QUANTITATIVE
AND STATISTICAL
RESEARCH METHODS
FROM HYPOTHESIS TO RESULTS
WILLIAM E. MARTIN KRISTA D. BRIDGMON
Chapter 13
UNDERSTANDING QUANTITATIVE LITERATURE AND RESEARCH
LEARNING OBJECTIVES
o Learn guidelines to interpret quantitative research articles in the professional literature.
o Apply the interpretation guidelines to a quantitative research article.
RESEARCH INTERPRETATION FOR CONSUMERS
The following guide is intended as a practice device for the interpretation of published research in education and the behavioral sciences.
I. The Initial Exposure
The initial exposure to a research study should begin by reading the entire study with a rather casual, relaxed approach with little attention to given details. The objective at this point is to gain a general overview. As one progresses through the five guidelines that follow, more and more attention to detail will become necessary; however, one should not stop and spin wheels at any point where a particularly difficult problem presents itself. Instead, continue on through until more clarification is gained and then return to the difficult problem when a more comfortable solution is available.
1. Read the entire study through rather casually to gain a general overview.
2. Ascertain the central theme of the study, the basic rationale for the research, and the relationship of the study to other research.
3. Determine the existing theory, if any, to which the research is addressed, or speculate as to the basic theoretical framework.
4. Identify the subject(s) of the study and the research setting or environment if it is localized.
5. Determine the basic nature of the research (e.g., historical, descriptive, or experimental). Studies of an inferential nature can usually be detected at this point.
442 • CHAPTER 1 3
II. The Research Question(s)
Every research study should have a question or questions formulated prior to its initiation. It is the nature of the question or questions and the approach toward resolution that distinguish one form of research from another. Quite often a primary question will be subdivided into subquestions. This isa matter of style and can be helpful for both the researcher and the consumer. All too often, however, the research questions (RQs) either are not stated or are stated poorly. In the event that the research questions are not stated, it is necessary for the consumer to speculate by formulating a tentative guess. One may then return later and revise the question as more information is gained.
• Ascertain the question(s) asked.
• Determine whether the criteria for good research questions are met, and if not, restate the RQs as you believe the researcher intended them to be stated, applying the criteria. This is an important step toward a clearer interpretation of the remainder of the study. A good research question must be clear, unambiguous, and in question form. It must ask about the relationship between two or more variables and must imply the possibility of empirical testing.
III. Follow a Single Question All the Way Through
The research question is the primary reference point for interpretation and evaluation of any research study. It is for this reason that all of the guidelines that are presented here should be applied to each research question individually and completely, one RQ at a time. It is quite easy for the consumer of research to become lost in the middle of the study among all
UNDERSTANDING QUANTITATIVE LITERATURE AND RESEARCH • 443
the jargon, charts, and statistics. When this happens, simply return to the specific research question and start over.
IV. The Hypotheses
Just as every research study should have at least one question, it should also have some systematic means of answering the question. This is usually, though not always, accomplished through the use of hypotheses. In that it is conceivable that more than one hypothesis is required to help answer a single research question, depending on the style of the author and how broadly the question is formulated, each hypothesis should be followed through individually and completely. All four of these guidelines should be followed by the consumer before starting on a second hypothesis.
1. For each research question identified in the study, ascertain the hypothesis (or hypotheses) intended by the researcher to help answer that research question. Formal hypotheses should be stated in two forms: the research or alternative hypothesis (f/a), which is what the researcher is actually guessing the true situation to be, and the null hypothesis (f/o), which provides a mathematical zero reference point for a formal statistical test.
2. For each research hypothesis (f/a), determine the corresponding null hypothesis (/-/0) that is actually tested.
3. Determine whether the stated alternative and null hypotheses meet the criteria for good hypotheses, and if not, state or restate them applying the criteria. They should be stated as you feel the researcher meant to state them.
4. If only the null hypothesis is stated, which is unfortunately too often the case, speculate and state the research
444 • CHAPTER 1 3
hypothesis (/-/a) in an acceptable manner. Usually when only the null hypothesis (H0) is presented it is safe to assume that the alternative hypothesis (/-/a) is a nondirec-tional hypothesis. This means that the researcher is not predicting a greater-than or less-than relationship, but instead is simply waiting to see what will happen. In this case a two-tailed test of statistical significance will be used in hypothesis testing.
V. The Variables
Because the very essence of research is centered on the relations between and among variables, a thorough knowledge of variables and their classifications is imperative for research interpretation. Very generally, variables are classified into three categories according to their relative purpose or existence in a study: independent variable (IV), dependent variable (DV), and extraneous variable (EV). Extraneous variables are contaminating, are unwanted, and need to be controlled in some way.
• Identify each variable important to the hypothesis and research question, and classify it as independent or dependent if appropriate within the context of the study. Some studies are more concerned with simply the association of a large number of variables, in which case this may not be appropriate. If independent, classify as either active or attribute.
• Identify all extraneous variables mentioned by the researcher for each hypothesis. These are variables that behave like independent variables and confound or contaminate the study. Determine any steps taken by the researcher to control for those extraneous variables mentioned.
UNDERSTANDING QUANTITATIVE LITERATURE AND RESEARCH • 445
VI. The Operational Definition (OD)
Every variable in a study must have an operational definition (OD). The OD is specific (though not necessarily unique) to the study. It is the precise way the variable is measured in the study, and its nature is the prerogative of the researcher.
• Identify the operational definition of each variable.
• The consumer should take time to learn about any operational definitions with which he or she is unfamiliar at this point (e.g., Q-sort technique, semantic differential, various published psychological instruments, etc.).
• Keep notes for later use.
• Determine the level of measurement (scaling) of each variable's operational definition, and distinguish between those considered continuous and those considered discrete.
VII. The Population under Study
At this point the consumer should take a closer look at exactly whom the study is concerned with.
• Determine the precise population under study. Quite often the researcher does not define the population clearly. If this is the case, one must speculate the best one can.
• Determine the actual subjects comprising the sample in the study, the number acquired, and the method of acquiring the subjects (sampling).
VIII. The Basic Research Design
The nature of the design of the research is dependent upon the research questions asked and the researcher's approach
446 • CHAPTER 1 3
toward answering those questions. Because of the large number of possible designs, the consumer should have some background knowledge of general research designs to assist him or her in identification. The simpler experimental or quasi-experimental attempts are usually the easiest to detect because of their emphasis on matched or random assignment and control or comparison groups. Other designs may be more difficult for the consumer to discern. At any rate, the consumer should glean what information is possible regarding the basic design from the context of the study. It may also prove helpful to diagram the design if possible.
IX. The Collection of the Data
• Determine precisely how the data were collected for each variable.
• Familiarize yourself with any procedures mentioned that are unknown to you at this point (e.g., survey techniques, mechanical devices, psychometric instruments, etc.). This is obviously related to operational definitions in many instances.
• Keep notes on new information for later use.
X. The Analysis of the Data
Most research studies are fairly clear as to how the data were analyzed. However, adequate interpretation requires some knowledge of basic statistics and the common symbols encountered. In addition, one should be familiar with the basic hypothesis-testing process. The serious consumer of published research will keep a notebook handy with acquired information that can be added to and drawn from continuously. The consumer must eventually acquire an interpretive knowledge of the
UNDERSTANDING QUANTITATIVE LITERATURE AND RESEARCH • 447
most commonly seen statistics (e.g., Z, t, F(the simpler models), chi-square, and r). These statistics, along with their use and the information they yield, may be found in any elementary statistics text. When more sophisticated techniques are encountered (e.g., multiple regression, factor analysis, discriminate analysis, or canonical correlation), use information resources related to multivariate statistics.
• Identify and define each statistical procedure employed and note the specific variables involved.
• Identify and define each symbolic expression encountered and determine the precise meaning with respect to the analysis presented.
• Take notes on information gained for future use.
XI. The Presentation of the Results
The presentation of the results is necessarily related to the analysis of the data and will usually be a combination of narrative and tables or graphs. It is important for the consumer to pay careful attention to exactly what the researcher has presented.
• Identify and define all terms and symbols presented in the results.
• Determine which null hypotheses, if any, were rejected and the level of significance for rejection.
• Stay with a table or graph until you know you understand what is presented.
XII. The Conclusions and Interpretations
The conclusions of the study should be based on the analysis of the hypotheses, if hypotheses were tested, and should be
448 • CHAPTER 1 3
clearly related to the original research question(s). The consumer's focus of attention should be on whether the research questions were answered, and on the final conclusions of the study regarding the questions.
• Ascertain the specific conclusion for each finding presented.
• Determine the researcher's interpretation with respect to each conclusion individually and in concert with respect to the research question(s).
Source: From Dennis A. Gay, PhD, University of Northern Colorado. By permission of Professor Gay.
The researchers of this study used the Rauscher et al. recommendations to plan a replication study to "be a faithful replication of the central conditions of the Rauscher et al. experiment." However, there were differences in this replication study that included: (1) only one posttreatment assessment was used, (2) random assignment to condition group was used to create equivalent groups, (3) the time interval was lengthened by 24 hours to 48 hours between pretest and treatment condition.
4. What conceptual, methodological, and measurement theories undergird this study? This information can be found in the introduction but also throughout the article.
• The overarching theory relates to cognitive learning theory.
• The more immediate theories are:
• Musical (complexly structured) experience of short duration (long duration) improves spatial abilities.
• Music as a mood-induction technique affects performance on cognitive tasks.
5. Who are the participants and what is the research setting?
Describe the characteristics of the sample participants.
UNDERSTANDING QUANTITATIVE LITERATURE AND RESEARCH • 449