Reading & Evaluating Scholarly Literature
Evaluating the Introduction
1. What is the problem the study is addressing?
2. What are the questions investigated or hypotheses tested?
3. What previous work or theories informed the study?
4. How is this work different from the previous studies?
5. What are the contexts of the study that may have affected the results (e.g. environmental
conditions, lab conditions, timing questions, etc…)?
6. Is the design suitable for the types of questions to be answered?
7. Is the sample size representative of the larger population (was it big enough)?
8. Are the data collection instruments and procedures likely to have measured all the important characteristics with reasonable accuracy?
9. Does the data analysis appear to have been done with care, and were appropriate analytical techniques used?
10. Are the results statistically significant?
11. What is the pattern of results for each broad question that was addressed?
12. Which conclusions appear to be well supported by the pattern of results; and which conclusions do not appear to be well supported?
13. Which conclusions are well supported by what is generally known about the problem and by prior research findings?
Hints for evaluating the Results and Discussion - Common errors when generating the conclusions and recommendations are the following:
1. Stating conclusions that the researcher thinks are correct and important, but for which
the study provides no support.
2. Generalizing well beyond the questions, contexts, population, and interventions that
were actually studied.
3. Not adding cautions when there are important limitations in methods and/or their
execution in the study.
4. Falsely interpreting statistical significance and the lack thereof
5. Selectively focusing on some results while ignoring others and the pattern of results
6. Moving from inference to values and speculation.