Many different forms of evidence are used in academic work in order to answer different types of questions. Different disciplines often focus on some types of evidence over others, depending upon the type of inquiry they are interested in conducting. This article is a brief introduction to some of the major types of evidence that one is likely to encounter in researching a controversial claim or a topic of interest, along with a short discussion of the relative strengths and weaknesses of that form of evidence.
- Case study: A detailed analysis of one or a small number of specific historical incidents, with the aim of understanding their progression, why things developed as they did, and perhaps comparing one case to another. Often used in history, political science, law, corporate management, and the study of geologic and evolutionary history. Case studies provide a great deal of detailed information about selected cases of relevance to a particular question, however they are often liable to multiple conflicting interpretations, and it is unclear how similar or different current or future conditions will be compared to those examined in the case study, making it difficult to make generalised conclusions on the basis of such evidence alone.
- Observation: A generic term for any method of collecting evidence that does not entail collection of detailed statistics (this would come under statistical analysis), experimental manipulations, or active engagement beyond passive recording of some phenomena. Very common in botany, astronomy, and geology. An excellent method for collecting facts without affecting the system of interest, but typically not very useful for gaining a deeper understanding of those facts.
- Textual analysis: The careful reading and systematic study of texts, typically written documents but also including photographs, film, and other media. Such analysis often focuses on the language used, the tools of persuasion implemented by the author, comparison of details with other texts to determine likely authorship, and interpretation of the implications and presuppositions of a text in order to better understand the mindset of the author and audience of the text. Common in linguistics, sociology, history, and media studies. Very useful for casting light on the purposes, presuppositions, goals, and mindset of a person or groups of people (from particular cultures, time periods, etc), but often subject to multiple interpretations and different readings.
- Surveys and interviews: Recording the answers to a series of set questions given to one person or a group of people. Answers can take the form of selecting from several set options, providing numerical ratings, giving short answers, or lengthy unstructured interviews where a person is invited to ‘tell their story’. Frequently used in sociology, psychology, political science, and medicine. Useful for collecting standardised data from different people, allowing a comparison of their beliefs, perspectives, experiences, and attitudes. Limited by low response rates, inaccuracy or dishonesty of respondents, misleading or confusing wording of questions, and the imagination of researchers as to what they should ask.
- Experiments: Conducting an experiment involves arranging at least two systems which are the same in every way, except that some intervention or treatment is performed on one (the experimental group) but not on the other (the control group). Since both systems were initially identical, any subsequent difference between them should be due exclusively to the effect of the intervention or treatment, and not to any other factors. Most commonly used in chemistry, physics, cell biology, medicine, and psychology. Experiments have the advantage of being able to provide fairly unambiguous results, without the concern of being biased by uncontrolled differences or extraneous variables. Limited by the difficulty in correctly performing experimental proceedures, limited applicability to systems we are unable to experimentally manipulate, and limited ability to generalise results to real-world settings outside the artificial experimental setup.
- Statistical analysis: Any quantitative data that is collected from any real world system can be subject to statistical analysis, from simple calculation of averages through to complex multivariate regressions. Statistical analyses are commonly used in physics, genetics, economics, psychology, sociology, and business. Such methods have the advantage of providing more objective results that are less easily disputable than the qualitative results of case studies or textual analysis, however their main disadvantage (aside from the fact that the relevant mathematical manipulations are very often performed incorrectly), is that outside an experimental setting it is often very difficult to make any clear inferences about how or why the data came out as they did, as it is very difficult to distinguish between competing hypotheses or variables that were not controlled for in collecting the data.
- Formal modelling: Involves developing a mathematical, graphical, or computational model of some system or process based on a set of initial assumptions, and then examining the behaviour of the model to see how well it replicates the behaviour of the real world system. Used increasingly in many different disciplines, but particularly common in physics, chemistry, engineering, and economics. This method can be extremely useful when it leads to the development of comparatively simple, well validated formal models that allow us to better understand the behaviour of the systems in question. Its main weaknesses lie in the difficulty in deciding which initial assumptions are most appropriate, and in knowing to what degree the simplified world of the model resembles the real world.
- Ethnography: A method involving living and working very closely with a group or community for an extended period of time, often weeks or months, and making very detailed, rich descriptions of their beliefs, practices, and behaviour. It is most commonly used in anthropology, and differs from interviews or regular observation in that the researcher does not attempt to remain aloof, but becomes intimately involved in the lives of those they are studying. They often will refrain from asking extensive questions explicitly (as in interviews), but rely on observations and discussions which take place naturally over the course of daily interaction. Ethnography can provide very detailed information about the customs, norms, and beliefs of various groups which may be impossible to obtain through any other method. The method is limited by the possibility that the researcher may bias their results through their participation. Ethnographic studies will seldom be able to identify particular causes or distinguish between competing explanations as to why things are a certain way.
Different research methods: Useful introduction to different research methods and when to use them from Explorable
Research methods: A brief outline of some different types of research design