3
Models of Research Design and their Application to Semi-Structured Depth Interviewing
MODEL-BUILDING, MODEL-TESTING SEQUENCE
In social research I am concerned with building and testing descriptive and explanatory models of the realities with which I am concerned. Any particular model is a simplified version of a more complex social reality. Just as no map can include everything about the territory of which it is a representation - a map that excluded nothing would be an identical full-size reduplication of the original - so no model can include everything about the reality it represents.
In general, social research moves from model-building to testing the model that was built. Sometimes, this is described as the move from exploratory research where an unknown area is given a preliminary mapping to theory-testing research where the current provisional map is tested against reality. Whether the metaphor is of mapping or of modelling, the argument about sequence is the same.
At any given moment of an overall research cycle, particular types of interviews may be used for model-building or model-testing; in any given round of interviews, particular types of interview session may be used for model-building or for model-testing; in a given interview, the interviewer may shift between model-building and model-testing activity. However, given that you cannot test a model until you have built one, the general sequence is from model-building to model-testing at all these levels.
How to Think the Relation of Models to Evidence: Questions, Concepts and Indicators, the Problem of Operationalization/lnstrumentation
Collingwood and the Language of Theory-questions and Theory-answers
A theory, or a particular theoretical proposition, is an assertion about reality. Such an assertion is an answer to a question. The question and the answer need to be taken together.
The historian and philosopher R.G. Collingwood just before World War II wrote rather clearly about this in his Autobiography as follows:
52
CONCEPTS AND APPROACHES TO DEPTH INTERVIEWING
'I began by observing that you cannot find out what a man means by simply studying his spoken or written statements, even though he has spoken or written with perfect command of language and perfectly truthful intention. In order to find out his meaning, you must also know what the question was (a question in his own mind and presumed by him to be in yours) to which the thing he has said or written was meant as an answer____
A highly detailed and generalized proposition must be the answer, not to a vague and generalized question, but to a question as detailed and particularized as itself.
For example, if my car will not go, I may spend an hour searching for the cause of its failure. If, during this hour, I take out number one plug, lay it on the engine, turn the starting handle, and watch for a spark, my observation "number one plug is all right" is an answer not to the question "Why won't my car go?" but to the question "Is it because number one plug is not sparking that my car won't go?" Any one of the various experiments I make during the hour will be the finding of an answer to some such detailed and particularized question.
The question "Why won't my car go?" is only a kind of summary of all these taken together. It is not a separate question asked at a separate time, nor is it a sustained question which I continue to ask for the whole hour altogether. Consequently, when I say "Number one plug is all right", this observation does not record one more failure to answer the hour-long question, "What is wrong with my car?" It records a success in answering the three-minute long question, "Is the stoppage due to failure in the number one plug?"____1
It seemed to me that truth, if that meant the kind of thing which I was accustomed to pursue in my ordinary work as a philosopher or historian - truth in the sense of which a philosophical theory or a historical narrative is called true, which seemed to me the proper use of the word - was something that belonged not to a single proposition, nor even as the coherence theorists maintained to a complex of propositions taken together, but to a complex consisting of questions and answers____
I could hazard a few statements about it...
• Each question and each answer in a given complex had to be relevant or appropriate, had to "belong" both to the whole and to the place it occupied in the whole.
• Each question had to "arise"; there must be that about it whose absence I condemn when I refuse to answer a question on the grounds that it "doesn't arise".
• Each "answer" must be the "right" answer to the question it professes to answer: by "right", I do not mean "true". The "right" answer to a question is the answer which enables us to get ahead with the process of questioning and answering.'
(Collingwood, 1939: 31-2, 37, paragraphing modified)
Collingwood's final paragraph seems very important, and can be commented upon as it affects research questioning and the answering of research questions by the researcher (the context of Collingwood's discussion above). (Later on, we will see how it affects interview questioning and the answering of interview questions by interviewees, to which Collingwood's discussion can be cautiously extended.)
1 I would argue that it is both.
MODELS OF RESEARCH DESIGN
53
I wish to argue that the conceptual framework in terms of which research-questions are posed and in terms of which research-answers have to be given is provided by a theory-language (theoretical discourse). Few are likely to argue with such a proposition, but often its implications are underestimated.
Social research devoted to theoretical development rests upon a number of distinctions which are not obvious to common-sense and often neglected by qualitative researchers. Perhaps the crucial distinction is between a (social science) concept and its indicators.
Consider the relation between a theory-language in which 'social class' is a crucial characteristic to be established about a person, and interview evidence. What interview evidence would help to establish the person's 'social class'?
You might decide that social class cannot be determined from somebody's talk, and only a document indicating source of income would be a relevant indicator. Consequently no verbal statements by an interviewee would be indicators of social class, if the concept was defined in this way.
Alternatively, you might decide that the indicator of social class was the capacity or otherwise to engage in 'middle-class talk' (Bernstein's elaborated code). In this case, documentary evidence about source of income or anything else would be useless as an indicator of social class; only an interview or an overheard spontaneous conversation would provide evidence about the capacity to engage in 'middle-class talk in a middle-class accent'.
Alternatively, your concept of social class might involve both source and level of income, customary mode of talking, and type of occupational activity. This would mean that the concept had three indicators, and that interview evidence might be a strong indicator for one of them (mode of talk) and a weaker indicator for others.
The researcher who decides to use multiple indicators for any particular theoretical concept in their conceptual framework has a problem: what happens if the indicators point in different directions? The person talks with a public-school accent; they present documentary evidence of living on minimum welfare benefits; they say that their work involves driving expensive cars and negotiating prices. Can this person be simply ascribed to a single social-class position? If so, which and why? If not, what can be inferred from the interview material? 'Interpreting evidence' involves exploring issues like these.
Theories in social science are couched in a 'theory-language' made up of a body of concepts, for which the indicators are often typically indirect and non-obvious.
If you wished to investigate whether a given organization was suffering from 'information overload', or a particular group of workers from 'alienation', or a particular empire from 'imperial overstretch', or a society from 'accelerated social polarization', you would need to define the key conceptual terms, and you would need to define what 'indicators' you would collect for each of the terms in order to determine whether the hypothesized condition or change of condition was actually occurring, and, if so, to what extent?
An 'empirical indicator' (EI) is a measurement, an observation, a datum, which is taken to be 'evidence' for a particular theoretical concept (TC) being in one 'state' or another (such as information overload or its opposite, high medium or low alienation, increasing or decreasing rates of social polarization, etc.).
Since Lazarsfeld, in social research the custom has often been that the more abstract the theoretical concept, the greater the number of 'indicators' that need to
54
CONCEPTS AND APPROACHES TO DEPTH INTERVIEWING
be examined. To establish somebody's 'address' or 'gender' is relatively easy and might normally be quickly established with rather few Els; to establish somebody's 'sexuality' or their 'social class' or their 'ideology' might require rather more Els and rather more complex argument.
In sociology, the principle of 'triangulation' has been put forward which suggests that you should consider looking for at least three empirical indicators for any particular moderately complex theoretical concept (Denzin, 1970).
How does this discussion of concepts and their difficult relation to different indicators relate to interviews? Our discussion of the conceptual framework needed to understand interviews seen as problematic should suggest the answer. The entire interview contains evidence for answering a variety of theory questions, but problematic evidence. The evidence is problematic because the relation between theoretical concepts and their empirical indicators is always across a gap, which one always has to be prepared to argue over. There is a gap which has to be filled by a justification of why this particular evidence should justify certain claims that a particular hypothesis or theorization has been supported or disconf irmed by a particular pattern of interview evidence. I need to argue a case as to why a pattern of data should be taken as indicating at the level of my theoretical discourse (in my theory-language) a particular 'reality'. Inferences from data-indicators to conceptual significance cannot be assumed: they must be argued for.
It may be useful to look at a model of sociological research in which the distinctions raised above are embodied.
CLASSIC MODEL OF PRE-CONCEPTUALIZATION, PRE-THEORIZATION AND STAGES OF THEORY-TESTING: ROSE-WENGRAF
Classical models of research design in general and social research design in particular assume that the 'conceptual structure and the theories couched in terms of that conceptual structure' have arrived en bloc from somewhere else, that in some sense they are 'given', and that the task that remains is that of testing such a pre-given theory by deriving from it testable hypotheses (all of which are couched in terms of the given, the received, conceptual structure). Formulated in terms of a received conceptual structure, the model assumes a pre-theorization and is only concerned with theory-testing/model-testing.
If you are going to start with a theorization-tesfmg model, then it is important to work the theorizations out very well in advance, as the 'model of the research cycle' set out below suggests.
Rose (1982: 14) developed and used to good effect a diagram which I have revised as Figure 3.1.2
2 I have added the items on the CF and the CRQ, and also added the words 'a descriptive or' to the description of what a Theory' is. First point: Very briefly, earlier writers tended not to distinguish between conceptual frameworks and assertions about reality (theories, propositions) couched in terms of them: both were called 'theory', and Rose's original formulation follows this tradition. I consider it to be very important to distinguish the two, both in relation to the conceptual frameworks of the researcher (Kuhn, 1970; Bachelard, 1999) and - though this is not our concern at the moment - in relation to the conceptual framework of the interviewee. Second point: Researchers are concerned to develop better accounts of reality, better models of entities and processes. Against positivist orthodoxy, I see no absolute distinction between description and explanation and I do not privilege causal explanation over understanding.
MODELS OF RESEARCH DESIGN
55
CF: conceptual framework: a set of concepts in terms of which questions can be asked and * answers can be given, theories hypothesized and theoretical propositions tested.
CRQ: a central research question 4-
couched in terms of the conceptual framework to which a 'Theory' is a hypothetical or strongly supported/refuted answer
(A) Theory: a descriptive or an explanatory statement about social phenomena <—
(B) Theoretical propositions: specific propositions 4-to be investigated in the study
(C) Operationalization/instrumentation: decisions made as to I     how to carry out empirical work - design of indicators for concepts, I    choice of units, sampling, technique of data collection (interviewing)
(D) Fieldwork: collecting interview and other data, _
|     practical problems of implementing Stage (C) decisions
(E) Results: data-processing and
analysis through strategies and procedures of interpretation lead to findings; their interpretation leads back to (C), (B), (A) as well as, sometimes, to modifications of CRQ and CF
FIGURE 3.1   Rose-Wengraf Model of the Research Process
<
The model involves a 'stages element' in that one proceeds down the arrows on the left-hand side of the diagram, but the 'return arrows' going back up the page on the right indicate that the researcher may have to 'return' to 'earlier stages' if the situation at 'later stages' requires it. In this version, it is problems with the 'results' that lead to a 'return' to the 'fieldwork' or other elements of the research design.
For other models of the research process, see de Vaus (1996), Black (1993: 5,10), Evertson and Green (reprinted in Rossmann and Marshall 1995: 20-1), Maykut and Morehouse (1994: 84).
The classic model of research involves a conceptual framework - which, once established, is normally not modified during the process of theory-testing - and, couched in terms of that conceptual framework, a theory composed of theoretical propositions which are also normally not modified during the process of theory-testing. Very often, the choice of samples is also not modified after the original selection has been made.
This is by no means the only model, however, of how research is done, because models, and the conceptual frameworks in terms of which models and theoriza-tions are couched, get revised, rectified and improved.
Research which is concerned for model-building and model-rectification very frequently changes its sampling strategy as the research proceeds; it puts forward theories not originally envisaged at the start of the research; it accounts itself particularly successful and important if it manages to contribute to a better conceptual framework than the one with which it started off. This can be called the
56
CONCEPTS AND APPROACHES TO DEPTH INTERVIEWING
'romantic' model of social research. For various reasons it is ideologically and often empirically associated with qualitative work.
The linkage of theoretical concepts with empirical indicators through an explicit operation is a crucial insight associated with quantitative research, but one underestimated, ignored or even denied by researchers declaring themselves to be 'qualitative'.
This is a serious error, but it can to some extent be understood. It should be stressed that the terms of 'operationalization' and even of 'instrumentation' have the hidden premise of a fixed prior theory, a premise which is characteristic of theory-testing quantitative research. In this model, the theory-language composed of agreed received theoretical concepts is already fixed, and intellectual ingenuity is confined to finding, at the level of empirical indicators, different operationalizations for those fixed theoretical concepts. Given such a fixity, the only conceptual design research question left open appears to be the identification of empirical indicators and their mutual relationship.
Qualitative researchers are more typically interested in the generation and rectification of theoretical concepts, and therefore reject the top-down assumption behind 'operational-izing pre-fixed theoretical concepts'. Where, as in all theory-building, model-building research, the theory is emergent from the research, then we have a reverse process upwards, the 'upwards inferring' of theorizations and even of theoretical concepts on the basis of examined evidence. This contrasts with the 'downwards' metaphysic of the term 'operationalization'. Such upwards inferences, such 'instrumentation/ interpretation', such invention and rectification of appropriate concepts, will be further discussed below.
To sum up: in this text, I argue for the existence of a gap between the level of theoretical concepts and theory-language and the level of empirical indicators and evidence, but I reject both any notion that the theoretical concepts are more fixed than the empirical indicators and any notion that the practice of 'interpreting evidence' can ever be fully reduced to an explicit mechanical operation that can be summed up in a verbal formulation called a 'theory'. Our practice of inter-level movement between concepts and material, between theory-language and empirical indications, is always more complex than any theorization I produce about it.
The second model I shall present as background to the discussion of semi-structured depth interviewing is Joseph Maxwell's model of design components and his useful conception of a well-designed research programme.
MAXWELL'S MODEL OF DESIGN COMPONENTS
A 'well-designed' object is an object whose component parts have been designed to be able to work together and in sequence such that the functions, or purposes, for which the object was designed are most likely to be served. The same functions, or purposes, can be carried out by different (well-designed) objects. What are the characteristics of a good, as opposed to a poor, 'research design'? I shall quote at length (and with permission) from Maxwell's account:
MODELS OF RESEARCH DESIGN
57
'Design in qualitative research is an iterative process that involves 'tacking'3 back and forth between the different components of the design, assessing the implications of
purposes, theory, research questions, and validity threats for one another ____The
model I present here has five components. These components are characterized by the issues that each is intended to address:
1 Purposes. What are the ultimate goals of this study? What issues is it intended to illuminate, and what practices will it influence? Why do you want to conduct it, and why should we care about the results? Why is the study worth doing?
2 Conceptual Context. What do you think is going on with the phenomena you plan to study? What theories, findings and conceptual frameworks relating to these phenomena will guide or inform your study, and what literature, preliminary research and personal experience will you draw on? This component of the design contains the theory that you already have or are developing about the setting or the issues that you are studying. There are four main sources for this theory: your own experience, existing theory and research, the results of any pilot studies or preliminary research that you've done, and thought experiments.
3 Research Questions. What, specifically, do you want to understand by doing this study? What do you not know about the phenomena you are studying that you want to learn? What questions will your research attempt to answer, and how are these questions related to each other?
4 Methods. What will you actually do in conducting this study? What approaches and techniques will you use to collect and analyse your data, and how do these constitute an integrated strategy? This component of your design consists of four main parts: your research relationship with the people you study, your site selection and sampling decisions, your data collection methods, and the data analysis techniques you use.
5 Validity. How might you be wrong? What are the plausible alternative explanations and validity threats to the potential conclusions of your study, and how will you deal with these? How do the data that you have, or that you could collect, support or challenge your ideas about what's going on? Why should I believe your results?' (Maxwell, 1996: 4-5)
See also Figure 3.2.
BRIEF NOTE ON ASSUMPTIONS BEHIND INFERENCES AND INTERPRETATIONS: INSTRUMENTALISTS vs REALISTS
Maxwell distinguishes a debate among interpreters of interview evidence. This is between what he calls 'instrumentalists' and 'realists'.
'Interviewing someone can only tell you what that person thinks or feels or values about what they think is real. It can never tell you what is actually real now or was actually real in the past.'
3 For a similar concept, see 'double-fitting' between theory and data as discussed by Baldamus (1982).
58
CONCEPTS AND APPROACHES TO DEPTH INTERVIEWING
FIGURE 3.2   Maxwell's Interactive Model of Research Design (Maxwell, 1965: 5)
Maxwell (1996: 56) identifies the type of assertion above as being 'instrumentalist'. He recounts the following history:
'Gail Lenehan, for her dissertation, proposed to interview nurses who specialize in treating sexual assault victims about their cognitive, behavioural, and emotional reaction to this work____Her research questions included the following:
1 What, if any, are the effects on nurses of working with rape victims?
2 Are there cognitive, psychological and behavioural responses to having experiences of rape "shared" with them, as witnessing victims' suffering after the assault?
Her proposal was not accepted [Maxwell continues the story], and the reviewers, in explaining their decision, argued (among other concerns) that
"the study relies solely on self-report data, but your questions do not reflect this limitation. Each question needs to be reframed in terms that reflect this limitation. Some examples might be: 'how do nurses perceive and report ... the effects of working with rape victims?' or 'what specific cognitive, psychological (emotional?) and behavioural responses do nurses report?""
Maxwell discusses the point:
Instrumentalists ... prefer to stick to what they can directly verify. Realists in contrast do not assume that research questions and conclusions about feelings, beliefs, intentions prior behaviour, effects and so on need to be reduced to, or reframed as, questions and conclusions about the actual data that one uses. Instead, they treat their data as fallible evidence about these phenomena, to be used critically to develop and
test ideas about the existence and nature of the phenomena____Each approach has its
risks____The main risk with realist questions ... is that your increased reliance on inference may lead you to draw unwarranted conclusions or to allow your assumptions or desires to influence your results. My own preference is to use realist questions and to
address as systematically as possible the validity threats that this approach involves____
continued
MODELS OF RESEARCH DESIGN
59
continued
What John Tukey (1962) said about precision is also true of certainly: "Far belter an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise".' (Maxwell, 1996: 56-7)
FIGURE 3.3 Maxwell's Interactive Model of Research Design - Modified for Semi-structured Depth Interviewing (SSDI) Method
It is because I am interested in going beyond people's interview self-report (truthful, partly deceptive, often self-deceptive, etc.) that I need to be aware that anything that is said, done, or apparently expressed in an interview is, as Maxwell points out, fallible evidence of extra-interview realities.
When you want to use the evidence in an argument about non-interview realities, your final research design and data analysis will need to address the validity threats that interview-evidence has to confront. Questions of validity are about the arguments in which you justify inferences from interview data (empirical indicators EI) to theorizations couched in terms of theoretical concepts (TC). However, validity questions will not be dealt with separately in this text in general terms, for reasons of space. Therefore, the key areas addressed in this book are on the diagonal from top right to bottom left in Figure 3.3. It means, in particular, unpacking the concept of 'methods' in respect of semi-structured depth interviews. This axis is therefore highlighted in the figure.