The Essential Guide to Doing Research TABLE 11.1 ACCESS AND POWER Using power Abusing power > Using official channels and protocols > Avoiding and skirting around official channels and protocols > Establishing points of contact > Going around or above the appropriate person's head > Using gatekeepers and insiders > Asking gatekeepers and insiders to act unethically or to go behind management's back > Building rapport > Ingratiate yourself to the point of becoming sycophantic > Leaving doors open > Becoming a nuisance > Offering something back > Making promises you cannot or do not intend to keep researched abused at any stage in the research process. As articulated in Table 11.1, if the subtleties of gaining access are not managed appropriately, it can be easy to abuse relationships and take advantage of individuals. SURVEYING First, a point of clarification ... in its broadest sense, to survey (v) means 'to look at or examine', while a survey (n) means 'an observation or an overview'. Now some research methods texts use these definitions and group a number of approaches/methods under the survey heading. However, when I use die word 'survey', both as a noun and a verb, I am using more specific definitions that relate to a particular social science method. Survey (n): Information gathered by asking a range of individuals the same questions related to their characteristics, attributes, how they live, or their opinions. Survey (v): The process of collecting such information. Knowing what to expect Now students often tell me they've decided to conduct a survey; after all, they've heard they are a relatively simple, straightforward, and cost-effective means of gathering credible data. Well I hate to break it to them, but this is a long way from the truth.Yes, a good survey does have the potential to: • reach a large number of respondents • represent an even larger population • generate standardized, quantifiable, empirical data DATA COLLECTION • be confidential and anonymous and • generate qualitative data through the use of open-ended questions But there seems to be a belief out there that just about anyone can construct a survey instrument, and conduct a survey. You just need to know who you want to target; what you want to ask; and have some modicum of common sense about you. But I'll tell you what ... social science surveying really needs to be considered a specialist activity. Constructing and adniinistering a survey that can generate credible and generalizable data is a truly difficult task. There are a whole lot of really crappy surveys out there that aren't worth the paper they're printed on, yet the data they generate is often reported as truth and used in all kinds of decision-making processes. If it is your intention to gather data using a survey and a survey instrument, you will need to have good support. In addition to getting a handle on the information presented here, it's worth talking to your supervisor and working through some of the recommended readings at the end of the chapter. Another option is to look into survey software.'Sphinx', accessible at www.scoalri.com, is a program designed to help you with all main aspects of surveying, mcluding questionnaire construction and data analysis. Basic survey types A survey is a survey. Well not quite. There are actually a few distinct types of survey, each suited to a different purpose. But if you are familiar with the various types, and know what you want to know (thereby knowing what you want your survey to do), choosing appropriate types will be a relatively easy task. Surveys can describe and/or explain: Descriptive surveys: These surveys pretty much do what they say - they describe. This might involve basic demographic information like age, socio-economic status, and gender; or more personal information such as voting patterns, how often respondents have sex, or how frequently they spank their children. They can also gather attitudinal information such as attitudes towards war, euthanasia, or abortion. The goal is to get a snapshot - to show or to describe your 'respondents' - and, if a representative sample is used, to be able to generalize that description to a larger population. Explanatory surveys: These surveys go beyond description (although they do gather descriptive data) and attempt to establish why things might be the way they are. For example, not only would an explanatory survey want to describe a population's attitude towards abortion, but it would also seek to establish what might shape and form those attitudes, i.e. the effect of family values or personal experience. An explanatory survey attempts to build more complex understandings and goes beyond description, or even correlation, in an attempt to determine cause and effect. Surveys might target full populations or samples of populations: 152 153 The Essential Guide to Doing Research DATA COLLECTION Census: This is a survey that does not rely on a sample. A census surveys every single person in a defined or target population. Cross-sectional surveys: This type of survey uses a sample or cross-section of respondents selected to represent a target population. The goal is to be able to generalize the findings of the sample to the population with a high degree of confidence. Survey can also be longitudinal and explore changing times or changing people: Trend surveys: A trend survey asks similar groups of respondents, or the same cross-section, the same questions at two or more points in time. For example, conducting three surveys over a 20-year period (1984, 1994, 2004) that asks teenagers about perceived personal and professional opportunities. This allows you to assess whether the attitudes of teenagers have changed over the past two decades - in other words, are teenagers the same now as they were in the 1980s. Panel study. A panel study involves asking the same (not similar) sample of respondents the same questions at two or more points in time. Using the example above, if you were to survey 18 year-olds in 1984, then you would survey these same individuals in 1994 when they were 28, and again in 2004 when they were 38. You would then be able to assess whether there was a shift in respondents' attitudes as they get older. Finally, surveys can be administered in a number of ways: Face-to-face surveys: This mode of surveying allows surveyors the opportunity to establish rapport, build trust, motivate respondents, clarify questions, read nonverbal cues, and probe appropriately. The response rate also tends to be good. However, face-to-face surveying is often a lengthy and expensive process, does not assure anonymity or even confidentiality, requires surveyor training, and may be affected by interviewer bias. Telephone surveys: Telephone surveys are relatively inexpensive and take minimal time, allow for coverage of a wide geographic area, and offer some assurance of anonymity and confidentiality. On the down side, the response rate, while better than self-administered surveys, is still lower than face-to-face surveys; it's easy to catch people at a bad time; respondents can hang up on you if they have had enough; and you are limited to surveying only those with a telephone. Self-administered surveys: These surveys can offer anonymity and therefore confidentiality. They also allow coverage of a wide geographic area, and give respondents the opportunity to complete questionnaires in their own time. Response rates, however, can be really low (unless you can distribute your survey 154 through an organization that lets respondents fill out the surveys as part of the school/or work day). And of course respondents cannot seek clarification. Self-administered surveys are often mailed, but can also be sent and received electronically. This can potentially save you thousands, but you are limited to surveying those with e-mail addresses or populations likely to be online. The survey process Conducting a 'good' survey is a. process that involves a whole lot of steps. Surveys require you to: plan the approach, process, tools, and all the contingencies; construct the survey instrument; run a trial or pilot; redevelop as appropriate; send out the survey; manage the responses; and finally analyze the data. Box 11.2 outlines the 25+ steps (some you will need to do more than once) involved in surveying. You may find this box helpful as both a guide and a checklist. Box 11.2 Steps (and more are a lot) far Conducting a 'Gond'Survey PLAN 2. 3 4. 5. : Realisticallyconsider issues of sampling, distribution, reminders, response rates and data management so that you are relatively sure a survey/ questionnaire will work. ' Develop a plan you can implement.if response rates are low Consider what asnents nf are: likely to be . -----,— r.™.uu„ mipicincni.ii response rates ar Consider what aspects of your research question(s) answered through a questionnaire. List, group, and categorize these 'aspects-Explore; whether there might be existing1 questionnaires or sets of ques-tions that address these 'aspects' that may be appropriate...... : CONSTRUCT 10. If relevant questionnaires or sets of questions exist - adopt, adapt, and modify, if your questionnaire requires the construction of any new questions, have a shot at drafting them. . ■ . Decide on the response categories for each question, considering both ? the effect of response categories on'responses themselves and how var- ■ ious types of response category translate to different data types that, demand.quite-distinct statistical treatment. . Carefully read each question and response choices, and think .about f whether your questions might be considered ambiguous, leading, con--frontinq, offensive, based on-unwarranted assumptions; double-barrelled,« or pretentious.'. ..... ■ :? .Rewrite the questions in relation to the considerations above and runt, therri past a few peers/supervisors for their assessment. Repeat this step » as many times as necessary to get each question as right as possible. * 155 mtf The Essential Guide to Doing Research 11. 12. 13. 14. Attempt to put your questions in an order .that will be logical ami o.-.sc respondents into your survey.' Write instructions for your respondents and ask your peers/supervisor whether they seem clear and unair.hi(jijo:js. Rewrite- as ner.e.ssviry. Construct a clear, logical, professional, and aesthetically pious nrj layout "and design. Write a cover letter. PILOT 15. IG. 17. Pilot your questionnaire with a few people who are similarto those in Get feedback- from the pilot group in relation to the questions themselves (see step 9); the overall layout and design; the effectiveness of the cover - letter; the usefulness of the instructions; and the length of time it took to corupleic !nc questionnaire. Attempt to create variables, code the pilot responses, and then plug it into a statistical program (or qualitative data analysis program) to fob if you are likely to encounter any issues when you input your main data. REDEVELOP "18. Make modifications based on both the feedback of the pilot group and j the quality of the data generated. 19. If modifications are substantial, start again from step 15. 20. Get ethical clearance for the final version of your questionnaire. 21. Distribute questionnaires; be sure to include instructions for return (address and return date) and possibly a self-addressed stamped envelope. 22. Sena oli: a reminder letter if response rates are low. 23. . Put low response rate plan.(see step 1) inlo nctior if not enough data: has been yatncied by you noadlino. " 24. : Record and manage responses as they áre received. 25: Turn to:Chapter 12 to work through management and analysis of the data! The survey instrument When it comes to conducting a survey, what students struggle with most is constructing the actual survey instrument. From drafting the questions through to layout and design, students are quite surprised at how much thought and work goes into the development of an instrument capable of generating credible data. DATA COLLECTION Formulating questions The first step in writing your questions is knowing what you want to ask. By the time you are ready to construct your survey instrument you should know what aspects of your research questions can be answered by your respondents. The second step is to attempt an initial drafting of questions related to each of these aspects. Now there are about 762 ways to ask the same question, and each of these ways has potential to generate quite different data, so knowing how to best draft your questions can be a real challenge. While there are volumes written on crafting survey questionnaires, a few basic rules can be applied that can help you avoid the pitfalls of leading, offending, or confusing your respondents. Using a number of examples, Box 11.3 covers diese basic rules. Box 11.3 Questions to Avoil! Writing good questions is about clarity and specificity, but there; are plenty of ways to make sure this doesn't happen. It is easy to construct questions that llllllifll^ POORLY WORDED Complex terms and language - oig words can offend nnd conf'jsf). li they're not necessary, why use them? Compare the two following agree/ disagree statements: 'Polysyllabic linguistic terminology can act to obscure connotations'.vs..[Big.words can be conniving'. Ambiguous questions it's vmy easy to v.n'.c- ambiguojs questions because frames of reference can be highly divergent. For example, consider I'io qi.e.-linns: Nov.' hig is your family?' or 'Do you use drugs?' Families can be nuclear or extended, or for children of separated parents, may include two households. Similarly, 'drugs'can be an ambiguous term. Some respondents will only consider, illegal drugs, while others may include prescription drugs. And of course, it would be impossible to know whether alcohol or cigEroitus were also considered. Double negatives-a significant proportion of respondents get confused when faced with double negatives, Consider the following agree/disagree statement: 'You are not satisfied with your job'. To state that you are satisfied in your job means that you need to choose disagree, which enn be quite 'confusing;. Double-barrelled questions - this is when you ask lor only one response to a question with more than one issue. For example, take:the;question:'Do you consider the President to be an honest and effective leader?' Respondents may think yes, effective - but definitely not honest. 156 157 The Essential Guide to Doing Research DATA COLLECTION biased, leading or loaded rtf • JRing true' statements - these are'statements that are easy to agree wlm'St pimply because of their 'ring true tone'. A couple of examples might be %uM 'feally can't rely on people these days' or 'Times maybe tough, butthefe " Strongly disagree ■ Disagree . Unsure Agree Strongly agree • • Likert scales offer a range of responses generally ranging from something like I -_rstronglyj]{sagree' to 'strongly agree'.. In Likert sc.aJJg§;you^need to consider: j | Irie number S'f points you will useYwTiethVryou will".fprc'e'a side; and whether'.""1 i you think'respondents will 'get on a roll' and keep circling a particulnr n-.jmbnr. Information and instruction Providing clear background information and lucid instructions is an essential part of good survey instrument construction. • Offering information — surveys need to include some background information that identifies the sponsor/university; clarifies the survey's purpose; assures anonymity/confidentiality; provides return information, including deadlines and return address; and offers thanks for time/assistance. This information can be included at the start of the survey or as a cover letter or e-mail. • Providing instruction — what might be self-evident to you, may not be so obvious to your respondents. Your instructions should introduce each section of the survey instrument; give clear and specific instructions for each question type; provide examples; and be easy to distinguish from the actual survey questions — in fact its a good idea to use a different font. It may take a couple of drafts to get your instructions as clear and helpful as possible, and it's advisable to ask your peers/supervisor if your instructions do the job. However, the real test will come when you pilot die instrument, review the data, and seek feedback from your pilot participants. Organization and length Once the elements of your survey are complete, you need to think about putting it together in a logical order that covers it all without being overly lengthy. • Lmgtii — instruments considered too long can be abandoned, returned incomplete, or filled in at random. I once filled in a survey for a gourmet chicken shop in order to get a free piece of chicken (well I was a poor student at the time).This thing turned out to be, no lie, nine pages long. How many finger-licking chicken questions can you answer? For me it turned out to be all of them (I wasn't leaving without my Kiev), but I think I was supposed to actually read the questions before ticking the boxes! • Organization - you are likely to find contradictory advice on whether to start or end with demographics, and where to place your most important questions. A lot of this depends on the nature of both your questions and your respondents; and you may want to pilot two different versions of your questionnaire if you are unsure how to lay it all out. One tried and true piece of advice, however, is that you don't want to start your survey with any questions that might be considered threatening. It is important to ease your respondents into your survey and save sensitive question for near the end. Layout and design You'd drink that all of the intense intellectual work that has gone into writing clear and unambiguous questions with appropriate, well thought out response categories that are accompanied by clear instruction and organized into a sensitive, logical, and manageable form would be enough to ensure a'good'survey. Not quite. Aesdretics counts! If your survey looks unprofessional (for example, poor photocopies, faint printing, messy and uninteresting layout etc.), two things can happen. First, respondents will be less likely to complete a survey that is unprofessional and lacking an aesthetically pleasing layout and design. Second, the potential for mistakes increases dramatically if surveys are cluttered, cramped, or messy. INTERVIEWING I'm going to start my discussion of interviewing with another point of clarification. When I talk about interviewing, I am referring to a process with the following definition: 160 161