Memory & Cognition 1980, Vol. 8 (1), 31-38 Memory aids people use: Two interview studies JOHN E. HARRIS MRC Applied Psychology Unit, Cambridge CB22EF, England In two interview studies subjects were asked what memory aids they used and how often. In particular, the reported frequencies of use of external aids such as shopping lists and memos were compared with those of internal aids that have been investigated by psychologists, such as the method of loci. Subjects in the first study were 15 male and 15 female university students, and in the second, a group of 30 adult women, most of whom were housewives. In both studies the aids reported to be used most were external ones. However, it is pointed out that the most commonly used methods of remembering must be internal, although these are not usually considered to be aids. Some difficulties that arise in the assessment of memory in interview and questionnaire studies are discussed. After years of neglect there has been, during the last 15 years, a steady increase in the attention experimental psychologists have paid to memory aids. Both the past neglect and the present interest have been discussed elsewhere (e.g., Brown & Deffenbacher, 1975; Higbee, 1978, in press, Note 1). Research on memory aids has been directed at a variety of questions, although most have to do with testing in the laboratory the effectiveness of what may be called internal aids. These are learned schemes for remembering specific types of information. They usually involve only internal, mental manipulation, such as the formation of mental images or bizarre associations or the composition of a story. However, there are also many memory aids that depend for their effectiveness upon external manipulation of the environment. This may involve writing in a diary, making a shopping list, setting a cooking timer, or even asking someone else to provide a reminder. Memory aids of this type will be referred to as external. Few studies have included external aids, and few have dealt with the use of memory aids in everyday life, although a number have gone some way in these directions. For example, some have looked at the use of internal aids by students (e.g., Carlson, Kincaid, Lance, & Hodgson, 1976; Gruneberg, 1973), and metamemory studies (see Flavell & Wellman, 1977) have investigated children's knowledge of methods of remembering, including external ones (e.g., Kreutzer, Leonard, & Flavell, 1975). Also, Hunter (1979) has included external devices in his discussion of memory aids. In his general category of "memory-aiding devices," he includes all artificial information storage, including written records, films, tapes, and computerized records, and he refers to "the communal memory store .... The second study was supported by the British Medical Research Council. The author thanks Alan Baddeley, John Duncan, and Arnold Wilkins for their comments on an earlier draft of the manuscript. Requests for reprints should be sent to John E. Harris, Medical Research Council Applied Psychology Unit, 15 Chaucer Road, Cambridge CB2 2EF, England. Copyright 1980 Psychonomic Society, Inc. 31 exemplified by a library or by the files of an office" (p. 5). A first step in investigating the use of memory aids in everyday life is to find out what aids, both internal and external, are used and how often. This was the purpose of the interview studies reported in this paper. METHOD Two studies are reported, the only major difference between the methods being in the populations from which the subjects were drawn. Subjects In the first study 30 students from Southampton University were interviewed. Their mean age was 21 years (range = 19- 27 years), and quotas of 15 male and 15 female students were set. The subjects were recruited as unpaid volunteers in the Students' Union building by the interviewer. Between 60% and 70% of the students who were accosted took part. In the second study 30 female members of the Cambridge Applied Psychology Unit subject panel were interviewed. They were mostly housewives, some having part-time jobs. Their mean age was 46 years (range = 23-67 years); only the two youngest and a widow were unmarried. In both studies each subject was interviewed separately. Interview Content and Procedure Within each study the same questions were asked in each interview and in the same order. The subjects replied orally, and the interviewer marked down the responses on a prepared score sheet. In the first part of the interview, the interviewer asked nine general questions, such as degree subject studied (student subjects) and marital status and number of children (Cambridge subjects). In both studies subjects were asked whether they had ever bought or read a book about memory improvement, and questions about whether they owned a diary,' calendar, or year planner for the current year and where any calendar or year planner was kept. In the second and principal part of the interview, the subjects were handed a list of memory aids and asked to rate on a 7-point scale how often they used each aid. The scale is given at the foot of Figure I. The points on the scale may appear to be strange choices. However, it was necessary to use the same scale for alI the aids whether they were used very frequently or hardly ever, and an alternative scale used in a pilot questionnaire had proved inadequate. 0090-502X/80/0 10031-08$01.05/0 32 HARRIS RESULTS AND DISCUSSION Table 2 Sheet Given to SUbjects in Last Part of Interview This section is divided into two subsections. The first subsection deals with the reported frequency of use of the aids asked about in the main section of the interview and with comparisons between different aids and subject REMEMBERING EVENTS, INFORMAnON, AND TO DO THINGS Below you will find examples of things you may need to remember from time to time. Please indicate (1) how you would set about remembering them and (2) how likely it is that you would be successful. 43 Probably Definitely Remember Remember 2 Even Chance Probably Forget Definitely Forget o effective, and psychologists have studied their products. However, use of these schemes is spontaneous and automatic, in the sense that they are not taught and that the user is probably not aware of using them. Therefore, such schemes represent normal memory operations and are not usually thought of as "mnemonics" or "memory aids." At the other extreme are the techniques that often have to be consciously learned and used, such as the "peg" and "loci" methods (see Table 1). Indeed, in anticipation of the results, it is probably the time and effort involved in learning aids of this type that limit the amount they are used. The important point here, however, is that there seems to be no clear cu toff point between these internal memory aids and what were referred to above as normal remembering schemes. Therefore, the internal aids included in the interview study were those encountered in the memory aid literature as having been investigated, plus some others that were merely mentioned. The aids were presented to the subjects on a typed sheet as shown in Table I. Question 18, the use of alarms for waking up, was not considered to be a memory aid, but it was included in order to help the subjects give clear responses to Questions 19 and 21, about the use of timed alarms for other purposes. Sometimes it was necessary for the interviewer to explain further some of the aids or to give examples of their possible use. This happened most often with the peg method and in checking that the subjects understood what was meant by an external aid in Question 30. In the last part of the interview, subjects were presented a typed sheet, as shown in Table 2. The table shows the form used for the Southampton students; the Cambridge subjects had a slightly different set of items (e.g., Question 37 was "An arrangement later today to meet a friend"). 33. A close friend's or relative's birthday. 34. An appointment with your dentist or doctor for, say, next week. 35. To take medicine or pills three times a day. 36. Where you have put something (pen, keys, handbag, etc.). 37. An appointment later today to see your tutor (or super- visor). 38. People's names (when you meet them for the first time). 39. People's faces (when you meet them for the first time). 40. To buy some bread on the way home. 41. A party at a friend's house in a month's time. 42. The new address of a close friend who has moved. 43. An important telephone number. 44. The meaning of a word you have come across for the first time. Table 1 List of Memory Aids Given to Subjects 10. Shopping lists. 11. First-letter "memory aids." For example, the first letters of "Richard Of York Gave Battle In Vain" give the colors of the rainbow. 12. Diary. 13. Rhymes. For example, "In fourteen hundred and ninetytwo Columbus sailed the ocean blue" aids memory of the date. 14.1he place method (method used since classical times). The items in a list to be remembered are imagined in a series of familiar locations. When recall is required one "looks" again in these familiar locations. 15. Writing on your hand what you need to remember. 16. The story method. For learning a list of items in order, a story is made up that connects them. 17.Mentally retracing a sequence of past events or actions in order to aid memory of something that happened, or to remember when you last had something you have now lost, and where you might have left it. 18. Alarm clock/radio (for waking up only). 19. Cooking timer with alarm or bell (for cooking purposes only). 20. Does your watch have an alarm system? (Yes/No) 21. Alarm clock/watch/radio/timer for purposes other than waking or cooking. -21a. Specify type(s) of alarm apparatus used. -21b.Specify what you use the alarm to remind you of. 22. The peg method. Before this method can be used, the learner has to learn to associate a word with each number up to, say, 50-e.g., "one is a bun; two is a shoe; three is a tree; ...." Once this has been learned, lists of items may be remembered in the following way. The first is imagined or associated in some way with "bun," the second with "shoe," and so on. 23. Turning numbers into letters (e.g., for telephone num- bers). 24. Memos (e.g., writing yourself special notes). -24a. Specify type of memo. 25. Face-name association. A mnemonic for learning people's names is to change their names into something meaningful and then to look for an unusual feature of their faces and then to associate the two; e.g., red-bearded Mr. Hiles may be imagined with hills growing out of his beard. 26. Alphabetical searching. When trying to recall a name or word, one can go through the alphabet letter by letter to find the initial letter; e.g., "Does his name begin with A ... B ... ah yes, C! It's Clark." 27_Calendar/yearplanner/wall chart. -27a. Specify how used. 28. Asking someone else to remind you. 29. Leaving something in a special place so that it will be encountered at the time it needs to be remembered. 30. Other idiosyncratic external memory aids, e.g., knotted handkerchief, changing rings to unfamiliar positions on fingers, turning wristwatch to underside of wrist. -30a. Specify aids used. 31. Other methods of committing things to memory. - 31a. Please describe these clearly. 32. Other methods of retrieving things from memory. - 32a. Please describe these clearly. The external aids in the list came from two sources: first, external aids, which the interviewer judged a priori as being used in everyday life and had been used in the pilot questionnaire study, and second, external aids mentioned by subjects in the pilot study in response to a question about other aids they used. With internal aids it was more difficult to know which to include. Some schemes, such as one that leads to clustering in free recall of categorized lists, appear to be widely used and MEMORY AIDS PEOPLE USE 33 groups. The second subsection reports briefly on the responses to questions in the other parts of the interview. Discussion of the possible discrepancy between reported frequency of use and actual frequency of use is left for the final section, which includes discussion of some more general problems of subjective assessment of memory phenomena. Frequency of Use Responses to items in the second and principal part of the interview will be reported separately for the two studies and for the three types of item (specific external, specific internal, and general open-ended); comparisons will be made where appropriate. Questions on specific items. There were nine items concerning specific external memory aids (use of an alarm for waking up is excluded, as it is not considered to be use of a memory aid) and nine concerning specific internal aids. The bar charts in Figure 1show the reported frequency of use for these aids in the first study (the lowest frequency external and internal aids are not included but are reported below). The abscissa in each bar chart represents the level of use of an aid; the ordinate represents the number of students, out of a total of 30, who claimed to use it at a particular level. The top row shows the eight reportedly most used external aids, and the bottom row shows the equivalent internal aids. Both rows have the more frequently used aids on the left, although the categories used and distributions found do not allow perfect ordering. As regards the least used internal aid, turning numbers into letters, none of the students reported ever having used it. The response frequencies for the least used external aid, use of an alarm fat purposes other than cooking or waking, were (left-to-right ordering, as in Figure 1) 24, 5, 0, 0,1,0, and O. The most apparent observation is how few of the internal aids were reported to be in frequent use, despite the fact that one might expect students to be among the most frequent users of these mnemonics. Interestingly, the two internal aids reported to be used most, mental retracing of events or actions and alphabetical searching, are rather different from the other aids in the study. They are both pure retrieval strategies, in that no special encoding effort needs to have been made in order to use them. The other internal aids the students were asked about provide schemes for encoding or learning. It is mainly these schemes that have been investigated, and yet they are seldom used. This preference among internal aids for retrieval strategies over schemes for learning or encoding may reflect a higher probability of usefulness for the retrieval strategies. Learning and encoding schemes must be employed before the user even knows whether "normal" ~m: 100". '5 ~~~~: 97"'. '5 ~~~~; n O / o '5 m~:S3"0 '5 ~~l~: 53",. ~O15 IS '5 MA. t/7",. ~~1kR: .0°,. EVEA. u"/., ~;:;~: 37°'. ~1~:R: 13.,. ';,~~~P,:20°,. >1 PER • USED· USEO- WEEK: 71. EvEA . •0·'...., ~~~~A, 43"10 :~;;~ to >1 PER WEEK: 0·'0 23~ '5 o ~012)456 5HOPPINC LIST 26t:J 15 :>1 PEA. 10 WEEK" 0.'. ~~~~: IlOt. ~:~~ 7°'0 A!t.l(INC SOWEQHE TO REMIND DIARY "015 15 EVEA. USED· 73°'. to >1PER 10 WEEI<: 3°/<1 '5 ~m:80% >,PEA WEEK: 3°,. 10 ~~: 97°'. ~ ~~::~ 23-/. elO < 15 °0123~S6 00123456 MENTAL PEG R[TRACINC O-NEVER USlO; '-UPTO TWICE IN LAST' ~NTHSJ 2-2 OR LESS'TIMES IN LAST" WEEkS; KEY 3-2 OR LESS TIMES IN LAST 2 WEEkS; .=t3 TO5 TIMES IN LAST :2WEEkS; 5- 6 TO 10 Tltwt£S IN l, ...ST 2 WEEkS; 6=f1 OR MORE TIUES IN LAST 2 wEEkS Figure 1. Reported frequency of use of memory aids by students (N = 30). The vertical scale is the number of students in each category 0 to 6. 34 HARRIS unaided memory would be sufficient, and he may, therefore, be wasting time and effort. Artificial retrieval strategies, on the other hand, need only be employed when it is already known that normal methods have failed. A comparable though more extreme pattern of results emerged in the study of 30 female Cambridge subjects. The equivalent frequencies for this group are given in Figure 2. Again the external aids were reported to be used more than the internal ones, and the most used internal aids were the pure retrieval strategies. The most obvious difference between Figures 1 and 2 is the far greater reported use of diaries and (writing on) calendars by the Cambridge women (Mann-Whitney Z values were 2.04, p < .05, and 5.13, p < .0001, respectively; all Z values in this section are corrected for ties, and p values are two-tailed). They also made more use of shopping lists (Z = 3.55, p < .001) and alarm cooking timers (Z = 3.18, p < .01) than did the students. Since half the students were male and all the Cambridge subjects were female, a sex difference in the use of aids could have contributed to these differences between the two studies. This interpretation receives some support from a comparison of the female and male students' use of external aids. Of the eight external aids shown in Figure 1, seven were reported to be used more by female than by male students, although in only one case (memos) did this reach two-tailed significance on a Mann-Whitney Utest [U(l5,15)=64, p<.05]. This suggestion of a sex difference in the use of external aids by students, whose life-style of lectures, private study, social and recreational activities is not so heavily sexdependent as among most other adult groups, led to additional questions being asked in the second study about the role of remembering birthdays and social events, the responses to which are reported below. The different pattern of use of external aids in the two studies may also have had a life-style or age component, independent of any sex difference. A comparison of the 15 female students with the 30 Cambridge women shows the older group reported more use of shopping lists (Mann-Whitney Z = 2.68, p < .01), writing on calendars (Z = 4.00, P < .0001), and alarm cooking timers (Z = 2.67, p < .01). The older group also declared less use of writing on their hands as a reminder than did the female students (Z = 2.45, p < .025). Open-ended questions. The general, open-ended questions asked at the end of the main section produced rather low-frequency responses, only 31% of the total 180 (3 questions X 60 subjects), eliciting any use of other aids at all. Even among these the frequency of use was still usually rated as very low. The first of these questions (Number 30) was about other external aids and gave as examples knotted hand- 1S loll ~~~~100°,. ~ >IPER 0 -c10 WEEK;63 10 16 1'~[V[A. no,.1S '5 ~~~~: lOOo'o 15 ~~~~: 97°/. tS [VER • 15 ~~~~: 13°1. 15 ~~~~: 930'. 15 USEO' ~~i~; 90'. USEO:'? J. >1 ~[tl > I PEA 0 >1 PER :~~~~ 40·'. ~I[~~'': 20·'. WEEK : 0·'.WEEK; 50 /. >1 PEA :37·'.>1 PER WEEK; 47°'. WEEK to W[[1(:630,. 10 10 10 '0 10 10 -' < Z a: w 5 >- X 15 ~~~~: 100 0,. V') > I PER o WEEK: 37°/. :;:'0 10 16 EVER 0 16 15 USEO:701.'5 ~~::~ a·'. 10 10 ~~i~: 47°/. ,.1 PER WEEK=O'o 2·~EVER • 15 USED- 20 1. > I PER 0 WEEK ; 0 '. 10 10 10 10 -' -c Z a: 5 w >- Z J,A[NTAl RETRACING OsNEVERUSEO; ,-UP TO "WICE IN LAST 6~ONTHS; 2_20R LESS liMES IN LAST 4 WEEKS; KEY 3".. 2 Oil LESS TI'-4ESIN LAST 2 WEEkS; 4-3 TO 5 TIMES IN LAST 2 W£EkSj 5=6 T010Tn,us IN LAST 2 W££KS; 6=11 OR MORE TIMES IN LAST 2 WEEKS. Figure 2. Reported frequency of use of memory aids by female Cambridge Panel subjects (N = 30). The vertical scale is the number of subjects in each category 0 to 6. MEMORY AIDS PEOPLE USE 35 Southampton 5 19 5 Cambridge 13 7 6 Table 3 Number of Subjects Mentioning at Least One Memory Aid in Response to a Question About Aids not Already Mentioned Note-Each figure is from a total of 30 subjects. Question 30 = other external aids; Question 31 = other methods ofcommitting things to memory; Question 32 = other methods of retrieving things from memory. kerchiefs, changing rings to unfamiliar positions on fingers, and turning a wristwatch to the underside of wrist. These examples are aids that are purely symbolic in the sense that they are not themselves the things that need to be remembered. This is in contrast, for example, with leaving a library book in an unusual place in order to remember to return the book itself to the library; this type of aid is covered by Question 29. Symbolic aids may be very bizarre and idiosyncratic, since they have no logical connection with what they are being used to remember, and, presumably, the more outrageous the misplacement or distortion, the greater the chance that it will be noticed. However, such symbolic external aids may be less effective than ones clearly related to what needs to be remembered. I have described elsewhere (Harris, 1978) some criteria for the effectiveness of cues provided by external memory aids. For example, an effective cue should be a specific reminder for the action required; in contrast, a symbolic aid such as a knotted handkerchief may only remind its user that something must be remembered but not what that something is. Interestingly, only one Southampton student and five Cambridge subjects reported ever having used a knotted handkerchief, and all six rated it in the lowest usage category, less than three times in the last 6 months. The failure of symbolic aids to satisfy the specificity criterion makes the limited reported use less surprising. A breakdown by question and study of the three general open-ended questions is given in Table 3. On Question 31, more students than Cambridge panel subjects gave examples of other methods they had used of committing things to memory, [x2 (1) =8.21, p <.01]. This presumably reflects students' need to remember lectures and to review for examinations. This was often made explicit in their answers (e.g., "Making notes actually helps me remember for exams"). On the other hand, for Question 30 significantly more of the panel subjects gave examples of other external aids they had used [X2(1) =3.88, p < .05]. Among the internal methods mentioned by the Cambridge subjects in response to the open-ended questions were some shopping mnemonics. One of these methods was to remember the total number of items to be bought, as well as what they are. This ensures that one does not terminate the shopping trip too early, leaving some items unbought, and also allows one to make a confident decision to go home when the right number of items has been bought. The second shopping mnemonic mentioned was a pure retrieval one. It consists of imagining the contents of the bread bin, the larder, the refrigerator, and so on, as last seen, in order to remember how empty they were. This can then be used as a basis for what needs to be bought. It is clearly related to the method of loci, although that involves encoding by imagining things in places where they have never been and so is not a pure retrieval method. There are also other related shopping mnemonics, such as mentally running through a checklist of items ("Have I got any bread, butter, eggs, ... ?") or meals ("Have I got something for lunch tomorrow, dinner on Saturday, ... ?"); both these versions were mentioned by a third Cambridge subject. Frequency of use and other studies. Subjects have also been questioned about the use of memory aids in other studies. Gruneberg (1973) found that some 53% of graduates who returned a questionnaire reported having used mnemonics (at least "rarely") to remember information from essays while preparing for psychology final examinations. The first of three relevant questions that Kreutzer et aI. (1975) asked children concerned what methods they could think of that would ensure that they would remember to take their skates to school the next day. The second was a similar question about remembering a friend's party. The third question, concerning a friend trying to work out the age of his dog, was about what he could do to help himself remember which Christmas he had received the puppy. Kreutzer et a!. (1975) categorized the methods given by the children as being either "inner" or "outer," a distinction similar to the one made between internal and external aids in this paper. The three questions all produced far more outer than inner methods as responses from the children in all four age groups (approximately 6, 7, 9, and 11 years). The ratios of outer to inner methods given as responses by all 80 subjects were 98 : 14 for the first question described above, 97: 24 for the second, and 62: 27 for the third. Perlmutter's (1978) battery of objective and subjective assessments included questions on memory strategy use. Her subject groups were defmed by age (means = 23 vs. 62 years), education, and sex. Unfortunately, Perlmutter gave no details of how responses were categorized or scored. She merely stated, "Most subjects reported that they often used memory strategies. For example, they reported using appointment books and shopping lists, and said they deliberately tried to relate information they wanted to remember to other information they thought would increase the probability that it would be remembered. An analysis of variance of 3231 Question 30 Subject Group 36 HARRIS Note-Each figure is from a total of30 subjects. Southampton 23 12 3 Cambridge 28 21 21 Table 5 Ownership During Current Year of Diaries, Calendars, and Dated Planners or Wall Charts composite scores of memory strategy use failed, however, to reveal any systematic group differences in overall strategy use" (Perlmutter, 1978, pp. 335-336). SOME PROBLEMS IN SUBJECTIVE ASSESSMENT OF MEMORY PHENOMENA While laboratory studies have produced objective data concerning whether particular memory aids can be used effectively, they can tell us nothing about what aids people actually do use to cope with their normal memory loads. Adequate theory and understanding in any area of memory and cognition must take account not only of what people can do but also of what they actually do in normal situations. While objective data on what people normally do is hard to obtain, a number of recent investigations into memory and related topics have used a variety of self-report methods. These include questionnaires (e.g., Herrmann & Neisser, 1978; Perlmutter, 1978; Broadbent, Note 2), diaries (e.g., Reason, 1976), and structured interviews (e.g., the present studies). The Problem of Validation When subjective methods are used for monitoring remembering and forgetting in the real world, it is tempting to quote, by way of validation (as do Herrmann & Neisser, 1978) studies that have shown a close relationship between actual performance and knowledge of (or belief or confidence in) that performance. This work includes the "feeling of knowing" (Hart, 1965), "sense of direction" (Kozlowski & Bryant, 1977), and the often found correlation of confidence with being correct in, for example, recognition experiments. Perhaps the most important in the present context is the work on metamemory by Flavell and others (see Flavell & Wellman, 1977). This work has started to map out the developmental changes in children's knowledge about remembering, showing, for example, the increasing accuracy with which they can predict their memory performance in different situations (e.g., Flavell, Friedrichs, & Hoyt, 1970; Markman, 1973; Yussen & Levy, 1975). The metamemory of adults has received less direct attention. However, the oldest groups in some of the developmental studies have been young adults (e.g., 21 years, Salatas & Flavell, 1976; 20 years, Yussen & Levy, 1975). These experimental groups have shown that adults do have considerable and often accurate information about their memories and knowledge about strategies for remembering. Despite these studies we do not yet know what are the boundaries within which subjective data are reasonably accurate. What is known is that some situations lie outside these boundaries. Consider, for example, a study by Wilkins and Baddeley (1978). Subjects were instructed to operate a button on a small box at four fixed times each day; a clock system inside the box recorded the time on each occasion the button was pushed. Wilkins and Baddeley found that when people forgot to push the button, they often forgot to record this forgetting, although they had been instructed to Dated Planner or WallChart 27 1 19 10 Wife's Joint Role Responsibility Date- Only Calendar 1 o Diary Husband's Role Subject Group Question Birthdays Social Events Table 4 Responses to Questions on Whose Role It Is Within the (Female) Respondents' Families to Remember Oose Friends' and Relatives' Birthdays and Social Events Such as a Meal With Friends Additional Questions None of the 60 subjects reported ever having bought a book about memory improvement; 4 reported having read such a book, and another 16 reported having read passages of similar advice in a book or magazine or having heard it broadcast. Responses to the questions (only asked the Cambridge subjects) about the role of remembering birthdays and social events are shown in Table 4. They show that the wives overwhelmingly regarded these tasks as their role. Responses to questions on ownership of diaries, calendars, and wall charts are shown in Table 5. The wider ownership of these aids among the Cambridge subjects is in accordance with the more frequent usage these subjects reported. In the final section of the interview, subjects were asked how they would set about remembering a number of things and how likely they would be to remember them. Often, subjects gave replies such as "I would just try to remember" and could not explain how. Another problem was multiple replies, such as "I might write it down, in which case I would remember, or I might just commit it to memory, in which case I would probably forget." Therefore, the answers to these questions cannot be used to present a fair comparison of the use of different memory aids. However, they did in many cases provide a consistency check against the main section questions. This is discussed below. Note-Rows sum to 29 because one of the two unmarried subjects was unable to respond. The other unmarried subject responded on the basis of her arrangements with her fiance; similarly, the widowed subject responded on the basis of her arrangements with her late husband. make a note of it when they realized they had forgotten. Another example is provided by Warrington and Sanders (1971). Although old people report that they can recall distant memories better than recent ones, some objective tests show the opposite trend. These examples illustrate the dangers of inductive validation. Reliability and Intennethod Checks Despite the problems encountered in attempting to validate subjective memory assessment, there are data checking procedures whose positive outcome is a necessary but not sufficient condition for an assessment to be valid. One commonly used check is that of test-retest reliability (e.g, Herrmann & Neisser, 1978). However, this cannot show up consistent distortions produced by a particular method of assessment. Such a distortion was revealed to have occurred in the questionnaire that was used as a pilot for the interview studies reported in this paper. In the pilot questionnaire, Southampton University students (26 students returned the questionnaire out of 50 who received it) were asked how often they used different sorts of memory aids, using the response categories never, rarely, sometimes, and often. However, the results seemed to indicate that the scale might be producing a range effect and that, for example, subjects rated once a week as rare for a diary, but as often for first-letter mnemonics. In the main studies, therefore, a more objective (7-point) scale was used, with points such as "up to twice in the last 6 months" being less open to varying interpretation. The results (see Figure 1) confirmed the earlier suspicion that in the pilot study the response category labels had allowed a substantial range effect, eliminating the frequency differences between internal and external aids that successfully emerged in the main study. This throws doubt on comparisons between items that have been rated on frequency scales with points such as "sometimes" or "often." For example, Herrmann and Neisser (1978) used the points "never," "once in a while," "now and then," "about half the time," "fairly often," "very often," and "always." Most of the questions were about forgetting things in particular situations. As well as being open to the type of range effect mentioned above, this scale confounds the frequency of the situation with the frequency of forgetting given the situation: The fourth and seventh points on the scale seem to indicate relative frequency (i.e., frequency of forgetting with respect to possible situations), whereas the second, third, fifth, and sixth points seem to indicate absolute frequency (i.e., frequency of forgetting with respect to time); "never" can obviously refer to either relative or absolute frequency. Returning to the inability of test-retest procedures to show up consistent distortions produced by a particular method, it is obviously advantageous to perform a within-study, intermethod check; then not only statistical results can be cross-checked, but also raw data from individual protocols. This was attempted in the present MEMORYAIDS PEOPLE USE 37 studies by not only asking subjects what memory aids they used, but also including a subsequent section with questions on how they would cope with certain everyday memory situations. Sometimes it turned out in the responses to the latter questions that the subjects frequently used aids not mentioned in the earlier section. For example, it was clear that many of the Cambridge subjects made extensive use of address books, telephone books, and birthday books, but among those who declared use of them in response to the memory situation questions, very few had mentioned them in response to the earlier open-ended questions on memory aids, and there were no specific questions about them. For aids about which there were specific questions in the first section (e.g., diaries), such discrepancies hardly occurred. In other words, if subjects stated that some of the items asked about in the later section would be remembered by using a diary, then they had rated a diary as being used at least fairly often in the earlier section. Specific vs. General Questions So it appears that the subjects had some sort of difficulty in answering the general, open-ended questions that they did not have when answering specific questions about individual types of memory aid. It seems likely that this failure to recall certain much used aids represents a retrieval failure. In other words a specific question actually mentioning an aid is a sufficient retrieval cue for the appropriate information to be accessed, whereas the generalized question, "What other memory aids do you use and how often do you use them?" is not. This points to two potentially empirical questions. The first is whether specific questions always lead to greater consistency (or even validity) than do general ones. The second question is whether it is possible to provide some sort of retrieval plan or cues to help achieve accurate answers to general questions. This might be done by givingspecific examples of the general memory phenomenon the question is about (e.g., "How often do you forget you have already done something and start to do it again, like winding your watch or pouring yourself a drink?"). There are obvious dangers in such a technique. Assuming for the moment that general questions are difficult to answer and specific ones easier, the respondent may answer the question as if it were "How often do you pour yourself a drink twice?" and go no further than the given examples in thinking of situations in which he attempts to do things twice. Additionally, such specific questions may not apply to a particular respondent-he may never need to wind his watch. Questions of exactly the sort exemplified have been asked in recent studies (e.g., Herrmann & Neisser, 1978; Question 15). An alternative approach to overcoming retrieval problems in general questions is that used by Sunderland, Harris, and Baddeley (Note 3). One part of a battery of tests and questions used in a study of closed head injury and memory was the use of a special "diary." Every 38 HARRIS evening for I week the subjects went through a checklist of possible memory failures that they might have suffered during the day. Restricting the target period of search in this way may alleviate the retrieval problem, first, because it is a small, well defined area to search, and second, because, since the events are recent, they may be more accessible. CONCLUDING REMARKS Finally, a caveat should be added to the finding that the external aids were reported to be used more than the internal ones. In the Method section of this paper, it was explained that there is no clear division between internal memory aids and normal remembering, so that the internal aids specifically mentioned in the interview studies were chosen from the psychological literature on memory aids. It may be that psychologists have not chosen to investigate or mention the most commonly used internal aids: Had there been specific questions on other internal methods of remembering, such as the shopping mnemonics mentioned in the Results section, the results might have been rather different. One might even choose to differentiate internal memory aids and normal remembering schemes on the basis of frequency of use itself. The complex semantic, episodic, working, and other memory processes upon which cognitive functioning depends presumably represent normal memory, but, in contrast with the classical mnemonics, the rememberer cannot report on their frequency of use. However, the dependence of momentto-moment functioning upon these processes leads to the inevitable conclusion that, whatever may be the case with aids, internal methods of remembering far outstrip external ones in frequency of use. REFERENCE NOTES I. Higbee, K. L. A century of research on mnemonics: We're back where we started. Paper presented at the meeting of the Western Psychology Association, Los Angeles, April 1976. 2. Broadbent, D. E. The Cognitive Failure Questionnaire (CFQ). In J. E. Harris (Convener), Assessment ofcognitive functioning in everydaylife. Symposium presented at the Conference of the British Psychological Society, London, 1979. 3. Sunderland, A., Harris, J. E., & Baddeley, A. D. Everyday memory following closed head injury. Manuscript in prepara- tion. REFERENCES BROWN, E., & DEFFENBACHER, K. Forgotten mnemonists. Journal of the History of the Behavioral Sciences, 1975, 11, 342-349. CARLSON, R. F., KINCAID, J. P., LANCE, S., & HODGSON, T. Spontaneous use of mnemonics and grade point average. Journal ofPsychology, 1976,92, 117-122. FLAVELL, J. H., FRIEDRICHS, A. G., & HOYT, J. D. Developmental changes in memorization processes. Cognitive Psychology, 1970,1,324-340. FLAVELL, J. H., & WELLMAN, H. M. Metamemory. In R. V. Kail & J. W. Hagan (Eds.), Perspectives on the development of memory and cognition. Hillsdale, N.J: Erlbaum, 1977. GRUNEBERG, M. M. The role of memorization techniques in finals examination preparation-A study of psychology students. Educational Research, 1973, 15,134-139. HARRIS, J. E. External memory aids. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.), Practicalaspects of memory. London: Academic Press, 1978. HART, J. T. Memory and the feeling-of-knowing experience. Journal ofEducational Psychology, 1965,56,208-216. HERRMANN, D. J" & NEISSER, U. An inventory of everyday memory experiences. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.), Practical aspects of memory. London: Academic Press, 1978. HIGBEE, K. L. Some pseudo-limitations of mnemonics. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.), Practicalaspects ofmemory. London: Academic Press, 1978. HIGBEE, K. L. Recent research on visual mnemonics: Historical roots and educational fruits. Review of Educational Research, in press. HUNTER, 1. M. L. Memory in everyday life. In M. M. Gruneberg & P. E. Morris (Eds.), Applied problems in memory. London: Academic Press, 1979. KOZLOWSKI, L. T., & BRYANT, K. J. Sense of direction, spatial orientation and cognitive maps. Journal of Experimental Psychology: Human Perception and Performance, 1977, 3, 590-598. KREUTZER, M. A., LEONARD, C., & FLAVELL, J. H. An interview study of children's knowledge about memory. Monographs ofthe Society for Research in Child Development, 1975, 40(1, Serial No. 159). MARKMAN, E. Factorsaffecting the young child's ability to monitor his memory. Unpublished doctoral dissertation, University of Pennsylvania, 1973. PERLMUTTER, M. What is memory aging the aging of? Developmental Psychology, 1978, 14,330-345. REASON, J. T. Absent minds. New Society, November 4, 1976, pp. 244-245. SALATAS, H., & FLAVELL, J. H. Retrieval of recently learned information: Development of strategies and control skills. Child Development, 1976,47,941-948. WARRINGTON, E. K., & SANDERS, H. 1. The fate of old memories. Quarterly Journal of Experimental Psychology, 1971, 23, 432-442. WILKINS, A. J., & BADDELEY, A. D. Remembering 10 recall in everyday life: An approach to absent-mindedness. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.), Practical aspects ofmemory. London: Academic Press, 1978. YUSSEN, S. R., & LEVY, V. M., JR. Developmental changes in predicting one's own span of short-term memory. Journal of Experimental Child Psychology, 1975, 19, 502-508. NOTE 1. North American readers should note that in Britain an appointment book is called a diary (the word diary was used with this meaning throughout the study) and that the total number of such diaries published annually in the United Kingdom is about 20 million. (Received for publication June 7, 1979; revision accepted October 12, 1979.)