Impact Evaluation Report 16 Rukmini Banerji James Berry Marc Shotland Education The impact of mother literacy and participation programmes on child learning Evidence from a randomised evaluation in India August 2014 International Initiative for Impact Evaluation About 3ie The International Initiative for Impact Evaluation (3ie) was set up in 2008 to meet growing demand for more and better evidence of what development interventions in low- and middle-income countries work and why. By funding rigorous impact evaluations and systematic reviews and by making evidence accessible and useful to policymakers and practitioners, 3ie is helping to improve the lives of people living in poverty. 3ie Impact Evaluations 3ie-supported impact evaluations assess the difference a development intervention has made to social and economic outcomes. 3ie is committed to funding rigorous evaluations that include a theory-based design, use the most appropriate mix of methods to capture outcomes and are useful in complex development contexts. About this report 3ie accepted the final version of this report, The impact of mother literacy and participation programmes on child learning: evidence from a randomised evaluation in India, as partial fulfilment of requirements under OW2.153 issued under Open Window 2. The content has been copyedited and formatted for publication by 3ie. Due to unavoidable constraints at the time of publication the content of the report was not changed to conform to 3ie editorial style. All of the content is the sole responsibility of the authors and does not represent the opinions of 3ie, its donors or its Board of Commissioners. Any errors and omissions are also the sole responsibility of the authors. All affiliations of the authors listed in the title page are those that were in effect at the time the report was accepted. Any comments or queries should be directed to the corresponding author, James Berry at jimberry@cornell.edu. Funding for this impact evaluation was provided by 3ie’s donors, which include UKaid, the Bill & Melinda Gates Foundation, Hewlett Foundation and 12 other 3ie members that provide institutional support. A complete listing is provided 3ie website at www.3ieimpact.org/en/about/3ie-affiliates/3ie-members. Suggested citation: Banerji, R, Berry, J and Shortland, M, 2014. The impact of mother literacy and participation programmes on child learning: evidence from a randomised evaluation in India, 3ie Impact Evaluation Report 16. New Delhi: International Initiative for Impact Evaluation (3ie) 3ie Impact Evaluation Report Series executive editors: Jyotsna Puri and Beryl Leach 3ie editors: Radhika Menon and Markus Olapade Production manager: Lorna Fray Managing editor: Omita Goyal Assistant managing editor: Kanika Jha Assistant production manager: Rajesh Sharma Copy editor: Lucy Southwood Proofreader: Mathew P.J. Cover design: John F McGill Printer: Via Interactive Cover photo: Ravi Gupta/JPAL-South Asia © International Initiative for Impact Evaluation (3ie), 2014 The impact of mother literacy and participation programmes on child learning: evidence from a randomised evaluation in India Rukmini Banerji Annual Status of Education Report Centre, India James Berry Cornell University, USA Marc Shotland Abdul Latif Jameel Poverty Action Lab, USA 3ie Impact Evaluation Report 16 August 2014 ii Acknowledgements We thank Annie Duflo for collaboration and insights on the study design and implementation. We are also grateful to Jessica Chan, Nandini Gupta, Ravi Gupta, Rachna Nag Chowdhuri and Nikhil Wilmink for superb work coordinating the field activities in Rajasthan and Bihar. Laurel Wheeler and Kevin Kho provided excellent research assistance. This research was funded by the International Initiative for Impact Evaluation (3ie). All errors are our own. The views expressed in this report are not necessarily those of 3ie or its members. iii Abstract We report the results of a randomised evaluation of three programmes designed to improve home learning environment among rural households in India. Households were assigned into one of four groups that received either: (1) adult literacy classes for mothers, (2) training for mothers on how to enhance their children’s learning at home, (3) a combination of the first two interventions, or (4) nothing. The latter serves as the control group. We find that mothers in the first three groups perform 0.11, 0.06 and 0.15 standard deviations better (respectively) on a combined language and math test when compared to the control group. We find that the three programmes had statistically significant effects of 0.04, 0.05 and 0.07 standard deviations on children’s math scores (respectively), but only the combined intervention had significant effects on language scores. We also find that the interventions increased women’s empowerment, mothers’ participation in child learning and the presence of education assets in the home. The key policy implication of our study is that the effect sizes may not justify the programmes as a tool to improve child learning alone, but they can serve as a useful tool to policymakers interested in influencing child learning, mothers’ learning and the home learning environment. iv Contents Acknowledgements .................................................................................................ii Abstract................................................................................................................iii List of figures and tables .......................................................................................v Abbreviations and acronyms ................................................................................vi 1. Introduction.................................................................................................... 7 2. Programme description..................................................................................10 3. Conceptual framework and theory of change.................................................12 3.1 Conceptual framework ................................................................................12 3.2 Theory of change .......................................................................................14 4. Data collection ...............................................................................................15 5. Results – mothers..........................................................................................16 5.1 Estimating equation....................................................................................16 5.2 Programme take-up....................................................................................17 5.3 Test scores ................................................................................................19 5.4 Intermediate outcomes ...............................................................................20 5.5 Empowerment and time use ........................................................................20 6. Results – children ..........................................................................................22 6.1 Test scores ................................................................................................22 6.2 Intermediate outcomes ...............................................................................23 7. Impact heterogeneity ....................................................................................24 7.1 Mothers.....................................................................................................25 7.2 Children ....................................................................................................25 8. Internal and external validity ........................................................................26 8.1 Internal validity..........................................................................................26 8.2 External validity .........................................................................................26 8.3 Cost-effectiveness ......................................................................................28 9. Policy recommendations ...............................................................................30 10. Conclusion. .............................................................................................31 Appendix A: Location selection............................................................................45 Appendix B: Power calculations.......................................................................... 47 References...........................................................................................................56 v List of figures and tables Figure 1 Factors that contribute to and influence child learning1 ..................................13 Table 1 Randomisation check ...................................................................................32 Table 2 Take-up of mother literacy classes.................................................................33 Table 3 Determinants of mother take-up ...................................................................34 Table 4 Mother learning...........................................................................................35 Table 5 Family assistance in mother learning .............................................................32 Table 6 Empowerment ............................................................................................33 Table 7 Mother time................................................................................................34 Table 8 Child learning .............................................................................................35 Table 9 Child schooling............................................................................................36 Table 10 Mother-child participation ...........................................................................37 Table 11 Child time use ...........................................................................................38 Table 12 Education assets........................................................................................39 Table 13 Mother perceptions ....................................................................................40 Table 14 Heterogeneity in impact .............................................................................41 Table15 a: Heterogeneity in impact outcome: child test scores 42 Table15 b: Heterogeneity in impact outcome: child test scores.....................................43 Table 161: Cost effectiveness of interventions............................................................44 Appendix Table 1: Baseline means of variables in Rajasthan and Bihar samples .............49 Appendix Table 2a: Question-wise literacy baseline means ..........................................50 Appendix Table 2b: Question-wise numeracy baseline means.......................................51 Appendix Table 3: Mother question-wise treatment effects – language ..........................52 AppendixTable 4: Mother question-wise treatment effects – math............................53 Appendix Table 5: Child question-wise treatment effects – language.............................54 Appendix Table 6: Child question-wise treatment effects – math ..................................55 vi Abbreviations and acronyms ASER Annual Status of Education Report CHAMP Child Home Activities and Materials Packet IV instrumental variables ML mother literacy NGO non-governmental organisation Obs observations OLS ordinary least squares sd standard deviation 7 1. Introduction Improving the quality of primary education in the developing world remains a crucial issue for researchers and policymakers alike. While developing countries have made significant gains towards universal enrollment, with a net enrollment rate of 90 per cent in low- and middle-income countries in 2011 (World Bank Group 2013), learning has not matched this progress. For example, a 2012 survey in India found that 96 per cent of rural primary school-aged children were enrolled, but only 38 per cent could read a simple story (ASER Centre 2013). Low-quality education is often considered the result of a low-quality education system, characterised by poor school infrastructure, limited materials, inappropriate pedagogy and low-quality teachers. To address this issue, researchers are building evidence on interventions that improve education in developing countries through changes in inputs at school (Kremer et al. 2013). However, low learning levels can also be attributed to the home environment: in low-income households parents spend less time on educational activities with their children, are less productive with the time they spend, have lower expectations and allocate fewer resources to education. All of these factors are believed to be directly related to the parents' low education levels. While the correlations between parents’ education levels, the home environment and ultimately child outcomes are strong, endogeneity makes it difficult for researchers to establish causal links (Thomas et al. 1996). However, for policymakers who face a generation of parents with already low levels of education, perhaps the more important question is: if the household environment is indeed an important factor in a child’s education, can policies targeted toward parents help promote a better home learning environment? In this case, evidence on effective interventions is much more limited, as we discuss below. With the aim of influencing the home environment, some policymakers have initiated adult education campaigns, and this movement has been gaining momentum. In India, the National Literacy Mission was launched in 1988 ‘to impart functional literacy to non-literates in the age group of 15–35 years in a time-bound manner’ (Government of India 2014). In 2009, the Prime Minister of India launched Saakshar Bharat (literate India), the revised version of the National Literacy Mission, aiming to achieve an 80 per cent literacy rate and reduce the gender literacy gap to 10 per cent by 2012 (The Hindu, 2009). Many other countries and donors are investing in such programmes, in part because they could promote children’s schooling (UK Department for International Development 2008). Unfortunately, evidence of the effectiveness of such programmes on child learning is sparse, especially in contexts where parents have little to no formal education. This study is designed to evaluate whether child learning can be improved through interventions focused on improving the mother's human capital and/or through interventions that work with the mother but are focused on enhancing at-home learning for the child. We present the results of a randomised evaluation of three interventions in rural India designed by Pratham, an Indian education NGO,1 to improve child learning through increased mother literacy and direct encouragement of home learning. 1 Pratham is a large, India-wide NGO specialising in child literacy and numeracy. For more information, see www.pratham.org 8 We test for these effects by randomly assigning villages to one of four groups. In the first group, mothers in the village are offered the mother literacy (ML) intervention: daily literacy and math classes. In the second, mothers are given the child home activities and materials packet (CHAMP) intervention: materials, activities, and training each week to promote enhanced involvement in their children’s education at home. In the third, mothers are offered both the literacy and enhanced home learning intervention (ML-CHAMP). The fourth group serves as a control with no intervention. The evaluation was carried out in 480 villages in the states of Bihar and Rajasthan. In each state, 240 villages were randomly assigned in equal proportions to the four groups. We provide evidence that these programmes can affect a broad set of learning outcomes as well as improve children's home learning environment. For mothers, the ML programme increased learning outcomes by 0.11 standard deviations (sd), CHAMP increased test scores by 0.06 sd, and ML-CHAMP increased test scores by 0.15 sd. We also find significant impacts of each of the three programmes on an aggregate measure of women’s empowerment outcomes. Turning to the results for children, we find that the ML, CHAMP and ML-CHAMP increased child math scores by 0.04, 0.05 and 0.07 sd, respectively. The only significant impacts on language scores were in the combined interventions. We find little evidence that the programmes affected formal schooling behaviour, but each of the three interventions affected mother’s self-reported participation in child learning and educational assets in the home. The evidence is therefore consistent with the interventions improving child learning by changing the home environment, particularly through increased productivity of the time children spend studying. However, we cannot rule out that the interventions affected children directly through either child attendance at ML classes or through direct participation in CHAMP sessions. We also note that these are short-term effects, measured after the programmes had only been running for one year. More research is needed to study whether these effects persist well after the programmes have ended. Our child learning effects are small relative to similar interventions that target child learning: a recent survey by Kremer et al. (2013) lists 15 interventions that have statistically significant effects sizes ranging from 0.14 to 0.6 sd, while our estimate for the combined ML-CHAMP programme is 0.07. However, we find effects on a broader set of outcomes of policy interest. Our study's key policy implication is therefore that, while the effects sizes may not justify the programmes as a tool to improve child learning alone, they can be a useful tool for policymakers who are interested in influencing child learning, mother learning and the home learning environment. Our study adds to the literature that asks whether the skills believed to help parents influence their child’s learning can be acquired as an adult. Such programmes fall into three categories: (1) adult literacy, (2) child participation and (3) family literacy. The latter typically bundle the first two together, along with other components such as job training and remedial education for children, in different combinations. 9 These programmes are also implemented in different contexts: (1) in higher-income countries, where parents have had some personal experience with a formal education system and varying levels of literacy (2) in developing countries, where there is much less exposure to formal education and literacy levels are far lower. We focus our literature review on research in developing countries, as parents in poor countries are less likely to have had substantial experience of formal education and are therefore likely to respond differently to these programmes. However, we also highlight particularly relevant studies from higher-income countries. Although several evaluations attempt to establish the impact of developing-country adult literacy programmes on adults and children, much of the research has methodological limitations. Some studies find significant impacts of adult literacy programmes on adult learning using ex-post comparison with non-participants (Carron 1990; Ortega and Rodriguez 2008). In a randomised evaluation of a programme providing cell phones to participants in existing adult education classes in Niger, Aker et al. (2012) find significant impacts on math and literacy scores. However, they do not evaluate the adult literacy programme, per se. Research on the effects of adult literacy programmes on children’s outcomes is sparse, and these studies tend to rely on retrospective selection of a comparison group (Aoki 2005; Abadzi 2003).2 There are few existing studies evaluating developing-country participation programmes that encourage parents to be more involved in their children’s schooling. Bekman (1998) evaluates a Turkish programme that trained mothers to help educate their children at home. Using a matching procedure to construct a comparison group, the study finds that the programme has large effects on child learning. In the developed country context, the randomised evaluation of a programme in France to enhance parental involvement in their adolescent children's education found significant positive effects on parental and student participation, student attitudes and students’ grades in school (Avvisati et al. 2014). In the family literacy movement, we only know of one randomised evaluation: St Pierre et al.’s (1993) evaluation of the National Even Start Program in the US. The authors find no statistically significant effects on child learning, performance or parental help with studies. However, the sample size was small and take-up was low. Our study adds to the prior literature by providing, as far as we are aware, the first randomised evaluation of (1) an adult literacy programme, (2) a participation programme, and (3) a combined family literacy programme in a developing country. We also examine impacts on both adult and child outcomes, a feature that is relatively rare in prior literature. 2 Although there are numerous evaluations of adult literacy programmes in the US, much of the research also suffers from methodological limitations (Beder 1999). 10 Our paper is structured as follows. Section 2 discusses the programmes and context. Section 3 outlines a conceptual framework for the programmes' effects on child learning. Section 4 covers the study design, data collected and analysis. Section 5 describes the results for mothers and Section 6, the results for children. Section 7 analyses heterogeneity in test score impacts. Section 8 discusses internal and external validity and includes a cost-effectiveness analysis. We discuss implications for policy in Section 9 and set out our conclusions in Section 10. 2. Programme description The interventions were conducted in two blocks (district subdivisions) of Bihar's Purnia district and two blocks of Rajasthan's Ajmer district. Pratham selected Bihar and Rajasthan for two key reasons. First, both states share low literacy levels. According to the latest census, they have the lowest female literacy rates in India, at roughly 53 per cent each (Government of India 2011a). They have similar children’s education outcomes, with 48 per cent of rural children in grades 3 to 5 reading at Grade 1 level in both states, just below the national average of 54 per cent (ASER Centre 2013). Pratham selected the intervention districts within each state because they had existing programmes and infrastructure in those areas. Within the intervention districts, the blocks were selected because they did not have any preexisting Pratham programmes. Second, while education outcomes are similar between the two states, Rajasthan and Bihar provide distinct contexts in which to study the interventions.3 Rajasthan ranks substantially higher than Bihar along several other key dimensions of economic development. Bihar has the lowest GDP per capita of any state in the country, and while Rajasthan is below the national average, its per capita GDP is double that of Bihar (Central Statistics Office of India 2013). Similarly, in Rajasthan 67 per cent of households have electricity – about the national average – while in Bihar, ranked last among Indian states, only 16 per cent of households have electricity (Government of India 2011b). Households in our sample broadly follow these patterns. Appendix Table 1 displays the differences in baseline education, wealth, demographic and time use variables between the two states. The average education level for a mother in our sample is under one year for Rajasthan and Bihar, and both have similar scores on our baseline test, with Rajasthan mothers scoring slightly higher in math. Child learning levels are also slightly higher in the Rajasthan sample. Households in the Rajasthan sample have substantially more assets and are more likely to be electrified. Women in the Rajasthan sample spend more time working (46 hours per week compared to 26), while women in the Bihar sample spend more time per week reading to their children or helping with homework (2.4 hours per week compared with 1.4). In addition to these contextual differences, each intervention was implemented by a separate local team and was supervised by separate state-level Pratham leadership. 3 In Section 7 we explore heterogeneity in the programme effects on mother and child learning by state. 11 In each state, Pratham selected 240 hamlets for the randomisation, based on a target number of households (the approximate size that could support one mother literacy class) and geographic distance from other target locations to limit spillovers. In Rajasthan, where villages are typically far apart, one appropriately-sized hamlet per village was selected, and the randomisation was effectively conducted at village level. In Bihar, where hamlets may be close to one another (whether in the same village or in different villages), hamlets of the target size were included if they were sufficiently far from other included hamlets.4 For ease of exposition, we refer to the randomisation unit as a ’village’ throughout. In each state, 60 villages were randomly assigned to each of the four treatment groups. Randomisation was stratified geographically to allow Pratham to organise its monitoring structure based on a known number of programme villages in each area. The 240 villages in each state were first divided into geographically proximate ‘clusters’ of 20 villages. These clusters were further divided into two ‘phases’, which determined the order of the rollout of the programmes. The Pratham team first rolled out the interventions in Phase 1 villages and began in Phase 2 villages approximately three weeks later. The randomisation was stratified by the resulting 24 groups of 10 villages in each state. Assignment within each cluster of 20 villages was balanced such that each intervention was implemented in exactly five villages. Pratham designed and implemented three interventions in each location. Each intervention was implemented for approximately one year. A census was conducted in each village of the sample to determine a list of target mothers for recruitment (with children aged 5–8). These mothers were targeted to maximise the precision of estimated effects on children just beginning formal education, as it was hypothesised that the programmes would have the greatest effects on children who were just beginning to develop the most basic reading and math skills. Twenty-two mothers were randomly selected to be targeted. If there were fewer than 22 such mothers in the village, all mothers were targeted. On average, there were 18.5 mothers in each village in the study. The targeting strategy used in the evaluation may differ from that used in other adult literacy programmes. For example, the Saakshar Bharat programme typically recruits women from the vicinity of the teacher’s home who are interested in attending adult literacy classes. In our case, it was not possible to use a sample of this type, because it was difficult to gauge interest without holding classes to see which mothers would attend. When asked directly, almost all women said they were interested in the classes. So we decided to recruit a fixed set of mothers who would be interested. Because adult literacy programmes typically have low take-up, we decided to focus on mothers with young school-aged children. 4 Appendix A details the location selection procedure within the study blocks. 12 While our recruitment strategy was designed to be simple and replicable, it may result in different impacts from programmes using other recruitment strategies. For example, without explicit targeting, only the most motivated mothers may attend, which might have more impact on them or their children. Alternatively, a strategy that targeted different types of women could have different effects on mothers of young children if it changed the composition of the classes. Our classes typically consisted of both targeted and non-targeted women, but we cannot rule out the possibility that our targeting strategy created a particular class composition that in turn affected the results on targeted women. The ML intervention consisted of daily literacy classes held in the villages. In each location, a volunteer was recruited from the community to teach classes for two hours a day at the time and place that was most convenient to interested women. Volunteers used a version of Pratham’s Read India methodology. This approach, already shown to be effective in teaching children to read (Banerjee et al. 2010), was modified to suit the interests of adults. While ML classes were open to any who wished to attend, Pratham staff and volunteers were given a list of target mothers to recruit into the classes. The CHAMP intervention was designed to engage mothers with their children’s learning at home. Once a week, a Pratham staff member visited each target mother and gave her a worksheet to help her child complete. Mothers were also shown how to review their children’s school notebooks, discuss their children's learning with their school teacher and encourage their children to do schoolwork at home. The combined intervention included both the ML and CHAMP interventions. The MLCHAMP intervention was not integrated; both interventions were simply conducted in the same villages with the same target group of mothers. 3. Conceptual framework and theory of change This section presents a conceptual framework and theory of change for the ML and CHAMP interventions. Although the focus in this section is on how the programmes can influence child learning, we recognise that some of the framework's intermediate outcomes – in particular, mother learning and empowerment – could be important policy objectives in themselves. 3.1 Conceptual framework In theory, the amount that children learn at home is a function of the time they spend on educational activities and the productivity of that time spent. Similar factors may contribute to both time and productivity: child preferences, available educational inputs or assets, time parents spend monitoring educational activities and/or directly instructing, and the productivity of the time parents spend. In other words, children are likely to spend more time learning and will be more productive learners when their parents dedicate resources and productive time to their education. The amount of resources and time that parents spend on child learning, and the productivity of those inputs, can in turn be influenced by a number of factors. We identify three key factors relevant to our context, and motivate the discussion of these using relevant correlations in our baseline data. 13 First, parents’ own human capital, and their experience with – and awareness of – the process of learning, all influence the productivity of the time and inputs they provide (Rosenzweig and Wolpin 1994). Our baseline data show that more literate mothers are more likely to help their children with homework and have more educational materials (books, magazines and stationery) at home; their children are also more likely to be at higher learning levels. Similarly, the children of mothers who identify in-home responsibilities for their education have higher test scores. Second, parents’ expectations and aspirations can directly influence child motivation, parents’ own time allocation, and the amount of resources they dedicate to educational assets in the home.5 According to our baseline data, households who believe their child will complete at least 12th Grade are more likely to have educational materials at home, to help their children at home and to have children with better learning outcomes. Third, if mothers have a relative preference for educational outcomes, their own empowerment – defined broadly as control over decision making in the household – may serve as a key intermediate step in procuring educational assets or allocating their time accordingly (Duflo 2012). Our empowerment index, described in Section 5.5, is predictive of child outcomes, educational assets and participation. This conceptual framework can be summarised by the following figure. Figure 1 Home inputs that contribute to child learning 1 Although it is not shown in the figure, there may also be feedbacks across the various sources of home inputs – for example, human capital, experience, awareness and relative empowerment may all influence parental aspirations. In our baseline, literacy, education level and awareness of activities are all correlated with aspirations and our empowerment index. 5 Nguyen (2008) provides evidence that parental expectations of the returns of education influence child schooling outcomes. 14 3.2 Theory of change One driving assumption behind ML and CHAMP is that mothers in particular would like to help their children learn, but often lack the skills, awareness and/or experience to do anything at home, and therefore do not dedicate as much productive time or resources as they would like. Nearly every mother in our baseline reported that they believed the parents could do more (than sending their kids to school) to improve learning; 86 per cent said that parents have specific responsibilities toward their children’s education, beyond sending them to school. However, only one-third listed any in-home activities (time spent telling stories, helping with, checking or monitoring the completion of homework) that might help improve their learning; 30 per cent said they did not know what responsibilities parents might have. By design, ML was intended to directly influence mothers' human capital, or skills. Classes focused on basic literacy and numeracy. Improving these skills would increase the productivity of time and inputs that mothers provide to their children. Specifically, if they were more numerate or literate, mothers would better understand the concepts their children are learning and can therefore use that knowledge to teach children directly as a supplement to school work. These classes could also give mothers experience in the learning process as adults. While different from child learning – for example, it doesn’t involve the formal education system – mothers may still be able to translate their own experience of learning into an understanding of how their children learn. A better understanding of the importance of practicing concepts, for example, could induce mothers to motivate and monitor their children at home. CHAMP was intended to increase mothers’ awareness of the process of learning, time spent on learning and assets available in the home, but not human capital or preferences directly. Parents were given materials and guidance on how to interact with their children at home to foster their learning. In this case, the theory is that if mothers were given specific materials and instructed on activities known to promote learning, they would be able to mimic the environment and interactions that children of more educated parents have. Again, this increase in awareness is intended to increase the mothers' inputs and productivity of these inputs. CHAMP could also improve mothers' human capital: while not directly giving instruction on how to read, write or do math, their interactions – with Pratham staff and with their own children – may result in mothers learning new skills too. These interventions could influence people's preferences, aspirations and expectations. In ML, direct exposure to the process of learning could increase mothers' interest in child learning and their aspirations for their children. CHAMP could increase mothers' perception of the value of their children’s education through their involvement in child learning. 15 Either intervention could also promote a sense of empowerment. To the extent that ML and CHAMP influence mother learning, this could increase mothers’ ability to make decisions related to education or household decisions more broadly. Through direct exposure to a classroom environment, ML could also provide the skills and confidence that mothers need to make education decisions within the household. Although CHAMP provides exposure to the learning process through a slightly different mechanism, this could similarly empower mothers to make education decisions. In addition, by giving mothers new skills and exposing them to education, both interventions could increase mothers’ aspirations for their daughters, relative to their sons. Finally, both ML and CHAMP could directly affect child learning. If children attend the ML classes (along with, or in place of, their mothers) it could influence their own motivation, the time they spend on educational activities and the productivity of that time. If children are present when CHAMP materials and activities are being demonstrated, this could impact their productivity, time allocation and preferences, which is independent from the interaction with their mother. 4. Data collection Baseline data were collected from selected households at the onset of the interventions and endline data after approximately one year. Data collection consisted of standardised tests and household surveys. The standardised tests, designed to evaluate a basic set of Hindi and math skills, were developed by the ASER Centre, Pratham’s research arm, and were an expanded version of the ASER Centre’s standard assessment tool used each year in their Annual Status of Education Report.6 At the baseline, the tests were administered to all eligible mothers, target children, other children in the household in Grades 1–4 and children aged 4 and below who were going to be enrolled in school in the following year. The endline testing included all mothers and children tested at baseline, and the remaining children who were aged 3 or 4 at the baseline. These tests were scored on a 20-point scale for children in both the baseline and endline, a 24-point scale for mothers at baseline, and a 28-point scale for mothers at endline. The mothers’ test was the same as the child’s test, but included several additional questions that related to the material taught in the mother literacy classes. Minor additions and deletions were made in the testing instruments between baseline and endline. For the purposes of the analysis, test scores were normalised based on the control group means and sd in each round of testing. This was done separately for the mothers and the children. In addition to the primary standardised testing instruments, at the baseline other household members were given very short tests designed to quickly assess whether they could read simple sentences and do basic subtraction. 6 The ASER tool is used in the ASER Centre’s national assessments of child learning and is administered to approximately 450,000 children annually (ASER Centre 2013). 16 The household surveys were administered to eligible mothers. The baseline contained modules on basic household demographics, asset ownership, schooling status of children in the household, the mother's perceptions of education, and the mother’s time use. They were also asked about the time use of one child aged 5–8 in the household (in the cases where there was more than one such child, one was randomly selected). The endline survey repeated the measures of the baseline survey, with the exception of demographics, and included additional questions on empowerment. Table 1 contains descriptive statistics from the baseline tests and surveys and compares the means of the variables between each treatment group and the control group. Out of 60 comparisons performed, six are significant at the 10 per cent level; three at the 5 per cent level. No variable is jointly significantly different at the 10 per cent level between the three treatment groups and the control group (not shown). On the whole, this suggests that the randomisation was successful in creating comparable groups. Appendix Tables 2a and 2b detail the weighting procedure and the weighted test scores for each question in the test. The average baseline mother scores were 3.0/10 for literacy and 3.1/14 for numeracy. Mothers scored the highest on the most basic competencies on the test, such as picture recognition, letter recognition, writing one’s own name and number recognition. Child scores averaged 2.9/10 in literacy, and 2.9/10 in numeracy. As with the mothers, the children scored highest on the most basic competencies. Out of 8,857 mothers tested at baseline, 8,552 (97 per cent) were re-tested for the endline. Child tests are available for 14,575 out of 15,502 (94 per cent) of children tested at the baseline. 5. Results – mothers 5.1 Estimating equation Throughout the analysis we use the following estimating equation: (1) In this equation, is the outcome for individual i, in household h, in village v. ML, CHAMP, and MLCHAMP are dummies indicating the treatment status of the village. Y01 is the baseline value of the outcome of interest (when measured). G is a dummy for stratum, as described in Section 2 above. is the individual error term, clustered by village, the level of randomisation. 17 5.2 Programme take-up Take-up of the mother literacy classes is analysed in Table 2. Compared with the control group, approximately 32 per cent more mothers in the ML treatment and 37 per cent in the ML-CHAMP treatment reported ever having attended the classes.7 Children attended the classes as well; they were 21 per cent more likely to ever attend in the ML treatment, and 27 per cent more likely to ever attend in the ML-CHAMP treatment, compared with the control group. According to our focus group discussions and interviews, mothers reported selfmotivation as a primary reason for attending ML classes. The primary excuses for absence were lack of free time, a perception that there was little value – some said that it was too late for them to benefit from education, and Pratham should instead concentrate on teaching their children directly – and a lack of support from the rest of the family. By the same token, we found a similar number of examples where mothers saw value and had support from family members. The quality and innovation of volunteers appeared to be a critical factor. For example, one volunteer posted a sign outside with mothers’ names and attendance. Another would bang a drum in the village before every class. At the other end of the spectrum, when volunteers were less motivated, mothers in one-on-one interviews blamed the irregularity of classes as a reason for not attending. In Table 3 we quantitatively analyse the determinants of mother take-up of ML literacy classes in the ML and ML-CHAMP treatments. We regress class attendance on a set of variables including household characteristics, child schooling behaviour, mother's education, mother's experience with literacy classes, work behaviour, empowerment and participation in child learning. Column 1 includes a dummy for Bihar state, while Column 2 includes the full set of stratification unit dummies. Household composition could affect demands on a mother’s time (in the case of very young children), mother’s interest in improving child learning (for schoolaged children) and other household resources that could free up mother’s time (in the case of older household members). However, we have little evidence of a relationship between take-up and our three measures of household composition. Mothers of primary school-aged children are 1–2 per cent more likely to attend, although the coefficient is significant at the 10 per cent level in one of the two specifications. We also find no evidence for a relationship between a mother’s weekly housework or market work hours and take-up. We find weak evidence that mothers with children in school are slightly more likely to attend, but child test scores are not significantly related to take-up. Mothers are, however, 7 per cent less likely to attend when their children are in private school. In this case, mothers who are already sending their children to private school may not see the classes as important to support child learning. 7 It is important to note that only 7per cent of mothers in the control group attended classes. There are three potential explanations for this non-compliance: (1) Some control group mothers could have attended some classes in treatment villages. (2) A government programme – Saakshar Bharat – that was conducted in the spring of 2012 in 11 villages in Bihar. Research staff monitored this programme carefully. Where they were set up, classes were held for approximately one week, and were held in both treatment and control villages. (3) Some mothers may have misunderstood the survey question. 18 The relationship between mother's education and attendance is non-monotonic: mothers are more likely to attend when they have some education, but more years of education makes them less likely to attend. In addition, mothers scoring higher on the baseline test are significantly more likely to attend the classes. Taken together, these results suggest that mothers are more interested in attending when they have some education, but the classes are not attractive at higher levels of education. Because the test only covered the most basic competencies, this implies a positive relationship between test scores and attendance over the levels covered by the test, but attendance could be lower for women at higher-level competencies. A mother's experience with literacy classes in the past is a strong predictor of attendance in the Pratham classes: mothers who have attended in the past are 8 per cent more likely to attend during the intervention. Similarly, members of a self-help group are 9–10 per cent more likely to attend, implying that mothers with experience of meeting in women's groups are more comfortable attending classes. Finally, mothers in Bihar had 11 per cent higher take-up than those in Rajasthan. This mirrors Pratham and research staff observations that mothers in Bihar were on average more motivated and had more time to attend, and volunteers were more readily available to teach there. In Column 2 of Table 3, we replace the dummy for Bihar with the 48 stratification unit dummies. As indicated above, the stratification unit varies both by geographic area and implementation phase. In this case, differences in take-up could be driven by geography, Pratham staff and volunteer characteristics, and perhaps by slight changes in implementation across phases. The stratification unit dummies are highly jointly significant, implying that implementation or geographic differences did affect take-up. CHAMP was a door-to-door intervention where Pratham staff visited each mother in her household. Although complete data on CHAMP take-up is pending, administrative data from Rajasthan implies that 97 per cent of mothers were visited at least once, 12 times during the year, on average. In most villages 15 sessions were held, and the average attendance rate is 82 per cent across all targeted mothers. Interviews with parents suggest that the child's inherent interest and ability were key determinants for parental engagement. Some identified the child's lack of interest and ability as the reason for not investing in the process. Others were less involved because their child was able to complete the worksheets on their own. These parents often felt that their primary responsibility was to monitor that the worksheets were completed; not necessarily to explain the activities. 19 5.3 Test scores Turning to the results on mother learning, Table 4 presents the effect of the treatment groups on mothers’ normalised test scores. All three programmes had statistically significant impacts on literacy, numeracy and combined test scores. The ML programme improved mothers’ test scores by 0.09 sd in Hindi and 0.12 sd in numeracy and 0.11 sd overall. The last column in the table presents instrumental variables (IV) estimates of the effects of programme take-up on learning, instrumenting take-up with assignment to the ML treatment. To account for spillovers within households, take-up is defined as either the mother or the child attending a class at least once. Using this method, the effect of take-up is 0.33 sd overall.8 We note that in our context, a number of control mothers did attend classes, thereby not complying with the original treatment assignment. Therefore, the IV estimate represents the effect of class attendance on test scores only for the mothers who were induced to take up the classes because they were in an ML village rather than a control village. This estimate is valid under monotonicity: that is, we must assume that mothers who would have attended in the control group were not less likely to attend by virtue of being assigned to the treatment group. This assumption would be violated if, for example, mothers preferred to attend classes in villages other than their own. However, because social ties tend to be stronger within villages, rather than across villages, monotonicity is likely satisfied in this case. The CHAMP programme improved mothers’ test scores as well. Test scores improved by 0.04 sd in Hindi, 0.07 sd in numeracy, and 0.06 sd overall. The effect of the combined intervention on total test scores was 0.15 sd. While this is slightly lower than the sum of the effects of the ML and CHAMP interventions, we cannot reject that the ML-CHAMP effect equals the sum of the effects of the two individual interventions (p-value = 0.335). Appendix Tables 3 and 4 display the treatment effects on each question of the test for language and math, respectively. For comparability across questions, the maximum score for each question is re-scaled to one. On the language portion of the test, ML and ML-CHAMP interventions had the largest effects on more basic skills such as reading letters, reading simple words, and writing the mother’s name. For example, mothers in the ML group were 3.5 per cent more likely to be able to read letters, while mothers in the ML-CHAMP group were 4.7 per cent more likely to read letters, compared with the endline control group mean of 17.3 per cent. The point estimates for CHAMP, on the other hand, were modest and positive (about 0.5–1.5 per cent) on most questions, although most of the estimated effects are not statistically significant. 8 Note that the exclusion restriction in the IV estimation assumes that the mother literacy classes influenced learning only through attendance in the classes. This assumption would be violated, if, for example, mother learning was influenced by the attendance of other members in the community. 20 On the math portion of the test, all three interventions had the strongest effects on the number recognition questions, the most basic skills tested. For example, the mothers’ ability to identify digits 1–9 was 7, 3 and 11 per cent higher in ML, CHAMP and ML-CHAMP respectively, compared to the control group mean of 47 per cent. Interestingly, all three interventions also had statistically significant effects on the mothers' ability to complete the division word problem in addition to the more basic math skills. This suggests that either classes attracted some relatively more numerate mothers, or that the programmes were particularly effective in mental math – in other words, solving word problems – in addition to the more basic skills. The fact that the interventions affected more basic skills is supported by our qualitative observations. In focus groups, mothers in ML classes identified basic skills such as the ability to write their own name and those of family members, and the ability to dial numbers on a cell phone as the primary skills they expected to learn. In CHAMP, qualitative observations suggested that were more actively engaged in understanding basic activities on the worksheets, rather than simply monitoring their children’s work. This could explain why CHAMP improved mothers’ basic number recognition. 5.4 Intermediate outcomes The programmes could have affected mother learning through a variety of channels. In addition to the more direct effects that mother literacy and child participation could have had on mother learning, there are a number of indirect channels. Section 6 analyses changes in the home environment, including having access to education assets at home – for example, books and slates – and the mothers’ involvement in child learning. We find that the programmes did increase both assets and mothers’ involvement in child learning, both of which could have had feedbacks to mother learning. We also find evidence that the programmes induced others in the households to help the mothers learn. Table 5 analyses whether the mother reported learning various skills from family members. We find that significantly more mothers in the ML and ML-CHAMP treatments reported learning any of the skills from family members, from 21 per cent in the control group to 26 per cent in ML and MLCHAMP. For the CHAMP interventions, we find smaller coefficients, and the coefficients are significant only for learning about counting and counting change. 5.5 Empowerment and time use This subsection examines the effects of the programmes on women’s empowerment and time use. These indicators are both potential channels through which the programme could have affected mother and child learning, as well as important outcomes in and of themselves. 21 We first turn to the programmes’ effects on women’s empowerment. We include 19 variables from the survey instrument reflecting a number of underlying aspects of empowerment. First, we include a set of variables reflecting the mother’s ability to make decisions and carry out tasks on her own. Second, we include a set of variables indicating whether the mother is involved in certain household decisions. Third, we include a set of variables reflecting beliefs about own and daughters’ education. Finally, we include a measure of happiness.9 Using these variables, we construct an index of empowerment using the methodology from Kling et al. (2007). Each variable is normalised by subtracting the control group mean and dividing the result by the control group sd. The resulting normalised variables are then averaged to create the index. We construct separate indices for both the baseline and endline. The baseline index contains fewer elements than the endline index, as additional empowerment questions were added to the endline questionnaire. Table 6 presents the programme effects on the index and its components. Using our index, we find that each of the three treatments has positive and statistically significant impacts on empowerment. The estimated effects of the ML and CHAMP programmes were both 0.04 sd, both significant at the 5 per cent level. Turning to the components of the index, both the ML and CHAMP interventions had significant impacts on whether the mother counts change, beliefs about adult daughter’s choices and beliefs that the mother should be responsible for her children’s education. The ML intervention also had impacts on several variables more directly related to mother literacy and numeracy (mother signing her name, considering herself literate, the value of goods she could buy on her own) and beliefs about a wife’s level of education relative to her husband's. The CHAMP intervention had a significant impact on leaving the village without adult accompaniment and a small negative impact on self-help group membership. We next turn to the effects of the programmes on women’s time use. Andrabi et al. (2012) find that women with more education spend more time with their children. In Table 7, we examine whether the ML, CHAMP and combined programmes affect mothers’ time use in this manner. Across all measures, we see little evidence that the programmes impacted time use. The combined interventions increased weekly hours spent on paid work by one hour per week (significant at the 10 per cent level), and livestock work by 0.5 hours per week (significant at the 5 per cent level). These effects could be a result of increased productivity brought about by the interventions. Because of the number of comparisons made in this table, these results could have arisen due to pure chance.10 Indeed, we fail to reject that all 33 coefficients in the table are zero (p-value: 0.38). 9 We note that involvement in work activities could also arise as a result of empowerment, or could provide a greater sense of empowerment (Duflo 2012). We do not include work in our index, as our available measures do not include detail on the type of work or on women’s own earnings. 10 These effects could also be the result of increased productivity working, either through an increase in literacy or numeracy, or through increased interactions with volunteers or other women. In addition, increases in empowerment could have led women to work more. However, this was not an expected result, and we have somewhat limited data on labour supply to explore the mechanisms. 22 6. Results – children 6.1 Test scores Table 8 presents the effects of the treatment groups on child test scores. All children tested at the endline are included in this table, including the younger children not tested at baseline. The regressions include a dummy variable for missing values of the baseline test scores. All three interventions had significant impacts on numeracy skills: the effect size is 0.04 sd for ML, 0.05 sd for CHAMP and 0.07 sd for ML-CHAMP. The effects of ML and CHAMP on literacy and cumulative scores are not statistically different from zero. However, ML-CHAMP had statistically significant effects on literacy (0.05 sd) and cumulative scores (0.06 sd). The last column of Table 8 follows the mother test score results in Table 4 by presenting IV estimates of the effect on child learning of a mother or child attending the ML classes. The IV estimate of the effect of attendance – 0.11 sd for numeracy –is significant at the 5 per cent level. Because the reduced-form estimates are not significant for literacy or cumulative test scores, it is not surprising that the IV estimates are not significant at conventional levels. Appendix Tables 5 and 6 disaggregate the test score effects by individual question. As with the mothers’ results, we re-scale the questions so that the maximum possible score for each is 1. The results for language are displayed in Appendix Table 5. The ML intervention did not have a statistically significant impact on any competency, and the estimated magnitudes are very small and inconsistently signed. For the CHAMP intervention, the magnitudes of the coefficients on each question are positive, but none reaches statistical significance. The ML-CHAMP intervention had positive and statistically significant impacts on the child’s ability to read letters, complex words and paragraphs. Appendix Table 6 displays question-wise results for numeracy. Across all three interventions, the largest effects are concentrated in the more basic number recognition questions. For example, child scores were 2.3, 4.0 and 3.9 per cent higher on the question that asked the child to identify the digits 1 to 9, compared with the endline control group mean of 56 per cent. Discussions with mothers, volunteers and Pratham staff found that there was more demand from mothers to learn math. This could have driven the stronger child results for math, in that their mother's enthusiasm for learning math could have led them to engage more with their children on the subject. 23 6.2 Intermediate outcomes This section analyses impacts of the treatment groups on intermediate outcomes. We start by discussing outcomes that relate to learning outside of the home. Table 9 presents the programmes' impacts on school participation. We find no evidence that the individual programmes affected current enrollment, regular attendance or recent absences, although we find a small positive impact on school attendance of the combined programme. Finally, we find a statistically significant increase in monthly tuition expenditures for the ML group, but the effects are smaller and statistically insignificant in the ML and ML-CHAMP groups. On balance, this table shows limited, if any, impact of the interventions on schooling outcomes. We note, however, that enrollment is relatively high among our study population. In interviews, many parents claimed that the most they could do within the formal schooling system is send their children to school and cited poor quality and poor access as barriers they could not overcome.11 Table 10 presents a set of indicators of mothers' participation in their children’s schooling. We constructed an index of mother involvement using nine measures – including indicators of school visits, helping with homework and talking to the child and others about the child’s studies – and following the procedure outlined above. These survey questions were asked about the randomly selected child; hence the sample size is equal to the sample of mothers. We find positive and statistically significant impacts of all three programmes on the index of indicators. The magnitudes are approximately 0.04 for ML, 0.07 for CHAMP and 0.05 for ML-CHAMP. While both ML and CHAMP had statistically significant impacts on the mother looking at the child’s notebook, talking to the child about their studies and talking to others about the child’s studies, CHAMP had impacts on the mother knowing whether the child received homework and on helping her child with homework. This is encouraging, given the qualitative observations from early in the intervention which suggested that mothers in CHAMP villages were learning how to monitor CHAMP worksheets, but were not transferring these skills over to school homework. 11 While one father expressed dissatisfaction with the schooling system, claiming that ‘like any government employee, the teachers don’t do any work...the best solution is to send the children to private school’, he lamented that ‘there is no private school in the vicinity, so they are still studying [at the government school]’. One mother explained, ‘if the teachers taught for a couple of hours in a day, our children would at least learn something’, and later expressed that ‘all [Pratham] could do [to help] is spend money and send [her child] to a better school’. Another mother complained that ‘she and her husband want to shift [their children] into a private school. [But] there is no private school nearby so they are contemplating their options.’ 24 Table 11 presents the estimated impacts of the programme on the child’s weekly time use. Overall, there were very few impacts. The combined ML-CHAMP intervention increased time spent on homework by 0.3 hours per week, statistically significant at the 5 per cent level. While the effect of individual interventions fail to reach statistical significance at the 5 per cent level, the magnitude of the CHAMP effect is 0.2 hours per week, significant at the 10 per cent level, suggesting that the ML-CHAMP effect could be driven primarily by CHAMP. The ML and ML-CHAMP interventions also have significant impacts on time spent in household business. An increase in labour supply among children is similar to the labour supply increases found in the mothers' time use analysis. In the case of children, increases in labour supply could be due to complementarities with the mother’s labour supply or because the interventions made children more productive in household businesses. As with the mothers’ labour supply results, however, more work is needed to understand the mechanisms behind these impacts. Table 12 presents the treatment effects on the presence of education assets in the home, including pencils, school books, other books, newspapers/magazines and slates. For the ML intervention, the only statistically significant effect is on the presence of school books, with an estimated magnitude of 0.018. The CHAMP intervention, on the other hand, had a statistically significant effect on the presence of pencils, school books, other books and newspapers/magazines. The combined intervention increased the presence of school books, other books and slates. (Note that pencils are present in 95 per cent of comparison group households, so minimal movement on this indicator is unsurprising.) Finally, in Table 13 we turn to a set of indicators that reflect a mother’s aspirations for her child and her perceptions of her child’s reading ability. We do not find statistically significant impacts for any of the interventions on the highest grade that the mother aspires her child to pass. In interviews, mothers often reported inherent child ability and future achievement as a given, and not something they had much control over. We do find that the CHAMP and combined interventions increased the mother’s perceptions of her child’s reading and math ability. When compared to the child’s actual ability, however, the CHAMP and combined programmes caused mothers to be overly optimistic: the absolute difference between the mother’s perception and measured child ability increased for the CHAMP and combined interventions. 7. Impact heterogeneity We use the following estimating equation to examine heterogeneity in treatment effects: In this equation, Vari is the interacted variable and the remainder of the variables are defined as in equation (1). 25 7.1 Mothers Table 14 examines heterogeneity in treatment effects of the interventions on mother test scores. We focus on heterogeneity by the state where the intervention took place, the mother’s baseline score, mother’s age and mother’s education level. The first three columns of Table 14 examine heterogeneity by state. There is evidence that the ML and ML-CHAMP interventions were more effective in Bihar. For example, the effects of the ML intervention were 0.02 sd higher for language, 0.13 sd higher for math and 0.09 sd higher for composite scores in Bihar compared with Rajasthan. The latter two results are significant at the 1 per cent level. Similarly, the ML-CHAMP intervention increased language, math and combined scores by 0.08, 0.12 and 0.11 sd more in Bihar than in Rajasthan. There are no significant differences in treatment effects across states for the CHAMP intervention. The greater effectiveness of the ML and ML-CHAMP interventions in Bihar is consistent with the fact that mothers were 11 per cent more likely to attend the classes in Bihar (see Section 5.2). Columns 4 through 7 examine heterogeneity by the mother’s baseline test score. Although mothers did take up the mother literacy classes more often at higher test scores, there is little evidence that ML and ML-CHAMP were more effective for mothers with higher or lower test scores. ML-CHAMP increased composite scores significantly more for mothers who scored lower at the baseline, but the interaction effects on the disaggregated language and math scores are insignificant and inconsistently signed. On the other hand, the CHAMP intervention was significantly more effective for mothers with higher initial test scores. This result implies that a basic level of literacy and numeracy was helpful for mothers to improve their own learning in CHAMP. The remaining columns of Table 14 examine heterogeneity by mother’s age and education level. There are few significant interaction effects and the magnitudes are small. There is evidence that ML-CHAMP was more effective for less educated mothers in increasing numeracy and composite scores, although these interactions are not reflected in either the ML or CHAMP interventions. 7.2 Children Tables 15a and 15b examine heterogeneity in treatment effects on child test scores. Table 15a uses the same set of variables that were used in the analysis for mothers. Overall, there is little evidence of heterogeneity by any of these variables. Unlike the effects found for mothers, there is no significant heterogeneity by state, although the point estimates do suggest that ML and ML-CHAMP were more effective in Bihar. The ML intervention had significantly stronger effects on numeracy and composite scores among children with older mothers. 26 Table 15b examines heterogeneity by child age, child baseline score and gender. Again, there are few large or statistically significant effects. There is some evidence that ML-CHAMP was more effective in improving literacy and composite scores for older children. The only other statistically significant interaction in the table suggests that lower-scoring children performed better in numeracy in the MLCHAMP intervention, although this heterogeneity is not reflected in language or composite scores. 8. Internal and external validity 8.1 Internal validity As a randomised evaluation, we are fairly confident of this study's internal validity. In theory, the intention to treat estimates could be biased by (1) initial imbalance of the treatment and control groups, (2) attrition and (3) spillovers or crossovers. As described in Section 4, the treatment and control groups were balanced from the onset. Attrition was also very low. Out of 8,857 mothers tested at baseline, 8,552 (97 per cent) were re-tested for the endline. Child tests are available for 14,575 out of 15,502 (94 per cent) of children tested at the baseline. Last, spillovers and crossovers were largely contained by the unit of randomisation. If spillovers and crossovers were indeed a problem, our intention to treat estimate would give us the lower-bound of the (internally valid) impact of the programme. 8.2 External validity The success of a programme depends on the interaction of (a) the concept, (b) the implementation and (c) the context. Its external validity depends on how these same factors interact in new areas where the programme is implemented. The concepts being tested here are specific adult and family literacy programme models. One question is whether this model is gold plated, or alternatively, replicable and scalable. The model relies on four main factors: (1) the pedagogy and material, (2) the recruitment, training, retention and motivation of local Pratham staff (for village mobilisation, CHAMP implementation, and monitoring), (3) the recruitment, training, retention and motivation of volunteers to run ML classes and (4) the recruitment, retention and motivation of mothers. The pedagogy and material are replicable. In fact, they would likely be improved in future iterations of the programme. During the first few months of classes, material was used that the programme team later deemed to be ineffective. In theory, if replicated, these mistakes could be avoided. Some of the material designed for this programme was given to Bihar for its state-wide adult literacy campaign. The recruiting methods used were not unlike the methods used by Pratham in its other interventions. Pre-intervention mobilisation lasted, on average, two days per treatment village for both Pratham staff and mothers. However, Pratham is particularly skilled at mobilising communities and, in particular, volunteers. It is unlikely that a government-run programme could extract similar levels of intrinsic motivation to work and volunteer. That said, modest monetary incentives could be used to generate sufficient labour supply. 27 One aspect to note is that the interventions targeted mothers of children aged 5–8, and our effects are estimated for this group. These mothers were identified through a village census conducted by the research team. The target criterion was chosen in order to maximise power to detect effects on this group of children, as it was hypothesised that these children would be most likely to experience the programmes' effects. In order to replicate the interventions with the same target group, an implementer would have to identify this group, either through a community census or anther means. This is unlikely to be cost-prohibitive, as children and their mothers could be identified through schools or door-to-door interactions within the community rather than through a formal census. If a broader set of women was targeted, our estimated effects would not necessarily apply to women of other age groups or those with other characteristics. However, our estimated effects would apply to targeted mothers of young schoolaged children, provided that the spillovers across targeted women are similar for the different targeting mechanisms. Retention and motivation may be the most difficult aspects to replicate – particularly of staff and volunteers. However, both motivation and retention were major challenges for this programme, and would likely be a challenge for others. Here, financial incentives could actually be more effective than the volunteer model. While altruistic motivation could wane, especially as the novelty wears off and challenges arise, financial incentives would be contingent on retention and could be made contingent on performance. Overall, each intervention spent US$17–40 per mother for an 11-month intervention. The cost is therefore unlikely to be prohibitive for scaling up. It would be difficult to suggest that the implementation of this programme was exceptional. The largest challenges to implementation involved the timely production of material, retention of volunteers and mothers, and determining the right period to hold classes. For example, harvest-time classes could only be held at night (if at all). And night-time classes required external sources of light, necessitating the purchase of solar lamps. We spent a significant amount of time learning this and then procuring the lights, which resulted in long periods with no teaching activities. This waste could be easily avoided in future replications or scale-ups. Although we argue that the programme is replicable, we note that differences in implementation by different implementing organisations may yield different results. In particular, because take-up and retention of mothers and volunteers was challenging, organisations that do not monitor programme implementation may face much lower take-up. Section 10 provides further discussion of this point, using the example of Saakshar Bharat, India’s state-run adult literacy programme. 28 The intervention locations were purposefully chosen (rather than being randomly chosen, and representative of a larger geographical area). As discussed in Section 2, Pratham selected Bihar and Rajasthan due to low literacy levels in both states and the differences between the states along other dimensions. The intervention districts within each state were selected because of existing Pratham programmes and infrastructure in those areas. Within the intervention districts, the blocks were selected because they did not have any pre-existing Pratham programmes. However, the specific blocks do not appear to be outliers relative to the districts or state as a whole. Households appear similar to state-wide characteristics. As noted in Section 2, running the interventions in multiple states in different areas of the country aids external validity of the evaluation. Although the interventions were identical in both states, they were implemented by different local teams and supervised by separate state-level Pratham leadership. And while learning levels in both states were similar, the differences in mothers' wealth and pre-existing activities presented distinct implementation challenges in each area. This suggests that the programme may be externally valid to the rest of those two states, other states with similar characteristics, and in particular, poor and illiterate districts of other states. We are less comfortable suggesting that these programmes' material and pedagogy are appropriate to poor and illiterate regions outside of India, and suggest that more replication studies in other regions would be useful. 8.3 Cost-effectiveness This section presents a summary of programme costs and a discussion of costeffectiveness. Our overall finding is that for child learning, the cost per unit improvement is less than that of other interventions deemed effective. We note, however, that we find effects along a number of other dimensions that cannot be explicitly included in the cost-effectiveness calculation. As with most cost-effectiveness comparisons across studies, we note that differences in target population, competencies tested, testing instruments, local prices and methods of calculating costs may limit comparability. Our costeffectiveness calculation follows the methodology in Kremer et al. (2013). We report costs in US dollars, converted using the exchange rate as of when the study began. Panel A of Table 16 provides a breakdown of costs for each programme. Because ML-CHAMP was simply a combination of ML and CHAMP, the costs of ML-CHAMP were approximately equal to the sum of the ML and CHAMP interventions. 29 We separate the total costs into three components. First, we use the actual cost incurred to pay for Pratham staff. In the ML programme, Pratham staff monitored the volunteers and in the CHAMP programme, they implemented the intervention. Second, we compute the opportunity cost of the ML volunteers’ time. This is based on the proportion of time spent on the classes (two hours per day) and the average wage in non-agricultural occupations computed from the India Labour Bureau.12 These costs were incurred only in the ML and ML-CHAMP interventions. Finally, we include costs of training, monitoring and materials. These were substantially higher in the ML intervention than in the CHAMP intervention because each mother attending ML classes received Pratham learning materials. Overall, as a result of volunteer opportunity costs and higher training, monitoring and materials costs, the ML intervention was substantially more costly per targeted mother (US$34 for ML versus US$17 for CHAMP). To arrive at the sd improvement per child per US$100 spent, we divide the total costs of each programme by the total number of beneficiaries and the estimated treatment effects reported in Table 8. The results are displayed in Panel B of Table 16. Following Kremer et al. (2013), we report cost-effectiveness only for interventions with statistically significant impacts on a particular test. CHAMP is most cost-effective for math and composite scores, while ML-CHAMP is most effective in improving literacy.13 To compare our results with a broad set of alternative interventions aimed at improving child learning, we compare our impact estimates for children with those reviewed in Kremer et al. (2013). Our estimates, ranging 0.04 to 0.07 sd, fall below the range of statistically significant estimates in the Kremer et al. (2013) study. This latter set of estimates ranges from 0.14 sd to 0.6 sd. Turning to cost-effectiveness, 14 out of the 15 studies in Kremer et al. (2013) that found statistically significant impacts are more cost-effective. Even though the interventions may be less cost-effective in improving child test scores than others, our study examines and finds impacts on a broad set of outcomes, including mother learning, women’s empowerment and the home learning environment. A full cost-effectiveness analysis would take into account the full set of impacts relative to costs. However, given the limited number of studies that examine these outcomes, in addition to the lack of a consistent framework to compare cost-effectiveness along these dimensions, we are unable to perform this broader comparison. Without an explicit comparison with alternative interventions, we tentatively conclude that the interventions studied here are cost-effective considering the broad set of outcomes affected and the relatively low cost of the interventions (between US$17 and US$50 per beneficiary household). Our study should serve as a starting point for future work to deepen the evidence base so that more explicit cost-effectiveness comparisons can be made in the future. 12 http://labourbureau.nic.in 13 Panel C of Table 16 shows the results of the equivalent cost-effectiveness calculation for mother learning. The three interventions produce very similar gains per US$100 spent. The ML intervention is most cost-effective for literacy, while the CHAMP intervention is most costeffective for math and composite scores. As noted in the introduction, our study is the first to provide a rigorous evaluation of the effects of adult literacy or home input interventions on adult learning outcomes. As such, we are unable to compare our effects on mothers with those found in other studies. 30 9. Policy recommendations The key policy implication of this study is that the ML, CHAMP and combined MLCHAMP programmes can be an effective tool for policymakers who are interested in improving child learning, mother learning, mother empowerment and the home learning environment. As discussed above, there may be more cost-effective interventions for improving child learning alone, but we are able to provide rigorous evidence of effectiveness over a much broader set of outcomes than those studied in the literature on child learning interventions. We emphasise, however, that replication is crucial for understanding whether these results apply to other contexts. As discussed in Section 8.2, while performing the study in two different states using different Pratham implementation teams aids external validity, more work is needed to understand whether the effects found here apply to different populations and different implementing organisations. In addition to the standard caveats in extrapolating our results to other contexts, we note that these results are short-term and were taken at the end of one year of implementation. Through future research, we hope to study longer-run impacts. Many learning interventions are subject to fade-out of effects over time (see, for example, Banerjee et al. 2007). In many programmes, however, the interventions do not produce long-lasting institutional improvements in either the school or home environment. If the programmes studied here have indeed caused a fundamental shift in the home learning environment, learning effects could persist as a result. Our experience has shown that take-up, attendance and general implementation are critical challenges in these types of programmes. Saakshar Bharat, India’s state-run adult literacy programme, follows a similar model to ML. One might therefore expect that a properly implemented Saakshar Bharat programme in the populations studied may have similar effects to those we find here. However, our field observations of Saakshar Bharat indicate that the programme is not implemented very thoroughly. Even though Saakshar Bharat was scheduled to run throughout our intervention areas at the same time as our programmes, our field teams noted little to no Saakshar Bharat activity in these areas once teachers had been recruited. Thus, while our study shows that these programmes can be effective when implemented properly, we also note that the planned implementation process is not always followed. 31 10. Conclusion Adult literacy and participation programmes are increasing in popularity, frequency, funding and influence – particularly in developing countries. Proponents and policymakers draw an explicit link between parents' education and child welfare outcomes when advocating for such programmes. The underpinning theory starts with the observation that parent education levels are strongly correlated with child outcomes and draws on further evidence that the relationship is causal (rather than due to other factors such as inherent ability or cultural preferences, which could lead to both outcomes independently). Educating parents in adulthood, the theory goes, will shift preferences toward demanding: (1) more quantity, and higher-quality, education, (2) more household resources toward more educational assets at home, (3) more time spent educating their children at home and (4) increased productivity of that time. However, there is very little rigorous evidence on whether these programmes are actually effective in the developing country context. We show that adult literacy and participation programmes targeting mothers in rural India were effective at educating parents by improving mothers’ basic literacy and numeracy skills. These programmes also had an impact on measures of women’s empowerment, educational assets in the home and mothers' participation in child learning. Lastly, they improved learning levels of younger school-aged children. Literacy classes were more effective at educating the mothers than the participation programme, while the participation programme was most effective at improving child learning outcomes. The results on learning (for mothers and children) were highest when the two interventions were combined, suggesting that the two interventions are at least additive, and not substitutes. We find that the programmes influenced a number of intermediate outcomes that could in turn have affected child learning. However, we cannot isolate the most important of these factors in the effectiveness of the programmes. Understanding the importance of each mechanism is a key area for future research. Nonetheless, our evaluation shows that literacy and participation programmes can impact both mother and child learning. This is encouraging evidence for policymakers who are looking to improve adult and child learning, as well as the home education environment. 32 11. Tables Table 1 Randomisation check Mean Relative to control Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) Assets First Principal Component of Durables Ownership -0.0328 0.00923 0.0924 0.0282 8888 [2.261] [0.0866] [0.0952] [0.0906] Main Income Source of Household Farming 0.431 0.0251 0.00974 0.0414* 8819 [0.495] [0.0230] [0.0250] [0.0226] Wages 0.431 [0.495] -0.0121 [0.0215] -0.0149 [0.0226] -0.0370* [0.0216] 8819 Other Number of Household Members 0.447 [0.497] -0.0129 [0.0133] 0.00521 [0.0143] -0.00445 [0.0131] 8819 Target-Aged Children (5 yrs to 8 yrs) 1.453 [0.612] -0.0351* [0.0184] -0.0293 [0.0199] -0.0158 [0.0179] 8888 Other Primary-Aged Children (4 yrs to 9 yrs) 1 [0.960] 0.0475 [0.0298] 0.0217 [0.0307] 0.0467* [0.0275] 8888 Younger Children (Less than 4 yrs) 0.942 [0.909] -0.0161 [0.0299] 0.0115 [0.0335] 0.0639** [0.0303] 8888 Older Children (More than 9 yrs) Mothers’ Test Scores 3.269 [1.751] 0.00905 [0.0632] 0.114* [0.0687] 0.0932 [0.0634] 8888 Mother Literacy 2.993 [2.474] 0.0442 [0.110] 0.134 [0.131] 0.0548 [0.115] 8857 Mother – Numeracy 3.022 [3.376] 0.0885 [0.153] 0.147 [0.181] 0.0822 [0.160] 8857 Mother - Composite Children's Test Scores 6.015 [5.616] 0.133 [0.259] 0.281 [0.307] 0.137 [0.271] 8857 Children - Literacy 2.803 [2.370] 0.0603 [0.0883] 0.0730 [0.0908] 0.0824 [0.0876] 15502 Children- Numeracy 2.770 [3.032] 0.114 [0.111] 0.131 [0.117] 0.0857 [0.109] 15502 Children- Composite Other Members' Reading/Math 5.573 [5.233] 0.175 [0.195] 0.204 [0.204] 0.168 [0.192] 15502 Other Members- Can Read? 0.380 [0.485] -0.00151 [0.0162] 0.0274 [0.0178] 0.0163 [0.0175] 13891 Other Members- Can Do Math? Parent Education 0.249 [0.433] 0.00460 [0.0146] 0.0249 [0.0170] 0.0233 [0.0147] 13891 Mother’s Education Level 0.764 [2.282] 0.0475 [0.102] 0.152 [0.118] 0.0694 [0.103] 8864 Father’s Education Level 3.876 [4.438] -0.150 [0.203] 0.133 [0.226] 0.234 [0.213] 8181 Mother Has Past Experience with Literacy Classes Child Gender 0.117 [0.321] -0.00839 [0.0123] -0.00825 [0.0130] -0.0209* [0.0124] 8635 Child Is Male 0.521 [0.500] -0.0133 [0.0108] -0.00804 [0.0110] -0.0214** [0.0106] 15500 Note: Columns 2, 3 and 4 display the differences in means between each treatment group and the control group. Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 33 Table 2 Take-up of ML classes Mean OLS: Impact of treatment in endline Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) Knew about ML classes 0.218 [0.413] 0.402*** [0.0226] -0.00558 [0.0199] 0.451*** [0.0217] 8581 Mother attended ML classes 0.0710 [0.257] 0.321*** [0.0184] -0.00101 [0.0128] 0.368*** [0.0177] 8581 Child attended with mother 0.0252 [0.157] 0.161*** [0.0126] 0.0000870 [0.00727] 0.218*** [0.0133] 8581 Child attended alone 0.0380 [0.191] 0.140*** [0.0115] -0.00884 [0.00728] 0.179*** [0.0132] 8511 Notes: Columns 2, 3 and 4 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies. Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 34 Table 3 Determinants of mother take-up Dependent Variable: Mother Attended (1) (2) 1st Principal Component of Durables -0.00227 -0.000612 [0.00644] [0.00526] # Children 0-4 -0.000514 -0.000447 [0.0101] [0.00924] # Children 5-14 0.0165* 0.0110 [0.00840] [0.00694] # Adults 15+ -0.00707 -0.00195 [0.00597] [0.00491] Total hours worked per week 0.000456 0.000389 [0.000398] [0.000370] Selected child age 0.00486 0.0115* [0.00946] [0.00664] Selected child is a girl -0.00143 -0.0146 [0.0177] [0.0153] Selected child in school 0.0333 0.0473** [0.0271] [0.0200] Selected child in private school -0.0808** -0.0878*** [0.0364] [0.0332] Selected child’s test score 0.00121 -0.00725 [0.0113] [0.0101] Father’s education level -0.00585** -0.00331 [0.00234] [0.00210] Mother’s total test score 0.282** 0.311*** [0.120] [0.109] Mother’s education > 0 -0.0304*** -0.0289*** [0.00610] [0.00562] Mother’s education: years 0.0587*** 0.0563*** [0.0145] [0.0126] Mother’s age -0.00210 -0.00270** [0.00138] [0.00120] Has mother attended adult- 0.0832*** 0.0777*** literacy classes before [0.0295] [0.0250] Self Help Group Member 0.0967*** 0.0910*** [0.0268] [0.0239] Baseline Empowerment Index 0.0594** 0.0615*** [0.0255] [0.0230] Baseline Mother- 0.0255* 0.0197 Child Participation Index [0.0151] [0.0129] State = Bihar 0.0996*** [0.0316] Mean of Dep. Var. 0.425 R-Squared 0.0600 0.0869 P-value: test that all stratification unit dummies N =0 0.000 Notes: Columns 1 and 2 display estimated coefficients of a regression of mother and child attendance respectively on the determinants in each row. * denotes significance at 0.10; ** at 0.05; *** at 0.01 31 Table 4 Mother learning Baseline mean Endline mean OLS: Impact of treatment in endline First stage IV All Obs Control ML CHAMP ML-CHAMP N P-value: additive effects Attend literacy class Impact of literacy class (1) (2) (3) (4) (5) (6) (7) (8) (9) Literacy 0.0430 0.115 0.0913*** 0.0400** 0.126*** 8552 0.848 0.341*** 0.261*** [1.055] [1.329] [0.0185] [0.0193] [0.0188] [0.0189] [0.0529] Numeracy 0.0616 -0.0158 0.120*** 0.0693*** 0.159*** 8552 0.226 0.353*** [1.065] [1.017] [0.0167] [0.0158] [0.0173] [0.0493] Total 0.0560 0.0414 0.111*** 0.0587*** 0.150*** 8552 0.385 0.325*** [1.066] [1.153] [0.0151] [0.0142] [0.0158] [0.0439] Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values. Column 7 displays the p-value of the test that the coefficients ML+CHAMP=ML-CHAMP. Column 8 displays the impact of assignment to the mother literacy treatment group on literacy class attendance by mother or child. Column 9 displays the impact of literacy class attendance on the dependent variables, using assignment to the ML treatment group as an instrument for attendance. Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 32 Table 5 Family assistance in mother learning Endline mean OLS: Impact of treatment in endline Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) Family member taught mother: ANY 0.206 [0.404] 0.0507*** [0.0146] 0.00683 [0.0154] 0.0560*** [0.0146] 8581 Family member taught mother: to write her name 0.173 [0.378] 0.0414*** [0.0131] -0.00953 [0.0135] 0.0502*** [0.0130] 8581 Family member taught mother: counting 0.0818 [0.274] 0.0509*** [0.0106] 0.0200* [0.0107] 0.0577*** [0.0105] 8581 Family member taught mother: household accounts 0.0537 [0.226] 0.0230*** [0.00825] 0.0118 [0.00826] 0.0285*** [0.00828] 8581 Family member taught mother: counting change 0.0514 [0.221] 0.0314*** [0.00820] 0.0194** [0.00860] 0.0295*** [0.00956] 8581 Note: Columns 2, 3 and 4 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies. Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 33 Table 6 Empowerment Baseline mean Endline mean OLS: Impact of treatment in endline All obs Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) (6) Empowerment Index -0.00625 [0.391] 0.000225 [0.378] 0.0409*** [0.0143] 0.0360** [0.0147] 0.0695*** [0.0148] 8539 Times left village in the past month 1.405 [2.244] 1.146 [1.672] 0.0338 [0.0607] 0.0741 [0.0627] 0.0977 [0.0661] 8581 Left without adult accompaniment (% of mothers) 0.127 [0.333] 0.114 [0.318] 0.00319 [0.00998] 0.0286*** [0.0103] 0.00449 [0.0103] 8581 Left village without permission (% of mothers) 0.0258 [0.159] 0.0168 [0.129] -0.00426 [0.00366] 0.000597 [0.00390] -0.00313 [0.00372] 8581 Signed name on official documents 0.538 [0.499] 0.562 [0.496] 0.0630*** [0.0140] 0.0127 [0.0134] 0.0829*** [0.0138] 8581 Counts change 0.876 [0.330] 0.869 [0.337] 0.0250** [0.0117] 0.0227* * [0.0114] 0.0421*** [0.0112] 8581 Caught mistakes counting change 0.310 [0.463] 0.318 [0.466] 0.0151 [0.0172] -0.00742 [0.0169] 0.0269 [0.0174] 8581 Considers self-literate 0.235 [0.424] 0.0479*** [0.0164] 0.0165 [0.0176] 0.0730*** [0.0170] 8581 Value of goods can buy alone 2442.0 [2259.3] 130.4 [83.58] 80.68 [91.02] 165.3* [91.72] 8581 Does not believe husband should be more educated 0.380 [0.485] 0.350 [0.477] 0.0441*** [0.0163] 0.0169 [0.0174] 0.0567*** [0.0170] 8581 Does not believe daughter should be at home or married when 18 0.0539 [0.226] 0.0439 [0.205] -0.00217 [0.00631] 0.00592 [0.00649] 0.00308 [0.00648] 8581 Believes daughter should be doing further studies 0.161 0.383 0.0478*** 0.0701*** 0.0954*** 8581 / what she wants / paid work outside home [0.368] [0.486] [0.0169] [0.0183] [0.0186] Would have wanted to study up to: grade level 5.620 [4.434] -0.432*** [0.161] 0.183 [0.181] -0.261 [0.159] 8581 Member of self help group 0.277 [0.447] 0.330 [0.470] 0.00665 [0.0171] -0.0303* [0.0169] -0.00403 [0.0160] 8581 Happiness 3.101 [1.439] 0.0556 [0.0501] 0.0784 [0.0511] 0.0558 [0.0470] 8581 Involved in purchasing utensils, cot or cycle 0.586 [0.493] 0.0271 [0.0181] 0.0118 [0.0180] 0.0247 [0.0194] 8581 Involved in purchasing educational materials 0.479 [0.500] 0.0209 [0.0183] 0.0188 [0.0179] 0.0383** [0.0183] 8581 Involved in deciding girl or boy enrollment 0.519 [0.500] 0.0114 [0.0173] 0.00638 [0.0173] 0.0156 [0.0175] 8581 Involved in deciding girl or boy school type 0.522 [0.500] 0.0115 [0.0170] 0.00966 [0.0174] 0.0287* [0.0171] 8539 Mother/ both should be responsible for child's education 0.717 [0.451] 0.0409*** [0.0138] 0.0322** [0.0138] 0.0424*** [0.0136] 8581 Notes: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values (where available). The empowerment index is an average of z-scores of the other variables in the table, using the control group means and sd. The baseline empowerment index only includes indicators for which data were collected. Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 34 Table 7 Mother time Use baseline mean Endline mean OLS: Impact of treatment in endline All obs Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) (6) Help with homework (weekly hours) 1.686 [2.903] 2.313 [2.704] 0.114 [0.0919] 0.126 [0.0973] 0.0602 [0.0918] 8519 Read (weekly hours) 0.201 [1.056] 0.324 [1.332] -0.0116 [0.0397] -0.0217 [0.0365] 0.00582 [0.0422] 8399 Play with child (weekly hours) 0.255 [1.309] 1.322 [3.172] 0.0720 [0.110] 0.0544 [0.116] -0.0272 [0.108] 8472 Share stories (weekly hours) 0.383 [1.201] 0.515 [1.401] 0.0196 [0.0464] -0.00556 [0.0438] 0.0358 [0.0507] 8514 Paid work (weekly hours) 26.81 [18.53] 31.27 [20.93] 1.022* [0.604] 0.487 [0.587] 0.975 [0.610] 8547 Livestock work (weekly hours) 9.242 [7.020] 9.528 [6.745] 0.171 [0.246] -0.253 [0.231] 0.505** [0.241] 8573 Collect animal feed (weekly hours) 6.828 [6.601] 0.209 [0.261] -0.161 [0.266] 0.178 [0.274] 8577 Collect wood (weekly hours) 3.302 [4.962] 0.0804 [0.179] -0.125 [0.193] 0.00144 [0.189] 8570 Housework (weekly hours) 22.20 [8.844] 18.86 [7.918] 0.365 [0.304] 0.252 [0.301] 0.316 [0.286] 8581 Buy supplies (weekly hours) 4.832 [6.188] 1.231 [2.656] -0.0582 [0.0901] -0.0654 [0.0905] -0.0182 [0.0877] 8567 Look after children (weekly hours) 5.751 [4.035] 4.640 [3.695] -0.209** [0.106] -0.143 [0.109] 0.0737 [0.104] 8581 Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values (where possible). Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 35 Table 8 Child learning Baseline mean Endline mean OLS: Impact of treatment in endline First stage IV All obs Control ML CHAMP MLCHAMP N P-value: MLCHAMP = ML + CHAMP Attend literacy class Impact of literacy class (1) (2) (3) (4) (5) (6) (7) (8) (9) Literacy 0.0253 0.134 -0.00229 0.0288 0.0537*** 18282 0.331 0.352*** 0.000552 [1.008] [1.130] [0.0192] [0.0197] [0.0186] [0.0199] [0.0548] Numeracy 0.0306 0.127 0.0374** 0.0469** 0.0685*** 18282 0.552 0.114** [1.014] [1.058] [0.0185] [0.0189] [0.0182] [0.0523] Total 0.0292 0.134 0.0194 0.0387** 0.0632*** 18282 0.841 0.0635 [1.012] [1.085] [0.0176] [0.0183] [0.0171] [0.0499] Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values. Missing value dummies are included for children not tested at baseline. Column 7 displays the p-value of the test that the coefficients ML+CHAMP=ML-CHAMP. Column 8 displays the impact of assignment to the mother literacy treatment group on literacy class attendance by mother or child. Column 9 displays the impact of literacy class attendance on the dependent variables, using assignment to the ML treatment group as an instrument for attendance. Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 36 All Obs Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) (6) Monthly tuition fees 14.25 [36.68] 20.92 [53.28] 3.564* [1.926] 2.114 [1.962] 1.299 [1.839] 8438 Child is enrolled 0.775 [0.418] 0.0122 [0.0118] 0.0136 [0.0116] 0.0144 [0.0118] 25053 Child is/will be enrolled 0.905 [0.293] -0.00203 [0.00741] 0.00483 [0.00724] 0.00964 [0.00722] 25053 Child attends school 0.833 [0.373] 0.845 [0.362] -0.00486 [0.0109] 0.00349 [0.0108] 0.0193* [0.0106] 25053 Child attends private school 0.114 [0.318] 0.0805 [0.272] 0.00565 [0.00847] -0.000640 [0.00859] 0.00427 [0.00883] 25053 Hours spent in school 3.642 [1.971] 4.063 [1.599] 0.0554 [0.0554] 0.0827 [0.0537] 0.0885 [0.0558] 8475 Days missed per month 2.152 [4.753] -0.0820 [0.175] -0.0206 [0.183] -0.133 [0.164] 7383 Days missed in last week 2.825 [2.469] 1.379 [1.946] 0.0275 [0.0710] -0.0572 [0.0760] -0.0361 [0.0683] 6980 Table 9 Child schooling Baseline mean Endline mean OLS: Impact of treatment in endline Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values (where available). Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 37 Table 10 Mother-child participation Baseline mean Endline mean OLS: Impact of treatment in endline All Obs Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) (6) Mother-Child Participation Index 0.0231 [0.643] 0.0123 [0.512] 0.0371** [0.0184] 0.0634*** [0.0196] 0.0507** [0.0198] 8231 Take child to school (number of times/week) 0.255 [1.095] 0.336 [1.130] -0.0385 [0.0380] 0.0370 [0.0420] -0.00939 [0.0357] 8451 Visit school (% of mothers) 0.128 [0.335] 0.155 [0.362] 0.00511 [0.0119] 0.0136 [0.0129] 0.0132 [0.0123] 8451 Visit school (% of mothers, not because of bullying or for fees) 0.0969 [0.296] 0.0763 [0.266] 0.00894 [0.00951] 0.0157* [0.00903] 0.00824 [0.00895] 8451 Know whether child received homework (% of mothers) 0.762 [0.426] 0.0158 [0.0151] 0.0321** [0.0150] 0.0189 [0.0162] 8479 Help child with homework (% of mothers) 0.325 [0.469] 0.708 [0.455] 0.0158 [0.0153] 0.0419*** [0.0146] 0.0258* [0.0154] 8479 Time spent helping per week (weekly hours) 1.686 [2.903] 2.313 [2.704] 0.114 [0.0919] 0.126 [0.0973] 0.0602 [0.0918] 8519 Look at notebook (% of mothers) 0.126 [0.332] 0.216 [0.412] 0.0300** [0.0149] 0.0651*** [0.0157] 0.0495*** [0.0152] 8572 Talk to child about school: number of times per week 3.090 [3.018] 0.222** [0.101] 0.194* [0.105] 0.247** [0.105] 8438 Talk to others about child's studies: number of times per week 0.551 [0.497] 1.609 [2.222] 0.235*** [0.0778] 0.181** [0.0773] 0.251*** [0.0797] 8521 Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values (where available). The mother-child participation index is an average of z-scores of the other variables in the table, using the control group means and sd. The baseline participation index only includes indicators for which data were collected. Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 38 Table 11 Child’s time use Baseline Mean Endline Mean OLS: Impact of treatment in endline All Obs Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) (6) Homework (weekly hours) 2.992 [4.103] 3.760 [4.243] -0.00138 [0.149] 0.241* [0.141] 0.302** [0.138] 8331 Reading (weekly hours) 0.258 [1.253] 0.460 [1.571] -0.0451 [0.0485] -0.00993 [0.0526] 0.0549 [0.0531] 7942 Drawing/painting (weekly hours) 0.465 [1.339] 0.698 [1.543] 0.00414 [0.0531] 0.0743 [0.0520] 0.0901* [0.0527] 7902 Playing with an adult (weekly hours) 0.448 [2.001] 0.547 [1.953] 0.000218 [0.0653] -0.0281 [0.0654] -0.0997 [0.0622] 8337 Tuition (weekly hours) 1.848 [4.428] 2.263 [4.861] 0.194 [0.180] 0.151 [0.194] 0.0103 [0.174] 8416 Television (weekly hours) 3.832 [5.644] 3.673 [4.934] -0.112 [0.169] 0.144 [0.175] 0.0408 [0.164] 8339 Housework (weekly hours) 3.182 [4.211] 3.552 [3.987] 0.0547 [0.132] 0.115 [0.129] 0.129 [0.130] 8408 Household business (weekly hours) 1.175 [3.550] 1.786 [3.705] 0.229* [0.128] 0.0882 [0.134] 0.328** [0.128] 8407 Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values (where available). Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 39 Table 12 Education assets Baseline Mean Endline Mean OLS: Impact of treatment in endline All Obs Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) (6) Education assets in home: pencil (% of household) 0.930 [0.256] 0.945 [0.227] -0.0000251 [0.00850] 0.0148** [0.00727] 0.0123 [0.00795] 8581 Education assets in home: school books 0.906 [0.292] 0.0179* [0.00978] 0.0168* [0.00935] 0.0264*** [0.00990] 8581 Education assets in home: other books/comics 0.229 [0.420] 0.245 [0.430] 0.0164 [0.0155] 0.0364** [0.0169] 0.0410** [0.0159] 8581 Education assets in home: newspaper/ magazine 0.122 [0.328] 0.0533 [0.225] 0.0105 [0.00789] 0.0301*** [0.00881] 0.00867 [0.00789] 8581 Education assets in home: slate 0.891 [0.312] 0.0125 [0.0103] -0.00444 [0.0104] 0.0259*** [0.00974] 8581 Education assets in home: none 0.0154 [0.123] -0.00161 [0.00398] -0.00581 [0.00357] -0.00197 [0.00375] 8581 Education assets index 0.0176 [0.651] 7.45e-09 [0.570] 0.0344 [0.0212] 0.0613*** [0.0206] 0.0708*** [0.0211] 8581 Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values (where available). Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 40 Table 13 Mother perceptions Baseline Mean Endline Mean OLS: Impact of treatment in endline All Obs Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) (6) Number of things for which parents are responsible 0.730 [0.773] 1.345 [1.025] 0.0559 [0.0426] 0.0434 [0.0433] 0.0668* [0.0392] 8888 Number of things mother can do to help child 1.246 [0.950] 1.856 [1.151] 0.0454 [0.0436] 0.00462 [0.0486] 0.0816* [0.0441] 8888 Mother thinks child will likely pass 8th standard 0.798 [0.402] 0.818 [0.386] -0.000437 [0.0138] 0.0121 [0.0134] 0.0134 [0.0134] 8490 Mother thinks child will likely pass 12th standard 0.579 [0.494] 0.608 [0.488] 0.00665 [0.0177] 0.0168 [0.0175] 0.0124 [0.0174] 8482 Highest standard to which mother aspires for child to study 9.881 [2.817] 10.13 [3.079] -0.0180 [0.125] 0.0929 [0.130] 0.222 [0.139] 3200 Mother's perception of child's reading ability 1.645 [1.252] 2.452 [1.589] -0.0421 [0.0513] 0.132*** [0.0508] 0.105** [0.0512] 7595 Mother's perception of child's math ability 1.954 [1.593] 2.558 [1.612] 0.00103 [0.0544] 0.237*** [0.0551] 0.160*** [0.0577] 7711 Reading: Absolute value of difference between mother's guess and child score 1.085 [1.089] 1.601 [1.373] -0.0256 [0.0429] 0.103** [0.0445] 0.0315 [0.0426] 7235 Math: Absolute value of difference between mother's guess and child score 1.265 [1.302] 1.476 [1.275] -0.0403 [0.0444] 0.161*** [0.0486] 0.0997** [0.0458] 7350 Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values (where available). Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 41 Table 14 Heterogeneity in impact Outcome: Mother Test Scores Interacted Variable State = Bihar Mother Baseline Score Mother Age Mother Education Level Literacy Numeracy Composit e Literacy Numeracy Composite Literacy Numeracy Composite Literacy Numeracy Composite (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) ML 0.0810* ** 0.0533* ** 0.0657* ** 0.0903* ** 0.120* ** 0.111** * 0.0615 0.0610 0.0596 0.0988* ** 0.122** * 0.117** *[0.026 6] [0.019 9] [0.018 3] [0.018 0] [0.0164 ] [0.0148 ] [0.0768 ] [0.0615] [0.0563 ] [0.0153 ] [0.0165 ] [0.0142 ]CHAMP 0.0681 ** 0.0740* ** 0.0773* ** 0.0370 ** 0.0659* ** 0.0557* ** 0.0933 0.164** * 0.138** 0.0264 * 0.0544* ** 0.0459* **[0.027 2] [0.019 6] [0.017 7] [0.018 1] [0.0150 ] [0.0135 ] [0.0768 ] [0.0601] [0.0564 ] [0.0148 ] [0.0156 ] [0.0127 ]ML-CHAMP 0.0865* ** 0.0973* ** 0.0958* ** 0.125* ** 0.161* ** 0.151** * 0.179* * 0.123* 0.150** 0.131** * 0.173** * 0.162** *[0.026 5] [0.022 1] [0.020 1] [0.018 1] [0.0170 ] [0.0155 ] [0.0792 ] [0.0635] [0.0604 ] [0.0163 ] [0.0176 ] [0.0153 ]Variable 0.221* ** 0.0811 0.169* ** 1.182* ** 0.899* ** 1.059** * - 0.00077 6 - 0.00692* ** - 0.00258* * 0.149** * 0.0711* ** 0.0917* **[0.082 5] [0.050 4] [0.059 8] [0.026 0] [0.0117 ] [0.0141 ] [0.0015 8] [0.00124 ] [0.0011 9] [0.0087 1] [0.0059 4] [0.0056 0]Variable x ML 0.0201 0.130* ** 0.0890* ** 0.0221 - 0.0095 2 - 0.00602 0.00092 1 0.00179 0.00160 - 0.0133 - 0.00567 -0.0104 [0.036 9] [0.032 1] [0.029 4] [0.032 4] [0.0171 ] [0.0192 ] [0.0021 5] [0.00181 ] [0.0016 3] [0.0103 ] [0.0081 4] [0.0075 9]Variable x CHAMP - 0.0561 - 0.0099 6 - 0.0375 0.0571 * 0.0371 ** 0.0369* * - 0.0016 6 - 0.00298 * - 0.00247 0.0051 8 0.00852 0.00607 [0.037 6] [0.029 9] [0.026 5] [0.032 6] [0.0159 ] [0.0180 ] [0.0021 2] [0.00172 ] [0.0016 1] [0.0090 5] [0.0077 2] [0.0063 5]Variable x ML- CHAMP 0.0788 ** 0.123* ** 0.109* ** 0.0370 - 0.0358* * - 0.0118 - 0.0016 5 0.00110 0.00001 76 - 0.0107 - 0.0198** * - 0.0176** * [0.037 0] [0.033 3] [0.030 4] [0.033 0] [0.0170 ] [0.0181 ] [0.0022 0] [0.00188 ] [0.0017 4] [0.0097 1] [0.0073 5] [0.0064 4]N 8552 8552 8552 8552 8552 8552 8552 855 2 8552 8528 8528 8528 Note: Each column displays the results of a regression of the mother's normalised literacy, numeracy or composite test score on treatment dummies, the interaction variable indicated, and interactions of the variable and treatment dummies. Regressions control for baseline test scores (except where the interacted variable is the baseline score itself) and stratum dummies. Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 42 Table 15a Heterogeneity in impact outcome: child test scores Interacted Variable State = Bihar Mother Baseline Score Mother Age Mother Education Level Literacy Numerac y Composit e Literacy Numeracy Composite Literacy Numeracy Composite Literacy Numeracy Composite (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) ML - 0.0029 9 0.028 1 0.014 7 - 0.0010 7 0.0385* * 0.0207 -0.101 - 0.0849 -0.0946 - 0.0067 9 0.0371* * 0.0173 [0.027 7] [0.024 3] [0.024 2] [0.0187 ] [0.0179 ] [0.0170 ] [0.071 8] [0.063 5] [0.0622 ] [0.0188 ] [0.0182 ] [0.0172 ]CHAMP 0.023 1 0.0511 ** 0.038 6 0.0226 0.0453* * 0.0350* * - 0.0300 - 0.0251 -0.0317 0.0116 0.0402* * 0.0275 [0.028 2] [0.025 5] [0.025 2] [0.0174 ] [0.0180 ] [0.0167 ] [0.073 3] [0.062 6] [0.0620 ] [0.0179 ] [0.0182 ] [0.0170 ]ML-CHAMP 0.043 8 0.0494 ** 0.0480 ** 0.0525* ** 0.0682* ** 0.0625* ** 0.0200 0.121* 0.0800 0.0457* * 0.0655* ** 0.0579* **[0.026 9] [0.022 4] [0.022 3] [0.0174 ] [0.0176 ] [0.0162 ] [0.075 2] [0.068 5] [0.0669 ] [0.0183 ] [0.0177 ] [0.0166 ]Variable - 0.0695 - 0.0243 - 0.040 6 0.0775* ** 0.0638* ** 0.0743* ** - 0.00481 *** - 0.00235 * - 0.00381* ** 0.0336* ** 0.0302* ** 0.0316* **[0.066 7] [0.060 0] [0.058 6] [0.0125 ] [0.0110 ] [0.0112 ] [0.0015 2] [0.0012 5] [0.0012 3] [0.0061 0] [0.0059 7] [0.0057 8]Variable x ML 0.0015 3 0.016 9 0.0085 4 0.0049 6 0.0172 0.0079 5 0.0030 3 0.00379 ** 0.00352 * 0.0064 8 0.0031 7 0.0045 2[0.038 3] [0.036 3] [0.034 9] [0.0174 ] [0.0151 ] [0.0151 ] [0.0020 9] [0.0018 7] [0.0018 0] [0.0080 7] [0.0080 7] [0.0076 6]Variable x CHAMP 0.010 3 - 0.0076 2 0.0001 33 0.0298 * 0.0141 0.0177 0.0018 0 0.0022 3 0.0021 7 0.0147 * 0.0046 1 0.0089 6[0.039 3] [0.037 2] [0.036 3] [0.0178 ] [0.0152 ] [0.0158 ] [0.0020 9] [0.0018 3] [0.0017 7] [0.0084 4] [0.0076 0] [0.0075 3]Variable x ML- CHAMP 0.018 2 0.035 0 0.027 9 0.0217 0.0225 0.0219 0.0010 3 - 0.0016 4 - 0.00053 9 0.0081 5 0.0030 1 0.0055 2[0.037 2] [0.035 5] [0.033 7] [0.0184 ] [0.0150 ] [0.0155 ] [0.0022 1] [0.0020 0] [0.0019 3] [0.0091 8] [0.0079 5] [0.0080 3]N 18282 18282 18282 17823 17823 17823 18282 18282 18282 18234 18234 18234 Note: Each column displays the results of a regression of the child's normalised literacy, numeracy or composite test score on treatment dummies, the interaction variable indicated, and interactions of the variable and treatment dummies. Regressions control for baseline test scores (except where the interacted variable is the baseline score itself) and stratum dummies. Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 43 Table 15b Heterogeneity in impact outcome: child test scores Interacted Variable Child Age Child’s Baseline Score Gender Literacy Numeracy Composite Literacy Numeracy Composite Literacy Numeracy Composite (13) (14) (15) (16) (17) (18) (19) (20) (21) ML 0.0256 0.0188 0.0226 -0.00954 0.0385* 0.0165 -0.00873 0.0360 0.0164 [0.0351] [0.0331] [0.0300] [0.0210] [0.0198] [0.0186] [0.0235] [0.0231] [0.0214] CHAMP 0.0110 0.0306 0.0230 0.0172 0.0460** 0.0325* 0.0370 0.0588** 0.0496** [0.0350] [0.0338] [0.0316] [0.0203] [0.0197] [0.0186] [0.0238] [0.0240] [0.0222] ML-CHAMP -0.0367 0.0389 0.00669 0.0547*** 0.0723*** 0.0658*** 0.0554** 0.0609*** 0.0598*** [0.0326] [0.0342] [0.0310] [0.0198] [0.0194] [0.0179] [0.0225] [0.0228] [0.0209] Variable 0.0412*** 0.0685*** 0.0486*** 0.970*** 0.888*** 0.951*** 0.0481*** 0.0829*** 0.0651*** [0.00558] [0.00551] [0.00516] [0.0133] [0.0118] [0.0102] [0.0180] [0.0185] [0.0166] Variable x ML -0.00456 0.00283 -0.000608 0.00909 0.00940 0.00925 0.0137 0.00442 0.00707 [0.00597] [0.00548] [0.00487] [0.0201] [0.0161] [0.0150] [0.0248] [0.0253] [0.0226] Variable x CHAMP 0.00265 0.00259 0.00244 0.0195 -0.0173 0.000399 -0.0156 -0.0225 -0.0207 [0.00603] [0.00546] [0.00514] [0.0198] [0.0163] [0.0150] [0.0263] [0.0257] [0.0237] Variable x ML- CHAMP 0.0143** 0.00447 0.00890* 0.0193 -0.0302* -0.00619 -0.00200 0.0175 0.00862 [0.00555] [0.00548] [0.00488] [0.0184] [0.0166] [0.0142] [0.0263] [0.0257] [0.0239] N 18281 18281 18281 14575 14575 14575 18282 18282 18282 Note: Each column displays the results of a regression of the child's normalised literacy, numeracy or composite test score on treatment dummies, the interaction variable indicated, and interactions of the variable and treatment dummies. Regressions control for baseline test scores (except where the interacted variable is the baseline score itself) and stratum dummies. Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 44 Table 16 Cost-effectiveness of interventions (in US$) Intervention ML CHAMP ML-CHAMP Panel A: Cost Summary Pratham staff $37,521 $30,699 $68,219 Volunteer time $21,653 -- $21,653 Training, monitoring, materials $14,861 $4,730 $17,289 Total $74,035 $35,428 $107,162 Panel B: Standard Deviation Improvement per $100 spent--Children Literacy -- -- $4.29 Math $4.33 $1.70 $3.36 Composite -- $2.06 $3.64 Children affected 4,572 4,447 4,653 Panel C: Standard Deviation Improvement per $100 spent--Mothers Literacy $3.73 $4.19 $3.95 Math $2.84 $2.42 $3.13 Composite $3.07 $2.85 $3.32 Mothers affected 2,176 2,115 2,151 Note: Costs incurred in rupees converted to dollars using 2011 exchange rate of 46.7 rupees/dollar. Volunteer time estimated based on average daily wage in non-agricultural occupations. Cost-effectiveness estimates in Panels B and C computed based on total costs of each programme divided by the (a) effect sizes reported in Tables 4 and 8, for mothers and children, respectively, (b) number of beneficiaries and (c) 100. 45 Appendix A: Location selection Because of the slightly different organisation of villages in Rajasthan and Bihar, a different selection procedure was used in each state. The procedure focused on finding distinct geographic units, called hamlets, in which the programmes could run, while limiting spillovers. Hamlet eligibility was therefore determined by size, according to the number of households, and distance from other target hamlets. Size and location of hamlets were determined from rapid rural assessments, conducted in study blocks. In Rajasthan, dispersed clusters of villages comprise larger geographic units known as gram panchayats. Villages are divided into smaller hamlets known as mohellas. Hamlets in Rajasthan met the size eligibility requirements if they contained between 40 and 100 households, whereby a household is defined as a family that eats from one kitchen.14 To limit spillovers, one hamlet per village was selected. All the villages in two blocks – Kekri and Bhinay – were targeted for the intervention in Rajasthan. Within each village, first preference was given to hamlets with 60–80 households, as Pratham and the research team determined this to be the approximate size to support one adult literacy class. In each village, the hamlet with 60–80 households was selected unless there was more than one, in which case one hamlet of that size was chosen at random. If there were no hamlets in a village with 60–80 households, second preference was given to hamlets of 40–100 households. Again, if there was only one hamlet in a target village with 40–100 households, it was selected; otherwise, one hamlet of that size was selected at random. To identify a total of 240 target hamlets, the boundaries of Kekri and Bhinay were extended into a third block. Target hamlets were identified using the same procedure used in Kekri and Bhinay until 240 had been selected. The average size of study hamlets was 74 households. Targeted households contained 7.0 members, on average. In Bihar, the village boundaries are less distinct and villages are much denser than in Rajasthan. Each panchayat has multiple revenue villages, with each revenue village comprising smaller hamlets known as tolas (the equivalent of a mohella in Rajasthan). Within each revenue village, there is typically a main village and hamlets that surround the main village. In Bihar, hamlets were considered eligible if they contained between 25 and 150 households15 and if they were at least 500 metres from any other target hamlet. 14 In one instance, a hamlet containing more than 100 households was split into smaller synthetic hamlets for the purposes of the intervention. 15 The household criteria differed between Rajasthan and Bihar because the criteria for Rajasthan would not have produced a sufficient number of eligible hamlets in Bihar. Due to the higher upper bound on number of households, Pratham agreed to hold more than one class in a target hamlet where necessary in Bihar. 46 All revenue villages and hamlets in two blocks – Dhamdaha and B. Kothi – were targeted for the intervention. To limit spillovers, hamlets in Bihar were selected only if their boundaries were 500 metres or more from the boundaries of other target hamlets.16 If an eligible hamlet was closer than 500 metres from another, the one with 40–80 households was selected, as Pratham determined this to be the approximate size to support one adult literacy class. If more than one hamlet contained 40–80 households within the 500-metre radius, one was randomly selected for the intervention. Second preference was given to a hamlet of 25–150 households where no hamlet in the 500-metre radius contained between 40 and 80 households.17 If the eligible hamlets were in an adjacent row, the hamlets at the ends of the row were selected.18 The selection process yielded 269 eligible hamlets. Of those, 240 hamlets were randomly selected, and the remainder were used by Pratham for pilot activities. The average size of study hamlets was 68 households. Targeted households contained 6.6 members, on average. 16 GPS coordinates were used to confirm distances between the boundaries of hamlets. Distances were checked between hamlets within revenue villages as well as across revenue villages. 17 In any given 500-metre radius, if there were no hamlets of 40–80 households but multiple hamlets of 25–150 households, one hamlet of 25–150 households was selected at random. 18 In two cases, target hamlets were eliminated because Pratham determined that adult literacy rates were too high to sustain classes. 47 Appendix B: Power calculations The following section is a formatted version of the power calculations section of the original 3ie grant proposal. We look to prior Pratham interventions when setting a reasonable effect size. The data used for our power calculations come from an evaluation of the Pratham Read programme in Uttar Pradesh in 2005–2006. This evaluation used the literacy test included in the ASER survey to measure child literacy levels both at a baseline and at an endline 12 months later. Banerjee et al. (2007) found that being in a school and grade that was assigned a balsakhi (Pratham-trained tutor) resulted in a 0.15 sd increase in test scores in the first year of the programme, and a 0.25 sd increase in year two. Meanwhile, fewer than 50 per cent of the students ever interacted with the balsakhi; those who did not interact with the balsakhi did not appear to improve relative to the control group. Our study, which uses child literacy levels as the primary outcome measure and basis for sample size, will similarly observe endline literacy levels nine months after baseline measurements. We assume similar impact and take-up – a 0.15 sd average treatment effect on children and a 50 per cent mother participation rate in the literacy programme or the materials intervention – implying a 0.3 sd impact on children whose mothers participated. We are relatively confident in this compliance figure given past experience in these communities, and because the mothers have already signalled their interest in improving their children’s education by sending their young children to reading classes. The estimated take-up rate will be refined through pilots. According to data from the Uttar Pradesh study, baseline reading levels and other demographic covariates explain a little more than 60 per cent of the variance in endline reading levels. Hence, our power calculations use R2=0.62. Conducting an analysis of variance of reading level for children aged 5–8 in the Uttar Pradesh study, and defining the village as our cluster, we find an intra-cluster correlation (rho) of 0.079. Using the above inputs, and accepting a type two and type one error rate of 80 per cent and 5 per cent, respectively, we calculate that the minimum number of clusters required to detect a 0.15 sd impact is 108 (implying 54 for one treatment and 54 for the control). We round the villages-per-group number from 54 up to 60 and apply it to the other two interventions. Our power calculations show that if we survey 20 children per village (and their mothers), we require 60 villages per treatment group to distinguish an impact of 0.15 sd. Since we aim to detect this impact within each state, this sample is required for each state independently. Therefore, our sample totals to 240 villages per state, or 480 villages total. Pooling the two states together, the minimum detectable effect would be 0.1 sd. Further pooling the different treatments that include mother literacy (ML + MLCHAMP) or mother-child activities (CHAMP + ML-CHAMP) will allow detection of even smaller impacts when compared to the control. Based on these (and the above) assumptions, we believe our power calculations are relatively conservative. It is important to note that the effect size applies to both the comparison between the intervention groups and the control group, as well as the comparison between interventions. For example, the impact of the combined intervention will have to be greater than that of the individual interventions by at least 0.15 sd for an advantage to be detectable. We hypothesise that these individual interventions are highly complementary, and thus that this difference is not unreasonable. 48 We will also look at differential outcomes in subgroups that are socially marginalised to greater and lesser degrees. It is important to note that the assumptions in our power calculations do not account for this sub-group analysis within each state. However, as mentioned above, pooling the two states together can increase our sample and thus allow such comparisons. For example, while our target population of illiterate children with illiterate mothers defines an already deeply marginalised group, some of the children enrolled in reading classes will have semi-literate mothers, and will be included in both the sample and population. Therefore, on top of measuring the improvement in our target population's learning levels, these additional data will allow us to measure improvement vis-a-vis the less marginalised groups, such as children with semi-literate mothers. Additionally, as described in the heterogeneity section, we will examine differential outcomes for girls and boys. Though girls of this age range in these regions do not perform lower than the corresponding boys, they may be considered marginalised for other social reasons. 49 Appendix Table s Appendix Table 1 Baseline means of variables in Rajasthan and Bihar samples Rajasthan Bihar (1) (2) Education and learning Mother’s education level 0.741 0.922 (2.173) (2.480) Father’s education level 4.819 3.124 (4.436) (4.296) Mother's weighted baseline literacy score 3.220 2.871 (2.535) (2.548) Mother's weighted baseline numeracy score 3.612 2.601 (3.504) (3.391) Child attends school 0.840 0.827 (0.366) (0.378) Child attends private school 0.155 0.0240 (0.362) (0.153) Child's weighted baseline literacy score 3.144 2.626 (2.546) (2.224) Child's weighted baseline numeracy score 3.022 2.730 (3.018) (3.116) Assets and demographics First principal component of durables ownership 1.250 -1.202 (2.281) (1.397) Does household have electricity 0.808 0.154 (0.394) (0.361) Roof has cement, stone, metal, beams or plastic 0.981 0.626 (0.136) (0.484) Family's largest source of income was self-employed agriculture or rent agriculture 0.517 0.385 (0.500) (0.487) Family's largest source of income was agricultural wages, regular wages or irregular wages 0.311 0.543 (0.463) (0.498) Number of children (aged 14 and younger) in household 4.671 5.038 (1.777) (1.657) Number of adults (aged 15 and older) in household 3.682 2.980 (2.099) (1.395) Mother’s time use Hours weekly spent on housework 21.27 23.10 (8.374) (9.185) Hours weekly spent on agricultural, paid, and livestock work 46.11 26.38 (17.78) (19.03) Hours weekly spent looking after, telling stories to, or playing with kids 5.300 7.456 (3.945) (4.886) Hours weekly spent helping with homework or reading with kids 1.378 2.385 (2.458) (3.876) Note: Standard deviations in parentheses. 50 Appendix Table 2a Question-wise literacy baseline means Baseline Endline Baseline Weighted Score Weight Weight Mothers Children (1) (2) (3) (4) Identify pictures 2 2 1.495 1.458 (0.388) (0.566) Read letters 2 2 0.392 0.690 (0.712) (0.838) Read simple words 1 1 0.146 0.251 (0.341) (0.416) Read complex words 1 1 0.131 0.214 (0.327) (0.393) Read paragraph 2 2 0.197 0.250 (0.596) (0.662) Read Story * -- 2 Write own name * 1 1 0.565 (0.496) Write child's name * 1 1 0.117 (0.321) Write village name * -- 2 Literacy score 10 14 3.042 2.863 (2.547) (2.390) Note: Columns 1 and 2 display the weights used in aggregating the test questions. * Denotes that the question appeared only on the mother test. 51 Appendix Table 2b Question-wise numeracy baseline means Baseline Endline Baseline Weighted Score Weight Weight Mothers Children (1) (2) (3) (4) Subtraction word problem * 1 1 0.402 (0.490) Division word problem * 1 1 0.228 (0.420) Read digits 1-9 1 1 0.593 0.626 (0.471) (0.466) Identify digits 1-9 1 1 0.448 0.529 (0.459) (0.474) Read digits 11-20 1 1 0.299 0.449 (0.443) (0.492) Identify digits 11-20 1 1 0.192 0.349 (0.371) (0.457) Identify numbers 21-99 1 1 0.103 0.163 (0.286) (0.339) Single digit addition 1 1 0.127 0.266 (0.333) (0.442) Double digit addition 1 1 0.101 0.207 (0.302) (0.405) Single digit subtraction 1 1 0.0781 0.151 (0.268) (0.358) Double digit subtraction 2 2 0.0865 0.124 (0.407) (0.483) Tell time: 10:30 * 0.5 0.5 0.117 (0.212) Tell time: 1:40 * 0.5 0.5 0.0826 (0.186) Dial a number read out loud * 1 1 0.238 (0.426) Numeracy score 14 14 3.096 2.863 (3.483) (3.075) Note: Columns 1 and 2 display the weights used in aggregating the test questions. * Denotes that the question appeared only on the mother test. 52 Appendix Table 3 Mother question-wise treatment effects: language Baseline Mean Endline Mean OLS: Impact of treatment in endline All Obs Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) (6) Identify pictures 0.747 [0.194] 0.775 [0.186] 0.00275 [0.00477] 0.00759 [0.00471] 0.0116** [0.00450] 8552 Read Letters 0.196 [0.356] 0.173 [0.338] 0.0351*** [0.00593] 0.00927 [0.00580] 0.0474*** [0.00609] 8552 Read simple words 0.146 [0.341] 0.115 [0.308] 0.0165*** [0.00472] 0.0117** [0.00486] 0.0244*** [0.00499] 8552 Read complex words 0.131 [0.327] 0.0922 [0.278] 0.00540 [0.00436] 0.00973** [0.00460] 0.0108** [0.00432] 8552 Read paragraph 0.0985 [0.298] 0.0776 [0.268] 0.00243 [0.00421] 0.00714 [0.00507] 0.00334 [0.00443] 8552 Read story 0.0627 [0.242] 0.00303 [0.00968] 0.0141 [0.0112] 0.00571 [0.00986] 8580 Write own name 0.565 [0.496] 0.556 [0.497] 0.0794*** [0.0145] 0.00835 [0.0134] 0.0907*** [0.0143] 8552 Write child's name 0.117 [0.321] 0.105 [0.306] 0.0212*** [0.00668] 0.00537 [0.00615] 0.00571 [0.00986] 8551 Write name of village 0.0856 [0.280] 0.00843 [0.0113] 0.0136 [0.0133] 0.0159 [0.0116] 8580 Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values (where available). Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 53 Appendix Table 4 Mother question-wise treatment effects: math Mean Endline Mean OLS: Impact of treatment in endline All Obs Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) (6) Read digits 1-9 0.593 [0.471] 0.627 [0.460] 0.0967*** [0.0134] 0.0457*** [0.0129] 0.140*** [0.0128] 8552 Identify digits 1-9 0.448 [0.459] 0.469 [0.460] 0.0661*** [0.0103] 0.0304*** [0.0100] 0.109*** [0.0108] 8552 Read digits 11-20 0.299 [0.443] 0.298 [0.442] 0.0677*** [0.00997] 0.0204** [0.00965] 0.0779*** [0.0101] 8552 Identify digits 11-20 0.192 [0.371] 0.182 [0.361] 0.0228*** [0.00677] 0.0130* [0.00737] 0.0399*** [0.00752] 8552 Identify numbers 21-99 0.103 [0.286] 0.0964 [0.276] 0.0140*** [0.00435] 0.00636 [0.00441] 0.0147*** [0.00439] 8552 Single digit addition 0.127 [0.333] 0.129 [0.335] 0.0124 [0.00752] 0.0134* [0.00727] 0.0321*** [0.00762] 8552 Double digit addition 0.101 [0.302] 0.0781 [0.268] 0.00200 [0.00550] 0.00555 [0.00592] 0.0137** [0.00565] 8552 Single digit subtraction 0.0781 [0.268] 0.0692 [0.254] 0.00651 [0.00574] 0.00445 [0.00580] 0.00592 [0.00570] 8551 Double digit subtraction 0.0432 [0.203] 0.0580 [0.234] 0.0127* [0.00674] 0.0134* [0.00689] 0.00608 [0.00615] 8552 Double digit subtraction with carry over 0.0355 [0.185] 0.00662 [0.00735] 0.0147* [0.00882] 0.00253 [0.00728] 8580 Subtraction word problem 0.402 [0.490] 0.240 [0.427] 0.0239 [0.0149] 0.0238 [0.0160] 0.0381*** [0.0145] 8552 Division word problem 0.228 [0.420] 0.134 [0.340] 0.0264** [0.0110] 0.0271** [0.0115] 0.0238** [0.0115] 8552 Tell time at 10:30 0.233 [0.423] 0.252 [0.434] 0.00952 [0.0111] 0.00460 [0.0129] -0.00427 [0.0124] 8552 Tell time at 1:40 0.165 [0.371] 0.0767 [0.266] 0.0107 [0.00771] 0.00830 [0.00773] -0.00331 [0.00765] 8552 Dial a number that is read out to her 0.238 [0.426] 0.261 [0.440] 0.0287** [0.0111] 0.00522 [0.0119] 0.0283** [0.0112] 8552 Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values (where available). Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 54 Appendix Table 5 Child question-wise treatment effects: language Baseline Mean Endline Mean OLS: Impact of treatment in endline All Obs Control ML CHAMP ML-CHAMP N (1) (2) (3) (4) (5) (6) Identify Pictures 0.729 [0.283] 0.758 [0.285] 0.000473 0.0104* [0.00633] [0.00625] 0.00966 [0.00603] 18282 Read Letters 0.345 [0.419] 0.387 [0.430] 0.00110 0.0146 [0.00892] [0.00917] 0.0272*** [0.00894] 18282 Read simple words 0.251 [0.416] 0.271 [0.421] -0.00517 0.0124 [0.00826] [0.00862] 0.0121 [0.00818] 18282 Read complex words 0.214 [0.393] 0.175 [0.352] 0.00116 0.00983 [0.00642] [0.00633] 0.0122** [0.00618] 18282 Read paragraph 0.125 [0.331] 0.117 [0.321] -0.00211 0.00215 [0.00620] [0.00621] 0.0134** [0.00611] 18282 Read story 0.0757 [0.265] 0.00713 0.00749 [0.00875] [0.00838] 0.0101 [0.00823] 18282 Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values (where available). Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 55 Appendix Table 6: Child question-wise treatment effects: math Mean Endline Mean OLS: Impact of treatment in endline All Obs Control ML CHAMP MLCHAMP N (1) (2) (3) (4) (5) (6) Read digits 1-9 0.626 [0.466] 0.652 [0.459] 0.0249*** [0.00938] 0.0227** [0.00925] 0.0397*** [0.00885] 18282 Identify digits 1-9 0.529 [0.474] 0.560 [0.474] 0.0203** [0.00856] 0.0385*** [0.00876] 0.0399*** [0.00861] 18282 Read digits 11-20 0.449 [0.492] 0.487 [0.496] 0.0195* [0.0101] 0.0275*** [0.0102] 0.0482*** [0.00974] 18282 Identify digits 11-20 0.349 [0.457] 0.387 [0.470] 0.00338 [0.00936] 0.0189* [0.00987] 0.0332*** [0.00887] 18282 Identify numbers 21-99 0.163 [0.339] 0.200 [0.367] 0.00104 [0.00724] 0.00285 [0.00713] 0.00800 [0.00692] 18282 Single digit addition 0.266 [0.442] 0.311 [0.463] 0.0130 [0.0104] 0.0254** [0.0104] 0.0192* [0.0102] 18282 Double digit addition 0.207 [0.405] 0.196 [0.397] 0.0231** [0.00946] 0.00822 [0.00914] 0.00790 [0.00880] 18282 Single digit subtraction 0.151 [0.358] 0.171 [0.377] 0.00757 [0.00838] 0.0193** [0.00896] 0.0143* [0.00842] 18282 Double digit subtraction 0.0622 [0.241] 0.133 [0.339] 0.0132 [0.00912] 0.0153* [0.00914] 0.0131 [0.00918] 18282 Double digit subtraction with carry over 0.0579 [0.234] 0.0150* [0.00779] 0.00115 [0.00742] 0.00637 [0.00693] 18282 Note: Columns 3, 4 and 5 display estimated coefficients of a regression of the outcome in each row on treatment group dummies, controlling for stratification unit dummies and baseline values (where available). Standard errors are clustered at the village level. * denotes significance at 0.10; ** at 0.05; *** at 0.01 56 References Abadzi, H, 2003. Adult literacy: a review of implementation experience. The World Bank Operations Evaluation Department. Aker, JC, Ksoll, C and Lybbert, TJ, 2012. Can mobile phones improve learning? evidence from a field experiment in Niger. American Economic Journal: Applied Economics, 4(4), pp.94-120. Andrabi, T, Das, J and Khwaja, AI, 2012. What did you do all day?: maternal education and child outcomes. Journal of Human Resources, 47, pp.873–912. Aoki, A, 2005. Assessing learning achievements and development impact: Ghana’s national functional literacy program. Australian Journal of Adult Learning, 45(1), pp.47- 62. ASER Centre, 2013. Annual status of education report (rural) 2012. Avvisati, F, Gurgand, M, Guyon, N and Maurin, E, 2014. Getting parents involved: a field experiment in deprived schools. Review of Economic Studies, 81(1), pp.57-83. Banerjee, A, Cole, S, Duflo, E and Linden, L, 2007. Remedying education: evidence from two randomized experiments in India. Quarterly Journal of Economics, 122(3), pp.1235– 64. Banerjee, AV, Banerji, R, Duflo, E, Glennerster, R and Khemani, S, 2010. Pitfalls of participatory programs: Evidence from a randomized evaluation in education in India. American Economic Journal: Economic Policy, 2(1), pp.1–30. Beder, H, 1999. The outcomes and impacts of adult literacy education in the United States. National Center for the Study of Adult Learning and Literacy Report 6, (6). Bekman, S, 1998. A fair chance: an evaluation of the mother-child program. MotherChild Education Foundation Publication 13. Carron, G, 1990. The functioning and effects of the Kenya literacy program. African Studies Review, 33(3), pp.97–120, ISSN 00020206. Central Statistics Office of India, 2013. Statement: per capital net state domestic product at constant (2004-2005) prices. Technical report, Central Statistics Office. Available at: Duflo, E, 2012. Women empowerment and economic development. Journal of Economic Literature, 50(4), pp.1051–79. Government of India, 2011a. Census of India 2011 Provisional Population Totals. Retrieved July 13, 2014 from http://censusindia.gov.in/2011-prov- results/prov_results_paper1_india.html. Government of India, 2011b. Census of India 2011 Source of Lighting: 2001-2011. Retrieved July 13, 2014 from http://www.censusindia.gov.in/2011census/hlo/Data_sheet/India/Source_Lighting.pdf. 57 Government of India, 2014. Adult Education – Overview. Retrieved July 13, 2014 from http://mhrd.gov.in/adulteducation. The Hindu, 2009. ‘Manmohan launches National Mission for Female Literacy’. September 8, 2009. Kling, JR, Liebman, JB and Katz, LF, 2007. Experimental analysis of neighborhood effects. Econometrica, 75(1), pp.83–119. Kremer, M, Brannen, C and Glennerster, R, 2013. The challenge of education and learning in the developing world. Science, 340(6130), pp.297-300. Nguyen, T, 2008. Information, role models and perceived returns to education: experimental evidence from Madagascar. Mimeo, MIT. Ortega, D and Rodriguez, F, 2008. Freed from illiteracy? a closer look at Venezuela’s misión robinson literacy campaign. Economic Development and Cultural Change, 57(1), pp.1–30. Rosenzweig, MR and Wolpin, KI, 1994. Are there increasing returns to the intergenerational production of human capital? maternal schooling and child intellectual achievement. The Journal of Human Resources, Vol.29, No.2. St. Pierre, T, Swartz, J, Murray, S, Deck, D and Nickel, P, 1993. National evaluation of the Even Start Family Literacy Program: report on effectiveness. US Department of Education – Office of Policy and Planning. Thomas, D, Schoeni, RF and Strauss, J, 1996. Parental investments in schooling: the roles of gender and resources in urban Brazil. Labor and Population Program, Working Paper Series, 96-02. UK Department for International Development, 2008. Adult literacy: an update. Department for International Development Briefing. World Bank Group, 2013. World development indicators, education statistics. Technical report, The World Bank Group. 58 Publications in the 3ie Impact Evaluation Report Series The following reports are available from http://www.3ieimpact.org/en/publications/3ie- impact-evaluation-reports/3ie-impact-evaluations/ The promise of preschool in Africa: A randomised impact evaluation of early childhood development in rural Mozambique, 3ie Impact Evaluation Report 1. Martinez, S, Naudeau, S and Pereira, V (2012) A rapid assessment randomised-controlled trial of improved cookstoves in rural Ghana, 3ie Impact Evaluation Report 2. Burwen, J and Levine, DI (2012) The GoBifo project evaluation report: Assessing the impacts of community-driven development in Sierra Leone, 3ie Impact Evaluation Report 3. Casey, K, Glennerster, R and Miguel, E (2013) Does marginal cost pricing of electricity affect groundwater pumping behaviour of farmers? Evidence from India, 3ie Impact Evaluation Report 4. Meenakshi, JV, Banerji, A, Mukherji, A and Gupta, A (2013) Impact evaluation of the non-contributory social pension programme 70 y más in Mexico, 3ie Impact Evaluation Report 5. Rodríguez, A, Espinoza, B, Tamayo, K, Pereda, P, Góngora, V, Tagliaferro, G and Solís, M (2014) The impact of daycare on maternal labour supply and child development in Mexico, 3ie Impact Evaluation Report 6. Angeles, G, Gadsden, P, Galiani, S, Gertler, P, Herrera, A, Kariger, P and Seira, E (2014) Social and economic impacts of Tuungane: final report on the effects of a communitydriven reconstruction programme in the Democratic Republic of Congo, 3ie Impact Evaluation Report 7. Humphreys, M, Sanchez de la Sierra, R and van der Windt, P (2013) Paying for performance in China’s battle against anaemia, 3ie Impact Evaluation Report 8. Zhang, L, Rozelle, S and Shi, Y (2013) No margin, no mission? Evaluating the role of incentives in the distribution of public goods in Zambia, 3ie Impact Evaluation Report 9. Ashraf, N, Bandiera, O and Jack, K (2013) Truth-telling by third-party audits and the response of polluting firms: Experimental evidence from India, 3ie Impact Evaluation Report 10. Duflo, E, Greenstone, M, Pande, R and Ryan, N (2013) An impact evaluation of information disclosure on elected representatives’ performance: evidence from rural and urban India, 3ie Impact Evaluation Report 11. Banerjee, A, Duflo, E, Imbert, C, Pande, R, Walton, M and Mahapatra, B (2014) Targeting the poor: evidence from a field experiment in Indonesia, 3ie Impact Evaluation Report 12. Atlas, V, Banerjee, A, Hanna, R, Olken, B, Wai-poi, M and Purnamasari, R (2014) Scaling up male circumcision service provision: results from a randomised evaluation in Malawi, 3ie Impact Evaluation Report 13. Thornton, R, Chinkhumba, J, Godlonton, S and Pierotti, R (2014) Providing collateral and improving product market access for smallholder farmers: a randomised evaluation of inventory credit in Sierra Leone, 3ie Impact Evaluation Report 14. Casaburi, L, Glennerster, R, Suri, T and Kamara, S (2014) 59 A youth wage subsidy experiment for South Africa, 3ie Impact Evaluation Report 15. Levinsohn, J, Rankin, N, Roberts, G and Schöer, V (2014) The impact of mother literacy and participation programmes on child learning: evidence from a randomised evaluation in India, 3ie Impact Evaluation Report 16. Banerji, R, Berry, J and Shortland, M (2014) Assessing long-term impacts of conditional cash transfers on children and young adults in rural Nicaragua, 3ie Impact Evaluation Report 17. Barham, T, Macours, K, Maluccio, JA, Regalia, F, Aguilera, V and Moncada, ME (2014) Impact of malaria control and enhanced literacy instruction on educational outcomes among school children in Kenya: a multi-sectoral, prospective, randomised evaluation, 3ie Impact Evaluation Report 18. Brooker, S and Halliday, K (2014) A randomised evaluation of the effects of an agricultural insurance programme on rural households’ behaviour: evidence from China, 3ie Impact Evaluation Report 19. Cai, J, de Janvry, A and Sadoulet, E (2014) Environmental and socioeconomic impacts of Mexico's payments for ecosystem services programme, 3ie Impact Evaluation Report 20. Alix-Garcia, J, Aronson, G, Radeloff, V, Ramirez-Reyes, C, Shapiro, E, Sims, K and Yañez-Pagans, P (2014) Shelter from the storm: upgrading housing infrastructure in Latin American slums, 3ie Impact Evaluation Report 21. Galiani, S, Gertler, P, Cooper, R, Martinez, S, Ross, A and Undurraga, R (2014) Impact Evaluation Series International Initiative for Impact Evaluation c/o Global Development Network 2nd Floor, West Wing, ISID Complex Plot No. 4, Vasant Kunj Institutional Area New Delhi – 110070 India 3ie@3ieimpact.org Tel: +91 11 4323 9494 While significant progress has been made in low- and middle-income countries towards the enrolment of children in schools, it has not led to an equivalent increase in their learning. A 2012 survey by the Annual Status of Education Report Centre in India found that 96 per cent of rural primary school-aged children were enrolled, but only 38 per cent could read a simple story. This study evaluates whether child learning can be improved through interventions that focus on increasing a mother’s literacy and enhancing home learning. The impact evaluation found that all of the interventions had statistically significant effects on children’s maths scores. Only a combined intervention that focused on both a mother’s literacy as well as training for enhancing home learning had significant effects on language scores. www.3ieimpact.org