| Notes |
Rogers, T. & Feller, A. (2018). Reducing student absences at scale by targeting parents’ misbeliefs. Nature Human Behaviour.
Reducing Student Absences at Scale by Targeting Parents’ Misbeliefs
Todd Rogers
Harvard Kennedy School Harvard University
Avi Feller
Goldman School of Public Policy University of California, Berkeley
Student attendance is critical to educational success, and is increasingly the focus of educators, researchers, and policymakers. We report the results of a randomized experiment examining in- terventions targeting student absenteeism. Parents of 28,080 high-risk Kindergarten through 12th grade students received one of three personalized information treatments repeatedly throughout the school year or received no additional communication (control). The most effective versions reduced chronic absenteeism by 10% or more, partly by correcting parents’ biased beliefs about their stu- dents’ total accumulated absences. The intervention reduced student absences comparably across grade levels, and reduced absences among untreated cohabiting students in treated households. This intervention is easy to scale and is more than one order of magnitude more cost effective than current absence-reduction best practices. Educational interventions that inform and empower parents, like those reported here, can complement more intensive student-focused absenteeism interventions.
Student absenteeism in the United States is astonish- ingly high. Among US public school students, over 10% are chronically absent each year (defined as missing 18 or more days of school)1,2. The rates are even higher in low- income, urban districts3. The consequences of chronic absenteeism are significant. For students, absences ro- bustly predict academic performance2,4,5,6,7, high school graduation8,9, drug and alcohol use10, criminality11,12, and risk of later life adverse outcomes13. For schools and districts, student absenteeism is often a key perfor- mance metric, and, in many states, absenteeism is tied directly to performance evaluations and funding14. Poli- cymakers have recently redoubled their efforts to reduce absences, such as in the Every Student Succeeds Act15 and in an Obama Administration initiative that aimed to reduce chronic absenteeism by 10% each year16. Meeting goals like these, however, is challenging. Existing best practices, such as assigning students mentors or social workers, have limited effect, can be difficult to scale, and can be expensive. A randomized experimental evaluation of a mentor program in a population similar to the one studied in this paper estimated that the program reduced absenteeism at a cost of around $500 per incremental day of attendance generated17.
This manuscript reports results from a large-scale ran- domized experiment evaluating an intervention that re- duces student absenteeism18 . The intervention deliv- ered personalized information through repeated rounds of mail-based messaging targeting key misbeliefs held by parents of at-risk students (N=28,080). The most effec- tive treatment arm reduced total absences by 6% and the fraction of students who were chronically absent by over 10% relative to a control group. The approach is ex- tremely cost-effective, costing around $6 per additional day of student attendance generated — more than one order of magnitude more cost-effective than one current best-practice intervention18. A key feature of the scal-
ability of this intervention is that it is also particularly easy to implement with fidelity in other school districts19. We find that the intervention reduced student absences comparably across all grade levels, and reduced absences among untreated cohabiting students in treated house- holds.
The intervention targets two biased beliefs held by par- ents of high-absence students: beliefs about total ab- sences and beliefs about relative absences. First, parents severely underestimate their children’s total absences (to- tal absences bias). A pilot survey of parents of high- absence students in our partner school district shows that parents underestimate their own students’ absences by a factor of two (9.6 estimated absences vs. 17.8 actual absences). Our experiment finds that providing total ab- sences information as part of the treatment reduces par- ents’ biased total absences belief, and nearly doubles the absence-reducing impact compared to similar treatment regimes that lack the total absences information. Second, parents are severely miscalibrated about how their chil- dren’s absences compare to those of their classmates (rel- ative absences bias). In the same pilot survey, only 28% of parents whose students have higher-than-average ab- sences accurately reported that their students had missed more school than their classmates. Our experiment finds that providing relative absences information as part of the treatment reduces parents’ biased relative absences belief. Providing this information in the way that we did, however, has no appreciable impact on student absences compared to similar messages without this information.
Parents’ beliefs about their children’s total absences may be inaccurate because bounded attention can make it challenging to sustain over time the attention needed to keep an accurate running tally of absences for an en- tire school year20,21. This may lead to parents being un- certain about their children’s summative total absences. Amidst this uncertainty parents may believe that their
1
Reducing student absences at scale by targeting parents’ misbeliefs.
students have missed far fewer total days of school than they actually have because they are motivated to hold fa- vorable views about their students. Since children can be central to parents’ own identities, biased total absences beliefs may benefit parents by allowing them to think more positively about themselves (i.e., “self-enhancement motive”)22. Logically, correcting parents’ biased beliefs about total absences will not necessarily lead to increased parent motivation to reduce student absences. The moti- vational effect of correcting this bias will depend on par- ents’ belief about whether the marginal cost to students of additional absences is increasing or decreasing. At the end of this paper, we report a simple survey experiment suggesting that parents do, in fact, believe that there are increasing marginal educational costs of incremental absences.
Parents’ beliefs about their children’s relative absences may be inaccurate because they have little direct expo- sure to when their children’s peers are absent, and also because of parents’ self-enhancement motives. Research has found that parents display a similar overconfidence as the residents in Garrison Keillor’s fictional town of Lake Wobegon in believing that “...all the students are above average”23. Our pilot survey suggests that this Lake Wobegon effect applies also to parents’ beliefs about how their children’s attendance compares to that of their peers23.
Informing parents about their children’s total absences
could be thought of as a form of personalized information
intervention. Other interventions delivering personalized
information have had sizable impacts on consequential
behaviors. For example, providing senior citizens with
price information about multiple prescription drug in-
surance options for which they are eligible can improve
the efficiency of insurance plan selection24. Notifying
inattentive cell phone subscribers with timely personal-
ized information when they have exceeded their allot-
ted usage changes subscribers’ phone usage behavior25.
Parents are particularly potent targets for interventions
that communicate personalized information developed to
change student behavior (like the total absences messag-
ing) for several reasons: parents are active investors in
their children’s human capital26, they can allocate re-
wards and punishments to students27, and there is evi-
dence that they have incomplete knowledge and bargain-
ing power with respect to their children’s human capital
investments28. In line with this reasoning, several infor-
mation interventions targeting parents have been found
to change student behaviors. Mailing information about
school quality to parents influences which schools stu-
dents enroll in and how well they subsequently perform
academically29. Recently, a range of personalized infor-
mation interventions have found that providing parents
with timely information about their students’ behaviors
and performance in school can increase student perfor- mance30,31,32.
Informing parents about their students’ relative ab- sences has less clear normative implications for affecting
their students’ subsequent absences. However, research across a wide range of policy areas has found that the dis- closure of social comparison information regarding con- sequential behaviors can result in conformity. This has been observed for charitable giving33,34, water and re- source conservation35 , 36 , 37 , energy conservation38 , 39 , 40 , job selection41 and motivation to participate in elec- tions42,43. At the end of this paper, we speculate as to why increasing parental accuracy regarding relative ab- sences by adding this information to the treatments does not result in the conformity this previous research might predict.
We conducted our experiment in partnership with the School District of Philadelphia (SDP), the eighth-largest school district in the United States. We conducted a pi- lot study in the spring prior to the launch of the main experiment. In brief, the pilot study assessed two main questions. First, does sending mailings to parents re- garding their children’s total absences indeed decrease absenteeism? Second, does including the absences of the typical student (relative comparison) lead to a greater decrease in absenteeism? We tested these questions by randomly assigning 3,007 households in the School Dis- trict of Philadelphia to one of three experimental con- ditions: Total Absences, Relative Absences, and Control. Those assigned to Total Absences and Relative Absences received two rounds of mail treatments in the spring 2014 semester. Both treatments reduced the number of ab- sences by about 0.7 days (6% relative to control) over a 14-week period. While both treatment conditions were significantly different from control, we were not able to distinguish whether the effect on those in the Total Ab- sences and Relative Absences conditions differed. See SOM for additional details.
The following year, we conducted our main experiment with an eligible population of 161,922 students across the entire district. The student population is racially diverse: 53% of enrolled students are Black/African American, 19% are Hispanic/Latino, and 14% are White/Caucasian (as of the 2013-2014 school year). Almost three-quarters of all SDP students qualify for Free or Reduced Price Lunch, and a third of all students in Philadelphia live in households below the Federal Poverty Level, making it the poorest major city in the United States. SDP has a budget of over $13,000 per student per year. Finally, 58% of all students scored “Below Basic” on the 2014- 2015 Math Pennsylvanian System of School Assessment (PSSA) exams.
In the summer preceding the 2014-2015 School Year (SY), we sent opt-out consent forms to the parents of all eligible students. After relevant exclusions (see “Meth- ods”), we were left with 40,326 eligible students in 32,437 households in our experiment universe. We randomly as- signed households in equal numbers to a control group or to one of three treatment conditions, with randomization stratified by school, grade, and prior-year absences (see SOM). Random assignment was balanced across covari- ates (see SOM).
2
Reducing student absences at scale by targeting parents’ misbeliefs.
Households assigned to the control group received no additional contact beyond normal school communications (e.g., report cards, school announcements, parent-teacher conferences; see SOM). Households assigned to treatment received up to five rounds of treatment mail through- out the school year. All treatments within each round were sent on the same day and have the same overall ap- pearance; the treatments differed only in their content, with treatments associated with each condition adding an additional piece of information. See Figure 1. Re- minder treatments reminded parents of the importance of absences and of their ability to influence them. To- tal Absences treatments added information about stu- dents’ total absences. Relative Absences treatments fur- ther added information about the modal number of ab- sences among focal students’ classmates. Data reported in the first treatment, mailed in October 2014, reflected absences from the previous school year. Data reported in the remaining treatments, mailed January 2015 — May 2015, reflected current-year absences. The total cost of the treatment was around $6.60 per household for pro- duction and labor (see SOM).
Not all parents assigned to the treatment conditions received all of the five treatment mailings, or the same regime of mailings over the course of the year. First, we were unable to send treatments to parents who moved during the school year without leaving valid forwarding information. Second, when student absences were espe- cially low — either overall or compared to their class- mates — parents received the most informative treat- ment the district permitted for that round (see SOM). On average, we sent treatment condition households 4.2 mailings over the school year (Reminder=4.24; Total Ab- sences =4.21; Relative Absences =4.18). As we discuss next, we therefore based our analysis on random as- signment to treatment condition (i.e., Intent-to-Treat), rather than the number of treatment rounds received.
This experiment evaluates the effectiveness of using parental engagement to improve student attendance. We address three main research questions:
1. Does contacting guardians and encouraging them to improve their child’s attendance reduce ab- sences?
2. Does communicating to guardians the total number of days their child missed reduce absences?
3. Does communicating to guardians the total num- ber of days their child missed as compared to the absences of a typical student reduce absences?
We also address these exploratory research questions:
4. Do these interventions impact the attendance of other students in the household not explicitly men- tioned in the mailings?
5. Do the treatment effects differ for students in early grade-levels (Kindergarten through 5th grades)
compared to later grade-levels (6th through 12th grades)?
At the end of the school year, between June 20, 2015 and June 25, 2015, we surveyed parents to assess whether assignment to treatment condition also affected parent beliefs (survey N=1,268; American Association for Public Opinion Research Response Rate 2 of 23.0%). The sur- vey had two primary purposes. First, we wanted to assess the internal validity of the study and include several ma- nipulation checks. To this end, the survey included a set of questions to address whether the guardians received, read, and understood the mail. Second, we wanted to assess the impact of the treatments in parent beliefs. To this end, the survey included a set of questions about the importance of attendance and parental beliefs about their role in ensuring that their children got to school.
A secondary purpose of the survey was to assess the impact of the treatments on parental behavior. Because we surveyed a minority of the experimental universe, the responses are informative of the mechanisms underlying the experimental treatment effects, but are not conclusive evidence of the mechanisms. The full survey and the survey analysis plan are included in the SOM.
We find that random assignment to treatment signif- icantly reduced student absences relative to the Con- trol group (joint FRT p<0.001). Students in the Con- trol group were absent 17.0 days on average (all means regression-adjusted; SE=0.1 days); students in the Re- minder condition were absent 16.4 days on average (SE=0.1 days); students in the Total Absences condi- tion were absent 16.0 days on average (SE=0.1 days); and students in the Relative Absences condition were ab- sent 15.9 days on average (SE=0.1 days). Therefore, the ATE for the Reminder condition relative to the Con- trol group was -0.6 days (FRT p<0.001). Adding ab- solute absences information nearly doubled this effect: the ATE for the Total Absences condition relative to the Control group was -1.1 days (FRT p<0.001), and rel- ative to the Reminder condition it was -0.4 days (FRT p<0.001). However, adding relative absences information did not affect student absences: absences among those in the Relative Absences condition were nearly identical to those in the Total Absences condition (ATE=0.0 days compared to Total Absences, FRT p=0.19). See Figure 2. We found a similar pattern for chronic absenteeism: 36.0% of students in the Control group were chronically absent (SE=0.5pp), compared to 33.0% in the Reminder condition (SE=0.5pp), 32.4% in the Total Absences con- dition (SE=0.5pp), and 31.9% in the Relative Absences condition (SE=0.5pp). Thus, compared to students in the Control group, students in the Reminder condition were 8% (or 3pp) less likely to be chronically absent, students in the Total Absences condition were 10% (or 3.6pp) less likely to be chronically absent, and students in the Relative Absences condition were 11% (or 4.1pp) less likely to be chronically absent.
We used the fact that the focal student was randomly assigned to assess spillover in households with two or
3
Reducing student absences at scale by targeting parents’ misbeliefs.
Figure 1. Sample mailings from each treatment condition. The Reminder treatment (a) reminds parents of the
importance of absences and of their ability to influence them. The Total Absences treatment (b) builds on (a) and adds information about students’ total absences. The Relative Absences treatment (c) builds on (b) and adds information about the modal number of absences among focal students’ classmates. Focal students were randomly assigned to one of these three treatment conditions or a control group (N=6,994 Control; 7,041 Reminder; 7,037 Total Absences; 7,008 Relative Absences). See SOM for more information on the randomization strategy.
more qualifying students (N=5,028 households). Among non-focal students in households assigned to the Re- minder condition, there was no evidence of spillover ef- fects (ATE=0.0 days; SE=0.4 days). Among non-focal students in households assigned to the Total Absences and Relative Absences conditions, spillover effects were nearly as large as the effects for focal students (Total Ab- sences : ATE=-1.0 days, SE=0.4 days; Relative Absences : ATE=-1.0 days, SE=0.4 days).
Daily attendance data allowed us to examine the im- pact of the treatments over time. Across all three treat- ment conditions, we find modest evidence that the impact was larger in the week immediately following delivery of each treatment round compared with the two subsequent weeks (Reminder: 0.18 v. 0.09 days/week, p=.05; Total Absences: 0.25 v. 0.16 days/week, p=.04; Relative Ab- sences: 0.24 v. 0.18 days/week, p=.12; all comparisons versus Control). This action-and-backsliding pattern is similar to that observed in other repeated, personalized interventions38 . Additionally, across the treatment con- ditions, the first round of treatment tends to have a non- significantly smaller effect than the average of the sub- sequent rounds. For example, the first treatment round in the Relative Absences condition reduced absenteeism by 0.22 days, and the average of the remaining treat- ments in the condition reduced absenteeism by 0.28 days per treatment round (see SOM). Though this is not sta- tistically robust evidence of increasing impact per treat-
ment, it is consistent with an interpretation that effects do not decrease with each successive round of treatment (e.g., that parents do not appear to be habituating to treatments). This suggests that increased frequency of communication to parents about absenteeism may reduce absenteeism even more. Additionally, one intriguing pos- sible mechanism for the lack of habituation to repeated rounds of treatment is that the first treatment may have alerted parents to the possibility that they may be com- municated with again during the school year specifically regarding their children’s absenteeism. This could create a heightened sense in parents that their children’s absen- teeism was being carefully monitored and thus generate increased effort to reduce absenteeism44 .
We found no evidence of meaningful treatment effect variation by student grade-level, though statistical power was limited. If such variation was indeed minimal, this suggests that the treatment effect did not result from informing parents that their children had been cutting school. After all, 18 year-old seniors in high school are far more likely to covertly cut school than 7 year-old first graders, yet both age groups show comparable ef- fect sizes. We found no evidence of meaningful treat- ment effect variation by gender, race, or total absences in the previous school year. However, as discussed in the SOM, we found meaningful variation in quantile treat- ment effects. This approach compared a given quan- tile for students assigned to the control group (e.g., the
4
Reducing student absences at scale by targeting parents’ misbeliefs.
median) with the corresponding quantile of students as- signed to treatment (in this case, we pooled the treat- ment conditions). In particular, we found a quantile treatment effect of around 1 day at the median of each group (around 3 weeks absent for students in Control) compared with a quantile treatment effect of around 0.5 days at the 10th percentile by absences of each group (around 1 week absent for students in Control). Esti- mates at much higher quantiles are imprecise. While inherently exploratory, these results suggest that there is indeed meaningful treatment effect heterogeneity not captured by pre-treatment covariates.
Finally, we explored the impact of the intervention on end-of-year standardized test scores for students in grades 4 through 8. Unfortunately, we were severely sta- tistically under-powered to detect realistic impacts. As a benchmark, several recent studies have tried to quan- tify the impact of attending an additional school day on test scores. Using within-sibling comparisons with data from SDP, Gottfried (2011)7 found that an addi- tional day is associated with a roughly 0.01 SD increase in test scores. Using ECLS-K and administrative data from North Carolina, Gershenson et al. (2017)4 found effects about half this size for comparable populations6. For the group for which we have end-of-year test scores, the pooled impact on attendance through the test date was 0.6 days (SE=0.1 days); thus, we would realistically expect test score impacts on the order of 0.001 to 0.006 SDs. The experiment was dramatically underpowered to study effects of this size, with a minimal detectable effect of around 0.03 SDs. Given the importance of stan- dardized test scores, however, we nonetheless assessed the impact on this outcome, pre-registering our concerns about statistical power. Unsurprisingly, we found no sig- nificant effect on end-of-year standardized test scores for students in grades 4 through 8 (for pooled treatments, Math ATE=-0.001 SD, SE=0.012 SD; Reading ATE=- 0.015 SD, SE=0.012 SD).
The end-of-year survey confirmed that parents ac- tually received and remembered the treatments: 57% (SE=2pp) in the three treatment conditions recalled re- ceiving the treatments compared to 26% (SE=3pp) in Control (p<0.001). The survey also showed that assign- ment to the Reminder condition did not change parents’ reports of the importance of absences or parents’ role in reducing absences. This suggests that the Reminder treatments primarily focused parents’ attention on ab- sences45, but did not affect their relevant beliefs; par- ents’ attitudes about attendance across seven questions did not differ across conditions (F-test p=0.48).
We next examined whether informing parents of their children’s total number of absences corrected parents’ bi- ased beliefs about these absences. Parents’ total absences bias was calculated as the difference between parents’ self-reported absences and their students’ actual absences (this pattern holds across different measures as well). See Figure 3. Informing parents of their children’s total ab- sences indeed corrects this bias: parents in Control and
Figure 2. Absences by treatment condition. The average number of absences is compared for each treatment group. Bars show regression-adjusted means with error bars +/-1 SE. The joint FRT p-value for the null hypothesis of no impact is p<0.001. The ATE for the Reminder condition relative to the Control group is -0.6 days, while the ATE for the Total Absences condition relative to the Control group is -1.1 days. Absences among those in the Relative Absences condition were nearly identical to those in the Total Absences condition. At a cost of around $6 per incremental day of attendance, this makes the Total Absences treatment an order of magnitude more cost-effective than current absence-reduction best practices.
the Reminder condition under-reported their children’s absences by 6.1 days (SE=0.6 days), roughly 50% more than parents in the Total Absences and Relative Absences conditions (2.8 days; SE=0.6 days; ATE=-3.2; SE=0.9). Adding total absences information to the treatments re- duced parents’ biased beliefs and reduced absences, sug- gesting that parents’ total absences bias inhibits them from reducing actual student absences. Adding total ab- sences information may have also increased the amount of attention people devoted to the treatments, amplify- ing the cognitive accessibility and perceived importance of student absences. Though we cannot fully rule out that interpretation, we note that the change in parent beliefs is aligned with the proposed parent belief mecha- nism.
Finally, we assessed whether providing parents with in- formation about typical absences corrected parents’ bi- ased beliefs about their children’s relative absences. Par- ents’ relative absences bias was calculated by asking par- ents whether their students were absent “more,” “about the same,” or “fewer” days than their children’s typi- cal classmates (this pattern holds across different mea- sures). Among parents of students in Control, the Re- minder condition, and the Total Absences condition, 9.2% (SE=1pp) responded correctly, compared to 16.2%
5
Reducing student absences at scale by targeting parents’ misbeliefs.
(SE=2pp) among parents of students in the Relative Ab- sences condition [ATE=7.1pp, p=0.001]. See Figure 3. Adding relative absences information to the treatments corrected parents’ relative absences bias, but did not af- fect actual student absences. This suggests that par- ents’ biased beliefs about their children’s relative ab- sences does not inhibit parents from reducing actual stu- dent absences.
This experiment makes three primary contributions. First, it develops and evaluates a cost-effective and scal- able intervention that addresses a critical social problem. Second, it suggests that correcting parents’ biased be- lief about how many total absences their children have accumulated causes parents to reduce student absences. Third, it suggests that using extreme social comparisons to correct parents’ biased belief about how their chil- dren’s absences compare to their children’s classmates’ absences does not cause an appreciable change in stu- dent absences.
Not attending school negatively affects student, school, school district, and national success. Yet, little research has validated interventions that can be implemented at scale with fidelity relatively inexpensively. The inter- vention reported here is both highly scalable and cost- effective at reducing at-risk students’ absences, costing around $6 per incremental school day generated. We fo- cus on assessing the intervention’s costs per additional student day rather than assessing costs versus benefits because we believe that student attendance is of intrinsic importance. As a practical matter, however, quantify- ing the impact of an additional student day is also quite challenging, due largely to data limitations. Fortunately, there is an active literature seeking to address this prob- lem4; we hope that future research can shed additional light on this.
One common best practice for reducing absenteeism — absence-focused mentors — has been found to have sub- stantially higher costs per student-day than the interven- tion reported in this manuscript. The only randomized controlled experiment evaluating this absence-reduction strategy involved a very similar at-risk population as the one studied in this manuscript (see SOM for discussion of population similarities). Students in Chicago Public Schools grades 1 through 8 (N=1,972) were randomly as- signed a mentor17. Mentors reduced absenteeism among participating students in grades 5 through 7 by 3.4 days, and had no statistically significant effect on students in grades 1 through 4. The mentors did not affect students’ GPA, but appeared to reduce failed courses among the subgroup of students in grades 5 through 7. The men- tors cost around $1,700 per year per treated student, leading the researchers to estimate that mentors cost around $500 per additional day of attendance generated for students in grades 5 through 7. (See also Balfanz & Byrnes (2013)46 for a quasi-experimental evaluation of absence-focused mentors targeting a population that also includes students with more extreme absenteeism.) The only other absenteeism intervention strategy that
Figure 3. Treatments corrected parents’ biased beliefs. Regression-adjusted means and standard errors based on end-of-experiment survey responses; error bars +/-1 SE; orange bars represent treatment conditions that included the relevant information. Informing parents of their students’ total absences corrected their beliefs about their students’ total number of absences (a). When asked at the end of the year to report the number of absences, parents in the Total Absences and Relative Absences treatment conditions, both of which included information on students’ total absences, were roughly 50% more accurate than parents in the Control and Reminder treatment conditions. Adding relative absences information to the treatments in the Relative Absences condition corrected parents’ biased beliefs around relative absences (b). About 16% of parents in the Relative Absence condition responded correctly when asked how their student compared to his/her typical classmates.
has been assessed with a randomized controlled exper- iment involved absence-focused SMS text messages; it produced no measurable impacts on absenteeism47.
We note that despite its cost effectiveness, the mail- based intervention reported in this manuscript is not a substitute for more intensive approaches that address the
6
Reducing student absences at scale by targeting parents’ misbeliefs.
deep personal and structural challenges facing students, families, and communities. After all, this intervention reduces chronic absenteeism by around 10% and cannot reach all at-risk students. No single intervention is a panacea; rather system-level change will require many such interventions woven together. By harnessing the intervention we report, schools can better target educa- tional resources and personnel towards difficult absen- teeism challenges that require more active and personal involvement.
One possible explanation for the additive impact of providing total absences information is that parents be- lieve that there are increasing repercussions for every ad- ditional day of school missed. In other words, parents appear to believe that the marginal educational cost of absences is increasing. We conducted an online survey experiment to examine this further. Parents of students in grades kindergarten through twelfth grade recruited on Amazon’s Mechanical Turk (N=255) were randomly assigned to one of two conditions. Half were asked to imagine that their child had been absent six days as of about half way through the school year, and the other half were asked to imagine that their child had been ab- sent twelve days as of halfway through the school year. They were all asked “How much would being absent from school tomorrow affect your child’s success in school this school year? ” Parents who imagined that their child had accumulated relatively many absences reported that be- ing absent tomorrow would more negatively affect their child’s success than did parents who imagined that their child had accumulated relatively few absences, t253 = -4.33, p=.002. (See SOM.) This is consistent with an interpretation that the Total Absences result arises be- cause parents believe there are increasing marginal costs for each additional absence their child accumulates.
In our pre-registered analysis plan, we predicted that adding the personalized social comparison information to the treatment would reduce student absences com- pared to not including that information. This predic- tion was informed by the impact social comparisons have had in other domains. In fact, the Relative Absences treatment was modeled after the robust and widely stud- ied OPOWER home energy report intervention, the cen- tral feature of which is personalized social comparison information38 , 48 . The content of the home energy re- ports is informed by social psychology research showing that personalized social comparison information added to personalized energy use information resulted in en- ergy use conformity40. While the evaluations of the spe- cific OPOWER intervention have not isolated the mo- tivational impact of the personalized social comparison information, other research has. For example, related research has shown that adding personalized social com- parison information to personalized water use informa- tion resulted in water use conformity relative to just per- sonalized water use information35.
There are many possible reasons that correcting rela- tive absences bias did not result in additional reduction
in student absences. For example, perhaps the average gap between students’ actual absences and their peers’ absences was so large that it discouraged parents49 , 50 . Across all rounds of treatment for those assigned to the Relative Absences condition, the average ratio of own- student absences to comparison-student absences was 5 to 1. It is conceivable that this gap seemed insurmount- able and so discouraged parents. Or, perhaps relative comparisons tend to be less motivating in domains that are especially identity-central (e.g., parental support of education) because they elicit especially strong counter- arguing and rationalization. Or, perhaps the treatment was simply too weak. Figure 3 shows that the relative absences information increased by 50% the fraction of parents who accurately reported that their students’ had missed more school than their classmates — and yet the vast majority of parents still displayed relative absences bias. We hope future research will explore why reduc- ing relative absences bias failed to result in additional parental motivation to reduce their children’s absences.
The treatment effects were nearly as large on other students living in the targeted households as they were on the focal students. This suggests that analyses of household-level interventions that do not incorporate intra-household spillover effects dramatically underesti- mate intervention cost effectiveness51. It could be that this spillover arose from students directly influencing other students within their households, or from the inter- ventions motivating parents to influence the absenteeism of all students within their households. We cannot de- termine the mechanism from the current study, though follow up research could tease these apart.
This research examined absenteeism among students, though absenteeism is an important challenge for most organizations. Employee absenteeism in the US is esti- mated to cost organizations $202 billion each year52. Un- doubtedly, the specific targeting and content of absence- reducing interventions will differ when directed at em- ployee absences53, though the research we report could provide useful insight for a program of work on this topic.
Additionally, this research examined a personalized in- formation intervention aimed at changing focal individu- als’ behaviors by communicating to influential third par- ties — in this case, parents. For sensible reasons, the vast majority of research on information interventions targets the focal individuals directly. However, influen- tial third parties are common in the world. They exist in workplaces (e.g., managers can influence employees), in healthcare settings (e.g., doctors can influence patients), in personal finance settings (e.g., financial advisors can influence investors), and within households (e.g., spouses can influence each other). The present research suggests that targeting influential individuals may be a particu- larly promising strategy for behavior change.
More research is also needed on the general interven- tion approach studied in this manuscript. Though rig- orously examined and replicated with the pilot study re- ported in the SOM, this intervention targeted just two
7
Reducing student absences at scale by targeting parents’ misbeliefs.
biased parent beliefs, when there are many possible be- liefs to target. It targeted a specific sample of students in one large urban district, when there are many diverse student samples that merit study. We look forward to future research exploring other biased parent beliefs and broadening the sample frame.
In conclusion, parents of low-income and minority stu- dents are often seen as a contributing cause of student failure54,55. As we see it, this “deficit” view of parents hinders educational innovation, especially for students in Kindergarten through 12th grade. The intervention we report here shows that an “asset” view of parents can un- lock new interventions that empower parents as partners in improving student outcomes30,31,32,56.
METHODS
Sample selection. We used administrative records from the School District of Philadelphia in the summer preceding the 2014-2015 School Year (SY) to identify all students enrolled in a public school in the district. This yielded an initial study universe of 161,922 students in 103,408 households. On August 15, 2014, we sent opt- out forms to the parents of these students. In 146 (0.1%) households, the consent forms bounced back as being bad addresses, and 923 (0.9%) households opted out of the study. After excluding opt-outs and bounce backs, we determined our initial sample on the basis of the follow- ing criteria. We excluded students who enrolled in “non- regular status” schools, defined as any public schools in the district with a special designation, such as special- ized schools (like schools for the blind), alternative edu- cation schools, charter schools, and online schools. We excluded students who were in 12th grade at the end of the 2013-2014 SY or whose administrative records in- dicated that the student had graduated, withdrawn, or was otherwise not enrolled as of June 2014. We excluded students who were currently enrolled in the schools used for the pilot study in Spring 2014, as well as any stu- dents who were enrolled in one of these schools at the time of the pilot study but who subsequently enrolled in a different school. We excluded students for whom ad- ministrative records indicated that (1) the student was not stably housed (i.e., homeless) or that (2) the student had an Individual Education Plan (i.e., learning disabil- ity). For students experiencing homelessness, we were concerned about the reliability of their addresses. Stu- dents with IEPs were excluded as these students were more likely to have many justifiable absences and we did not want to interfere with other conversations that these students and their parents were having with the district and schools. We excluded students whose indicated home language was not one of our consent form languages (En- glish, Albanian, Arabic, Mandarin, French, Khmer, Rus- sian, Spanish, or Vietnamese) or whose home language was missing in administrative records. This was an ethi- cal concern related to human subjects research because it
meant that the informed consent mailing may not have been understood, and thus parents may not have had the opportunity to opt-out of the study. Finally, we ex- cluded all students who had a perfect attendance record in the previous school year because of high year-over- year within student correlation in absenteeism, which we found to be roughly r =0.6.
We further narrowed the sample universe as follows. We then selected all students whose 2013-2014 SY ab- sences met the following criteria. First, we selected stu- dents who were absent 3 or more days more than their typical classmate (defined as the modal student in the participating student’s school-grade — e.g., the modal student among all 11th graders at Washington High School). We anticipated that students with very good attendance might not show meaningful treatment effects due to ceiling effects. If there were multiple modes, we selected the minimum mode. More individual students were clustered around the minimum mode as compared to the maximum mode, so we concluded that the min- mode was a better representation of the “typical” student in a class. And second, we selected students whose ab- sences did not exceed 2 standard deviations more than the mean student in their school-grade in 2013-2014 SY. We excluded students who had missed over 2 s.d. more days than the mean student in their school-grade for two reasons: (1) in consultation with the district, we deter- mined that many of these students likely no longer lived in the district, but had failed to inform the district, and (2) we hypothesized that these students were more likely than others to be experiencing a grave challenge that might make them less treatment responsive (e.g., an un- documented health or family event). In households with multiple qualifying students (19%), we randomly selected one student from the household to be the focal student, with treatment messages focused on this student. Before post-randomization exclusions, this yielded 32,437 focal students from the 40,326 eligible students in our main study universe.
We also made the following exclusions after random- ization. We excluded all students in households that were randomly assigned to a condition, but in which the fo- cal student transferred out of the district before the first mail treatment was sent (N=1,242 students). We then excluded all students in a household if the focal student had missing or incomplete attendance data in the 2014- 2015 SY (N=2,166 students); we also excluded all stu- dents in a household if a sibling of a focal student was missing outcome data in the 2014-2015 SY (N=358). Fi- nally, we excluded all students in a household if the focal student transferred outside the district between the time we sent the first mailing and the end of the school year (N=1,896) (i.e., students who were lost to follow up); we also excluded all students in a household if a sib- ling of a focal student left the district (N=203). These post-randomization “attrition rates” did not differ across condition (χ2 p=0.75). In the SOM, we show that im- pacts are nearly identical under a Missing at Random
8
Reducing student absences at scale by targeting parents’ misbeliefs.
assumption (Table S7). The final student sample is 53% African American, 20% Hispanic, 52% female, 28% in high school, and 74% free or reduced-price lunch quali- fied. The final sample consists of 28,080 households (con- taining 34,461 eligible students) across 203 schools. The 28,080 focal students are the focus of the primary analy- ses presented below.
Table 1: Sample Size of Experiment Universe after Exclusions
from the date of the first mailing through the end of the school year. This outcome includes both excused and unexcused absences; the results are consistent examin- ing these outcomes separately. As discussed in the SOM, secondary outcomes include standardized test scores and number of tardies. We use the following demographic control variables: student gender, whether the student has Low English Proficiency (LEP), speaks English as the primary home language, is eligible for Free and Re- duced Price Lunch, or is Black/African American. We also control for the number of days absent in the prior school year and in the current school year prior to ran- domization. Finally, we control for school and grade (i.e., fixed effects) unless otherwise stated. As a practical mat- ter, the data quality is excellent overall, with minimal missingness. We address this and sample attrition in more detail in the SOM.
The distribution of baseline absences in our final ex- perimental universe was similar to that of the consent universe (see SOM). Among the 110,299 students in the consent universe for whom we have baseline data, the average number of absences in 2013-2014 was 15.3 days (SD=17.7 days) with a minimum of zero and a maximum of 171 days absent. The average number of baseline ab- sences in our final experimental universe was 16.3 days (SD=10.4 days), with a minimum of three and a maxi- mum of 97. The tighter distribution of absences in our experimental universe is by design. First, we excluded high absence outliers, students who were absent more than 2 standard deviations above the mean student in their school-grade, which represented less than 1% of the consent universe. Second, we excluded students who had prior absences within three days (or fewer) of the modal number of absences for their specific school-grade, which represented about 15% of the consent universe.
Analysis. Prior to obtaining any information on outcomes, we pre-registered a detailed analysis plan (#AEARCTR-0000829, www.socialscienceregistry.org). The SOM provides extensive details on the analysis methods. We assess the impact of random assignment on student attendance in two ways. First, we use Fisher Randomization Tests (FRT) to obtain exact p-values for the sharp null hypothesis of no impact (Rosenbaum, 2002). This is a non-parametric approach that is fully justified by the randomization itself. Second, we use lin- ear regression to estimate the Average Treatment Effects (ATE) of random assignment to each treatment condi- tion, with covariate adjustment for student-level demo- graphics and prior absences as well as the student’s school and grade. The SOM provides additional details on the procedure for multiple test correction.
Ethical procedures. The consent procedures, study design and protocols, and all treatment materials for the pilot, main experiment, and phone surveys were approved by Harvard University’s Committee on the Use of Human Subjects (IRB13-2911; IRB14-2747; IRB14-4651).
Exclusion criterion
All students
Focal students
Initial eligible universe
161,922
Pre-randomization
Consented (did not opt-out)
160,413
School type
104,088
Likely enrolled in 2014-2015 SY
95,419
Pilot study schools
90,430
Student characteristics
76,167
Home language for consent form
71,205
Perfect attendance in 2013-2014 SY
69,387
Low absenteeism
42,914
Extremely high absenteeism
40,384
Students in overly large households
40,326
One focal student per household [randomly selected]
32,437
Post-randomization
Transferred out of district before first treatment
39,084
Missing or incomplete 2014-2015 SY attendance data
36,560
Transferred out of district between beginning and end of experiment 2014-2015 SY
34,461
Final number of focal students
28,080
Data. We obtained all of our student- and household- level data from school district administrative records. The primary outcome is the total number of absences
9
Reducing student absences at scale by targeting parents’ misbeliefs. REFERENCES
Code availability. All final analyses were conducted using StataSE and R. Code used to generate the results presented in the paper and Supplementary Information are available from the corresponding author upon reason- able request and with the permission of SDP.
Data availability. Restrictions apply to the availability of the data used to support the findings of this study, and so they are not publicly available. Data may be available from the authors upon reasonable request and with the permission of SDP after a formal review by the District’s Research Review Committee.
Acknowledgements. We thank the Laura and John Arnold Foundation, Pershing Square Venture Fund for Research on the Foundations of Human Behavior, and IES/ICF/REL MidAtlantic #14JTSK0003 for funding support. We thank Jessica Lasky-Fink, John Ternovski, and Shruthi Subramanyam for research support. We thank Tonya Wolford, Adrienne Reitano and William Hite for district partnership and collaboration. We thank Bob Balfanz, Guillaume Basse, Max Bazerman, Peter Bergman, Hedy Chang, Luke Coffman, David Deming, Craig Fox, Hunter Gehlbach, Alex Gelber, Francesca Gino, Ed Glaeser, Michael Gottfried, Don Green, Hilary Hoynes, Leslie John, Gary King, David Laibson, Marc Laitin, Sendhil Mullainathan, Mike Nor- ton, Lindsay Page, Lamar Pierce, Sean Reardon, and Josh Schwartzstein for feedback on earlier drafts.
Author contributions. TR and AF designed the experiment, oversaw data analysis, and wrote this manuscript.
Competing interests. TR and AF had no compet- ing financial interests while this project was being con- ducted. In light of the results of this and other projects TR and AF started an organization to help US schools implement this intervention to reduce student absen- teeism. It is called In Class Today and worked with two school districts at the time of initial submission, includ- ing the school district in which the experiment reported in this manuscript was conducted, The School District of Philadelphia.
Correspondence. Correspondence should be addressed to Todd Rogers (email: todd rogers@hks.harvard.edu).
Supplementary information is available for this pa- per.
REFERENCES
1Balfanz, R. & Byrnes, V. The importance of being there: A report on absenteeism in the nation’s public schools. Bal- timore, MD: Johns Hopkins University School of Education, Everyone Graduates Center, 1-46 (2012).
2Gottfried, M. A. Evaluating the relationship between stu- dent attendance and achievement in urban elementary and middle schools: An instrumental variables approach. Ameri- can Educational Research Journal 47(2), 434-465 (2010).
3Nauer, K., Mader, N., Robinson, G. & Jacobs, T. A better picture of poverty: What chronic absenteeism and risk load reveal about NYC’s lowest income elementary schools. New York: Center for New York City Affairs (2014).
4Gershenson, S., Jacknowitz, A. & Brannegan, A. Are stu- dent absences worth the worry in US primary schools? Edu- cation Finance and Policy 12(2), 137-165 (2017).
5Allensworth, E. M. & Easton, J. Q. What matters for staying on-track and graduating in Chicago public highs schools: A close look at course grades, failures, and atten- dance in the freshman year. Chicago: Consortium on Chicago School Research (2007).
6Goodman, J. Flaking out: Student absences and snow days as disruptions of instructional time Working Paper No. 20221 (National Bureau of Economic Research, 2014)
7Gottfried, M. A. The detrimental effects of missing school: Evidence from urban siblings. American Journal of Education 117(2), 147-182 (2011).
8Byrnes, V. & Reyna, R. Summary of state level analysis of early warning indicators. Baltimore, MD: Johns Hopkins University School of Education, Everyone Graduates Center (2012).
9Schoeneberger, J. Longitudinal attendance patterns: de- veloping high school dropouts. The Clearinghouse: A Jour- nal of Educational Strategies, Issues and Ideas 85(1), 7-14 (2012).
10Henry, K. L., & Thornberry, T. P. Truancy and escalation of substance use during adolescence. Journal of Studies on Alcohol and Drugs 71(1), 115-124 (2010).
11Baker, M. L., Sigmon, J. N. & Nugent, M. E. Truancy reduction: Keeping students in school. Juvenile Justice Bul- letin (2001).
12Jacob, B. A. & Lefgren, L. Are idle hands the devil’s workshop? Incapacitation, concentration, and juvenile crime. The American Economic Review 93(5), 1560-1577 (2003).
13Rohrman, D. Combating truancy in our schools — a com- munity effort. NASSP Bulletin 77(549), 40-45 (1993).
14Ely, T. L. & Fermanich, M. L. Learning to count: School finance formula count methods and attendance-related stu- dent outcomes. Journal of Education Finance 38(4), 343-369 (2013).
15Every Child Succeeds Act of 2015. (2015). Pub. L. 114- 95.
16Lynch L., Burwell S., Castro J. & Duncan A. Joint letter on chronic absenteeism. (2015). Retrieved from http://www2.ed.gov/policy/elsec/guid/secletter/151007.html
17Guryan et al. The effect of mentoring on school atten- dance and academic outcomes: a randomized evaluation of the Check & Connect program Working Paper-16-18 (North- western University Institute for Policy Research Working Pa- per Series, 2017).
18Sutphen, R. D., Ford, J. P. & Flaherty, C. Truancy in- terventions: A review of the research literature. Research on
10
Reducing student absences at scale by targeting parents’ misbeliefs. REFERENCES
Social Work Practice 20(2), 161-171 (2010).
19O’Donnell, C. L. Defining, conceptualizing, and measur-
ing fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educa- tional Research 78(1), 33-84 (2008).
20Chugh, D. & Bazerman, M. H. Bounded awareness: What you fail to see can hurt you. Mind & Society 6(1), 1-18 (2007).
21Simons, D. J. & Chabris, C. F. Gorillas in our midst: Sus- tained inattentional blindness for dynamic events. Perception 28(9), 1059-1074 (1999).
22Sedikides, C., Gaertner, L. & Toguchi, Y. Pancultural self-enhancement. Journal of Personality and Social Psychol- ogy 84(1), 60 (2003).
23Lee, J. Y. Garrison Keillor: A Voice of America (Uni- versity Press of Mississippi, 1991).
24Kling, J. R., Mullainathan, S., Shafir, E., Vermeulen, L. & Wrobel, M. Comparison friction: Experimental evidence from Medicare drug plans Working Paper No. 17410 (Na- tional Bureau of Economic Research, 2012)
25Grubb, M.D., & Osborne, M. Cellular service demand: Biased beliefs, learning, and bill shock. American Economic Review 105(1), 234-271 (2015).
26Becker, G. S. A theory of social interactions. Journal of Political Economy 82(6), 1063-1093 (1974).
27Heckman, J. J. & Mosso, S. The economics of human de- velopment and social mobility Working Paper 19925 (National Bureau of Economic Research, 2014).
28Bursztyn, L. & Coffman, L. C. The school decision: Fam- ily preferences, intergenerational conflict, and moral hazard in the Brazilian favelas. Journal of Political Economy 120(3), 359-397 (2011).
29 Hastings, J.S., Weinstein, J.M. Information, school choice, and academic achievement: Evidence from two ex- periments Working Paper No. 13623. National Bureau of Economic Research, 2007)
30 Bergman, P. Parent-child information frictions and
human capital investment: Evidence from a field ex-
periment. New York: Columbia University (2015).
http://www.columbia.edu/ ̃psb2101/BergmanSubmission.pdf
31
32Kraft, M. A. & Rogers, T. Teacher-to-parent communi- cation: Experimental evidence from a low- cost communica- tion policy. Society for Research on Educational Effectiveness (2014).
33Frey, B. S. & Meier, S. Social comparisons and pro-social behavior: Testing “conditional cooperation” in a field exper- iment. The American Economic Review 94(5), 1717-1722 (2004).
34Shang, J. & Croson, R. A field experiment in charita- ble contribution: The impact of social information on the voluntary provision of public goods. The Economic Journal 119(540), 1422-1439 (2009).
35Ferraro, P. J., Miranda, J. J. Price, M. K. The persistence of treatment effects with norm-based policy instruments: ev- idence from a randomized environmental policy experiment. The American Economic Review 101(3), 318-322 (2011).
36Ferraro, P. J. & Price, M. K. Using nonpecuniary strate- gies to influence behavior: evidence from a large-scale field experiment. Review of Economics and Statistics 95(1), 64- 73 (2013).
37Goldstein, N.J. & Cialdini, R.B. Normative influences on consumption and conservation behaviors. Social Psychology of Consumer Behavior 273-296 (2009).
38Allcott, H. & Rogers, T. The short-run and long-run effects of behavioral interventions: Experimental evidence from energy conservation. The American Economic Review 104(10), 3003-3037 (2014).
39Nolan, J. M., Schultz, P. W., Cialdini, R. B., Goldstein, N. J. & Griskevicius, V. Normative social influence is under- detected. Personality and Social Psychology Bulletin 34(7), 913-923 (2008).
40Schultz, P. W., Nolan, J. M., Cialdini, R. B., Goldstein, N. J. & Griskevicius, V. The constructive, destructive, and reconstructive power of social norms. Psychological Science 18(5), 429-434 (2007).
41Coffman, L. C., Featherstone, C. R. & Kessler, J. B. Can social information affect what job you choose and keep? American Economic Journal: Applied Economics 9(1), 96- 117 (2017).
42 Gerber, A. S. & Rogers, T. Descriptive social norms and motivation to vote: Everybody’s voting and so should you. The Journal of Politics 71(1), 178-191 (2009).
43Keane, L. D. & Nickerson, D. W. When reports depress rather than inspire: A field experiment using age cohorts as reference groups. Journal of Political Marketing 14(4), 381- 390 (2015).
44Rogers, T., Green, D. P., Ternovski, J. & Young, C. F. Social pressure and voting: A field experiment conducted in a high-salience election. Electoral Studies 46, 87-100 (2017).
45Karlan, D., McConnell, M., Mullainathan, S. & Zinman, J. Getting to the top of mind: How reminders increase saving. Management Science 62(12), 3393-3411 (2016).
46Balfanz, R. & Byrnes, V. Meeting the challenge of com- bating chronic absenteeism. Baltimore, MD: Johns Hopkins University School of Education, Everyone Graduates Center (2013)
47Balu, R., Porter, K. & Gunton, B. Can informing parents help high school students show up for school? mdrc, 1-10 (2016).
48Allcott, H. Social norms and energy conservation. Jour- nal of Public Economics 95(9-10), 1082-1095 (2011).
49Rogers, T. & Feller, A. Discouraged by peer excellence: Exposure to exemplary peer performance causes quitting. Psychological Science 27(3), 365-374 (2016).
50Beshears, J., Choi, J. J., Laibson, D., Madrian, B. C. & Milkman, K. L. The effect of providing peer information on retirement savings decisions. The Journal of Finance 70(3), 1161- 1201 (2015).
51Nickerson, D. W. Is voting contagious? Evidence from two field experiments. American Political Science Review 102(01), 49-57 (2008).
52Goetzel, R. Z., Hawkins, K., Ozminkowski, R. J. & Wang, S. The health and productivity cost burden of the “top 10” physical and mental health conditions affecting six large US employers in 1999. Journal of Occupational and Environmen- tal Medicine 45(1), 5-14 (2003).
53ten Brummelhuis, L.L., Johns, G., Lyons, B.J., & ter Ho- even, C. Why and when do employees imitate the absenteeism of co-workers? Organizational Behavior and Human Decision Processes 134, 16-30 (2016).
54Robinson, K. & Harris, A. L. The broken compass: parental involvement with children’s education. The Journal of Educational Research 108(4), 345-346 (2014).
Bergman, P. & Rogers, T. The impact of defaults on tech- nology adoption, and its underappreciation by policymakers. Faculty Research Working Paper Series RWP17-021. Cam- bridge: MA 2017
11
Reducing student absences at scale by targeting parents’ misbeliefs. REFERENCES
55Valencia. R.R. The Evolution of Deficit Thinking: Edu- cational Thought and Practice. (Routledge, 1997).
56Henderson, A. T. & Mapp, K. L. A new wave of evidence:
The impact of school, family, and community connections on student achievement. National Center for Family and Com- munity Connections with Schools (2002).
12
|