Knowing Value
Does college selectivity reduce obesity? A partial identification approach
Giorgio Brunello et al.
Health Economics, forthcoming
Abstract:
We use data from the National Longitudinal Study of Adolescent to Adult Health to investigate whether the quality of tertiary education -- measured by college selectivity -- causally affects obesity prevalence in the medium run (by age 24–34) and in the longer run (about 10 years later). We use partial identification methods, which allow us, while relying on weak assumptions, to overcome the potential endogeneity of college selectivity as well as the potential violation of the stable unit treatment value assumption due to students interacting with each other, and to obtain informative identification regions for the average treatment effect of college selectivity on obesity. We find that attending a more selective college causally reduces obesity, both in the medium and in the longer run. We provide evidence that the mechanisms through which the impact of college selectivity on obesity operates include an increase in income, a reduction in physical inactivity and in the consumption of fast food and sweetened drinks.
GPT Takes the SAT: Tracing Changes in Test Difficulty and Students' Math Performance
Saannidhya Rawat & Vikram Suresh
University of Cincinnati Working Paper, August 2024
Abstract:
Scholastic Aptitude Test (SAT) is crucial for college admissions but its effectiveness and relevance are increasingly questioned. This paper enhances Synthetic Control methods by introducing Transformed Control, a novel method that employs Large Language Models (LLMs) powered by Artificial Intelligence to generate control groups. We utilize OpenAI's API to generate a control group where GPT-4, or ChatGPT, takes multiple SATs annually from 2008 to 2023. This control group helps analyze shifts in SAT math difficulty over time, starting from the baseline year of 2008. Using parallel trends, we calculate the Average Difference in Scores (ADS) to assess changes in high school students' math performance. Our results indicate a significant decrease in the difficulty of the SAT math section over time, alongside a decline in students' math performance. The analysis shows a 71-point drop in the rigor of SAT math from 2008 to 2023, with student performance decreasing by 36 points, resulting in a 107-point total divergence in average student math performance. We investigate possible mechanisms for this decline in math proficiency, such as changing university selection criteria, increased screen time, grade inflation, and worsening adolescent mental health. Disparities among demographic groups show a 104-point drop for White students, 84 points for Black students, and 53 points for Asian students. Male students saw a 117-point reduction, while female students had a 100-point decrease. This research highlights the need to reconsider the SAT's role in admissions and to update educational strategies to enhance high school math performance.
Choosing to learn: The importance of student autonomy in higher education
Simon Cullen & Daniel Oppenheimer
Science Advances, July 2024
Abstract:
Despite strong evidence that autonomy enhances motivation and achievement, few interventions for promoting student autonomy in higher education have been developed and empirically tested. Here, we demonstrate how two autonomy-supportive policies effectively increase classroom attendance and subject mastery. First, in a randomized controlled field study, we explored the effect of allowing students to choose whether to make their attendance mandatory (i.e., a component of their course grades). We found that nearly all students used the opportunity as a pre-commitment device and were subsequently more likely to attend class than were students whose attendance had been mandated. Second, in a multi-year cohort study, we explored the effect of allowing students to opt out of a challenging, high-effort assessment stream, finding that students given greater autonomy invested more effort into their assignments and attained greater proficiency with the material. We discuss other opportunities for applying choice architecture to improve learning, motivation, and well-being in higher education.
In-person learning during the pandemic: Student take-up and school-level effects of remote and hybrid instruction on student outcomes
Stephen Ross et al.
Proceedings of the National Academy of Sciences, 13 August 2024
Abstract:
While studies have examined the effects of schools offering in-person learning during the pandemic, this study provides analysis of student enrollment decisions (remote versus in-person) in response to schools providing in-person learning opportunities. In Connecticut during the 2020–21 school year, we find that student take-up of in-person learning opportunities was low with students on average enrolled in-person for only half of the days offered, and take-up was even lower in schools with larger shares of disadvantaged students. The provision of in-person learning opportunities has been previously shown to mitigate pandemic learning losses. By exploiting data on actual enrollment, we show that the protective benefits of in-person learning are twice as large as previously estimated once we account for the low rates of student take-up. Finally, we provide evidence suggesting that a key mechanism behind the benefits of in-person learning is alleviating the burden faced by schools and teachers in delivering remote education. First, we show that the benefits to individual students of their in-person learning are substantially smaller than the overall benefits a student receives from their school average level of in-person enrollment. Second, we show that a combination of remote and in-person learning (hybrid) with a full-time on-line presence of students when at home was worse than hybrid learning with students never or only partially online. This second finding is consistent with qualitative evidence showing that teachers found hybrid learning especially challenging when having to manage both in-person and remote students for the entire class period.
The Rise of Teamwork and Career Prospects in Academic Science
Mabel Andalón et al.
NBER Working Paper, August 2024
Abstract:
Teamwork has become more important in recent decades. We show that larger teams generate an unintended side effect: individuals who finish their PhD when the average team in their field is larger have worse career prospects. Our analysis combines data on career outcomes from the Survey of Doctorate Recipients with publication data that measures team size from ISI Web of Science. As average team size in a field increased over time, junior academic scientists became less likely to secure research funding or obtain tenure and were more likely to leave academia relative to their older counterparts. The team size effect can fully account for the observed decline in tenure prospects in academic science. The rise in team size was not associated with the end of mandatory retirement. However, the doubling of the NIH budget was associated with a significant increase in team size. Our results demonstrate that academic science has not adjusted its reward structure, which is largely individual, in response to team science. Failing to address these concerns means a significant loss as junior scientists exit after a costly and specialized education in science.
The Returns to Experience for School Principals
Brendan Bartanen et al.
American Educational Research Journal, forthcoming
Abstract:
Despite increasing recognition of the importance of high-quality school leadership, we know remarkably little about principal skill development. Using administrative data from Tennessee, Oregon, and New York City, we estimate the returns to principal experience as measured by student outcomes, teacher hiring and retention patterns, and teacher and supervisor ratings of principals. The typical principal leads a school for only 3 to 5 years and leaves the principalship after 6 to 7 years. We find little evidence that school performance improves as principals gain experience, despite substantial improvement in supervisor ratings. Our results suggest that strategies intended to increase principal retention are unlikely to improve school outcomes absent more comprehensive efforts to strengthen the link between principal skill development and student and school outcomes.
The effects of teacher tenure on productivity and selection
Kevin Ng
Economics of Education Review, August 2024
Abstract:
I examine productivity and selection effects of K-12 teacher tenure by leveraging variation from New Jersey’s TEACHNJ Act, which extended the pretenure period. Using a difference-in-differences design, I evaluate the productivity effects of tenure by comparing fourth-year tenured and pretenured teachers. I find math value-added declines but English language arts value-added and ratings remain unchanged. Focusing on labor market effects, I compare teachers hired before and after TEACHNJ within the same district and experience level. TEACHNJ disproportionately increased male and Black teacher turnover, as the policy was tied to subjective evaluation criteria. TEACHNJ did not impact the quality of the teacher labor market as measured by value-added, though higher rated teachers often filled new vacancies. This matches principal–agent models where schools only use ratings to guide personnel decisions. Overall, tenure generates small declines in math value-added, while reforms tied to subjective evaluations disproportionately increase male and Black teacher turnover.
Generative AI Can Harm Learning
Hamsa Bastani et al.
University of Pennsylvania Working Paper, July 2024
Abstract:
Generative artificial intelligence (AI) is poised to revolutionize how humans work, and has already demonstrated promise in significantly improving human productivity. However, a key remaining question is how generative AI affects learning, namely, how humans acquire new skills as they perform tasks. This kind of skill learning is critical to long-term productivity gains, especially in domains where generative AI is fallible and human experts must check its outputs. We study the impact of generative AI, specifically OpenAI's GPT-4, on human learning in the context of math classes at a high school. In a field experiment involving nearly a thousand students, we have deployed and evaluated two GPT based tutors, one that mimics a standard ChatGPT interface (called GPT Base) and one with prompts designed to safeguard learning (called GPT Tutor). These tutors comprise about 15% of the curriculum in each of three grades. Consistent with prior work, our results show that access to GPT-4 significantly improves performance (48% improvement for GPT Base and 127% for GPT Tutor). However, we additionally find that when access is subsequently taken away, students actually perform worse than those who never had access (17% reduction for GPT Base). That is, access to GPT-4 can harm educational outcomes. These negative learning effects are largely mitigated by the safeguards included in GPT Tutor. Our results suggest that students attempt to use GPT-4 as a "crutch" during practice problem sessions, and when successful, perform worse on their own. Thus, to maintain long-term productivity, we must be cautious when deploying generative AI to ensure humans continue to learn critical skills.
Kumon in: The Recent, Rapid Rise of Private Tutoring Centers
Edward Kim, Joshua Goodman & Martin West
Harvard Working Paper, July 2024
Abstract:
The increasing prevalence of private tutoring has received minimal scholarly attention in the United States. We use over 25 years of geocoded data on the universe of U.S. private tutoring centers to estimate the size and growth of this industry and to identify predictors of tutoring center locations. We document four important facts. First, from 1997 to 2022, the number of private tutoring centers more than tripled, from about 3,000 to 10,000, with steady growth through 2015 before a more recent plateau. Second, the number and growth of private tutoring centers is heavily concentrated in geographic areas with high income and parental education. More than half of tutoring centers are in areas in the top quintile of income. Third, even conditional on income and parental education, private tutoring centers tend to locate in areas with many Asian American families, suggesting important differences by ethnic or cultural identity in demand for such services. Fourth, we see only marginal evidence that prevalence of private tutoring centers is related to the structure of K-12 school markets, including the prevalence of private schools and charter or magnet school options. The rapid rise in high-income families’ demand for this form of private educational investment mimics phenomena observed in other spheres of education and family life, with potentially important implications for inequality in student outcomes.
The return to classroom instruction time in private and public schools
Jeffrey Schiman & Rand Ressler
Empirical Economics, August 2024, Pages 449–464
Abstract:
Private school students outperform their public school peers on standardized tests. Extensive effort has been devoted to testing whether the private–public gap is attributable to the schools themselves or simply due to peer effects or positive selection into private schools. Receiving far less attention is the extent to which the return to specific schooling inputs differs between private and public schools. We find evidence of an overall positive effect of class time on academic achievement and little evidence of a premium to time in private schools. Indeed, the benefit of added time appears similar in both settings. The lack of a private school premium to class time is consistent with the notion of positive selection into private schools.
Are Two Teachers Better Than One? The Effect of Coteaching on Students with and without Disabilities
Nathan Jones & Marcus Winters
Journal of Human Resources, July 2024, Pages 1180-1206
Abstract:
Coteaching, in which a general education teacher and special education teacher collaboratively provide instruction to students with and without disabilities in the same classroom, is widely endorsed as a strategy to give instructional support to students with disabilities within inclusive environments. We leverage longitudinal administrative data in Massachusetts to provide the first causal estimate for the effect of coteaching across a large public school system. We find evidence that coteaching leads to statistically significant test score improvements for both students with and without disabilities. However, the benefits for students with disabilities are much smaller than reported in prior studies.
Students With Growth Mindset Learn More in School: Evidence From California’s CORE School Districts
Susana Claro & Susanna Loeb
Educational Researcher, forthcoming
Abstract:
Previous research provides evidence that developing a growth mindset -- believing that one’s capabilities can improve -- promotes academic achievement. Although this phenomenon has undergone prior study in a representative sample of ninth graders in the United States, it has not been studied in representative samples of other grade levels or with standardized assessment measures of achievement rather than more subjective grades. Using a rich longitudinal data set of more than 200,000 students in Grades 4 through 7 in California who we followed for a year until they were in Grades 5 through 8, this article describes growth mindset gaps across student groups and confirms, at a large scale, the predictive power of growth mindset for achievement gains. We estimate that a student with growth mindset who is in the same school and grade level and has the same background and achievement characteristics as a student with a fixed mindset learns 0.066 SD more annually in English language arts, approximately 18% of the average annual growth or 33 days of learning if we assume learning growth as uniform across the 180 days of the academic year. For mathematics, the corresponding estimates are 0.039 SD, approximately 17% of average annual growth or 31 days of learning.