Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
. Author manuscript; available in PMC 2016 Mar 31.
Published in final edited form as:
PMCID: PMC4815037
NIHMSID: NIHMS761610

Do Stimulant Medications Improve Educational and Behavioral Outcomes for Children with ADHD?

Janet Currie, Mark Stabile, and Lauren Jones
Author information ► Copyright and License information ►

Abstract

We examine the effects of a policy change in the province of Quebec, Canada which greatly expanded insurance coverage for prescription medications. We show that the change was associated with a sharp increase in the use of stimulant medications commonly prescribed for ADHD in Quebec relative to the rest of Canada. We ask whether this increase in medication use was associated with improvements in emotional functioning or academic outcomes among children with ADHD. We find little evidence of improvement in either the medium or the long run. Our results are silent on the effects on optimal use of medication for ADHD, but suggest that expanding medication in a community setting had little positive benefit and may have had harmful effects given the average way these drugs are used in the community.

Introduction

Over the past twenty years, mental disabilities have overtaken physical disabilities as the leading cause of activity limitations in children. Today, ADHD is three times more likely than asthma to be contributing to childhood disability in the United States (Currie and Kahn, 2011). Recent research indicates that children with ADHD have lower standardized test scores than others (including their own siblings) and are more likely to be placed in special education, to repeat grades, and to be delinquent (; ; , 2007; , ). Moreover, untreated children with ADHD impose significant costs on their classmates by disrupting learning and/or diverting teacher resources ().

According to the most recent data from the Centers for Disease Control and Prevention, approximately eleven percent of U.S. children aged 4 to 17 have ever been diagnosed with ADHD and more than half of them are taking stimulant medications such as Ritalin for their condition (; Centers for Disease Control and Prevention, 2005).1 Both diagnosis and treatment rates are lower outside the U.S., but have been rapidly increasing ().

Despite, or perhaps because of the millions of children taking stimulants, drug treatment for ADHD remains controversial. The National Institute of Mental Health recommends treatment with stimulants and says that they are safe if used under medical supervision (). However, concerns continue to surface about both short term side effects, and possible side effects due to long-term use. For example, the U.S. Food and Drug Administration voted in 2006 to recommend a warning label describing the cardiovascular risks of stimulant drugs for ADHD (). Other side effects can include decreased appetite, insomnia, headache, stomach ache, dizziness and mood changes including anxiety and depression (, ). Some studies have also found growth deficits in treated children (). Aside from the possibility of physical side effects, inappropriate use of stimulant medication could also harm children by stigmatizing them or by crowding out other interventions that might be more helpful.

Lack of evidence regarding long-term benefits of stimulant medications is a key element of this controversy. Drugs are often prescribed with the goal of helping children to be successful in school. If the drugs do not actually lead to scholastic benefits in the medium and long run, then the case for subjecting children to even a small risk of side effects is weakened. The main problems involved in assessing the long-run efficacy of stimulant medication are first, that most drug trials follow children only for a short time – between one and two months after treatment () – and second, that families (and children) choose whether or not to seek treatment for ADHD, and whether to take medication if it is prescribed.

Our paper assesses the medium and long run benefits of treatment for ADHD with stimulant medication using longitudinal data from the National Longitudinal Survey of Canadian Youth (NLSCY), and a unique policy experiment which expanded insurance coverage for drugs in Quebec in 1997. Our study improves on the previous literature in many respects. First, we have a large sample of children who have been followed from 1994 to 2008. We are able to observe medium term outcomes such as grade repetition and math scores, as well as long term outcomes like graduation from high school and whether children ever attended college. Moreover, we know whether children were taking stimulant medication as of each wave. An important feature of the NLSCY is that all children were assessed for ADHD symptoms, so we do not have to deal with selection into diagnosis. A third innovation is that we are able to exploit exogenous variation in the availability of drugs due to the policy experiment. Fourth, in our analysis of medium term outcomes we are able to use individual fixed effects to control for unobservable differences between children that might influence both treatment and outcomes.

We find that the introduction of the prescription drug insurance program increased the use of stimulants in Quebec relative to the rest of Canada. However, we find no evidence that the performance of children with ADHD improved. In fact, the increase in medication use among children with ADHD is associated with increases in the probability of grade repetition, lower math scores, and a deterioration in relationships with parents. When we turn to an examination of long-term outcomes, we find that increases in medication use are associated with increases in the probability that a child has ever suffered from depression and decreases in the probability of post secondary education among girls.

The rest of the paper is laid out as follows. Section 1 reviews the previous literature about the consequences of ADHD for child outcomes and the controversy surrounding ADHD medications. Section 2 discusses our data and Section 3 discusses methods. The results appear in Section 4 and Section 5 concludes.

Background

In view of the importance of ADHD and the fact that stimulant medications have been used for many years, it is perhaps surprising that most of the evidence regarding their efficacy relates to short time horizons. Controlled studies suggest that medication improves attention, short-term memory, performance on quizzes, homework completion, and note-taking (; ; ; ). It is often assumed that these improvements will translate into future academic gains, but few studies actually track children longer than a few months. Moreover argue that the positive short-run effects on attention and behavior may be over-estimated given publication bias towards positive findings. An additional concern is that the doses that yield the most desirable behavior may not be calibrated to achieve the greatest possible improvement in cognitive functioning ().

One of the most widely known longer term studies of the effects of medication for ADHD is based on the U.S. National Institute of Mental Health 14 month Multimodal Treatment study (MTA). It is important to note that this study did not compare medication to non-treatment, instead, the MTA compared different types of treatment. Specifically, the MTA randomized 579 children with ADHD into four arms: Stimulants alone; behavioral therapy alone; stimulants plus behavioral therapy; or usual community care, which involved treatment with stimulants but with possibly less than optimal dosages. Blinded classroom observations did not find any significant differences in behavior between the four groups. At the end of 14 months, 49.8% of children reported mild side effects, 11.4% reported moderate side effects, and 2.9% reported severe side effects ().

discuss a long-term follow up of children from the MTA study which included 436 of the original study children and 261 “controls” who were randomly selected from the same schools and grades 24 months after the original study began and matched with treatment children by age and gender. They find that 6 to 8 years following the initial intervention, there were still no differences between the children in the four treatment groups. They also find that the treatment children were worse off than the “controls” on virtually every measure but it is important to note that these controls were not part of the original randomized design so this comparison does not constitute an experimental evaluation of the long term benefits of drug treatment compared to non-treatment. Of those originally assigned to take medications, 62% had stopped taking them by the time of the follow up which is remarkable in itself since it suggests dissatisfaction with the drug regimen. However, adjusting for this attrition did not affect the differences between treated children and control children.

follow 370 children with ADHD from a 1976-1982 birth cohort study. They obtained the complete school record, as well as medical records with information about stimulant use for each child. They found that in this sample of children with ADHD diagnoses, longer duration of stimulant use was associated with reductions in absences and retention in grade but had no effect on school dropout. However, endogeneity of stimulant use makes these results difficult to interpret. If the children with the worst attention difficulties were most likely to take medication, then any positive effects of medication would be biased towards zero. Alternatively, if children from the best backgrounds were most likely to take stimulants properly, then this might bias the analysis towards finding a positive effect.

Zoega et al. (2009) use registry data from Iceland, which has a measured prevalence of ADHD and a usage of stimulant medication that is similar to the U.S. They linked information from medical records to a data base of national scholastic examinations for children born between 1994 and 1996 who took standardized tests at fourth and seventh grade. In order to deal with the endogeneity of treatment, they include only children who were “ever treated” between the ages of 9 to 12, and focus on whether they were treated sooner or later. They find that children with ADHD suffered declines in test taking relative to other children, but that ADHD children who started medication earlier experienced slower declines than those who started medication later. Again, this design suffers from endogeneity, this time in terms of the choice of when treatment was started. It is possible, for instance, that children start medication in response to some crisis, and then experience reversion to their mean performance.2

use data from the Early Childhood Longitudinal Study— Kindergarten Class of 1998-1999 to examine the effect of medication use on standardized math and reading test scores for 594 children with ADHD. They estimate first differenced models in order to control for constant aspects of the child's background. A limitation of their data is that questions about medications were asked only in fifth grade, so it was assumed that children who were not taking medication at fifth grade had never taken it. They find that children with ADHD who took medication had higher mathematics and reading scores than other children with ADHD, though they still lagged behind their non-ADHD peers. However, if children with ADHD are on different trajectories then their non-ADHD peers, then it is not clear that estimating the model in first differences will adequately control for the endogeneity of medication use.

use Danish registry data and variations in the prescription patterns of physicians to identify the effect of ADHD medication on hospital contacts, criminal activity and a limited set of school performance measures. They find that physician treatment patterns vary significantly, and that among children who receive treatment, hospital contacts decrease as do the number of interactions with police. While they find little difference in test scores for treated versus non-treated children, they note that treated children are less likely to take the exam. One limitation of their study is that higher income children were significantly more likely to go to doctors who prescribed medication more frequently which suggests that the probability of receiving a prescription was correlated with economic status.

Our study provides new evidence regarding the medium- and long-term effects of stimulants use for ADHD in a nationally representative sample of Canadian children by taking advantage of a policy experiment that expanded access to these drugs.3 In 1997, the Canadian province of Quebec adopted a mandatory prescription drug insurance law.4 Before 1997, many residents of Quebec received private prescription drug insurance from their employers while others went without drug insurance. The new law stipulated that all Quebecers had to be insured. Those who did not have insurance through their employer were required to participate in a new provincial public plan (). Premiums and deductibles were scaled according to income and some segments of the population received coverage for free including children whose parents were covered. Premiums were collected along with the filing of the Quebec tax return to ensure compliance with the law (). Details on the premiums, deductibles and co-insurance rates over time are presented in the data appendix.

As a result of the insurance mandate and public plan, drug insurance rates rose quickly in Quebec. Using data from the National Population Health Survey and Community Health and Social Survey, both of which contain information on whether or not individuals hold prescription drug insurance,5 we calculate coverage rates in both Quebec and in the rest of Canada. Whereas the rate of drug insurance coverage pre-reform in 1996 was 55%, it jumped to 84% in 1998 and continued to rise to 89% by 2003. Drug coverage rates in the rest of Canada averaged 65% in 1996 and rose slowly over time to an average of 76% by 2003 (Table 1). Overall the jump in Quebec far exceeds the rise in coverage taking place in the rest of the country as Quebec was the only province that instituted a universal coverage mandate.

Table 1
Changes over time in prescription drug insurance rates in Canada versus the rest of Canada

Our identification strategy, then, is to first explore the increase in the use of stimulants that accompanied the increase in drug coverage and then to relate the increase in drug use to medium and long-run child outcomes. Since it is possible that there were divergent trends in outcomes in Quebec and Canada which were independent of the introduction of the prescription drug law, we focus on the effects of the law on children who had high levels of ADHD symptoms prior to the passage of the law. The overall argument is that if an expansion in drug use is beneficial, then we should see an improvement in the performance of children with ADHD in Quebec relative to the rest of Canada.

Data

We use data from the NLSCY, a national longitudinal data set which began with an initial sample of children ages 0 to 11 and their families in 1994. In the second wave of data collection in 1996, 15,871 of these children were surveyed (a reduced sample due to budget restrictions). We use the children born in 1985 or later who appear in both the 1994 and 1996 surveys as the base sample for this study. Follow-up surveys were conducted biannually up to 2008, producing up to 8 potential survey responses for each child. For responses pertaining to children under age 16, the survey collected information from the person most knowledgeable (PMK) about the child, while older children (16 and older) were responsible for completing the survey themselves.

We employ distinct approaches to evaluating the medium and long-term effects of stimulant use, and our sample depends on the approach in question. To investigate medium-term outcomes, we exploit the panel nature of the NLSCY and restrict the sample to observations collected at ages 0 through 16. For the oldest children in the sample – those born in 1985 or 1986 – we are able to observe up to 3 observations per child, while we use up to 7 survey responses for the youngest children. Our medium-term outcomes are not collected for all ages, however, and we further restrict our medium-term base sample as data availability requires.7 The data appendix provides information about the maximum number of observations potentially available for each measure, and the number actually available given attrition.

For the long-term analysis we focus on outcomes that are reported by the youths themselves at ages 16 and later. For the most part, these are measured only once for each child, like high school graduation. With the exception of the self-assessed depression score – which we construct by averaging all available scores for each child in order to better capture whether the child was ever depressed and the persistence of depression – variables are defined according to their last observed value. Our long-term outcomes sample therefore consists of children aged 0-9 in 1994 who remain in the sample until at least age 16, tracked through 2008, with one observation per child. Due mostly to attrition, the base long-term sample is composed of 8,643 children born in 1985 or later, surveyed in both 1994 and 1996, and followed thereafter.

We measure ADHD using questions that are asked to parents about symptoms of ADHD. ADHD is always diagnosed through the use of questions similar to those included in the survey. Parents are asked to rate on a scale of 0 to 2 how often their child demonstrates five behaviors common among those who suffer from ADHD. Answers to these five questions are summed to produce an ADHD score that ranges between 0 and 10, where higher scores indicate a higher level of ADHD symptoms. The questions used are listed in the data appendix, along with the questions used to construct all outcome variables. One of the strengths of the NLSCY data for this analysis is that these screener questions are administered to all children aged 2 to 11 years old, rather than to only diagnosed cases. We use the ADHD screener score collected in 1996 as our measure of the child's ADHD symptoms. Using the 1996 measurement allows us to obtain a pre-policy measurement of the severity of any child's ADHD symptoms.

Our information on stimulant use for both the medium- and long-term analyses is derived from a survey question that asks whether the child takes, “any of the following prescribed medication on a regular basis: Ritalin or other similar medication.” This question is asked about all children age 15 and younger. Approximately 9 percent of sample children in Quebec, and 5 percent of sample children in the rest of Canada report ever having used stimulants. Stimulant use has increased slowly in Canada from less than 2 percent in 1994 to around 4 percent in 2008. Figure 1 shows that in Quebec, stimulant use tracked the rest of Canada closely prior to the policy change, but began to increase significantly following the policy change in 1997.

Figure 1
Stimulant use by survey year for Quebec versus the rest of Canada

Figure 2 provides additional evidence that the policy change led to significant increases in stimulant use. For all respondents who indicate ever using stimulants, we plot the fraction that commenced use in each cycle. If stimulant use is unrelated to outside factors, then uptake rates across survey cycles should exhibit a more-or-less smooth trend, with approximately equal proportions commencing use in any one year, peaking when the sample has the most children at peak diagnosis ages (6-10) and declining as the sample ages and diagnosis becomes less frequent. This is the pattern we see for children living outside of Quebec. For children in Quebec, however, there is a distinct spike in uptake rates in 2000 and 2002, following the policy change.

Figure 2
Stimulant uptake patterns in Quebec versus rest of Canada

The NLSCY also asks about other chronic conditions, some of which could also have been affected by increased drug coverage. Specifically, the survey asks whether, “a health professional has ever diagnosed any of the following long-term conditions…” where the listed conditions include: any type of allergy, bronchitis, heart conditions, epilepsy, cerebral palsy, kidney conditions, mental handicaps, learning disabilities, attention deficit disorder, emotional or psychological difficulties, eating disorders, autism, migraines, or any other chronic condition. We use these questions to test the robustness of our findings in two ways. First, we exclude children who had other (physical) chronic conditions from the sample and repeat our analyses.

Second, we examine children with asthma who may have gained access to, “Ventolin, inhalers or puffers for asthma” with expanded drug coverage. The increase in stimulant use was particularly pronounced relative to other medications such as the use of inhalers for asthma which did not increase disproportionately in Quebec relative to the rest of Canada (Figure 3). Thus, although the law was intended to increase access to all types of necessary medications, it seems to have had a disproportionate impact on prescriptions for stimulants.8 It is perhaps unsurprising that the impact of the policy change on stimulants is large while we observe little impact on ventilators. It seems plausible that the elasticity of demand for mental health drugs is larger than the elasticity for asthma medication and recent media reports on physician practices suggests that there may be greater variation in the use of stimulant medication to treat children with behavioral problems (). Further, the price of stimulant medications is large relative to asthma medications9.

Figure 3
Ventilator use by survey year for Quebec versus the rest of Canada

We focus on outcomes that are intended to capture the child's behaviour, emotional state, and human capital accumulation in both the medium and longer run. The analysis of our medium-term outcomes involves a panel analysis of repeated observations over time for the same child They include: an unhappiness score, a rating of the child's relationship with his or her parents over the past 6 months10, whether the child repeated a grade in the past two years and a mathematics score which is age-standardized to have a mean of zero and a standard deviation of one. Mathematics tests were administered in schools to children in grades two through ten and are based on the Canadian Achievement Tests.11

While the medium-term analysis is conducted using multiple outcome values for each child collected over time, the long-term analysis only employs one observation for each child. The long-term outcomes we consider include: an indicator for high school graduation, an indicator for having attended or graduated from a post-secondary institution, and a self-assessed depression score composed of six questions asked of all respondents aged 16 and older. In the case of the self-assessed depression score, we average all available self-assessed scores collected as of 2008 in order to construct an overall measure of the child's adolescent depression symptoms. The educational outcomes measure, by wave 8, whether the child had graduated from high school and whether he or she ever attended any post-secondary education.

Descriptive statistics for stimulant use, the outcome variables, and key independent variables for both the medium and long-term samples are shown in Table 2 (referred to as Samples 1 and 2, respectively). The table shows means separately for Quebec and the rest of Canada. The increase in stimulant use in Quebec vs. the rest of Canada is apparent in the first half of the table, and the second half of the table shows that a much larger number of children had ever used stimulants in Quebec by the end of the sample period. It is apparent that there are some differences in mean outcomes in Quebec vs. the rest of Canada, though the baseline child and family characteristics are fairly similar.

Table 2
Stimulant use, ADHD symptoms, and child and family characteristics

Methods

We begin by estimating the effect of the policy change on the use of stimulants in a difference-in-difference framework. The estimating equation takes the form:

Stimit=α+βPostit−1+λQueit−1+φPostit−1∗Queit−1+XitΠ+τt+pi+εit
(1)

where Stimit is a dichotomous variable equal to one if the PMK reports that child i is currently taking stimulant medication in year t, τ are survey year fixed effects and p are province fixed effects. Postit is a variable that identifies those survey responses collected from children after 1996, Queit identifies responses from children in Quebec, and their interaction indicates the treatment group. In this specification, we compare children in Quebec to children in other provinces, before and after the policy change. The vector X includes family income, whether the person most knowledgeable about the child is an immigrant, whether the person most knowledgeable about the child (the survey respondent) is male or female, the sex of the child, birth order, family size, whether there are two parents present in the family, the mother's age at birth, whether the mother had a teen birth, and child-age dummies. To allow for delayed uptake in medication treatment, as well as time for the medication to take effect, we lag the policy change variable by one period (both the province of residence and the indicator for being post policy change).We expect a positive coefficient estimate on the Postit * Queit interaction term, implying that increased access in post-reform Quebec led to expanded use of stimulant medication.

A limitation of the difference-in-differences approach is that there may be post 2007 differences in outcomes between Quebec and the rest of Canada for other reasons. Therefore, we focus on a triple difference specification that focuses on those children most likely to benefit from increased stimulant use in response to the policy change: Those with the worst initial ADHD symptoms. These models add an additional level of interaction terms to equation (1) – the ADHD score for the child between the ages of 2 to 11, measured in 1996 (pre policy change) – in order to estimate a difference-indifference-in-difference (DDD) model, comparing children with worse underlying ADHD symptoms (measured before the reform) in post-reform Quebec to other children. This model is specified as:

Stimit=α+βPostit−1+λQueit−1+γADHD96i+ηQueit−1∗ADHD96i+φPostit−1∗Queit−1+δPostit−1∗ADHD96i+θPostit−1∗Queit−1∗ADHD96i+XitΠ+τt+pit+εit,
(2)

where ADHD96i is the child's 1996 ADHD symptom score.12 Using this approach, we are able to isolate the effect of the reform on stimulant use among children with worse ADHD symptoms, net of any pre-existing differences in stimulant use across time, geography, and severity of symptoms. In this specification, we expect that the estimate of θ should be positive.

When we examine medium term outcomes, we focus on versions of equations (1) and (2) that include child specific fixed effects. In these models, the effects are identified through changes in stimulant use for the same child before and after the policy change. The ability to control for child fixed effects obviates concerns about possible changes in the sample of children over time.

We use the same DDD framework (equation (2)) to examine the effect of the policy change on outcomes: if stimulant use improves outcomes, and children with worse symptoms are more likely to be treated post reform, then children with worse ADHD symptoms in post-reform Quebec – should demonstrate post-reform improvements in outcomes relative to their peers.

In order to examine the longer-term effects of an increase in stimulant use, we next use the sample with one long-term observation per child and estimate a quasi-first stage regression where the dependent variable is an indicator that equals 1 for children who ever reported using stimulant medication between ages 0 and 15 (EverRit). We construct a policy exposure variable intended to capture the number of years the child was eligible for the new prescription drug regime: The total number of under age 16 survey responses for the child that occurred post 1996 (PostYrs). We then interact this lifetime exposure window variable with a Quebec indicator and the 1996 ADHD symptom score to create a parallel to (2):

EverRiti=α+βPostYrsi+λQue94i+γADHD96i+ηQue94i∗ADHD96i+φPostYrsi∗Que94i+δPostYrsi∗ADHD96i+θPostYrsi∗Que94i∗ADHD96i+X94iΠ+p94i+εi
(3)

Equation (3) is estimated using one observation per child and includes measures that are constructed at different periods in the child's life. Here the vector X includes controls measured as of 1994. The maximum number of years that a child can be treated depends on his or her age in year 1 of the survey (1994). We include age/cohort dummies to control for the fact that different children will be observed for different lengths of time. After estimating the relationship between lifetime stimulant use and exposure to the policy, we use equation (3) to examine the relationship between stimulant use and long term outcomes.

Given that we observe individual level information on stimulant medication, an alternate specification would be to estimate the impact of the policy using instrumental variables. However, an IV specification would require us to assume that the legislation only affected children through its effects on stimulant medications and there might be other pathways. Some possibilities include effects on other medications for children, onparent medications, on household budgets, and schools pushing harder for kids to be put on medications because they are now perceived as affordable. Further, the IV specification requires a separate first stage for each subsample and we felt that it would be simpler to present the results using the difference-in-differences framework.

We perform a number of robustness checks to control for other health and learning disabilities that the child may have, as well as to specifically control for other contemporaneous policy changes that occurred in Quebec over this period. We discuss these checks following the presentation of our main results.

Results

We first examine the effect of the policy change on the probability of stimulant use in our sample as well as the relationship between exposure to the policy change and the number of years that a child used stimulants. Table 3 presents the results. Columns 1 and 2 report the difference-in-differences results without and with child fixed effects. In both cases we see an increase in the probability of using stimulants of approximately 2.5 percentage points for children in Quebec after the policy change. Columns three and four of Table 3 report the triple difference estimates (the D-D interacted with the child's 1996 ADHD score). Here the preferred fixed effect estimate suggests an increase of approximately 0.43 percentage points with each one unit increase in ADHD scores, which is quite similar to the OLS estimates without fixed effects of 0.48 percentage points. At the average ADHD score, this is a 1.15 percentage point change in stimulant use compared to the average baseline number of children on stimulants of 2 percent.

Table 3
Effects of law change on stimulant use

We also estimate a similar “first stage” model for our longer-term analysis by examining the relationship between exposure to the policy change and ever taking stimulants, as described in equation (3). These results are presented in columns 5 and 6 of Table 3. The DDD estimate suggests a 0.56 percentage point increase in the probability of ever taking stimulants on a baseline of 4 percent, however, the coefficient is imprecisely estimated and is significant only at the 90 percent level of confidence. Again, while this is a fairly small overall change in stimulant use it reflects a large change relative to baseline.

Having established that the policy change resulted in a reasonably large change in the use of stimulants we now turn to examining both the medium and longer term consequences of this change. Table 4 presents the estimates for medium term outcomes. All columns include child specific fixed-effects. The difference-in-differences estimates suggest consistently worse outcomes post policy change in Quebec though, even with the inclusion of child fixed effects, these differences could possibly reflect divergent trends in Quebec and the rest of Canada. Therefore, we prefer to focus on the DDD estimates. These also suggest a significant negative effect of the policy change in terms of grade repetition, math scores, and relationships with parents. For example, the coefficient on the triple interaction in the “grade repetition” model suggests that for each one unit increase in ADHD scores, the probability that a child repeated a grade between waves increased post policy change by 1.28 percentage points on a baseline 7 percent repetition rate.

Table 4
Child Fixed Effects Estimate of Exposure to Policy on Contemporaneous Outcomes

Turning to the long term outcomes, Table 5 shows estimates of equation (3). In the triple difference framework, the estimates suggest that the only long term effect is on unhappiness – there is no statistically significant effect of exposure to the policy on high school completion or post-secondary schooling among those with higher ADHD scores.

Table 5
Effects of Exposure to the Policy on Long-Term Outcomes

These estimates cast doubt on the idea that the diffusion of stimulant use improved academic outcomes among those with ADHD, and raise the possibility that children were actually harmed. There are several possible mechanisms that could be at work. First, many of the known side effects of stimulant use have to do with children's emotional wellbeing; direct effects on unhappiness or depression may therefore not be surprising. It is also possible that stimulants have direct effects on children's cognitive abilities, particularly if dosages are not optimized for the individual child. A second possible mechanism is that stimulant use might crowd-out other therapies or learning strategies that could be more beneficial to the child. A third possibility is that stigma associated with an ADHD diagnosis and stimulant use is harmful to the child. In order to further assess these possibilities we turn to a separate analysis by gender.

Estimates by Gender

There are well-documented differences in ADHD prevalence and in the use of stimulants between boys and girls: For example, find that 15% of U.S. boys and only 7% of U.S. girls have ever been diagnosed with ADHD. Figure 4 plots stimulant use rates for Quebec versus the rest of Canada separately for boys and girls. Due to NLSCY data release rules, we have pooled observations by two-survey year time periods. Thus, the first point in the graphs shows the rate of stimulant use indicated in 1994 and 1996 survey responses and it is our pre-policy observation; the remaining points represent stimulant use rates for post-policy years. What is clear is that while both boys and girls increased stimulant use substantially after the policy change, the effect is much larger among boys.

Figure 4
Trends in Stimulant Use By Gender, Quebec vs. Rest of Canada

Table 6 shows our estimates of the effect of the policy change on take up of stimulants by gender. Column (1) shows that among boys, there was a strong increase in the use of stimulants in Quebec post policy change. However, column (2) shows that there was no differential impact among children with higher ADHD scores. In other words boys with low levels of ADHD symptoms were just as likely to take up stimulants as those with high ADHD scores post policy change, suggesting that the marginal boy taking stimulants had lower levels of ADHD symptoms post policy change. Columns (3) and (4) show the comparable estimates for girls. The story for girls is quite different, suggesting that the increase in stimulant use post policy change was concentrated among girls with high ADHD scores, and that there was no increase in usage among girls with low ADHD scores.

Table 6
Effects of the Law Change on Stimulant Use By Gender

Table 7 shows estimates of the medium-term impacts of the policy on boys and girls. The estimates for girls suggest that the negative effects of the policy change are confined to girls who had initially high ADHD scores, which makes sense, since these are the only girls who increased stimulant use as a result of the policy change. Among these girls, there are increases in unhappiness, deteriorations in relations with parents, and reductions in math scores.

Table 7
Child Fixed Effects Estimate of Exposure to Policy on Contemporaneous Outcomes, by Gender

For boys, the DD coefficient estimates on the Quebec post-policy indicator suggest that the policy change is associated with an increase in grade repetition among all boys; however, this result could be part of a general trend towards greater use of grade repetition among boys in Quebec. Among boys with higher ADHD scores, there are deteriorations in relations with parents and an even larger increase in grade repetition post policy change. These estimates suggest that the upswing in stimulant use following the policy change had larger negative effects on boys with ADHD than on those without, even though stimulant use increased for boys with and without ADHD symptoms. It is possible that the negative effects of increased stimulant use – for example the crowding out of other types of intervention –were greater for boys with more severe ADHD symptoms since they had greater need for these interventions.

Turning to the results for long-term outcomes which are shown in Table 8, the estimates suggest that the policy impacted girls with ADHD but not boys. Specifically girls with higher initial ADHD scores were more likely to have suffered from depression, and less likely to have any post-secondary education, the more they were exposed to the post-policy change regime.

Table 8
Effects of Exposure to the Policy on Long-Term Outcomes

Robustness checks

We performed a number of specification checks to test the robustness of our findings. First we re-estimated the triple difference models excluding children with physical chronic conditions. These children may have benefited from increased access to other medications, which could have affected outcomes as well. However, the estimates are quite similar in this sub-sample. Estimates are shown in Appendix Table 4. Since asthma is the most common physical chronic condition among the children in our sample, we also asked whether there was an increase in ventilator use following the policy change. Such an improvement in the treatment of asthma could have had independent effects on children's outcomes. We find insignificant coefficients on the DDD estimates for an increase in ventilator use, unlike our estimates for increases in the use of stimulants.13 The results are reported in Appendix Table 4. We also re-estimated all our main specifications excluding children who reported the use of ventilators in order to verify that any increase in the use of ventilators was not having an indirect effect on our estimates of the effects of stimulants. Our results are robust to excluding these children from the sample.

A second possible concern is that our triple difference, despite focusing on the children who were most likely to benefit from stimulant use, could be picking up the effect of contemporaneous policy changes. One important policy change that happened around the same time was the introduction of subsidized day care in Quebec. find negative effects of exposure to subsidized day care programs in Quebec on a number of child outcomes. To make sure that we are not confounding these two policy changes, we re-estimated our models limiting the sample to children born in 1991 or earlier – that is, to those unaffected by the childcare policy change. Although this restriction greatly reduces the sample size, we continue to find negative effects on math scores and grade repetition. These estimates are reported in Appendix Table 4.

We have focused above on unhappiness and depression given that these are the most prevalent mental health conditions (besides ADHD) in our sample. However, given that other measures of mental health are available, we also created a composite mental health measure. Using the unhappiness score, along with similarly constructed scores measuring anxiety and physical aggression, we standardized and then averaged the scores to construct an overall composite mood score. We continued to find positive and significant coefficient estimates (reflecting an increase in mood and behavioral problems) for girls. These results are also reported in Appendix Table 4.

Finally we conduct a series of placebo tests using data excluding observations from Quebec. We define placebo policy change dates every two years, from 1995 to 2005, and policy change regions in Ontario, British Columbia, the prairie provinces (Alberta, Saskatchewan and Manitoba), and the maritime provinces (Newfoundland, Nova Scotia, New Brunswick and Prince Edward Island). We then estimate equations (2) and (3) for each placebo year-region combination, resulting in a total of 24 placebo DDD coefficient estimates for each model. We plot the distributions of these estimates in Appendix Figures 1 and 2; in both cases, the vertical line denotes the DDD coefficient estimate derived from the model estimation using the true policy change in Quebec in 1997. The figures reveal that the true coefficient estimates fall in the tails of the placebo distributions, suggesting that if the policy had been ineffective – as the placebo changes are by definition – we would be very unlikely to have generated estimates as large in magnitude as those that we find. The lack of any systematic or robust relationship between the experiment and the stimulant use outcomes in the placebo context provides some confidence that we are not picking up a spurious correlation in the true policy experiment setting.

Discussion and Conclusions

This paper examines the effect of a “natural experiment” in Quebec that greatly expanded access to stimulant medication, and the take up of stimulants among children with ADHD. One might have anticipated that increases in access to medication would be associated with improved outcomes among these children. Instead, we actually find some evidence of negative effects. Some of these negative effects are consistent with known possible side effects of stimulant medication, especially depression.

We find little evidence of positive effects on academic outcomes or schooling attainment. In fact, we find deterioration in important academic outcomes including grade repetition and math scores. When we examine the effects of the policy by gender, we find that stimulant use among boys increased greatly, but that it increased equally among boys with high and low levels of initial ADHD symptoms. Among girls, the increase in stimulant use was more concentrated among children with initially high levels of ADHD symptoms. However, the increase in stimulant use among girls with ADHD was associated with increases in unhappiness and the probability that a girl had depression, decreases in math scores, and a decline in the probability of having any post-secondary education.

Our findings of potentially negative effects associated with the increase in stimulant medications use raise the question of mechanisms. How is it possible that an increase in the utilization of medication for ADHD could be associated with worse academic performance?

One possibility is that an increase in the availability of stimulants makes it more likely that a child will be treated for ADHD and that treatment triggers harmful social stigma or other consequences, such as being placed in special education.14 A second possibility is that medication is a substitute for other types of cognitive and behavioral interventions that might be necessary to help the child learn. By making children less disruptive, ADHD medication could decrease the attention that they receive in the average classroom and reduce the probability that the child receives other needed services. A third possibility is that the medication itself, particularly if the dosage is not appropriately tailored, could have negative effects on emotional wellbeing and learning.

It is important to acknowledge that this is an ecological study which does not shed light on the question of whether optimal medication use could be beneficial. It is clear that many children use stimulant medication in a haphazard manner. For example, on average, among those who ever report using stimulants in our data, children use stimulants for about 30% of the survey years we observe them. Moreover, the average child who is ever reported to use stimulants switches twice over the observation period (between the time they are ages 4-7 and age 15, depending on how old they were in 1994). While it is possible that some of this churning is measurement error, recall that in the MTA most children had stopped taking medications 6 to 8 years after follow up. In addition, while we have no information about dosage, it seems likely that many children are taking doses of ADHD that are not calibrated to achieve optimal results, even in terms of short-term behavioral effects.

What our results do speak to, is the effect of a large increase in the use of ADHD medications in a community, given the usual standard of care available to Quebec children. In Quebec, as in the U.S., any doctor can prescribe stimulants, and it is not necessary to have expertise treating ADHD in order to do so. Hence, it is not surprising that some use is sub-optimal. Our results suggest that observers of the large increases in the use of medication for ADHD in Canada, the U.S., and other countries are right to be concerned.

​

Highlights

  • - We analyze childhood use of ADHD medications in a province that expanded drug coverage.
  • - Use of stimulants increased without improving medium or long-term outcomes.
  • - There was no improvement even in children with the worst pre-existing ADHD symptoms.
  • - Increases in drug use were associated with increases in depression among girls.

Supplementary Material

sup1

Acknowledgments

We are grateful to the Canadian Institutes of Health Research for financial support and to seminar participants at the Aarhus Workshop on Children's Human Capital, Uppsala University, Sciences-Po, McGill, McMaster, UCLA, Portland State University, and Columbia for helpful comments.

Footnotes

1Schwarz and Cohen tabulate data from the 2011-2012 wave of the National Survey of Children's Health. Methylphenidate (sold under the trade names Ritalin, Biphentin, and Concerta) is the most commonly used central nervous system stimulant for ADHD. Others include: dextroamphetamine (Dexedrine); and mixed amphetamine salts (Adderall) (Therapeutics Initiative, 2010).

2Another issue is that the authors define the start of therapy to be the first prescription after a period of at least 11 months without previous prescriptions for ADHD. This suggests that some of the “later starters” may in fact have started ADHD drugs earlier and then stopped them again.

3 adopt a somewhat similar identification strategy, examining the effects of increases in access to medication that occurred as a result of expansions of Medicaid coverage on rates of suicide, injury, and crime in eligible populations.

4Quebec implemented a subsidized day care program in September of that same year. In the first few years the program focused on older children (4-6) and expanded to include younger children later on (). To ensure that our instrument is not conflating the two programs we replicate our estimates focusing on children who are older than the day care ranges by the time the daycare program took place. Our main results are quite similar in this specification.

5The NPHS (1994, 1996 and 1998) and CCHS (2002 and CCHS (2003) are both publicly available data sets that ask questions about prescription drug coverage. The NLSCY, the main source of data for our analysis does not ask questions on prescription drug coverage.

6Quebec's public plan formulary explicitly lists Ritalin as covered. The reimbursement for the drug the price for 100 20mg tablets was $53.06.

7Most of the short-term behavioral outcomes are only collected at ages 2 to 11 years. The educational outcomes are only available for school-aged children, and thus are collected starting at age 6. The question assessing the quality of the child's relationships are asked for children aged 4-9.

8Although there are many fewer children with other medications, we also checked to see whether there were increases in the fractions of children taking tranquilizers or drugs for epilepsy. We did not find any increases in these categories.

9The Quebec formulary for 2012 lists the price of an inhaler as $5 for 200 doses compared to $106. for 200 20mg tablets of Ritalin().

10The relationship questions are indicators that equal 1 if the PMK has reported that the child has gotten along with the person in question “quite well” or “very well” over the previous six month period.

11The NLSCY began collecting a reading test score in its first three cycles but dropped this measure in subsequent cycles. With regard to the mathematics test, as we explored differences between boys and girls we verified that the distribution of test scores is almost identical for the two groups.

12Currie and Stabile (2007) show non-parametric Lowess plots which indicate that short-term test scores and grade repetition vary approximately linearly with ADHD scores, and that the 90th percentile of the ADHD score (which corresponds approximately to a threshold for diagnosis) is similar in Canada and the U.S. We therefore use linear average ADHD scores in our analysis.

13We use an indicator for asthma diagnoses as the third difference in this robustness check.

14The NLSCY dataset does not include information on special education.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributor Information

Janet Currie, Princeton University and NBER.

Mark Stabile, University of Toronto and NBER.

Lauren Jones, Cornell University.

References

  • Aizer A. Peer Effects, Institutions, and Human Capital Accumulation: The Externalities of ADD. NBER Working Paper # 14354. 2009
  • American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th. 1994.
  • Baker Michael, Gruber Jonathan, Milligan Kevin. Journal of Political Economy. 4. Vol. 116. University of Chicago Press; 2008. Universal Child Care, Maternal Labor Supply, and Family Well-Being; pp. 709–745.
  • Barbaresi WJ, Katusic SK, Colligan RC, Weaver AL, Jacobsen SJ. Modifiers of Long-Term School Outcomes for Children with Attention-Deficit/Hyperactivity Disorder: Does Treatment with Stimulant Medication Make a Difference? Results from a Population Based Study. J Dev Behav Pediatr. 28(4):2007, 274–287. [PubMed]
  • Bedard A, Jain U, Johnson SH, Tannock R. Effects of Methylphenidate on Working Memory Components: Influence of Measurement. J Child Psychol Psychiatry. 2007;28(9):872–880. [PubMed]
  • Cuellar Alison, Markowitz Sara. Medicaid Policy Changes in Mental Health Care and Their Effect on Mental Health Outcomes. Health Economics, Policy and Law. 2007;2(1):23–49. [PubMed]
  • Currie Janet, Kahn Robert. Children with Disabilities. 1. Vol. 22. Princeton: The Future of Children; 2012. Children with Disabilities: Introducing the Issue.
  • Currie Janet, Stabile Mark. Child Mental Health and Human Capital Accumulation: The Case of ADHD. Journal of Health Economics. 2006 Nov;25(6):1094–1118. [PubMed]
  • Currie Janet, Stabile Mark. Mental Health in Childhood and Human Capital. In: Jonathan Gruber., editor. Perspectives on the Problems of Disadvantaged Youth. Chicago: University of Chicago Press for NBER; 2009.
  • Centers for Disease Control and Prevention. Mental Health in the United States: Prevalence of Diagnosis and Medication Treatmetn for Attention-Deficit/Hyperactivity Disorder: United States, 2003. MMWR Morb Mortal Wkly Rep. 2005;54(34):842–847. [PubMed]
  • Dalsgaard S, Skyt Nielson H, Simonsen M. Long-term Consequences of ADHD Medication Use for Children's Human Capital Devleopment. Aarhus University Working Paper. 2013
  • Douglas VI. Cognitive Control Processes in Attention-Deficit/Hyperactivity Disorder. In: Quay HC, Hogan AE, editors. Handbook of Disruptive Behavior Disorders. New York, NY: Plenum Press; 1999.
  • Evans SW, Pelahm WE, Smith BH, et al. Dose-Response Effects of Methylphenidate on Ecologically Valid Measures of Academic Performance and Classroom Behavior in Adolescents with ADHD. Exp Clin Psychopharmacol. 2001;9(2):163–175. [PubMed]
  • Fletcher Jason, Wolfe Barbara. Child Mental Health and Human Capital Accumulation: The Case of ADHD Revisited. Journal of Health Economics. 2008;27(3):794–800. [PubMed]
  • Fletcher Jason, Wolfe Barbara. Long-Term Consequences of Childhood ADHD on Criminal Activities. The Journal of Mental Health Policy and Economics. 2009;12:119–138. [PMC free article] [PubMed]
  • Gouvernment du Quebec. La Politique du Medicament. Gouvernment du Quebec; 2007. www.msss.gouv.qc.ca.
  • Gouvernment du Quebec. Regulation Respecting the List of medications covered by Quebec's basic prescription drug plan. [last accessed May 6, 2014]; http://www.ramq.gouv.qc.ca/en/regie/legal-publications/Pages/list-medications.aspx.
  • Griffin Susan, Weatherly Helen, Richardson Gerry, Drummond Mike. Methodlogical Issues in Undertaking Independent Cost-Effectiveness Analysis for NICE: The Case of Therapies for ADHD. The European Journal of Health Economics. 2008;9(2):137–145. [PubMed]
  • Joshi SV, Adam HM. ADHD, Growth Deficits, and Relationships to Psychostimulant Use. Pediatr Rev. 2002;23(2):67–68. [PubMed]
  • Miech Richard, Caspi Avshalom, Moffitt Terrie, Wright Bradley Entner, Silva Phil. Low Socioeconomic Status and Mental Disorders: A Longitudinal Study of Selection and Causation During Young Adulthood. American Journal of Sociology. 1999;104:1096–1131.
  • Molina BS, Hinshaw SP, Swanson JM, et al. The MTA at 8 Years: Prospective Follow-Up of Children Treated for Combined Type ADHD in a Multisite Study. J Am Acad Child Adolesc Psychiatry. 2009;48(5):484–500. [PMC free article] [PubMed]
  • Morgan Steve. Quebec's Drug Insurance Plan: A Prescription for Canada? Centre for Health Services and Policy Research, University of British Columbia, Working Paper HPRU:98:2D. 1998 Feb;
  • The MTA Cooperative Group. A 14-Month Randomized Clinical Trial of Treatment Strategies for Attention-Deficit/Hyperactivity Disorder. Arch Gen Psychiatry. 1999;56:1073–1086. [PubMed]
  • Nagin Daniel, Tremblay Richard E. Trajectories of Boys' Physical Aggression, Opposition, and Hyperactivity on the Path to Physically Violent and Nonviolent Juvenile Delinquency. Child Development. 1999;70:1181–1196. [PubMed]
  • Schwarz A. The Selling of Attention Deficit Disorder. New York Times. 2013 Dec;14
  • Nissen SE. ADHD Drugs and Cardiovascular Risk. N Engl J Med. 2006;354(14):1445–1448. [PubMed]
  • Pelham WE, Carlson C, Sams SE, Vallano G, Dixon MJ, Hoza B. Separate and Combined Effects of Methylphenidate and Behavior Modification on Boys with Attention Deficit-Hyperactivity Disorder in the Classroom. J Consult Clin Psychol. 1993;61(1):506–515. [PubMed]
  • Polanczyk G, De Lima MS, Horta BL, Biederman J, Rohde LA. TheWorldwide Prevalence of ADHD: A Systematic Review and Metaregression Analysis. American Journal of Psychiatry. 2007;164(6):942–948. [PubMed]
  • Pomey Marie Pascale, Forest PG, Palley H, Elisabeth Martin. Public/Private Partnerships for Prescription Drug Coverage: Policy Formulation and Outcomes in Quebec's Universal Drug Insurance Program, with Comparisons to the Medicare Prescription Drug Program in the United States. The Millbank Quarterly. 2007;85(3):469–498. [PMC free article] [PubMed]
  • Schachter HM, Pahm B, King J, et al. How Efficacious and Safe is Short-Acting Methylphenidate for the Treatment of Attention-Deficit Disorder in Children and Adolescents? A Meta-Analysis. CMAJ. 2001;165:1475–1488. [PMC free article] [PubMed]
  • Scheffler Richard M, Brown Timothy, Fulton Brent, Hinshaw Stephen, Levine Peter, Stone Susan. Positive Association Between Attention-Deficit/Hyperactivity Disorder Medication Use and Academic Achievement During Elementary School. Pediatrics. 2009;123:1273–1279. [PubMed]
  • Schwarz Alan, Cohen Sarah. ADHD Seen in 11 percent of U.S. Children as Diagnoses Rise. New York Times. 2013 Mar 31;
  • Therapeutics Initiative. What is the Evidence for Using CNS Stimulants to Treat ADHD in Children? Therapeutics Letter. 2008 Mar-May;69
  • U.S. National Institute of Mental Health. Attention Deficit Hyperactivity Disorder. [accessed August 2012]; http://www.nimh.nih.gov/health/publications/attention-deficit-hyperactivity-disorder/medications.shtml.
  • Wigal Tim, Swanson James, Regino Roland, Lerner Marc, Soliman Ihab, Steinhoff Ken, Gurbani Suresh, Wigal Sharon. Stimulant Medications for the Treatment of ADHD: Efficacy and Limitations. Mental Retardation and Developmental Disabilities Research Reviews. 1999;5:215–224.
  • Zoega Helga, Rothman Kenneth, Huybrechts Krista, Olafsson Orn, Baldursson Gisli, Almarsdottir Anna, Jonsdottir Solveig, Haldorsson Matthias, Hernandez-Diaz Sonia, Valdimarsdottir Unnur. A Population Based Study of Stimulant Drug Treatment of ADHD and Academic Progress in Children. Pediatrics. 2012 Jul;130(1):e53–e62. [PubMed]

Formats:

Save items

Similar articles in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

Recent Activity

ClearTurn OffTurn On

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...