Abstract
Objective
New ADHD subtypes identified through latent class analysis have been recently proposed. Here, we assess the accuracy of simple rules based on symptom counts for the assignment of youths to clinically relevant population-derived ADHD subtypes: severe inattentive (SI) and severe combined (SC).
Method
Data from 9,675 twins and siblings from Missouri and Australia aged 7-19 years were analyzed using continuous and categorical models of ADHD symptomatology using principal components analysis (PCA) and subtyping by DSM-IV and by latent class criteria. Cut-points were derived for classifying SI and SC subtypes by positive predictive value, negative predictive value, percent positive agreement, and Matthews’ coefficient of agreement.
Results
PCA suggested 2 underlying factors: total number of symptoms and symptom type, with SI and SC latent class subtypes clearly mapping to distinct areas on a plot of these factors. Having 6 or more total symptoms and fewer than 3 hyperactive-impulsive symptoms accurately predicts the latent class SI subtype. The latent class SC subtype was best identified by 11 or more total symptoms and 4 or more hyperactive-impulsive. DSM-IV ADHD subtype criteria accurately identified the SC subtype, but only poorly for the SI subtype.
Conclusions
Symptom counts criteria allow the simple and accurate identification of subjects with severe ADHD subtypes defined by LCA. Such simple symptom counts corresponding to screening cut-points selected latent class derived SI subtype subjects with greater precision than DSM-IV criteria.
Keywords: ADHD, subtype, clinical
Introduction
Current diagnostic criteria set forth by the Diagnostic and Statistical Manual of Mental Disorders, Version IV (DSM-IV) divide ADHD into three subtypes: predominantly inattentive, predominantly hyperactive-impulsive and combined, based on the presence of at least 6 of 9 inattentive symptoms, 6 of 9 hyperactive/impulsive symptoms, or both.1 Additional diagnostic criteria include impairment at school or home before age 7 and absence of other diagnoses that could explain the ADHD symptoms, such as autism. The definition of these DSM-IV thresholds incorporated measures of academic and social impairment, Children's Global Assessment Scale score and clinician DSM-III-R ADHD diagnosis.2, 3
Original psychometric analyses of the 18 ADHD symptoms with factor analysis identified two highly correlated underlying factors (inattention and hyperactivity/impulsivity).4 In contrast to factor analysis, which assumes a continuum of disease, latent class analysis (LCA) is a version of mixture modeling that aims to separate individuals into discrete, phenotypically homogenous subgroups. From the standard 18 ADHD symptoms, LCA consistently identifies 7 to 8 highly heritable and distinct classes in several population-based samples5-7: mild and severe inattentive subtypes, mild and severe combined subtypes, mild and severe hyperactive-impulsive subtypes, a group of subjects with few symptoms, and a final, less defined group. Three of these latent classes (severe inattentive, severe hyperactive-impulsive, and severe combined) demonstrate modest (severe inattentive) to excellent (severe combined) agreement with the similarly named DSM-IV ADHD subtype.5-9
Compared to DSM-IV ADHD subtypes, the population based LCA subtypes could offer several advantages, in particular in being more phenotypically homogeneous and genetically distinct.8, 9 Agreement is high across maternal, paternal and teacher reports, especially for the severe classes.10 Recently, we showed that LCA-derived subtypes are more stable at five-year follow up than those in DSM-IV11 and that the DSM-IV age of onset before 7 years criterion is unreliable.11 The severe inattentive subtype is more homogeneous, averaging fewer than 2 endorsed hyperactive-impulsive symptoms, while the DSM-IV definition of the predominately inattentive type allows up to 5. As with their DSM-IV counterparts, the LCA severe inattentive and severe combined subtypes are associated with marked impairment in academic and social settings. As judged by Child Behavior Checklist (CBCL) criteria 12, these subtypes are characterized by increased attention problems and social problems subscales.13 The severe inattentive and combined subtypes are more likely to have poor grades and to have had an Individual Education Plan (school program to address specific special education needs) compared to DSM-IV and other population-defined subtypes.14 These ADHD subtypes also have high levels of autistic symptoms and motor problems, especially the severe combined type.15, 16
In the present study, we provide a direct comparison of DSM-IV and LCA subtypes, with a view toward addressing a shortcoming to the adoption of this alternative classification approach in clinical practice: translating the many possible symptom endorsement profiles of the 18 DSM-IV ADHD symptoms to specific ADHD subtype assignments. For LCA-derived subtypes to be useful clinically, the main motivation for such research, the novel subtypes must be easy to identify. We thus describe the utility of a simple ADHD symptom counting approach to assigning youth to severe ADHD subtypes for the DSM-IV and latent class classification systems. The findings suggest that revisions in nosology, such as anticipated in DSM-V, can incorporate results from a variety of approaches that produce simple assignment algorithms practical for clinical applications.
Method
Description of Samples
Data for this study came from three distinct population-based samples (Table 1): The population-representative MOAFTS (Missouri Adolescent Female Twin Study) examined risk factors for alcoholism in young women and adolescents; MOTWIN (Missouri Twin Study) identified both male and female twins from the Missouri Twin Registry for a genetic epidemiological study of ADHD; and ATAP (Australian Twin ADHD Project) is a large mail survey of child and adolescent twins and siblings participating in the Australian National Health and Medical Research Council Twin Registry. Only subjects with complete information on all 18 ADHD symptoms were included in the present analyses, for a final sample of 9,675 individuals in 4,323 families. Half the subjects from each twin pair or family were randomly assigned to a training set (N = 4,838) to develop the classification rule and the remainder to a validation set (N=4,837).
Table 1.
Sample Characteristics
| Australian Twin ADHD Project (ATAP) (N=3,984) | Missouri Twin Study (MOTWIN) (N=1,521) | Missouri Adolescent Female Twin Study (MOAFT) (N=4,170) | |
|---|---|---|---|
| Country of Origin | Australia | US- Missouri | US- Missouri |
| Assessment Method | Mailed Questionnaire | Semi-structured Phone Interview | Semi-structured Phone Interview |
| Reporter | Parent | Parent | Parent |
| Sex | |||
| Males | 1,981 | 991 | 0 |
| Females | 2,000 | 530 | 4,168 |
| Missing | 3 | 0 | 0 |
| Age (years) | |||
| Mean ± SD | 11.25±2.93 | 12.20 ± 3.29 | 14.86 ± 2.67 |
| Range | 4-19 | 7-19 | 10-23 |
| Latent Class Subtype | |||
| Few Symptoms | 2,404 | 865 | 2,320 |
| Mild Inattentive | 453 | 203 | 589 |
| Talkative-Impulsive | 315 | 68 | 362 |
| Mild Combined | 368 | 139 | 276 |
| Mild Hyperactive | 88 | 55 | 200 |
| Severe Combined | 179 | 125 | 143 |
| Severe Inattentive | 112 | 100 | 167 |
| Severe Hyperactive | 67 | 26 | 113 |
| DSM-IV Subtype | |||
| No ADHD | 3,465 | 1,181 | 3,639 |
| Inattentive | 268 | 195 | 292 |
| Hyperactive/Impulsive | 102 | 41 | 124 |
| Combined | 149 | 104 | 115 |
*** Table 1 about here ***
Missouri Adolescent Female Twin Study
The 4,212 participants in the Missouri Adolescent Female Twin Study (MOAFTS) were identified through a database of all live-birth twin pairs born in Missouri between 1968 and 1997.17 All individuals (N = 4170 out of 4,212) with complete information on the 18 DSM-IV ADHD symptoms as provided in parent interviews about their children, were included in this analysis. Assessments were conducted by telephone using the Semi-Structured Assessment for the Genetics of Alcoholism (SSAGA) which exhibits good reliability for all disorders assessed, including ADHD.18
Missouri Twin Study
The Missouri Twin Study (MOTWIN) is composed of 1,630 male and female twins identified from the Missouri Twin Registry from 1996 through 2002 as part of a genetic epidemiological study of ADHD. Of these, 70 twins also participated in MOAFTS, and 1,521 of the 1,560 unique to MOTWIN had complete information on all 18 DSM-IV ADHD symptoms by parental report. Unlike MOAFTS and ATAP (below) which are population representative samples, a mild selective sampling scheme was applied during the recruitment of MOTWIN families: if either twin exhibited three or more present or past inattentive symptoms on a brief screening interview (N=564 families). In addition, an additional 104 families with a child with a CBCL anxious/withdrawn subscale score in the upper 95th percentile and 183 randomly selected families (across birth years) were enrolled in MOTWIN to address issues of rater bias and to serve as controls, respectively.19
Parental reports on offspring were collected with the Missouri Assessment for Genetic Studies of Children (MAGIC), a semi-structured interview based on DSM-IV diagnoses.20 The MAGIC demonstrates high inter-rater reliability and excellent stability of diagnosis for parental and offspring report of ADHD (alpha ≥ 0.79).20
Australian Twin ADHD Project
The Australian Twin ADHD Project (ATAP) is a large mail survey of child and adolescent twins and siblings from the Australian National Health and Medical Research Council Twin Registry (ATR).21 Typically, ATR participants were enrolled shortly after birth. This sample consists of 3,100 individual twins and 1,275 non-twin siblings from 1,550 families assessed in the second wave of data collection. Of these 4,275 subjects, 3,984 had complete ADHD symptom information by parental report. Information on DSM-IV ADHD symptoms was collected using the Australian Twin Behavior Rating Scale (ATBRS), a maternal report questionnaire assessing the presence or absence of behavior in the prior six months.22 In the ATBRS, ADHD symptoms are rated on a 4-point scale (1= “not at all”, 2= “just a little/sometimes”, 3= “pretty much/often”, 4= “very much/often”). As done in previous studies, 22, 23 these categories were collapsed to correspond with the “absent” (categories 1 and 2) and “present” (3 and 4) DSM-IV categories.
Statistical Methods
Analyses were conducted using Stata Version 10 (College Station, TX). To validate the use of symptom counts in later analyses and provide a template onto which DSM-IV and latent class subtypes can be compared, we first applied principal components analysis (PCA) to the 18 DSM-IV ADHD symptoms, starting from the polychoric correlation matrix and using only observations from the training sample. Extracted factors were not rotated in order to avoid placing any preexisting diagnostic conventions on results. This approach takes into account inter-item correlations both within and between the inattentive and hyperactive-impulsive dimensions.
Second, we used latent class model parameter estimates obtained in previous population studies to assign subjects to their most likely ADHD latent class (eight class solution24). Latent class analysis (LCA) is a variant of mixture modeling that seeks to identify subtypes (“classes”) of individuals based on their responses to a set of correlated items. Based on the results of LCA (probability of item endorsement given class and class prevalence), individuals can be assigned to their most likely latent class.25 Two of the identified classes, SI (severe inattentive) and SC (severe combined) will serve as phenotypes of interest for subsequent analyses. Given its low prevalence, the severe hyperactive-impulsive subtype was not analyzed separately.
Third, we used the training set data to define symptom count thresholds to identify SI and SC subjects. The use of symptom counts was validated by an examination of the relationship between the continuous principal components scores and the discrete assignments to the SI and SC latent classes. The resulting classification was subjected to Receiver Operating Curve (ROC) analysis where area under the curve (AUC) is computed to examine the ability of symptom count cut points to accurately classify subjects based on a trade-off between sensitivity and specificity. The AUC provides a measure of correct classification of disease status but is an imperfect measure of the accuracy of the screening instrument. Therefore, we also calculated the positive predictive value (PPV), negative predictive value (NPV) and percent positive agreement (PPA). Percent positive agreement (PPA) is a variant of percent correctly classified and is the preferred measure to examine the reliability of an instrument when the prevalence of the disease is low.26 The Matthew correlation coefficient27 (φ) is, roughly stated, a summary of these four values and was used as a final measure of agreement between latent class assignment and prediction based on symptom counts. Values of φ range from -1 to 1, with 0 indicating prediction equal to having a diagnosis chosen at random, 1 being perfect prediction and values less than 0 indicating worse than random prediction. As mentioned above, half the subjects from each family were randomly selected to form a training data set and the remaining observations used for validation. The training data alone were used to determine sensitivity, specificity, PPV, and NPV for particular combinations of symptom count cut points. The accuracy of the decision rule was then assessed in the validation set combining across all samples as well as for each study separately.
Results
Sample Characteristics
The final analysis sample (Table 1) consisted of 4,079 individual monozygotic twins, 4,532 individual dizygotic twins, and 1,127 non-twin siblings with a mean age of 12.5 ± 3.3 years at assessment. The predominantly female (69%) characteristic is largely due to MOAFTS being 100% female, while the ATAP and MOTWIN samples exhibit a more equal gender distribution (50% female in ATAP; 35% in MOTWIN).
Overall, 4% of subjects were assigned to the severe inattentive latent class and 4.6% of subjects to the severe combined class. Ninety-nine percent of subjects assigned to the severe inattentive latent class were also assigned to the DSM-IV ADHD predominantly inattentive subtype (Table 2). However, the converse is not true, with only 50% of subjects in the DSM-IV predominantly inattentive subtype assigned to the severe inattentive latent class.6-8 Approximately 77% of the severe combined latent class subjects were assigned to the DSM-IV combined ADHD subtype while 92.9% of the DSM-IV combined subtype subjects were assigned to the severe combined latent class. Notably, subjects in the severe inattentive latent class were found to have a minimum of 7 total symptoms (mean = 8.9 ± 1.5), with 5-9 inattentive symptoms (mean = 7.7 ± 1.0) and 0-3 hyperactive impulsive (mean = 1.2 ± 1.1) symptoms. Compared to severe inattentive cases, severe combined subjects demonstrate a greater total number of symptoms (15.2±1.8, range 11-18, p<0.001); more hyperactive-impulsive symptoms (mean 7.1±1.6, range 4-9 symptoms; p<0.001); but similar numbers or inattentive symptom (8.1±1.0, range 5-9, p=0.61).
Table 2.
Cross-tabulation of DSM-IV and Latent Class ADHD Subtypes
| DSM-IV ADHD Subtype |
|||||
|---|---|---|---|---|---|
| Latent Class Subtype | No ADHD | Inattentive | Hyperactive/Impulsive | Combined | Total |
| Few Symptoms | 5,527 | 0 | 0 | 0 | 5,527 |
| Mild Inattentive | 1,197 | 66 | 0 | 0 | 1,245 |
| Talkative-Impulsive | 736 | 0 | 9 | 0 | 745 |
| Mild Combined | 489 | 210 | 59 | 25 | 783 |
| Mild Hyperactive | 341 | 0 | 2 | 0 | 343 |
| Severe Combined | 0 | 102 | 3 | 342 | 447 |
| Severe Inattentive | 2 | 377 | 0 | 0 | 379 |
| Severe Hyperactive | 11 | 0 | 194 | 1 | 206 |
| Total | 8,285 | 755 | 267 | 368 | 9,675 |
Principal Components Analysis
PCA of parental reports of current ADHD symptoms confirms previous results and suggests two main factors accounting for 62% and 11% of the variance, respectively (Table 3). All 18 symptoms load roughly equally on the first factor, which thus corresponds to total number of symptoms endorsed, irrespective of type (inattentive versus hyperactive-impulsive). The median factor loading is 0.34 for inattention symptoms (range: 0.30 to 0.41) and 0.38 for hyperactive/impulsive symptoms (range: 0.26 to 0.46). On the second factor, all nine inattentive items load negatively (median: -0.32, range: -0.42 to -0.18) while all nine hyperactive-impulsive items have positive factor loadings (median: 0.40, range: -0.27 to 0.48). The second factor thus corresponds to type of symptom endorsed (inattentive vs. hyperactive-impulsive). Having a similar number of inattentive and hyperactive-impulsive symptoms would thus result in a symptom type factor loading close to zero.
Table 3.
Factor loadings for the 18 DSM-IV ADHD Symptoms From Principal Components Analysis
| Factor 1: Number of Symptoms Factor | Factor 2: Symptom Type Factor | |
|---|---|---|
|
Inattention | ||
| Fails to give attention to detail | 0.34 | −0.36 |
| Difficulty sustaining attention | 0.38 | −0.32 |
| Does not seem to listen | 0.41 | −0.25 |
| Does not follow instructions/fails to finish | 0.35 | −0.35 |
| Difficulty organizing | 0.34 | −0.42 |
| Avoids/dislikes tasks that require sustained effort | 0.35 | −0.30 |
| Loses things needed for tasks and activities | 0.33 | −0.30 |
| Is easily distracted by extraneous stimuli | 0.30 | −0.18 |
| Is often forgetful in daily activities | 0.34 | −0.40 |
|
Hyperactivity-Impulsivity |
||
| Fidgets with hands or feet or squirms | 0.38 | 0.28 |
| Often leaves seat when remaining in seat is expected | 0.46 | 0.27 |
| Often runs about or climbs excessively when inappropriate | 0.39 | 0.40 |
| Often has difficulty playing or engaging in leisure activities quietly | 0.39 | 0.41 |
| Is often “on the go” or acts as if “driven by a motor” | 0.32 | 0.44 |
| Often talks excessively | 0.26 | 0.44 |
| Often blurts out answers before questions are completed | 0.37 | 0.48 |
| Often has difficulty waiting turn | 0.38 | 0.37 |
| Often interrupts or intrudes on others | 0.33 | 0.32 |
Because the factor loadings have relatively similar magnitude for all items, there is a simple (though not perfect) relationship between the semi-continuous factor scores and symptom counts, as well as between the factor scores and the discrete DSM-IV or the LCA derived ADHD subtypes. In Figure 1, the vertical axis represents the first PCA factor scores (total number of symptoms) and the horizontal axis, scores for the second factor from the PCA (symptom type). First, the correlation between total symptom count and the first component score is almost perfect (~0.99). Second, subjects assigned to DSM-IV inattentive, hyperactive and combined subtypes cluster in clearly distinct areas. These areas correspond to type and number of symptoms endorsed in a particular group. The severe inattentive class corresponds to the inattentive extreme of the symptom type factor and is positioned midway on the symptom number dimension of the graph. Children with combined subtype ADHD are characterized by a large total number of symptoms and cluster equally between the inattentive and hyperactive-impulsive extremes and at the extreme of the symptom number dimension.
Figure 1.
Plot of the two factor scores for each observed response pattern for the 18 ADHD symptoms, with Factor 1 (overall number of symptoms) on the vertical axis and Factor 2 (symptom type) on the horizontal axis. (a) DSM-IV classification: The two intersecting lines correspond to observations with exactly 6 inattentive (dashed) and 6 hyperactive symptoms (solid) and define four distinct regions corresponding to the DSM-IV subtypes. Because the factor loadings are relatively similar across all items, the regions are clearly delineated. To reflect the characteristics of the sample, these lines were derived using two separate lowess regressions of Factor 1 score on Factor 2 score: one with all observations with exactly 6 inattentive symptoms, and the other with all observations with exactly six inattentive/impulsive symptoms. (b) Latent class subtypes: A similar plot for the latent class nosology: severe inattentive (green), severe combined (red), severe hyperactive (blue) and the other 5 classes (orange). The DSM-IV thresholds from Figure 1a are repeated as dashed lines for completeness. Solid lines outline the regions defined by the proposed cut-offs. The region for severe combined is at the top and center of the graph, with 11 or more total symptoms (horizontal line) and at least 4 hyperactive (sloped line). The region for severe inattentive is middle and left, with 6 or more total symptoms (horizontal) and less than 3 hyperactive (sloped). The relationship between latent class assignment and the factor scores is more complex. As discussed in the text, the prediction is not perfect and the defined area for, e.g., the severe combined class will capture some observations from the pure hyperactive class.
The relationship between factor scores and latent class assignment is not as simple. Unlike DSM-IV subtypes which are defined directly from symptom counts, latent class assignments are derived from probabilities of symptom endorsements. However, a similar clustering is observed, indicating that the latent classes under consideration do map to an elevated level of severity.
Determination of Symptom Count Thresholds
DSM-IV and LCA defined subtypes, thus target similar regions of the severity spectrum (as defined by factor scores), which requires a redefinition of thresholds if symptom counts are to be used. Tables 4 and 5 summarize the results of ROC analyses for the severe inattentive and severe combined latent class subtypes, respectively conducted using the training set data. The SI class averages very few hyperactive-impulsive symptoms. A classification rule with less than 3 hyperactive-impulsive symptoms and an overall total of 6 or more symptoms fairly accurately identifies the severe inattentive subtype on all indicators (PPV=76.3%) and produces a high rate of correct classification of affected subjects (PPA=81.1%, φ=0.80). These cut-points perform equally well in the validation set, across genders and studies, though slightly less so for ATAP males. Note that the SI class is poorly identified by DSM-IV thresholds for predominantly inattentive subjects (6 or more inattentive symptoms, fewer than 6 hyperactive/impulsive symptoms): while the sensitivity is high (99%), the PPV is low (34%) (Table 4).
Table 4.
Calculated Area Under the Curve (AUC), Sensitivity, Specificity, Positive Predictive Value (PPV), Negative Predictive Value (NPV), Percent Positive Agreement (PPA), and Matthew's Correlation Coefficient (Phi) for the Severe Inattentive ADHD Latent Class Subtype
|
Severe Inattentive Latent Class- Training Set (n=4,838 in Training Set, n=193 SIA in Training Set) Empirically Derived Cut-points
| |||||||
|---|---|---|---|---|---|---|---|
| AUC | Sensitivity | Specificity | PPV | NPV | PPA | Phi | |
| ≥ 6 total, <3 HY | 0.93 | 86.5% | 98.9% | 76.3% | 99.4% | 81.1% | 0.80 |
| ≥ 6 total, <4 HY | 0.98 | 98.5% | 97.1% | 58.1% | 99.9% | 73.1% | 0.74 |
| ≥ 6 total, <5 HY | 0.97 | 100% | 94.6% | 43.7% | 100% | 60.8% | 0.59 |
| ≥ 7 total, <3 HY | 0.82 | 63.7% | 99.6% | 86.6% | 98.5% | 73.4% | 0.73 |
| ≥ 7 total, <4 HY | 0.87 | 75.7% | 98.5% | 68.2% | 98.9% | 71.7% | 0.71 |
| ≥ 7 total, <5 HY | 0.87 | 77.2% | 96.6% | 48.4% | 99.0% | 59.5% | 0.59 |
| ≥ 8 total, <3 HY | 0.71 | 42.5% | 100% | 97.6% | 97.7% | 59.2% | 0.77 |
| ≥ 8 total, <4 HY | 0.77 | 54.4% | 99.3% | 75.5% | 98.1% | 63.3% | 0.63 |
| ≥ 8 total, <5 HY | 0.77 | 56.0% | 97.8% | 51.2% | 98.2% | 53.5% | 0.51 |
| ≥ 9 total, <3 HY | 0.60 | 19.7% | 100% | 100% | 96.8% | 32.9% | 0.44 |
| ≥ 9 total, <4 HY | 0.65 | 31.1% | 99.6% | 77.9% | 97.2% | 44.4% | 0.51 |
| ≥ 9 total, <5 HY | 0.66 | 32.6% | 98.6% | 48.5% | 97.2% | 39.0% | 0.38 |
| DSM-IV Predominantly Inattentive Subtype Criteria – ≥6 IA, ≤ 5 HY | |||||||
| Training set | 0.96 | 99.5% | 92.0% | 34.2% | 100% | 50.9% | 0.56 |
| Validation set | 0.96 | 99.5% | 91.9% | 32.9% | 99.9% | 49.5% | 0.55 |
|
Severe Inattentive Latent Class- Validation Set (n=4,837 in Validation Set, n=186 SIA in Validation Set) ≥ 6 total, <3 HY rule
| |||||||
|---|---|---|---|---|---|---|---|
| AUC | Sensitivity | Specificity | PPV | NPV | PPA | Phi | |
| Combined Validation Set | 0.91 | 82.8% | 99.1% | 78.6% | 99.3% | 80.6% | 0.80 |
| MOAFTS (n=76 SIA) | 0.93 | 86.8% | 99.2% | 79.5% | 99.5% | 83.0% | 0.82 |
| ATAP Males (n=36 SIA) | 0.87 | 75.0% | 98.6% | 65.9% | 99.1% | 70.1% | 0.69 |
| ATAP Females (n=21 SIA) | 0.83 | 66.7% | 99.6% | 77.8% | 99.3% | 71.8% | 0.71 |
| MOTWIN Males (n=42 SIA) | 0.96 | 92.9% | 98.9% | 88.6% | 99.3% | 90.7% | 0.90 |
| MOTWIN Females (n=11 SIA) | 0.86 | 72.7% | 99.2% | 80.0% | 98.9% | 76.2% | 0.75 |
IA = inattentive symptom, HY = hyperactive/impulsive symptom, SIA = Severe Inattentive Subtype, ATAP = Australian Twin ADHD Project, MOTWIN = Missouri Twin Study, MOAFT = Missouri Adolescent Female Twin Study
Table 5.
Calculated Area Under the Curve (AUC), Sensitivity, Specificity, Positive Predictive Value (PPV), Negative Predictive Value (NPV), and Percent Positive Agreement (PPA) for the Severe Combined ADHD Latent Class Subtype
|
Severe Combined Latent Class- Training Set (n=4,838 in Training Set, n=221 SCMB in Training Set) Empirically Derived Cut-points
| |||||||
|---|---|---|---|---|---|---|---|
| AUC | Sensitivity | Specificity | PPV | NPV | PPA | Phi | |
| ≥ 11 total, ≥4 HY | 0.99 | 99.6% | 99.0% | 83.0% | 99.9% | 90.5% | 0.90 |
| ≥ 12 total, ≥4 HY | 0.97 | 94.6% | 99.7% | 93.7% | 99.7% | 94.1% | 0.94 |
| ≥ 13 total, ≥4 HY | 0.90 | 80.5% | 99.9% | 98.9% | 99.1% | 88.8% | 0.89 |
| ≥ 14 total, ≥4 HY | 0.80 | 59.7% | 100% | 100% | 98.1% | 74.8% | 0.77 |
| DSM-IV Combined Subtype Criteria≥6 IA, ≥6 HY | |||||||
| Training set | 0.99 | 100% | 99.71% | 93.44% | 100% | 96.6% | 0.97 |
| Validation set | 0.99 | 100% | 99.66% | 92.43% | 100% | 85.9% | 0.96 |
|
Severe Combined Latent Class- Validation Set (n=4,837 in Validation Set, n=226 SCMB in Validation Set) ≥ 11 total, ≥4 HY | |||||||
|---|---|---|---|---|---|---|---|
| AUC | Sensitivity | Specificity | PPV | NPV | PPA | Phi | |
| Combined Validation Set (n=226 SCMB) | 0.99 | 99.8% | 98.9% | 81.5% | 99.9% | 89.7% | 0.89 |
| MOAFTS (n=74 SCMB) | 0.99 | 100% | 98.7% | 73.3% | 100% | 86.9% | 0.85 |
| ATAP Males (n=62 SCMB) | 0.99 | 100% | 98.5% | 81.6% | 100% | 90.8% | 0.90 |
| ATAP Females (n=22 SCMB) | 0.99 | 100% | 99.2% | 73.3% | 100% | 85.7% | 0.85 |
| MOTWIN Males (n=55 SCMB) | 0.99 | 100% | 98.8% | 91.7% | 100% | 93.8% | 0.95 |
| MOTWIN Females (n=13 SCMB) | 0.99 | 100% | 99.6% | 92.9% | 100% | 98.1% | 0.96 |
IA = inattentive symptom, HY = hyperactive/impulsive symptom, SCMB= Severe Combined Subtype, ATAP = Australian Twin ADHD Project, MOTWIN = Missouri Twin Study, MOAFT = Missouri Adolescent Female Twin Study
With regards to the severe combined subtype, a cut-off of 11 or more total symptoms of which 4 or more are hyperactive-impulsive symptoms, results in high values for all screening accuracy measures (Table 5). A slightly higher percent age of affected subjects is correctly classified when raising the total number of symptoms threshold to 12 or more (PPA=94%, φ=0.94) while keeping the criterion of 4 or more hyperactive-impulsive symptoms, but the benefits are minimal. A cut-point calculated using the DSM-IV combined subtype criteria (6 or more inattentive and 6 or more hyperactive-impulsive symptoms) demonstrated perfect sensitivity and NPV. Replication of these analyses in the validation set produced nearly identical results (Table 5). In summary, subjects were correctly classified as severe inattentive if they endorse 6 or more total symptoms of which 4 or more are inattentive. Membership in the severe combined class was determined by the presence of 11 or more total symptoms of which 4 or more must be of the hyperactive-impulsive type.
Discussion
These results show that it is possible to identify clinically relevant latent class ADHD subtypes using symptom counts only (Tables 4 and 5). Symptom utility estimates from the DSM-IV field trials indicated similar predictive ability for all 18 ADHD symptoms. Inattentive and hyperactive-impulsive items were found to better identify subjects with those named characteristics (ex: inattentive symptom 1 with primarily inattentive type) than cross-dimension (ex: inattentive symptom 1 with primarily hyperactive/impulsive type).28 This would argue for the retention of all 18 symptoms in future revisions of diagnostic schedules, such as DSM-V or ICD-11. The present analysis confirms recommendation, as the factor analysis found that all symptoms have similar factor loadings. This allows the derivation of a simple symptom count algorithm to identify ADHD subtypes derived from more complex statistical approaches and their translation into criteria that are easily applicable to real life clinical situations. Briefly, a total of 6 or more symptoms, of which fewer than three are hyperactive-impulsive, identify the severe inattentive subtype. A total of eleven or more, with 4 or more hyperactive-impulsive symptoms identify the severe combined subtype. Fewer than 6 total symptoms identify the absence of clinically significant ADHD. Consideration of these findings may help increase the utility of future diagnostic schema for ADHD.
The results presented here expand on earlier efforts to identify, in population samples, homogeneous ADHD subtypes with significant clinical impairments. The correspondence between the latent class defined ADHD subtypes and the corresponding DSM-IV subtypes is imperfect (Table 2). On the one hand, severe inattentive latent class subjects are almost perfectly classified as predominantly inattentive subtype by DSM-IV criteria (N=377 out of 379). On the other, of the 755 DSM-IV predominantly inattentive subtype subjects in our sample, a large number fall into the severe combined, mild combined and mild inattentive latent classes (N = 102, 210 and 66, respectively). The DSM-IV predominantly inattentive type allows endorsement of up to 5 hyperactive-impulsive symptoms without meeting criterion, while the latent class severe inattentive definition allows only up to two. The ratio of inattentive to hyperactive-impulsive symptoms appears to be important as the “pure” inattentive subjects endorsed fewer hyperactive-impulsive symptoms than the DSM-IV predominately inattentive subtype. Many of the subjects classified as mild combined by LCA and members of a clinically relevant subtype would have been diagnosed as predominantly inattentive by DSM-IV criteria, resulting in a less homogenous group. This may explain in part why twin studies suggest that the severe inattentive latent class is a genetically distinct subtype of ADHD, while results for the DSM-IV predominately inattentive subtype are equivocal.8, 24 Using population-defined criteria for the severe inattentive subtype may be especially valuable in assessing genetic risk factors for this disease. The alternative cut-points presented here capitalize on this difference and provide a method to identify this under-diagnosed, yet impaired, subtype.13
Principal Component Analysis of this very large sample suggests two factors (symptom number and symptom type) in the DSM-IV ADHD symptom structure (Table 3). This is fully consistent with previous factor analyses of DSM-IV ADHD symptom data that suggested two correlated factors, one inattentive and one hyperactive-impulsive. 4 A simple rotation of the uncorrelated symptom number factor and the type factor recreates the correlated inattention and hyperactive- impulsive factors described previously. The use of the uncorrelated factor structure rather than the previously identified correlated inattentive and hyperactive factors allows easy visual identification of ADHD subtypes by either classification. Subjects belonging to the clinically relevant and markedly impaired severe inattentive and severe combined latent classes cluster in distinct areas on a plot of these two underlying factors.
This is not the first empirical study of ADHD nosology. Faraone and colleagues used ROC analysis to examine the efficiency of diagnostic criteria creating limited symptom clusters for DSM-III-R ADD subjects versus subjects with ADD and comorbidity.29 Similar methods have been employed to predict impairment in subjects with DSM-IV ADHD.30
Limitations of this study include using only symptoms from the DSM-IV definition of ADHD. There may be additional symptoms that would further discriminate between syndrome appearance and severity. For example, items describing sluggish cognitive tempo (ex: daydreams) were included as potential ADHD symptoms during the DSM-IV field trials. The inclusion of sluggish cognitive tempo symptoms does not significantly alter the structure of the latent class derived ADHD subtypes.31 Additionally, we did not include symptoms of disorders commonly comorbid with ADHD in this analysis since previous work did not indicate differential clustering by ADHD latent class subtype.9
Another limitation of the study is that, due to its low prevalence, we could not meaningfully analyze the predominately hyperactive-impulsive ADHD subtype. No age or sex specific criteria were incorporated in the development of ADHD subtypes for both nosologies, but our results indicate that the classification rule remains accurate across gender, country and method of data collection.
Further, the data come from three twin and family samples. Inclusion of non-twin siblings and random allocation of one number of each family to a training set and the remainder to validation sets in the current analyses helps avoid biases in results due to correlations between twins and siblings. While the prevalence of DSM-IV and latent class subtypes in twins may vary from the general population, no comparable studies have been conducted in non-twin samples. As noted in Neuman et al19 the prevalences of ADHD subtypes in Australian non-twin siblings of twins are similar to those reported in Missouri twins.7, 19, 24 Similarly, no differences in the prevalence or familiality of these ADHD subtypes were found between non-twin sibling pairs and dizygotic twins in the Australian sample.24 Additionally, the prevalences of ADHD and individual ADHD subtypes in the ATAP sample were similar to those in an Australian national epidemiologic sample. Finally, the MOTWIN study is slightly enriched with DSM-IV primarily inattentive subtype subjects. The effects on the present results are limited since the screening instrument used for selection demonstrated low PPV 19 and the key step of assigning subjects to a particular latent class was done using parameter estimates derived from population representative samples.
Despite these limitations, our study has several strengths. These analyses were conducted on three large, international population-based samples of subjects from Missouri and Australia. Overall, this sample is representative of the twin populations and the data were collected using highly reliable diagnostic interviews (MOAFT and MOTWIN) and questionnaires (ATAP). Because the standard 18 ADHD symptoms from the DSM-IV ADHD diagnosis were used to create alternative symptom cutoffs, our analysis can thus easily be replicated by others. As demonstrated in the present study, clinically significant ADHD latent class subtypes can be assigned by both researchers and clinicians using parent report of the current presence or absence of the 18 DSM-IV symptoms that are available on a variety of widely used screening questionnaires and diagnostic interviews.
Acknowledgments
The authors gratefully acknowledge the support of NIH grants MH52813(RDT); MH083823(AAT) and ES013678, MH074272(HEV). The Australian work has been supported by the NHMRC (Australia) and the Australian Twin Registry.
The authors thank Andrew C. Heath for access to the Missouri Adolescent Female Twin Study (MOAFTS) dataset and the Missouri Department of Vital Statistics for access to Missouri Birth Records. This work was conceived and conducted under the guidance of Dr. Richard D. Todd, who unfortunately passed away from complications of leukemia.
Footnotes
Disclosure: Dr. Hay has received funding for expenses by Janssen-Cilag to present at their unrestricted educational seminars, and by Janssen-Cilag, Eli Lilly, UCB, and Shire at their unrestricted meeting of the International Collaboration on ADHD and Substance Abuse. No fees were involved. The other authors report no conflicts of interest.
Contributor Information
Heather E. Volk, Division of Biostatistics, Department of Preventive Medicine, Keck School of Medicine, USC, Los Angeles, CA..
Alexandre A. Todorov, Department of Psychiatry, Washington University School of Medicine, St. Louis, MO..
David A. Hay, School of Psychology, Curtin University of Technology, Perth, Australia..
Richard D. Todd, Department of Psychiatry and Department of Genetics at Washington University School of Medicine, St. Louis MO..
References
- 1.American Psychiatric A . Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) American Psychiatric Association; Washington, D.C.: 1994. [Google Scholar]
- 2.Setterberg S, Bird H, Gould M. Parent and Interviewer Version of the Children's Global Assessment Scale. Columbia University; New York: 1992. [Google Scholar]
- 3.Lahey BB, Applegate B, McBurnett K, et al. DSM-IV field trials for attention deficit hyperactivity disorder in children and adolescents. Am J Psychiatry. 1994;151:1673–1685. doi: 10.1176/ajp.151.11.1673. [DOI] [PubMed] [Google Scholar]
- 4.McBurnett K, Pfiffner LJ, Frick PJ. Symptom properties as a function of ADHD type: an argument for continued study of sluggish cognitive tempo. J Abnorm Child Psychol. 2001;29:207–213. doi: 10.1023/a:1010377530749. [DOI] [PubMed] [Google Scholar]
- 5.Hudziak JJ, Heath AC, Madden PF, et al. Latent class and factor analysis of DSM-IV ADHD: a twin study of female adolescents. J Am Acad Child Adolesc Psychiatry. 1998;37:848–857. doi: 10.1097/00004583-199808000-00015. [DOI] [PubMed] [Google Scholar]
- 6.Neuman RJ, Todd RD, Heath AC, et al. Evaluation of ADHD typology in three contrasting samples: a latent class approach. J Am Acad Child Adolesc Psychiatry. 1999;38:25–33. doi: 10.1097/00004583-199901000-00016. [DOI] [PubMed] [Google Scholar]
- 7.Rasmussen ER, Neuman RJ, Heath AC, Levy F, Hay DA, Todd RD. Replication of the latent class structure of Attention-Deficit/Hyperactivity Disorder (ADHD) subtypes in a sample of Australian twins. J Child Psychol Psychiatry. 2002;43:1018–1028. doi: 10.1111/1469-7610.00229. [DOI] [PubMed] [Google Scholar]
- 8.Todd RD, Rasmussen ER, Neuman RJ, et al. Familiality and heritability of subtypes of attention deficit hyperactivity disorder in a population sample of adolescent female twins. Am J Psychiatry. 2001;158:1891–1898. doi: 10.1176/appi.ajp.158.11.1891. [DOI] [PubMed] [Google Scholar]
- 9.Volk HE, Neuman RJ, Todd RD. A systematic evaluation of ADHD and comorbid psychopathology in a population-based twin sample. Journal of the American Academy of Child & Adolescent Psychiatry. 2005;44:768–775. doi: 10.1097/01.chi.0000166173.72815.83. [DOI] [PubMed] [Google Scholar]
- 10.Althoff RR, Copeland WE, Stanger C, et al. The latent class structure of ADHD is stable across informants. Twin Res Hum Genet. 2006;9:507–522. doi: 10.1375/183242706778025008. [DOI] [PubMed] [Google Scholar]
- 11.Todd RD, Huang H, Henderson CA. Poor utility of the age of onset criterion for DSM-IV attention deficit/hyperactivity disorder: recommendations for DSM-V and ICD-11. Journal of Child Psychology and Psychiatry and Allied Disciplines. 2008;49:942–949. doi: 10.1111/j.1469-7610.2008.01892.x. [DOI] [PubMed] [Google Scholar]
- 12.Achenbach TM. Child behavior checklist/4-18. Dept. of Psychiatry, University of Vermont; Burlington, VT: 1991. [Google Scholar]
- 13.Volk HE, Henderson C, Neuman RJ, Todd RD. Validation of population-based ADHD subtypes and identification of three clinically impaired subtypes. American Journal of Medical Genetics. 2006;141 B:312–318. doi: 10.1002/ajmg.b.30299. [DOI] [PubMed] [Google Scholar]
- 14.Todd RD, Sitdhiraksa N, Reich W, et al. Discrimination of DSM-IV and latent class attention-deficit/hyperactivity disorder subtypes by educational and cognitive performance in a population-based sample of child and adolescent twins. J Am Acad Child Adolesc Psychiatry. 2002;41:820–828. doi: 10.1097/00004583-200207000-00014. [DOI] [PubMed] [Google Scholar]
- 15.Reiersen AM, Constantino JN, Volk HE, Todd RD. Autistic traits in a population-based ADHD twin sample. J Child Psychol Psychiatry. 2007;48:464–472. doi: 10.1111/j.1469-7610.2006.01720.x. [DOI] [PubMed] [Google Scholar]
- 16.Reiersen AM, Constantino JN, Todd RD. Co-occurrence of motor problems and autistic symptoms in attention-deficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry. 2008;47:662–72. doi: 10.1097/CHI.0b013e31816bff88. [DOI] [PubMed] [Google Scholar]
- 17.Heath AC, Madden PA, Grant JD, McLaughlin TL, Todorov AA, Bucholz KK. Resiliency factors protecting against teenage alcohol use and smoking: influences of religion, religious involvement and values, and ethnicity in the Missouri Adolescent Female Twin Study. Twin Res. 1999;2:145–55. doi: 10.1375/136905299320566013. [DOI] [PubMed] [Google Scholar]
- 18.Bucholz KK, Cadoret R, Cloninger CR, et al. A new, semi-structured psychiatric interview for use in genetic linkage studies: a report on the reliability of the SSAGA. J Stud Alcohol. 1994;55:149–158. doi: 10.15288/jsa.1994.55.149. [DOI] [PubMed] [Google Scholar]
- 19.Neuman RJ, Sitdhiraksa N, Reich W, et al. Estimation of prevalence of DSM-IV and latent class-defined ADHD subtypes in a population-based sample of child and adolescent twins. Twin Res Hum Genet. 2005;8:392–401. doi: 10.1375/1832427054936646. [DOI] [PubMed] [Google Scholar]
- 20.Todd RD, Joyner CA, Heath AC, Neuman RJ, Reich W. Reliability and stability of a semistructured DSM-IV interview designed for family studies. Journal of the American Academy of Child & Adolescent Psychiatry. 2003;42:1460–8. doi: 10.1097/00004583-200312000-00013. [DOI] [PubMed] [Google Scholar]
- 21.Hay DA, McStephen M, Levy F, Pearsall-Jones J. Recruitment and attrition in twin register studies of childhood behavior: the example of the Australian Twin ADHD Project. Twin Res. 2002;5:324–8. doi: 10.1375/136905202320906039. [DOI] [PubMed] [Google Scholar]
- 22.Hay D, McStephen M, Levy F. The diagnostic genetics of ADHD symptoms and subtypes. In: Levy F, Hay D, editors. Attention, genes and ADHD. Brunner-Routledge; 2001. pp. 35–57. [Google Scholar]
- 23.Rasmussen ER, Neuman RJ, Heath AC, Levy F, Hay DA, Todd RD. Replication of the latent class structure of Attention-Deficit/Hyperactivity Disorder (ADHD) subtypes in a sample of Australian twins. Journal of Child Psychology and Psychiatry and Allied Disciplines. 2002;43:1018–28. doi: 10.1111/1469-7610.00229. [DOI] [PubMed] [Google Scholar]
- 24.Rasmussen ER, Neuman RJ, Heath AC, Levy F, Hay DA, Todd RD. Familial clustering of latent class and DSM-IV defined attention-deficit/hyperactivity disorder (ADHD) subtypes. J Child Psychol Psychiatry. 2004;45:589–98. doi: 10.1111/j.1469-7610.2004.00248.x. [DOI] [PubMed] [Google Scholar]
- 25.McCutcheon AL. Latent Class Analysis. Sage Publications; Newbury Park: 1987. [Google Scholar]
- 26.Szklo M, Nieto FJ. Epidemiology: beyond the basics. Aspen; Gaithersburg MD: 2000. ill 23 cm edition, Vol. Book xvi. [Google Scholar]
- 27.Baldi P, Brunak S, Chauvin Y, Andersen CA, Nielsen H. Assessing the accuracy of prediction algorithms for classification: an overview. Bioinformatics. 2000;16:412–24. doi: 10.1093/bioinformatics/16.5.412. [DOI] [PubMed] [Google Scholar]
- 28.Frick PJ, Lahey BB, Applegate B, et al. DSM-IV field trials for the disruptive behavior disorders: symptom utility estimates. J Am Acad Child Adolesc Psychiatry. 1994;33:529–39. doi: 10.1097/00004583-199405000-00011. [DOI] [PubMed] [Google Scholar]
- 29.Faraone SV, Biederman J, Sprich-Buckminster S, Chen W, Tsuang MT. Efficiency of diagnostic criteria for attention deficit disorder: toward an empirical approach to designing and validating diagnostic algorithms. J Am Acad Child Adolesc Psychiatry. 1993;32:166–74. doi: 10.1097/00004583-199301000-00024. [DOI] [PubMed] [Google Scholar]
- 30.Mota VL, Schachar RJ. Reformulating attention-deficit/hyperactivity disorder according to signal detection theory. J Am Acad Child Adolesc Psychiatry. 2000;39:1144–51. doi: 10.1097/00004583-200009000-00014. [DOI] [PubMed] [Google Scholar]
- 31.Todd RD, Rasmussen ER, Wood C, Levy F, Hay DA. Should sluggish cognitive tempo symptoms be included in the diagnosis of attention-deficit/hyperactivity disorder? J Am Acad Child Adolesc Psychiatry. 2004;43:588–97. doi: 10.1097/00004583-200405000-00012. [DOI] [PubMed] [Google Scholar]


