Abstract
Whereas the feasibility and effectiveness of Internet-based epidemiologic research have been established, methodological support for the quality of such data is still accumulating. We aimed to identify sociodemographic differences among members of a French cohort according to willingness to provide part of one's 15-digit national identification number (personal Social Security number (PSSN)) and to assess response consistency based on information reported on the sociodemographic questionnaire and that reflected in the PSSN. We studied 100,118 persons enrolled in an Internet-based prospective cohort study, the NutriNet-Santé Study, between 2009 and 2013. Persons aged 18 years or more who resided in France and had Internet access were eligible for enrollment. The sociodemographic profiles of participants with discordant data were compared against those of participants with concordant data via 2-sided polytomous logistic regression. In total, 84,442 participants (84.3%) provided the first 7 digits of their PSSN, and among them 5,141 (6.1%) had discordant data. Our multivariate analysis revealed differences by sex, age, education, and employment as regards response consistency patterns. The results support the quality of sociodemographic data obtained online from a large and diverse volunteer sample. The quantitative description of participant profiles according to response consistency patterns could inform future methodological work in e-epidemiology.
Keywords: cohort studies, data quality, e-epidemiology, epidemiologic methods, general population, Internet, prospective studies, response consistency
The feasibility and effectiveness of Internet-based epidemiologic research are becoming evident, especially as regards the recruitment of very large, diverse, geographically dispersed, and hard-to-reach populations (low socioeconomic strata, risky behavior profiles), the ability to incorporate novel exposure assessment tools (physical activity, dietary behavior) which capitalize on state-of-the-art automated computer technology (1), and the rapidly expanding access to the Internet across social strata (2, 3). Such research has demonstrated both logistic advantages (cost-effectiveness, convenience regarding place/time of survey completion, ease of converting data to an analyzable format) and scientific advantages (improved quality and quantity of exposure measurement, complex research designs, study of sensitive topics or rare conditions) over traditional methods (3–9). Nonetheless, despite some evidence of the validity of epidemiologic data obtained online (7), of data concordance between Internet-based and paper-and-pencil versions of the same instruments (10, 11), and of high Internet data test-retest reliability (10), some authors caution that deliberate falsification, misrepresentation, and inadvertent or duplicate entries might be common issues in Web-based research (9).
Methodological studies are beginning to emerge that capitalize on the unique opportunities available in e-epidemiology for examining data quality. For example, in a Web-based study on sexual risk behavior utilizing a rigorous validation protocol, Konstan et al. (12) reported that nearly all invalid data (11% of all entries) were due to repeat survey submissions from the same participants. Another study in the same domain found that participants providing invalid data (entries with similar Internet Protocol addresses or e-mail addresses, erratic response patterns) were more likely to be younger, to have less formal education, and to engage in sexual risk behaviors compared with their counterparts providing valid information (9). In turn, the reinterview (including repeated questions about place of birth, education, and citizenship) designed to assess response error within the 2011 American Community Survey Internet Test revealed encouraging evidence regarding the quality of data collected online (13). Finally, asking for personal information (demographic characteristics, e-mail address) in a Web-based study was shown to provoke privacy concerns and dropout (14).
Given the current evidence, further efforts appear necessary with respect to the validation and refinement of e-epidemiologic research (2). Thus, our 2 main objectives in the present methodological study were to identify differences in participant characteristics according to the optional provision of the first 7 digits (reflecting personal demographic information) of their national identification number, which is their personal Social Security number (PSSN), and to cross-reference the demographic information provided in the baseline questionnaire and the PSSN in order to assess its concordance via data consistency indices. Our main reason for asking participants to provide part of their PSSN was the possibility of addressing issues of data quality and reliability.
METHODS
Data source
To our knowledge, the ongoing NutriNet-Santé Study (Etude NutriNet-Santé) is the first large (>100,000 enrollees to date), exclusively Web-based, general-population, prospective worldwide cohort study aimed at elucidating the relationship between nutrition (eating habits, dietary patterns, nutrient intake, nutritional status, physical activity) and health. The study is positioned to serve as a resource for investigating directional, etiological, and mechanistic hypotheses pertaining to that relationship (15). The cohort study was launched in France in May 2009 and has a planned 10-year recruitment and follow-up period. Participants are recruited via a combination of traditional and online strategies, including a vast multimedia campaign (television, radio, Internet, national/regional newspapers, billboards). The inclusion criteria pertain to residence in France (including overseas territories), age ≥18 years, and Internet access. Registration and participation take place online via a dedicated and secure website (www.etude-nutrinet-sante.fr) (15). This study was approved by the institutional review board of the French National Institute for Health and Medical Research (INSERM) and by the French National Information and Citizen Freedom Committee (CNIL). Upon provision of informed consent and an electronic signature, each person receives a confirmation e-mail with a personal log-in number and a password. Each registered user has 3 weeks to complete in full the baseline set of 5 questionnaires (sociodemographic characteristics and lifestyle, health status, diet, physical activity, and anthropometric characteristics) before being considered enrolled and active (15).
National identification number (PSSN)
At baseline, participants are given the option (but are not required) to provide the first 7 digits of their 15-digit PSSN. The actual wording of the question is as follows: “What are the first 7 digits of your Social Security number? Reminder: All information is confidential and will be used strictly for scientific purposes.” The possible responses include: 1) provision of the PSSN data; 2) “I don't know”; and 3) “I do not wish to respond.” In France, the PSSN (Numéro d'inscription au répertoire des personnes physiques (NIR)) is a unique and confidential national identification number that is assigned at birth (or upon employment for tax purposes, for persons born abroad) and is present on each person's Social Security card (carte vitale) (16). The first 7 digits pertain to one's sex (1 digit), year and month of birth (4 digits), and department of origin (2 digits). For sex, the possible values (with some very rare exceptions) are 1 for male and 2 for female; the year of birth is given by the last 2 digits, and the code for month of birth (also with some very rare exceptions) ranges from 01 to 12. The department codes range from 01 to 99. Exceptions pertain to cases where the information is provisional or unknown or the person has a particular status. Regarding sex, for example, a provisional code of 3 might be assigned to foreign-born males who are in the process of establishing residence/employment in France. In turn, in 1964 and 1967, some department code changes were introduced following modifications in French geographical divisions and definitions (17). Such changes included a switch from a department code of 75 or 78 for persons born in Paris to 91, 92, 93, 94, or 95 (which were previously used as department codes for Algeria, Tunisia, and Morocco) after 1964; a switch from a department code of 20 for persons born in Corsica to 2A or 2B; and a switch from 01 or 38 at birth to 69 after 1967. At the preliminary stage of our analysis, we strove to identify and account for such coding changes before considering any potential discrepancies as reporting error. Finally, in very rare cases, a month of birth in the range of 31–42 or 50–99 could be assigned to persons whose vital records were incomplete or whose month of birth was unknown.
Statistical analysis
For the present analysis, we used data from persons who enrolled in the NutriNet-Santé Study between May 2009 and May 2013. Among the participants who had agreed to provide PSSN information, we evaluated the concordance/consistency of demographic data (sex, year of birth, month of birth, and department of origin) by constructing 4 data consistency indices. These indices were based on cross-tabulations of the information reported on the baseline sociodemographic questionnaire and that reflected in the PSSN. Each index had a value of 0 for concordance or 1 for discordance. Next, we totaled the scores for the 4 indices in order to create a summary discordance measure (values 0–4).
Adjusted associations between the sociodemographic characteristics and the summary discordance measure were estimated via polytomous logistic regression models, without making any assumptions regarding the linearity of the associations. Polytomous logistic regression generalizes the binary logistic regression model, allowing the dependent (response) variable to have more than 2 categories (18). For these analyses, we used a 3-level dependent variable: 0 = full concordance (reference), 1 = single-item discordance, and 2 = multiple-item discordance. The multivariate model included the following covariates: sex, age (18–29, 30–54, or ≥55 years), birthplace (France, including overseas territories, vs. abroad), marital status (married/cohabiting vs. other), education (up to secondary level, undergraduate, or graduate), employment (unemployed/disabled/homemaker/student, blue-collar, white-collar, or retired), area of residence (rural, urban with ≤200,000 inhabitants, or urban with >200,000 inhabitants), and monthly household income (≤€1,799, €1,800–€2,299, €2,300–€3,700, or >€3,700 (≤$2,457, $2,458–$3,140, $3,141–$5,034, and >$5,034 as of January 29, 2014)). Adjusted odds ratios and 95% confidence intervals are reported.
Finally, we performed sensitivity analyses with cumulative logit models. In fact, while the distance between each of the 2 response categories of our 3-level dependent variable was not consistent, these categories were nonetheless sequentially ordered. The proportional odds assumption inherent in a cumulative logit model was tested via the score test (19, 20) and the Pearson goodness-of-fit statistic. All analyses were conducted with SAS, version 9.3 (SAS Institute, Inc., Cary, North Carolina).
RESULTS
Through May 2013, there were 100,118 enrolled and active participants in the NutriNet-Santé cohort with complete baseline data (78.2% women; mean age = 46.2 (standard deviation, 14.6) years). In total, 84,442 participants (84.3%) had elected to provide the first 7 digits of their PSSN, whereas 15,676 participants (15.7%) had declined to provide that information. Sociodemographic characteristics of participants according to their provision of the PSSN are summarized in Table 1.
Table 1.
Characteristics of Participants According to Provision of the First 7 Digits of the Personal Social Security Number (n = 100,118), NutriNet-Santé Study, France, 2009–2013
| PSSN Data Provided (n = 84,442) |
PSSN Data Not Provided (n = 15,676) |
|||
|---|---|---|---|---|
| No. | % | No. | % | |
| Sex | ||||
| Male | 19,445 | 23.0 | 2,337 | 14.9 |
| Female | 64,997 | 77.0 | 13,339 | 85.1 |
| Age group, years | ||||
| 18–29 | 10,607 | 12.6 | 5,058 | 32.3 |
| 30–54 | 44,077 | 52.2 | 7,573 | 48.3 |
| 55–74 | 28,381 | 33.6 | 2,924 | 18.7 |
| ≥75 | 1,377 | 1.6 | 121 | 0.8 |
| Birthplace | ||||
| France (including overseas territories) | 80,268 | 95.1 | 14,385 | 91.8 |
| Abroad | 4,174 | 4.9 | 1,291 | 8.2 |
| Marital status | ||||
| Single | 13,474 | 16.0 | 4,645 | 29.6 |
| Married, cohabiting | 61,869 | 73.3 | 10,017 | 63.9 |
| Widowed, divorced, separated | 9,088 | 10.8 | 1,011 | 6.4 |
| Other | 11 | 0.0 | 3 | 0.0 |
| Area of residence | ||||
| Rural | 18,164 | 21.5 | 3,168 | 20.2 |
| Urban | ||||
| <20,000 inhabitants | 12,702 | 15.0 | 2,187 | 14.0 |
| 20,000–200,000 inhabitants | 15,084 | 17.9 | 2,848 | 18.2 |
| >200,000 inhabitants | 37,868 | 44.8 | 7,014 | 44.7 |
| Other | 624 | 0.7 | 459 | 2.9 |
| Educational level | ||||
| Primary school or less | 14,070 | 16.7 | 2,705 | 17.3 |
| Secondary school | 16,739 | 19.8 | 3,762 | 24.0 |
| College degree | 25,519 | 30.2 | 4,550 | 29.0 |
| Graduate degree | 27,372 | 32.4 | 4,506 | 28.7 |
| Other | 742 | 0.9 | 153 | 1.0 |
| Employment status | ||||
| Unemployed, disabled, homemaker | 9,812 | 11.6 | 2,303 | 14.7 |
| Student | 4,899 | 5.8 | 3,003 | 19.2 |
| Farmer, manual labor | 1,300 | 1.5 | 307 | 2.0 |
| Artisan, self-employed | 1,385 | 1.6 | 219 | 1.4 |
| Office worker, skilled labor | 29,950 | 35.5 | 5,454 | 34.8 |
| Managerial, professional staff | 19,849 | 23.5 | 2,677 | 17.1 |
| Retired | 16,576 | 19.6 | 1,532 | 9.8 |
| Other | 671 | 0.8 | 181 | 1.2 |
| Monthly household income | ||||
| <€900 (<$1,229)a | 8,302 | 9.8 | 2,486 | 15.9 |
| €900–€1,199 ($1,229–$1,638) | 5,571 | 6.6 | 1,054 | 6.7 |
| €1,200–€1,799 ($1,639–$2,457) | 21,475 | 25.4 | 3,401 | 21.7 |
| €1,800–€2,299 ($2,458–$3,140) | 12,613 | 14.9 | 1,633 | 10.4 |
| €2,300–€2,699 ($3,141–$3,672) | 7,886 | 9.3 | 963 | 6.1 |
| €2,700–€3,700 ($3,673–$5,034) | 12,360 | 14.6 | 1,296 | 8.3 |
| >€3,700 (>$5,034) | 8,615 | 10.2 | 830 | 5.3 |
| Not provided | 7,620 | 9.0 | 4,013 | 25.6 |
Abbreviation: PSSN, personal Social Security number.
a Conversion rate as of January 29, 2014.
Among participants who had provided PSSN data, we cross-tabulated the demographic information reported on the baseline questionnaire and that reflected in the PSSN in order to assess its concordance/discordance (Table 2). In total, 79,301 participants (93.9%) had fully concordant demographic data, while 5,141 participants (6.1%) had discordant data. With respect to the 4 individual data items, sex was discordant for 0.4% of the participants, while department of origin was discordant for 1.9% of the participants. There were 620 individuals (0.7%) with discordance in all 4 demographic items.
Table 2.
Consistency of Responses Regarding Demographic Data Obtained From the Baseline Questionnaire and the First 7 Digits of the Personal Social Security Number (n = 84,442), NutriNet-Santé Study, France, 2009–2013
| No. | % | |
|---|---|---|
| Full concordance | 79,301 | 93.9 |
| Single-item discordance | 2,525 | 3.0 |
| Sex | 300 | 0.4 |
| Year of birth | 190 | 0.2 |
| Month of birth | 429 | 0.5 |
| Department of origin | 1,606 | 1.9 |
| Multiple-item discordance | ||
| 2 data items | 1,312 | 1.6 |
| 3 data items | 684 | 0.8 |
| 4 data items | 620 | 0.7 |
The polytomous logistic regression analysis (Table 3) revealed a number of differences between persons with concordant and discordant demographic data. Participants aged 18–29 years (odds ratio (OR) = 0.72, 95% confidence interval (CI): 0.62, 0.84) and those aged 55 years or more (OR = 0.64, 95% CI: 0.56, 0.73) were significantly less likely to have single-item discordance than were participants aged 30–54 years. In addition, birth abroad (vs. birth in France) was significantly associated with 40% reduced odds of single-item discordance. Next, compared with white-collar workers, persons who were unemployed, disabled, homemakers, or students were 15% more likely to have single-item discordance. Regarding multiple-item discordance, we observed increased differentiation among the participants. Males (vs. females) and those aged 18–29 years (vs. 30–54 years) were 34% and 31% less likely, respectively, to have multiple-item discordance. However, those born abroad (vs. born in France), those with no more than a high school diploma (vs. a graduate degree), and those with blue-collar occupations or who were unemployed, disabled, homemakers, or students had increased odds of presenting multiple-item discordance. For example, participants who were unemployed, disabled, homemakers, or students were over 3 times more likely to have multiple-item discordance compared with their counterparts in white-collar positions (OR = 3.06, 95% CI: 2.75, 3.39). No significant associations were observed as regards area of residence or monthly household income.
Table 3.
Multivariate Associations Between the Participant Sociodemographic Profile and the Demographic Data Discordance Index,a NutriNet-Santé Study, France, 2009–2013
| Single-Item Discordanceb |
Multiple-Item Discordancec |
|||
|---|---|---|---|---|
| Adjusted OR | 95% CI | Adjusted OR | 95% CI | |
| Sex | ||||
| Male | 0.94 | 0.84, 1.04 | 0.66 | 0.59, 0.74 |
| Female | 1 | Reference | 1 | Reference |
| Age group, years | ||||
| 18–29 | 0.72 | 0.62, 0.84 | 0.69 | 0.59, 0.80 |
| 30–54 | 1 | Reference | 1 | Reference |
| ≥55 | 0.64 | 0.56, 0.73 | 1.16 | 1.03, 1.30 |
| Birthplace | ||||
| France (including overseas territories) | 1 | Reference | 1 | Reference |
| Abroad | 0.60 | 0.47, 0.78 | 1.86 | 1.60, 2.17 |
| Marital status | ||||
| Single, divorced, separated, widowed | 0.99 | 0.89, 1.09 | 0.66 | 0.59, 0.74 |
| Married, cohabiting | 1 | Reference | 1 | Reference |
| Area of residence | ||||
| Rural | 1.02 | 0.91, 1.14 | 1.06 | 0.95, 1.18 |
| Urban with ≤200,000 inhabitants | 0.93 | 0.84, 1.03 | 0.92 | 0.83, 1.02 |
| Urban with >200,000 inhabitants | 1 | Reference | 1 | Reference |
| Educational level | ||||
| Up to secondary school | 1.12 | 1.00, 1.26 | 1.36 | 1.21, 1.52 |
| College degree | 1.04 | 0.93, 1.16 | 1.12 | 0.99, 1.25 |
| Graduate degree | 1 | Reference | 1 | Reference |
| Employment status | ||||
| Unemployed, disabled, homemaker, student | 1.15 | 1.02, 1.30 | 3.06 | 2.75, 3.39 |
| Blue-collar | 1.15 | 0.92, 1.44 | 1.57 | 1.25, 1.97 |
| White-collar | 1 | Reference | 1 | Reference |
| Retired | 1.10 | 0.94, 1.29 | 1.08 | 0.94, 1.25 |
| Monthly household income | ||||
| ≤€1,799 (≤$2,457)d | 0.98 | 0.87, 1.09 | 1.03 | 0.92, 1.16 |
| €1,800–€2,299 ($2,458–$3,140) | 0.96 | 0.84, 1.10 | 1.08 | 0.94, 1.24 |
| €2,300–€3,700 ($3,141–$5,034) | 1 | Reference | 1 | Reference |
| >€3,700 (>$5,034) | 0.97 | 0.83, 1.13 | 1.09 | 0.93, 1.28 |
Abbreviations: CI, confidence interval; OR, odds ratio.
a Two-sided polytomous logistic regression with “full concordance” used as the reference category.
b Discordance in one of the 4 demographic data items (sex, year of birth, month of birth, department of origin).
c Discordance in 2 or more demographic data items (sex, year of birth, month of birth, department of origin).
d Conversion rate as of January 29, 2014.
The results of the sensitivity analyses via cumulative logit models indicated that the proportional odds assumption was violated (data not tabulated), thus reinforcing the adequacy of the polytomous logistic regression models, which allowed the regression coefficients for each covariate to vary across the response categories.
DISCUSSION
Despite the burgeoning literature focused on the design and feasibility of Web surveys (2, 8, 21), evidence about the consistency, quality, and reliability of research data in the domain of e-epidemiology is currently insufficient. With the present study, we aimed to begin filling this gap in knowledge. Specifically, we observed that over three-quarters of online participants were comfortable providing part of their PSSN, which—because of its length (15 digits) and confidential nature—is infrequently solicited in France. It should be noted that participants were asked for their PSSN and other personal information at the beginning of the survey, which could potentially provoke privacy concerns and dropout (14). Requesting such data early on had logical as well as practical advantages in terms of commitment and retention in the study. While it may have an adverse impact on the representativeness of the cohort (introducing potential self-selection bias), early dropout might offer the advantage of improved follow-up rates.
We found encouraging evidence of demographic data consistency in approximately 94% of the sample with available PSSN information. Only 0.7% of that subsample had discordant data regarding all 4 demographic items. Given the absence of a gold standard as regards demographic data quality, the cross-referencing of 2 separate data sources served as a means of assessing data quality and reliability. To our knowledge, this is the first methodological study to examine issues related to response consistency in a large, ongoing population-based online cohort.
Our multivariate analyses revealed a number of differences in participant sociodemographic profiles with regard to response consistency as well as the extent of data discordance. The strongest evidence pertained to employment status, which played a role in both single- and multiple-item discordance. Specifically, compared with white-collar workers, persons who were unemployed, disabled, homemakers, or students were 15% more likely to have single-item discordance and over 3 times more likely to have multiple-item discordance. Male sex emerged as a significant predictor of decreased multiple-item discordance but not single-item discordance. Interestingly, persons aged 55 years or more (vs. those aged 30–54 years) were less likely to have single-item discordance but somewhat more likely to have multiple-item discordance. We could speculate that participants belonging to that age group might represent a more heterogeneous subsample in terms of both cognitive abilities (attention, concentration, memory) and sensory abilities (vision) compared with their younger counterparts, owing to cognitive aging or age-related comorbidity. Added to the complexity of the PSSN composition, this might partly explain the tendency for older participants to be more likely to evidence inconsistency regarding all 4 sociodemographic items than regarding a single item. In turn, level of formal education appeared to play a role with respect to multiple-item discordance, with persons who had up to a secondary school education being 36% more likely to have such data discordance than persons with graduate-level education. Having less formal education has previously been associated with the provision of invalid online data in human immunodeficiency virus/acquired immunodeficiency syndrome research (9). Finally, monthly household income or area of residence did not appear to play a role in response consistency in our study.
Whereas methodological research about Internet-derived data quality is scant, existing recommendations pertain to the utility of identified (as in our study) rather than anonymous responses, the need for computing data consistency indices, and the need for using both automated and manual validation techniques in order to increase confidence in the quality of data collected online and in the findings of such research (12, 22). For example, a recent methodological study identified both random and nonrandom “careless” response patterns in online surveys, for which different consistency indices were needed (22). Those authors reported that approximately 10%–12% of the participants (all of whom were undergraduate students completing a lengthy survey for course credit) were categorized as “careless” responders (22). An increased likelihood of satisficing by providing many “don't know” responses and by exhibiting decreased differentiation on rating scales has also been reported in Web-based research (23, 24).
It has been suggested that Internet-based epidemiologic surveys may present better data quality than that obtained via traditional research methods owing to the incorporation of validation checks and also to the reduced likelihood of socially desirable responding (24) or self-censorship (25). In fact, neither telephone-based surveys nor paper-and-pencil surveys are regarded as a gold standard in epidemiologic research (2, 26). However, evidence suggests that persons who respond to Web-based questionnaires are similar to those who typically participate in traditional-mode surveys in terms of age, sex, and socioeconomic status (8). Findings from 2 pilot studies carried out within the NutriNet-Santé cohort have indicated superior data quality in the Web version than in the paper version of both the anthropometric and demographic questionnaires, as the paper version was associated with data entry errors and missing/aberrant/inconsistent data. Such issues were prevented from occurring in the Web versions owing to integrated quality controls (11, 25). In addition, another pilot study conducted within the NutriNet-Santé cohort compared Web-based self-administered 24-hour dietary records against 24-hour dietary recalls collected by a trained dietitian (27). Apart from the considerable logistic simplification and cost savings associated with online survey administration, the findings of that study revealed strong agreement between the two methods, as evidenced by the intraclass correlation coefficients computed for food intake and by the energy-adjusted Pearson correlation coefficients computed for nutrient intake (27). Results from another methodological study revealed that the inclusion of invalid data (entries with similar Internet Protocol or e-mail addresses, erratic survey patterns) in the main analyses did not change the direction or statistical significance of the correlations of interest yet changed the precision of the estimated relationships (9). However, those authors cautioned that the removal of all suspicious entries might be an overly conservative and inappropriate strategy for improving data quality, as it might lead to the exclusion of valid data, to decreased statistical power and sample diversity, and to biased conclusions (9).
The principal limitation of this study was the lack of information on participation or refusal rates, since the call for recruitment is not delivered to a predefined and exhaustive list of individuals. Another limitation concerns the fact that both data sources used for response consistency assessment pertained to self-reports. In other words, we were not able to evaluate demographic data consistency against an independent, objective source. In addition, it is possible that some of the reasons for data discordance might pertain to inadvertent reversal of the information. For example, respondents might unintentionally report their year of birth after the month of birth (by reversing the positions of the year and month codes). In the case of discordance, it would appear more likely that the error came from the PSSN (which might be difficult to memorize given its length) than from the sociodemographic questionnaire, where built-in controls include, for example, a fixed range of acceptable years of birth and a fixed range of postal codes for each region of residence. Next, we did not report on differences across the sociodemographic characteristics according to provision of the PSSN, because the very low P values from such comparison tests could not convey meaningful information either about the practical importance of the results or about potential confounders.
In this study, we relied on polytomous logistic regression, which is a relatively flexible modeling strategy, not imposing any restrictions on the sequential ordering of the response categories. Unlike cumulative logit models, in polytomous logistic regression models the regression coefficients associated with each covariate are allowed to vary across levels of the ordinal dependent variable.
In conclusion, our methodological work supports the consistency/concordance of pivotal sociodemographic data, such as sex, date of birth, and place of birth, obtained online for research purposes. In addition, the present findings fill gaps in knowledge by providing a quantitative description of both participant profiles and potential bias according to response consistency patterns. Generally, the results of this study advance understanding in the relatively novel field of e-epidemiology by providing insight into opportunities for investigating issues related to data concordance, quality, and reliability.
ACKNOWLEDGMENTS
Author affiliations: Nutritional Epidemiology Research Group (UREN), Sorbonne-Paris-Cité, Unite Mixte de Recherche University of Paris 13, National Institute of Health and Medical Research (INSERM) Unité 557, French Agronomic Research Institute (INRA) Unité 1125, National Conservatory of Arts and Crafts (CNAM), Bobigny, France (Valentina A. Andreeva, Pilar Galan, Chantal Julia, Emmanuelle Kesse-Guyot, Serge Hercberg); Nutritional Epidemiology and Surveillance Group, French Institute for Health Surveillance (INVS), University of Paris 13, Sorbonne-Paris-Cité, Bobigny, France (Katia Castetbon, Serge Hercberg); and Department of Public Health, Avicenne Hospital, Bobigny, France (Chantal Julia, Serge Hercberg).
All authors contributed equally to the present work.
This work was supported by the French Ministry of Health (DGS), the French Institute for Health Surveillance (INVS), the National Institute for Prevention and Health Education (INPES), the Foundation for Medical Research (FRM), the National Institute for Health and Medical Research (INSERM), the National Institute for Agricultural Research (INRA), the National Conservatory of Arts and Crafts (CNAM), and the University of Paris 13.
We express our gratitude to the French Institute for Public Health Research (IRESP). We thank Charlie Menard, Gwenael Monot, Paul Flanzy, Mohand Ait-Oufella, Yasmina Chelghoum, Laurent Bourhis, Nathalie Arnault, Thi Hong Van Duong, Dr. Fabien Szabo, and Stephen Besseau for their assistance with data management and Anne-Sylvie Monot-Berroyer and Rachida Mehroug for their assistance with the logistic coordination of the NutriNet-Santé Study.
Conflict of interest: none declared.
REFERENCES
- 1.Schatzkin A, Subar AF, Moore S, et al. Observational epidemiologic studies of nutrition and cancer: the next generation (with better observation) Cancer Epidemiol Biomarkers Prev. 2009;18(4):1026–1032. doi: 10.1158/1055-9965.EPI-08-1129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ekman A, Litton JE. New times, new needs; e-epidemiology. Eur J Epidemiol. 2007;22(5):285–292. doi: 10.1007/s10654-007-9119-0. [DOI] [PubMed] [Google Scholar]
- 3.Funk KL, Elder CR, Lindberg NM, et al. Comparison of characteristics and outcomes by initial study contact (website versus staff) for participants enrolled in a weight management study. Clin Trials. 2012;9(2):226–231. doi: 10.1177/1740774511432727. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Baer A, Saroiu S, Koutsky LA. Obtaining sensitive data through the Web: an example of design and methods. Epidemiology. 2002;13(6):640–645. doi: 10.1097/00001648-200211000-00007. [DOI] [PubMed] [Google Scholar]
- 5.Couper MP. Issues of representation in eHealth research (with a focus on web surveys) Am J Prev Med. 2007;32(5 suppl):S83–S89. doi: 10.1016/j.amepre.2007.01.017. [DOI] [PubMed] [Google Scholar]
- 6.Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17(9):643–653. doi: 10.1016/j.annepidem.2007.03.013. [DOI] [PubMed] [Google Scholar]
- 7.Lieberman DZ. Evaluation of the stability and validity of participant samples recruited over the Internet. Cyberpsychol Behav. 2008;11(6):743–745. doi: 10.1089/cpb.2007.0254. [DOI] [PubMed] [Google Scholar]
- 8.van Gelder MM, Bretveld RW, Roeleveld N. Web-based questionnaires: the future in epidemiology? Am J Epidemiol. 2010;172(11):1292–1298. doi: 10.1093/aje/kwq291. [DOI] [PubMed] [Google Scholar]
- 9.Bauermeister J, Pingel E, Zimmerman M, et al. Data quality in web-based HIV/AIDS research: handling invalid and suspicious data. Field Methods. 2012;24(3):272–291. doi: 10.1177/1525822X12443097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Ritter P, Lorig K, Laurent D, et al. Internet versus mailed questionnaires: a randomized comparison. J Med Internet Res. 2004;6(3):e29. doi: 10.2196/jmir.6.3.e29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Touvier M, Méjean C, Kesse-Guyot E, et al. Comparison between web-based and paper versions of a self-administered anthropometric questionnaire. Eur J Epidemiol. 2010;25(5):287–296. doi: 10.1007/s10654-010-9433-9. [DOI] [PubMed] [Google Scholar]
- 12.Konstan JA, Rosser BRS, Ross MW, et al. The story of subject naught: a cautionary but optimistic tale of Internet survey research. J Comput Mediat Commun. 2005;10(2) Article 11. [Google Scholar]
- 13.Horwitz R, Tancreto JG, Zelenak MF, et al. Washington, DC: US Census Bureau; 2012. Data quality assessment of the American Community Survey Internet response data: final report. [Google Scholar]
- 14.O'Neil KM, Penrod SD, Bornstein BH. Web-based research: methodological variables’ effects on dropout and sample characteristics. Behav Res Methods Instrum Comput. 2003;35(2):217–226. doi: 10.3758/bf03202544. [DOI] [PubMed] [Google Scholar]
- 15.Hercberg S, Castetbon K, Czernichow S, et al. The NutriNet-Santé Study: a web-based prospective study on the relationship between nutrition and health and determinants of dietary patterns and nutritional status. BMC Public Health. 2010;10:242. doi: 10.1186/1471-2458-10-242. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Institut National de la Statistique et des Etudes Economiques. Personal Social Security number (NIR) [in French] (http://www.insee.fr/fr/methodes/default.asp?page=definitions/nir.htm. ). (Accessed July 1, 2013)
- 17.Colliez JP. The personal Social Security number (NIR) [in French] INSEE Economie de la Réunion. 2001:24–25. 1st trimester. [Google Scholar]
- 18.Hosmer DW, Lemeshow S. Applied Logistic Regression. New York, NY: John Wiley & Sons, Inc.; 2000. [Google Scholar]
- 19.Allison PD. Logistic Regression Using the SAS System: Theory and Application. Cary, NC: SAS Institute, Inc.; 1999. [Google Scholar]
- 20.Gameroff MJ. Using the proportional odds model for health-related outcomes: why, when, and how with various SAS procedures; SUGI 30 Proceedings, Philadelphia, Pennsylvania, April 10–13, 2005; (Statistics and Data Analysis, Paper 205-30). (http://www2.sas.com/proceedings/sugi30/205-30.pdf. ). (Accessed January 29, 2014) [Google Scholar]
- 21.Fan W, Yan Z. Factors affecting response rates of the web survey: a systematic review. Comput Hum Behav. 2010;26(2):132–139. [Google Scholar]
- 22.Meade AW, Craig SB. Identifying careless responses in survey data. Psychol Methods. 2012;17(3):437–455. doi: 10.1037/a0028085. [DOI] [PubMed] [Google Scholar]
- 23.Heerwegh D, Loosveldt G. Face-to-face versus web surveying in a high-Internet-coverage population: differences in response quality. Public Opin Q. 2008;72(5):836–846. [Google Scholar]
- 24.Nagelhout GE, Willemsen MC, Thompson ME, et al. Is web interviewing a good alternative to telephone interviewing? Findings from the International Tobacco Control (ITC) Netherlands survey. BMC Public Health. 2010;10:351. doi: 10.1186/1471-2458-10-351. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Vergnaud AC, Touvier M, Méjean C, et al. Agreement between web-based and paper versions of a socio-demographic questionnaire in the NutriNet-Santé study. Int J Public Health. 2011;56(4):407–417. doi: 10.1007/s00038-011-0257-5. [DOI] [PubMed] [Google Scholar]
- 26.Dillman DA, Smyth JD. Design effects in the transition to web-based surveys. Am J Prev Med. 2007;32(5 suppl):S90–S96. doi: 10.1016/j.amepre.2007.03.008. [DOI] [PubMed] [Google Scholar]
- 27.Touvier M, Kesse-Guyot E, Méjean C, et al. Comparison between an interactive web-based self-administered 24 h dietary record and an interview by a dietitian for large-scale epidemiological studies. Br J Nutr. 2011;105(7):1055–1064. doi: 10.1017/S0007114510004617. [DOI] [PubMed] [Google Scholar]