Dietary assessment - Misreporting
The categorical demonstration of under-reporting in dietary assessment during the late 1980s / early 1990s was revolutionary at the time. It is noteworthy to point out that at that time, the focus of obesity research was on identifying a metabolic cause of obesity, as self-reported dietary intakes did not show intakes that were higher than in lean counterparts (Prentice et al, 1989). The development of doubly labelled water (DLW) as a gold standard measure of energy expenditure was fundamental to this area of dietary assessment (Schoeller et al, 1982). Much of the pioneering work was undertaken at the Dunn Nutrition Unit, Cambridge UK by Drs Alison Black, Gail Goldberg and colleagues.
The detection of mis-reporting is really investigating the validity of the dietary data obtained. Often energy intake is used as a proxy for dietary intake. If energy intake (EI) is underestimated, it is probable that the intakes of other nutrients are also underestimated (Livingstone & Black, 2003). Validation of dietary reports therefore often concentrates on comparison with measures or estimates of energy expenditure. This is based on the laws of thermodynamics and fundamental principles of physiology. Assuming an individual is weight stable i.e. in a state of energy balance, energy intake must equal total energy expenditure (TEE).
DLW is the gold standard measure of energy expenditure (Schoeller, 2002) but is very expensive to use in research. This may limit its use. A comprehensive review of studies which have compared reported energy intake to energy expenditure measured by DLW has been undertaken (Livingstone & Black, 2003). Many showed the presence of underreporting. A review of 25 studies, which used DLW on at least 2 occasions to evaluate reported EI, concluded that within-individual variation is 8% (Black and Cole 2000).
Alternatively, standard equations, which estimate basal metabolic rate (BMR) can be applied e.g. Scholfield (Scholfield et al, 1985), and a physical activity level (PAL) factor added to the calculated BMR to provide an estimate of TEE. Measuring BMR rather than estimating it is not needed in epidemiological studies (Black 2000a). Estimates of TEE will be improved if a measure of physical activity is made e.g. accelerometry. A physical activity questionnaire at best may rank people in broad activity categories. A direct measure of basal (or resting) MR can be made by indirect calorimetry.
In weight stable person:
Energy expenditure = Energy intake ± ∆ body stores
Energy intake = energy expenditure
Energy expenditure : BMR = energy intake : BMR
PAL =energy intake : BMR
The following summary of PAL values have been calculated from DLW measurements of TEE. For each activity, it is assumed that overall activity is as detailed in the table.
|Chair-bound or bed-bound||1.2|
|Seated work with no option of moving around and little or no strenuous leisure activity||1.4-1.5|
|Seated work with discretion and requirement to move around but with little or no strenuous leisure activity||1.6-1.7|
|Significant amounts of sport or strenuous leisure activity (30-60 min, 4-5 times per week)||+0.3|
|Strenuous work or very active leisure||2.0-2.4|
Evaluation of reported energy intake
When evaluating energy intake data consider if the reported EI is representative of habitual intake or a valid estimate of actual intake during the period of investigation.
To evaluate energy intake it should be expressed as a multiple of measured or predicted BMR. This should then be compared to predicted energy requirements also expressed as a multiple of BMR (ie PAL value). A cut-off value of EI:BMR should be calculated below which, it is unlikely, that reported intake represents either habitual intake or random low intake.
The mostly widely known cut-offs are the Goldberg cut-offs (Goldberg et al 1991). The cut-offs seek to evaluate mean EI and evaluate individual EI.
Goldberg and colleagues (Goldberg et al, 1991) developed two cut-offs for the agreement between PAL and reported energy intake/BMR and their application was first demonstrated by Black et al (1991). In the first, ‘CUT-OFF 1’, PAL was set at 1.35, the minimum plausible value for most individuals who are weight stable. Subsequent work in this area led to a recommendation that this cut-off no longer should be used because it fails to account for biological variability and measurement error in estimating both energy intake and expenditure (Black, 2000b). It was also noted that it underestimated underreporting in individuals whose activity was above a sedentary level.
CUT-OFF 2 differs from the previous cut off as total energy expenditure or PAL varies according to the population or individuals under study. As with CUT-OFF 1 it involves a statistical comparison between reported energy intake and BMR accounting for biological variability and measurement error. Originally only a lower cut-off was developed but if the activity is known or can be assumed an upper limit can be determined (Black, 2000b) i.e. the 95% confidence limits for the ratio of reported energy intake/BMR and PAL.
The principles of the Goldberg CUT-OFF 3 are outlined in the following document.
To calculate cut-offs the following must be measured or estimated:
- PAL of population concerned
- Between-individual variability in PAL
- Within-individual variation in energy intake
- Number of days of dietary assessment
- Precision of estimated vs measured BMR
- Variation in repeated measurement of BMR
- Number of individuals in sample
The following points are crucial to the fundamental principles of the cut-off for mean EI:BMR:
- Assumes EI:BMR = PAL
- Uses an appropriate PAL for the sample
- Calculates the upper and lower 95% confidence limits allowing for variation in EI, BMR, PAL, d (number of days of diet) and n (number of individuals)
- Mean EI:BMR outside the 95% confidence limit is unlikely to be a valid measure of intake during the period of measurement
- Valid habitual intake should equal PAL
The aim of using a cut-off is to improve the sensitivity and specificity of the test for under-reporting:
- Sensitivity – the proportion of those who under-report and who are identified
- Specificity – the proportion of those who are not identified as under-reporters and who have not under-reported
The following points are crucial to the fundamental principles of the cut-off for individual EI:BMR:
- Assumes EI:BMR = PAL
- Calculates 95% confidence limit for n=1
- 95% confidence limits are wide
- Cut-off at 1.1 identifies about 2/3 of under-reporters
- Sensitivity is limited
- Individual information on physical activity is pivotal to identify over-reporters and all under-reporters
Use of the Goldberg cut-offs
The cut-offs have been used inappropriately or without a thorough understanding of the principles (Livingstone and Black 2003); published papers should state the criteria used for calculating cut-offs.
It has been demonstrated in sensitivity and specificity analysis using PAL values based on WHO categories (WHO, 1985) that the Goldberg cut-offs have high specificity (0.97 for men and 0.98 for women), but relatively low sensitivity (0.76 for men and 0.85 for women) (Black, 2000a). In another evaluation, the sensitivity and specificity of heart rate and the cut-offs were compared. Results showed that the sensitivity of EE by heart rate was 0.50 by diet diary and 1.00 by diet history; specificity was 0.98 and 1.00 respectively. Designating individuals into low/medium or high levels of activity and using a WHO appropriate PAL did not change sensitivity but specificity dropped to 0.98 for the diet diary and 0.97 for the diet history. Basing the cut-offs on a PAL of 1.55 reduced the sensitivity to 0.33 for the diet diary and 0.00 for the diet history; specificity was unchanged. The sensitivity of all cut-offs based on PALs for the diary was 0.50 in adults and 0.25 in children (Livingstone et al, 2003).
Using a PAL of 1.55 has been shown to identify approximately 50% of under-reporters only (Livingstone and Black 2003). At the group level, the Goldberg cut-off can be used to assess the overall bias in a study providing a population appropriate PAL is used. The cut-off has low sensitivity and poor specificity at the individual level (Livingstone and Black 2003).
Another limitation of the cut-offs is that they only identify the extreme of under-reporting (McCrory et al, 2002). It is incorrect to use the percentage difference between reported EI and either measured or predicted TEE to determine the degree of misreporting in individuals as it does not account for the error in measuring or estimating these values.
Alternative and simpler cut-offs have been proposed based on the work above. These cut-offs (McCrory et al, 2002) do use the percentage difference between TEE predicted from standard equations and reported EI, and takes account of the within-individual error in these parameters. The TEE used is that of Vinken et al (1999). Using this approach when examining relationships between diet and health in a US national cohort, the use of ±1 SD cut-offs was said to be preferable to the ±2 SD cut-offs for excluding inaccurate reports (McCrory et al, 2002).
Estimating under-reporting in dietary surveys
One study has compared estimated under-reporting by individualised estimates of energy requirements to the Goldberg cut-offs using the UK 2002 National Diet Survey data (Rennie et al, 2007). Physical activity diaries and 7-day weighed diet diaries were kept concurrently. The estimated energy requirements identified 75% men and 77% women as under-reporters similar to the 80% and 88% levels found using the Goldberg cut-offs. The individualised method did allow a quantification of under-reporting of EI. The authors speculated on the importance of measures of physical activity and questions to identify under eating during the dietary assessment in identifying true under-reporting.
Characteristics of under-reporters
It has been shown in numerous studies that overweight and obese individuals under-report more than lean counterparts (Rennie et al, 2007). Under-reporting has also been demonstrated in young people, and increases with age and body weight (Rennie et al, 2005). Under-reporting leads to bias in the types of foods reported by certain individuals. Consistent characteristics of under-reporters have not been shown (Rasmussen et al, 2007). It has been demonstrated that individuals who under-report on one occasion are likely to do so on subsequent occasions, therefore repeated or more prolonged periods of assessment will not overcome this source of error (Black and Cole, 2001)
Implications of under-reporting
The implications of under-reporting are that they are likely to attenuate any diet and disease relationships; the phenomenon is particularly troublesome in the study of obesity (Livingstone and Black 2003). The selective reporting of foods has hampered the definition of food patterns and the subsequent derivation of food based guidelines (Becker et al, 1999).
All dietary assessment methods are prone to mis-reporting-this can be either under- or over-reporting. Dietary assessments with low validity may also be caused by under-eating or over-eating. Often in prospective methods and recalls, under-reporting is apparent, whether through conscious or unconscious omission. In FFQs there may be an effect of social desirability, where foods perceived as ‘good’ are over-reported, whilst foods perceived as ‘bad’ are under-reported. Under-reporting is more likely in certain groups of people and with certain nutrients. People who are obese or overweight more often report implausibly low intakes of energy than do those who are normal weight. Some nutrients such as fat and sugar are more likely to be under-reported. This may be because these nutrients are often eaten as snack foods, which are more prone to inadvertent or deliberate failure to record having been eaten. An increased understanding of the reasons why people under-report is an important area of future study.
- Cut-offs can provide help for identifying under-reporters
- Measure physical activity, ideally by an objective measure such as accelerometry, to improve estimates of energy expenditure
- Weigh individual pre- and post-dietary assessment
- Measure dietary restraint
- Any researcher working with study individuals must be aware that their attitude, general demeanour, and non-verbal and verbal language conveys messages which may affect the likelihood of under-reporting
- Conducting focus groups to explore behaviours around dietary assessment may provide an insight into the likelihood of mis-reporting (Black and Cole, 2001)
- Critically examine all dietary data