Wednesday, February 16, 2005


Missing Clinical Information
During Primary Care Visits
Peter C. Smith, MD
Rodrigo Araya-Guerra, BA
Caroline Bublitz, MS
Bennett Parnes, MD
L. Miriam Dickinson, PhD
Rebecca Van Vorst, BA
John M. Westfall, MD, MPH
information (patient information
such as demographics,
medical history, medications, test
results, and family structure)1 is an essential
part of all medical care; it is particularly
crucial for primary care to be
able to fulfill what the Institute of Medicine
and others consider to be its defining
task of coordinating comprehensive
care across the health care system.2-7
Unfortunately, multiple barriers complicate
the collecting, synthesizing, recording,
and sharing of clinical information,
including privacy regulations,
decentralized medical systems, inadequate
interprofessional communication,
the transfer of patients’ care within
and across care settings, and the rapid
turnover of patients’ insurance plans.8-14
Accordingly, physicians may not have
clinical information available when it is
important for a patient’s care.
Missing clinical information has
been implicated in injurious adverse
events.9,11,15-21 Elder et al9 reported that
missing clinical information was associated
with 15.6% of all reported errors
in primary care, most of which
were perceived by clinicians as likely
to be harmful, and was implicated in
every major category of medical error.
In the only research studying missing
clinical information directly,22 Canadian
emergency department physicians
reported that 15.3% For editorial comment see p 617. of visits had
Author Affiliations: Department of Family Medicine, University
of Colorado Health Sciences Center, Denver.
Corresponding Author: Peter C. Smith, MD, Department
of Family Medicine, University of Colorado Health
Sciences Center at Fitzsimons, PO Box 6508, Mail Stop
F496, 12474 E 19th Ave, Bldg 402, Aurora,CO80045-
0508 (
Context The coordinating function of primary care is information-intensive and may
be impeded by missing clinical information. However, missing clinical information has
not been explicitly investigated in the primary care setting.
Objective To describe primary care clinicians’ reports of missing clinical information.
Design, Setting, and Participants Cross-sectional survey conducted in 32 primary
care clinics within State Networks of Colorado Ambulatory Practices and Partners
(SNOCAP), a consortium of practice-based research networks participating in the
Applied Strategies for Improving Patient Safety medical error reporting study. Two hundred
fifty-three clinicians were surveyed about 1614 patient visits between May and
December 2003. For every visit during 1 half-day session, each clinician completed a
questionnaire about patient and visit characteristics and stated whether important clinical
information had been missing. Clinician characteristics were also recorded.
Main Outcome Measures Reports of missing clinical information frequency, type,
and presumed location; perceived likelihood of adverse effects, delays in care, and additional
services; and time spent looking for missing information. Multivariate analysis
was conducted to assess the relationship of missing information to patient, visit, or
clinician characteristics, adjusting for potential confounders and effects of clustering.
Results Clinicians reported missing clinical information in 13.6% of visits; missing
information included laboratory results (6.1% of all visits), letters/dictation (5.4%),
radiology results (3.8%), history and physical examination (3.7%), and medications
(3.2%).Missing clinical information was frequently reported to be located outside their
clinical system but within the United States (52.3%), to be at least somewhat likely to
adversely affect patients (44%), and to potentially result in delayed care or additional
services (59.5%). Significant time was reportedly spent unsuccessfully searching for
missing clinical information (5-10 minutes, 25.6%; _10 minutes, 10.4%). After adjustment,
reported missing clinical information was more likely when patients were
recent immigrants (odds ratio [OR], 1.78; 95% confidence interval [CI], 1.06-2.99),
new patients (OR, 2.39; 95% CI, 1.70-3.35), or had multiple medical problems compared
with no problems (1 problem: OR, 1.09; 95% CI, 0.69-1.73; 2-5 problems: OR,
1.87; 95% CI, 1.21-2.89; _5 problems: OR, 2.78; 95% CI, 1.61-4.80). Missing clinical
information was less likely in rural practices (OR, 0.52; 95% CI, 0.29-0.92) and
when individual clinicians reported having full electronic records (OR, 0.40; 95% CI,
Conclusions Primary care clinicians report that missing clinical information is common,
multifaceted, likely to consume time and other resources, and may adversely
affect patients. Additional research on missing information is needed to focus on validating
clinicians’ perceptions and on conducting prospective studies of its causes and
JAMA. 2005;293:565-571
©2005 American Medical Association. All rights reserved. (Reprinted) JAMA, February 2, 2005—Vol 293, No. 5 565
important information missing at the
time of the encounter that was very
likely to result in patient harm. Such
harm could include otherwise avoidable
drug interactions or duplications,
missed or delayed diagnoses, missed immunizations,
unnecessary testing and
procedures, and the downstream effects
of such events.23
Despite its potential impact on the
essential coordination function of primary
care, missing clinical information
has not yet been explicitly investigated
in this setting. To begin to
describe this phenomenon, we surveyed
primary care clinicians about
clinical information reported as missing
during patient care visits.
This study was conducted within the
State Networks of Colorado Ambulatory
Practices and Partners (SNOCAP),
a consortium of Colorado practices and
practice-based research networks. These
include practices from the Colorado Research
Network (CaReNet) and the High
Plains Research Network (HPRN). Although
CaReNet focuses on the care of
underserved patients,24 it has a diverse
membership including academic, private,
and community practices and encompasses
both private and publicly
funded entities.HPRNsettings are in rural
and frontier communities across
northeastern Colorado.25 All 38SNOCAP
practices participating in the Applied
Strategies for Improving Patient Safety error
reporting project26 were invited to
participate. Six practices with only 1 clinician
were excluded to protect anonymity,
and first-year residents were excluded
because they were unlikely to be
familiar with practice information systems.
Clinicians in CaReNet were surveyed
between May and August 2003
and those inHPRNbetween August and
December 2003.
A 2-part cross-sectional survey of primary
care clinicians was created using
a modified Delphi technique.27 For each
visit, an anonymous study questionnaire
asked the clinician about patient
variables, including age and sex; whether
the patient had moved to the United
States within the last 5 years; and the
number of active medical problems. The
respondent was also asked whether this
was the patient’s first visit to the practice,
if he or she was the patient’s usual
primary clinician, and “Do any communication
barriers exist with this patient?”
(a broad question intended to include
such barriers as language
discrepancy, severe dementia, and developmental
delay). The clinician was
asked to indicate patient race (all that apply:
white, black, Asian, Native American,
do not know) and ethnicity (Hispanic,
non-Hispanic, do not know) to
determine if these variables were associated
with missing clinical information.
The respondent was then asked, “Was
any existing information, important for
the care of this patient, unavailable at the
time of the visit?” The questionnaire explained
that this referred only to information
known to exist. The term “important
for the care of this patient” was
not further defined but was intended to
capture essential but not necessarily urgent
information. To study the entire
scope of missing information, we included
information that might not always
be reasonably expected to be available
at the visit. For example, we asked
whether missing clinical information was
located outside the practice (eg, in the
hospital or in another state) or inside the
practice (eg, a misplaced chart or malfunctioning
electronic systems). Because
we wanted to assess information
missing at the time that most medical decisions
are made, clinicians completed
the questionnaire at the end of each visit.
Thus, clinical information initially missing
but found prior to the end of the visit
was not classified as missing, whereas information
found after the visit had ended
was still classified as missing.
If clinical information was reported
missing, clinicians answered additional
questions pertaining to that information.
They choseamongnonmutually exclusive,
fixed-response options that also
had an “other” option accompanied by
space for free text. These questions included
(1) the type of information reported
as missing; (2) whether they
thought the missing information likely
resided within or outside their clinical
system (defined as their practice and any
associated hospital, university, or community
health system) orwithin or outside
the United States; (3) whether, as a
consequence of the information being
missing, they thought the patient was
likely to have a delay in care or require
additional medical services; and (4)
whether the clinician or a staff member
had attempted to find the information,
and if not, why not. If clinicians searched
for the missing information but didn’t
find it during the visit, they were asked
to estimate the time spent looking (_1,
1-4, 5-10, or_10 minutes). Finally, they
recorded on a 5-point Likert scale their
estimate of “How likely is this missing
information to adversely affect the patient’s
well being?”, with anchors ranging
from “not at all likely” to “very
likely.” The questionnaire instructions
asked only that this be considered in the
context of the patient’s medical care but
did not define “adversely affect.” These
estimates of adverse effects were not confirmed
or otherwise characterized.
Asecond clinician questionnaire asked
for the clinicians’ own demographic information
and specialty, whether they
were physicians or midlevel clinicians
(nurse practitioner or physician assistant),
and whether they were residents.
The questionnaire also asked the respondents
to choose the single best description
of their practice’s information system:
paper charts, partial or hybrid
electronic medical records (EMRs), or
full (EMRs). Finally, clinicians reported
whether or not they had electronic
access in their office to patient data
from their primary hospital.Wedid not
assess the extent to which each respondent
used any existing electronic systems.
The survey was reviewed by experts
in medical error and communication to
maximize face and content validity and
was pilot tested by experienced clinicians.
28 The study questionnaire was
limited to 1 page to maximize response
rate; average completion time
was less than 1 minute. No patient or
566 JAMA, February 2, 2005—Vol 293, No. 5 (Reprinted) ©2005 American Medical Association. All rights reserved.
clinician identifying information was included
on the questionnaires, and the
study was approved as exempt by the
Colorado Multiple Institutional Review
Board and all necessary local institutional
review boards.
Data Collection
Each participating clinician completed
the study questionnaire at the end of every
consecutive patient visit during 1
half-day clinic session. Each clinician also
completed 1 anonymous clinician questionnaire.
Recent preexisting network
surveys provided data on practice size,
estimated by the number of full-time
equivalent clinicians at each practice.
Network data were used to determine
which of these were residency practices
to assess whether their unique structure
influenced missing clinical information
independent of the behavior of
resident vs nonresident physicians. Because
residents are frequently away from
the clinics, they were considered 0.3 fulltime
equivalents. The month of data collection
for each practice was recorded.
Statistical Analysis
Missing clinical information rates, frequency
distributions, and means (SDs)
were calculated for all variables of interest.
The intraclass correlation coefficient
was computed to assess potential
clustering effects. The intraclass correlation
coefficient for patients within physicians
was 0.076, indicating the need to
use methods appropriate for clustered
data. To determine whether missing
clinical information was associated with
patient demographics, visit characteristics,
and practice or clinician factors, generalized
linear mixed models (multilevel
models) were used with missing
clinical information (yes/no) as the outcome
(logit link) to extend the traditional
logistic regression model to accommodate
the hierarchical structure of
the data (SAS Proc MIXED with
GLIMMIX macro).29 Variance components
at each level were examined to determine
whether random effects should
be retained (clinician, practice). After accounting
for clinician-level variability,
variability at the practice level was not
significant (P_.20). Thus a 2-level model
was used (patient, clinician). Sensitivity
analyses were performed by strata
when cell frequencies were adequate.
Significance from the generalized linear
mixed models was determined using
the F statistic, a joint significance test
of global differences among any categories.
Statistical significance was defined
as P_.05 (2-tailed test). To study characteristics
associated with reported missing
clinical information, power calculations
indicated that a sample of 340
events of missing information per group
was necessary to detect a 10% absolute
difference in rates of missing clinical information
in a 2-group comparison with
80% power, assuming an intraclass correlation
coefficient of 0.08 (variance inflation
factor, 1.48) and a missing information
rate of 13% in 1 group. All
analyses were performed using SAS 8.2
(SAS Institute Inc, Cary, NC).
A total of 253 clinicians in 32 practices
returned study questionnaires for 1614
visits. Eight of these practices were rural
HPRN sites and 24 were urban/
suburban CaReNet practices. Six invited
practices, representing 34 clinicians,
did not participate; reasons included extreme
weather, an influenza outbreak,
and being too busy with practice or concomitant
surveys. Participating and nonparticipating
practices did not differ significantly
in size (P=.26), rurality
(P=.33), or whether they were residency
practices (P=.64). Although the
number of clinicians within the networks
is constantly changing, we estimated
that the 253 participants represent
71% of all network clinician fulltime
equivalents. Of these 253 clinicians,
7 did not complete the clinician survey,
leaving 51 patient visits without clinician
information. As a result, clinician information
was available for 1563 patient
visits (96.8%).
The results of univariate analyses of
patient, visit, clinician, and practice
characteristics are shown in TABLE 1
and TABLE 2. Diverse age groups and
both sexes were well-represented. Clinicians
characterized most patients as
white (74.6%) but one third of patients
as Hispanic. Half of all patients
had at least 2 active medical problems,
while relatively few were characterized
as first-time patients (13.0%)
or recent immigrants (5.1%). Most respondents
were family physicians. Most
practices were nonrural and reported
electronic access to inpatient data.
Clinical information considered important
was reported to be missing at the
time of the visit in 220 (13.6%) of 1614
visits, and many visits had more than 1
type of information missing (TABLE 3).
Clinicians reported that the types of information
missing included (as a percentage
of total visits) laboratory results
(6.1%), letters/dictation (5.4%),
radiology results (3.8%), history and
physical examination (3.7%), and medi-
Table 1. Patient and Visit Characteristics*
No. (% of Total)
(n = 1614)
Age, y
_17 422 (26.1)
18-39 473 (29.3)
40-64 469 (29.1)
_65 235 (14.6)
Incomplete 15 (0.9)
Male 554 (34.3)
Female 1051 (65.1)
Incomplete 9 (0.6)
White 1204 (74.6)
Nonwhite 227 (14.1)
Unknown or incomplete 183 (11.3)
Hispanic 538 (33.3)
Non-Hispanic 846 (52.4)
Unknown or incomplete 230 (14.3)
Moved to United States
in last 5 y
Yes 83 (5.1)
No 1399 (86.7)
Unknown or
132 (8.2)
Active medical problems
0 259 (16.0)
1 503 (31.2)
2-5 654 (40.5)
_5 152 (9.4)
Incomplete 46 (2.9)
First visit to the practice
Yes 210 (13.0)
No 1389 (86.1)
Unknown or incomplete 15 (0.9)
Usual care clinician
Yes 1011 (62.6)
No 555 (34.4)
Unknown or incomplete 48 (3.0)
Communication barriers exist
Yes 156 (9.7)
No 1424 (88.2)
Unknown or incomplete 34 (2.1)
*All data are by clinician report.
©2005 American Medical Association. All rights reserved. (Reprinted) JAMA, February 2, 2005—Vol 293, No. 5 567
cations (3.2%). In 97 (44.0%) of these
visits, clinicians reported that missing information
was at least somewhat likely
to adversely affect the patient (Table 3).
Clinicians believed the missing information
was outside their clinical system in
57.3% of visits with missing information.
They also reported that someone attempted
to find the missing information
in 125 (56.8%) of these visits. For
45 (36.0%) of these 125 visits, clinicians
reported spending at least 5 minutes
looking for missing clinical information.
They also reported that during
36 (28.8%) of the 125 visits, staff spent
at least 5 minutes looking for missing information.
Clinicians believed that missing
information would likely result in
either delayed care or at least 1 duplicative
medical service in 59.5% of visits
with missing information (Table 3).
Associations between missing clinical
information and patient, visit, clinician,
and practice characteristics, separately
and in combination, were tested
using multilevel models adjusted for
clustering of patients within physicians
(TABLE 4). Increased reporting of missing
clinical information was significantly
associated with first visit (odds ratio
[OR], 2.39; 95% confidence interval
[CI], 1.70-3.35), rural clinician (OR,
0.52; 95% CI, 0.29-0.92), immigration
within 5 years (OR, 1.78; 95% CI, 1.06-
2.99), and number of active medical
problems (no problems vs 1 problem:
OR, 1.09; 95% CI, 0.69-1.73; 2-5 problems:
OR, 1.87; 95% CI, 1.21-2.89; _5
problems: OR, 2.78; 95% CI, 1.61-
4.80). Clinical information was equally
likely to be reported missing regardless
of electronic access to information at
one’s primary hospital, the size of the
practice, the month of data collection,
whether physicians were residents, or
whether the setting was a residency practice.
Family physicians had rates of visits
with missing information similar to
those of other physicians (13.2% vs
14.4%; P=.61). While physicians had a
smaller percentage of missing clinical information
than did midlevel clinicians
(13.4% vs 26.5%), the small numbers of
visits for which midlevel clinicians reported
missing information (n=9) precluded
further analysis.
Within a given practice, there was
only 81% agreement on average among
clinicians on how to classify the practice’s
charting system. Accordingly, we
assessed practices’ charting systems using
both individual clinician report and
clinician concurrence, determined by
taking the response most often reported
by the clinicians within each
practice. Only 17 clinicians indicated
that their offices had full EMRs. When
compared with respondents who reported
having hybrid EMRs or paper records,
clinicians who reported having
full EMRs were significantly less likely
to report missing clinical information
(Table 4), while reporting a partialEMR
did not confer a difference (OR, 0.88;
95% CI, 0.60-1.28). However, when using
the practice-level variable of clinician
concurrence rather than individual
report, no benefit was seen for
practices determined to have full EMRs
(OR, 0.60; 95% CI, 0.25-1.40).
We studied primary care clinicians’ reports
about missing clinical information
during patient visits and their beliefs
about its potential consequences. In
nearly 1 in 7 visits, they reported that
clinical information important for the patient’s
care was missing. Although laboratory
reports and dictations or letters
predominated, clinicians reported that
the missing information originated from
a variety of sources and often included
more than 1 type. In 44% of the visits
with missing information, clinicians believed
the patient would be at least somewhat
likely to be adversely affected. If
validated by future research, these results
could have serious implications for
the 220 million primary care visits that
occur in the United States each year.30
Poon et al31 found that 83% of surveyed
physicians had reviewed at least
1 test result in the previous 2 months that
they would have wanted to know about
earlier, despite having fairly advanced
electronic information systems. It is not
surprising that in our study clinicians and
staff spent significant amounts of time
looking for missing information, especially
when they believe it often leads to
delayed care, duplicative services, or potential
adverse effects for their patients.
We did not validate these time estimates,
and based on other research32 clinicians
may have overestimated the
amount of time spent unsuccessfully
looking for missing information. However,
by excluding any time spent during
the visit that resulted in finding the
information (so that it was not classified
as missing), or time spent looking
for missing information after the visit was
over, we may have underestimated the
total lost time related to searching. This
may represent less time available for direct
patient care, a further reduction in
a resource that is already under threat
from other competing demands.
Wefound relatively few predictors of
missing clinical information. Clinicians
were more likely to report missing clinical
information during visits in which the
patient had recently moved to the United
States, was new to a practice, or had multiple
medical problems. These factors
have been implicated in missing information–
related medical errors and adverse
events in other settings.10,12,15,33
Table 2. Clinician and Practice
No. (% of Total)
(n = 253)
Clinician type
Nurse practitioner 14 (5.5)
Physician assistant 23 (9.1)
Physician 209 (82.6)
Incomplete 7 (2.8)
Family physician 203 (80.2)
General internist 18 (7.1)
General pediatrician 19 (7.5)
Obstetrician 1 (0.4)
Incomplete 12 (4.8)
Yes 106 (41.9)
No 132 (52.2)
Incomplete 15 (5.9)
Rural practice
Yes 28 (11.1)
No 225 (88.9)
Medical records
Paper 144 (56.9)
Partial/hybrid electronic 84 (33.2)
Full electronic 17 (6.7)
Incomplete 8 (3.2)
Electronic access
to inpatient data
Yes 214 (84.6)
No 29 (11.5)
Do not know/incomplete 10 (3.9)
*All data are by clinician report.
568 JAMA, February 2, 2005—Vol 293, No. 5 (Reprinted) ©2005 American Medical Association. All rights reserved.
Rural clinicians were less likely to report
missing information than urban or
suburban clinicians, perhaps because of
simpler and more self-contained systems
of care, with fewer clinicians and
facilities compared with urban areas. It
is possible that the influence of broader
systemic factors on missing clinical information
that could not be discerned in
this study may overwhelm such patient,
clinician, or practice factors.
Clinicians reporting a fullEMRin their
practice were significantly less likely to
report missing clinical information, but
this did not eliminate the problem. Missing
information was believed more likely
to be outside the clinical system than
within it and therefore may be beyond
the reach of an EMR. The lack of impact
of partial EMRs and electronic access
to hospital data on adverse events
has been found in other settings.11,18We
found no difference in reports of missing
information when we used the concurrence
among clinicians within a practice
to determine theEMRvariable. This
difference from individual report may indicate
that familiarity with or actual use
of an EMR is a better predictor of effective
information management than the
mere presence of an EMR.
This study has several important limitations.
The data are cross-sectional and
based on clinician report, including
patient race and ethnicity, which may
be less accurate than patient selfidentification.
Several network practices
reported being too busy to participate.
Although this number was small,
had they participated the rate of reported
missing clinical information may
have been slightly higher. There was no
independent verification that questionnaires
were completed on every consecutive
patient in each clinic session.
The definition of information that was
“important for the care of this patient”
was open to broad interpretation by the
respondent. Such information may be
both important and urgent (eg, an allergy
to a newly prescribed medication)
or important but not urgent (eg, a written
advance directive for a patient with
dementia, or urinary microalbumin results
for a patient with diabetes).
Table 3. Missing Clinical Information: Categories, Visit Characteristics, and Perceived
Variable Visits, No. (%)
Categories of Missing Clinical Information (n = 220)*
Laboratory results† 99 (45.0)
Letters/dictation 87 (39.5)
Radiology results 62 (28.2)
History and physical examination 59 (26.8)
Current and prior medications 51 (23.2)
Pathology results‡ 33 (15.0)
Immunization records 27 (12.3)
Procedures 16 (7.3)
Other§ 11 (5.0)
Visit Characteristics
Perceived likelihood of missing clinical information to adversely affect the patient’s
well-being (n = 220)
1 (Not at all likely) 52 (23.6)
2 (Not very likely) 68 (30.9)
3 (Somewhat likely) 52 (23.6)
4 (Likely) 30 (13.6)
5 (Very likely) 15 (6.8)
Incomplete 3 (1.4)
Where is information likely to reside? (n = 220)*_
Within own clinical system 92 (41.8)
Outside clinical system but in United States 115 (52.3)
Outside United States 11 (5.0)
Do not know 5 (2.3)
Attempted to obtain the information? (n = 220)
No (clinician or staff ) 95 (43.2)
Why not?* (n = 95)
Not critical 55 (57.9)
Unlikely to succeed 32 (33.7)
Too busy 17 (17.9)
Other 14 (14.7)
Yes (clinician or staff ) 125 (56.8)
Reported time clinician spent looking unsuccessfully, min (n = 125)
_1 23 (18.4)
1-4 53 (42.4)
5-10 32 (25.6)
_10 13 (10.4)
Incomplete 4 (3.2)
Reported time staff spent looking unsuccessfully, min (n = 125)
_1 39 (31.2)
1-4 23 (18.4)
5-10 20 (16.0)
_10 16 (12.8)
Incomplete 27 (21.6)
Perceived Consequences of Missing Clinical Information (n = 220)¶
Delay(s) in care 56 (25.5)
Additional laboratory testing 49 (22.3)
Additional visit(s) 46 (20.9)
Additional imaging 24 (10.9)
Other# 18 (8.2)
*Percentages total more than 100% because questionnaire options were not mutually exclusive.
†Blood chemistry, urinalysis, and hematology.
‡Biopsy specimens and cytology, including Papanicolaou smears.
§Includes pediatric growth data, notes about telephone calls, and parts of charts or entire charts.
_A visit may have had 2 or more pieces of missing information residing in different places.
¶Questionnaire items were not mutually exclusive. Either a delay in care or an additional medical service was reported
as a likely outcome in 131 (59.5%) of 220 visits.
#Included additional time spent by patients and clinicians looking for the missing information and communicating it on
the telephone with hospitals, specialists, pharmacies, and each other; additional time spent reconciling divergent
information; problems with missing information that will not be resolved by the next visit; potentially missed diagnoses
or improper therapeutics; and potentially duplicated vaccinations.
©2005 American Medical Association. All rights reserved. (Reprinted) JAMA, February 2, 2005—Vol 293, No. 5 569
To explore the widest possible scope
of the problem of missing clinical information,
there was no requirement that
having the information available during
the visit was reasonable. Expecting
prior medical records at a first visit may
not yet be realistic in many practices, and
primary efforts to remedy the problem
may best focus on limiting missing information
for existing patients. However,
these findings suggest that robust,
long-term solutions may need to
include transfers of care across care settings,
even across international borders.
34 One model for a solution is the
Continuity of Care Record, a data standard
that enables diverse information
systems to share aminimal clinical data
set whose components closely mirror the
types of missing information reported in
this study,35 that has the potential to be
disseminated via portable memory devices
or secure e-mail or Web servers,
and that can be printed and given directly
to patients or new clinicians.
Because clinicians were not given a
specific definition of an adverse effect
from missing clinical information, their
responses may have considered outcomes
ranging from minor inconvenience
to financial hardship to actual
physical injury. We did not validate or
characterize these estimates of potential
adverse effects. Although other studies
have demonstrated that errors related
to missing clinical information are
common and can adversely affect patients,
8,9,11,15,20,36-38 future research should
focus on the actual impact of missing
information on patients, clinicians,
practices, and systems of care.
Although we did not validate the accuracy
of clinician report of missing clinical
information, a recent directobservation
study indicated that primary
care physicians’ reports of events during
patient visits are highly accurate.32
We did not confirm whether information
reported as missing actually existed
and, if it did, whether it was truly
inaccessible to the clinician or was functionally
missing (ie, actually available but
not found when needed). Clinicians may
have reported nonexistent information
(such as a laboratory test ordered but
never actually performed) as missing.
Conversely, they may have reported information
as missing that was actually
at their fingertips but that they did not
or could not access (such as results buried
inside a thick paper record). We did
not determine how well the practices’
electronic systems were functioning or
used during the study, which may have
transiently altered the rate of missing information.
However, busy clinicians
making medical decisions during clinic
Table 4. Patient, Visit, Clinician, and Practice Characteristics Associated With Reported
Missing Clinical Information
Visits With Missing
Clinical Information, % OR (95% CI)*
Patient characteristics, y
Age category
_17 11.4 0.67 (0.42-1.07)
18-39 13.5 0.91 (0.59-1.38)
40-64 14.7 1.00 (0.66-1.51)
_65 14.0 Reference
Male 15.0 1.20 (0.92-1.58)
Female 12.7 Reference
Nonwhite 14.5 1.09 (0.73-1.62)
White 12.6 Reference
Hispanic 13.0 0.82 (0.59-1.14)
Unknown 17.0 1.23 (0.82-1.84) .18
Non-Hispanic 13.1 Reference
No. of active medical problems
0 9.7 Reference
1 10.7 1.09 (0.69-1.73)
2-5 15.9 1.87 (1.21-2.89)
_5 19.1 2.78 (1.61-4.80)
Moved to United States within last 5 y
Yes 21.7 1.78 (1.06-2.99)
No/unknown 12.9 Reference
Visit characteristics
First visit
Yes 24.3 2.39 (1.70-3.35)
No/unknown 12.0 Reference
Usual clinician
Yes 12.6 0.81 (0.62-1.07)
No/unknown 15.4 Reference
Communication barriers
Yes 16.7 1.30 (0.85-2.00)
No/unknown 13.3 Reference
Clinician and practice characteristics
Resident physician
Yes 12.7 0.93 (0.64-1.35)
No 14.1 Reference
Rural clinician
Yes 8.2 0.52 (0.29-0.92)
No 14.5 Reference
Medical records‡
Full electronic record 6.5 0.40 (0.17-0.94)
Paper or partial electronic record 14.3 Reference
Electronic access to inpatient data
Yes 13.4 0.96 (0.61-1.52)
No 15.1 Reference
Abbreviations: CI, confidence interval; OR, odds ratio.
*Multilevel, univariate logistic regression adjusted for clustering of patients within physicians.
†P values represent significance of F statistic for all categories.
‡As reported by individual clinicians.
570 JAMA, February 2, 2005—Vol 293, No. 5 (Reprinted) ©2005 American Medical Association. All rights reserved.
visits need information systems that are
both effective and efficient. Because most
medical decisions are made during patient
visits, clinicians may not distinguish
between actually missing and functionally
missing information.
Although this is a state-level survey,
our sample included diverse clinicians
and patients from a variety of practices
in multiple geographic, economic, and
demographicsettings.Althoughthe racial
and ethnic composition of our sample
was different from national norms, we
found no differential rates of missing
clinical information basedonrace or ethnicity.
We therefore believe that these
results should be generalizable.
This is the first direct study of missing
clinical information in primary care,
in contrast to retrospective detection of
missing information as the etiology of a
medical error or adverse event. It demonstrates
reports of a high frequency of
missing important clinical information,
with a wide array of potential impact on
patient care. Additional research on missing
clinical information should focus on
validating clinicians’ perceptions and
conducting prospective studies of its actual
causes and sequelae.
Author Contributions: Dr Smith had full access to all
of the data in the study and takes responsibility for
the integrity of the data and the accuracy of the data
Study concept and design: Smith, Araya-Guerra,
Parnes, Westfall, Pace.
Acquisition of data: Smith, Araya-Guerra, Dickinson,
Van Vorst, Westfall, Pace.
Analysis and interpretation of data: Smith, Araya-
Guerra, Bublitz, Parnes, Dickinson, Westfall, Pace.
Drafting of the manuscript; critical revision of the
manuscript for important intellectual content: Smith,
Araya-Guerra, Bublitz, Parnes, Dickinson, Van Vorst,
Westfall, Pace.
Statistical analysis: Araya-Guerra, Bublitz, Parnes,
Dickinson, Van Vorst.
Obtained funding: Smith, Pace.
Administrative, technical, or material support: Araya-
Guerra, Westfall, Pace.
Study supervision: Smith, Araya-Guerra, Dickinson,
Westfall, Pace.
Financial Dislosure: None reported.
Funding/Support: This study was funded by the American
Academy of Family Physicians Foundation and the
Joint AAFP/F-AAFP Grant Awards Council (MIA Care:
the Missing Information in Ambulatory Care Study, grant
G0307RS) (Dr Smith) and in part by the Agency for
Healthcare Research and Quality (Applied Strategies for
Improving Patient Safety, grantU18HS11878) (Dr Pace).
Role of the Sponsors: None of the funding sources
had any role in the design and conduct of the study;
the collection, preparation, or interpretation of the data;
or the preparation or approval of the manuscript.
Previous Presentations: An earlier draft of this article
was presented as a distinguished paper at the North
American Primary Care Research Group annual meeting;
October 10-13, 2004; Orlando, Fla.
Acknowledgment: We thank Tillman Farley, MD, and
Marc Ringel, MD, for their early conception and support
of this study and Linda Niebauer, Elizabeth Staton,
and especially Sherry Holcomb for their invaluable
assistance.Wethank all the participating SNOCAP
practices for their dedicated participation: AF Williams
Family Medicine, Colorado Springs Osteopathic
Foundation and Family Medicine Center, Comprehensive
Family Care Center, PC, Denver Health
Medical Plan Clinic, Exempla St Joseph Family Practice,
Generations Health Care, High Street Internal
Medicine, Internal Medicine–AOP, Kids Care Clinic,
La Casa-Quigg Newton Health Center, Lowry Family
Health, Mariposa Family Health, Park Hill Family
Health, People’s Clinic, Plains Medical Center Limon
and Strasburg Clinics, Rose Family Medicine, Saint
Mary’s Family Practice, Salud Family Health Centers
(Brighton Family Health, Commerce City, Ft Lupton,
Ft Morgan, and Sterling), Southern Colorado Family
Practice, St Anthony’s Family Medicine Center West,
Swedish Family Medicine, University Family Medicine
(Park Meadows and Westminster), Westside Family
Health Center Pediatric and Teen Clinic, Wray Clinic,
and Yuma Clinic.
1. Smith R. What clinical information do doctors need?
BMJ. 1996;313:1062-1068.
2. Gorman PN. Information needs of physicians. J Am
Soc Inf Sci. 1995;46:729-736.
3. Ebell MH, Frame P. What can technology do to,
and for, Family Medicine? FamMed. 2001;33:311-319.
4. Martin JC, Avant RF, Bowman MA, et al; Future
of Family Medicine Project Leadership Committee. The
Future of Family Medicine: a collaborative project of
the family medicine community. Ann Fam Med. 2004;
2(suppl 1):S3-S32.
5. Donaldson MS, Yordy KD, Lohr KN, Vanselow
NA, eds. Primary Care: America’s Health in a New Era.
Washington, DC: Institute of Medicine; 1996.
6. Parkerton PH, Smith DG, Straley HL. Primary care
practice coordination versus physician continuity. Fam
Med. 2004;36:15-21.
7. Aspden P, Corrigan JM, Wolcott J, Erickson SM, eds.
Patient Safety: Achieving a New Standard for Care.
Washington, DC: Institute of Medicine; 2004.
8. Committee on Quality Health Care in America.
Crossing the Quality Chasm: A New Health System
for the 21st Century. Washington, DC: Institute of
Medicine; 2001.
9. Elder NC, Vonder Meulen M, Cassedy A. The identification
of medical errors by family physicians during
outpatient visits. Ann Fam Med. 2004;2:125-129.
10. Gandhi TK, Sittig DF, Franklin M, Sussman AJ,
Fairchild DG, Bates DW. Communication breakdown
in the outpatient referral process. J Gen Intern Med.
11. Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates
DW. The incidence and severity of adverse events affecting
patients after discharge from the hospital. Ann
Intern Med. 2003;138:161-167.
12. Poon EG, Haas JS, Louise Puopolo A, et al. Communication
factors in the follow-up of abnormal
mammograms. J Gen Intern Med. 2004;19:316-323.
13. Bourguet C, Gilchrist V, McCord G; Northeastern
Ohio Network Research Group. The consultation
and referral process. J Fam Pract. 1998;46:47-53.
14. Roulidis ZC, Schulman KA. Physician communication
in managed care organizations: opinions of primary
care physicians. J Fam Pract. 1994;39:446-451.
15. Wilson RM, Harrison BT, Gibberd RW, Hamilton
JD. An analysis of the causes of adverse events from
the Quality in Australian Health Care Study. Med J
Aust. 1999;170:411-415.
16. Cook RI, Render M, Woods DD. Gaps in the continuity
of care and progress on patient safety. BMJ.
17. Bhasale AL, Miller GC, Reid SE, Britt HC. Analysing
potential harm in Australian general practice: an incident
monitoring study.MedJAust. 1998;169:73-76.
18. Gandhi TK, Weingart SN, Borus J, et al. Adverse
drug events in ambulatory care. N Engl J Med. 2003;
19. Lesar TS, Briceland L, Stein DS. Identification ofmedication
prescribing errors. JAMA. 1997;277:312-317.
20. Holden J, O’Donnell S, Brindley J, Miles L. Analysis
of 1263 deaths in four general practices. Br J Gen
Pract. 1998;48:1409-1412.
21. Dovey SM, Phillips RL, Green LA, Fryer GE. Consequences
of medical errors observed by family
physicians. Am Fam Physician. 2003;67:915.
22. Stiell A, Forster AJ, Stiell IG, van Walraven C. Prevalence
of information gaps in the emergency department
and the effect on patient outcomes. CMAJ. 2003;
23. Toxic cascades: a comprehensive way to think
about medical errors.AmFam Physician. 2001;63:847.
24. Parnes B, Main DS,Holcomb S, PaceWD.Tobacco
cessation counseling among underserved patients: a
report from CaReNet. J Fam Pract. 2002;51:65-69.
25. Van Vorst R, Westfall JM; High Plains Research
Network, University of Colorado Health Sciences Center
Department of Family Medicine. Assessment and
Evaluation of Hispanic Access to Health Care Services
in Western Yuma County. 2001. Available at:
.asp. Accessed December 20, 2004.
26. Fernald DH, Pace WD, Harris DM, et al. Event reporting
to a primary care patient safety reporting system.
Ann Fam Med. 2004;2:327-332.
27. Hasson F, Keeney S, McKenna H. Research guidelines
for the Delphi survey technique. J Adv Nurs. 2000;
28. Fink A, ed. How to Measure Survey Reliability
and Validity. Vol 7. Thousand Oaks, Calif: Sage; 1995.
29. Littell R,Milliken GA, Stroup WW, Wolfinger RD.
SAS System for Mixed Models. Cary, NC: SAS Institute
Inc; 1996.
30. Cherry DK, Burt CW,Woodwell DA. National Ambulatory
Medical Care Survey: 2001 Summary: Advance
Data From Vital and Health Statistics No. 337.
Hyattsville, Md: National Center for Health Statistics;
31. Poon EG, Gandhi TK, Sequist TD, Murff HJ, Karson
AS, BatesDW.“I wish I had seen this test result earlier!”:
dissatisfaction with test result management systems in
primary care. Arch Intern Med. 2004;164:2223-2228.
32. Gilchrist VJ, Stange KC, Flocke SA, McCord G,
Bourguet CC. A comparison of the National Ambulatory
Medical Care Survey (NAMCS) measurement
approach with direct observation of outpatient visits.
Med Care. 2004;42:276-280.
33. Leape LL, Bates DW, Cullen DJ, et al; ADE Prevention
Study Group. Systems analysis of adverse drug
events. JAMA. 1995;274:35-43.
34. Burton LC, Anderson GF, Kues IW. Using
electronic health records to help coordinate care.
Milbank Q. 2004;82:457-481.
35. Kibbe DC, Phillips RL Jr, Green LA. The continuity
of care record. Am Fam Physician. 2004;70:1220,
36. Brennan TA, Leape LL, Laird NM, et al. Incidence
of adverse events and negligence in hospitalized
patients. N Engl J Med. 1991;324:370-376.
37. Kohn LT, Corrigan JM, Donaldson MS; Committee
on Quality of Health Care in America. To Err Is
Human: Building a Safer Health System. Washington,
DC: Institute of Medicine; 2000.
38. Woolf SH, Kuzel AJ, Dovey SM, Phillips RL Jr. A
string of mistakes: the importance of cascade analysis
in describing, counting, and preventing medical
errors. Ann Fam Med. 2004;2:317-326.
©2005 American Medical Association. All rights reserved. (Reprinted) JAMA, February 2, 2005—Vol 293, No. 5 571


Post a Comment

<< Home