HEALTH IT AND PATIENT SAFETY
PREPUBLICATION COPY: UNCORRECTED PROOFS
Appendix B: Literature Review Methods
Literature Tables
B-1 Systematic Reviews
B-2 Studies Examining Patient Safety and Health IT
TABLE B-1
Systematic Reviews
Study Study Purpose Relevant Findings
Health IT
Component Time Frame Sample Size Outcome Measures
Black, A. D., J. Car, C Pagliari, C.
Anandan, K. Cresswell, T. Bokun,
B. McKinstry, R. Procter, A. Majeed,
A. Sheikh. 2011. The impact
of eHealth on the quality and
safety of health care: A systematic
overview. Public Library of
Science Med 8(1): e1000387.
Overview 1997 to 2010 53 systematic
reviews
Benefits and risks associated
with various eHealth systems
(i.e., legibility, accessibility,
efficiency, patient disengagement,
and increased costs)
To determine the
impact of eHealth on
the quality and safety
of health care by
conducting a systematic
review of current
systematic reviews
–	There is insufficient empirical evidence in the literature to
establish the impact of eHealth on the quality and safety
of health care
–	Evidence supporting eHealth is weak and inconsistent
–	The presence of negative consequences cited in the literature
indicates a need to further evaluate the risks associated
with eHealth
Harrington, L., D. Kennerly, and
C. Johnson. 2011. Safety issues
related to the electronic medical
record (EMR): Synthesis
of the literature from the last
decade, 2000–2009. Journal
of Health Care Management
56(1):31-43.
Overview 2000 to 2009 24 studies Rates of potential adverse
drug events (ADEs)
Identify problems
associated with health
IT that health care
leaders need to be
aware of when implementing
health
IT systems
–	Although health IT can be associated with greater patient
safety (e.g., resolve legibility problems), it can lead to
unintended consequences such as
•	Increases in coordination load for clinicians resulting in
new opportunities for error
•	Overdependence on health IT, particularly when incorrect
information is entered into the system, and making errors
•	Alert fatigue
–	When implementing health IT systems, health care leaders
must be aware of potential problems and be prepared to address
them
Shamliyan, T. A., S. Duval, J. Du,
and R. L. Kane. 2008. Just what
the doctor ordered. Review of
the evidence of the impact of
computerized physician order
entry system on medication
errors. Health Services Research
43(1 Pt 1):32-53.
CDS 1990 to 2005 12 studies Rate of MEsTo determine if electronic
ordering with
CDS lowers medication
errors (MEs) as compared
to handwritten
orders
–	Computerized physician order entry (CPOE) can be
associated with a reduction in MEs, particularly when
used with a CDS
–	CPOE use is associated with
•	66 percent reduction in total prescribing errors in adults
(odds ratio [OR] 0.34; 95 percent confidence interval
[CI] 0.22 to 0.52)
•	A positive tendency in children, but not statistically
significant (OR .31; 95 percent CI 0.09 to 1.02)
Pearson, S.-A., A. Moxey, J.
Robertson, I. Hains, M. Williamson,
J. Reeve, and D. Newby.
2009. Do computerised clinical
decision support systems for
prescribing change practice?
A systematic review of the
literature (1990-2007). Health
Services Research 9:154.
Clinical decision
support
(CDS)
1990 to 2007 56 studies Effectiveness of CDS systems
in supporting prescribing
during the following:
–	Initiation of treatment
–	Before drug selection
–	After drug selection
To determine CDS systems’
impact on specific
aspects of prescribing
–	CDS systems were more effective after drug selection in
•	Flagging key safety issues (e.g., drug–drug interaction
(DDI) alerts and warnings against prescribing potentially
inappropriate medications for the elderly)
•	Medication messages, such as suggesting alternative
drug treatments
Tan, K., P. R. F. Dear, and S. J.
Newell. 2005. Clinical decision
support systems for neonatal
care. Cochrane Database of
Systematic Reviews
(2):CD004211.
CDS 1966 to 2007 3 studies
–	2 randomized
control
trials
–	1 randomized
crossover
trial
–	Mortality within first 28 days
of life
–	Mortality within the first year
of life
–	Effects on physician or
nursing staff performance
–	Staff satisfaction or
compliance
–	Costs
To assess how newborn
mobility and mortality
is affected by the use
of CDS in CPOE, computerized
physiological
monitoring, diagnostic,
and prognostic systems
There are too few randomized trials and data to determine
the benefits or harms of CDS systems in neonatal care
Wolfstadt, J. I., J. H. Gurwitz, T.
S. Field, M. Lee, S. Kalkar, Wei
Wu, and P. A. Rochon. 2008 The
effect of computerized physician
order entry with clinical
decision support on the rates of
adverse drug events: A systematic
review. Journal of General
Internal Medicine 23(4):451-458.
CDS 1994 to 2007 10 studies Rate of ADEsTo determine how
CPOE systems with
CDS components
impact the rate of ADEs
–	More research is needed to determine the impact
of CPOE systems with CDS components
–	5 of 10 studies focusing on CDS systems’ impact on
ADE rates found a significant reduction in the number
of ADEs
Ammenwerth, E., P. Schnell-Inderst,
C. Machan, and U. Siebert.
2008. The effect of electronic
prescribing on medication errors
and adverse drug events:
A systematic review. Journal of
the American Medical Informatics
Association 15:585-600.
CDS e-
prescribing
1992 to 2004 27 controlled
field and
pretest-posttest
studies
Relative risk of
–	MEs and
–	ADEs
To determine the effects
of CPOE on the relative
risk reduction of MEs
and ADEs
–	Studies show that the implementation of CPOE, especially
those with CDS, can reduce the relative risk of MEs and ADEs
–	However, these studies
•	Differ substantially in setting and design
•	Were often weak, with many before–after trials and
insufficient descriptions to assess the comparability
of the study and control groups
–	23 studies showed a 13-99 percent relative risk reduction
for MEs
–	4 studies showed a 30-84 percent relative risk reduction
for ADEs
–	6 studies showed a 35-98 percent relative risk reduction
for potential ADEs
Conroy, S., D. Sweis, C. Planner,
V. Yeung, J. Collier, L. Haines,
and I. C. K. Wong. 2007. Interventions
to reduce dosing
errors in children: A systematic
review of the literature. Drug
Safety 30(12):1111-1125.
CDS e-
prescribing
–	Pre August
2004;
–	September
2004 to
October
2006
28 studies Medication error rateTo determine the effect
of CPOE systems, with
and without CDS, on the
risk of dose calculation
errors in pediatric
medicine
–	In most studies, CPOE with CDS was associated with a large
reduction in medication error rate
–	Some studies showed a medication error rate of zero after
the implementation of CPOE with CDS
–	One study showed a significant increase in mortality after the
implementation of CPOE
Durieux, P., L. Trinquart, I. Colombet,
J. Nies, R. T. Walton, A.
Rajeswaran, M. Rège-Walther,
E. Harvey, and B. Burnand.
2008. Computerized advice
on drug dosage to improve
prescribing practice. Cochrane
Database of Systematic Reviews
(3):CD002894.
CDS e-
prescribing
1966 to 2007 23 studies –	Change in the behavior of
the health care provider
(e.g., changes in the dose
of drug used)
–	Change in the health of
patients resulting from
computerized advice
(e.g., ADEs)
To determine impact of
computerized advice on
drug dosage
–	Use of computerized advice had no effect on adverse reactions
–	Computerized advice for drug dosage resulted in the following
benefits:
• Increasing
–	Initial dose (SMD 1.12, 95 percent CI 0.33 to 1.92)
–	Serum concentrations (SMD 1.12, 95 percent
CI 0.43 to 1.82)
• Reducing
–	Time to therapeutic stabilization (SMD _0.55, 95 percent
CI _1.03 to _0.08)
–	Risk of toxic drug level (rate ratio 0.45, 95 percent CI
0.30 to 0.70)
–	Length of hospital stay (SMD _0.35, 95 percent CI _0.52
to _0.17)
Reckmann, M. H., J. I. Westbrook,
Y. Koh, C. Lo, R, and O.
Day. 2009. Does computerized
provider order entry reduce
prescribing errors for hospital
inpatients? A systematic
review. Journal of the American
Medical Informatics Association
16(5):613-623.
e-prescribing 1998 to 2007 13 studies Rate of prescription errorsTo evaluate the evidence
regarding whether
CPOE systems reduce
prescribing errors among
hospital inpatients
–	Little to no evidence to supports the hypothesis
that CPOE systems reduce prescription errors
–	Studies supporting the link between CPOE use and a
reduction in errors are limited by modest study sample
sizes and designs:
•	Data collections were usually limited to no more than
two wards
•	Control groups were generally not used
•	Severity of data was generally not reported
George, J., and P. S. Bernstein.
2009. Using electronic medical
records to reduce errors and
risks in a prenatal network.
Current Opinion in Obstetrics
& Gynecology 21(6):527-531.
EHR 1999 to 2009 n/a –	Improvement in the delivery
of patient care
–	Complete documentation
of a patient’s history
–	Reduction in medication
errors
To review the impact of
implementing EMRs on
quality of obstetrics care
–	It is not clear how EMRs affect patient outcomes
–	EMR implementation is associated with high costs; however,
EMRs can provide for better quality of care because
•	Benefits can outweigh the use over the long term
–	EMRs allow for more complete, accurate, and rapid access to
data
Coiera, E., J. I. Westbrook, and
J. C. Wyatt. 2006. The safety
and quality of decision support
systems. Methods of Information
in Medicine 45:20-25.
CDS e-
prescribing
n/a n/a –	Error ratesTo identify and determine
the frequency of
errors associated with
CDS systems
–	Poorly designed, implemented, and used CDS systems can
lead to harm
–	The level of CDS performance is dependent on complex
sociotechnical interactions within the system
–	Understanding the safety issues surrounding CDS can lead
to more safely designed systems and contribute to safer
outcomes delivered
by busy or poorly resourced clinicians
PREPUBLICATION COPY: UNCORRECTED PROOFS
TABLE B-2
Studies Examining Patient Safety and Health IT
Study
Study
Purpose Method Relevant Findings
Health IT
Component Time Frame Sample Size Outcome Measures
Amarasingham, R., L. Plantinga,
M. Diener-West, M. J.
Gaskin, and N. R. Powe. 2009.
Clinical information technologies
and inpatient outcomes.
Archives of Internal Medicine
169(2):108–114.	
Overview	 December
2005 to May
2006	
41 hospitals	 –	Use of health IT as
measured by the Clinical
Information Technology
Assessment Tool (CITAT)
–	Mortality
–	Complications
–	Costs
–	Length of stay	
To determine the
relationship of cost
and rate of incidence
with the increase use
of health IT
–	Higher health IT usage was associated with
lower complications
–	A 10-point increase (using the CITAT scoring system)
in the automation of notes and records was associated
with a 15 percent decrease in the adjusted odds of fatal
hospitalizations (OR 0.85; 95 percent CI 0.74 to 0.97)
–	Increased use of CPOE was associated with a decrease
of deaths by
•	Myocardial infarction, 9 percent
•	Coronary artery bypass graft procedures,
55 percent
–	Increased use of CPOE and CDS systems was
associated with a 16 percent decrease in complications
(OR 0.84; 95 percent CI 0.79 to 0.90)
–	Use of CPOE and CDS systems resulted in
lower admission costs:
•	Test results, –$110
•	Order entry, –$132
•	Decision support, –$538 (p < 0.05)
Cross-
sectional
study	
Culler, S. D., J. N. Hawley, V.
Naylor, and K. J. Rask. 2007.
Is the availability of hospital
IT applications associated
with a hospital’s risk adjusted
incidence rate for patient
safety indicators: Results
from 66 Georgia hospitals.
Journal of Medical Systems
31(5):319–327.	
Overview 	 August to
December
2003	
66 acute care
community
hospitals
The availability of IT applications
compared with
a hospital’s risk adjusted
incidence rate per 1,000 hospitalizations
for anesthesia
complications, death in lowmortality
DRGs, decubitus
ulcers, failure to rescue,
foreign body left during
procedure, iatrogenic Pneumothorax,
infection due to
medical care, postoperative
hip fracture, postoperative
hemorrhage or hematoma,
postoperative physiologic
and metabolic derangement,
postoperative respiratory
failure, postoperative pulmonary
embolism or deep vein
thrombosis, postoperative
sepsis, postoperative wound
dehiscence, accidental puncture
or laceration 	
To determine how the
availability of health
IT components affects
the risk adjusted
incidence rate for
Patient Safety Indicators
(PSIs)	
Very little statistically significant correlation between the
availability of health IT applications and PSIs
–	Greater availability of health IT was associated with
significantly lower rates of postoperative hemorrhage
or hematoma
–	All other PSIs were not significantly affected or had
significantly increased rates
Observational
study	
Johnson, K., D. Chark, Q. X.
Chen, A. Broussard, and S. T.
Rosenbloom. 2008. Performing
without a net: Transitioning
away from a health
information technologyrich
training environment.
Academic Medicine
83(12):1179–1186.	
Overview	 2004 to 2005	 –	255 practitioners
who
transferred
from health
IT–rich
to health
IT–poor en-
vironments
–	73 practitioners
who
transferred
to an equally
health
IT–rich
environment
The self-reported impact on
perception of health IT on
–	Safety
–	Evidence-based practice
–	Efficiency
–	Communication	
To determine how
transferring from a
health IT–rich environment
to a health
IT–poor environment
affects practitioners’
self-perceptions of
competence, practice
efficiency, and
patient safety
Transition from a health IT–rich to a health IT–poor environment
is associated with a perception of decreased
– Safety (p < 0.02)
–	Efficiency (p < 0.0001)
Cross-
sectional
survey	
Magrabi, F., M. Ong, W. Runciman,
and E. Coiera. 2009.
An analysis of computerrelated
patient safety incidents
to inform the development
of a classification.
Journal of the American
Medical Informatics Association
17:663–670.	
Overview	 July 2003 to
June 2005	
42,616
incidents	
Rate and type of computer
related incidents	
To determine the
frequency and type
of patient safety
incidents that are
associated with computer
use problems	
–	Of all reported incidences, the rate of computerrelated
incidents was 0.2 percent, of which
•	55 percent were machine related (software and
hardware related)
•	45 percent were caused by human–computer
interaction problems
–	Consequences of computer-related incidents:
•	3 percent of incidents resulted in harm
•	34 percent resulted in no noticeable consequences
•	38 percent resulted in noticeable consequences,
but no harm
Retrospective
analysis	
Agostini, J. V., J. Concato, and
S. K. Inouye. 2007. Use of a
computer-based reminder to
improve sedative-hypnotic
prescribing in older hospitalized
patients. Journal of the
American Geriatrics Society
55(1):43–48.
Alerts	 N/A –	Preimple-
mentation:
12,356
patients
–	Postimple-
mentation:
12,153
patients
Frequency of prescription
of four sedative-hypnotic
drugs:
–	Diphenhydramine
–	Diazepam
–	Lorazepam
–	Trazodone	
To determine whether
a computerized
reminder system
improves sedativehypnotic
prescribing
in hospitalized older
people
–	Implementation of computerized reminder system
improved sedative-hypnotic prescribing for older
persons in acute care
–	95 percent of patients were successfully directed to a
safer sedative-hypnotic drug or a nonpharmacological
sleep protocol
Prospective
pre- and
postinterven-
tion	
van der Sijs, H., L. Lammers,
A. van den Tweel, J. Aarts,
M. Berg, A. Vulto, and
T. van Gelder. 2009. Timedependent
drug–drug interaction
alerts in care provider
order entry: Software may
inhibit medication error
reductions. Journal of the
American Medical Informatics
Association 16(6):864–868.	
Alerts	 October 2004
to July 2007	
–	Study 1:
8 internal
medicine
wards
–	Study 2:
28 internal
medicine
wards
Significant drug administration
error rates
To determine the
effect of alerts on the
rate of errors caused
by time-dependent
drug–drug interactions
(TDDIs)
–	Significant drug administration errors were
insufficiently reduced
–	Significant drug administration error rates were
reduced in the first study by 20.2 percent (p < 0.05)
–	Second study found a reduction of 1.5 percent of
significant drug administration errors (p > 0.05)
Retrospective
analysis	
Isaac, T., J. S. Weissman,
R. B. Davis, M. Massagli, A.
Cyrulik, D. Z. Sands, and S. N.
Weingart. 2009. Overrides of
medication alerts in ambulatory
care. Archives of Internal
Medicine 169(3):305–311.	
Alerts January to
September
2006	
–	233,537
medication
safety alerts
–	2,872
clinicians	
Acceptance rates of alerts 	To determine the
acceptance rate of
medication alerts
and whether the type
and severity of alert
affects acceptance
rates	
–	Clinicians override most medication alerts
–	61.6 percent of generated alerts were
high-severity alerts
–	Clinicians accepted
•	9.2 percent of drug interaction alerts
•	23.0 percent of allergy alerts
•	10.4 percent of high-severity interaction alerts
•	7.3 percent of moderate-severity interaction alerts
•	7.1 percent of low-severity interaction alerts
Retrospective
study	
Lin, C. P., T. H. Payne, W. P.
Nichol, P. J. Hoey, C. L. Anderson,
and J. H. Gennari. 2008.
Evaluating clinical decision
support systems: Monitoring
CPOE order check override
rates in the Department of
Veterans Affairs’ computerized
patient record system.
Journal of the American Medical
Informatics Association
15(5):620–626.	
Alerts	 –	Period 1:
January 4,
2006, to
January 6,
2006
–	Period 2:
January 9,
2006, January
11, 2006	
908 critical
order checks	
–	Frequency of order
check types
–	Frequency of order
check overrides by
order check type	
To compare the 2001
and 2006 override
rates of critical order
checks (computergenerated
recommendation
and alerts for
potential medication
allergies, interactions
or overdosing)
–	Clinicians override critical drug–drug and drug-allergy
order checks at a high rate
–	Critical override rate for January 2006 and 2001
(2001 data taken from a previous study):
•	DDIs:
–	2006, 87 percent
–	2001, 88 percent (p = 0.85)
•	Drug-allergy
–	2006, 81 percent
–	2001, 69 percent (p = 0.005)
Retrospective
analysis	
Raebel, M. A., J. Charles, J.
Dugan, N. M. Carroll, E. J.
Korner, D. W. Brand, and D. J.
Magid. 2007. Randomized trial
to improve prescribing safety
in ambulatory elderly patients.
Journal of the American Geriatrics
Society 55(7):977–985.	
Alerts	 May 2005 to
May 2006	
59,680
patients age
65 and older	
Rate at which inappropriate
medications were dispensed	
To evaluate the
effectiveness of a
computerized system
designed to alert
pharmacists when
patients aged 65
and older were
newly prescribed
potentially inappropriate
medications	
–	Alerts can decrease the dispensing rate of potentially
inappropriate medication prescribed
–	Percentage of patients prescribed inappropriate
medication:
•	Control group, 2.2 percent
•	Intervention group. 1.8 percent (p = 0.002)
Prospective,
randomized
trial	
PREPUBLICATION COPY: UNCORRECTED PROOFS
TABLE B-2 Continued
Study
Study
Purpose Method Relevant Findings
Health IT
Component Time Frame Sample Size Outcome Measures
Sellier, E., I. Colombet, B.
Sabatier, G. Breton, J. Nies, E.
Zapletal, J. B. Arlet, D. Somme,
and P. Durieux. 2009. Effect of
alerts for drug dosage adjustment
in inpatients with renal
insufficiency. Journal of the
American Medical Informatics
Association 16(2):203–210.	
Alerts August 2006
to August
2007	
–	603 patients
–	38 physicians
–	2 medical
departments
within one
hospital	
Proportion of inappropriate
first prescriptions, according
to recommendation	
To determine the
impact of CPOE
alerts at the time of
ordering medication
on the rate of inappropriate
medication
prescribed	
–	Alerts did not significantly impact the rate of
inappropriate first prescriptions (19.9 vs. 21.3 percent
[p = 0.63])
–	Residents made fewer errors with alert system
(OR 0.69; 95 percent CI 0.41 to 1.15)
–	Senior physicians made more inappropriate prescriptions
with alert system (OR 1.88; 95 percent CI 0.91
to 3.89)
Controlled
trial
Shah, N. R., A. C. Seger, D.
L. Seger, J. M. Fiskio, G. J.
Kuperman, B. Blumenfeld, E.
G. Recklet, D. W. Bates, and T.
K. Gandhi. 2006. Improving
acceptance of computerized
prescribing alerts in ambulatory
care. Journal of the
American Medical Informatics
Association 13(1):5–11.
Alerts 2004 to
January 2005
–	701 clinicians
–	31 adult
primary care
practices	
–	The extent an alert design
minimizes workflow
interruptions
–	Clinician acceptance rates
of selective alerts
–	The specific types of
alerts clinicians accepted
most frequently
–	The reasons clinicians gave
for overriding alerts
To determine whether
a tiered alert system,
one which assigned
alerts of less serious
magnitude to a noninterruptive
display,
would increase clinical
acceptance of more
serious interruptive
drug alerts
–	The implementation of tiered alerts successfully
raised clinician acceptance of more selective
interruptive alerts, as compared to acceptance
rates of previous studies
–	Total number of drug alerts under tiered system: 18,115
•	71 percent were noninterruptive, less serious alerts
•	29 percent were interruptive, more serious alerts
–	Acceptance of interruptive alert under tiered system:
67 percent
Observational
study
Singh, H., E. J. Thomas, D. F.
Sittig, L. Wilson, D. Espadas,
M. M. Khan, and L. A. Petersen.
2010. Notification of abnormal
lab test results in an electronic
medical record: Do any
safety concerns remain? The
American Journal of Medicine
123(3):238–244.	
Alerts	 May to
December
2008	
–	One large
Veterans
Affairs mul-
tispecialty
ambulatory
clinic
–	5 satellite
clinics
–	78,158 tests
–	1,163 transmitted
alerts
–	Unacknowledged alerts
–	Lack of timely follow-ups
to alerts (not responded
to within 30 days)	
To determine whether
automated notifications
of abnormal
laboratory alerts in an
EMR received timely
follow-up actions
Safety concerns remain due to unacknowledged alerts
and lack of follow-up:
– 10.2 percent of all alerts were unacknowledged
–	6.8 percent of all alerts lackd timely follow-up
Retrospective
cohort study	
van der Sijs, H., A. Mulder, T.
van Gelder, J. Aarts, M. Berg,
and A. Vulto. 2009. Drug
safety alert generation and
overriding in a large Dutch
university medical centre.
Pharmacoepidemiology and
Drug Safety 18(10):941–947.	
Alerts	 –	In wards:
25 days
–	Hospital:
24 months 	
–	Observation
study:
•	2 internal
medicine
wards
•	6 residents
(3 per ward)
•	515 prescrip-
tions
–	Retrospective
analysis:
371,261
prescribed
orders from
one hospital
–	Rate of alerts
–	Alert types
–	Rate of overrides	
To determine rate and
types of drug safety
alerts generated and
overridden	
–	Observation study of internal medicine wards:
•	Drug safety alerts were generated in 34 percent
of all orders, 91 percent of which were overridden
•	Type of alert (frequency/override rate)
–	Drug safety alert (56/98 percent)
–	Overdose (15/89 percent)
–	Duplicate orders (29/80 percent)
–	Retrospective analysis of entire hospital:
•	20.2 percent of all orders were overridden
•	59 percent of all alerts were DDI overrides,
of which
–	22.4 percent were low-level alerts
–	54.5 percent were medium-level alerts
–	19.3 percent were high-level alerts
–	3.8 percent were unknown
–	In wards:
Disguised
observation
study
of internal
medicine
wards
–	Hospital:
Retrospective
analysis
for entire
hospital	
van der Sijs, H., T. van Gelder,
A. Vulto, M. Berg, and J. Aarts.
2010. Understanding handling
of drug safety alerts:
A simulation study. International
Journal of Medical
Informatics 79(5):361–369.
Alerts Unknown –	18 physicians
–	35 orders of
predefined
patient
cases
–	211 generated
alerts
–	Errors in responding
to alerts
–	Reason why alert was
handled incorrectly
To determine frequency
and reason
why drug safety alerts
generated by a CPOE
are handled incorrectly
–	30 percent of all the generated alerts were handled
incorrectly because
•	An incorrect action was chosen (24 percent of all
alerts) and/or
•	The action was based on incorrect reasoning
(16 percent of all alerts)
–	Types of errors:
•	Rule-based, 63 percent
•	Knowledge-based, 24 percent
•	Skill-based, 13 percent
–	25 percent of respondents demonstrated signs of
alert fatigue
Disguised
observation
study
Ohsaka, A., M. Kobayashi,
and K. Abe. 2008. Causes of
failure of a barcode-based
pretransfusion check at the
bedside: Experience in a
university hospital. Transfusion
Medicine 18(4):216–222.
Barcode April 2004
to December
2007
43,068 blood
components
transfused
–	Rate at which transfusions
were performed without
electronic barcode
checking
–	Reasons for not performing
checks, including
•	Human errors
•	Handheld device errors
•	System errors and
•	Wristband errors
To determine the
reason for the failures
to check bedside
barcode identification
before blood admin-
istration
–	2.2 percent of transfusions were performed without
electronic checking
–	Reasons for not performing check:
•	Human error, 84.7 percent
•	Handheld device error, 7.7 percent
•	System error, 5.2 percent
•	Wristband error, 2.4 percent
Retrospective
analysis
Franklin, B. D., K. O’Grady,
P. Donyai, A. Jacklin, and N.
Barber. 2007. The impact
of a closed-loop electronic
prescribing and administration
system on prescribing
errors, administration errors
and staff time: A before-andafter
study. Quality & Safety in
Health Care 16(4):279–284.
Barcode –	3 to 6
months pre-
intervention
–	6 to 12
months
postinter-
vention
–	2,319 newly
written
medications,
–	906 oppor-
tunities
for error
–	56 drug
rounds
–	Percentage of new medication
orders with a prescribing
error
–	Percentage of doses
with medication administration
errors
–	Percentage of medication
administered without
checking patient identity
–	Time spent prescribing
and providing a ward
pharmacy service
–	Nursing time spent on
medication tasks
To examine how
prescribing and
administration errors,
confirmation of
patient identity, and
staff time are affected
by a closed-loop
electronic prescribing,
automated dispensing,
barcode patient
identification and
electronic medication
administration record
(eMAR) system
–	eMAR can reduce errors and increase patient identification
rates but is associated with an increase in time
spent on medication-related tasks
–	Prescribing errors were reduced by 1.8 percent
(p < 0.001)
–	Medication administration errors were reduced by
2.7 percent (p = 0.005)
–	Patient identification not checked before administration
of medication was reduced by 63.7 percent
(p < 0.001)
–	Time staff spent prescribing rose 24 seconds
(from 15 to 39 seconds) (p = 0.03)
–	Time per drug administration round was reduced by
10 minutes (from 50 to 40 minutes) (p = 0.006)
–	Nursing time on medication tasks outside of drug
rounds rose 7.6 percent (from 21.1 to 28.7 percent)
(p = 0.006)
Prospective
cohort study
Koppel R., T. Wetterneck, J.
L. Telles, and B. Karsh. 2008.
Workarounds to barcode
medication administration
systems: Their occurrences,
causes, and threats to patient
safety. Journal of the American
Medical Informatics Association
15(4):408–423.
Barcode 2003 to 2006 –	A four-
hospital,
929-bed
east coast
health care
system
–	One
academic
tertiary-care
hospital
–	Workarounds
–	Potential MEs associated
with workarounds
To identify workarounds
used with
barcode medication
administration systems
(BCAS) and the
potential MEs those
workarounds may
cause
15 types of workarounds which could lead to patient
harm were discovered, including
•	User administers medication without scanning
patient ID
•	Patient barcode ID placed on another object,
not on patient,
•	User gives partial dose but electronically
documents full dose
–	Observing
and shadowing
staff
–	Interviews
–	Participating
in staff
meetings
–	Analyzing
override
logs
Poon, E. G., C. A. Keohane, C.
S. Yoon, M. Ditmore, A. Bane,
O. Levtzion-Korach, T. Moniz,
J. M. Rothschild, A. B. Kachalia,
J. Hayes, W. W. Churchill,
S. Lipsitz, A. D. Whittemore,
D. W. Bates, and T. K. Gandhi.
2010. Effect of bar-code
technology on the safety of
medication administration.
New England Journal of Medicine
362(18):1698–1707.
Barcode February to
October 2005
–	6,723
medication
administra-
tions
–	3,082
order tran-
scriptions
–	Errors in timing of medication
administration that
were early or late by more
than 1 hour
–	Errors unrelated to timing
–	Transcription errors
To assess how the
rates of errors in
order transcription
and medication
administration are
impacted by the
implementation
of the barcode
eMAR system
–	Use of the barcode eMAR was associated with
a substantial reduction in
•	Rate of errors in order transcription (p < 0.001)
•	Medication administration (p < 0.001)
•	Potential adverse drug events (p < 0.001)
–	Early or late medication administration errors were
reduced by 27.3 percent (p = 0.001)
–	Nontiming errors had a relative reduction rate of
41.4 percent (p < 0.001)
Prospective,
quasi-experi-
mental
PREPUBLICATION COPY: UNCORRECTED PROOFS
TABLE B-2 Continued
Study
Study
Purpose Method Relevant Findings
Health IT
Component Time Frame Sample Size Outcome Measures
Poon, E. G., J. L. Cina, W.
Churchill, N. Patel, E. Featherstone,
J. M. Rothschild, C. A.
Keohane, A. D. Whittemore,
D. W. Bates, and T. K. Gandhi.
2006. Medication dispensing
errors and potential adverse
drug events before and
after implementing bar code
technology in the pharmacy.
Annals of Internal Medicine
145(6):426–434.
Barcode –	November
and December
2003
–	May and
September
2004
–	115,164
preimple-
mentation
doses
–	253,984
postimple-
mentation
doses
–	Rates of dispensing errors
and potential ADEs related to
•	Carousel fill process
(medications
are scanned during
stocking, retrievals are
directed by the carousel
machine, and only
1 dose per batch was
scanned during filling)
•	2-day fill process
(medication doses
are stocked manually,
retrieved by hand, and
each dose was scanned
during filling)
•	Alternate zone fill
(medication doses
are manually stocked,
manually retrieved,
and only 1 dose per
batch was scanned)
To determine the effect
of barcode technology
on dispensing
errors and ADEs
–	A substantial decrease in dispensing errors and potential
ADEs were associated with the implementation of
three kinds of barcode systems:
–	2-day fill process reduced
•	Dispensing errors by 96 percent (p < 0.001)
•	Potential ADEs by 97 percent (p < 0.001)
–	Carousel fill process reduced
•	Dispensing errors by 93-96 percent (p < 0.001)
•	Potential ADEs by 86-97 percent (p < 0.001)
–	Alternate zone fill reduced
•	Dispensing errors by 93 percent (p < 0.001)
•	Potential ADEs by 86 percent (p <0.001)
Before-andafter
study
using direct
observations
Graham, T. A., A. W. Kushniruk,
M. J. Bullard, B. R. Holroyd,
D. P. Meurer, and B. H. Rowe.
2008. How usability of a
web-based clinical decision
support system has the potential
to contribute to adverse
medical events. AMIA Annual
Symposium Proceedings
6:257–261.
CDS 2006 to 2007 7 attending
emergency
physicians
Number and type of ADEsTo determine how usability
of CDS systems
graphic interfaces
contribute to ADEs
422 events were recorded, including
–	Events where the system precluded the
desired choice
– Subjects either ignored or overrode the
CDS system purposefully
– Subjects not having specific options to
select common conditions
Observational
studies
Fitzgerald, M., P. Cameron,
C. Mackenzie, N. Farrow, P.
Scicluna, R. Gocentas, A.
Bystrzycki, G. Lee, G. O’Reilly,
N. Andrianopoulos, L. Dziukas,
D. J. Cooper, A. Silvers,
A. Mori, A. Murray, S. Smith,
Y. Xiao, D. Stub, F. T. McDermott,
and J. V. Rosenfeld.
2011. Trauma resuscitation
errors and computer-assisted
decision support. Archives of
Surgery 146(2):218–225.
CDS January 2006
to February
2008
–	1 level
I adult trauma
center
–	1,171 patients
–	Error (deviation from
trauma care algorithms)
rate per patient treated
–	Morbidity
To determine whether
management errors in
the first 30 minutes of
trauma resuscitation
can be reduced by
CDS use
Management error rates reduced by CDS use:
–	Control, 0.4 reduction per patient (2.53 to 2.13,
p = 0.004)
–	Intervention, 0.17 reduction per patient (2.30 to 2.13,
p = 0.04)
Prospective,
open,
randomized,
controlled
interventional
study
Graumlich, J. F., N. L. Novotny,
G. Stephen Nace, H.
Kaushal, W. Ibrahim-Ali, S.
Theivanayagam, L. William
Scheibel, and J. C. Aldag.
2009. Patient readmissions,
emergency visits, and adverse
events after software-assisted
discharge from hospital:
Cluster randomized trial.
Journal of Hospital Medicine
4(7):E11–19.
CDS –	November
2004 to
January
2007
–	Follow-up
occurred
for 6 months
after dis-
charge
631 inpatients –	Hospital readmission
rates within 6 months
of discharge
–	Emergency department
visit within 6 months
–	Postdischarge adverse
event within 1 month
To determine the
impact of a CPOE discharge
software application
that prompts
physicians to enter
pending prescriptions
and orders, automatically
generating
discharge reports,
discharge papers,
patient instructions,
and prescriptions
Differences between CPOE software and CPOE software
tailored to discharge were insignificant:
–	Hospital readmission, 37.0 percent vs. 37.8 percent
(p = 0.894)
–	Emergency department visit, 35.4 percent vs. 40.6
(p = 0.108)
–	Adverse event within 1 month, 7.3 percent vs.
7.3 percent (p = 0.884)
Cluster randomized
controlled
trial
Wadhwa, R., D. B. Fridsma,
M. I. Saul, L. E. Penrod, S.
Visweswaran, G. F. Cooper,
and W. Chapman. 2008. Analysis
of a failed clinical decision
support system for management
of congestive heart failure.
AMIA Annual Symposium
Proceedings 6:773–777.
CDS July to September
2006
112 patients –	Sensitivity
–	PPV
–	Frequency of alert and
false positives
–	Physician responses
to alerts
To determine if CDS
systems can successfully
identify patients
with primary congestive
heart failure
(CHF)
–	CDS systems performed poorly
–	CDS systems had problems with false negatives:
•	Sensitivity, 0.79
•	Positive predictive value (PPV), .11
–	CDS systems had excessive alerts:
•	CDS system issued multiple alerts (74 percent of
patients)
•	CDS systems issued alerts for patients without
primary CHF (63 percent)
–	Physicians did not respond to alerts the first time
Retrospective
analysis
Campbell, E. M., D. F. Sittig, K.
P. Guappone, R. H. Dykstra,
and J. S. Ash. 2007. Overdependence
on technology: An
unintended adverse consequence
of computerized
provider order entry. AMIA
Annual Symposium Proceedings
5:94–98.
CPOE N/A 5 hospitals Type of ADEsTo identify adverse
consequences caused
by overdependence
on CPOE systems
Overdependence can lead to the following adverse
consequences:
–	Lack of backup systems can lead to chaos when
systems are down
–	Users may trust inaccurate data
–	Clinicians, who are overdependent on systems,
may not be able to adequately efficiently perform
without the system
Observational
analysis
Ramnarayan, P., G. C. Roberts,
M. Coren, V. Nanduri, A. Tomlinson,
P. M. Taylor, J. C. Wyatt,
and J. F. Britto. 2006. Assessment
of the potential impact
of a reminder system on the
reduction of diagnostic errors:
A quasi-experimental study.
Medical Informatics & Decision
Making 6:22.
CDS
diagnosing
February to
August 2002
–	76 subjects
•	18 consul-
tants
•	24 registrars,
•	19 senior
house
officers
•	15 students
–	751 case
episodes
–	Diagnostic errors of
omission (DEO): failing
to include all clinically
important diagnoses, after
consultation with webbased
pediatric diagnostic
reminder system
To evaluate the impact
of a web-based
pediatric diagnostic
reminder system that
suggests important
diagnoses during
clinical assessment
on the quality of clinical
decisions during
acute assessment
–	A web-based pediatric diagnostic reminder system
can reduce the rate of errors
–	The mean count of DEOs fell from 5.5 to 5.0
(p < 0.001) after implementation
–	Reminder system prompted an order of an important
test in 10 percent of case episodes
Quasi-experi-
mental
Gurwitz, J. H., T. S. Field, P.
Rochon, J. Judge, L. R. Harrold,
C. M. Bell, M. Lee, K.
White, J. LaPrino, J. Erramuspe-Mainard,
M. DeFlorio,
L. Gavendo, J. L. Baril, G.
Reed, and D. W. Bates. 2008.
Effect of computerized provider
order entry with clinical
decision support on adverse
drug events in the long-term
care setting. Journal of the
American Geriatrics Society
56(12):2225–2233.
CDS
ePrescribing
August 2006
and August
2007
–	1,118 pa-
tients;
–	29 resident
care units
–	Number and severity of
ADEs
–	Whether ADEs were pre-
ventable
To determine if CPOE
with CDS are effective
at preventing ADEs in
long-term care
–	CDS did not reduce preventable ADEs
–	No significant differences found between CDS and
control groups:
•	ADEs, 1.06 (95 percent CI 0.92 to 1.23)
•	Preventable ADEs, 1.02 (95 percent CI 0.81 to 1.30)
•	Severe ADEs, 1.07 (95 percent CI 0.82 to 1.40)
Cluster,
randomized
control trial
Bedouch, P., B. Allenet, A.
Grass, J. Labarère, E. Brudieu,
J.-L. Bosson, and J. Calop.
2009. Drug-related problems
in medical wards with a
computerized physician order
entry system. Journal of Clinical
Pharmacy & Therapeutics
34(2):187–195.
CDS
ePrescribing
November
2001 to April
2003
8,152 patients Rate and type of drug-related
problems
To determine the
type and frequency
of drug-related problems
that occur during
the use of CPOE
system with CDS
–	33 drug-related problems per 100 admissions
–	Most common drug-related problems were the following:
•	Nonconformity to guidelines or contraindication
(29.5 percent)
•	Improper administration (19.6 percent)
•	Drug interaction (16.7 percent)
•	Overdose (12.8 percent)
Prospective
study
Abarca, J., L. R. Colon, V.
S. Wang, D. C. Malone, J.
E. Murphy, and E. P. Armstrong.
2006. Evaluation of
the performance of drugdrug
interaction screening
software in community and
hospital pharmacies. Journal
of Managed Care Pharmacy
12(5):383–389.
CDS
ePrescribing
2004 –	8 community
pharmacies,
–	5 hospital
pharmacies
–	6 mock
patients
–	25 medica-
tions
–	37 DDIs
–	16 clinically
meaningful
DDIs
DDI alerting system’s me-
dian:
–	Sensitivity
–	Specificity
–	PPV
–	Negative predictive
value (NPV)
To determine the
effectiveness of DDI
screening software in
identifying significant
DDIs
–	Significant variation in effectiveness among hospital
pharmacy computer systems, even among systems
manufactured by the same vendor
–	Computer systems correctly classified 12 of the
16 DDI pairs
–	Median sensitivity: 16 DDI pairs was 0.88 (range
0.81-0.94)
–	Median specificity of the systems was 0.91 (range
0.67-1.00)
–	Median PPV score was 0.88 (range 0.68-1.00)
–	Median NPV was 0.91 (range 0.86-0.95)
Observational
study
PREPUBLICATION COPY: UNCORRECTED PROOFS
TABLE B-2 Continued
Study
Study
Purpose Method Relevant Findings
Health IT
Component Time Frame Sample Size Outcome Measures
Eslami, S., A. Abu-Hanna, N.
F. de Keizer, and E. de Jonge.
2006. Errors associated with
applying decision support by
suggesting default doses for
aminoglycosides. Drug Safety
29(9):803–809.
CDS
ePrescribing
May 2002 to
December
2004
–	1 Dutch tertiary
adult
intensive
care unit
–	392 pre-
scriptions
–	253 patients
–	Rate of potential ADEsTo determine the
impact of a CPOE
system that supplies
default doses when
ordering on the rate
of potential ADEs
–	Default doses led to a significant increase in
potential ADEs
–	Default dose was wrong in 73 percent of the orders
–	Rate of potential ADEs:
•	Patients with renal insufficiencies who were given
default dosages, 86 percent
•	Patients with renal insufficiencies who were not
given default doses, 53 percent (p < 0.0001).
Retrospective
analysis
Bertsche, T., J. Pfaff, P. Schiller,
J. Kaltschmidt, M. G.
Pruszydlo, W. Stremmel, I.
Walter-Sack, W. E. Haefeli,
and J. Encke. 2010. Prevention
of adverse drug reactions in
intensive care patients by personal
intervention based on
an electronic clinical decision
support system. Intensive Care
Medicine 36(4):665–672.
CDS
ePrescribing
–	3-month
control
phase
–	3-month
intervention
phase
265 patients
(136 control,
129 interven-
tion)
–	Number of DDIs
–	DDI-related ADEs
To determine how
the rate of DDIs and
DDI-related ADEs in
intensive care patients
were effected when
senior clinicians’
written prescription
orders were
typed into a CDS and
recommendations
are printed for senior
clinicians
–	Printed CDS recommendations were associated with
a decrease in DDI and DDI-related ADEs
–	The number of patients with at least one DDI at the
end of the study decreased by 18 percent (p = 0.02)
–	The relative risk of a patient having at least one
DDI-related adverse event decreased by 43 percent
(p = 0.01)
Prospective
controlled
intervention
cohort study
Colpaert, K., B. Claus, A.
Somers, K. Vandewoude, H.
Robays, and J. Decruyenaere.
2006. Impact of computerized
physician order entry
on medication prescription
errors in the intensive care
unit: A controlled cross-sectional
trial. Critical Care 10(1).
CDS
ePrescribing
80 patient
days
–	2,510 pre-
scriptions
–	2 paperbased
units
–	1 computerized
unit
MPEs (minor and serious)To determine if the introduction
of a computerized
intensive
care unit (ICU) system
with a moderate
level of CDS reduced
the incidence and
severity of medication
prescription errors
(MPEs)
–	The computerized unit had significantly lower occurences
and severity of medication errors in the ICU
–	MPEs:
•	Computer unit, 3.4 percent
•	Paper unit, 27 percent (p < 0.001)
–	Minor MPEs:
•	Computer, 9
•	Paper, 225 (p < 0.001)
–	Serious MPEs:
•	Computer, 12
•	Paper, 35 (p < 0.001)
Prospective,
randomized
controlled
cross-sectional
trial
Cunningham, T. R., E. S. Geller,
and S. W. Clarke. 2008. Impact
of electronic prescribing in a
hospital setting: A process-focused
evaluation. International
Journal of Medical Informatics
77(8):546–554.
CDS
ePrescribing
N/A –	Intervention:
194 physi-
cians
–	Control: 159
physicians
–	Compliance with medication-ordering
protocols
–	Mean duration to first dose
of antibiotics
Examine the effects
of natural implementation
of a CPOE
system with CDS
(consisting of a
campaign promoting
the general awareness
and benefits of CPOE,
system training, and
allowing physicians to
continue using paper
medication orders)
–	CPOE medication orders were associated with
significantly greater compliance
–	Rate at which orders were 100 percent compliant:
•	CPOE orders, 59.8 percent
•	Paper orders at the intervention site, 46.7 percent
compliant (p < 0.001)
•	Paper orders at the control site, 46.6 percent
(p < 0.001)
–	Mean duration to first dose of antibiotics:
o CPOE orders, 185 min
o Paper orders, 326.2 min (p < 0.001)
Multiple-baseline,
quasi-
experimental
study, with a
nonequivalent
control site
Dallenbach, M. F., P. A. Bovier,
and J. Desmeules. 2007. Detecting
drug interactions using
personal digital assistants
in an out-patient clinic. QJM:
An International Journal of
Medicine 100(11):691–697.
CDS
ePrescribing
N/A –	1,801 drug
Prescrip-
tions;
–	591 consecutive
patients
The drug interaction
database’s
–	Sensitivity
–	Specificity
–	PPV
To determine if the
drug interaction database
(ePocrates Rx)
can correctly identify
clinically significant
adverse drug interactions
in an outpatient
setting
–	The drug interaction database can be an efficient tool
to reduce prescription error
–	Sensitivity: 81 percent (95 percent, CI 77 to 85 percent)
–	Specificity: 88 percent (95 percent, CI 86 to
89 percent)
Retrospective
chart review
Galanter, W. L., D. B. Hier, C.
Jao, and D. Sarne. 2010. Computerized
physician order entry
of medications and clinical
decision support can improve
problem list documentation
compliance. International
Journal of Medical Informatics
79(5):332–338.
CDS
ePrescribing
N/A 1,011 alerts –	Alert validity
–	Alert yield
–	Accuracy of problem
list additions
To test a CDS
mechanism that helps
maintain an electronic
problem list in a realtime
clinical environ-
ment
CDS was able to improve the problem list with minimal
diagnostic inaccuracies
–	Alert validity: 96±1 percent
–	Alert yield: 76±2 percent
–	Accurate problem list additions: 95±1 percent
Observational
study
Gandhi, T. K., S. B. Bartel,
L. N. Shulman, D. Verrier,
E. Burdick, A. Cleary, J. M.
Rothschild, L. L. Leape, and
D. W. Bates. 2005. Medication
safety in the ambulatory
chemotherapy setting.
Cancer 104(11):2477–2483.
CDS
ePrescribing
March to
December
2000
–	1,606
patients
•	1,380 adult
patients
under a
CMO system
•	226 pediatric
patients
under a
handwritten
system orders
(HWO)
–	10,112
medication
orders
–	1,602 charts
Medication error rateTo determine the
effect of a computerized
medication
ordering (CMO)
system on outpatient
chemotherapy
–	No significant difference between the medication error
rates of adult patients with CMO and pediatric patients
with handwritten orders
–	Medication error rate:
•	Adult patients with CMO, 3 percent (249/8,008)
•	Pediatric patients with HWO, 3 percent (57/2,104)
–	The relatively low medication error rate in children
may have been due to a high proportion of pediatric
patients receiving investigational protocols with very
specific dosing and dose modification parameters
Prospective
cohort study
Glassman, P. A., P. Belperio, A.
Lanto, B. Simon, R. Valuck, J.
Sayers, and M. Lee. 2007. The
utility of adding retrospective
medication profiling to
computerized provider order
entry in an ambulatory care
population. Journal of the
American Medical Informatics
Association 14(4):424–431.
CDS
ePrescribing
June 2001 to
January 2002
913 patients –	ADE rates
–	ADE severity, characterized
into four categories:
1. 	Laboratory or test
abnormality
2.	Symptoms, not serious
or serious
3.	Disability, cognitive
or physical
4.	Death
–	Preventability (determined
by the presence of an associated
conflict)
To determine whether
adding a medication
profiling program
to a CPOE system
improves safety
–	Addition of a medication profiling program did
not increase safety
–	ADE incidence had no significant difference:
•	Usual care, 37 percent
•	Provider feedback groups, 45 percent (p = 0.06)
–	ADE severity was similar:
•	Usual care group, 51 percent
•	Provider feedback group, 58 percent (95 percent
CI for the difference –15, 2 percent)
–	ADE preventability did not differ with feedback:
•	Usual care group, 16 percent
•	Provider feedback group, 17 (95 percent CI for
the difference –7 to 5; p = 0.79)
Retrospective
review
Han, Y. Y., J. A. Carcillo, S. T.
Venkataraman, R. S. B. Clark,
S. Watson, T. C. Nguyen, H.
L. Bayir, and R. A. Orr. 2005.
Unexpected increased mortality
after implementation of a
commercially sold computerized
physician order entry
system. American Academy
of Pediatrics 116.6:1506–1513.
CDS
ePrescribing
–	13 months
before
implementation
of
CPOE
–	5 months
after imple-
mentation
1,942 children Mortality rateTo determine whether
the mortality rate
among children transported
for specialized
care is reduced by the
implementation of
CPOE with CDS
An increased odds of mortality was associated with the
implementation of CPOE (OR 3.28; 95 percent CI 1.94-
5.55)
– Pre-CPOE mortality rate: 3.86 percent
– Postimplementation: 6.57 percent (p < 0.001)
Retrospective
analysis
Holdsworth, M. T., R. E. Fichtl,
D. W. Raisch, A. Hewryk, M.
Behta, E. Mendez-Rico, C. L.
Wong, J. Cohen, S. Bostwick,
and B. M. Greenwald. 2007.
Impact of computerized
prescriber order entry on the
incidence of adverse drug
events in pediatric inpatients.
Pediatrics 120(5):1058–1066.
CDS
ePrescribing
–	Pre-CPOE
period:
September
2000 to May
2001
–	Post-CPOE
period: April
to October
2004
–	Pre-CPOE:
1,197 admis-
sions
–	Post-CPOE:
1,210 admis-
sions
Number of ADEsTo evaluate how the
incidents and types of
ADEs in hospitalized
children are impacted
by the use of a CPOE
system with substantial
decision support
–	ADEs were substantially reduced by CPOE with CDS
–	Total ADEs
•	Pre-CPOE: 76
•	Post-CPOE: 37 (RR of total ADEs in post-CPOE
compared to pre-CPOE 95 percent CI 0.43
to 0.95).
–	Preventable ADEs
•	Pre-CPOE: 46
•	Post CPOE: 26
•	RR: 0.56 (95 percent CI 0.34 to 0.91)
–	Potential ADEs
•	Pre-CPOE: 94
•	Post CPOE: 35
•	RR: 0.37 (95 percent CI 0.25 to 0.55)
Prospective
cohort study
PREPUBLICATION COPY: UNCORRECTED PROOFS
TABLE B-2 Continued
Study
Study
Purpose Method Relevant Findings
Health IT
Component Time Frame Sample Size Outcome Measures
Jani, Y. H., M. A. Ghaleb, S. D.
Marks, J. Cope, N. Barber, and
I. C. K. Wong. 2008. Electronic
prescribing reduced prescribing
errors in a pediatric renal
outpatient clinic. Journal of
Pediatrics 152(2):214–218.
CDS
ePrescribing
July 1, 2005
to July 31,
2006
–	520 patients
–	2,242 items
prescribed
–	1,141 pre-
scriptions
Prescribing error rates:
–	Items missing essential
information
–	Items judged illegible
–	Number of patient visits
that were error-free
To evaluate how the
rate of incidents, type
of prescribing errors,
and the number
of error-free visits
are affected by an
electronic prescribing
(EP) system with CDS
–	Overall prescribing error rate was significantly reduced:
•	77.4 percent for handwritten items (95 percent
CI 75.3 to 79.4 percent)
•	4.8 percent for EP (95 percent CI 3.4 to
6.7 percent)
–	Items missing essential information:
•	73.3 percent pre-EP (95 percent CI 71.1 to
75.4 percent)
•	1.4 percent post-EP (95 percent CI 0.7 to
2.6 percent)
–	Items judged illegible:
•	12.3 percent pre-EP (95 percent CI 10.8 to
14 percent)
•	Zero percent post-EP
–	Percentage of patient visits that were error-free
increased by 69 percent (95 percent CI 64 to
73.4 percent)
Before-andafter
study
Kadmon, G., E. Bron-Harlev,
E. Nahum, O. Schiller, G. Haski,
and T. Shonfeld. 2009.
Computerized order entry
with limited decision support
to prevent prescription
errors in a PICU. Pediatrics
124(3):935–940.
CDS
ePrescribing
September
2005 to
September
2007
5,000 orders Rates of ADEs and medication
prescription errors
(MPEs) per period
–	Period 1: 1 month pre-CPOE
–	Period 2: 1 year post-CPOE
–	Period 3: post-CDS system
–	Period 4: post-change in
prescription authorization
To compare the
impact of prescription
error rates of weightbased
dosing in a
pediatric ICU (PICU)
with the introduction
of a CPOE without
a CDS system and a
subsequent introduction
of a CPOE with
a CDS
–	Introduction of CPOE alone had no significant impact
on potential errors:
•	Period 1 ADE, 2.5 percent
•	Period 2 ADE, 2.4 percent
•	Period 1 MPE, 5.5 percent
•	Period 2 MPE, 5.3 percent
–	Introduction of CDS system significantly lowered the
rate of potential errors:
•	Period 3 ADE, 0.8 percent (p < .05).
•	Period 4 ADE, 0.7 percent (p < 0.005)
•	Period 3 MPE, 3.8 percent (p < 0.05)
•	Period 4 MPE, 0.7 percent (p < 0.0005)
Retrospective
cohort study
Metzger, J., E. Welebob, D.
W. Bates, S. Lipsitz, and D. C.
Classen. 2010. Mixed results
in the safety performance
of computerized physician
order entry. Health Affairs
29(4):655–663.
CDS
ePrescribing
April to
August 2008
62 hospitals Percentage of simulated
orders detected that would
have led to serious ADEs
To determine the abilities
of different CPOE
systems with CDS in
different hospitals
–	Wide variation in the ability of CDS systems to detect
serious ADEs
–	Range for the percentage of test orders detected that
would have caused fatalities: 10 to 82 percent
–	Mean percent of test orders detected:
•	Overall score, 44.3 (SE: 2.3)
•	CPOE systems that are easy to implement,
61.4 (SE: 2.4)
•	CPOE systems that require configuration and
customization, 24.8 (SE: 2.6)
Quasi-experi-
mental
Poller, L., M. Keown, S. Ibrahim,
G. Lowe, M. Moia, A. G. Turpie,
C. Roberts, A. Van den Besselaar,
F. J. M. Van der Meer, A.
Tripodi, G. Palareti, C. Shiach,
S. Bryan, M. Samama, M.
Burgess-Wilson, A. Heagerty,
P. Maccallum, D. Wright, and J.
Jespersen. 2008. An international
multicenter randomized
study of computer-assisted
oral anticoagulant dosage vs.
medical staff dosage. Journal
of Thrombosis and Haemostasis
6(6):935–943.
CDS
ePrescribing
June 2002
to December
2006
–	32 centers
–	13,219
patients
–	6,503
patients
randomized
to medical
staff
–	6,716 randomized
to
computer-
assisted
dosage
Comparison of dosing by
–	Safety
–	Effectiveness
–	Clinical events by group
To compare the safety
and effectiveness
of two computerassisted
anticoagulant
dosage programs
(PARMA 5 or DAWN
AC) with dosage by
experienced medical
staff delivered
–	The computer-assisted dosage programs were
found to be similar or more effective than the manual
delivery group
–	Clinical events by computer-assisted dosage was not
significantly different (incidence rate ratio = 0.90;
95 percent CI 0.80–1.02; p = 0.10)
–	Clinical events for patients with deep vein thrombosis
or pulmonary embolism were reduced by 24 percent
(p = 0.001)
Multicenter,
randomized
trial
Saverno, K.R., L. E. Hines, T.
L. Warholak, A. J. Grizzle, L.
Babits, C. Clark, A. M. Taylor,
D. C. Malone. 2011. Ability of
pharmacy clinical decisionsupport
software to alert
users about clinically important
drug-drug interactions.
Journal of the American Medical
Informatics Association 18:
32–37.
CDS
ePrescribing
December
2008 to November
2009
–	64 pharma-
cies
–	24 software
vendors
Sensitivity, specificity,
positive predictive value,
negative predictive value,
and percentage of correctly
detected DDIs present
in a simulated patient’s
medical orders
To determine the effectiveness
of a pharmacy
CDS software
in detecting DDIs
–	Many pharmacy CDS systems inadequately
identified DDIs
–	28 percent of pharmacies correctly identified
eligible DDIs and non-DDIs
–	Median percentage of correct DDI responses:
89 percent
–	Median sensitivity to detect well-established
interactions: 0.85 (range 0.23-1.0)
–	Median specificity: 1.0 (range 0.83-1.0)
Quasi-experi-
mental
Smith, D. H., N. Perrin, A. Feldstein,
X. Yang, D. Kuang, S.
R. Simon, D. F Sittig, R. Platt,
and S. B. Soumerai. 2006. The
impact of prescribing safety
alerts for elderly persons in an
electronic medical record: An
interrupted time series evaluation.
Archives of Internal Medicine
166(10):1098–1104.
CDS
ePrescribing
October 1999
to December
2002
209 family
practitioners
and internal
medicine
clinicians
Number of dispensings of
nonpreferred and preferred
drugs per 10,000 population
To determine if the
use of potentially contraindicated
agents in
elderly persons is reduced
by implementing
a CPOE system
with CDS
–	Implementation of CPOE system with CDS is
significantly correlated with a sudden reduction in
the rate of initial dispensing of nonpreferred agents
among elderly persons
–	Rate of dispensing nonpreferred agents (per 10,000):
•	Preimplementation, 21.9
•	Postimplementation, 16.8 (p < 0.01)
Interrupted
time series
analysis
Niès, J., I. Colombet, E. Zapletal,
F. Gillaizeau, P. Chevalier,
and P. Durieux. 2010. Effects
of automated alerts on unnecessarily
repeated serology
tests in a cardiovascular
surgery department: A time
series analysis. Health Services
Research 10:70.
CDS
laboratory
January 2004
to December
2007
–	Pre-CDS:
3,480 viral
serology
tests
–	Post-CDS:
2,095 HBs
antigen tests
performed
Proportion of unnecessarily
repeated HBs antigen tests
To determine whether
CDS reminders of
previous existing
serology results
would result in
less unnecessarily
repeated HBs
antigen tests
–	Implementation of CDS system stopped the rising
rate of unnecessarily repeated HBs antigen tests
–	Mean proportion of unnecessarily repeated HBs
antigen tests:
•	Pre-CDS, increase of 0.4 percent per month
(95 percent CI 0.2 to 0.6, p < 0.001)
•	Post-CDS, decrease of 0.4 percent per month
(95 percent CI –0.7 to –0.1 percent, p = 0.02)
Time series
analysis
Koppel, R., J. P. Metlay, A.
Cohen, B. Abaluck, A. R.
Localio, S. E. Kimmel, and
B. L. Strom. 2005. Role of
computerized physician order
entry systems in facilitating
medication errors. Journal of
the American Medical Association
293(10):1197–1203.
CPOE 2002 to 2004 –	261 hospital
house staff
–	5 focus
groups
–	32 one-onone
interviews
and
observations
Examples of medication
errors caused or worsened
by the CPOE system
To identify and quantify
the role of CPOE
in facilitating prescription
error risks
CPOE system caused or exacerbated 22 types of
medication error risks, including
–	Fragmented CPOE displays that prevent a coherent
view of patients’ medications
–	Pharmacy inventory displays mistaken for
dosage guidelines
–	Ignored antibiotic renewal notices placed on paper
charts rather than in the CPOE system
–	Separation of functions that facilitate double dosing
and incompatible orders
–	Inflexible ordering formats generating wrong orders
Quantitative
and qualitative
study
DesRoches C. M., E. G.
Campbell, C. Vogeli, J. Zheng,
S. R. Rao, A. E. Shields, K.
Donelan, S. Rosenbaum, S. J.
Bristol, and A. K. Jha. 2010.
Electronic health records’
limited successes suggest
more targeted uses. Health
Affairs 29(4):639–646.
EHR March to
September
2008
2,952 institu-
tions
Hospital Quality Alliance
summary scores
To determine the
impact of EHRs
on quality of
medical care
–	No relationship between the adoption of EHRs
and quality of care was found
–	Hospital Quality Alliance summary scores (percent):
•	Acute myocardial infarction:
–	Comprehensive EHR adoption, 97.5
–	Basic EHR adoption, 96.4
–	No EHR adoption, 96.3 (p = 0.24)
•	Congestive heart failure:
–	Comprehensive EHR adoption, 91.2
–	Basic EHR adoption, 90.5
–	No EHR adoption, 89.1 (p = 0.08)
•	Pneumonia:
–	Comprehensive EHR adoption, 93.2
–	Basic EHR adoption, 92.9
–	No EHR adoption, 92.4 (p = 0.33)
•	Surgical Care Improvement Project measures:
–	Comprehensive EHR adoption, 93.7
–	Basic EHR adoption, 93.3
–	No EHR adoption, 92.0 (p = 0.01)
Observational
analysis
(survey)
PREPUBLICATION COPY: UNCORRECTED PROOFS
TABLE B-2 Continued
Study
Study
Purpose Method Relevant Findings
Health IT
Component Time Frame Sample Size Outcome Measures
Ferris, T. G., S. A. Johnson, J.
P. T. Co, M. Backus, J. Perrin,
D. W. Bates, and E. G. Poon.
2009. Electronic results
management in pediatric
ambulatory care: Qualitative
assessment. Pediatrics
123(Suppl 2):S85–S91.
EHR N/A –	86 respon-
dents
–	18 surveyed
practices
–	8 of the surveyed
practices
were
interviewed
Physicians responses
to questions regarding
–	Patient safety
–	Effectiveness of care
–	Availability of results
–	Confidence that results
would not be lost
To determine the
impact of electronic
results management
(ERM) systems on
pediatric care
–	Fully adopted ERMs reported an increase in efficiency,
safety, and physicians satisfaction
•	72 percent of practitioners reported an increase
in patient safety
•	63 percent reported an increase in more
effective care
–	Partially adopted ERMs resulted in a perceived
decrease in safety and efficiency
Observational
Analysis (sem-
istructured,
key informant
interviews and
surveys)
Gearing, P., C. M. Olney, K.
Davis, D. Lozano, L. B. Smith,
and B. Friedman. 2006.
Enhancing patient safety
through electronic medical
record documentation of vital
signs. Journal of Health care
Information Management
20(4):40–45.
EHR 2006 1,236 vital sign
sets
Error rateTo compare the error
rate of electronic vital
signs (EVS) documentation
to paper
documentation
–	Medical error rates can be reduced almost by half
through the use of EVS documentation
–	Medical error rate:
•	EVS documentation: 4.4 percent
•	Paper chart: 10 percent
Prospective
cohort study
Gordon, J.R.S., T. Wahls,
R.C. Carlos, I.I. Pipinos, G.E.
Rosenthal, and P. Cram. 2009.
Failure to recognize newly
identified aortic dilations in
a health care system with an
advanced electronic medical
record. Annals of Internal
Medicine 151(1):21–27.
EHR 2003 –	2 hospitals
–	440 patients
with abdominal
aortic
abnormali-
ties
Percentage which
–	Dilations were not
recorded in EMR
–	Abnormalities were
documented in EMR
To determine the
frequency which clinicians
enter CT-documented
dilations of
the abdominal aorta
into the EMR
–	A substantial proportion of new aortic dilations were
not recorded in the EMR
–	58 percent of dilations were not recorded in the EMR
by clinical teams within 3 months of the CT
–	No EMR documentation of abnormalities existed for
29 percent of surviving patients during a mean followup
of 3.2 years
Retrospective
cohort study
Lo, H. G., L. P. Newmark, C.
Yoon, L. A. Volk, V. L. Carlson,
A. F. Kittler, M. Lippincott, T.
Wang, and D. W. Bates. 2007.
Electronic health records in
specialty care: A time-motion
study. Journal of the American
Medical Informatics Association
14(5):609–615.
EHR –	Pre-EHR:
May 2002
to August
2003
–	Post-EHR:
December
2002 to May
2004
–	5 outpatient,
urban specialty
care
clinics
–	Pre-EHR: 15
physicians
treating 157
patients
–	Post-EHR: 15
physicians
treating 146
patients
Average adjusted total time
spent per patient on
–	Direct patient care
–	Indirect patient care (writing,
reading, and involved
actions such as writing
emails, reading patient
charts, or finding digitized
radiographs)
–	Administration
–	Miscellaneous
To determine if EHRs
decrease the amount
of time spent per
patient by specialized
clinicians in cardiology,
dermatology,
endocrine, and pain
clinics
–	EHR use slightly, but not significantly, increased
average adjusted total time spent per patient across
all specialties:
•	Pre-EHR, 28.8 min
•	Post-EHR, 29.8 min (p = 0.83)
–	Change in time for
•	Direct patient care, 0.26 min (p = 0.85)
•	Indirect patient care (write), 2.1 min (p = 0.21)
•	Indirect patient care (read), 1.8 min (p = 0.07)
•	Indirect patient care (other), –0.53 min (p = 0.49)
•	Administration, –0.40 min (p = 0.55)
•	Miscellaneous, –3.1 min (p = 0.03)
Prospective
study
O’Donnell, H. C., R. Kaushal,
Y. Barrón, M. A. Callahan, R.
D. Adelman, and E. L. Siegler.
2008. Physicians’ attitudes
towards copy and pasting
in electronic note writing.
Journal of General Internal
Medicine 24(1):63–68.
EHR June to
August
2007
–	2 medical
facilities
–	253 physi-
cians
–	Reported use of CPF
–	Reported opinions toward
CPF
To determine physician
use and attitudes
towards copy and
paste functions
(CPFs) in computerized
note writing
–	90 percent of respondents used CPF
–	70 percent used CPF almost always or most of
the time
–	71 percent believed electronic records developed
with CPF are more likely to contain mistakes and
outdated information
Cross-sectional
survey
Parente, S.T., and J.S. McCollough.
2009. Health information
technology and patient
safety evidence from panel
data. Health Affairs 28(2):
357–360.
EHR 1999 to 2002 N/A PSIs:
–	Infection due to
medical care
–	Postoperative hemorrhage
or hematoma
–	Postoperative pulmonary
embolism or deep vein
thrombosis (DVT)
To determine the impact
of PACS, EMRs,
and nurse charts on
patient safety
–	PACS and nurse charts showed no statistical significance
on patient safety
–	EMR use was the only health IT application that
showed a significant effect on patient safety
•	EMRs were significantly correlated with the
reduction of infection rates
•	Two infections are avoided per year at an
average hospital
•	EMR use became more effective over time
Retrospective
analysis
Smith, L.B., L. Banner, D.
Lozano, C. Olney, and B.
Friedman. 2009. Connected
care: Reducing errors through
automated vital signs data
upload. Computers Informatics
Nursing 27(5):318–323.
EHR October to
November
2006
9,084 vital
sign data ele-
ments
–	Vital sign
documentation errors
–	Omission errors
–	Transcription errors
–	Transmission errors
To evaluate the accuracy
of vital sign
data when collected
from an automated
vital sign monitor,
transmitted through
an infrared port to
a personal digital
assistant (PDA), and
then automatically
uploaded into an EMR
–	A significant reduction in documentation errors was
associated with the use of direct electronic upload into
EMR (p < 0.001)
–	Vital signs captured on paper and then typed into the
EMR were incorrect 4.4 percent of the time
–	Vital signs electronically uploaded directly into EMR
were incorrect:
•	Total errors, 0.66 percent
•	Omission errors, 0.58 percent
•	Transcription errors, 0.08 percent
•	Transmission errors, 0 percent
Prospective
observational
study
Zhou, L., C. S. Soran, C. A.
Jenter, L. A. Volk, E. J. Orav,
D. W. Bates, and S. R. Simon.
2009. The relationship between
electronic health record
use and quality of care over
time. Journal of the American
Medical Informatics Association
16(4):457–464.
EHR 2000 to 2005 1,181 physi-
cians
–	Length of EHR use
–	Change in 6 HEDIS quality
measures after EHR
implementation:
•	Cancer screening
•	Diabetes care
•	Asthma care
•	Well child and
adolescent visit
•	Behavioral and
mental health
•	Women’s health
To examine the extent
of EHR usage and
how that use over
time affects the quality
of different ambulatory
care practices
–	No association between length of time using EHR
and quality of care was found for any of the six
quality measures
–	By 2005:
•	Adoption of EHRs doubled since 2000
•	Average length of EHR use was 4.8 years
Cross-sectional
study
Hanuscak, T. L., S.L. Szeinbach,
E. Seoane-Vazquez, B. J.
Reichert, and C. F. McCluskey.
2009. Evaluation of causes
and frequency of medication
errors during information
technology downtime. American
Journal of Health-System
Pharmacy 66(12):1119–1124.
ePrescribing February and
May 2007
32 respon-
dents
MEs occurring while health
IT systems were down
To determine the rate
of MEs when health IT
systems are down
–	Standard protocols and backup systems did not
prevent MEs during a downtime of a health IT system
–	16 percent of errors occurring during downtime had
the potential to cause patient harm
Survey
Santell, J. P., J. G. Kowiatek,
R. J. Weber, R. W. Hicks,
and C. A. Sirio. 2009. Medication
errors resulting from
computer entry by nonprescribers.
American Journal
of Health-System Pharmacy
66(9):843–853.
ePrescribing July 2001
to December
2005
–	693 unique
facilities
–	90,001 medication
error
records that
were the
result of computer
entry
by nonpre-
scribers
Rates and causes of MEsThe characteristics
of medication errors
associated with the
use of computer
order-entry systems
by nonprescribers
–	Computer systems can create new opportunities
for error
–	Percentage of harm associated with computer-entry
errors: 0.99 percent (only national level data reported)
–	Causes for error included
•	Inaccurate or omitted transcription, 30 percent
•	Documentation, 19.5 percent
•	Procedure or protocol not followed, 21.7 percent
–	Most computer-entry errors occurred in the inpatient
pharmacy department: 49.3 percent
Retrospective
analysis of
records
submitted to
MEDMARX
PREPUBLICATION COPY: UNCORRECTED PROOFS
TABLE B-2 Continued
Study
Study
Purpose Method Relevant Findings
Health IT
Component Time Frame Sample Size Outcome Measures
Singh, H., S. Mani, D. Espadas,
N. Petersen, V. Franklin, and
L. A. Petersen. 2009. Prescription
errors and outcomes
related to inconsistent information
transmitted through
computerized order entry:
A prospective study. Archives
of Internal Medicine
169(10):982–989.
ePrescribing 4-month
period
–	One tertiary
care facility
–	55,992 new
prescrip-
tions
–	Percentage of orders
containing inconsistent
communication
–	Percentage of orders with
inconsistent information
that could cause moderate
to severe harm
To identify the nature
and frequency of
errors related to
inconsistent information
entered into a
CPOE system
–	Inconsistent communication in CPOE systems creates
significant risks to safety
–	0.95 percent of orders contained inconsistent
information
–	Approximately 20 percent of errors could have
resulted in moderate to severe harm
Prospective
study
Walsh, K. E., W. G. Adams,
H. Bauchner, R. J. Vinci, J. B.
Chessare, M. R. Cooper, P. M.
Hebert, E. G. Schainker, and C.
P. Landrigan. 2006. Medication
errors related to computerized
order entry for children.
Pediatrics 118(5):1872–1879.
ePrescribing –	April to June
2002
–	3 to 12
months after
implementation
of
computerized
order
entry
–	One urban
teaching
hospital
–	6,916
medication
orders
–	1930
patient-days
Severity and rate of MEs per
1,000 patient days
To determine the
frequency of MEs
associated with a
CPOE system
–	Although CPOE systems can introduce new pediatric
MEs, serious computer-related pediatric computer
errors are uncommon
–	Study yielded 37 serious MEs per 1,000 patient-days;
rate of serious computer-related pediatric errors was
3.6 errors per 1,000 patient-days
Retrospective
analysis
Zhan, C., R. W. Hicks, C. M.
Blanchette, M. A. Keyes, and
D. D. Cousins. 2006. Potential
benefits and problems with
computerized prescriber order
entry: Analysis of a voluntary
medication error-reporting
database. American Journal
of Health-System Pharmacy
63(4):353–358.
ePrescribing 2003 –	570 facilities
–	235,164
medication
error reports
Rate and type of errorsTo compare the rate
of prescribing errors
in facilities with
and without CPOE
systems
–	Due to underreporting, MEDMARX data cannot
be used to determine if CPOE prevents errors
–	However, types of errors related to CPOE systems
can be determined:
•	Dosing errors, 51.4
•	Unauthorized drug, 3.6
•	Wrong patient, 3.5
•	Wrong time, 2.7
Retrospective
analysis from
MEDMARX
reports
Ali, N. A., H. S. Mekhjian, P.
L. Kuehn, T. D. Bentley, R.
Kumar, A. K. Ferketich, and S.
P. Hoffmann. 2005. Specificity
of computerized physician
order entry has a significant
effect on the efficiency of
workflow for critically ill patients.
Critical Care Medicine
33(1):110–114.
ePrescribing May 2000 to
May 2002
91 Patients –	Orders for complex
ICU care
–	Use of higher-efficiency
CPOE order paths
To compare the
effects patient care
of a standard CPOE
on ICU to that of
a modified CPOE
designed for ICU
patient care
–	CPOE specifically designed for ICU care significantly
increased efficiency
–	 Under the modified CPOE system, significant
reductions in orders per patient were found for
•	Vasoactive infusions from 4.8 to 2.2 (p < 0.01),
•	Sedative infusions, and from 6.4 to 2.9 (p < 0.01),
•	Ventilator management from 13.1 to 6.9 orders/
patient (p < 01).
–	Significant increase in orders executed through ICUspecific
order sets occurred after system modifications
Retrospective
before and
after cohort
study
Callen, J., R. Paoloni, A.
Georgiou, M. Prgomet, and
J. Westbrook. 2010. The rate
of missed test results in an
emergency department: An
evaluation using an electronic
test order and results viewing
system. Methods of Information
in Medicine 49(1):37–43.
ePrescribing January 2002
to November
2004
–	4 clinical
units
in 2 large
Australian
teaching
hospitals
–	Hospital A:
Used CPOE
for 10-plus
years
–	Hospital B:
Orders
ordered
manually
–	Comments regarding use
of health IT made by clinicians
during interviews
–	Observations regarding use
of health IT made by study
staff on site
Determine the impact
of long term use of a
CPOE system
–	Different clinical environments and diversity among
clinicians affect the way clinicians ordered lab tests
and therefore the safety of the CPOE system
–	Diversity of physicians’ test management practices
need to be understood, analyzed, and accommodated
before and during the CPOE implementation
Cross-sectional
qualitative
study
Condren, M., I. J. Studebaker,
and B. M. John. 2010. Prescribing
errors in a pediatric clinic.
Clinical Pediatrics 49(1):49–53.
ePrescribing February to
April
2007
–	3,523
records
–	1,802 new
prescrip-
tions
Rates of
–	Errors entered into the
EMR
–	Incomplete prescription
–	Dosing outside
recommended range
–	Drug selection error
–	Documentation error
–	Administration error
To identify the rate
and type of prescribing
errors occurring
in a pediatric clinic
with an EMR system
–	9.7 percent of all prescriptions were found to contain
prescribing errors
–	Types and rates of error:
•	Incomplete prescription, 42 percent
•	Dosing outside recommended range, 34 percent
•	Drug selection error, 14.5 percent
•	Documentation error, 6.5 percent
•	Administration error, 1.25 percent
Prospective
cohort study
Devine, E. B., R. N. Hansen, J.
L. Wilson-Norton, N. M. Lawless,
A. W. Fisk, D. K. Blough,
D. P. Martin, and S. D. Sullivan.
2010. The impact of computerized
provider order entry
on medication errors in a
multispecialty group practice.
Journal of the American Medical
Informatics Association
17(1):78–84.
ePrescribing July 2004 to
November
2007
10,169 pre-
scriptions
–	5,016
handwritten
–	5,153
electronic
Percent change in ME rate
and ADE rate
To determine an
ambulatory CPOE
system’s effect on
medication errors
and ADEs
–	A significant reduction in medication errors is
associated with the use of a CPOE system
–	Adjusted odds of an error occurring postimplementation
of CPOE: 70 percent lower than preimplementation
(OR: 0.30; 95 percent CI 0.23 to 0.40; p < 0.001)
–	Frequency of errors declined from 18.2 percent to
8.2 percent
–	Largest reduction of errors as a result of CPOE
implementation for the following:
•	Illegibility (97 percent)
•	Inappropriate abbreviations (94 percent)
•	Missing information (85 percent)
Quasi-experimental
study
Magrabi, F., S. Y. W. Li, R. O.
Day, and E. Coiera. 2010. Errors
and electronic prescribing:
A controlled laboratory
study to examine task complexity
and interruption effects.
Journal of the American
Medical Informatics Association
17(5):575–583.
ePrescribing N/A 32 doctors –	Types of prescribing errors
–	Error rate
To determine the
impact of interruptions
and task
complexity caused
by CPOE systems
on the types and rates
of prescribing errors
–	Most errors were “slips” in updating and creating EHRs,
such as selecting incorrect:
•	Medications
•	Doses
•	Routes
•	Formulations
•	Frequencies of administration
–	Among the several types of prescribing errors
that were observed, the rates for each type of error
ranged from
•	0.5 percent (incorrect medication selected) to
•	16 percent (failure to enter patent allergy
information).
Observational
analysis
Nam, H. S., S. W. Han, S. H.
Ahn, J. Y. Lee, H. Y. Choi, I.
C. Park, and J. H. Heo. 2007.
Improved time intervals by
implementation of computerized
physician order entrybased
stroke team approach.
Cerebrovascular Diseases
23(4):289–293.
ePrescribing –	Pre-CPOE:
June 2003
to May 2004
–	Post-CPOE:
June 2004
to May 2005
–	Pre-CPOE:
14 patients
–	Post-CPOE:
25 patients
Time intervals from patient
arrival to:
–	Registration
–	CT scan
–	Thrombolysis
To determine if the
implementation of
CPOE reduces the
time interval from a
patient’s arrival at the
emergency department
to thrombolysis
–	Implementation of the CPOE significantly shortens
the median time interval from arrival to evaluation
and treatment
–	Median time intervals (minutes) from arrival to
•	Registration:
–	Pre-CPOE, 5
–	Post-CPOE, 5
(p = 0.52)
•	CT scan
–	Pre-CPOE, 34
–	Post-CPOE, 19
(p = 0.01)
•	Thrombolysis:
–	Pre-CPOE, 79
–	Post-CPOE, 56
(p < 0.01)
Quasi-experimental
study
Oyen, L. J., R. A. Nishimura,
N. N. Ou, J. J. Armon, and M.
Zhou. 2005. Effectiveness of
a computerized system for
intravenous heparin administration:
Using information
technology to improve patient
care and patient safety. The
American Heart Hospital Journal
3(2):75–81.
ePrescribing 2001 to April
2003
–	419 patients
using
HepCare
–	98 using
standard
care
–	Percentage of patients
achieving goal activated
partial thromboplastin
time values (aPTT)
–	Time to achieve the
goal aPTT
To evaluate the
impact of HepCare
(a computerized
heparin nomogram
system) on heparin
safety
–	Significant improvements in safety, quality assurance,
and targeted aPTT values were associated with HepCare
use
–	Mean percentage of aPTT values at goal:
•	HepCare, 44 percent
•	Standard care, 27 percent (p < 0.01)
–	Percent of patients reaching at least one goal aPTT
within 24 hours:
•	HepCare, 54 percent
•	Standard care, 13 (p < 0.01)
Cohort study
PREPUBLICATION COPY: UNCORRECTED PROOFS
TABLE B-2 Continued
Study
Study
Purpose Method Relevant Findings
Health IT
Component Time Frame Sample Size Outcome Measures
Pirnejad, H., Z. Niazkhani, H.
van der Sijs, M. Berg, and R.
Bal. 2008. Impact of a computerized
physician order
entry system on nurse-physician
collaboration in the
medication process. International
Journal of Medical
Informatics 77(11):735–744.
ePrescribing Nov 2003 to
Apr 2004
–	Six internal
medicine
wards
–	Pre-CPOE:
76 nurses
–	Post-CPOE:
73 nurses
Concerns respondents
expressed with the medication
ordering system
To determine if CPOE
impedes nurse–physician
collaboration and
thereby undermines
the efficiency and
safety of the medication
process
–	Although the CPOE system addressed concerns of
paper-based systems, new concerns were associated
with the CPOE system
–	Concerns with paper-based system:
•	Illegibility of handwritten medication data,
64.3 percent
•	Poor overview of current medication, 46.4 percent
•	Slowness of system, 46.4 percent
–	Concerns with CPOE system:
•	No possibility to check what medication had already
been administered to a patient, 52.2 percent
•	Less possibility for nurses to correct physicians’
prescription errors, 43.5 percent
Prospective
cohort study
(survey)
Shulman, R., M. Singer, J.
Goldstone, and G. Bellingan.
2005. Medication errors: A
prospective cohort study of
hand-written and computerised
physician order entry in
the intensive care unit. Critical
Care 9(5):R516–R521.
ePrescribing 65 weeks –	2,429 CPOE
prescrip-
tions
–	1,036 HWP
prescrip-
tions
Rate of MEsTo compare the rate
of MEs with handwritten
prescribing
(HWP) system to a
CPOE system without
CDS in an intensive
care unit
–	Rate of MEs is significantly lowered by introduction
of CPOE:
•	HWP, 6.7 percent
•	CPOE, 4.8 percent (p < 0.04)
–	Strong linear trend of a declining proportion of
MEs over time (p < 0.001)
–	Moderate and major errors (such as harms without
any associated causes or can lead to death) are still
of significant concern after implementation of CPOE
(nonintercepted and intercepted harm: 0.9 percent)
Prospective
cohort study
Sinopoli, D. J., D. M. Needham,
D. A. Thompson, C. G.
Holzmueller, T. Dorman, L.
H. Lubomski, A.W. Wu, L. L.
Morlock, M. A. Makary, and P.
J. Pronovost. 2007. Intensive
care unit safety incidents
for medical versus surgical
patients: A prospective
multicenter study. Journal of
Critical Care 22(3):177–183.
ePrescribing July 2002 to
June 2004
–	646 incidents
involving
adult
medical
patients
–	707 incidents
involving
adult
surgical
patients
Reported safety incidentsTo determine the
differences in harm
related to CPOE in
medical and surgical
patients
Incidents related to CPOE occurred more often in
medical patients than surgical patients:
–	Surgical, 6 percent
–	Medical patients, 13 percent (p ≤ 0.001)
–	Increased frequency of incidents may be due to
more medication orders being made in medical
patients group
Multicenter
prospective
study
Thompson, D.A., L. Duling,
C.G. Holzmueller, T. Dorman,
L.H. Lubomski, F. Dickman,
M. Fahey, L.L. Morlock, A.W.
Wu, and P.J. Pronovost. 2005.
Computerized physician order
entry, a factor in medication
errors: Descriptive analysis of
events in the Intensive Care
Unit Safety Reporting System.
Journal of Clinical Outcomes
Management 12(8):407–412.
ePrescribing Unknown 18 intensive
care units
Rate of incidence reported
to an anonymous web-based
incident reporting system
To determine how the
rate of medication
errors are affected by
the implementation of
a CPOE system
–	55 incidents were related to CPOE
–	CPOE incidents that resulted in a medication error:
85 percent
–	Types of errors:
•	User errors, 67 percent of the error and
near misses
•	Software errors, 20 percent
•	Computer malfunction, 13 percent
Observational
analysis
Weant, K. A., A. M. Cook,
and J. A. Armitstead. 2007.
Medication-error reporting
and pharmacy resident experience
during implementation
of computerized prescriber
order entry. American Journal
of Health System Pharmacy
64(5):526–530.
ePrescribing –	Pre-CPOE:
Sept to
Oct 2003
–	Post CPOE:
Sept to
Oct 2004
1 neurosurgical
ICU
Number of ordering errorsTo compare the
number and type of
medication errors
reported before and
after the implementation
of CPOE
–	Number of ordering errors increased fivefold
after CPOE implementation (0.938 vs. 1.839 per
1,000 doses)
–	Number of errors resulting in patient harm decreased
following CPOE implementation (0.137 vs. 0.0152 per
1,000 doses)
Prospective
cohort study
Agrawal, A., and W. Y. Wu.
2009. Reducing medication
errors and improving systems
reliability using an electronic
medication reconciliation
system. Joint Commission
Journal on Quality & Patient
Safety 35(2):106–114.
Other health
IT-assisted
care
August 2006
to December
2007
–	120 unique
MedRecon
events
over initial
2-week pilot
study
–	19,356
unique
MedRecon
events during
17-month
study period
–	Unintended discrepancy
rate between a patient’s
home medications and admission
medication orders
–	Compliance with the MedRecon
process
To evaluate the effectiveness
of an
electronic Medication
Reconciliation System
(MedRecon) system
with computerized
alerts
–	Medication errors on admission were substantially
reduced by an electronic MedRecon System
–	Unintended discrepancy rate was reduced from
20 to 1.4 percent
–	Physician adherence with the MedRecon process:
•	Preimplementation, 34 percent
•	Postimplementation, 98-100 percent
Observational
study
Agrawal, A., W. Wu, and I. Khachewatsky.
2007. Evaluation
of an electronic medication
reconciliation system in inpatient
setting in an acute care
hospital. In Building Sustainable
Health Systems, edited
by K. A. Kuhn, J. R. Warren,
and T.-Y. Leong. Conference
Proceedings: 12th World
Congress on Health (Medical)
Informatics. Amsterdam: IOS
Press. Pp. 1027–1031.
Other health
IT–assisted
care
Unknown 3,426 consecutive
inpatient
admission
MedRecon
events
Type and rate of discrepancies
between medications
taken at home and what
is recorded in the admission
orders
To evaluate the
performance of an
electronic MedRecon
system
–	Compared to the literature, discrepancy rate between
patient’s home medication history and admission
orders was low (3.12 percent)
–	Most common type of discrepancy was omission
of a home medication (56.52 percent)
Observational
analysis
Schnipper, J. L., C. Hamann,
C. D. Ndumele, C. L. Liang,
M. G. Carty, A. S. Karson, I.
Bhan, C. M. Coley, E. Poon, A.
Turchin, S. A. Labonville, E.
K. Diedrichsen, S. Lipsitz, C.
A. Broverman, P. McCarthy,
and T. K. Gandhi. 2009. Effect
of an electronic medication
reconciliation application and
process redesign on potential
adverse drug events:
a cluster-randomized trial.
Archives of Internal Medicine
169(8):771–780.
Other health
IT–assisted
care
May to June
2006
–	2 hospitals
–	322 patients
–	160 control
patients
•	Residents
documented
medication
histories in
admission
notes
•	Pharmacists
reviewed
medication
orders
•	Physicians
wrote
discharge
orders without
access
to preadmission
medication
histories
–	162 inter-
ventions:
electronic
medication
reconciliation
program integrated
into
the CPOE
–	Preadmission medication
list compared to medical
histories taken from medical
team
–	Number of unintentional
medication discrepancies
with potential for causing
harm per patient: potential
adverse drug event (PADE)
To determine the
effect of a web-based
medication reconciliation
intervention on
medication discrepancies
with potential for
adverse drug events
–	Computerized medication reconciliation tool
decreased unintentional medication discrepancies
with potential for patient harm.
–	Control Group: 230 unintentional medications
discrepancies with potential for patient harm
(1.44 PADEs per patient)
–	Intervention group: 170 unintentional medication
discrepancies with potential for patient harm
(1.05 PADEs per patient)
–	Adjusted relative risk (ARR): 0.72 (95 percent CI
0.52 to 0.99)
–	Because the effect differed among the hospitals,
integration issues are likely important for successful
implementation
•	Hospital 1 (ARR, 0.60; 95 percent CI, 0.38 to 0.97)
•	Hospital 2 (ARR, 0.87; 95 percent CI 0.57 to 1.32)
(p = 0.32 for test of effect modification)
Cluster, randomized
trial
PREPUBLICATION COPY: UNCORRECTED PROOFS
TABLE B-2 Continued
Study
Study
Purpose Method Relevant Findings
Health IT
Component Time Frame Sample Size Outcome Measures
Porter, S. C., R. Kaushal, P. W.
Forbes, D. Goldmann, and
L. A. Kalish. 2008. Impact of
a patient-centered technology
on medication errors
during pediatric emergency
care. Ambulatory Pediatrics
8(5):329–335.
Patient
engagement
June 2005 to
June 2006
–	1,410 parent–
child pairs
–	835 pairs
under usual
care
–	575 pairs
used
ParentLink
Number of medication errorsTo identify the effect
of ParentLink (application
that obtains child’s
medication/allergy
history and provides
tailored advice to
both parents and
clinicians) on the rate
of medication errors
–	Use of ParentLink had no significant impact
on medication errors
–	Number of errors per 100 patients:
•	Control, 173
•	Intervention, 134 (p = 0.35)
Quasi-
experimental
intervention
study
McAlearney, A. S., J. Vrontos,
Jr, P. J. Schneider, C. R. Curran,
B. S. Czerwinski, and C.
A. Pedersen. 2007. Strategic
work-arounds to accommodate
new technology: The
case of smart pumps in hospital
care. Journal of Patient
Safety 3(2):75–81.
Smart-
pumps
March to April
2005
–	24 nurses
–	4 focus
groups
–	Nurses’ perceptions of
smartpumps
–	Examples of how nurses
work around smartpump
problems
To assess nurses’
attitudes towards
computerized intravenous
infusion pumps
with decision support
(smartpumps)
–	Nurses largely perceive smartpumps to be beneficial
to patient safety
–	Nurses use a number of workarounds to overcome
issues with smartpumps, which may lead to new
sources of error
–	Examples of workarounds:
•	Bypassing both the decision support and dose
mode safety features to give doses that are not
contained in the smartpump’s dosage library
•	Placing pillows over the pump to quiet alerts that
could not be turned off
Focus groups
Claridge, J. A., J. F. Golob,
Jr., A. M. A. Fadlalla, B. M.
D’Amico, J. R. Peerless, C. J.
Yowler, and M. A. Malangoni.
2009. Who is monitoring
your infections: Shouldn’t
you be? Surgical Infections
10(1):59–64.
Surveillance 12 months 769 patients Number, sensitivity, and
specificity of patients
diagnosed with VAP by a
panel of doctors compared
to number of patients diagnosed
by SIC-IR and IC
To compare the Surgical
Intensive CareInfection
Registry
(SIC-IR) (a health IT
system integrated into
the hospital’s laboratory
information
system and medication
administration
record for automatic
data loading) with
traditionally trained
infection control
teams’ (IC) ability to
identify ventilatorassociated
pneumonia
(VAP) in critically
ill patients
–	SIC-IR was more accurate in diagnosed VAPS than
the IC team
–	Number of patients diagnosed VAPs by
•	Physician panel: 40
•	SIC-IR, 39
•	IC, 22
–	Sensitivity for identifying VAP:
•	SIC-IR, 97 percent
•	IC, 56 percent
–	Specificity for identifying VAP:
•	SIC-IR, 100 percent
•	IC, 99 percent
Prospective
analysis
Jha, A. K., J. Laguette, A.
Seger, and D. W. Bates. 2008.
Can surveillance system
identify and avert adverse
drug events? A prospective
evaluation of a commercial
application. Journal of the
American Medical Informatics
Association 15(5):647–653.
Surveillance N/A –	2,407
patients
screened
–	266 alerts
–	Frequency and types of
alerts produced
–	Frequency which alerts
were associated with ADEs
and potential ADEs
–	Potential financial impact
of monitoring for ADEs
To determine whether
Dynamic Pharmacovigilance
(a health
IT system that monitors
laboratory and
pharmacy data and
uses preset rules to
determine whether an
ADE may occur) can
successfully identify
and prevent ADEs in
a community hospital
–	Dynamic Pharmacovigilance can be an effective tool
for identifying and preventing ADEs
–	11.3 percent of the studied alerts were considered
substantially important to warrant contacting
the physician
–	23 percent of high priority alerts were associated
with an ADE (95 percent CI 12 to 34 percent)
–	15 percent were associated with a potential ADE
(95 percent CI 6 to 24 percent)
Prospective
study
van der Sijs, H., R. Bouamar, T.
van Gelder, J. Aarts, M. Berg,
and A. Vulto. 2010. Functionality
test for drug safety alerting
in computerized physician
order entry systems. International
Journal of Medical
Informatics 79(4):243–251.
ePrescribing 2006 to 2007 6 different
CPOE systems
The sensitivity and specificity
to detect drug safety
problems
To determine the
effectiveness of
CPOE systems’
alert functions
–	There is a large variations in different CPOE’s ability
to detect and alert clinicians to drug safety problems
–	Sensitivity: 0.38 to 0.79
–	Specificity: 0.11 to 0.84
Comparative
evaluations
PREPUBLICATION COPY: UNCORRECTED PROOFS