Jacobs Journal of Emergency Medicine

Implementing an Evidence-Based Algorithm for CT in the Emergency Department

*Sherri L LaVela
Department Of Veterans Affair, Institute For Public Health And Medicine, Northwestern University, Chicago, United States

*Corresponding Author:
Sherri L LaVela
Department Of Veterans Affair, Institute For Public Health And Medicine, Northwestern University, Chicago, United States

Published on: 2018-04-23


Objective: A substantial percentage of children who get CT scans after apparently minor head trauma, may not need them. Here we evaluate the impact of a quality improvement intervention on the appropriate use of head CT scanning in children presenting to the pediatric emergency department (ED).

Materials and Methods: Retrospective patient chart review to assess the impact of an intervention on adherence to an evidence-based algorithm for CT head scanning in a pediatric ED. Charts for all pediatric ED encounters for the pre-implementation period (August-September 2010) and the post-intervention period (November- December 2010). To identify variables associated with algorithm adherence we used a multivariate regression model.

Results: ED encounters were evaluated (n=346). Controlling for patient age, medical encounter reason, and health care provider type, odds of CT head scan algorithm adherence were 4.3x higher during the post-intervention time period (vs. pre-), p=0.0027. Odds of following the algorithm were lower for patients 2 years), p=0.0462. Medical encounter reason (trauma/non-trauma) and provider type were not associated with algorithm adherence.

Conclusion: A relatively simple, low-cost intervention resulted in an improvement in appropriate use of CT scans of the head, consistent with the evidence-based algorithm, in children presenting to the pediatric ED for traumatic or non-traumatic reasons.


Computerized Tomography, Radiation, Emergency Department, Quality Improvement, Pediatrics


The use of computerized tomography (CT), a valuable imaging tool, has been increasing rapidly. Throughout the world, a substantial percentage of children exposed to ionizing radiation receive it in the form of CT scans. Due to the potential for increased radiation exposure to children undergoing these scans, pediatric CT is a public health concern. The question of overuse has become a point of concern in the delivery of health care. By 2008, in the US, CT was performed in 5.9% of all pediatric visits to the emergency department (ED) [1]. Larson et al. [1] determined that from 1995 to 2008, the number of pediatric ED visits that included CT examination increased from 0.33 to 1.65 million, a fivefold increase and a compound annual growth rate of 13.2% [1]. Approximately 70 million CT scans are performed annually and between 5% and 10% of these are administered to children [1]. Use of CT for diagnostic evaluation is associated with radiation exposure and potential cancer risk [2]. The risk is believed to be higher in children because of their smaller bodies, lower shielding capabilities, and longer life expectancy [3-5]. Due to growing concerns about cancer risks that may be associated with CT imaging it is essential to reduce radiation exposure from CT scans that may be unnecessary. Rehani [6] defines overutilization of imaging as any application where imaging is not likely to improve patient outcome. This prompted several expert organizations such as the American College of Radiology and the Royal College of Radiology to provide appropriateness criteria [7,8]. Evaluation of healthcare professionals’ response and adherence to the criteria on appropriate use of CT scanning is warranted to understand adoption and implementation of evidencebased interventions [9].
Literature suggests that a substantial percentage of children who get CT scans after apparently minor head trauma may not need them. In an evaluation of records of more than 42,000 children from 25 medical centers, Kupperman et al. [10] found that one in five children aged 2 and older and nearly one quarter of children under 2 years of age who received CT scans following head trauma did not need them because they had a very low risk of developing serious brain injury compared with the cancer risks of the radiation dose. In addition, Mooney et al. [11] reported provider reluctance to adhere to recommended guidelines for imaging following a head injury in a cohort of children. Implementation science is the study of methods to promote the integration of evidence into healthcare practice [9]. Such interventions that promote the adoption and use of evidence-based guidelines/ algorithms are needed to guide appropriate use of CT scans. Hence, efforts are needed to clarify and encourage appropriate use of CT scanning in children and to streamline guideline-driven practice in the ED setting. The overall purpose of this study was to assess the impact of a quality improvement (QI) initiative intended to promote health care provider’s adherence to an evidence-based practice algorithm that guides appropriate use of CT head scanning in a sample of all children presenting to the ED.

Materials and Methods

Design: Outcomes were evaluated before and after a quality improvement intervention to improve evidence-based ED CT scanning utilization. This study was approved by the Northwest Community Hospital Institutional Review Board.

Intervention: The quality improvement intervention was implemented during October 2010 (between pre- and post-implementation data collection) and included use of provider education and clinical championing to promote evidence-based practice (EBP). Evidence-based practice (EBP) is an approach that specifies the way in which health professionals or other decision-makers should make decisions by identifying evidence that exists for a practice and rating it according to how scientifically sound it is. The goal of EBP is to eliminate unsound or risky practices in favor of those that have better outcomes [12]. The current study intervention entails QI efforts and training on the application of an evidence-based algorithm to guide the appropriate use of head CT scans in evaluating pediatric ED patients. All physicians and nurse practitioners working in the pediatric ED were in-serviced on the algorithmic decision tree prior to the post-intervention period. The in-service was offered multiple times to ensure reaching the entire full staff, although each individual provider was only expected to attend once. Several of the physicians who were board-certified in pediatric emergency medicine became early adopters of the algorithm, and helped champion its use. In addition, copies of the algorithm were posted through-out the pediatric ED. CT ordering was implemented by the ordering physician/ practitioner through a computerized ordering system. The hospital was in the process of implementing computerized provider order entry into its electronic health records; however, it was not available in the ED during the study period. Therefore, there were no ‘forced’ mechanisms within the electronic health records for adherence to the algorithm. Adherence was optional after the implementation.

The algorithm decision-tree based intervention was adopted from evidence-based research and included a predictive model that identified level of risk for clinically important traumatic brain injury in children [10]. The algorithm included predictive rules for children younger than 2 years of age and for children 2 years of age and older, for several potential reasons, e.g., younger patients’ greater sensitivity to radiation, minimal ability to communicate (pre-verbal/ verbal), and different mechanisms and risks for brain injury [10]. The predictive rule identified pre-verbal (under age 2) children as being at very low risk for traumatic brain injury when the child had normal mental status, no scalp hematoma, no loss (or less than 5 seconds) of consciousness, nonsevere injury mechanism, no palpable skull fracture, and normal behavior. The predictive rule for children 2 years of age and older was the same, but also included loss of consciousness, severe headache, and vomiting as factors. For children younger than 2 years of age, the sensitivity and negative predictive value were each 100%. For children 2 years of age and older, the sensitivity was 96.9% and the negative predictive value was 99.95% [10].

Subjects: Encounters for all children from birth to 17 years of age presenting to the pediatric ED at Northwest Community Hospital, Illinois during the study timeframe were evaluated. Encounters of infants/children presenting primarily to other departments such as neonatal intensive care units, outpatient subspecialty departments, or those presenting primarily to the radiation department as a scheduled outpatient procedure were not evaluated.

Setting: At Northwest Community Hospital, the volume of patients seen in the main ED (not inclusive of the pediatric ED) during 2010 and 2011 was 58,932 and 58,268, respectively. Pediatric ED patient volume for 2010 was 14,308 and for 2011 was 15,032. The pediatric ED is open for twelve hours only. During closure times, all patients are seen in the adult ED by board certified emergency medicine physicians. The pediatric ED is staffed by physicians who are all board certified in pediatric emergency medicine and pediatric nurse practitioners. There is a dedicated pediatric emergency nursing staff as well.

Data Source: Retrospective patient chart reviews were conducted to assess the impact of the intervention on provider adherence to an evidence-based algorithm for CT head scanning. Using a standardized data collection/chart abstraction tool, patient charts were reviewed by a trained chart abstractor for all pediatric ER encounters for the pre-implementation period (August-September 2010) and the postintervention period (November-December 2010).

Measures: Several variables were collected for each encounter.Patient age was dichotomized as ‘less than 2 years old’ and ‘2 years and older;’ this was done because the algorithmic decision tree varied based on these two age distinctions. Reason for medical encounter included: head trauma, headache, seizure, syncope, intoxication, and other (e.g., assault, ataxia, dizziness, hydrocephalus, miscellaneous medical, neck pain, psychological/behavioral, shaking, vertigo, and vomiting). The Glasgow Coma Score (GCS) could range from 3-15 and was used as a neurological measure of the patient’s state of consciousness, with a GCS of 3 representing a coma state and a GCS of 15 representing a fully awake and aware patient. Health care provider type was either physician or nurse practitioner and represented the provider handling the ER encounter. The dependent variable was algorithm adherence status, determined by whether the algorithmic decision tree was appropriately followed to either use or not use a head CT scan for each encounter.

Analyses: Descriptive statistics were used to describe the characteristics associated with encounters (details about the patient, encounter, and provider). Bivariate analyses were used to assess differences in encounters in which the algorithm was adhered to vs. not adhered to. A multivariate logistic regression was used to identify variables independently associated with the dependent variable, algorithm adherence. Collinearity was assessed for select variables for inclusion in the model to attain the model with the best fit. The model, adjusted for confounders, was used to generate odds ratios (ORs) and 95% confidence intervals (CIs). The final model included: intervention period (pre/post), age category of patients (under 2 years/2 years and older), medical encounter reason, and health care provider type as covariates. An alpha level of 0.05 was used to determine statistical significance. Statistical analyses were performed with SAS 9.2 (SAS Institute Inc., Cary, NC) and Stata Version 10 software (Stata Corporation, Texas, US).


A total of 346 encounters were evaluated during the study period (pre-intervention = 127, post-intervention = 219). Most encounters were for children 2 years of age and older (82.7%). The majority of ER encounters were for trauma reasons (84.4%), followed by headache/migraine (6.1%). The median GCS score was 15 (range 8-15). In most cases, physicians (61.8%) handled the encounters; 38.2% of cases were handled by nurse practitioners. The CT scan algorithm was adhered to for the large majority of cases (93.5%).

                                                                                                                         Table 1. Overall Demographics/Characteristics1.

Table 1 shows patient characteristics, ER encounter reasons, provider type, and algorithm adherence status.

Unadjusted bivariate comparisons of variables by algorithm adherence vs. non-adherence are shown in Table 2. There were 8 missing values for the algorithm variable, leaving a sample of 338 for the analyses. In patients under 2 years of age, the proportion of cases in which the algorithm was not adhered to (31.8%) was greater than the proportion of cases in which the algorithm was adhered to (16.1%), there was trend toward statistical significance for this association (p=0.06). There were no differences in medical encounter reason, GCS score, or provider type by algorithm adherence status. However, the bivariate analyses showed that in the post-intervention period, in a greater proportion of the encounters the algorithm was adhered to vs. non-algorithm adherence (65.5% vs. 31.8%, respectively), p=0.0015).

                                                                                                                       Table 2. Bivariate comparisons of Algorithm Used vs. Algorithm not used

                                                             Table 3. Multivariate logistic regression: Variables independently associated with using the algorithm (algorithm adherence) (n=338)

Controlling for age category of patients, medical encounter reason, and health care provider type, odds of CT head scan algorithm adherence were 4.3x higher during the post-intervention time period (vs. pre-intervention period), p=0.0027. Odds of following the algorithm were lower for patients 2 years), p=0.0462. Reason for medical encounter and provider type were not statistically associated with algorithm adherence status.


In planning our intervention, we were guided by well-documented change management principles for changing clinician behavior [13]. Overcoming resistance to the use of the algorithm involved an education program design that provided a reason to change and a focused educational intervention on the evidence for the algorithm. We employed the use of early adopters (clinical champions) who were respected clinicians among their peers to influence change, provision of an easy to use clinical tool/algorithm, and feedback to reinforce providers. We found that internal discussions and feedback with occurred amongst physicians/nurse practitioners within the department that focused on topics such as each of use and strength of evidence supporting the algorithm, served to address common barriers to implementation such as resistance and trust in evidence [14]. Adjusting for potential confounders, the findings showed a 4 times higher odds of CT head scan algorithm adherence after the intervention. These findings suggest that a relatively simple, low cost quality improvement intervention of healthcare provider education and clinical championing can increase the use of an evidence-based algorithm designed to reduce the use of head CT scans in a pediatric emergency department population. This result has important clinical implications in that by enhancing the use of this algorithm, it is possible that the rate of inappropriate CT scanning would decline, which would reduce exposure of children to ionizing radiation. It is also possible that more media attention in recent years has led to increased awareness of radiation risk in children, in general [15] and may have impacted CT use in this study. An examination across 40 countries, found head CT to be the most common CT examination in children, representing about ¾ of all pediatric CT examinations [16], suggesting that head CT scans are particularly utilized and that the greatest achievements from a QI intervention such as the one used in the current study, may be seen cases of high CT use (in which there is much room for improvement).

In the current study, health care provider type (e.g., physician or nurse practitioner) was not associated with algorithm adherence. It is possible; however, that mid-level providers and physicians might have different tendencies to follow guidelines. There is limited literature published about whether or not one provider type might be more or less likely to adhere to clinical guidelines, or whether quality metrics may differ significantly between provider types [17] for CT scanning practices. However, a retrospective analysis of data from 227 ERs across 41 states, found that CT utilization varied training based on background of physicians, in that it was higher for emergency medicine boarded physicians vs. nonemergency-boarded physicians [18]. Furthermore, Weigner et al. [19] reported that emergency physicians sometimes ordered a CT scan that they did not feel was indicated (89%), to appease the patient or family (92%), and due to malpractice risk (85%).

A study of pediatric CT practice showed that being under 5 years of age was inversely associated with following set guidelines and with a higher incidence of inappropriate CT scans [16]. It is plausible that very young children are more difficult to clinically assess and that there may be higher anxiety among clinicians about missing key indicators in these patients, which can lead to over utilization of CT scans. Likewise, Maguire et al. [20] conducted a systematic review of clinical protection rules for the appropriate use of head CT scans for childhood head injury and reported that the rules performance for very young children was in general “poor” and better for age greater than 2 years old. This is consistent with the current study findings that the odds of algorithm adherence were lower for patients under 2 years of age. It is possible that the algorithm may have been more difficult to apply in pre-verbal children and this appears to be consistent with prior literature. It is possible that change in provider use of the CT head scan algorithm for the youngest children in the current study may have been observed with further experience over a longer study period; this warrants additional research. Interestingly, upon review of medical records of children under 2 years of age who went to an ER and received a skull fracture diagnosis, Reid, Liu, & Ortega [21] found of the 92 patients who received a head CT, that none had a clinically significant intracranial injury.

The authors’ findings suggest that observation may be a sufficient strategy for children under 2 years with a head injury exhibiting no clinical signs of intracranial injury who have a nondisplaced linear skull fracture on plain radiography. QI efforts to improve the efficiency and effectiveness of pediatric CT would be beneficial throughout the US healthcare system. This might include drawing from the multiple concerted efforts initiated by the International Atomic Energy Agency, e.g., prepare/use detailed guidelines, focused training on dosimetry skills, periodic reporting system, mentoring and motivating collaborations [22].

Limitations: A limitation of the project was the potential loss of study participants to the adult ED as the pediatric ED is not open twenty-four hours. We do not have the data available to assess sustainability of the impact from this QI project. Because data were collected through retrospective chart review, possible factors that influenced the reasoning behind decisions to order or not order a CT scan study (e.g., parental insistence) were not available. A potential limiting factor is that the study investigators conducted the chart data abstraction. Further because the study was performed at a single site, the findings may not be generalizable to other pediatric emergency departments.


Radiation exposure from CT scans ought to be kept as low as possible and alternative strategies, which do not involve ionizing radiation, should be considered if appropriate. There is a need for a good clinical decision making tools to guide head imaging decisions in children presenting to the ED with trauma and other diagnoses. Our quality improvement initiative that facilitated the implementation of an evidencebased algorithm led to an improvement in the guideline adherence for appropriate use of CT scans of the head the ED setting, for both trauma as well as non-trauma etiologies. The main implication is that a relatively simple, low-cost intervention resulted in at least a short-term improvement in appropriate use of CT scans of the head, consistent with the evidence-based algorithm, in children presenting to the pediatric ED. Further study is warranted to assess sustainability of the positive outcomes of the intervention and to identify barriers and facilitators to algorithm adherence and sustainability. The long-term benefit is that by adhering to the guidelines, we potentially can alter the risk of unnecessary radiation exposure (and potentially associated cancer) in children. In our study, as well as from the experiences of others, implementation of clinical predictive rules for head imaging has been challenging in children under 2 years of age. Further studies are needed to develop and evaluate algorithms for head CT imaging for the young patient population. A desirable tool would be evidence-based as well as addressing provider concerns and discomforts with foregoing imaging in appropriate cases, leading to a high degree of adherence among clinicians. Furthermore, as technology continues to advance, mechanistic modifications to equipment that lead to the lowest possible radiation exposure may present an extremely effective approach to controlling radiation exposure. However, such equipment may be costly and not widely available. At the time of the study, the Northwest Community Hospital was utilizing the fastest CT scanner available which minimized the amount of radiation exposure, but there were still concerns and questions about which cases and circumstances were appropriate for CT scanning. Research is warranted to address the question of whether use of an intervention to facilitate adoption of an evidence-based algorithm has more or less impact on outcomes than technically-advanced mechanical equipment (that alters the scanner to use the lowest radiation exposure possible).


None of the authors have a financial or other relationship that might signify a conflict of interest. This paper reflects the authors’ findings and conclusions and does not necessarily reflect the official position of the organizations in which the authors are affiliated.


1. Larson DB, Johnson LW, Schnell BM, Goske MJ, Salisbury SR, Forman HP. Rising use of CT in child visits to the Emergency Department in the United States, 1995-1998. Radiology. 2011, 259(3): 793-801.

2. Smith-Bindman R, Lipson J, Marcus R, Kim KP, Mahesh M et al. Radiation dose associated with common computed tomography examinations and the associated lifetime attributable risk of cancer. Archives of Internal Medicine. 2009, 169(22): 2078-2086.

3. Krille L, Zeeb H, Jahnen A, Mildenberger P, Seidenbusch M, Schneider K et al. Computed tomographies and cancer risk in children: A literature overview of CT practices, risk estimations and an epidemiologic cohort study proposal. Radiation and Environmental Biophysics. 2012, 51(2): 103-111.

4. Huser MM, Grosel JM. Children and radiation risk in the age of increased utilization of CT. Journal of the American Academy of Physician Assistants. 2013, 26(1): 36-40.

5. Pearce M S, Salotti JA, Little MP, McHugh K, Lee C, Kim KP et al. Radiation exposure from CT scans in childhood and subsequent risk of leukaemia and brain tumours: A retrospective cohort study. The Lancet. 2012, 380(9840): 499-505.

6. Rehani B. Imaging overutilization: Is enough being done globally? Biomedical Imaging and Intervention Journal. 2011, 7(1): e6.

7. American College of Radiology. ACR Appropriateness Criteria, 2013.

8. The Royal College of Radiologists. Making the Best Use of Clinical Radiology Services: Referral Guidelines (6th edn). London, England: The Royal College of Radiologists. 2007.

9. Gillam S, Siriwardena AN. Evidence-based healthcare and quality improvement. Quality in Primary Care. 22(3), 125- 132.

10. Kupperman N, Holmes JF, Dayan PS, Hoyle JD Jr, Atabaki SM et al. Identification of children at very low risk of clinically-important brain injuries after head trauma: A prospective cohort study. The Lancet. 2009, 374(9696): 1160–1170.

11. Mooney JS, Yates A, Sellar L, Shipway T, Roberts C, Parris R et al. Emergency head injury imaging: Implementing NICE 2007 in a tertiary neurosciences centre and a busy district general hospital. Emergency Medicine Journal. 2011, 28(9): 778-782.

12. Rycroft-Malone J, Seers K, Titchen A, Harvey G, Kitson A, McCormack B. What counts as evidence in evidence-based practice? Journal of Advanced Nursing. 2004, 47(1): 81-91.

13. Weber V, Bulger J. Implementing healthcare quality improvement: Changing clinician behavior. In E. R. Ransom, M. S. Joshi, D. B. Nash, S. B. Ransom (Eds.), The Healthcare Quality Book: Vision, Strategy, and Tools. Chicago, IL: Health Administration Press. 2008, 377-406.

14. Huynh TM. Innovators and early adopters of population health in healthcare: real and present opportunities for healthcare-public health collaboration. Healthcare Papers. 2014, 13(3): 53-57.

15. Menoch MJ, Hirsh DA, Khan NS, Simon HK, Sturm JJ. Trends in computed tomography utilization in the pediatric emergency department. Pediatrics. 2012, 129(3): 690-697.

16. Vassileva J, Rehani MM, Al-Dhuhli H, Al-Naemi HM, AlSuwaidi JS, Appelgate K et al. IAEA survey of pediatric CT practice in 40 countries in Asia, Europe, Latin America, and Africa: Part 1, frequency and appropriateness. American Journal of Roentgenology. 2012, 198(5): 1021-1031.

17. Fung V, Schmittdiel JA, Fireman B, Meer A, Thomas S, Smider N et al. Meaningful variation in performance: A systematic literature review. Medical Care. 2010, 48(2): 140- 148.

18. Kirsch TD, Hsieh YH, Horana L, Holtzclaw SG, Silverman M, Chanmugam A. Computed tomography scan utilization in emergency departments: A multi-state analysis. The Journal of Emergency Medicine. 2011, 41: 302-309.

19. Weigner MB, Dewar KM, Basham HF, Rupp VA, Greenberg MR. Impact of education on physician attitudes toward computed tomography utilization and consent. 2012, 43(5): e349-e353.

20. Maguire JL, Boutis K, Uleryk EM, Laupacis A, Parkin PC. Should a head-injured child receive a head CT scan? A systematic review of clinical prediction rules. Pediatrics. 2009, 124(1): 145-154.

21. Reid SR, Liu M, Ortega HW. Nondepressed linear skull fractures in children younger than 2 years: Is computed tomography always necessary? Clinical Pediatrics. 51(8): 745- 749.

22. Rehani MM, Tsapaki V. Impact of the International Atomic Energy Agency (IAEA) actions on radiation protection of patients in many countries. Radiation Protection Dosimetry. 2011, 147(1-2): 34-37.