Jacobs Journal of Emergency Medicine

Comparison of NIHSS Scores Determined by Emergency Physicians and Neurologists using Pre-Hospital Telemedicine Device

Department Of Emergency Medicine, Hospital Center Dijon, University Of Burgundy, Dijon, France

*Corresponding Author:
Department Of Emergency Medicine, Hospital Center Dijon, University Of Burgundy, Dijon, France

Published on: 2014-12-04


Introduction The management of acute stroke requires a rapid but appropriate clinical assessment of the patient’s neurologic deficit. This study investigates the comparative efficiency of emergency physicians and neurologists to determine the National Institute of Health Stroke Scale (NIHSS) score, the main tool for assessing the severity of a stroke and guiding possible subsequent fibrinolysis. Material and Methods A pre-hospital telemedicine device recently developed in Burgundy was used to compare the NIHSS score determined simultaneously by a neurologist at a remote location and emergency physicians at the patient’s beside. All examiners had been trained to determine this score. A Chi2 test and Spearman’s Rho test for statistical analysis were performed. 


Telemedicine, Acute Stroke, NIHSS, Emergency Physician

Copyright: © 2016 Marc


Acute stroke is one of the main causes of prolonged functional deficiency and mortality in France and the suspicion of stroke remains a frequent reason to resort to emergency healthcare services. Although stroke treatment has vastly improved, particularly since the development of fibrinolysis therapy, the pre hospital stroke management is still a matter of debate. Our state, Burgundy, is a large region and there are only two stroke units, which explains our interest in the telestroke. Indeed, in the pre-hospital phase, the neurological deficit and the severity of the acute stroke must be evaluated during the emergency call [1,2]. Diagnosing stroke severity quickly allows direct orientation towards an imaging unit, if appropriate, before fibrinolysis is performed, if indicated. The worst damage from a stroke often occurs within the first few hours.

The emergency physician is involved in the initial management from the call to the medical dispatching centre to admission to the emergency department. The first medical step in the management of stroke is the clinical assessment of the neurological deficit. Several reliable and well-validated scoring systems have been developed to evaluate stroke severity, which can determine the use of fibrinolysis therapy. Some of these have limitations [3] and the National Institute of Health Stroke Scale (NIHSS) is the most widely used.

The NIHSS is a 15-item scoring system and a reliable tool for the rapid assessment of stroke-related impairment [4,5]. It is a rapid reproducible tool and its interrater reliability has been well-established [6,7]. The scale graduates the level of consciousness, extraocular movements, visual fields, facial muscular function, upper and lower limb strength, sensory function, coordination (ataxia), language (aphasia), speech (dysarthria), and hemi-inattention (negligence). [table]

Table 1. National Institute of Health Stroke (NIHSS).

The initial NIHSS assessment appears to be a strong indicator of functional prognosis, since it evaluates neurological outcomes and the degree of recovery of stroke patients. Excellent outcomes were noted in 46% of patients with NIHSS scores of 7 to 10 and in only 23% of patients with scores of 11 to 15 [8].

The current era of telestroke for the management of stroke requires emergency physicians to be trained in certain skills usually reserved for neurologists, especially in determining indications for and performing fibrinolysis when a neurologist is not available. The present study investigated correlations between NIHSS scores determined by neurologists and by emergency physicians by using a prehospital telemedicine system. The main goal was to compare the reliability of the NIHSS score assessed remotely by the neurologist and that assessed locally by the emergency physician. The second aim was to assess the interrater reliability for NIHSS items, the time to complete the scale, and the quality of the telemedicine device.

Materials And Methods

Study Design

A pre-hospital telemedicine device, which receives images and sounds in real-time and is contained in a compact suitcase, was used (figure 1). This system, designed by the company W2NUMERICOM© (Pontailler-sur-Saone, France), includes software, an EDGE/3G internet key, a hand-free, head-mounted camera, an on-board transmission system using four batteries with four hours of autonomy and a Bluetooth headset. This system makes it possible to send multiple files to several destinations. Moreover, it guarantees the instantaneous and secure transmission of confidential medical data by a data encoding system.

In this preliminary study, the NIHSS score was assessed by a neurologist sitting in front of the screen of the telemedicine device at the university hospital stroke unit on the one hand, and by one or more emergency physicians close to the patient in pre-hospital settings. The emergency physicians had all been trained in determining this score. Each patient gave informed consent to participate and agreed to be filmed by the head-mounted camera or by the camera fixed above the patient. Neither the neurologist nor the emergency physicians had any prior clinical knowledge of the patients, and all were kept blinded to the examinations and scores of the other. The examiners all agreed to this investigation and accepted to be filmed. NIHSS Scores named “remote scores” were determined by the neurologist in the stroke unit whereas “bedside scores” were determined by the emergency physicians who were with the patients.

Figure 1. The prehospital telemedicine system.

All scorers had previously been trained using videotapes provided by the French Emergency Medicine Society. For the first 12 patients, we measured the time required by emergency physicians and the neurologist to determine the NIHSS score.

Statistical Data Analysis

We used the Chi2 test and Spearman’s Rho test for the statistical analyses.  p 0.8, excellent agreement; between 0.6 and 0.8, good agreement; between 0.4 and 0.6, moderate agreement; between 0.2 and 0.4, mediocre agreement and was considered significant. Interrater agreement between bedside and remote scores was measured by the kappa value for each item of the NIHSS. Published standards for interpretation of the kappa statistic were used (values >0.8, excellent agreement; between 0.6 and 0.8, good agreement; between 0.4 and 0.6, moderate agreement; between 0.2 and 0.4, mediocre agreement and


Sixteen patients were enrolled and 52 comparisons were made at the end of experimental sessions by three neurologists and five emergency physicians.

Table 2. kappa item reliability between remote and bedside examiners.

At the end of experiment, the NIHSS scores determined through the examination of the neurovascular unit stroke patients ranged from 1 to 21. The statistical study showed no significant difference between the neurologists’ scores and the emergency physician’s scores. The correlation coefficient between the two scores was considered excellent (rho = 0.97, p <0.001) (Figure 2)

Actually, some minor problems (light, camera position, connection problems, and sound quality) occurred during two sessions, but the sessions continued regardless. According to the kappa index (Table 2), six NIHSS items (level of consciousness, orientation, commands, dysarthria, motor leg, and motor arm) displayed excellent agreement, five items (visual fields, best gaze, facial palsy, sensory, language) good agreement and two items (ataxia, negligence) moderate agreement. The kappa index for each item is presented in the first column of Table 2.

Figure 2. Correlation between NIHSS determined by emergency physician and the neurologist.

The average time to determine an NIHSS score using the telemedicine device was 14 seconds longer than that for the emergency physician close to the patient (6.24 ± 2.3 versus 6.10 ± 2.4 minutes, p = 0.85). The time required by emergency physicians and the neurologist to determine the NIHSS are presented in Figure 3.

Figure 3. Time required by emergency physicians and neurologist to determine NIHSS scores.


The main findings of this study were the absence of any difference between the NIHSS scores determined by the neurologist and those determined by trained emergency physicians. Bedside and remote NIHSS scores did not differ by more than 3 points for any patient. It can thus be concluded that trained emergency physicians and neurologists are equally able to determine acute stroke severity, which underlines the fact that emergency physicians, like neurologists, are able determine whether fibrinolysis is indicated. In the literature, few studies have investigated such a comparison [9-11]. Moreover, the interrater agreement ranged from good to excellent in almost all items of the NIHSS for all patients. Agreement was moderate for only two items (ataxia, negligence). This moderate agreement could be due to the difficulty for emergency physicians to define cerebellar dysfunction and negligence precisely in hemiplegic patients because of their lack of experience in these specific items. Nevertheless, each physician involved as an examiner in our study had had previous training on the NIHSS. Continuous training in the use of the NIHSS appears to be necessary. In the literature, the least agreement was observed for facial palsy and leg motor subscales. These differences could be explained by the image quality or the angle of projection [12-14]

The wide use of telemedicine systems could be an effective response in a strategy to eliminate disparities in access to acute stroke care, to optimize the use of time of medical resources and to improve collaboration between health care providers [15-18]. Many studies have validated several telemedicine systems focused on various processes (video cell phone, high speed connection…) but only in in-hospital practice [19-24]. Our study showed that our pre-hospital system, using low-speed data transmission, was a feasible and reliable telemedicine method. Although it could be considered expensive (approximately USD 15,000, it can be used in pre-hospital settings [25] and in areas that are poorly covered by the third generation network service. This system may facilitate fibrinolysis delivery in areas that have insufficient healthcare cover for neurological diseases [26,27].


The study was based on the experience of a single university hospital and therefore the results may not be fully applicable to every hospital. Moreover, the previous training of emergency physicians was mandatory and was time-consuming and difficult to organize. To date, training of Burgundy emergency physicians to evaluate stroke by NIHSS and to perform fibrinolysis if indicated is only available in Dijon and it requires 2 days of theoretical training and 6 days of duty in the neurovascular unit. An examination at the end of this session can result in the delivery of certificate attesting the ability of the emergency physician to perform fibrinolysis.

Patients selected in this study did not present acute stroke at the moment of the examination. Some of them, however, had signs of a deficit from a previous stroke.




Our study demonstrated that trained emergency physicians and neurologists displayed equivalent skills in determining the NIHSS score. Interrater agreement was good to excellent for almost all items. The remote examination of acute stroke patients with our pre-hospital telemedicine system appeared feasible and reliable.

RT and MF conceived and carried out this study. RT, PYC and CT participated in the study as trained emergency physician examiners. MHB and YB represented the trained neurologist examiners. LSA designed and performed the statistical analysis of this study. RT wrote this article and CT participated in drafting the manuscript. MF takes the responsibility for the paper as the whole.

Funding and Support

All authors are required to disclose any and all commercial, financial and relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.ICMJE.org) The authors have stated that no such relationships exist. This study was presented in the Emergency Medicine Society Meeting (Urgences SFMU, Paris), June 2012, Paris, France.


The authors are grateful to Maurice GIROUD, MD, PHD head of the Neurology Department of Dijon University Hospital, for his help and assistance throughout this study. The authors would like to thank Mr P.LEBLANC, of the company W2NUMERICOM©, for his help and availability with the telemedicine device used in this study throughout the experiment.


1. Guidelines for Management of Ischaemic Stroke and Transient Ischaemic Attack 2008. The European Stroke Organisation (ESO) Executive Committee and the ESO Writing Committee. Cerebrovasc Dis. 2008, 25(5): 457– 507.

2. Adams Jr HP, del Zoppo, Alberts MA, Bhatt DL, Brass L, Grubb R et al. Guidelines for the early management of adults with ischemic stroke: a guideline from the American Heart Association/American Stroke Association Stroke Council, Clinical Cardiology Council, Cardiovascular Radiology and Intervention Council, and the Atherosclerotic Peripheral Vascular Disease and Quality of Care Outcomes in Research Interdisciplinary Working Groups: the American Academy of Neurology affirms the value of this guideline as an educational tool for neurologists. Stroke. 2007, 38(5): 1655-1711.

3. Brott T, Adams HP, Olinger CP, Marler JR, Barsan WG et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke. 1989, 20(7): 864-870.

4. Spilker J, Kongable G, Barch C, Braimah J, Brattina P et al. Using the NIH Stroke Scale to assess stroke patients. J Neurosci Nurs. 1997, 29(6): 384-392.

5. Wityk R, Pessin MS, Kaplan RF and Caplan LR. Serial assessment of acute stroke using the NIH Stroke Scale. Stroke. 1994, 25(2): 362-365.

6. Goldstein LB, Bertels C and Davis JN. Interrater reliability of the NIH Stroke Scale. Arch Neurol. 1989, 46(6): 660-662.

7. Lyden P, Brott T, Tilley B, Welch KMA, Mascha EJ et al. Improved reliability of the NIH Stroke Scale using video training. NINDS TPA Stroke Study Group. Stroke. 1994, 25(11): 2220-2226.

8. Adams HP Jr, Davis PH, Leira EC, Chang KC, Bendixen BH et al. Baseline NIH Stroke Scale score strongly predicts outcome after stroke: A report of the Trial of Org 10172 in Acute Stroke Treatment (TOAST). Neurol. 1999, 53(1): 126-131.

9. Meyer BC, Raman R, Chacon MR, Jensen M, Werner JD. Reliability of site-Independent telemedicine when assessed by telemedicine-Naive Stroke Practitioners. J Stroke Cerebrovasc. 2008, 17(4): 181-186.

10. Liman TG, Winter B, Waldschmidt C, Zerbe N, Hufnagl P et al. Telestroke Ambulances in Prehospital Stroke Management Concept and Pilot Feasibility Study. Stroke. 2012, 43(8): 2086-2090.

11. Gonzalez MA, Hanna N, Rodrigo ME, Satler LF, Waksman R. Reliability of prehospital real-time cellular video phone in assessing the simplified National Institutes of Health Stroke Scale in patients with acute stroke: A novel telemedicine technology. Stroke. 2011, 42(6): 457-507.

12. Wang S, Lee SB, Pardue C, Ramsingh D, Waller J et al. Remote evaluation of acute ischemic stroke: Reliability of national institutes of health stroke scale via telestroke. Stroke. 2003, 34(10): 188-192.

13. Handschu R, Littmann R, Reulbach U, Gaul C, Heckmann JG et al. Telemedicine in emergency evaluation of acute stroke: Interrater agreement in remote video examination with a novel multimedia system. Stroke. 2003, 34(12): 2842-2846.

14. Shafqat S, Kvedar JC, Guanci MM, Chang Yand Schwamm LH. Role for telemedicine in acute stroke: Feasibility and reliability of remote administration of the NIH Stroke Scale. Stroke. 1999, 30(10): 2141-2145.

15. Telemedic Pilot Project for Integrative Stroke Care Group, Audebert HJ, Schenkel J, Heuschmann PU, Bogdahn U et al. Effects of the implementation of a telemedical stroke network: the Telemedic Pilot Project for Integrative Stroke Care (TEMPiS) in Bavaria, Germany. Lancet Neurol. 2006, 5(9): 742-748.

16. Audebert J, Schenkel J, Heuschmann P, Bogdahn U, Haberl R. Effects of the implementation of a telemedical stroke network: the Telemedic Pilot Project for Integrative Stroke Care (TEMPiS) in Bavaria, Germany. Lancet Neurol. 2006, 5(9): 742-748.

17. Audebert HJ, Kukla C, Vatankhah B, Gotzler B, Schenkel J et al. Comparison of tissue plasminogen activator administration management between telestroke centers. The telemedical pilot project for integrative stroke care in Bavaria/Germany. Stroke. 2006, 37(7): 1822-1827.

18. Hess D, Wang S, Hamilton W, Lee S, Pardue C et al. REACH : feasibility of a rural telestroke network. Stroke. 2005, 36(9): 2018-2020.

19. Medeiros de Bustos E, Bouamra B, Chavot D, Moulin T. Telestroke in France, status in 2012 and further developments. Eur Res Telemed. 2012, 1: 12-18.

20. Schwab S, Vatankhah B, Kukla C, Hauchwitz M, Bogdahn U et al. Long-term outcome after thrombolysis in telemedical stroke care. Neurology. 2007,69(9): 898-903.

21. Johansson T, Wild C. Telemedicine in acute stroke management: Systematic review. Int J Technol Assess Health Care. 2010, 26(2): 149–155.

22. Allibert R, Ziegler F, Bataillard M, Gomes C, Jary A et al. Telemedicine and fibrinolysis in Franche-Comté. Rev neurol. 2012, 168(1): 40-48.

23. Schwamm LH, Holloway RG, Amarenco P, Audebert HJ, Bakas T et al. A Review of the evidence for the use of telemedicine within stroke systems of Care: A scientific statement from the American Heart Association/American Stroke Association. Stroke. 2009, 40(7): 2616-2634.

24. Schwamm LH, Rosenthal ES, Hirshberg A, Schaefer PW, Little EA et al. Virtual teleStroke support for the emergency department evaluation of acute stroke. Acad Emerg Med. 2004, 11(11): 1193-1197.

25. LaMonte MP, Xiao Y, Hu PF, Gagliano DM, Bahouth MN et al. Shortening time to stroke treatment using ambulance telemedicine: TeleBAT. J Stroke Cerebrovasc Dis. 2004, 13(4): 148-154.

26. Grotta JC, Burgin WS, El-Mitwalli A, Long M, Campbell M et al. Intravenous tissue-type plasminogen activator therapy for ischemic stroke: Houston experience 1996 to 2000. Arch Neurol. 2001, 58(12): 2009-2013.

27. Tatlisumak T, Soinila S, Kaste M. Telestroke networking offers multiple: benefits beyond thrombolysis. Cerebrovasc Dis. 2009, 27(Suppl 4): 21-27.