Assessment of inter-examiner agreement and variability in the manual classification of auditory brainstem response
Naves, K. F. P., Pereira, A. A., Nasuto, S. J., Russo, I. P. C. and Andrade, A. O. (2012) Assessment of inter-examiner agreement and variability in the manual classification of auditory brainstem response. BioMedical Engineering OnLine, 11. 86. ISSN 1475-925X
To link to this item DOI: 10.1186/1475-925X-11-86
Abstract Background: The analysis of the Auditory Brainstem Response (ABR) is of fundamental importance to the investigation of the auditory system behaviour, though its interpretation has a subjective nature because of the manual process employed in its study and the clinical experience required for its analysis. When analysing the ABR, clinicians are often interested in the identification of ABR signal components referred to as Jewett waves. In particular, the detection and study of the time when these waves occur (i.e., the wave latency) is a practical tool for the diagnosis of disorders affecting the auditory system. Significant differences in inter-examiner results may lead to completely distinct clinical interpretations of the state of the auditory system. In this context, the aim of this research was to evaluate the inter-examiner agreement and variability in the manual classification of ABR. Methods: A total of 160 ABR data samples were collected, for four different stimulus intensity (80dBHL, 60dBHL, 40dBHL and 20dBHL), from 10 normal-hearing subjects (5 men and 5 women, from 20 to 52 years). Four examiners with expertise in the manual classification of ABR components participated in the study. The Bland-Altman statistical method was employed for the assessment of inter-examiner agreement and variability. The mean, standard deviation and error for the bias, which is the difference between examiners’ annotations, were estimated for each pair of examiners. Scatter plots and histograms were employed for data visualization and analysis. Results: In most comparisons the differences between examiner’s annotations were below 0.1 ms, which is clinically acceptable. In four cases, it was found a large error and standard deviation (>0.1 ms) that indicate the presence of outliers and thus, discrepancies between examiners. Conclusions: Our results quantify the inter-examiner agreement and variability of the manual analysis of ABR data, and they also allows for the determination of different patterns of manual ABR analysis.
1. Hood LJ: Clinical Applications of the Auditory Brainstem response. San Diego: Singular Publishing Group Inc.; 1998. 2. Nodarse EM, Abalo MCP, López GS: Métodos de pesquisaje de las pérdidas auditivas a edades tempranas. Revista Electrónica de Audiologia 2006, 3:9–18. 3. Eggermont JJ: Electric and Magnetic Fields of Synchronous Neural Activity. In Auditory Evoked Potentials: basic principles and clinical application. Edited by Burkard RF, Eggermont JJ, Don M. Baltimore: Lippincott Williams & Wilkins; 2007:2–21. 4. Hall JW: New Handbook of Auditory Evoked Responses. Boston: Pearson Edication, Inc.; 2006. 5. Misulis KE: Potencial Evocado de Spehlmann. 2nd edition. Rio de janeiro, Brazil: Revinter Ltda; 2003. 6. Schwanke D: Exame de Potenciais Evocados Auditivos Utilizando Processador Digital de Sinais - DSPEA. Dissertação de Mestrado. Porto Alegre, Brazil: Universidade Federal do Rio Grande do Sul, Instituto de Informática; 2000. 7. Sininger YS: Source Analysis of Auditory Evoked Potentials and Filds. In The use of Auditory Brainstem Response in Screening for Hearing Loss and Audiometric Threshold Prediction. Edited by Burkard RF, Eggermont JJ, Don M. Baltimore: Lippincott Williams & Wilkins; 2007:254–274. 8. Martin WH, Shi BYB: Intraoperative monitoring. In Auditory Evoked Potentials: Basic Principles and Clinical Application. Edited by Burkard R, J.J E, Don M. Philadelphia: Lippincott Williams & Wilkins; 2007:355–384. 9. Katz J: Audiologia clínica. 3rd edition. New York: manole; 1989. 10. Garcia BG, Gaffney C, Chacon S, Gaffney M: Overview of newborn hearing screening activities in Latin America. Rev Panam Salud Publica 2011, 29:145–152. 11. Chomsky N: Three factors in language design. Linguistic Inquiry 2005, 36:1–22. 12. Fitcha WT, Hauserb MD, Chomsky N: The evolution of the language faculty: clarifications and implications. Cognition 2005, 97:179–210. 13. Vidler M, Parker D: Auditory brainstem response threshold estimation: subjective threshold estimation by experienced clinicians in a computer simulation of the clinical test. Int J Audiol 2004, 43:417–429. 14. Pediatrics AAO: Newborn and infant hearing loss: detection and intervention. Pediatricis 1999, 103:527–530. 15. Junqueira CAO, Colafêmina JF: Investigação da estabilidade inter e intra-examinador na identificação do P300 auditivo: análise de erros. Rev Bras Otorrinolaringol 2002, 68:468–478. 16. Fernandez R, George F: Validating the Bland-Altman Method of Agreement. In Western Users of SAS Software; Long Beach. Edited by W Conference. California: Long Beach; 2012. http://www.wuss.org/proceedings09/ 09WUSSProceedings/papers/pos/POS-Fernandez.pdf. 17. Porto MAA, Azevedo MF, Gil D: Auditory evoked potentials in premature and full-term infants. Baz J Otohrinolaryngol 2011, 77:622–627. 18. Don M, Ponton CW, Eggermont JJ, Kwong B: The effects of sensory hearing loss on cochlear filter times estimated from auditory brainstem response latencies. Acoustical Society of America 1998, 104:2280–2289. 19. Hernández JD, Castro FZ, Prat JJB: Normalización de los potenciales evocados auditivos del tronco cerebral I: resultados en una muestra de adultos normoyentes. Revista Electrónica de Audiologia 2003, 2:13–18. 20. Cd M, Manjón M, vinuales M, Menéndez C: Estúdio morfológico de los potenciales evocados auditivos de tronco Del encéfalo.Influência de la posición Del eletrodo de referência. Rev Neurol 2002, 34:84–88. 21. Vannier E, Adam O, Motsch J-F: Objective detection of brainstem auditory evoked potentials with a priori information from higher presentation levels. Artif Intell Med 2002, 25:283–301. 22. Antonelli AR, Bellotto R, Grandori F: Audiologic diagnosis of central versus eighth nerve and cochlear auditory impairment. Audiology 1987, 4:209–226. 23. Don M: Quantitative approaches for defining the quality and threshold of auditory brainstem responses. IEEE Engineering In Medicine & Biology Society 1989, 2:0761–0762. 24. Jacquin A, Causevic E, John ER, Prichep LS: Optimal denoising of brainstem auditory evoked response (BAER) for Automatic peak identification and brainstem assessment. In Book Optimal denoising of brainstem auditory evoked response (BAER) for Automatic peak identification and brainstem assessment. City: IEEE; 2006:1723–1726. 25. Acyra N, Ozdamarb O, Guzelis C: Automatic classification of auditory brainstem responses using SVM-based feature selection algorithm for threshold detection. Eng Appl Artif Intell 2006, 19:209–218. 26. Boston JR: Automated interpretation of brainstem auditory evoked potentials: a prototype system. IEEE Trans Biomed Eng 1989, 36:528–532. 27. Bradley AP, Wilson WJ: On wavelet analysis of auditory evoked potentials. Clin Neurophysiol 2004, 115:1114–1128.