Equivalence of the autism spectrum disorders diagnostics in children in telemedicine and face-to-face consultations: a literature review

Full Text

INTRODUCTION

Limitations on the availability and timeliness of specialized psychiatric care are a reality across the globe. This mainly has to do with the high prevalence of mental disorders and the shortage of staff in the specialized services of health care systems, especially in small settlements and those geographically remote from large medical and diagnostic centers, not to mention the obstacles involved in seeking the help of specialists because of the pervasive issue of stigmatization [1].

Autism spectrum disorder (ASD) is currently among the most challenging problems in pediatric psychiatry, due to its increasing rate of detection in recent decades, poorly defined etiopathogenetic factors, its diagnostic framework, and the therapeutic approaches used, as well as the need for long-term intensive complex treatment and rehabilitation. There are significant issues related to the scarcity of medical and diagnostic resources for families in rural or remote areas with a lower socioeconomic status [1, 2]. In such cases, ASD is diagnosed with a significant delay [3].

The active use of remote forms of assistance, including telemedicine consultations (TMCs), expanded during the epidemiological restrictions that came with COVID-19 around the world [4–6]. Russian specialists have accumulated sufficient experience in conducting TMCs in the “doctor-doctor” format, with remote interaction between healthcare professionals, including in psychiatry [7, 8]. At the same time, due to the multifactorial limitations relating to the availability of specialized medical care, the “patient-doctor” TMC format using video conferencing (VC) seems to be more in demand. A wide range of organizational and methodological issues related to medical care during remote interaction between healthcare professionals and patients or their legal representatives having to do with the regulation of the scope of medical intervention, the use of examination and treatment methods, quality assurance, information and clinical security remain unresolved [9, 10].

The goal of this review is to study how diagnostic tools compare to each other in the framework of telemedicine and in-person consultation as they apply to children with ASD. Our effort was based on a review of the extant scientific literature.

METHODS

We analyzed papers available on the PubMed, Web of Science, and eLibrary electronic databases for a period ranging from January 2017 to May 2023. Search terms included keywords such as “telemedicine diagnostics”, “telemedicine consultations”, “equivalence of telemedicine consultations”, “autism spectrum disorder”, and “children and teenagers”. Studies were considered eligible for analysis if they assessed the comparative validity of telemedicine (remote) and in-person consultations for the purpose of diagnosis, as well as the quantitative and qualitative assessment of ASD in children. Ninety-five articles were reviewed, and 43 of them were selected for analysis. In addition, we analyzed a number of related articles in Google Scholar and reviewed earlier longitudinal studies (up to 2017) and publications on diagnostic tools adapted for use in a remote format.

Descriptive analysis was used to summarize the obtained data.

RESULTS

Even before the COVID-19 pandemic hit, there was interest in developing and testing new remote forms of care for patients with ASD to improve access to diagnosis, treatment, and rehabilitation, as well as to increase the role and involvement of patients’ parents in the assessment procedures [11]. The relevance and growth of the research into the use of telemedicine in medical care for children and adolescents with ASD is evidenced by the change in the number of publications in systematic reviews. The publication by Sutherland et al. in 2018 [12], contains an analysis of the results of 14 studies, and the most recent review by Ellison et al. [13], conducted just 3 years after the previous one, already included 55 peer-reviewed articles.

The obtained data on the use of a remote format for diagnosing autistic disorders in childhood shall be divided into three parts: 1) ASD risk screening, 2) qualitative and quantitative diagnostics using standardized rating scales and procedures, and 3) clinical (clinical and psychopathological) examination. The distinction between the 2nd and 3rd options can be made only with some degree of conditionality, since in most of the analyzed studies, clinical diagnostics included the use of standardized assessment tools, which in many countries are provided for by the standards of medical care.

Telemedicine risk screening for autistic disorders

Most of the screening tools used are questionnaires in which the total scores obtained are compared against predetermined thresholds. The first level of screening assessment involves an initial survey in the general population of children in order to identify the risk (“red flags”, i.e., alarms) of ASD. First-level screening tools do not require special training, take minimal time, are conducted by parents or primary medical care professionals, but at the same time they have high sensitivity and low specificity, and therefore the probability of false positive is relatively high. The most popular and most studied first-level screening tool for assessing the risk of ASD validated around the world is The Modified Checklist for Autism in Toddlers, Revised with Follow-Up (M-CHAT-R/F) for children aged 16–30 months [14, 15].

Second-level screening tools have higher specificity, require special training and more time to interpret the results, and, accordingly, are used by trained specialists. These include the Social Communication Questionnaire (SCQ) [16] and the Checklist for Autism Spectrum Disorder (CASD) [17].

The high relevance of and potential demand for remote primary ASD risk screening have been noted in many studies in connection with the significant time gap between the onset of symptoms and the age of diagnosis [18]. According to Constantino et al. [19], the median age of diagnosis in the United States is above 4 years and 27% of children with ASD are not diagnosed by the age of 8 years, while the median age of diagnosis has not decreased in more than 15 years.

According to Qiu et al. [20], remote application of the Chinese version of the Checklist for Autism in Young Children CHAT-23-A for ASD screening showed a sensitivity and specificity of 0.92 and 0.90, respectively. It is believed that it is possible to replace the time-consuming, ineffective and expensive routine offline screening procedure in China with a telemedicine option on the web resource of the Network Center for Early Diagnosis of ASD¹ based on the WeChat platform.

An Indian study by Kadam et al. [21] compared the results of remote screening of 39 children for ASD (M-CHAT-R/F, analysis of 1–2 min home videos) and traditional in-person examination in accordance with DSM-5 diagnostic criteria. Remote assessment showed a correlation of 94.87% with the final diagnosis verified after 3 months as part of an in-person examination. Video-scoring agreement between two independent clinicians had a kappa correlation of 0.803, which was qualified as significant agreement.

A study by Colombo et al. [22] presented the result of an investigation of the first Italian online tool for using CHAT in 1,250 children via a mobile application using the LAMP platform for outpatient pediatricians called Web Italian Network for Autism Spectrum Disorder (WIN4ASD). It demonstrated effectiveness, efficiency, and sustainability of online screening in the primary health care system.

Remote diagnosis of autism spectrum disorder using rating scales and structured procedures

Prior to the COVID-19 pandemic, the development of special tools for remote diagnosis of ASD was rather slow; they have acccelerated in the last 2–2.5 years.

Conventional autism spectrum disorder diagnosis tools

The basic diagnosis of autistic disorders generally includes a structured observation of the child, learning their medical history data from parents, assessment of cognitive, speech and social adaptive functions, as well as a physical examination. Currently, TMC involves using the so-called “gold standard” tools for diagnosing ASD, which include a semi-structured interview with parents as assessment tools for in-person diagnosis: The Autism Diagnostic Interview-Revised (ADI-R) [23] and Structured Child Observation: Autism Diagnostic Observation Schedule (ADOS) [24].

A study by Reese et al. [25] described one of the first experiences with the use of videoconferencing for assessment procedures using ADI-R and ADOS (module 1) vs. a similar in-person assessment. Nearly 100% inter-specialist agreement (20 out of 21 cases) of diagnoses was shown; there were some difficulties in the assessment of socially directed pointing gesture and eye contact with the parent; the survey noted a high level of parental satisfaction.

Synchronous and asynchronous diagnostic approaches

Literature sources outside Russia commonly classify remote diagnostic approaches as synchronous or asynchronous according to the methods used to coordinate the actions of specialists and those receiving care [26, 27]. Synchronous options for remote ASD diagnosis involve monitoring a child’s spontaneous or stimulus-induced behavior in real time in the form of an online video conference. Asynchronous options are usually based on the analysis of video recordings of the child’s behavior. With asynchronous TMC, the transfer of information by the patient (legal representatives) and its processing by specialists occur at different times. Compared to online synchronous TMCs, such organization of interaction minimizes the difficulties of coordinating the schedules of care users and specialists; parents can record videos at convenient days and hours and record the most striking manifestations in the child’s behavior without being limited in time.

The article by Narzisi [26] presents a detailed and comprehensive model of telemedicine diagnostic and corrective care, using both synchronous and asynchronous algorithms for the interaction of a child and his legal representatives with specialists (Appendix 1 in the Supplementary). One of the essential components of this model is the algorithm for parents that describes the preparation of short videos illustrating the peculiarities of the child’s behavior at home. The scenarios included in this algorithm (spontaneous and directed play alone, with parents, siblings, eating together, problematic behavior) with some variations are universal for most tools of remote assessment of ASD manifestations. Video recordings should be made on different days for a more comprehensive understanding of the child’s behavior.

A study by Sutantio et al. [28] concerned the clarification of the validity of diagnosing ASD in children aged 18–30 months based on video recordings according to a protocol that included established scenarios. Diagnostic agreement with in-person consultations was 82.5%, sensitivity was 91.3%, and specificity was 70.6%. This has proved the significant reliability of remote assessment by video recordings vs. the in-person diagnosis of ASD.

According to the article by Riva et al. [29], the most popular structured tools for asynchronous remote assessment of ASD are the Naturalistic Observation Diagnostic Assessment (NODA), The Systematic Observation of Red Flags (SORF), and Brief Observation of Symptoms of Autism (BOSA) (Appendix 2 in the Supplementary).

In a pilot study of the NODA methodology by Nazneen et al. [30], parents easily used the system without prior training to record video materials, 96% of which were found to be clinically relevant for the diagnosis of autism. In 91% of cases, the diagnosticians using NODA Connect confidently (mean score 4.5 on a 5-point scale) arrived at a diagnostic result that aligned with the previous in-person examination of children by other specialists. Smith et al. [31] showed a diagnostic agreement between NODA and in-person diagnosis of 88.2%, sensitivity was 84.9%, and specificity was 94.4%.

At Florida State University, Dow et al. [32] investigated the psychometric properties of their proposed SORF technique in 228 children aged 18 to 24 months with ASD, with developmental delay and with typical development. Specificity and sensitivity were 63% and 73% for social communication and interaction disorders and 54% and 70% for manifestations of stereotypical forms of behavior. The most informative parameters were limited eye contact, looking into an adult’s face, pointing gesture, predominance of interest in non-living objects, adherence to certain non-functional objects, and actions. Pileggi et al. [33] tested SORF as a screening tool for early detection of ASD risk in 122 one-year-old younger siblings of children with confirmed autism. ASD was confirmed in younger siblings at the age of 24 months. With an optimal Composite threshold of 18, sensitivity was 0.77 and specificity was 0.76.

NIDA, Italy’s largest network of interdisciplinary services for observational research and early screening of ASD, has developed the TeleNIDA telemedicine tool for children aged 18–30 months. Parents provide 5-minute videos of their child’s behavior during free play, organized play with parents, eating, and book activities. The tool also has good psychometric properties compared to the “gold standard” in-person assessment [29].

In synchronous remote diagnostics, the tools for assessing the behavior of infants which cause difficulties even during in-person examination are of particular interest. Talbott et al. [34, 35] investigated the possibility of remote detection of ASD risk in 41 infants (mean age 10.51 months) using the Telehealth Evaluation of Development for Infants (TEDI). Inter-rater reliability ranged from 0.88 to 0.94 for most evaluation criteria, and retest reliability was 0.75, p <0.001 (mean interval between 2 tests 1.5 weeks, range 5–41 days).

A study by Kryszak et al. [36] evaluated the Autism Detection in Early Childhood-Virtual (ADEC-V) tool in 121 children aged 18–47 months. It showed high sensitivity (0.82) and specificity (0.78), significant correlation with the results of assessments using other standardized tools (CARS 2, ADI-R), and acceptable internal consistency (α=0.77).

Appendix 2 (in the Supplementary) provides a brief description of other structured tools for remote diagnosis of ASD manifestations based on the materials of the review by Berger et al. [37].

One of the most discussed ones, the TELE-ASD-PEDS (TAP) tool, was specifically developed for remote assessment of ASD in children without phrase speech under the age of 3 years before the COVID-19 pandemic. Currently, work is underway to validate the methodology and preliminary studies have shown a sufficient level of acceptability and convenience for both accompanying persons and specialists [38]. Authors in a separate study [39] compared parents’ perceptions of TAP possibilities with the Screening Tool for Autism in Two-Year-Olds (STAT) adapted for the videoconferencing format [40]. STAT includes assessment of a number of communicative actions when an adult initiates a joint game with a ball or a toy car, the presence of a request/demand of a child when presenting food, repetition of movements, and simple actions. The version for remote use of TELE-STAT contains additional instructions for certain experimental actions with the child, and the presence of eye contact is specified with the parents. Most parents found remote assessment using TAP and TELE-STAT convenient and meaningful, and they separately noted the advantage of these remote ASD assessment tools in the participation of specialists on only “one side of the screen”, which expands their availability and scalability.

The latest publication [39] of a project comparing the use of TAP and TELE-STAT with in-person assessment presents the results of a survey of 144 children aged 17 to 36 months, showing diagnostic agreement in 92% of cases. Diagnostic discrepancies were more often associated with a lesser severity of autistic symptoms or younger age of the children. A large study by McNally Keehn et al. [41] investigated the relationship between the clinical characteristics of 335 children aged 14 to 78 months and the effectiveness of remote diagnosis of ASD using TAP. For 85% of the examined children, including those with speech underdevelopment, the TMC format was sufficient to detect the symptoms of ASD; the presence of specific stereotypical behavior predicted the diagnosis to a greater extent.

We did not find information on remote diagnosis of ASD in Russia using rating scales and structured procedures in the available literature for the specified period.

Remote diagnosis of autism spectrum disorder and the possibilities of artificial intelligence

Developing tools for remote diagnosis of ASD using artificial intelligence (AI) algorithms seems promising [42–45].

For several years, the Cognoa laboratory (Palo Alto, USA) has been gradually validating an ASD screening tool using AI in the form of the Cognoa ASD mobile application [46, 47] — the Child Behavior Checklist to a novel mobile-health screening tool developed by Cognoa. Data for machine learning was collected from several repositories of the ADI-R and ADOS protocols; in an automatic mode and in a short time, the program evaluates the behavioral characteristics of children according to separate questionnaires for parents, specialists, and two short home videos. Abbas et al. [46] showed that the second-generation Cognoa advanced screening tool provided higher accuracy than standard screening tools (M-CHAT-R/F, SRS-II, SCQ) in the same age range. Sensitivity and specificity of 90% and 60% showed the potential of AI-based technology to improve and accelerate the detection of ASD in young children. The latest publication on a double-blind, multicenter, prospective cohort study [48] shows the results of testing the Cognoa tool vs. the diagnostic agreement of two or more independent specialists in a cohort of children aged 18–72 months with developmental delay (n=425, 29% prevalence of ASD). For the 31.8% of participants with a definite result (presence or absence of ASD), the positive predictive value was 80.8%, and the negative predictive value was 98.3%; sensitivity was 98.4%, and specificity was 78.9%. In the group with an “indeterminate” result due to insufficient detail of the input data, 91% of the children had one or more complex neurodevelopmental disorders. Thus, for almost a third of the sample, the Cognoa screening tool allowed timely, rapid diagnostic evaluation with a high degree of accuracy.

Clinical (clinical and psychopathological) remote diagnosis of autism spectrum disorder

In most of the studies, clinical diagnosis involved the use of some of the standardized assessment tools described above. A review of studies on the use of telemedicine diagnosis of ASD by Stavropoulos et al. [49] obtained data on the equivalence of diagnostic assessments compared with in-person consultations in the range of 80-91%. Six of the ten studies yielded a degree of sensitivity ranging from 75% to 100%, while five of the six studies demonstrated specificity values ranging from 68.75% to 100%.

Juarez et al. [50] used TMC to diagnose ASD in 62% of 45 children; in 13% of the cases, autism disorders could not be confirmed or excluded remotely. Matthews et al. [51] investigated the acceptability of diagnosing ASD in children, adolescents, and adults as part of a TMC deployed at an autism center in the U.S. Southwest during the COVID-19 pandemic. One hundred and two (84%) patients out of 121 completed the 6-month remote diagnosis program; for 91% (93 out of 102), it was sufficient to use only telemedicine procedures. In-person assessment was required for nine participants; according to surveys of specialists and parents of patients, the telemedicine model for diagnosing ASD was acceptable for most of the respondents.

The relevance of remote assessment of children aged 18–30 months with M-CHAT-R pre-set ASD risk based on video recordings with certain scenarios according to DSM-5 criteria was compared with similar in-person clinical diagnoses [52]. Diagnostic agreement was 82.5%, sensitivity was 91.3%, and specificity was 70.6%. The positive predictive value was 80.7%, and the negative predictive value was 85.7%.

In a comparative RCT of remote and in-person consultations for 23 patients with ASD aged 4 to 16 years, the diagnosis and treatment recommendations aligned in 96% of cases [53]. There were no differences in the satisfaction of patients and parents, 26% of children preferred the remote format, and 91% of parents preferred videoconferencing without the need to travel long distances for in-person psychiatric visits.

In the Russian-speaking segment, we found only a description of a pilot comparative ASD diagnosis study within the framework of TMC and in-person consultations, which was conducted at the Moscow State Budgetary Healthcare Institution “Scientific and Practical Center for Mental Health of Children and Adolescents named after G.E. Sukhareva of the Moscow Department of Health” [54]. There were 84 patients in the TMC group and 310 patients in the in-person consultations group. All consultations were conducted by one specialist and had a stable clear structure and duration. Mandatory blocks included observation and assessment of the child’s spontaneous behavior, structured situations of interaction with parents, with specialists (attending physician, psychologist, speech therapist, defectologist), and with a remote consultant. Fundamental differences were revealed only in the assessment of the interaction of a child with a remote consultant: the difficulty of assessing eye contact “through the screen,” the degree of subjective attitude of the patient to the consultant, the presence/absence of non-verbal reactions to the background visual, sound, and other stimuli that are not noticeable to the consultant due to fragmented image and sound from the patient and his/her environment (family members, animals, electronic gadgets, and much more). The TMC scenario includes additional clarification questions and actions (tests).

DISCUSSION

The remote format of interaction between specialists and consumers of diagnostic services coincides to the maximum extent with the tasks of ASD screening. Online screening allows one to conduct a primary examination in a much larger group of children aged 16–30 months thanks to the fact that it is easily accessible when placing simple tools with high sensitivity on various web resources; it does not require special training on the part of its users (parents, teachers, or specialists of the primary medical network). Studies conducted in different countries suggest the possibility and expediency of using the telemedicine format instead of the time-consuming and expensive routine offline screening procedure with limited productivity [20–22].

The COVID-19 pandemic has significantly accelerated and scaled up the development of special tools for remote diagnosis of ASD. The analysis of the publications presented in this review convincingly indicates a sufficient equivalence between assessment scales and structured procedures for the remote diagnosis of ASD and in-person examination with a high level of specificity, from 60.0 to 94.4%, sensitivity from 75 to 98.4%, and satisfaction of patients and their legal representatives. Most diagnostic tools are for children over 18 months of age, but tools are also available for the remote diagnosis of ASD in infants 6–12 months of age [34, 35].

Synchronous variants of clinical and psychopathological and remote diagnosis of ASD based on standardized tools are as close as possible to the in-person interaction between specialists and consumers of medical care; however, they require coordinated schedules of care recipients and specialists. There was an almost 100% consistency level between the online and offline formats of the so-called “gold standard for autism diagnosis” ADI-R and ADOS [25].

Asynchronous models of remote assessment of ASD symptoms use video recordings of the child’s behavior in their usual home settings, are free from organizational difficulties in coordinating the schedule of consultations, and provide video recording of the most characteristic manifestations at a convenient time for the required period of time. Typically, recommended video recording scenarios include focusing on the child’s spontaneous and directed play both alone and with parents and siblings, eating together, making requests, imitating actions, and problematic behavior. A number of studies have noted some difficulties in the remote assessment of a pointing gesture and eye contact in the videoconferencing mode, which requires additional clarification of the details of the corresponding manifestations by the persons accompanying the child [36, 39, 54].

A limitation of this review is the fact that a number of studies on the topic under consideration may have been omitted, because a systematic search strategy was not used in the selection of publications. In addition, the methodology and data quality of a number of studies were not sufficiently homogeneous.

CONCLUSION

These authors reviewed publications comparing a remote format for diagnosing autistic disorders in childhood as part of ASD risk screening and clinical diagnosis including the use of standardized rating scales and procedures.

We analyzed various structured tools for a qualitative and quantitative assessment of ASD symptoms developed and validated in different countries for use in the TMC format. A large number of studies have confirmed their acceptable equivalence to in-person diagnosis and sufficient applicability in young children, including infants in their first year of life. At the same time, the availability of these tools in Russian pediatric psychiatry practice is limited; one of the reasons is the need to purchase expensive licenses from copyright holders, which increases the relevance of developing domestic analogues. The introduction and widespread use of validated telemedicine diagnostic systems in clinical practice will contribute to the early detection of ASD and increase the timeliness and effectiveness of medical, corrective psychological, pedagogical, and habilitation interventions.

The active use of the remote diagnostic format can mitigate the limitations in the availability and timeliness of specialized care for children with ASD, which are among the most difficult problems of modern pediatric psychiatry.

In Russia, the “patient-doctor” format of TMC in the case of remote interaction between healthcare professionals and patients and/or their legal representatives has yet to take root, and, therefore, the various organizational, legal, clinical, and methodological aspects of remote care for ASD require further development. One of the relevant issues is the selection of valid diagnostic tools for remote symptom assessment with an evaluation of their agreement with the traditional face-to-face assessment procedures.

Authors’ contribution: O.Z. Khairetdinov: development of the idea and study design, search for publications on the topic of the article, obtaining data for analysis, writing the manuscript; L.I. Rubakova: development of the study design, analysis of the obtained data, discussion of the results and formation of conclusions. All authors contributed significantly to the study and article preparation, read and approved the final version before publication.

Funding: The research was carried out without additional funding.

Conflict of interest: The authors declare no conflicts of interest.

 

¹ http://gdz.fenghuaxinxi.com/admin/login

About the authors

Oleg Z. Khairetdinov

Scientific and Practical Center for Mental Health of Children and Adolescents named after G.E. Sukhareva

Author for correspondence.
Email: psycheas@yandex.ru
ORCID iD: 0000-0002-9842-3524
SPIN-code: 1510-7771
Scopus Author ID: 56558554800

Cand. Sci (Med.), Leading Researcher of the Scientific and Organizational Department, Scientific and Practical Center for Mental Health of Children and Adolescents named after G.E. Sukhareva

Россия, Moscow

Luciena I. Rubakova

Scientific and Practical Center for Mental Health of Children and Adolescents named after G.E. Sukhareva

Email: rubakova-m@rambler.ru
ORCID iD: 0009-0004-6224-9642

child psychiatrist of the Consulting and Diagnostic Department, Scientific and Practical Center for Mental Health of Children and Adolescents named after G.E. Sukhareva

Россия, Moscow

References

Supplementary files