Measures used to assess the effectiveness of asynchronous e-learning in medical education in Sub-Saharan Africa: a systematic review

Masterarbeit, Insitute of Public Health, 118 Seiten, engl.

Summary:

Background: Sub-Saharan Africa (SSA) has one of the weakest health indicators globally while having the fewest resources. A health workforce crisis further exacerbates the weak health systems in SSA, with a high level of ‘brain drain’ of medical professionals affecting the region. A promising method of tackling the health worker crisis is the use of e-learning to train medical professionals in SSA, with e-learning having the potential to increase access to high quality, upto-date, and standardized medical training in the most remote areas of the world, while diminishing reliance on teachers who are often in low-supply. However, proper evaluations of elearning interventions are imperative to determine the effectiveness of interventions in real-world settings. This is particularly important in the context of SSA where the implementation of elearning can often prove challenging due to inadequate ICT infrastructure, resources, and the difficulties with the sustainability of interventions. However, there exists a lack of adequate research on the effects of e-learning in low- and middle-income countries (LMICs), and in particular SSA.

Research Question and Objective: The research question of this systematic review was: What measures are used to assess the effectiveness of asynchronous e-learning within medical education in Sub-Saharan Africa? The objective was to review literature on elearning intervention evaluations in medical education in SSA and to identify, appraise, and recommend methods to evaluate e-learning in the Sub-Saharan context.

Methods: A systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and the Cochrane Collaboration’s steps for conducting systematic reviews. Seven databases were searched using a pre-defined search strategy and eligibility criteria, including PubMed (Medline), PsycINFO, ERIC, Web of Science, Science Direct, Cochrane Library, and Global Index Medicus. Grey literature was searched through the database POPLINE, Google Scholar, and the World Health Organization (WHO) website. Eligible studies for inclusion were original studies where an asynchronous e-learning intervention was implemented and evaluated within a medical training course or program in the context of SSA. Only English language articles published between January 2007 until June 11, 2017 were included. The results were screened and assessed for eligibility based on title, abstract, and full-text as per inclusion and exclusion criteria. Outcome measures of included studies were coded via inductive thematic analysis using NVivo® software. Lastly, included studies were assessed for quality using the Medical Education Research Study Quality Instrument (MERSQI) and Newcastle-Ottawa Scale (NOS), and risk of bias was assessed using an adapted ACROBAT-NRSI tool.

Results: Eleven studies matching all inclusion and exclusion criteria were included in this review. The asynchronous e-learning interventions of the included studies consisted of online modules, learning management systems (LMS), podcasts, and mobile learning (tablets/smartphones). The results of this systematic review showed that medical e-learning interventions in SSA use variable study designs, evaluation methods, and outcome measures when evaluating the effectiveness of interventions. Multiple study designs were used (i.e. mixed methods, qualitative, and quantitative), as well as various combinations of subjective and objective data collection methodologies. Surveys/questionnaires, focus groups, semistructured interviews, observation, usage tracking, document reviews, class grades, and knowledge/skills tests were used by studies to assess outcomes. Often, multiple outcome measures were assessed using diverse terminology and descriptions. The inductive thematic analysis revealed studies mainly assessed the outcome measures of ‘usefulness’, ‘usability’, ‘feasibility, and ‘acceptability’ within e-learning interventions. Studies were generally of ‘low to medium’ quality with an overall serious risk of bias in the measurement of outcome domain. Issues were identified with the validity of measurement instruments and subjective study designs. Various aspects of the results show similarity to previous literature, however differences remain. Although various methods of evaluation have been used or proposed, there was no standardized method of measuring the effectiveness of e-learning in either the literature or the included studies.

Conclusions and Recommendations: Currently there is no consensus on how effectiveness of asynchronous e-learning in medical education in SSA should be evaluated. The major outcome measures and evaluation methodologies as identified in this review serve as a valuable resource for researchers and evaluators designing evaluations of medical e-learning in SSA, and as a step towards the formulation of an appropriate framework for evaluations. Moving forward, the use of a framework for evaluations is highly recommended which is adapted to the context of medical e-learning in SSA with a clear definition of ‘effectiveness’ and links to broader health care outcomes and stakeholder needs. Improved reporting of e-learning evaluations to enhance the transparency and comparability of interventions is also recommended. Further research should focus on in-depth analysis of the relative weights of outcome measures on overall ‘effectiveness’ in the context of SSA, measuring the effect of e-learning interventions on the broader health system outcomes, and further systematic reviews comparing various e-learning modalities/delivery types with evaluation methods, both within and between contexts.