Herein lies the external evaluation results of the Smart Paper Technology Solution (aka MyChild Solution) in The Gambia which has assessed data quality, efficiency gains, operational costs, and the users’ experiences and perceptions associated with the intervention.
In 2017, Shifo Foundation (Shifo), the Ministry of Health & Social Welfare (MoH&SW) of The Gambia, Gavi, the Vaccine Alliance, Action Aid International The Gambia, IKARE, Swedish Postcode Foundation, and Jochnick Foundation started working together to implement The Smart Paper Technology Solution (SPT) in The Gambia. To date, the SPT is implemented in the Western Regions 1 and Western Region 2, two out of seven regions in The Gambia. The real-time progress of work in The Gambia can be seen here.
Since the inception of the programme in The Gambia, partners have identified key criteria to evaluate the effects of the programme and make informed decisions. This article summarises the results of the external evaluation conducted by Umeå University in May of 2018. The goal of the external evaluation was to assess four main attributes of the SPT in The Gambia, including 1) the data quality, 2) the annual operating costs, 3) the time efficiency gains for frontline health workers and 4) the users’ experiences and perceptions associated with the intervention.
Data Quality Assessment
The WHO Data Quality Review Toolkit was used as the framework to asses the data quality of the SPT over the course of three months. Data quality dimensions including 1) completeness and timeliness, 2) internal consistency and 3) external consistency were measured with this toolkit. The results indicated that completeness and timeliness of health facility reports were at 100%. The data was found to be internally and externally consistent. Moreover, the consistency between the reported data and the original records was at 99,95%. Lastly, the recording errors were between 0.7% to 1.5%. Therefore, the SPT scored highly in all of the data quality dimensions and it was concluded to generate high-quality data.
Administrative Time Efficiency
With the SPT, the frontline health workers’ administration time was reduced by 60% for each child who was fully immunised, receiving Vitamin A and deworming supplements.
Annual Operating Costs
The operating costs were measured by comparing the annual national cost of the current HMIS forms with the SPT in two different scenarios: excluding and including the monetary value of reduced time which is achieved with the SPT. When the monetary value of reducing the administration time with the SPT was excluded, the SPT (11,675.95 USD) was more expensive than the current HMIS (8,792.54 USD). When the monetary value of reducing the administration time was included, the SPT (3,944.10 USD) was cheaper than the current HMIS (8,792.54 USD).
Users’ Experiences and Perceptions
Interviews were conducted with frontline health workers and the regional health directorate staff to assess their perceptions of the SPT. The interviewees expressed how the SPT was user-friendly, how this solution delivered benefits to their health facility, and how the workload was less with the SPT when it was compared to the current HMIS. The health workers acknowledged the value of SMS messages sent to parents which remind them of vaccination due dates. The health workers also confirmed the importance of improving data quality. However, the interviewees also expressed how the initial registration of patients was time-consuming during the introduction of the intervention. The health workers also recommended that other health facilities adopt the SPT. In conclusion, the participants’ perceptions of the SPT were overall positive except for the initial registration of patients.
The external evaluators provided a number of recommendations such as better planning and projection of the necessary human resources required for the initial registration of children. They also recommended the MoH&SW staff to work more closely with data verification. It may also be beneficial to return the Smart Paper Forms to health facilities after they have been scanned for the purposes of increased ownership. The evaluators expressed that scaling-up the SPT would be beneficial and they recommend it.
The full reports and additional information about the Smart Paper Technology Solution (aka MyChild Solution) can be accessed using these links below:
- Assessment of MyChild Solution in The Gambia: Data quality, administrative time efficiency, operation costs, and users’ experiences and perceptions:https://bit.ly/2U14OGq
- About Smart Paper Technology Solution: https://shifo.org/en/solution/
- Project progress in The Gambia: https://shifo.org/en/work/gambia/