SciELO - Scientific Electronic Library Online

 
vol.92 issue3Compulsory patent licensing and local drug manufacturing capacity in AfricaDisability weights in the global burden of disease 2010 study: two steps forward, one step back? author indexsubject indexarticles search
Home Page  

Services on Demand

Journal

Article

Indicators

Related links

  • Have no similar articlesSimilars in SciELO

Share


Bulletin of the World Health Organization

Print version ISSN 0042-9686

Bull World Health Organ vol.92 n.3 Genebra Mar. 2014  Epub Nov 27, 2013

http://dx.doi.org/10.2471/BLT.13.122564 

Lessons from the Field

Monitoring polio supplementary immunization activities using an automated short text messaging system in Karachi, Pakistan

Suivi des campagnes de vaccination supplémentaire contre la poliomyélite en utilisant un système de messages courts automatiques à Karachi, au Pakistan

Seguimiento de las actividades complementarias de inmunización contra la poliomielitis mediante un sistema automático de mensajes de texto cortos en Karachi, Pakistán

رصد أنشطة التمنيع التكميلية ضد مرض شلل الأطفال باستخدام نظام الرسائل النصية القصيرة الآلي في كراتشي، باكستان

巴基斯坦卡拉奇使用自动短信系统监测小儿麻疹症强化免疫活动

Автоматизированная система отправки коротких текстовых сообщений и мониторинг дополнительных мероприятий по иммунизации от полиомиелита в Карачи (Пакистан)

AM Kazia 

A Murtazaa 

S Khojab 

AK Zaidia 

SA Alia 

aDepartment of Paediatrics and Child Health, The Aga Khan University, Stadium Road, PO Box 3500, Karachi 74800, Pakistan

bDepartment of Community Health Sciences, The Aga Khan University, Karachi, Pakistan

ABSTRACT

Problem

Polio remains endemic in many areas of Pakistan, including large urban centres such as Karachi.

Approach

During each of seven supplementary immunization activities against polio in Karachi, mobile phone numbers of the caregivers of a random sample of eligible children were obtained. A computer-based system was developed to send two questions – as short message service (SMS) texts – automatically to each number after the immunization activity: “Did the vaccinator visit your house?” and “Did the enrolled child in your household receive oral polio vaccine?” Persistent non-responders were phoned directly by an investigator.

Local setting

A cluster sampling technique was used to select representative samples of the caregivers of young children in Karachi in general and of such caregivers in three of the six “high-risk” districts of the city where polio cases were detected in 2011.

Relevant changes

In most of the supplementary immunization activities investigated, vaccine coverages estimated using the SMS system were very similar to those estimated by interviewing by phone those caregivers who never responded to the SMS messages. In the high-risk districts investigated, coverages estimated using the SMS system were also similar to those recorded – using lot quality assurance sampling – by the World Health Organization.

Lessons learnt

For the monitoring of coverage in supplementary immunization activities, automated SMS-based systems appear to be an attractive and relatively inexpensive option. Further research is needed to determine if coverage data collected by SMS-based systems provide estimates that are sufficiently accurate. Such systems may be useful in other large-scale immunization campaigns.

Résumé

Problème

La poliomyélite reste endémique dans de nombreuses régions du Pakistan, y compris dans les grands centres urbains tels que Karachi.

Approche

Pendant chacune des sept campagnes de vaccination supplémentaire contre la poliomyélite à Karachi, les numéros de téléphone portable des aidants familiaux d'un échantillon aléatoire d'enfants éligibles ont été obtenus. Un système informatique a été développé pour envoyer automatiquement deux questions – sous forme de messages courts (SMS) – à chaque numéro après la campagne de vaccination: «L'agent de vaccination vous a-t-il rendu visite?» et «L'enfant inscrit dans votre foyer a-t-il reçu le vaccin antipolyomélite oral?» Les personnes qui persistaient à ne pas répondre aux messages ont été appelées directement par un enquêteur.

Environnement local

Une technique d'échantillonnage par grappes a été utilisée pour sélectionner les échantillons représentatifs d'aidants familiaux de jeunes enfants à Karachi en général et des échantillons représentatifs de tels aidants familiaux dans trois des six districts «à haut risque» de la ville, où des cas de poliomyélites avaient été dépistés en 2011.

Changements significatifs

Dans la plupart des campagnes de vaccination supplémentaire étudiées, la couverture vaccinale estimée en utilisant le système de SMS était très similaire à celle estimée en interrogeant les aidants familiaux qui n'avaient jamais répondu aux messages SMS. Dans les districts à haut risque étudiés, la couverture estimée en utilisant le système de SMS était également similaire à celle rapportée – en utilisant la méthode de sondage par lots appliquée à l'assurance qualité – par l'Organisation mondiale de la Santé.

Leçons tirées

Pour le suivi de la couverture des campagnes de vaccination supplémentaire, les systèmes de SMS automatiques semblent être une option intéressante et relativement bon marché. Des recherches plus approfondies sont nécessaires pour déterminer si les données de la couverture recueillies par les systèmes de SMS fournissent des estimations suffisamment précises. De tels systèmes peuvent être utiles dans d'autres campagnes de vaccination à grande échelle.

Resumen

Situación

La poliomielitis sigue siendo una enfermedad endémica en muchas zonas de Pakistán, incluyendo grandes centros urbanos como Karachi.

Enfoque

Durante cada una de las siete actividades complementarias de inmunización contra la poliomielitis en Karachi, se obtuvieron los números de teléfono móvil de los cuidadores de una muestra aleatoria de niños elegibles. Se desarrolló un sistema computarizado para enviar dos preguntas, como mensajes del servicio de mensajes cortos (SMS), de forma automática a cada número tras la actividad de inmunización: «¿El vacunador visitó su casa?» y «¿El niño registrado de su hogar ha recibido la vacuna oral contra la poliomielitis?» Un investigador llamó directamente a quienes no contestaban en ninguna ocasión.

Marco regional

Se utilizó una técnica de muestreo por conglomerados para seleccionar las muestras representativas de los cuidadores de niños pequeños en Karachi, en general, y de dichos cuidadores en tres de los seis distritos de «alto riesgo» de la ciudad, donde se detectaron casos de poliomielitis en 2011.

Cambios importantes

En la mayoría de las actividades complementarias de inmunización investigadas, las coberturas de vacunas que se estimaron por el sistema de SMS fueron muy similares a las estimadas mediante entrevistas telefónicas a los cuidadores que nunca respondían a los mensajes SMS. En los distritos de alto riesgo investigados, las coberturas que se estimaron por el sistema de SMS también fueron similares a las que registró la Organización Mundial de la Salud, mediante un muestreo de garantía de calidad de los lotes.

Lecciones aprendidas

Los sistemas basados en SMS automatizados parecen ser una opción atractiva y relativamente económica para el seguimiento de la cobertura de las actividades de inmunización complementarias. Se necesitan más investigaciones para determinar si los datos de cobertura recogidos por los sistemas basados en SMS ofrecen estimaciones lo bastante precisas. Estos sistemas pueden ser útiles en otras campañas de inmunización a gran escala.

ملخص

المشكلة

لا يزال مرض شلل الأطفال متوطناً في العديد من المناطق في باكستان، بما في ذلك المراكز الحضرية الكبيرة مثل كراتشي.

الأسلوب

أثناء كل نشاط من أنشطة التمنيع التكميلية السبعة ضد مرض شلل الأطفال في كراتشي، تم الحصول على أرقام الهواتف الجوالة لمقدمي الرعاية لعينة عشوائية من الأطفال المؤهلين. وتم إنشاء نظام حاسوبي لإرسال سؤالين – في شكل رسائل نصية قصيرة (SMS) – تلقائياً إلى كل رقم بعد نشاط التمنيع: "هل قام عامل التطعيم بزيارة منزلكم؟" و "هل تلقى الطفل المسجل في منزلكم اللقاح الفموي ضد مرض شلل الأطفال؟" وتلقى من لم يقوموا بالرد بشكل متواصل اتصالًا هاتفياً مباشراً من الباحث.

المواقع

المحلية تم استخدام طريقة أخذ العينات الجماعية لتحديد عينات ممثلة لمقدمي الرعاية للأطفال الصغار في كراتشي بوجه عام، ولمقدمي الرعاية هؤلاء في ثلاث مناطق من بين الست "عالية المخاطر" في المدينة حيث اكتشفت حالات الإصابة بمرض شلل الأطفال في عام 2011.

التغيرات

ذات الصلة في معظم أنشطة التمنيع التكميلية التي تم البحث فيها، كانت تغطية اللقاح المقدرة باستخدام نظام الرسائل النصية القصيرة مشابهة للغاية لتلك المقدرة عن طريق المقابلات الهاتفية مع مقدمي الرعاية الذين لم يستجيبوا مطلقًاً للرسائل النصية القصيرة. وفي المناطق عالية المخاطر التي تم البحث فيها، تشابهت التغطية المقدرة باستخدام نظام الرسائل النصية القصيرة مع تلك التي تم تسجيلها - باستخدام العينات العشوائية لضمان الجودة - بواسطة منظمة الصحة العالمية.

الدروس

المستفادة لرصد التغطية في أنشطة التمنيع التكميلية، يبدو أن أنظمة الرسائل النصية القصيرة الآلية خيار رائع وغير مكلف نسبياً. وثمة حاجة إلى أبحاث إضافية لتحديد ما إذا كانت بيانات التغطية التي تم جمعها بواسطة أنظمة الرسائل النصية القصيرة تقدم تقديرات دقيقة بالقدر الكافي. وقد تكون هذه الأنظمة مفيدة في حملات التمنيع واسعة النطاق.

摘要

问题

小儿麻痹症依然是巴基斯坦很多地区的地方病,卡拉奇这样的大城市也不例外。

方法

在卡拉奇针对小儿麻痹症的七次强化免疫活动中,每次都随机抽取符合条件儿童,获得其照顾者的手机号码。开发基于计算机的系统,在免疫活动之后以短信服务(SMS)文本的方式向每个号码自动发送两个问题:“预防接种人员到过您家了吗?”和“您家里参与接种的儿童吃了口服小儿麻痹症疫苗了吗?”。如果一直没有回答,则由调查人员直接打电话。

当地状况

使用集群抽样方法选择卡拉奇的幼儿照顾者的代表性样本,包括该市一般性幼儿照顾者样本和2011年检测出小儿麻痹症病例的六个“高风险”区中三个区的幼儿照顾者样本。

相关变化

在研究的大多数强化免疫活动中,使用SMS系统估计的疫苗覆盖率和使用通过电话问询从不回复SMS短信的人群的方法而估计的覆盖率非常相似。在调查的高风险区域,使用SMS系统估计的覆盖率和世卫组织使用批次质量保证抽样所记录的覆盖率也很相近。

经验教训

对于强化免疫活动的覆盖率监控,基于短信的系统看起来是更有吸引力且相对便宜的选择。需要进一步研究确定基于SMS系统收集的覆盖率数据是否足够准确。这类系统可能在其他大规模免疫活动中发挥作用。

Резюме

Проблема

Полиомиелит остается эндемичным во многих регионах Пакистана, включая такие крупные городские центры как Карачи.

Подход

В ходе проведения каждого из семи дополнительных мероприятий по иммунизации от полиомиелита в Карачи посредством случайной выборки были получены номера мобильных телефонов опекунов детей, подлежащих иммунизации. Была разработана автоматизированная система для автоматической отправки двух вопросов – в виде коротких текстовых сообщений (СМС) – на каждый номер после проведения иммунизации: “Посещал ли вакцинатор ваш дом?” и “Получал ли подлежащий иммунизации ребенок оральную полиомиелитную вакцину?” Респонденты, не отвечавшие на сообщения, опрашивались исследователем непосредственно по телефону.

Местные условия

Метод групповой выборки применялся для выбора репрезентативных групп опекунов детей младшего возраста в Карачи в целом и таких же опекунов в трех из шести городских районов высокого риска, где в 20011 г. были зарегистрированы случаи заболевания полиомиелитом.

Осуществленные перемены

В большинстве проведенных дополнительных мероприятий по иммунизации охват населения прививками, определенный с помощью СМС-сообщений, был очень схож с аналогичными данными, полученными посредством телефонного опроса респондентов, не отвечавших на СМС-сообщения. В исследованных городских районах высокого риска охват населения прививками, определенный с помощью СМС-сообщений, также был схож с данными, полученными Всемирной организацией здравоохранения посредством выборки для гарантированного контроля качества.

Выводы

Для мониторинга охвата населения прививками при проведении дополнительных мероприятий по иммунизации система отправки СМС-сообщений оказывается привлекательным и относительно недорогим вариантом. Необходимо проведение дальнейших исследований для определения точности оценок, основанных на данных по охвату прививками, полученных с помощью системы отправки СМС-сообщений. Подобные системы могут быть использованы в других крупномасштабных кампаниях по иммунизации.

Background

Pakistan is one of only three countries where poliovirus remains endemic despite extensive elimination efforts.1 Karachi, Pakistan's largest city, is now the world's only metropolitan area that has not been able to interrupt poliovirus transmission. Nine cases of poliomyelitis were confirmed in Karachi during 2011 and these cases were reported from six of the 18 “towns” or districts that form the metropolitan area.2 Although in 2012 no cases of polio were identified in Karachi, poliovirus was found in samples of the city's sewer water.1 The first case of polio identified in Pakistan in 2013 was in a child in Karachi.3

Problem

The use of supplementary immunization activities (SIAs) has helped several countries increase their immunization coverage against polio and has played a major role in decreasing the global incidence of polio by more than 99% since 1988.4 In Karachi, however, more than 100 such activities based on oral polio vaccine have failed to interrupt local poliovirus transmission.5 A major reason for this failure is poor vaccine coverage among vaccination-eligible children.6 The current system of monitoring the coverage of each SIA in Pakistan has several limitations. In this system, the children receiving vaccine during an SIA are marked with black “permanent” ink on their right little fingers.7 A third-party evaluation team then visits a randomly selected sample of households in the target area and counts the number of eligible children – that is, children aged less than 5 years – and the number of children who have black ink on their right little fingers.8 This monitoring system is resource-intensive, utilizes convenience sampling and is prone to misreporting bias since the same individuals often supervise both the vaccinations and the evaluation of vaccine coverage.9 It also exposes the members of the evaluation teams to the risk of violence, especially in certain locations with poor security. More than 16 vaccinators associated with polio vaccination campaigns have been assassinated in various parts of Pakistan since December 2012.10

In Pakistan, mobile phones have become very common over the past 10 years and there are currently more than 119 million mobile phone users.11 Many of these phones are used to send short message service (SMS) texts. In 2011, for example, 237.58 billion person-to-person texts – the equivalent of about 175 text messages per mobile phone per month – were generated in Pakistan.12 Mobile health (“m-health”) technology is increasingly being considered for various health interventions, such as sending reminders for clinic visits, immunizations and medication adherence.13 We recently wondered if m-health could be used to measure vaccine coverage in polio SIAs, particularly in resource-limited areas and those with poor security. We therefore conducted a feasibility study to assess if a low-cost automated system that could send SMS messages to the mobile phones of caregivers could be used to monitor the coverage of SIAs in Karachi.

Approach

The study protocol was approved by the ethical review committees of the Aga Khan University in Pakistan and the World Health Organization (WHO). Thirty clusters of households were randomly selected in each of three districts of Karachi – Gulshan-e-Iqbal, Baldia and SITE (Sindh Industrial and Trading Estate) – and across the entire city. A case of polio had been detected in each of the three study districts in 2011.2 Each cluster consisted of 200 to 250 households.8 Within each selected cluster, at least 28 households with a member aged less than 5 years were selected by systematic random sampling. If a selected house contained more than one child younger than 5 years, then one of the children was randomly chosen for enrolment. After informed consent was obtained from a caregiver, baseline demographic information on each enrolled child, the mobile phone number of one of the child's caregivers and the phone-owning caregiver's preferred language were recorded. The phone owners were told that they might be contacted – either via SMS texts or a direct phone call – after one or more forthcoming SIAs and that, if contacted, they would be asked two questions. Although the questions were basically “In the last SIA, did the vaccinator visit your house?” and “In the last SIA, did [named of enrolled child] receive oral polio vaccine?”, they – or their translation into the preferred languages of the caregivers – were slightly revised during the study to improve the probability of an adequate response. The revisions of the questions were based on feedback from the caregivers who had been asked the questions.

After each of seven SIAs run in Karachi, seven households were randomly selected from the 28 originally identified in each study cluster. The caregivers in these households were sent the two questions as SMS texts. Those who did not answer the text messages were contacted via direct phone calls and asked the same questions by an investigator. Coverage data obtained via text messages and phone calls were compared with the coverage data obtained – in the same SIAs – by WHO, using conventional lot quality assurance sampling.14

The SMS texts were sent automatically – and relatively cheaply – by a server-based software package developed specifically for our study. The two questions were sent as two separate texts, with a request for the caregiver to reply to each text with the letter “P” followed by a single digit: 1 if the answer was “yes”, 2 if the answer was “no” and 3 if the answer was “do not know”. As an incentive, 20 Pakistani rupees – equivalent to about 0.20 United States dollars – were credited to a caregiver's mobile phone each time the caregiver responded to an automated message. All questions – whether sent as texts or spoken over the phone – were posed in the caregiver's preferred language.

If no response to a text had been received within 24 hours, repeat messages were sent daily for up to 1 week. If there had been no reply to a question sent as an SMS text after 1 week, an investigator attempted to speak by phone to the caregiver involved, to obtain an answer to the question that had been posed and to determine why the caregiver had not sent a reply to the SMS message.

Results

Seven SIAs conducted in Karachi – six in 2012 and one in 2013– were monitored using the SMS-based system. Of the 840 caregivers who were sent the messages in each SIA, the median proportions who responded to the first SMS text – “In the last SIA, did the vaccinator visit your house?” – and to the second text – “In the last SIA, did [named of enrolled child] receive oral polio vaccine?” – were 23% (interquartile range, IQR: 19–30) and 14% (IQR: 10−21), respectively. Of the 4404 caregivers who were sent SMS messages but responded to none of them, 56% (IQR: 54–58) were successfully spoken to by an investigator, by phone. We failed to get any response – by text or phone call – from 1831 (32%) of the caregivers sent SMS messages: four refused to talk to an investigator, 208 never answered the phone when called at least three times by an investigator and 1619 could not be reached because they had changed their numbers, their phones were faulty, uncharged or otherwise unusable or, perhaps, because their phone numbers had been recorded incorrectly.

In the sample representing the entire city of Karachi, the lowest vaccine coverage was detected during the SIA in June 2012, when 59% of the caregivers who were sent SMS messages and successfully contacted said that a vaccinator had visited their household and the same proportion said that the enrolled child had received a vaccine dose. The highest coverage recorded in the sample representing the entire city was observed slightly earlier – in the SIA in April 2012 – when the corresponding proportions were 83% and 82%, respectively. The median coverage revealed for the entire city by the responses of the caregivers sent SMS messages was 74.5% (95% confidence interval, CI: 71.6–77.4).

Using the responses of the caregivers in the district-level samples who were sent SMS messages and successfully contacted, the median coverages by vaccinators were estimated to be 71% (95% CI: 68.2–73.8) in Gulshan-e-Iqbal, 79% (95% CI: 76.4–81.6) in Baldia and 81% (95% CI: 78.6–83.5) in SITE. The corresponding vaccine coverages among the enrolled children were 67% (95% CI: 63.9–70.1), 75% (95% CI: 72.1–78.0) and 78% (95% CI: 75.2–80.8), respectively.

In general, the data collected during phone calls to caregivers who had not responded to SMS messages indicated levels of vaccine coverage similar to those estimated from the responses to SMS messages (Table 1). However, the overall response rate to phone calls made by investigators was almost twice as high as the corresponding response rate to SMS messages (44% versus 23%). In the high-risk districts that were investigated, the vaccine coverages estimated by WHO – using clustered lot quality assurance sampling – were slightly higher than the estimates made using the SMS system (Table 1).

Table 1 Vaccine coverages estimated by three methods, Karachi, Pakistan, 2012–2013 

Study areas and date of SIA n a Estimated vaccine coverage,b % (95% CI)
SMS LQASc PCd
SITEe
March 2012 9/–/135 78 (51–100) ND 93 (89–97)
April 2012 21/–/81 95 (86–100) ND 93 (87–99)
June 2012 22/150/96 91 (79–100) 92 (88–96) 83 (75–91)
July 2012 24/–/87 83 (68–98) ND 90 (84–96)
September 2012 77/–/61 29 (15–43) ND 56 (44–68)
October 2012 37/50/68 86 (75–97) 82 (71–93) 82 (73–91)
April 2013 20/–/75 65 (44–85) ND 56 (45–67)
Baldia
March 2012 8/150/138 63 (30–96) 93 (89–97) 93 (89–97)
April 2012 14/200/77 93 (80–100) 92 (88–96) 96 (92–100)
June 2012 14/150/85 86 (68–100) 94 (92–98) 87 (80–94)
July 2012 18/–/69 94 (83–100) ND 91 (84–98)
September 2012 33/–/66 73 (58–88) ND 82 (73–91)
October 2012 31/50/75 97 (91–100) 88 (79–97) 88 (81–95)
April 2013 29/–/73 52 (34–70) ND 56 (45–68)
Gulshan-e-Iqbal
March 2012 15/100/108 60 (51–69) 91 (85–97) 80 (72–88)
April 2012 22/200/92 68 (49–87) 89 (84–93) 83 (75–91)
June 2012 38/150/77 82 (70–94) 89 (84–93) 81 (72–90)
July 2012 32/–/82 78 (64–92) ND 82 (74–90)
September 2012 46/–/51 61 (48–74) ND 76 (64–88)
October 2012 57/50/68 60 (47–73) 92 (84–100) 84 (75–93)
April 2013 32/–/73 72 (56–88) ND 66 (55–77)
Entire city of Karachi
March 2012 –/–/142 8 (0–21) ND 77 (70–84)
April 2012 24/–/84 92 (81–100) ND 89 (83–96)
June 2012 26/–/83 46 (27–65) ND 72 (62–82)
July 2012 35/–/82 86 (75–98) ND 89 (82–96)
September 2012 53/–/71 26 (14–38) ND 49 (37–60)
October 2012 43/–/72 84 (73–95) ND 85 (76–93)
April 2013 –/–/– ND ND ND

CI, confidence interval; LQAS, lot quality assurance sampling; ND, not determined; PC, phone calls; SIA, supplementary immunization activity; SMS, short message service.

a Numbers of households investigated using the SMS system/LQAS/PC that provided adequate data. We excluded the households of caregivers who claimed not to know if the eligible children in their care had been vaccinated.

b Estimated percentage of children younger than 5 years who received oral polio vaccine in each immunization activity.

c Data collected by the World Health Organization.14

d Data collected in phone calls to the caregivers who did not respond to SMS messages.

e Sindh Industrial and Trading Estate.

The major reasons given by caregivers for not responding to an SMS message were that they were too busy (36%), not interested in replying (32%) or unable to read the message (20%). The most common time of day for a caregiver to reply to an SMS message was the evening – between 18:00 and 22:00. Overall, 80% of the replies to SMS messages were received within three days of the messages being sent. Of the caregivers who responded to an SMS message asking if the enrolled child had received oral polio vaccine in the last SIA, 6% answered that they did not know.

Discussion

We were able to monitor the vaccine coverages achieved in SIAs in Karachi by using a low-cost, automated SMS-based system – augmented with follow-up phone calls to persistent non-responders. To the best of our knowledge, this study is the first to evaluate the effectiveness of mobile phones and automated SMS messages in monitoring the quality of polio SIAs.

The rate of response to the first SMS message that we sent out to caregivers was disappointing but it soon became clear that many of the targeted caregivers had simply not received the message because of problems with the modems then being used. The subsequent addition of a commercial SMS gateway (Sidat Hyder, Karachi, Pakistan) to the server-based SMS system substantially improved the percentage of targeted caregivers who received the SMS message that we had intended to send to them (data not shown). The rate of response to our automated SMS messages gradually increased over time, as we improved the format and language of the messages. It would be relatively easy to scale up our SMS-based system to cover SIAs or other vaccine campaigns at the provincial or country level. However, there may not be sufficient human resources to permit follow-up phone calls to be made to all of the non-responders in such large-scale campaigns.

Automated SMS-based monitoring allows for the separation of the vaccine delivery and monitoring teams. It eliminates errors made by human data collectors, reporting bias by field teams – who may exaggerate coverage to improve their apparent performance – and the risk of data collectors being attacked. The latter advantage makes SMS-based monitoring particularly valuable in security-compromised settings.10,15 If an SMS-based system is implemented, the human resources previously occupied with monitoring can focus exclusively on the delivery of vaccines.

In general, the methods that we and WHO used to estimate vaccine coverage in SIAs in Karachi gave similar results. However, there were some discrepancies. For example, for the SIA in March 2012, the coverage estimated using data collected with the SMS-based system was substantially lower than that estimated using lot quality assurance sampling (Table 1). However, the SMS gateway had not been deployed at this time and many of the caregivers who were sent SMS messages never received them. The overall response rate to our SMS messages was disappointingly low – despite the resending of messages to non-responders and the offer of a financial incentive for replying. It remains possible that the vaccine coverage in the households of caregivers who were never successfully contacted after being sent SMS messages differed from that in the households of responders. When investigators phoned most of the caregivers who were never successfully contacted after being sent SMS messages, they simply received a “number unobtainable” message or tone. It seems likely that many caregivers had changed their mobile phone numbers after the baseline survey. In Pakistan – as in Cambodia16 and other developing countries – it is easy to switch service providers and such providers offer attractive packages to gain new customers.

Another limitation of our study was that many of the caregivers contacted by SMS messages said that they simply did not know if the enrolled child in their household had been vaccinated. In many of these cases, the caregiver was responding while out of contact with the enrolled child and any other of the child's caregivers. This problem might be reduced by educating target populations – so that the caregivers who are to be contacted anticipate the contact and check on vaccinations within their households before that contact is made.

We could compensate for low response rates by increasing the number of caregivers sent SMS messages but if there is a systematic difference in the baseline characteristics of responders and non-responders, or of those who do and do not own a mobile phone, a larger sample will not reduce the risk of non-response bias. The use of mobile phones is now very common in Pakistan but it is not universal. Caregivers who are resistant to vaccinating children – who are important in the interruption of polio transmission – may well be unlikely to share their mobile phone numbers with those tasked with assessing vaccine coverage.

Although we collected mobile phone numbers and located children who were eligible for polio vaccination through household visits, the same information may become available online in the future. For example, if local laws, rules and regulations permit it, the phone numbers of potential caregivers could be collected from service providers and the identities and addresses of eligible children could be obtained from health records. Caregivers could be encouraged to provide the same data for inclusion in national registries. Vaccine coverage could then be assessed with very little human intervention.

In conclusion (Box 1), in the monitoring of SIA, SMS-based systems have the advantages of low cost, fairly representative sampling of the population, low burdens on monitoring staff and low risk of the coverage being exaggerated. These advantages make such systems an attractive alternative to more conventional methods of SIA monitoring, in Pakistan and elsewhere.

Summary of main lessons learnt

  • For the monitoring of coverage in supplementary immunization activities, automated systems based on short message service (SMS) texts appear to be an attractive and relatively inexpensive option.

  • Further research is needed to determine if coverage data collected by SMS-based system provide estimates that are sufficiently accurate.

  • Such systems may be useful in other large-scale immunization campaigns.

Acknowledgements

This study was funded by the World Health Organization (grant 2011/178719–0).

References

1.  Ahmed G, ul Fawwad SH. Polio – can it be eradicated from Pakistan? Pediatr Infect Dis J 2013;32:303. doi: http://dx.doi.org/10.1097/INF.0b013e31827db870 PMID:23558327 [ Links ]

2.  Weekly AFP surveillance update, Pakistan, week 52. Islamabad: World Health Organization; 2011. [ Links ]

3.  Weekly AFP surveillance update, Pakistan, week 41. Islamabad: World Health Organization; 2013. [ Links ]

4.  Dietz V, Cutts F. The use of mass campaigns in the expanded program on immunization: a review of reported advantages and disadvantages. Int J Health Serv 1997;27:767–90. doi: http://dx.doi.org/10.2190/QPCQ-FBF8-6ABX-2TB5 PMID:9399118 [ Links ]

5.  Khowaja AR, Khan SA, Nizam N, Omer SB, Zaidi A. Parental perceptions surrounding polio and self-reported non-participation in polio supplementary immunization activities in Karachi, Pakistan: a mixed methods study. Bull World Health Organ 2012;90:822–30. doi: http://dx.doi.org/10.2471/BLT.12.106260 PMID:23226894 [ Links ]

6.  Global polio emergency action plan 2012–2013. Geneva: World Health Organization; 2012. Available from: http://www.polioeradication.org/portals/0/document/resources/strategywork/eap_201205.pdf [accessed 20 October 2013]. [ Links ]

7.  Quoc Cuong H, Schlumberger M, Garba Tchang S, Ould Cheikh D, Savès M, Mallah B et al. Évaluation de la durée de vie des marquages sur l’ongle pendant les vaccinations polio au Tchad Santé 2010;20:143–8. PMID:21118787 [ Links ]

8.  Woodward SH. Description and comparison of the methods of cluster sampling and lot quality assurance sampling to assess immunization coverage. Geneva: World Health Organization; 2001. [ Links ]

9.  Sheikh S, Ali A, Zaidi AK, Agha A, Khowaja A, Allana S et al. Measles susceptibility in children in Karachi, Pakistan. Vaccine 2011;29:3419–23. doi: http://dx.doi.org/10.1016/j.vaccine.2011.02.087 PMID:21396902 [ Links ]

10.  Protecting our public health workers. Nat Rev Microbiol 2013;11:67. doi: http://dx.doi.org/10.1038/nrmicro2967 PMID:23441342 [ Links ]

11.  Nishtar S, Boerma T, Amjad S, Alam AY, Khalid F, ul Haq I et al. Pakistan’s health system: performance and prospects after the 18th Constitutional Amendment. Lancet 2013;381:2193–206. doi: http://dx.doi.org/10.1016/S0140-6736(13)60019-7 PMID:23684254 [ Links ]

12.  Proliferation of SMS & MMS in Pakistan with emphasis on premium rate SMS services: SMS/MMS report 2012. Islamabad: Pakistan Telecommunication Authority; 2012. Available from: http://www.pta.gov.pk/media/sms_mms_report_130313_1.pdf [accessed 4 July 2013]. [ Links ]

13.  Déglise C, Suggs LS, Odermatt P. SMS for disease control in developing countries: a systematic review of mobile health applications. J Telemed Telecare 2012;18:273–81. doi: http://dx.doi.org/10.1258/jtt.2012.110810 PMID:22826375 [ Links ]

14.  Assessing vaccination coverage levels using clustered lot quality assurance sampling: field manual. Geneva: World Health Organization; 2012. Available from: http://www.polioeradication.org/Portals/0/Document/Research/OPVDelivery/LQAS.pdf [accessed 4 July 2013]. [ Links ]

15.  Callaway E. Public health: polio’s moving target. Nature 2013;496:290–2. doi: http://dx.doi.org/10.1038/496290a PMID:23598322 [ Links ]

16.  Bisits Bullen PA. Operational challenges in the Cambodian mhealth revolution. J Mob Tech Med 2013;2:20–3. doi: http://dx.doi.org/10.1258/jtt.2012.110810 PMID:22826375 [ Links ]

Competing interests:

None declared.

Received: March 29, 2013; Revised: September 10, 2013; Accepted: October 14, 2013

Correspondence to AM Kazi (e-mail: momin.kazi@aku.edu).