Making Sense of Post New Vaccines Introduction Evaluations

Background

A post-introduction evaluation (PIE) according to the WHO manual [1] is the assessment of the overall impact of the introduction of a new vaccine(s) on a country’s national immunization programme. It focuses on a range of programmatic aspects, such as pre-introduction planning, vaccine storage and wastage, logistics of administering the vaccine, and community receptiveness to the vaccine. It rapidly identifies problem areas needing correction within the immunization programme either pre-existing or resulting from the introduction of a new vaccine, and provides valuable lessons for future vaccine introductions. WHO recommends that all countries which have introduced a new vaccine conduct a PIE. WHO has developed a comprehensive manual and to tool to guide country teams to conduct PIE effectively. The manual suggests that, a PIE should be done between 6 and 12 Opinion months after introduction of the new vaccine. The PIE methodology consists of using questionnaires, checklists, observation of practices and recording reviews, at all levels of the health service, including cold and dry vaccine storage areas and points of vaccine administration, such as health facilities. I have been involved in four PIE activities; two in Ghana [2] Eritrea [3] and Nigeria [4]. The process at each level – national, regional, district and the health centre took 2-3 hours administering a questionnaire and observing practical activities. Key activities conducted were (i) administration of questionnaire as already said, (ii) physical inspection of the cold chain especially vaccine stock, temperature monitoring, records keeping, waste disposal sites (incinerators and pits); (iii) observation of practical immunization sessions and exit interviews with care givers (iv) preparation of comprehensive report with recommended actions to improve challenges and gaps in the immunization programme of each country. This script is not intended to undermine the authenticity and modalities of the PIE structure but to enhance the achievements of the purpose and objectives of it.

Introduction Pathway

Ghana introduced measles second dose (MSD), pneumococcal conjugate vaccine (PCV) and rotavirus (Rota) vaccine in 2012 and conducted the PIE in 2013. Ghana will introduce Meningococcal A conjugate vaccine (Men A) in November 2016. Eritrea introduced MSD and Rota in 2014 and conducted PIE in February 2015 and planned to introduced PCV and inactivated Polio vaccine (IPV) in July same year (2015). Nigeria introduced IPV in 2015 and started introduction of PCV-10 in some States as a start, and conducted PIE in March 2016. Nigeria will continue with the introduction of PCV in the remaining states. Tables 1-3 are some of the recommendations from the PIE conducted in Ghana (2013); Eritrea (2015) and Nigeria (2016).

manual should be put in a document of its own and disseminated.

• Provide written guidelines to Districts and HFs; HCW should remember to sensitize caregivers and mothers about possible adverse events
• Encourage community and HF to report AEF

1 Training

2 Performance

Monitoring

3 Vaccine Management

4 Cold Chain

• States and Local Government Authorities (LGAs) in collaboration with local partners should conduct refresher trainings for immunization on the identified gaps. In case of cascade trainings, step down trainings should be followed up;
• States and LGAs teams should use monthly LGAs Immunization Officers and healthcare worker meetings as opportunity to train healthcare workers on data management, immunization formulae and archiving of routine immunizationdocuments;

• National Public Health and Child Development Authority (NPHCDA) should review and develop a monitoring chart that includes more than one antigen (in case for antigens with less than 3 doses).
• State, LGA and health facilities should develop PCV-10 and IPV monitoring charts including any other new vaccines introduced;

• NPHCDA in collaboration with partners should procure sufficient fridge and freeze tags;
• States and LGAs should facilitate distribution of these temperature monitoring devices to all health facilities with vaccines refrigerators;

• States and LGAs with support from NPHCDA should carry out bi-annual cold chain assessment, establish the number of the broken down equipment and repair accordingly. The assessment should also provide data for replacement of aged/obsolete equipment.
• All LGAs should have cold and dry stores for the storage of vaccines and injection materials respectively;

• All States, LGAs and Health facilities using electricity as a source of power to run refrigerators, should have a backup source in case of power outage.

6 AEFI

7 Waste management

• States and LGAs should conduct clinical sensitizations on AEFI and ensure revitalization of AEFI committees. Prior to sensitization, States distribute AEFI booklets/protocols/kits to all LGAs and subsequently to all facilities.
• NPHCDA should consider to incorporate the AEFI reporting as a key indicator to be reported for performance based financing schemes
• NPHCDA should finalize review of waste management policy/policy guidelines and disseminate to the States and LGAs for implementation.
• All states should install incinerators and conduct regular on site supportive supervision by management to ensure incinerators are well operated and maintained.

Discussion

Whilst contemplating on how to make sense (effective and efficient) of PIE, the following issues came up for discussion and are worthy of consideration. Should there be PIE in a country for every new vaccine introduced –This may not be necessary because there would have been enough information from the last PIE to guide the introduction of the new vaccine. For example as these three countries have conducted PIE for any of these new vaccines: Rota, MSD PCV or IPV then there should be no need when the same countries introduce Men a vaccine using the same rigorous PIE process. A simplified questionnaire focusing simply on cold chain expansion, training, social mobilization, AEFI monitoring, recording tools is enough. Issues like coverage, wastage, dropout etc and other detailed information as before may not be relevant. Should PIE for same vaccine be conducted in every country–This may also not be necessary because previous report in one country can be used to guide the introduction of the vaccine in another country. There is the need to share PIE reports from neighbouring countries planning to introduce same vaccines. For example if country A has conducted PIE following introduction of PCV and country B is also planning to introduce same vaccine (PCV), then the PIE report from country A should guide country B to minimize the weaknesses observed in country A. This also suggests that reports from Ghana, Eritrea and Nigeria and other PIEs should facilitate effective introduction of new vaccines in other countries.Rigorous PIE may not be required but a more flexible tool as suggested above may then be required insome of these countries following experiences that are available. Should the PIE tool be maintained as it is?–The current PIE tool uses close to three hours to conduct an assessment in one facility and this can be very stressful to both the facility team and the evaluators. The Tool, as it is now, is overloaded with repeated and overlapping questions. The specifics cannot be explained in this script but a technical committee to review and simplify it would be more useful to all parties. Experience from the PIEs so far conducted shows that some of the questions can be taken out. It appears PIE is gradually replacing comprehensive EPI Review. To modify PIE to cover issues required in traditional EPI Review is a good innovation if so then the arguments above hold that PIE in its current form should not be conducted repeatedly and frequently in one country. What happens after PIE – Some ideas for post PIE are here for discussion. (i) Countries need to develop follow up improvement plans to address the gaps identified by the PIE. (ii) HQ/AFRO /IST must share PIE reports with countries especially reports that will enhance effective introduction of particular vaccines as indicated above. (iii) HQ/AFRO/IST must have post PIE assessment plan to follow up on countries improvement plans. This should be a very simple checklist taking not more than an hour to conduct at the facility to encourage countries to make sure most of the recommendations from the PIE are addressed.

Conclusion

PIE is a relevant component of the new vaccine introduction process and must be made to remain relevant and useful to countries. It reveals the strengths and weaknesses of the immunization programme in the country. Its objectives are to: a. Document the process of introduction of new vaccine(s) into the immunization program.

b. Identify operational, logistical and environmental challenges associated with the introduction of the Vaccine (s) and provide recommendations to address them c. Inform future program planning and implementation in the same setting, or in other Geographic areas/ countries from the lessons drawn from the PIE.

To achieve these objectives, there is need for:

a. post PIE follow-ups,

b. review and simplification of the current PIE tool and c. A look at the number of PIEs one country can conduct irrespective of how many times a new vaccine is introduced in the country. These will enhance the achievements of the purpose and objectives of PIE.