Analysis of NITI AAYOG (National Institution for Transforming India) Health Index Report on the Ranking of States and Union Territories: Round 1 (2014-2016)-V1

Background/Rationale

India has committed to adopting the Sustainable Development Goals (SDGs) for ending poverty, protecting the planet, and ensuring prosperity for all to be fulfilled by year 2030. Goal 3 of SDGs is about ensuring healthy lives with promoting well-being for all. National Institution for Transforming India- (NITI) Aayog had started the Health Index initiative for achieving desirable health outcomes. India’s improvement in life expectancy, maternal and child mortality, reducing fertility, are falling short on several national and global targets. There are variations across States and Union Territories of India in their health needs and systems performance. NITI Aayog aims to bring change in population health by spirit of co-operative and competitive federalism; NITI Aayog measures the annual performance of States and Union Territories (UTs), and rank States and UTs on the basis of incremental change. Healthy States and union territories can make India able to reap demographic dividend is the key motto. In year 2017 the NITI Aayog with the Ministry of Health and Family Welfare (MoHFW) and the World Bank initiated an annual Health Index for knowing Performance and Incremental Performance across all 36 states and UTs. NITI Aayog has been mandated as the nodal agency responsible for attaining the commitments under the SDGs. It was necessary to develop a tool for measuring outcomes in the health sectors to provide feedback to all stakeholders on what we have set out to achieve, deviations, if any, to be pointed out in time to ensure necessary correction. It is true that summarizing the complexities and condensing it in an Index has limitations. Health Outcomes Index seeks to capture the annual progress of States and Union Territories (UTs) through 3 varieties of indicators – Outcomes, Governance and Processes. The NITI Aayog works in collaboration with the Ministry of Health and Family Welfare, with technical assistance from the World Bank.

Objectives

1. To develop a composite Health Index based on key health indicators.
2. To ensure States’ participation and ownership.
3. Transparency by using an independent validation of data by an independent agency.
4. To generate Health Index scores and rankings for the States and UTs.

Study Design

This novel study was a cross-sectional retrospective observational epidemiological study. The Health Index consists of a set of indicators in the domains of Health Outcomes, Governance and Information, and Key Inputs/ Processes. Health Outcomes are assigned the highest weight, indicators were selected on the basis of their importance and availability of reliable data at least annually from pre- existing data sources such as the Sample Registration System (SRS), Civil Registration System (CRS) and Health Management Information Systems (HMIS). Data on indicators is included for Index calculations only after validation by the IVA. A composite Index is calculated as a weighted average of various indicators, for a base year (BY) and a reference year (RY). The change in the Index score of each State from the base year to a reference year is the annual incremental progress of each State. States and UTs were grouped in 3 categories to ensure comparison among similar entities, namely 21 Larger States, 8 Smaller States, and 7 UTs.

Setting

For calculation of Index values and ranks, data was submitted online and validated by an Independent Validation Agency (IVA). The States were previously sensitized about the process for data submission through workshops and mentor agencies (Table 1). Data was submitted by participants States and UTs through online portal hosted by NITI Aayog and data from pre-existing sources in the public domain was pre-entered. After validation of data by an IVA it was used as an input into automated generation of Index values and ranks on the web-portal. The data was verified by IPE Global, an IVA prior to computing the Index and ranks for all States and UTs of India.

Table1: List of mentor agencies.

This novel study was the first of its kind which was conducted over a period of eighteen months. The World Bank, experts in statistics and health systems, public health, and economics were consulted for the development of the Index. The States and UTs participated for finalization of the indicators/variables, workshops for sharing the methodology, process of data submission.

Participants

All states and UTs of India were participants. Multiple

stakeholders as discussed above contributed to the Index development: The various Index was developed by NITI Aayog with help of World Bank, States and UTs, the Ministry of Health and Family Welfare (MoHFW), domestic and international sector experts and other development partners Categorization of States and UTs for ranking were based on the size, and administration. The States were ranked in three categories, namely Larger States, Smaller States and UTs [1] (Table 2).

This categorization was adopted due to the following reasons: 1. The SRS data on health outcomes (NMR, U5MR, TFR and SRB) were not available for 8 Smaller States and 7 UTs, 2. Reliable estimates for these outcome indicators/ variables based on raw data obtained from SRS for the Smaller States and UTs could not be derived due to statistically small sample size and insufficient number of events. Variables The main criteria for inclusion of indicators/variables were the availability of reliable data with at least an annual frequency. The output Index is a weighted composite Index based on indicators/variables in 3 fields: (1) Health Outcomes; (2) Governance and Information; and (3) Key Inputs/Processes. Each domain was assigned a weight based on its importance. The indicator values are scaled from 0 to 100 for generating composite Index scores and performance rankings for base year (BY) (2014-15) and RY (reference year) (2015-16). The annual incremental progress made from BY to RY is used to generate incremental ranks. Table 3 shows the number of indicators/variables in each domain and sub-domain along with weights, while Table 4 provides the detailed Health Index with indicators/variables, their definitions, data sources, and specifics of base and reference years.

*The data for indicator no. 1.2.6 related to out of pocket expenditure was available only for 2015-16 and hence was used to calculate independently the RY Index and rank. Table 3: Health Index: Summary.

Data Sources/Measurement

The Health Index consists of 24 indicators/variables related to Health Outcomes, Governance and Information, and Key Inputs/Processes Table 4 provides Health Index- indicator details and data sources.

Sub-domain 2.2 – Governance (Weight – 60)

Average occupancy of an officer (in months), combined for following three posts at State level for last three years

1. Principal Secretary
2. Mission Director (NHM)
3. Director (Health Services) Average occupancy of an officer (in months), combined for following posts in last three years:

2.2.1 Average occupancy of a full-time officer (in months) for all the districts in last three years - District Chief Medical Officers (CMOs) or equivalent post (heading District Health Services) Average occupancy of a CMO (in months) for all the districts in last three years. State Report

2.2.2 DOMAIN 3 – KEY INPUTS/PROCESSES

BY: April 1, 2012-March 31, 2015 RY: April 1, 2013-March 31, 2016

State Report

BY: April 1, 2012- March 31, 2015 RY: April 1, 2013-March 31, 2016

Sub-domain 3.1 – Health Systems/Service Delivery (Weight – 200)

Vacant healthcare provider positions in public health facilities against total sanctioned healthcare provider positions for following cadres (separately for each cadre) during a specific year: a. Auxiliary Nurse Mid-wife (ANM) at sub-centres (SCs) b. Staff nurse (SN) at Primary Health Centres (PHCs) and Community Health Centres (CHCs) c. Medical officers (MOs) at PHCs d. Specialists at District Hospitals (Medicine, Surgery, Obstetrics and Gynaecology, Paediatrics, Anaesthesia, Ophthalmology, Radiology, Pathology, Ear- Nose-Throat (ENT), Dental, Psychiatry) Proportion of vacant healthcare provider positions (regular + contractual) in public health facilities

3.1.1 Availability of a functional IT- enabled HRMIS measured by the proportion of staff (regular + contractual) for whom an e-payslip can be generated in the IT-enabled HRMIS against total number of staff(regular + contractual) during a specific year.

Proportion of total staff (regular + contractual) for whom an e-payslip can be generated in the IT- enabled Human Resources Management Information System (HRMIS).

3.1.2 BY: As on March 31, 2015 RY: As on March 31, 2016

State Report

Indicator definition

BY: As on March 31, 2015 RY: As on March 31, 2016

State Report

Proportion of public sector facilities conducting specified number of C-sections* per year (FRUs) against thenorm of one FRU per 500,000 population during a specific year. Proportion of PHCsproviding all stipulated healthcare services** round the clock against the norm of one 24x7 PHC per 100,000 population during a specific year.

a. Proportion of specified type of facilities functioning as First Referral Units (FRUs) b. Proportion of functional 24x7 PHCs

3.1.3 Proportion of districts with functional CCUs [with desired equipment (ventilator, monitor, defibrillator, CCU beds, portable ECG machine, pulse oxymeter etc.), drugs, diagnostics and desired staff as per programme guidelines] against total number of districts.

Indicator definition modified 3.1.4 Proportion of districts with functional Cardiac Care Units (CCUs) Proportion of ANC registered within first trimester against total registrations Proportion of pregnant women registered for ANC within 12 weeks of pregnancy during aspecific year.

3.1.5 Proportion of births registered under CivilRegistration System (CRS) against the estimated number of births during a specific year.

3.1.6 Level of registration of births Proportion of Reporting Units (RUs) reporting in stipulated time period against total RUs, for Pand L forms during a specific year.

3.1.7 Completeness of IDSP reporting of P and L forms Proportion of CHCs that are graded above 3 points against total number of CHCs during a specific year.

3.1.8 Proportion of CHCs withgrading above 3 points State Report on number of functional FRUs, MoHFW data on required number of (FRUs State Report on number of functional 24x7 PHCs, MoHFW data on required number of PHCs BY: 2014-15 RY: 2015-16 BY: 2014-15 RY: 2015-16 BY: As on 3/31/2015 RY: As on 3/31/2016 State Report HMIS BY:2014-15 RY: 2015-16 Civil Registration System (CRS) [pre-entered]

BY: 2013 RY: 2014

Central IDSP, MoHFW Data [pre-entered]

BY: 2014RY: 2015

HMIS BY: 2014-15 RY: 2015-16

*Criteria for fully operational FRUs: SDHs/CHCs - conducting minimum 60 C-sections per year (36 C-sections per year for Hilly and North-Eastern States except for Assam); DHs - conducting minimum 120 C-sections per year (72 C-sections per year for Hilly and North-Eastern States except Assam). Criteria for functional 24x7 PHCs: 10 deliveries per month (5 deliveries per month for Hilly and North-Eastern States except Assam) # Centre NHM Finance data include the RCH exi-pool and NHM-Health System Strengthening exi-pool data (representing a substantial portion of the NHM funds) for calculating delay in transfer of funds. Table 4:** Health Index: Indicators/variables, definitions, data sources, base and reference years.

Bias

Grouping the states according to size was not enough. The researcher feels that population density/ per capita income/ literacy rate/ health workforce/ corruption-scam index etc. should be included for ranking states.

Study Size

All states and UTs of India were participants. Table 5 shows study period

Quantitative Variables

See table 4

Statistical Methods

Methodological details of constructing the Index- Computation of Index scores and ranks After validation of data by the IVA, data was used for the Health Index score calculations. Indicator value was scaled, based on the nature of the indicator, for positive indicators, where higher the value, better the performance, the scaled value (Si) for the indicator, with data value as Xi, was calculated as follows:

Scaled value (Si) for positive indicator = (Xi – Minimum value) x 100/ (Maximum value – Minimum value) For negative indicators where lower the value, better the performance (e.g. NMR, U5MR,) scaled value was calculated as follows:

Scaled value (Si) for negative indicator = (Maximum value – Xi) x 100/ (Maximum value – Minimum value) The minimum and maximum values of each indicator were ascertained based on the values for that indicator across States within the grouping of States (Larger States, Smaller States, and UTs) for that year. Indicator value lies between the ranges of 0 to 100; e.g. the State with the lowest institutional deliveries will get a scaled value of 0, while the State with the highest institutional deliveries will get a scaled value of 100. For a negative indicator such as NMR, the State with the highest NMR will get a scaled value of 0, while the one with the lowest NMR will get a scaled value of 100. Accordingly, the scaled value of other States will lie between 0 and 100 in both cases. Based on these scaled values (Si), a composite Index score was calculated for the base year and reference year by application of the weights using the formula:

Composite Index = (∑ Wi *Si)/ (∑ Wi) --Where Wi is the weight for ith indicator The composite Index score has been used for generating overall performance ranks. The difference between the composite Index score of reference and base years was the annual incremental performance. The ranking is primarily based on the incremental progress, however, rankings based on Index scores for the base year and the reference year performance calculated to provide the overall performance of the States and UTs.

Descriptive Data

Taking into account importance, availability (at least annually) of reliable data, 28 indicators/variables were included first. The availability and quality of data for all States was reviewed and 23 indicators/variables were retained and five indicators/ variables were dropped for calculating the performance in the base and reference years. However, Index scores and ranks for the RY were also calculated independently, based on 24 indicators/variables including an additional indicator on out-of-pocket expenditure, as the data for this was available only for 2015-16. Once the data was accepted by the IVA, the ranks were automatically generated by the portal hosted by the NITI Aayog. To ensure accuracy the indices and ranks were also manually calculated and cross-checked with the results from the portal and the final values were certified by the IVA.

Outcome Data

See Tables 6-17.

Most Improved Improved No Change Deteriorated Most Deteriorated Not Applicable

Table 6: Larger States: Health Outcomes domain indicators base and reference years. **The data shown in grey color is for ‘not applicable’ category wherein the States with TFR <= 2.1 (replacement level fertility) in both base and reference years are not considered for incremental change. #Data for this indicator is available and used only for reference year and hence this indicator comes under ‘not applicable’ category.

** Same data has been used for base and reference years due to overlapping periods of NFHS-4. Hence this indicator comes under ‘not applicable’ category. Table 7: Larger States: Governance and Information domain indicators base and reference years.

** Same data has been used for base and reference years due to overlapping periods of NFHS-4. Hence this indicator comes under ‘not applicable’ category. Table 13: Union Territories: Governance and Information domain indicators base and reference years.

Other Analyses

SRS-related indicators/variables estimates such as NMR were not available for Smaller States and UTs, these estimates could not be generated due to the insufficient sample size. In the Larger States category, MMR were not available separately for 08 states, previously four undivided States, and also for Himachal Pradesh and Jammu & Kashmir. In the case of Still Birth Rate (SBR), the IVA reported that data was unreliable. In case of proportion of pregnant women age 15-49 years who are anaemic, data on the appropriate denominator was not available in the HMIS. Proportion of people living with HIV (PLHIV) on ART excluded for the UTs since no ART centre was available in four UTs. NHM funds utilized by the end of 3rd quarter, data were not valid. Central data was used for a few indicators/variables such as PLHIV on antiretroviral therapy (ART), ‘average number of days for transfer of central NHM funds from State Treasury to implementation agency’ and ‘completeness of IDSP reporting of P and L forms’. The NFHS-4 data for out-of-pocket expenditure on drugs and diagnostics incurred per delivery in public health facilities was used in the RY Index. However, for the BY, this data was not available and could therefore not be factored in for generating BY ranks or incremental ranks or drawing comparisons between the base and reference years.

Key Results

There is a large gap in overall performance of States and UTs, overall performance ranged widely between 33.69 in Uttar Pradesh to 76.55 in Kerala. Similarly, among Smaller States, the Index score for overall performance varied between 37.38 in Nagaland to 73.70 in Mizoram, and among UTs this varied between 34.64 in Dadra & Nagar Haveli to 65.79 in Lakshadweep. Among the Larger States (table-15), Jharkhand, Jammu & Kashmir, and Uttar Pradesh are the top three in terms of annual incremental performance, while Kerala, Punjab, and Tamil Nadu ranked on top in terms of overall performance. In terms of incremental performance top three are Jharkhand (up 6.87 points), Jammu & Kashmir (up 6.83 points) and Uttar Pradesh (up 5.55 points). Jharkhand, Jammu & Kashmir, and Uttar Pradesh showed the maximum gains in improvement of health outcomes from base to RY.

Table 17: Union Territories: Incremental scores and ranks, with overall performance from base year to reference year and ranks Among UTs (Table 17), Lakshadweep showed both the highest annual incremental performance as well as the best overall performance The incremental measurement shows that about one- third of the States declined in their Health Indices in the RY as compared to the BY. Tables 18-21 provide a categorization of States and UTs based on the level of annual incremental performance and the overall performance.

The indicators/variables where most States and UTs need to focus include vacancies in key staff, establishment of functional district Cardiac Care Units (CCUs), quality accreditation of public health facilities, and institutionalization of Human Resources Management Information System (HRMIS). Additionally, almost all Larger States need to focus on improving the Sex Ratio at Birth (SRB).

Note: Overall Performance: The States are categorized on the basis of RY Index score range: Front-runners: top one- third (Index score>62); Achievers: middle one-third (Index score between 48 and 62), Aspirants: lowest one-third (Index score<=0), ‘Least Improved’ (incremental Index score between 0.01 and 2), ‘Moderately Improved’ (incremental Index score between 2.01 and 4), ‘Most Improved’ (incremental Index score>4.0).

Limitations

There is need for making outcome data available for smaller states, updated outcomes for non-communicable diseases and financial protection, robust programmatic data for continuous monitoring, were important issues, could not be addressed optimally in this first round.

Limitations of the Index

1. Non-availability of acceptable quality of data on an annual basis. 2. Paucity and uneven availability of private sector data in the HMIS.

3. Analytical tools could not be used to derive domain- specific weights. 4. For SRS data was available only for Larger States.

Interpretation

The Health Index score ranking is the first attempt at establishing an annual systematic tool for measurement of performance across States and UTs of health parameters. The results provide an important insight into the areas in which States have improved, stagnated or declined which will help in better targeting of interventions.

Generalizability

The States and UTs rank differently on performance, States and UTs at lower levels of the Health Index (lower levels of development of their health systems) are at an advantage in notching up incremental progress over States with high Health Index score. For example, Kerala ranks on top in terms of overall performance and at the bottom in terms of incremental progress mainly as it had already achieved a low level of Neonatal Mortality Rate (NMR) and Under-five Mortality Rate (U5MR) and replacement level fertility, leaving limited space for any further improvements.