PRISM Case Studies: Strengthening and Evaluating RHIS

MEASURE Evaluation has developed a conceptual framework to evaluate routine health information systems (RHIS), called Performance of Routine Information System Management (PRISM). The case studies presented in this 8-page paper illustrate how PRISM has been used in particular contexts to help design, strengthen, monitor, and evaluate RHIS in various settings: Mexico, South Africa, Uganda, and Pakistan.

As detailed here, the PRISM framework is grounded in a systems perspective, focusing on problem solving and continuous improvement - an important strategy, the author contends, particularly in developing countries were RHIS may be prone to weaknesses. PRISM highlights RHIS performance by gauging it in terms of improved data quality and continuous information use. In addition, PRISM places RHIS performance in the context of technical, behavioural, and organisational determinants whose relative importance, as measured by PRISM tools, generates possible interventions. And, finally, it provides a mechanism to assess the role of RHIS in improving health system performance.

PRISM tools are generic and need to be adapted to capture specific characteristics and situations; thus, the 4 case studies included here are designed to illustrate the characteristics of the PRISM strategy - and its utility - in concrete ways.

1. Mexico Health Management Information Systems (HMIS) - The PRISM Organizational and Behavioral Assessment Tool (OBAT) was used to guide Mexico's Ministry of Health (MOH) in its efforts to promote a culture of information. A survey of district-level (and higher-level) officials assessed to what extent they knew the purpose and methods for checking HMIS data quality. Gaps that were exposed (e.g., on average, 72% of respondents believed strongly that the MOH promotes use of HMIS information, but only 52% of the respondents showed how to use HMIS information) were then used to shape strategies. For instance, a website was created permitting all 33 state health departments access to the OBAT questionnaire; this reportedly enabled national authorities to prioritise interventions and produce a plan for the incoming government to improve information use and decision making at all levels.
2. South Africa District Health Information Systems (DHIS) - Data was collected by 30 University of Pretoria participants using all PRISM tools in an effort to assess the performance of the existing DHIS. "The PRISM framework highlighted systemic gaps in HMIS function and showed that the responsibility for improved performance rests with senior management and staff at all levels. The need for better coordination and harmony within the different components of RHIS was recognized, as well as the need for better self-assessment tools, better problem-solving skills and continuous improvement."
3. Uganda HMIS - In 2004, the National Health Information Resource Centre (NHIRC) asked the United States Agency for International Development (USAID)-funded UPHOLD project to support a situation analysis to identify gaps in HMIS performance using the PRISM framework as part of its effort to harmonise and integrate HIV/AIDS information. Findings from the resulting survey of 120 facilities in 12 districts were presented at a meeting that drew over 50 stakeholders including USAID and other national and international (e.g., United Nations) organisations. In general, a disconnect was found to exist among the categories examined (perception that an organization emphasises data quality and promotes the use of information, competence in checking data quality, and using information), and observed HMIS performance. Based on the findings, steps were taken to improve HMIS performance; for example, HIV/AIDS service indicators were harmonised, finalised, and integrated into revised monthly reporting forms and facility registers. Facility and district staff were then trained on the revised forms. A training manual on HMIS data quality and information use was developed, pre-tested, and integrated into trainings, and a data accuracy assessment checklist was developed.
4. Pakistan: Shift from HMIS to DHIS: In 2004, the HMIS team conducted a baseline situation analysis using PRISM tools to identify strengths and weaknesses in the existing HMIS. Through a process of consensus-building with stakeholders at all levels and based on the findings of the assessment study, a reform package was developed and pilot-tested in 4 districts. This pilot DHIS was then evaluated after 6 months; the results were then presented in meetings at federal, provincial, and pilot districts (in total, more than 1,000 government officials and stakeholders participated in meetings about the HMIS reform process and consensus building). Examples of the use of baseline and evaluation findings include: An HMIS training manual was developed for the revised data collection registers and forms; the HMIS team developed and pilot-tested a district software application for data entry, analysis, and report generation; and, during the pilot test, the facilities and the districts conducted monthly performance reviews using DHIS information and recorded decisions on a new DHIS register.

MEASURE Evaluation is funded by USAID and implemented by Carolina Population Center at the University of North Carolina (UNC) at Chapel Hill, NC, US, in partnership with Constella Futures, John Snow, Inc., Macro International Inc., and Tulane University.