Dev Site — You are viewing the development build. Go to Main Site
Health facility reporting rate
Overview
In the SNT workflow, routine surveillance data are often used to calculate key indicators such as malaria incidence, test positivity rate, and confirmed case counts. To interpret and use this data properly, we need to first assess their quality and completeness. One important assessment is of what proportion of health facilities are reporting consistently and completely across geography and time.
Reporting rates provide a simple and essential metric to evaluate the completeness of routine data. They help identify where gaps in reporting may affect the reliability of indicators used in decision-making and where routine surveillance should be strengthened. Reporting rates can also be used to estimate the number of additional cases (or other count indicator) that would have also been included in routine surveillance, if all facilities had reported.
This section outlines how to calculate, inspect, and save reporting rates using a reproducible approach, while grounding all choices in dialogue with the national malaria program and SNT team.
Objectives
Understand how to calculate reporting rates from routine health facility data
Visualize and interpret reporting rate patterns at any admin unit level
Compile and save validated reporting rate outputs for use in later SNT workflow steps
Defining and Calculating Reporting Rate for Routine Indicators
To ensure routine data can be used reliably in the SNT workflow, we need a clear and consistent method for calculating reporting rates across facilities and time. This section introduces a process to calculate monthly reporting rates for any routinely reported indicator.
What is reporting rate?
Reporting rate is the proportion of entities, such as health facilities or community health workers, in a given admin unit that reported on an indicator during a time period of interest.
In the example on this page, we are using monthly data from DHIS2, so we calculate monthly reporting rates. However, you should calculate reporting rate for the relevant reporting period in your dataset. For example, if you are analyzing weekly surveillance data, your reporting rate should be calculated on a weekly basis.
When presenting reporting rates, it is important to specify what indicator(s) are used to define the entity as reporting. In SNT, it is best practice to re-calculate reporting rate for each indicator of interest, as reporting practice may vary across indicators within the same facility. For example, a facility may prioritize reporting confirmed cases, as their stock replenishment depends on showing consistent reporting, but neglect to report all-cause outpatient visits.
An overall reporting rate can also be calculated for each entity in a given time period. In this case, a facility may only be defined as reporting if it reports suspected cases, tested cases, confirmed cases, and treated cases. This aggregated reporting rate should be at most the minimum of the reporting rate of each individual indicator.
Establishing the denominator: Which facilities are expected to report?
Before evaluating the proportion of facilites that reported in a given reporting period, we need to first determine the number of facilities that should report. To avoid underestimating the reporting rate and gaining and inaccurate assessment of the quality of surveillance, the SNT team may, for example, consider:
When calculating reporting rate for confirmed malaria cases, exclude facility types that do not test or treat malaria. For example, HIV clinics or maternity wards.
When calculating reporting rate for malaria admissions, exclude facility types that only handle outpatients, for example community health workers or health posts.
When calculating reporting rate for an indicator, exclude facilities that have closed, that are not yet active, or are temporarily nonfunctional. This avoids penalizing newly opened facilities that weren’t expected to report in earlier months, or facilities that are permanently or temporarily closed and therefore are not expected to report.
Up-to-date master facility lists (MFL) that track facility type and activity status are very helpful for determining which health facilities should be included in the denominator for reporting rate of each indicator, for each reporting period. In the absence of an MFL, or official determination of activity status, it is still possible to infer which health facilities should be excluded.
Code for excluding by specific facility type is included on this page.
Consult SNT team
Consult the SNT team to understand how to determine which facilities, if any, should not be included in the denominator for reporting rate. National practices vary, and the surveillance focal person on the SNT team should explain what would be appropriate for each indicator.
Calculating reporting rates
Once health facility activity status has been established, reporting rates can be calculated. These rates reflect the proportion of expected facilities that submitted valid data for a given indicator in a given time period.
For each indicator of interest, reporting rate is defined as:
\(a\) is the administrative unit (e.g. chiefdom or district)
\(t\) is the time period (e.g. “2022-03”)
\(o_{a,t}\) is the number of observed facilities in unit \(a\) during time \(t\)
\(e_{a,t}\) is the number of expected facilities in unit \(a\) during time \(t\)
Observed facilities are those that submitted a valid (non-missing) value for the indicator of interest during time \(t\).
Expected facilities are those who were expected to report during time \(t\) (see previous section). Remember to consult with the SNT team to decide what rules determine whether a facility is expected to report.
Worked example
PLEASE MAKE SURE THIS EXAMPLE IS USING ACTUAL NUMBERS FOR KAILAHUN DISTRICT IN MARCH 2022
Suppose we are calculating the reporting rate for total confirmed cases for Kailahun District in March 2022.
There are 6 health facilities in Kailahun that have ever submitted data on any key malaria indicator
All 6 submitted their first report on or before March 2022, so they are assumed to be active and expected to report that month
Of these, 4 facilities reported a valid value for conf (total confirmed cases) in March 2022 → 4 are observed reporting
The other two do not have a valid value for total confirmed cases (they show NA in the database) for March 2022
The reporting rate is calculated as:
\[
\text{Reporting Rate for Total Confirmed Cases}_{\text{Kailahun}, \text{Mar 2022}} = \frac{4}{6} = 0.67
\]
Weighted reporting rates
For some SNT applications, a weighted reporting rate may be of interest. The weighted reporting rate is an estimate of the proportion of an indicator’s expected total counts in a given admin unit over a given time period that was reported into routine surveillance.
This means that if a non-reporting facility generally reports fewer confirmed cases than average in its admin unit, the weighted reporting rate for confirmed cases is higher than the unweighted reporting rate (fewer cases are missing). Conversely, if a non-reporting facility generally reports relatively many confirmed cases for its admin unit, the weighted reporting rate for confirmed cases is lower than the unweighted reporting rate (more cases are missing).
Calculating weighted reporting rates
PLACEHOLDER EXPLANATION BELOW. WOULD BE GREAT TO INCLUDE THE SIMPLE EXAMPLE FROM BEA’S SLIDE DECK incl the illustration if possible
The monthly weighted reporting rate for each health district was determined as follows. For each calendar month (January through December), health facility weights were calculated by dividing the health facility’s average number of malaria cases reported for that month, across all years of data, by the district sum of the average number of malaria cases reported for that month, across all years of data. For dates in which a health facility is inactive, the average number of malaria cases reported for that month-and-year pair would be 0, and the weights for the health facilities in that district for that date would be calculated as usual. The district monthly weighted reporting rate was then calculated by summing the weights of the active health facilities. This value captures the proportion of expected confirmed malaria cases that are reported at the district level each month by the active health facilities included in the HMIS database.
How do I know whether to use unweighted or weighted reporting rate?
To assess the performance of the routine surveillance system, the unweighted reporting rate is likely to be more appropriate.
To estimate an admin unit’s unreported confirmed cases (as an example indicator), the weighted reporting rate may be the more accurate option, as it will account for the size of the non-reporting facility. If you choose to use weighted reporting rates, best practice is to also calculate the unweighted reporting rates for the same indicator, compare the two outputs, and discuss with the SNT team.
WOULD BE GREAT TO INCLUDE IN THE STEP BY STEP EXAMPLE VISUALIZATIONS. BOTH SEBASTIAN’S LINE PLOT VERSION AND OUSMANE’S HEATMAPS
Consult SNT team
If you think the weighted reporting rate might be the better option, produce reporting rates for using both methods and present to the SNT team. Discuss with the SNT team to understand how, where, when, and why the two reporting rates are different. Together with the SNT team, discuss which to use for downstream analysis.
Step-by-Step
Now that we’ve defined how reporting rates are constructed—by identifying active facilities and calculating observed reporting—we move into the step-by-step process for implementing this in code using example DHIS2 data from Sierra Leone. In this section, we walk through the steps for calculating and visualising monthly reporting rates. Each step is designed to guide you through the process. Follow the notes in the code, especially where edits are required.
To skip the step-by-step explanation, jump to the full code at the end of this page.
Objectives
Calculate monthly reporting rates
Visualise reporting rate over time, by indicator
Step 1: Import relevant packages and pre-processed routine data
In this step, we load the necessary packages to run this section, as well as the routine DHIS2 dataset that was initially processed in the DHIS2 Data Preprocessing section of this code library.
Step 3: Define function to calculate reporting rate
Now we define a function to calculate the monthly reporting rate, for a given indicator, at a given admin unit level. The function performs the following steps:
Count the number of non-NA reports for the indicator of interest (observed reports), aggregated by month and by the admin unit level requested by the user
Count the number of expected reports by month and admin uit unit level requested by the user, by aggregating dfden
Merge the two datasets (observed reports and expected reports, i.e. reporting rate numerator and denominator)
