User Support

INTRODUCTION

AHRQ QUALITY INDICATORS

PRPEARATION OF HCUP DATABASES

HCUP Data on Race/Ethnicity

Modifications to the HCUP Data

Creation of Analysis Files Derived from the HCUP Databases

STEPS TAKEN TO APPLY AHRQ QUALITY INDICATORS TO THE HCUP DATA

Review and Modify the AHRQ QI Software

Acquire Population-Based Data for Denominators

Assign QI Indicators to the HCUP Databases

Adapt Statistical Mthods to HCUP Data

SPECIAL ANALYSES

National Inpatient Hospital Costs Associated with Quality Indicators

Benchmarks for State Performance for the Quality Indicators

Medicaid and Uninsured Inpatient Stays and Aggregate Hospital Costs in the United States

National Rates for Inpatient Stays for Select Maternal Conditions

National Estimates of Emergency Department Utilization

TABLES

Table 1. AHRQ QIs Applied to the HCUP Data for the NHQDR

Table 2. Sources of 2016-2018 HCUP Inpatient Data for the NHQDR

Table 3. Editing Essential Data Elements for State-Level Reporting of QIs in the NHQDR

REFERENCES

APPENDIX A: NATIONALLY WEIGHTED ANALYSIS FILES FOR THE NATIONAL HEALTHCARE QUALITY AND DISPARITIES REPORT

APPENDIX B: EMERGENCY DEPARTMENT RATES FOR SELECTED AHRQ QUALITY INDICATORS

APPENDIX C: STATISTICAL METHODS

APPENDIX D: CAVEATS TO THE INTERPRETATION OF HCUP-BASED QI ESTIMATES REPORTED IN THE NHQDR

APPENDIX E: INPATIENT RATES FOR DELIVERY MEASURES

INTRODUCTION

The Agency for Healthcare Research and Quality (AHRQ) Quality Indicators (QIs) were applied to the Healthcare Cost and Utilization Project (HCUP) hospital discharge data for selected measures in the National Healthcare Quality and Disparities Report (NHQDR). The report measures and tracks trends in quality and disparities in terms of patient safety, person-centered care, care coordination, effective treatment, healthy living, and care affordability - and access to health care. A focus on priority populations summarizes quality and disparities in care for populations at elevated risk for receiving poor health care, which includes HCUP-based measures related to racial, ethnic, and socioeconomic factors for priority populations, including changes over time and across the urban-rural continuum. The NHQDR provides a comprehensive overview of the quality of health care received by the general population and disparities in care experienced by different racial, ethnic, and socioeconomic groups.

This document describes the preparation of the 2016-2018 HCUP data for the 2021 NHQDR, the steps taken to apply the AHRQ QIs to the HCUP data, and other analyses based on HCUP data that are not specific to the QIs but are developed for use in the NHQDR.

AHRQ QUALITY INDICATORS

The AHRQ QIs are measures of quality associated with processes of care that occur in an outpatient or an inpatient setting. The QIs rely solely on hospital inpatient administrative data and, for this reason, are screens for examining quality that may indicate the need for more in-depth studies. The AHRQ QIs include four sets of measures:

• Prevention Quality Indicators (PQIs) — or ambulatory care sensitive conditions — identify hospital admissions that evidence suggests could have been avoided, at least in part, through high-quality outpatient care (AHRQ, 2019).


• Inpatient Quality Indicators (IQIs) reflect quality of care inside hospitals and include measures of utilization of procedures for which there are questions of overuse, underuse, or misuse (AHRQ, 2019).


• Patient Safety Indicators (PSIs) reflect quality of care inside hospitals, by focusing on surgical complications and other iatrogenic events (AHRQ, 2019).


• Pediatric Quality Indicators (PDIs) reflect quality of care inside hospitals and identify potentially avoidable hospitalizations among children (AHRQ, 2019).

Prior to the 2018 NHQDR, the reports included longitudinal reporting of QI trends based on ICD-9-CM¹ coding. Because of the transition from ICD-9-CM to ICD-10-CM/PCS² on October 1, 2015, the 2021 NHQDR includes QI estimates for 2016, 2017 and 2018 and does not report on trends prior to 2016.

The following national and State-level QI estimates were constructed from the HCUP databases for the NHQDR:

• National inpatient rates by QI, overall and by subpopulations including community income, expected primary payer, and race/ethnicity


• State-level inpatient rates by QI, overall and by subpopulations including community income, expected primary payer, and race/ethnicity, for HCUP Partner organizations that agreed to participate in the NHQDR


• National rates for selected PQIs and PDIs in the emergency department (ED) setting.

The AHRQ QI measures generated for possible inclusion in the NHQDR are described in Table 1 at the end of this methods report.

PREPARATION OF HCUP DATABASES

The Healthcare Cost and Utilization Project (HCUP) is a family of healthcare databases and related software tools and products developed through a Federal-State-Industry partnership and sponsored by AHRQ. HCUP databases are derived from administrative data and contain encounter-level, clinical and nonclinical information including all-listed diagnoses and procedures, discharge status, patient demographics, and charges for all patients, regardless of payer (e.g., Medicare, Medicaid, private insurance, uninsured), beginning in 1988. These databases enable research on a broad range of health policy issues, including cost and quality of health services, medical practice patterns, patient safety, access to health care programs, and outcomes of treatments at the national, State and local market levels.

The following HCUP databases were used as the source of data for the NHQDR:

• The HCUP State Inpatient Databases (SID), including all inpatient stays regardless of payer from most, if not all, hospitals from participating HCUP Partner organizations in data years 2016-2018.


• The HCUP National Inpatient Sample (NIS), a nationally stratified sample of discharges from all hospitals in States that contributed data to HCUP in data years 2016-2018.


• The HCUP Nationwide Emergency Department Sample (NEDS), a nationally stratified sample of hospital-based emergency departments (EDs) (with information for both treat-and-release visits and those resulting in a hospital admission) in data years 2016-2018.

The list of HCUP Partner organizations that contribute to the HCUP databases each year is provided in Table 2.

HCUP Data on Race/Ethnicity

HCUP coding includes race and ethnicity in one data element (RACE). Because of variability in the collection of race and ethnicity information in the State data, HCUP maintains a uniform set of categories based on race definitions used in the 1977 Office of Management and Budget (OMB) Directive 15 using the combined race-ethnicity format (separate categories for Hispanic and five Non-Hispanic racial groups - White, Black, Asian or Pacific Islander, American Indian or Alaska Native (AIAN), and Other).

When a State and its hospitals collect Hispanic ethnicity separately from race, HCUP assigns the data to the combined race/ethnicity categorization and uses Hispanic ethnicity to override any other race category to create uniform coding across States.

There is limited reporting of AIAN in the HCUP data. In addition, in some areas of the country, care for the AIAN population is provided in Indian Health Service (IHS) hospitals which are not included in HCUP. For these reasons, AIAN discharges were combined with "Other" races for the NHQDR analyses.

The resulting NHQDR reporting categories for the HCUP data include: White Non-Hispanic; African American Non-Hispanic; Asian/Pacific Islander Non-Hispanic; Other Non-Hispanic; and Hispanic (of any race).

Modifications to the HCUP Data

In preparation for the NHQDR and its derivative products, the HCUP databases needed to be customized as indicated below:

1. The HCUP databases were augmented as necessary for the NHQDR analyses:
   
   
   • Impute for Missing Characteristics. For missing age, sex, race/ethnicity, ZIP Code, and expected primary payer data that occurred on a small proportion of discharge records, we used a "hot deck" imputation method (which draws donors from strata of similar hospitals and patients) to assign values while preserving the variance within the data.
   
   
   • Assign Additional Measures for Reporting. We assigned median household income quartile by linking Claritas ZIP Code demographic data to patient’s ZIP Code in the SID. Income quartiles were defined annually based on the distribution of the population in the United States.


2. Essential data elements were edited prior to use to determine if the SID had good quality data, and later informed which records and States could be used for national and State-level reporting in the NHQDR:
   
   
   • Race/Ethnicity. Race/ethnicity is an important reporting category for the NHQDR. Race/ethnicity edits were performed at the hospital-level using the HCUP data element RACE and include the following: (1) all discharges in the hospital (100 percent) had RACE coded as White and the hospital had more 2,500 discharges in the year; (2) more than 30 percent of the discharges in the hospital had the race reported as Other; (3) more than 50 percent of the discharges in the hospital had no information on the race/ethnicity of the patient (i.e., RACE was missing or invalid); and (4) no discharges in the hospital were coded as Black, Hispanic, Asian/Pacific Islander, or AI/AN.
   
   
   • Indicators that Diagnoses are Present on Admission (POA). Present on admission (POA) data are required for the PSI module software. Edit flags for POA were already included on the SID. There are three hospital-level and two discharge-level edits. The three hospital-level edits include: (1) POA is reported as "yes/present" on all diagnoses on all discharges in the hospital; (2) POA is reported as missing on all non-Medicare discharges in the hospital; and (3) POA is reported as missing on all nonexempt diagnoses³ for 15 percent or more of the discharges in the hospital. The two discharge-level edits include: (1) discharge is missing POA on all nonexempt diagnoses; and (2) Discharge is missing POA on all nonexempt secondary diagnoses. Hospitals in qualifying SID were excluded if they failed any POA edits.

Creation of Analysis Files Derived from the HCUP Databases

Beginning with the 2021 NHQDR, two versions of the nationally weighted NHQDR analysis file were used for reporting national QI estimates. Both used the SID, but were limited to community hospitals, excluding rehabilitation and long-term acute care (LTAC) facilities. The two versions were specific to the data elements needed for the different QI modules:

• For the PQIs, IQIs, and area-based PDIs, the nationally weighted file included data from SID that included information on race/ethnicity of the patient. After hospitals that failed the race/ethnicity edits were excluded, all remaining discharges in the selected SID were weighted to the universe of community hospitals⁴ in the United States, excluding rehabilitation and LTAC facilities.


• Because the PSIs and related PDIs require knowing if the patient safety event occurred in the hospital and the timing of procedures, a second version of the nationally weighted file was created using the following criteria: (1) the SID included the data elements indicating diagnoses were POA, (2) the SID included information on day of principal and secondary procedure days, and (3) the SID included information on the race/ethnicity of the patient. Hospitals in qualifying SID were excluded if they failed POA and race/ethnicity edits. All remaining discharges were weighted to the universe of community hospitals in the United States, excluding rehabilitation and LTAC facilities.

Table 3 shows which SID qualified for by race/ethnicity and PSI reporting for data years 2016-2018. A comparison of the nationally weighted NHQDR analysis file to the HCUP National Inpatient Sample (NIS) data is provided in Appendix A.

The editing of the race/ethnicity and POA data informed which records and States could be used for State-level reporting in the NHQDR. The HCUP SID were modified to create analytic files consistent across States.

• Subset to Community Hospitals. For the SID, we selected community hospitals and eliminated rehabilitation and LTAC hospitals.


• Weight for Missing Hospitals. Because some statewide data organizations do not report data for all community hospitals in the State, we weighted hospitals in the SID to the State's universe of hospitals in the American Hospital Association (AHA) Annual Survey Database based on hospital characteristics (teaching status, control, urban/rural location, and bed size).


• Weight for Missing Quarters. Discharges from hospitals operating for the entire year but not contributing data for one or more quarters were weighted up to annual estimates for that institution in the SID.

Weighting also accounted for all any discharges or hospitals excluded because of edit checks on race/ethnicity and POA.

STEPS TAKEN TO APPLY AHRQ QUALITY INDICATORS TO THE HCUP DATA

To apply the AHRQ Quality Indicators to HCUP hospital discharge data for the NHQDR, several steps were taken:

1. QI software review and modification
2. Acquisition of population-based data
3. Assignment of QIs to the HCUP databases
4. Identification of statistical methods

Review and Modify the AHRQ QI Software

We used AHRQ QI software v2020.1 that features ICD-10-CM/PCS coding. The AHRQ QI software v2020.1 calculates risk-adjusted rates for a subset of hospital measures and all area-based measures. Therefore, the 2021 NHQDR includes risk-adjusted QI estimates as available in the software.

The QI software was modified for the 2021 NHQDR in several ways.

• The AHRQ QI software was designed to produce rates at various levels including hospital, local area (county or Metropolitan/Micropolitan Statistical Areas), and State. The QI software was revised to produce national rates for the NHQDR using weighted discharge data.


• The AHRQ QI software was limited to reporting rates by age, sex, and race/ethnicity. The NHQDR included additional patient categories (community income level of the patient's ZIP Code, primary expected payer, and location) and hospital characteristics (region, ownership, teaching status, location, and bed size).


• The AHRQ QI software was designed to produce standard errors for unweighted data. For the NHQDR, we needed to produce statistics for weighted data from the stratified sample of discharges in the nationally weighted analysis files and test for significant differences.


• At AHRQ's request, we added two area-based QIs particularly relevant to the structure of the NHQDR, measuring immunization-preventable pneumococcal pneumonia and immunization-preventable influenza for patients aged 65 years and older.

Acquire Population-Based Data for Denominators

The next step was to acquire data for the numerator and denominator populations for the QIs. The AHRQ QIs measure an event that occurs in a hospital, requiring a numerator count of the event of interest and a denominator count of the population (within a hospital or geographic area) to which the event relates.

For the numerator counts of the AHRQ QIs, we used the HCUP databases. For the denominator counts, we identified two sources for all reporting categories.

• For QIs that related to providers, the HCUP data were used for national and State-level discharge denominator counts.


• For QIs that related to geographic areas, population ZIP-Code-level counts from demographic update data provided by Claritas (a vendor that compiles and adds value to the U.S. Bureau of Census data) were used for denominator counts. Claritas uses intra-census methods to estimate household and demographic statistics for geographic areas (Claritas).

Assign QI Indicators to the HCUP Databases

The four AHRQ QI program modules were applied to the prepared SID data using all available diagnoses and procedures reported by each State.

Adapt Statistical Methods to HCUP Data

Several statistical issues needed to be addressed when applying the AHRQ QI software to the HCUP data for the NHQDR, including the derivation of standard errors for weighted data and appropriate hypothesis tests.

• Standard Errors and Hypothesis Tests. Standard error calculations needed to take into account cluster and stratification aspects of the weighted analysis files. Standard errors were calculated for the NHQDR as described in the HCUP report entitled Inferences with HCUP State Databases Final Report (Houchens, et al., 2010). Using this method, we were able to calculate appropriate statistics for both national and State-level AHRQ QIs using the Statistical Analysis System (SAS) procedure called PROC SURVEYMEANS.


• Area-Based Measures Risk Adjustment Methodology. The risk adjustment methodology in the QI software was developed for county-level reporting. As in prior years’, Watson Health applied a direct standardization method for risk adjustment to the area-based measures (using by 18 5-year age groups and sex) to allow for the reporting by patient characteristics not included in the QI software such as community income quartile and the urban/rural location of the patient’s residence.


• Masking Rates for Statistical Reliability, Data Quality, and Confidentiality. QI estimates were included in the NHQDR if they reached a threshold defined by a relative standard error less than 30% and at least 100 weighted cases in the denominator. Estimates that did not meet this threshold were suppressed. Statistical calculations are explained in Appendix C to this report.

Some caution should be used in interpreting the QI statistics presented in the NHQDR. Appendix D details how these caveats relate to inter-State differences in data collection and other more general issues.

SPECIAL ANALYSES

National Inpatient Hospital Costs Associated with Quality Indicators

The NHQDR includes total national costs for the select PQIs and PDIs including the composites and ambulatory care sensitive conditions such as heart failure, diabetes, and asthma. Total national costs associated with these PQIs and PDIs were calculated overall and by community income quartile and race/ethnicity.

Total charges were converted to costs using the hospital-level HCUP Cost-to-Charge Ratio Files (CCR Files) based on Hospital Cost Report data from the Centers for Medicare & Medicaid Services (CMS).⁵ Costs reflect the actual costs of production, while charges represent what the hospital billed for the stay. Hospital charges reflect the amount the hospital charged for the entire hospital stay and do not include professional (physician) fees. To account for records with missing total charges and costs in the aggregate, the average cost for the QI was used to impute the missing values for that QI. This approach compensates for stays for which charges (and thus estimated costs) are not available.

Benchmarks for State Performance for the Quality Indicators

Based on a recommendation from the Institute of Medicine’s report on Future Directions for the National Healthcare Quality and Disparities Reports, benchmarks based on a straight average of the top 10 percent of reporting States were determined. For a benchmark to be calculated, rates for at least 30 States needed to be available.

Medicaid and Uninsured Inpatient Stays and Aggregate Hospital Costs in the United States

Information on Medicaid and uninsured inpatient stays and aggregate hospital costs in the United States for 2012–2018 were developed using the HCUP National Inpatient Sample (NIS). Medicaid and uninsured discharges were identified based on the expected primary payer of Medicaid, self-pay, no charge, and charity. Discharge counts and aggregate hospital costs were reported as a percentage of the total U.S and by hospital characteristics including region, ownership, teaching status, urban-rural location, and size of the hospital based on the number of beds.

National Rates for Inpatient Stays for Select Maternal Conditions

The nationally weighted NHQDR analysis file was used to examine differences in inpatient stays for select maternal conditions. Rates for the following measures were calculated for inpatient stays for women aged 12-55 years with any delivery diagnosis, procedure, or DRG, excluding those with any indication of abortion:

• Inpatient stays for severe maternal morbidity per 1,000 delivery hospitalizations. Severe maternal morbidity conditions were defined by the Center for Disease Control and Prevention. Information on the coding criteria is available at: www.cdc.gov/reproductivehealth/maternalinfanthealth/smm/severe-morbidity-ICD.htm. Rates were not risk-adjusted.


• Inpatient stays for venous thromboembolism or pulmonary embolism per 1,000 delivery hospitalizations. Any diagnosis was used to identify venous thromboembolism or pulmonary embolism (see Appendix E for detailed measures specifications). Rates were not risk-adjusted.


• Inpatient stays for postpartum hemorrhage per 1,000 delivery hospitalizations. Any diagnosis was used to identify postpartum hemorrhage (see Appendix E for detailed measures specifications). Rates were not risk-adjusted.


• Inpatient stays for eclampsia or preeclampsia per 1,000 delivery hospitalizations. Any diagnosis was used to eclampsia or preeclampsia (see Appendix E for detailed measures specifications). Rates were not risk-adjusted.

National Estimates of Emergency Department Utilization

The NEDS was used to examine national and regional differences in ED use for selected ambulatory care sensitive conditions

• ED visits for mental illness, substance use, and co-occurring mental illness and substance use. ED visits were identified by having a first-listed diagnosis related to mental illness, substance use, or co-occurring mental illness and alcohol or substance use (See Appendix B for detailed measures specifications). Claritas population data was used to calculate rates per 100,000 residents by age, sex, community income, urban-rural location of patient residence, and region of the United States. Rates were not risk-adjusted.


• ED visits for dental conditions. ED visits were identified by an ICD-10-CM principal diagnosis related to a dental condition (See Appendix B for detailed measures specifications). Claritas population data was used to calculate rates per 100,000 residents by age, sex, community income, and urban-rural location of patient residence. Rates were not risk-adjusted.


• ED visits for injuries. ED utilization for injuries was reported by trauma level and by age, sex, community income, urban-rural location of patient residence, and region of the United States (See Appendix B for detailed measures specifications). Rates were not risk-adjusted.


• ED visits for asthma for adults based on PQI 15 and for pediatrics based on PDI 14 (see Appendix B for detailed specifications).


• ED visits for heart failure based on PQI 8 (see Appendix B for detailed specifications).


• ED visits for dementia. ED visits in the U.S. for patients aged 65 years or older were identified by any ICD-10-CM diagnosis related to dementia (See Appendix B for detailed measure specifications). Rates for per ED visits and per 100,000 population aged 65 years and older were not risk-adjusted. Claritas population data was used to calculate rates per 100,000 population aged 65 years and older.

Additional details on the use of the NEDS for reporting are provided in Appendix B.

TABLES

Table 1. AHRQ QIs Applied to the HCUP Data for the NHQDR

This table includes the list of all AHRQ QI software v2020.1 QIs calculated using 2016-2018 HCUP inpatient data under the NHQDR task. Not all of the AHRQ QIs listed below were included in the NHQDR.

The QI specifications are consistent with the AHRQ software. For more information on QI definitions, please visit the AHRQ QI website (https://www.qualityindicators.ahrq.gov/Default.aspx).

Table 2. Sources of 2016-2018 HCUP Inpatient Data for the NHQDR

Sponsored by the Agency for Healthcare Research and Quality (AHRQ), HCUP is a family of databases, software tools, and products developed through the collaboration of State data organizations, hospital associations, private data organizations, and the Federal government. HCUP would not be possible without the contributions of the following data collection Partners from across the United States.

The reporting of State-Specific information in the National Healthcare Quality and Disparities Report (NHQDR) is dependent on the permission from the HCUP Partner. Use of the State Inpatient Databases (SID) in the nationally weighted NHQDR analysis file is dependent on the availability of essential data elements. The States that met the inclusion criteria for developing the national-level weights in the final NHQDR analysis file are noted below.

The notations used in Table 2 are defined below:

• Yes, all: Data from the SID were used to report on all four types of QIs (PQIs, IQIs, PSIs, and PDIs).


• Yes, subset: Data from the SID were used to report only on PQIs, IQIs, and area-based PDIs.
  • SID were excluded from the nationally weighted analysis file used for PSI and related PDI reporting if they did not meet POA, race/ethnicity, and or procedure day criteria


• No: Data from the SID were not used for national QI reporting.
  • SID did not include information on the race/ethnicity of the patient or were unavailable at the time the nationally weighted analysis file was constructed.

Table 3. Editing Essential Data Elements for State-Level Reporting of QIs in the NHQDR

REFERENCES

Agency for Healthcare Research and Quality. Inpatient Quality Indicators: Technical Specifications, AHRQ Quality Indicators, Version 2020.01 SAS. Rockville, MD: Agency for Healthcare Research and Quality, May 2020.

Agency for Healthcare Research and Quality. Patient Safety Indicators: Technical Specifications, AHRQ Quality Indicators, Version 2020.01 SAS. Rockville, MD: Agency for Healthcare Research and Quality, May 2020.

Agency for Healthcare Research and Quality. Pediatric Quality Indicators: Technical Specifications, AHRQ Quality Indicators, Version 2020.01 SAS. Rockville, MD: Agency for Healthcare Research and Quality, May 2020.

Agency for Healthcare Research and Quality. Prevention Quality Indicators: Technical Specifications, AHRQ Quality Indicators, Version 2020.01 SAS. Rockville, MD: Agency for Healthcare Research and Quality, May 2020.

Claritas. Claritas Demographic Data. Available: https://claritas.com/

HCUP Cost-to-Charge Ratio Files. Healthcare Cost and Utilization Project (HCUP). December 2019. Agency for Healthcare Research and Quality, Rockville, MD. Available: www.hcup-us.ahrq.gov/db/state/costtocharge.jsp.

HCUP State Inpatient Databases (SID). Healthcare Cost and Utilization Project (HCUP). 2017. Agency for Healthcare Research and Quality, Rockville, MD. www.hcup-us.ahrq.gov/sidoverview.jsp.

HCUP National Inpatient Sample (NIS). Healthcare Cost and Utilization Project (HCUP). 2017. Agency for Healthcare Research and Quality, Rockville, MD. www.hcup-us.ahrq.gov/nisoverview.jsp.

HCUP Nationwide Emergency Department Sample (NEDS). Healthcare Cost and Utilization Project (HCUP). 2017. Agency for Healthcare Research and Quality, Rockville, MD. www.hcup-us.ahrq.gov/nedsoverview.jsp.

Houchens R. Inferences with HCUP State Databases Final Report. HCUP Methods Series Report # 2010-05. ONLINE. October 12, 2010. U.S. Agency for Healthcare Research and Quality. Available: http://www.hcup-us.ahrq.gov/reports/methods.jsp.

IOM (Institute of Medicine). Future Directions for the National Healthcare Quality and Disparities Reports. Washington, DC: The National Academies Press, 2010.

APPENDICES

APPENDIX A: NATIONALLY WEIGHTED ANALYSIS FILES FOR THE NATIONAL HEALTHCARE QUALITY AND DISPARITIES REPORT

Beginning with the 2021 NHQDR, two versions of the nationally weighted NHQDR analysis file were used for reporting national QI estimates. Both used the SID and were limited to community hospitals, excluding rehabilitation and LTAC facilities. The two versions were specific to the data elements needed for the different QI modules:

• For the PQIs, IQIs, and area-based PDIs, the nationally weighted file included data from SID that included information on race/ethnicity of the patient. After hospitals that failed the race/ethnicity edits were excluded, all remaining discharges in the selected SID were weighted to the universe of community hospitals in the United States, excluding rehabilitation and LTAC facilities.


• Because the PSIs and related PDIs require knowing if the patient safety event occurred in the hospital and the timing of procedures, a second version of the nationally weighted file was created using the following criteria: (1) the SID included the data elements indicating diagnoses were POA, (2) the SID included information on day of principal and secondary procedure days, and (3) the SID included information on the race/ethnicity of the patient. Hospitals in qualifying SID were excluded if they failed POA and race/ethnicity edits. All remaining discharges were weighted to the universe of community hospitals in the United States, excluding rehabilitation and LTAC facilities.

In data year 2016, there were 49 eligible SID:

• 37 SID had all necessary data elements and were used for national estimates of all QIs (PQIs, IQIs, PSIs, and PDIs)
• 6 SID did not have POA and/or procedure date data and were thus excluded from national reporting of PSIs and related PDIs
• 6 SID did not have any race/ethnicity data (or were unavailable at the time the national file was built) and were not used for any national QI estimates (Figure A-1).

In data year 2017, there were 49 eligible SID:

• 40 SID had all necessary data elements and were used for national estimates of all QIs (PQIs, IQIs, PSIs, and PDIs).
• 6 SID did not have POA and/or procedure date data and were thus excluded from national reporting of PSIs and related PDIs
• 3 SID did not have any race/ethnicity data (or were unavailable at the time the national file was built) and were not used for any national QI estimates (Figure A-2).

In data year 2018, there were 49 eligible SID

• 41 SID had all necessary data elements and were used for national estimates of all QIs (PQIs, IQIs, PSIs, and PDIs).
• 6 SID did not have POA and/or procedure date data and were thus excluded from national reporting of PSIs and related PDIs
• 2 SID did not have any race/ethnicity data (or were unavailable at the time the national file was built) and were not used for any national QI estimates (Figure A-3).

Figure A-1. States Used for 2016 National QI Estimates

Figure A-2. States Used for 2017 National QI Estimates

Figure A-3. States Used for 2018 National QI Estimates

Evaluating the 2016-2018 Nationally Weighted NHQDR Analysis File

Table A-1 demonstrates the representation of the States (plus District of Columbia) used for each year by U.S. Census region.

Table A-1. Geographic Representation of the NHQDR Analysis File, 2016-2018

Table A-2 compares aggregated totals of various measures as a percent of the national measure.

Table A-2. Population Representation of the NHQDR Analysis File, 2016-2018

To further evaluate the reliability of national estimates produced by the nationally weighted NHQDR analysis file it is compare to the respective National Inpatient Sample (NIS). Tables A.3–A.18 contain the distribution of discharges in both files by key demographic and clinical data elements. Based on these analyses, the 2016-2018 nationally weighted analysis files appear to provide reliable national estimates when compared with the 2016-2018 NIS files.

Table A-3. Comparison of Census Region, NHQDR Analysis File and HCUP National Inpatient Sample, 2016

Table A-4. Comparison of Patient Age in Years at Admission, NHQDR Analysis File and HCUP National Inpatient Sample, 2016

Table A-5. Comparison of Patient Sex, NHQDR Analysis File and HCUP National Inpatient Sample, 2016

Table A-6. Comparison of Expected Primary Payer, NHQDR Analysis File and HCUP National Inpatient Sample, 2016

Table A-7. Comparison of Community Income Quartile Based on the Patient’s ZIP Code, NHQDR Analysis File and HCUP National Inpatient Sample, 2016

Table A-8. Comparison of Patient Race/Ethnicity, NHQDR Analysis File and HCUP National Inpatient Sample, 2016

Table A-9. Comparison of Location of Patient Residence, NHQDR Analysis File and HCUP National Inpatient Sample, 2016

Table A-10. Comparison of Length of Stay, Number of Diagnoses and Procedures on the Record, Total Charges, and Total Cost, NHQDR Analysis File and HCUP National Inpatient Sample, 2016

Table A-11. Comparison of Census Region, NHQDR Analysis File and HCUP National Inpatient Sample, 2017

Table A-12. Comparison of Patient Age in Years at Admission, NHQDR Analysis File and HCUP National Inpatient Sample, 2017

Table A-13. Comparison of Patient Sex, NHQDR Analysis File and HCUP National Inpatient Sample, 2017

Table A-14. Comparison of Expected Primary Payer, NHQDR Analysis File and HCUP National Inpatient Sample, 2017

Table A-15. Comparison of Community Income Quartile Based on the Patient’s ZIP Code, NHQDR Analysis File and HCUP National Inpatient Sample, 2017

Table A-16. Comparison of Patient Race/Ethnicity, NHQDR Analysis File and HCUP National Inpatient Sample, 2017

Table A-17. Comparison of Location of Patient Residence, NHQDR Analysis File and HCUP National Inpatient Sample, 2017

Table A-18. Comparison of Length of Stay, Number of Diagnoses and Procedures on the Record, Total Charges, and Total Cost, NHQDR Analysis File and HCUP National Inpatient Sample, 2017

Table A-19. Comparison of Census Region, NHQDR Analysis File and HCUP National Inpatient Sample, 2018

Table A-20. Comparison of Patient Age in Years at Admission, NHQDR Analysis File and HCUP National Inpatient Sample, 2018

Table A-21. Comparison of Patient Sex, NHQDR Analysis File and HCUP National Inpatient Sample, 2018

Table A-22. Comparison of Expected Primary Payer, NHQDR Analysis File and HCUP National Inpatient Sample, 2018

Table A-23. Comparison of Community Income Quartile Based on the Patient’s ZIP Code, NHQDR Analysis File and HCUP National Inpatient Sample, 2018

Table A-24. Comparison of Patient Race/Ethnicity, NHQDR Analysis File and HCUP National Inpatient Sample, 2018

Table A-25. Comparison of Location of Patient Residence, NHQDR Analysis File and HCUP National Inpatient Sample, 2018

Table A-26. Comparison of Length of Stay, Number of Diagnoses and Procedures on the Record, Total Charges, and Total Cost, NHQDR Analysis File and HCUP National Inpatient Sample, 2018

APPENDIX B: EMERGENCY DEPARTMENT RATES FOR SELECTED AHRQ QUALITY INDICATORS

The HCUP Nationwide Emergency Department Sample (NEDS) was used to examine national and regional differences in emergency department (ED) rates for selected AHRQ Prevention Quality Indicators (PQIs) and related Pediatric Quality Indicators (PDIs). The PQIs are measures of quality associated with processes and outcomes of care that occurred in an outpatient or an inpatient setting. The PQIs rely solely on hospital administrative data and, for this reason, are screens for examining quality that may indicate the need for more in-depth studies. Experts have suggested that using both inpatient and emergency room data may give a more accurate picture of avoidable visits/admissions for some ambulatory care sensitive conditions which can be identified by certain PQIs and PDIs.

The NEDS contains approximately 30 million ED events from about 950 hospital-based EDs. The NEDS includes information on ED visits that do not result in an admission (i.e., treat-and-release visits and transfers to another hospital) as well as discharge information on patients initially seen in the ED and then admitted to the same hospital. The NEDS was drawn from the SID and the State Emergency Department Databases (SEDD). Discharge-level weights included with the NEDS are used to produce national estimates. Several steps were taken to prepare the NEDS: (1) QI software review and modification, (2) acquisition of population-based data, (3) handling of missing data, and (4) identification of statistical methods.

1. QI Software Review. The AHRQ QI software v2020.1 PQIs and PDIs were developed for use with hospital inpatient discharge data. No guidelines for applying the AHRQ QIs to emergency department data were available when this analysis began. About 15 percent of records in the NEDS represent an ED admission. The selected QIs rely on the first-listed diagnosis code (DX1) to identify cases with the outcome of interest. For ED admissions, DX1 is the principal diagnosis code and reflects the condition established to be chiefly responsible for a patients’ admission to the hospital. Unfortunately, principal diagnosis is not clearly discernible for ED visits that do not result in admission. Coding instructions for outpatient data specify that the first-listed diagnosis is supposed to be the "reason for visit," which is different than the principal diagnosis. Even though DX1 in ED data is not necessarily the principal diagnosis, using DX1 preserves the concept from the PQI algorithm that the first code has higher priority than others. Therefore, this analysis used the first-listed diagnosis in the NEDS.


2. Acquisition of Population-Based Data. Similar to the development of the PQI and PDI rates based on the HCUP inpatient data, Claritas population data was used for the population denominators of the selected PQIs and PDIs.


3. Preparation of HCUP Data Next, a "hot deck" imputation method (which draws donors from strata of similar hospitals and patients) was used to assign missing values of patient age and sex. Patient age and sex are missing on less than 0.02 percent of the records in the 2017 NEDS.


4. Masking Rates for Statistical Reliability, Data Quality, and Confidentiality. PQI and PDI estimates were included in this analysis if they reached a threshold defined by a relative standard error less than 30 percent and at least 100 weighted cases in the denominator. Estimates that did not meet this threshold were suppressed and the corresponding table cell was marked with an asterisk.

Additionally, the NEDS was used to examine national and regional differences in emergency department (ED) visit rates for three additional condition categories: mental illness, substance use, and co-occurring mental illness and substance use; dental conditions; and injuries. The complete list of ICD-10-CM diagnosis codes for these analyses are listed below.

Mental Illness and Substance Use ICD-10-CM Diagnosis Codes

Table B-1. Mental Illness ICD-10-CM Diagnosis Codes

Table B-2: Substance Use ICD-10-CMS Diagnosis Codes

Dental Conditions ICD-10-CM Diagnosis Codes

K000, K001, K002, K003, K004, K005, K006, K007, K008, K009, K010, K011, K023, K0251, K0252, K0253, K0261, K0262, K0263, K027, K029, K030, K031, K032, K033, K034, K035, K036, K037, K0381, K0389, K039, K0401, K0402, K041, K042, K043, K044, K045, K046, K047, K048, K0490, K0499, K0500, K0501, K0510, K0511, K0520, K05211, K05212, K05213, K05219, K05221, K05222, K05223, K05229, K0530, K05311, K05312, K05313, K05319, K05321, K05322, K05323, K05329, K054, K055, K056, K06010, K06011, K06012, K06013, K06020, K06021, K06022, K06023, K061, K062, K063, K068, K069.

Injury ICD-10-CM Diagnosis Codes

S00-S99; T07-T34; T36-T50 with a 6th character of 1,2, 3, or 4 (except T36.9, T37.9, T39.9, T41.4, T42.7, T43.9, T45.9, T47.9, and T49.9 with a 5th character of 1, 2, 3, or 4); T51-T65; T66-T76; T79; T84.01; T84.02; and O9A.2-O9A.5.

Dementia ICD-10-CM Diagnosis Codes

Table B-3: Dementia ICD-10-CM Diagnosis Codes

APPENDIX C: STATISTICAL METHODS

This appendix explains the statistical methods and gives formulas for the calculations of standard errors and hypothesis tests. These statistics are derived from weighted SID and demographic population data provided by Claritas (a vendor that compiles and adds value to Bureau of Census data). Standard error calculations for the rates were based on the HCUP report entitled Inferences with HCUP State Databases Final Report (Houchens, et al., 2010). We will refer to this report simply as the SID Variance Report throughout this appendix. This method takes into account the cluster and stratification aspects when calculating these statistics on weighted data using the SAS procedure PROC SURVEYMEANS. For population counts based on Claritas data, there is no sampling error.

Even though the SID databases contain nearly all discharges from nearly all hospitals in the State, we treat the files as though they were drawn from an infinite population. We do not employ finite population correction factors in estimating standard errors. We take this approach because we view the outcomes as a result of myriad processes that go into treatment decisions rather than being the result of specific, fixed processes generating outcomes for a specific population and a specific year. We consider the SID to be samples from a “super-population” for purposes of variance estimation. Further, we assume the counts (of QI events) to be binomial.

1. Standard Error Estimates for Weighted Discharge Rates per 100,000 Population Using the 2017 Claritas Population Data for Area-Based QIs

The observed rate was calculated as follows:

Formula - this image shows the formula for observed rate.

w_i and x_i, respectively, are the weight and variable of interest for patient i in the NHQDR analysis file or SID. To obtain the estimate of S and its standard error, SE_s, we followed instructions in the SID Variance Report.

The population count in the denominator is a constant. Consequently, the standard error of the rate R was calculated as:

Formula - this image shows the formula for standard error.

Standard Error Estimates for Inpatient Rates per 1,000 Discharges Using Weighted Discharge Counts in Both the Numerator and the Denominator for Provider-based QIs

We calculated the observed rate as follows:

Formula - this image shows the formula for observed rate.

Following instructions in the HCUP SID Variance Report, we used PROC SURVEYMEANS to obtain estimates of the weighted mean, S/N, and the standard error of the weighted mean, SE_S/N. We multiplied this standard error by 1,000.

2. Significance Tests

Let R₁ and R₂ be either observed or adjusted rates calculated for comparison groups 1 and 2, respectively. Let SE₁ and SE₂ be the corresponding standard errors for the two rates. We calculated the test statistic and (two-sided) p-value:

Formula - this image shows the formula for significance tests.

where Z is a standard normal variate.

Note: the following functions calculate p in SAS and EXCEL:

SAS: p = 2 * (1 - PROBNORM(ABS(t)));

EXCEL: = 2*(1- NORMDIST(ABS(t),0,1,TRUE))

APPENDIX D: CAVEATS TO THE INTREPRETATION OF HCUP-BASED QI ESTIMATES REPORTED IN THE NHQDR

Some caution should be used in interpreting the QI statistics presented in the NHQDR. These caveats relate to inter-State differences in data collection and other more general issues.

State-Specific Rates

Organizations that collect statewide data generally collect data using the Uniform Billing formats (UB-92 or UB-04). However, not every statewide data organization collects all data elements nor codes them the same way. Uneven availability of a few data elements can underlie HCUP estimates, as noted next.

Number of Clinical Fields

One data collection issue relates to the number of fields that statewide data organizations permit for reporting patient diagnoses and procedures during the hospitalization. The mean number of diagnosis codes across all records in the 2017 SID used for the NHQDR was 11.33, while the mean number of procedure codes across all records was 1.61. Across the 2017 SID used for the NHQDR, there was a maximum of 111 diagnosis and 100 procedure fields.

Non-Resident Discharges

HCUP databases include discharges from all hospitals in a State, and may include non-residents, including foreign patients, which can bias the results for QIs using area-based denominators (State populations). We had no way to adjust the HCUP data to consistently exclude the non-resident discharges and include discharges for residents hospitalized in other States. Therefore, non-resident discharges were retained in the SID databases for the NHQDR analyses. Based on an analysis performed with the 2016 SID, the overall percentage of non-resident discharges within a State is 4 percent with a range from 1 percent to 45 percent. Most States were below 10 percent, but six States (MO, ND, SD, TN, VT, WV) and the District of Columbia had more than 10 percent of discharges in the SID that were for non-residents. The proportion of non-resident discharges and range across States in the 2016 SID is similar to that observed in prior years. This analysis was not repeated for States in the 2017 SID.

APPENDIX E: INPATIENT RATES FOR DELIVERY MEASURES

Table E-1: Delivery and Abortion ICD-10-CM diagnoses, ICD-10-PCS procedures, and DRGs

Table E-2: Pulmonary Embolism or Deep Vein Thrombosis ICD-10-CM Diagnosis Codes

Table E-3: Postpartum Hemorrhage ICD-10-CM Diagnosis Codes

Table E-4: Eclampsia or Preeclampsia ICD-10-CM Diagnosis Codes

¹ICD-9-CM: International Classification of Diseases, Ninth Revision, Clinical Modification.

² ICD-10-CM/PCS: International Classification of Diseases, Tenth Revision, Clinical Modification and Procedure Coding System.

³ Some diagnoses are exempt from having POA reported.

⁴ Community hospitals are defined by the AHA as "all non-Federal, short-term, general, and other specialty hospitals, excluding hospital units of institutions open to the public." Specialty hospitals included in the AHA definition of "community hospitals" are: obstetrics-gynecology, ear-nose-throat, short-term rehabilitation, orthopedic, and pediatric institutions. Community hospitals can include rehabilitation and long-term acute care (LTAC) facilities. These facilities can provide acute care services to patients who need long term hospitalization). Excluded from the AHA definition of "community hospitals" are long-term non-acute care hospitals, psychiatric hospitals, and alcoholism/chemical dependency treatment facilities.

⁵ HCUP Cost-to-Charge Ratio Files. Healthcare Cost and Utilization Project (HCUP). December 2019. Agency for Healthcare Research and Quality, Rockville, MD. Available: www.hcup-us.ahrq.gov/db/state/costtocharge.jsp.