How We Obtain Our Measures: Advantages and Limitations

All of the measures we report are obtained from the medical records of our partner practices and systems, not from insurance claims. This approach has remarkable advantages but it also has some limitations that we discuss below. Among participating practices with electronic medical records (EMRs), we obtain these measures on virtually all diabetic patients between the ages of 18 and 75 who visited the same doctors’ office at least twice during calendar year 2007. We exclude only patients whose diagnosis is “diabetes of pregnancy”. For partner practices that do not have EMRs, the same measures were collected using a standard protocol from a random sample of 50 charts of diabetes patients.

Both kinds of practices, those with and without EMRs, provide information about their patients’ neighborhoods, including average household income and maximum educational attainment, using information from the year 2000 U.S. census. No identifiers for patients or their doctors are shared with anyone outside of the clinical practice or health care organization. The Community Health Checkup is based on care provided to about 25,000 adult patients with diabetes in the region.

Advantages of our records-based approach

As mentioned above, our medical records-based approach has remarkable advantages but also some limitations as compared to using insurance claims. Advantages of our EMR-centered approach include our ability to: 1) measure our achievement on all of our patients, regardless of how they are insured, whether they have health insurance at all, or whether they change their insurance status; 2) accurately link doctors and their patients, facilitating appropriate attribution of care to specific doctors and their group practices; 3) obtain actual test results, and not simply whether the test was performed, allowing us flexibility in the standards (target values) we choose to establish, and to adopt different standards over time or for different interested users of our information; 4) obtain records of doctors’ prescriptions of medications, enabling us to report their intentions regarding treatments; and 5) report all information in a timely way (not requiring the practice’s submission and adjudication of insurance claims), enabling us to provide useful feedback for quality improvement to practices and practice leaders, and to patients.

It is noteworthy that this first Community Health Checkup Report, prepared during the second quarter of 2008, covers the entire calendar year 2007. In contrast, the most current claims-based data for national comparison purposes (see Part Two, Section H) pertain to actions and outcomes made a year earlier, in 2006. This unfortunate delay reflects the realities of insurance claims submission, processing, adjudication, analysis, and reporting. And, in the best of circumstances, this occurs without the benefit of knowing how we’re doing with uninsured patients, or those who have switched insurers.

Limitations of our records-based approach

In general, the limitations of the EMR-based approach, or to any practice-based approach in most organizations, relate to under-reporting performance on our achievement standards. These limitations theoretically can be quite important, and they can vary from standard to standard, and from one health care organization to another. Because many patients are free to obtain their care in different places, an approach that relies on practice-centered reporting relies on two factors: 1) whether the patient received a relevant service outside the home practice; or 2) if he/she did, whether the relevant “outside” service is documented in the medical record of the practice. Often, in our highly fragmented health system, even if a relevant test or treatment was received by the patient outside of his/her practice, details of the service are not communicated to the home practice or health care organization. For our diabetes measures and standards, this is likely to be an especially important problem in documenting eye examinations, because eye exams frequently are performed at free-standing eye clinics, leading to an under-estimate of practice achievement for this standard. In addition, until our practices have complete information on medication prescriptions that are filled by patients, we are unable to report “filled prescriptions”, instead being limited to documented doctors’ prescriptions for medications.

Exceptions to this “under-reporting” limitation occur in true systems of care, such as health maintenance organizations, where bills are generated for services obtained outside of the home practice, and documentation of these services is simpler. In Greater Cleveland, Kaiser Permanente is an example of a system in which reporting should be more complete.

At the other extreme are small practices that are not connected to larger organizations with sophisticated information systems or patient registries. In these types of practices, more of the relevant services are likely to be obtained outside of the home practice, creating challenges to comprehensively collecting information that is important for clinical performance measurement and reporting. Nonetheless, as documented by our region’s relatively resource-poor Federally Qualified Health Centers, committed smaller practices can collect and document relevant information in relatively inexpensive patient registries.