Ron,

This is a very important and challenging question, for which overly simple measures may be misleading. Here are some thoughts based upon my (somewhat limited) experience in this area.

1. CASE MIX: A single metric applied to all providers may be misleading. Some providers or clinics deal largely with the first visit for acute trauma, whereas others have longer term chronic patients; this may affect appropriateness of opioids. My case mortality rate when I was running in ICU would certainly be unacceptable for my current clinical practice. In several projects, we employed adjustment methods analogous to ascertaining such as how many prior visits to other providers involved an opioid prescription.

2. DO NOT AUTOMATICALLY APPLY TREATMENT GUIDELINES TO QUALITY MONITORING: Many treatment recommendations are “strong suggestions” and not easy to apply without understanding the individual case features (e.g., the clinician may have good reasons to feel it does not apply to the instant case). Some guideline recommendations depend upon data difficult to automatically ascertain (e.g., do not continue PT unless there has been a response to prior PT). Others, however, may be automatically and electronically ascertained for monitoring. (E.g., beta blockers after MI, follow-up visit after CTS surgery are easily identified from electronic records). See 2 articles below that describe this in more detail from the work we did with Rand Corporation. 

3. DECIDE WHETHER TO FOCUS ON THE PRACTICE OR ON THE INDIVIDUAL: Sometimes it is advisable to avoid the temptation to focus on identifying the “single outlier clinician”. Rather, in a widely dispersed system such as Banner, more stable and precise estimates should be obtainable for a clinic rather than for each individual. Improving the Average may sometimes be better than trying to find the severe outlier.

4. FOCUS ON MEANINGFUL SAMPLE: Rather than obtaining limited quality data on every case, it is sometimes better to systematically select appropriate sample cases for more in-depth analysis of guidelines compliance. We continue to use this approach in quality improvement work in workers compensation.

5. SIMPLE BUT SPECIFIC RULES: NIOSH sponsored work with both ATS and ACOEM to develop extremely simple decision support system rules that are very easily based upon automated review of EHR records. (Summaries of Bob McClellan and my committees’ recommendations are shown below).

6. PARTNERING WITH INSURERS OR EMPLOYERS: In the past, had the pleasure of working with two projects directly relevant to the employment setting. Both used uniquely occupationally relevant outcomes such as worker return and persistence at work or insurer reserve fund allocation. For externally facing clinics, it is necessary to establish strong but protected collaborations with a corporate employer or with a large insurer to obtain the data since the treating clinic rarely has this type of information. Accomplishing this in an ethical fashion requires extremely careful consideration about who will see the data; the “honest broker” approach used in many research studies may be particularly applicable.

7. SO WHAT? Ultimately, the monitoring process should depend on what you will do with the information obtained.

8. THE FUTURE?? ChatGPT

9.(Sorry for being so wordy! This is an extremely important & interesting but challenging problem). The references below are not necessarily the best, but are easy for me to find since they sit on my computer.

Phil Harber 

Dictated to Dragon voice dictation system. I apologize for any overlooked errors.

🌵🌵🌵🌵🌵🌵🌟

1. Quality measures for the diagnosis and non-operative management of carpal tunnel syndrome in occupational settings.Nuckols T, Harber P, Sandin K, Benner D, Weng H, Shaw R, Griffin A, Asch S; Carpal Tunnel Quality Group. J Occup Rehabil. 2011 Mar;21(1):100-19. doi: 10.1007/s10926-010-9260-6. PMID: 20737200

2. Selection of workers' compensation treatment guidelines: California experience. Harber P, Wynn BO, Lim YW, Mattke S, Asch SM, Nuckols TK. J Occup Environ Med. 2008 Nov;50(11):1282-92. doi: 10.1097/JOM.0b013e3181841fb4. Https://pubmed.ncbi.nlm.nih.gov/19001954/

3. Recommendations for a Clinical Decision Support System for Work-Related Asthma in Primary Care Settings. Harber P, Redlich CA, Hines S, Filios MS, Storey E. J Occup Environ Med. 2017 Nov;59(11):e231-e235. doi: 10.1097/JOM.0000000000001182. 

4.Using Electronic Health Records and Clinical Decision Support to Provide Return-to-Work Guidance for Primary Care Practitioners for Patients With Low Back Pain. McLellan RK, Haas NS, Kownacki RP, Pransky GS, Talmage JB, Dreger M. J Occup Environ Med. 2017 Nov;59(11):e240-e244. doi: 10.1097/JOM.0000000000001180..