No wonder so many in O&P are not involved in collecting formal outcome data on their patients; we are confused as to why, how and what. Those supposedly in-the-know are confused on what is an outcome and how to measure it. As defined by the Oxford dictionary, an outcome is, “…the way a thing turns out, a consequence.” Meriam-Webster defines an outcome as, “…something that follows as a result or consequence.” And the Cambridge dictionary defines an outcome as, “…one of the possible results in a probability (=likelihood) experiment.”
The April 2020 edition of the AOPA Almanac https://issuu.com/americanoandp/docs/4_-_april_final has stated. “As O&P facilities adopt evidence-based practices (EBP), measuring outcomes has become an essential part of patient care.” This is true, measuring outcomes is an essential part of patient care, but is impossible for O&P to adopt EBP at this time. Why? In short because the evidence O&P have published have been nullified by the 2017 CMS Consensus Report (https://www.cms.gov/Medicare/Coverage/DeterminationProcess/downloads/LLP_Consensus_Document.pdf) and the 2018 AHRQ report (https://effectivehealthcare.ahrq.gov/products/prosthesis/research) on lower limb prosthetics; we must start over. Why O&P cannot adopt an EBP will be discussed in more detail in next month’s article. But first let’s talk about why we should collect patient outcome data, how to measure them as this is the first step in getting to EBP protocols, and what outcomes to measure.
Why? - In my May post http://vanguardmetrics.biz/blog/april/ I noted the issues pointed out in detail, bullet point by bullet point, by the CMS relative to the provision of lower limb prosthetics. This report should be mandatory reading for everyone in O&P. Like it or not, this is your customer talking to you, pay attention! Yes, I said your customer, they pay you (the definition of customer) and they are your best, largest, and most powerful customer. They pay you in 14-days once you submit the claim electronically and they influence all other payers’ policies and reimbursements.
If we don’t like the next draft LCD, we had better be prepared with solid objective data to back up our position. That’s of course if the data backs up what we think, feel, believe, and guess. Everyone you ask THINKS what they do matters, everyone FEELS they are the best at what they do, everyone BELIEVES they do right by the patient, and everyone GUESSES their business is doing well. These are all very subjective emotional responses. But what do we really KNOW? What do we objectively know about providing an MPK to a patient? I recently had a young practitioner challenge me on this and stated she knew patients performed better and were more stable. The reply is which patients will benefit and which will not? Can you please define for me perform? What does more stable mean? What systemic medical condition are you addressing with the MPK? We need to focus the benefits of whatever we are doing with functional endpoints, which relate to the patients IADL’s allowing them to remain independent in their own home. We need a body of data to be able to answer these questions, which means we have to measure and collect this data as the cost of what we do increases.
How? –Today’s suggestions published in various industry trade journals and even presentations at the most recent AAOP meeting in Chicago are haphazard and anecdotal. This leaves those who read the journals or attend the meetings confused as to how to implement a program when they get home or back to the office the next day. The suggestions are, you should just start small and collect a few measures and build from there (https://issuu.com/americanoandp/docs/7_-_july), or they provide an anecdotal example of what a particular company is doing. None, as far as I am aware from what has been revealed at meetings or published in trade journals, is able to provide data on subsets of cohorts of patients. These all appear to be single subject data sets providing no guidance on future care standards.
While it is great we are being urged to collect data, the collection of disparate data from disparate sources makes it almost impossible to determine a standard of care, which the CMS Consensus report has said really does not exist. Further any attempt to combine this disparate data is at risk of being deemed invalid and not reliable.
Vanguard Metrics and Analytics has answered the above question of how, by constructing a consistent methodology, which is simple for the O&P practice to deploy so we are all collecting the same data.
What? – The biggest area of confusion is understanding the difference between a diagnostic clinical functional assessment and a patient-centric functional outcome, or the result of doing something. The “doing something” is the diagnostic clinical functional assessment. These assessments are things like a TUG, 6MWT, 2MWT, FSST, ABC, etc. tests. These are clinical tests administered by the clinician to help determine the functional abilities/inabilities of the patient. We do these to help form our clinical interventions; i.e. what device to provide. The “result” of doing something, the outcome or the human impact of what you have done, is essentially the report card the patient provides us. It’s not the report card we give ourselves! But this is not the patient’s freewheeling opinion.
There are mechanisms referred to as patient self-reports, which are deemed valid and reliable. These are surveys, with a defined set of questions, which relate to actual function and independence or quality of life, which have been validated as being neutral. In other words, the verbiage of the questions and possible answers do not lead the respondent to a particular answer. Thus, subjective responses become objective, valid, and reliable if administered so as not to inject bias. Then, these surveys need to be re-administered electronically (in vivo) at defined intervals of time to track the longitudinal changes in functional ability and quality of life.
The current examples provided today would suggest we should test against the test, which is neither valid or reliable and is highly biased. What I mean by testing against the test is the re-administering of the same diagnostic clinical functional assessment tests at some undefined point in time. This is highly biased as the same person administering the original test is more than likely administering the subsequent test. The tester has a bias, unconscious (for more on unconscious bias see https://www.amazon.com/Undoing-Project-Friendship-Changed-Minds/dp/0393254593) though it may be, in the result of the test. Additionally, these tests are performed inside of the clinical setting (in vitro) and are not real-world representations of actual function. Further these tests do not measure the psychosocial aspect of the patient and the care they have been provided. This is the quality of life aspect. For example, a patient may score well enough in a clinical functional test to be categorized as a K3 but if they are so self-conscious of themselves because of their body image they only use the prosthesis in or about their home are they actually functioning as a K3? Remember, the two government reports have indicated there is a lack of linkage between a patient’s stated potential function and their actual function so testing against the test, i.e. AMP test today and AMP test three weeks from now, doesn’t matter. Also, keep in mind K-levels are not a measure of function. At best they are a “potential” indicator of function but in reality, the K-levels are artificial barriers to reimbursement designed and implemented by the CMS in 1994 to mitigate their financial exposure to technology. O&P clinics who can demonstrate value going forward will be able to change the narrative away from price (discounts) to value armed with data. But a word of caution, be ready if the result is less than you expected. Often what we think, feel, believe, and guess, is not reality. Do not allow the dictates of the past to become the dogma preventing change from happening!
Be willing to look at the results dispassionately, ask the hard question – why? Make some hypotheses about what could be changed, implement the changes, then re-measure, reanalyze, and change again. Measuring patient outcomes are essentially your quality management program. This is continuous learning and is a value orientation you can market and honestly say, we are the best, let me show you!
Vanguard Metrics and Analytics has designed a program following the recommendations found within the CMS and AHRQ reports. It is simple for both the clinic to implement and for the patient to follow. For more information on how you can easily begin collecting and using patient-driven data contact