A regional hospital system serving Kentucky and Indiana engaged EdjAnalytics to improve the targeting logic for marketing campaigns around elective procedures which are strong contributors to the hospital’s profitability. The elective procedures of interest were Hip Replacement, Knee Replacement, and Bariatric Surgeries. The previous targeting logic relied primarily on age for Hip and Knee procedures and BMI for Bariatrics. Using the complete medical history and demographics information from the hospital’s Electronic Medical Records system offered the opportunity to identify the specific individuals most likely to require each procedure, allowing the hospital to reach the audience most likely to convert and to reduce wasteful marketing spend.
Edj received data covering 7.7 million total procedures executed over five years as well as information on demographics, labs, vitals, encounters, diagnoses, and medications used to identify each patient’s surgical risk. The data science team tested various machine learning techniques and achieved the strongest results with an Extreme Gradient Boosting method which creates a series of decision trees in order to assess likelihood of a specific outcome. The outputs of the models are predictions based on probability calculated after passing observations through all the decision trees. Patients are then classified as High, Medium, or Low Risk based on their probability scores. The High Risk group captures between 70 and 78% of the executed procedures.
Edj concluded that age and vitals, along with variables unique to each procedure such as previous history of the procedure, osteoarthritis, and obesity were the most valuable in assessing a patient’s surgical likelihood. The final models included approximately 80 variables in order to produce the most accurate individual predictions. While age and BMI are important indicators of surgical risk (particularly for Bariatrics and Knee Surgeries), the Edj team revealed that models using only age and BMI generated predictions with significantly lower accuracy. Vitals such as Respiratory Rate, Blood Pressure, Heart Rate, Temperature, Oxygen Saturation, and Mean Arterial Pressure helped improve the model’s predictions because they contain valuable information about a patient’s general health.
Top 15 features ranked by relative contribution to model accuracy
The final models achieved AUCs between 0.88 and 0.93 on unseen data, indicating extremely high accuracy in the predictions generated. AUC is a measure of how well a model balances the tradeoff between maximizing the rate of True Positives and minimizing the rate of False Positives; the closer to 1, the more accurate the model’s predictions.
The output of the models can be used to assess surgical likelihood of an individual patient over time given changes in their medical history as well as to generate targeting lists for marketing campaigns around the benefits of each procedure.
In order to illustrate how the models score each patient’s likelihood of undergoing a certain procedure, consider this example showcasing two white, male patients in their mid-to-late seventies and their probabilities of undergoing a hip replacement procedure:
The patient who does not undergo the procedure visits the hospital every six months and has no recorded abnormalities in his medical history. The other patient presents with hip pain and abnormal labs which causes his risk score to increase significantly for 3 quarters, after which the procedure occurs. Following the procedure, the patient’s risk score decreases until an ER visit occurs and arthritis is diagnosed, increasing his risk score once again.
In order to illustrate how the model outputs can be used to generate targeting lists for marketing, examine the count of patients who are classified High, Medium, and Low Risk for Knee Replacement procedures across one calendar year:
Among the 1.9 million patients within the hospital provider’s reach, there are 33,500 patients who were classified as High Risk for Knee Replacement during the prior year, and only 633 of those have undergone the procedure. This yields 32,867 patients who are strong candidates for Knee Replacement and prime targets for a marketing campaign about the benefits of the surgery. Circulation may be expanded by including members of the Medium Risk group as well.
Edj is partnering with the client’s marketing team to apply business logic to the model outputs in order to generate targeting lists for upcoming campaigns around the elective surgeries explored in this case study. The team plans to execute a test to understand the impact of the communications by comparing conversion rates of those contacted against a stratified control group of patients who were also strong candidates for the procedure but did not receive the communication.