Imagine you’re feeling achy. You have a cough, and you might have a fever. It’s flu season, so you want to have a doctor check you out. Almost a quarter of Americans now opt for a telehealth visit, which public health experts say has helped to keep sick people out of community spaces where they can spread illness.
But the transition to telehealth visits also means rewriting some of the rules clinicians depend on to make diagnostics decisions.
“We know that telemedicine is working identifying high-risk patients, but we know that we can do better also,” said Zane Billings, an epidemiology and biostatistics doctoral student in the University of Georgia’s College of Public Health.
Zane Billings
Billings and co-authors from UGA Public Health recently published a study in the Journal of the American Board of Family Medicine that tests the accuracy of existing clinical decision rules for flu in a telehealth setting using only patient-reported symptoms.
A clinical decision rule (CDR) is a diagnostic tool that uses symptoms, and sometimes tests, to help clinicians determine how likely it is that a patient has a particular disease, and more importantly, how severe that case could be. There are validated clinical decision rules for flu, but these are based on the clinician being able to assess a patient in person – for example, take their temperature or listen to their breathing.
But when that patient-clinician interaction happens over a screen, and the rule relies on only what information the patient can share, do the rules hold up?
To answer this question, the researchers turned to data collected from 250 students who visited a university health center between December 2016 and February 2017 with a host of symptoms that could be flu. Using detailed information on patient- and clinician-reported symptoms, as well as the patient’s actual flu diagnosis, they could analyze often how patients and clinicians agreed on their likelihood of flu.
“What we found was that for most of these symptoms, patients and clinicians did not agree very often,” said Billings.
This is important, he says, because patient and clinician disagreement may undermine the validity of current CDRs for flu.
“We want to know if a patient goes to a telemedicine appointment, can the patient self-report their symptoms and we still get an accurate diagnosis with that CDR?” said Billings.
“If patients and clinicians are agreeing with each other on what symptoms the patient has almost all of the time, that means you can pretty much drop in those patient-reported symptoms instead. Whereas if patients and clinicians systematically disagree, we then wouldn’t expect those diagnostic rules to work anymore.”
The next question the research team had to answer was whether a new CDR for flu could be developed based on patient-reported symptoms alone.
Using machine learning to create a new tool
Using three statistical methods, the team created a predictive model using all the information they could glean from patient reports to give them a percent chance that someone has flu. They also ran the model using clinician-reported symptoms, which, overall, yielded more accurate diagnoses.
Overall, the researchers couldn’t create a perfect fit to the patient data, which means that their new model isn’t ready for use in clinical practice. Billings says that there are several possible reasons why pulling an accurate diagnosis from the data proved challenging – in particular the study’s sample.
College-aged students tend to be healthy overall and at low risk for flu, and their symptoms can be harder to pin down.
“If they do get the flu, it usually won’t be very severe. They might not be reporting as many symptoms as we might see for other people. Whereas if you have little kids and elderly people who get the flu, they’re often reporting more symptoms. It would be easier to diagnose them with the flu based on the information that we can get,” he said.
Billings and his co-authors would like to see this work replicated in a sample that includes more people who are at higher risk of severe cases of flu because that is the end goal of creating an accurate CDR for telehealth – to determine who needs more intensive care and who should stay at home.
“Because if those people stay at home, people who aren’t in danger of complications stay at home, they’re not spreading the flu or whatever respiratory disease to other people,” said Billings.
The paper, “Use of Patient-Reported Symptom Data in Clinical Decision Rules for Predicting Influenza in a Telemedicine Setting,” can be access online here: https://www.jabfm.org/content/jabfp/36/5/766.full.pdf
Co-authors include Annika Cleven, Jacqueline Dworaczyk, Ariella Perry Dale, Mark Ebell, Brian McKay, and Andreas Handel.
