Could ChatGPT soon be used as a diagnostic tool?
A research team in London is looking into whether or not ChatGPT can be used as a diagnostic tool for medical learners and clinicians.
Dr. Amrit Kirpalani, a paediatric nephrologist at Children’s Hospital at London Health Sciences Centre, and his team of Schulich Medicine students, wanted to explore how ChatGPT could help learners and professionals.
To do so, they inputted 150 medical case challenges into ChatGPT, and the answers were analyzed based on accuracy, relevance and cognitive load.
According to Kirpalani, the results concluded that ChatGPT 3.5 accurately diagnosed 49 per cent of the 150 case challenges, but that the overall reliability was limited.
The team found that ChatGPT struggled with interpreting numerical values, neglected data points key to diagnoses and often overlooked important details which led to inaccurate assumptions.
However, the team was interested in ChatGPT’s ability to take complex cases and explain answers in a way that is understandable to those who may not possess complex medical knowledge.
“The cognitive load was relatively low, meaning that it was easy to understand,” said Branavan Nagarajan, a third-year Schulich Medicine medical student involved in the project. “This means ChatGPT can be a powerful tool for the learning process, helping both pre- and early-year medical students understand complex concepts because they are presented in a more manageable format. But when you combine this with the issues of accuracy, it highlights the importance of having someone with a higher level of medical knowledge to validate the outputs.”
Going forward, the research team believes there are exciting opportunities in terms of next steps.
“We are curious to analyze the performance of future iterations of ChatGPT, to see how much better it does in terms of relevancy, accuracy and cognitive load,” said Nagarajan. “There could be the opportunity to expand the parameters the program receives, allowing us to incorporate more numerical data, images and even clinical content such as patient history. We would be curious to see how this could impact the program’s diagnostic accuracy.”
Kirpalani said he would like to explore the potential of incorporating ChatGPT and AI literacy into formal medical training, adding, “Our results illustrate the importance of understanding how to use these AI tools properly to avoid receiving or spreading misinformation. A tool is only powerful if we know how to use it accurately.”
CTVNews.ca Top Stories
DEVELOPING MPs to vote today on Poilievre's motion of non-confidence in PM Trudeau
Members of Parliament will vote today on a motion that could — but likely won't — bring down the minority Liberal government, and the Conservatives have already put the House of Commons on notice that more such votes are coming.
'She was victimized by a predator': B.C. court reverses transfer of $1.4M townhouse in elder abuse case
A man who "systematically isolated, manipulated, deceived, abused, and exploited" an elderly North Vancouver woman has lost his ownership stake in her home.
Doug Ford says he wants to build a tunnel under Hwy. 401
Ontario Premier Doug Ford says he wants to build a tunnel under Highway 401 that would stretch from Brampton to Scarborough.
Yogurt recalled in Canada over risk of illness
A major yogurt maker is recalling one of its brands in Canada over concerns that it may cause illness in immunocompromised people.
Mortgage changes: Cheaper entry into housing market at steeper costs
Mortgage rule changes allow easier entry into the housing market with lower monthly payments, but also an increased cost of repaying a mortgage.
Coverage denied: Canadians hitting roadblocks with insurers after installing solar panels on their homes
More Canadians are installing solar panels on their homes, but some are facing challenges when it comes to getting home insurance after the panels are put in.
Population growth slows for first time since start of pandemic as Canadian government clamps down
Population growth in Canada slowed slightly in the last quarter of the year as the government made efforts to reduce temporary migration into the country, Statistics Canada revealed Wednesday.
Doing this for 20 seconds a day can relieve stress and anxiety
A 20-second session of self-compassionate touch reduced stress, increased kindness to participants and improved mental well-being, according to a recent study.
Israeli army chief says military is preparing for possible ground operation in Lebanon
Israeli army chief says military is preparing for possible ground operation in Lebanon