Babylon Health is best known for GP at Hand, a service which is supported by UK health secretary Matt Hancock and integrated into Samsung Health.

GP at Hand links patients with health experts 24/7 using video calls and can facilitate any prescriptions to be sent to local pharmacies. The service, however, has been criticised for an AI chatbot which repeatedly gave unsafe advice and for only taking on healthier, often younger individuals while redirecting cash away from local surgeries relied on by older and sicker patients.

Correct triaging is essential to ensure patients receive the appropriate care. As the world responds to the coronavirus pandemic, many of us will have seen the harrowing headlines from the worst-hit countries like Italy where doctors are having to essentially decide who is worth trying to save due to limited resources.

Having to make such decisions, on top of all the other pressures medical professionals are currently facing, is unimaginable. An automated system would help to reduce the mental impact from any doubt their decisions are correct.

The company used reinforcement learning, which uses rewards for completing tasks to incentivise an agent, to create their AI system.

Babylon Health’s agent learned “an optimised policy” based on 1,374 clinical vignettes crafted by experts. Each vignette was supported by 3.36 expert triage decisions on average, and each was independently reviewed by two clinicians.

The best performing model achieved an appropriateness score of 85 percent and a safety score of 93 percent, which is around the same for humans (84 percent appropriateness and 93 percent safety.)

If true, it’s an impressive result, but Babylon Health’s studies have been called into question in the past. Just three years ago, the company tried and failed to get a legal injunction to block the publication of a report from the NHS care standards watchdog.

In 2018, Babylon Health published a paper which claimed that its AI could diagnose common diseases as well as human physicians. The Royal College of General Practitioners, the British Medical Association, Fraser and Wong, and the Royal College of Physicians all issued statements disputing the paper’s claims.

As with the rest of Babylon Health’s solutions, there’s a lot of promise in what they’re aiming to do. However, the company’s history casts some doubt over whether these latest claims are as impressive as they seem.

You can find Babylon Health’s full paper on arXiv here (PDF)

The post Babylon Health says its AI can appropriately triage 85% of patients appeared first on AI News.

Babylon Health’s chatbot is available in the company’s GP at Hand app, a digital healthcare solution championed by health secretary Matt Hancock that was also integrated into Samsung Health since last year.

The chatbot aims to reduce the burden on GPs and A&E departments by automating the triage process to determine whether someone can treat themselves at home, should book an online or in-person GP appointment, or go straight to a hospital.

A Twitter user under the pseudonym of Dr Murphy first reached out to us back in 2018 alleging that Babylon Health’s chatbot was giving unsafe advice. Dr Murphy recently unveiled himself as Dr David Watkins and went public with his findings at The Royal Society of Medicine’s “Recent developments in AI and digital health 2020“ event in addition to appearing on a BBC Newsnight report.

Over the past couple of years, Dr Watkins has provided many examples of the chatbot giving dangerous advice. In one example, an obese 48-year-old heavy smoker patient who presented himself with chest pains was suggested to book a consultation “in the next few hours”. Anyone with any common sense would have told you to dial an emergency number straight away.

This particular issue has since been rectified but Dr Watkins has highlighted many further examples over the years which show, very clearly, there are serious safety issues.

In a press release (PDF) on Monday, Babylon Health calls Dr Watkins a “troll” who has “targeted members of our staff, partners, clients, regulators and journalists and tweeted defamatory content about us”.

According to the release, Dr Watkins has conducted 2,400 tests of the chatbot in a bid to discredit the service while raising “fewer than 100 test results which he considered concerning”.

Babylon Health claims that in just 20 cases did Dr Watkins find genuine errors while others were “misrepresentations” or “mistakes,” according to Babylon’s own “panel of senior clinicians” who remain unnamed.

Speaking to TechCrunch, Dr Watkins called Babylon’s claims “utterly nonsense” and questions where the startup got its figures from as “there are certainly not 2,400 completed triage assessments”.

Dr Watkins estimates he has conducted between 800 and 900 full triages, some of which were repeat tests to see whether Babylon Health had fixed the issues he previously highlighted.

The doctor acknowledges Babylon Health’s chatbot has improved and has issues around the rate of around one in three instances. In 2018, when Dr Watkins first reached out to us and other outlets, he says this rate was “one in one”.

While it’s one account versus the other, the evidence shows that Babylon Health’s chatbot has issued dangerous advice on a number of occasions. Dr Watkins has dedicated many hours to highlighting these issues to Babylon Health in order to improve patient safety.

Rather than welcome his efforts and work with Dr Watkins to improve their service, it seems Babylon Health has decided to go on the offensive and “try and discredit someone raising patient safety concerns”.

In their press release, Babylon accuses Watkins of posting “over 6,000” misleading attacks but without giving details of where. Dr Watkins primarily uses Twitter to post his findings. His account, as of writing, has tweeted a total of 3,925 times and not just about Babylon’s service.

This isn’t the first time Babylon Health’s figures have come into question. Back in June 2018, Babylon Health held an event where it boasted its AI beat trainee GPs at the MRCGP exam used for testing their ability to diagnose medical problems. The average pass mark is 72 percent. “How did Babylon Health do?” said Dr Mobasher Butt at the event, a director at Babylon Health. “It got 82 percent.”

Given the number of dangerous suggestions to trivial ailments the chatbot has given, especially at the time, it’s hard to imagine the claim that it beats trainee GPs as being correct. Intriguingly, the video of the event has since been deleted from Babylon Health’s YouTube account and the company removed all links to coverage of it from the “Babylon in the news” part of its website.

When asked why it deleted the content, Babylon Health said in a statement: “As a fast-paced and dynamic health-tech company, Babylon is constantly refreshing the website with new information about our products and services. As such, older content is often removed to make way for the new.”

AI solutions like those offered by Babylon Health will help to reduce the demand on health services and ensure people have access to the right information and care whenever and wherever they need it. However, patient safety must come first.

Mistakes are less forgivable in healthcare due to the risk of potentially fatal or lifechanging consequences. The usual “move fast and break things” ethos in tech can’t apply here. 

There’s a general acceptance that rarely is a new technology going to be without its problems, but people want to see that best efforts are being made to limit and address those issues. Instead of welcoming those pointing out issues with their service before it leads to a serious incident, it seems Babylon Health would rather blame everyone else for its faults.

Interested in hearing industry leaders discuss subjects like this? , , , AI &

The post Babylon Health lashes out at doctor who raised AI chatbot safety concerns appeared first on AI News.

Health secretary Matt Hancock has championed GP at Hand since 2017. The digital service has come into question for providing unsafe recommendations, and for taking money away from surgeries already struggling with cuts.

The service has operated through the Hammersmith and Fulham Clinical Commissioning Group (CCG) across London. Due to the popularity of the app, the CCG has ended up with a £21.6m funding gap while it plans £10m of additional cuts.

According to the CCG, it’s working on a “solution to the financial impact” of GP at Hand and expects reimbursement from NHS England. The CCG promises that patients in Hammersmith and Fulham will “not be disadvantaged” by the funding gap.

The Hammersmith and Fulham CCG’s total deficit swelled to £37m this year. Cuts are planned in a wide range of areas including overnight urgent care centres, out of hours GP services, radiology, cardiology, gynaecology, mental health services, dementia support, rehabilitation, and more.

Babylon said: “We fully expect Babylon GP at Hand to reduce the costs on the overall NHS,” adding the NHS’s financial structure is “not designed to meet the needs of people who choose digital-first care”.

GP at Hand has appealed primarily to the younger generation who are used to digital services. Babylon Health has, in the past, been criticised for picking healthier patients and leaving more complicated cases to surgeries. Due to each surgery being funded on a per patient basis, this means GPs are being left with less overall funding to cover patients which use their services more.

Ipsos Mori was commissioned by the NHS last year to assess the impact of Babylon Health’s partnership with the NHS. On Thursday, the research group published its findings and said there were “questions about the financial impact of the service on the wider health system”.

AI-powered healthcare solutions will one day fulfil their exciting promise, but rushing to market only risks patients’ wellbeing.

deepgeniusai.com/">AI & Big Data Expo events with upcoming shows in Silicon Valley, London, and Amsterdam to learn more. Co-located with the IoT Tech Expo, , & .

The post Babylon Health’s GP at Hand causes £21.6m funding gap appeared first on AI News.

The company’s GP at Hand app, which features the chatbot and can provide a video link with a doctor, was promoted by former UK digital secretary Matt Hancock and is backed by the NHS.

Furthermore, Samsung partnered with Babylon Health last year to integrate the service with compatible Galaxy devices

Babylon Health’s AI-powered chatbot aims to provide guidance on how a patient should proceed. The idea is to reduce the pressure on the health service from patients whose symptoms could be dealt with at home.

In theory, it’s a great idea and will one day be how we access healthcare. However, as AI News has reported in the past, it’s currently not robust enough and has presented advice which could result in fatalities.

Twitter user ‘Dr Murphy’ has been highlighting the failures of GP at Hand over the past few years.

One concerning example provided by Murphy was of an obese 48-year-old heavy smoker patient that’s presented himself with chest pains. Anyone with common sense would advise calling 999 immediately. However, the chatbot suggested booking a consultation “in the next few hours”.

This particular issue has since been rectified, but similar dangerous scenarios are still being discovered. The situation is a clear example of why rigorous, proven, and independent testing needs to be established before AIs can begin offering life or death advice.

In June last year, Babylon Health held an event bragging about its AI capabilities beating trainee GPs at the MRCGP exam used for testing their ability to diagnose medical problems.

Trainee GPs have an average passmark of 72 percent over the past five years. “How did Babylon Health do?” asked Dr Mobasher Butt at the event, a director at Babylon Health. “It got 82 percent”.

In recent days, the video of the event has been deleted from Babylon Health’s YouTube account. Even more intriguing, the company has removed all links to news coverage of it from the ‘Babylon in the news’ part of its website.

Here’s a before and after comparison:

In the left-hand screenshot, we can see coverage of the event prior to deletion. In a screenshot from today on the right, articles from Forbes and Newsweek have been erased.

The quiet deletion of the company’s AI test event from last year appears to be some admission that findings presented at the event were misleading. As of writing, the chatbot is still live.

We reached out to Babylon Health this morning for clarification on the reason for the deletions, but we’re reaching the end of play and have yet to hear back.

Update Babylon Health issued the following statement: “As a fast-paced and dynamic health-tech company, Babylon is constantly refreshing the website with new information about our products and services. As such, older content is often removed to make way for the new. “

deepgeniusai.com/">AI & Big Data Expo events with upcoming shows in Silicon Valley, London, and Amsterdam to learn more. Co-located with the IoT Tech Expo, , & .

The post Babylon Health erases AI test event for its chatbot doctor appeared first on AI News.

Following our story yesterday on Samsung’s partnership with Babylon Health, an NHS consultant reached out with his (puzzling) experience of Babylon’s technology:

Just when you thought the @babylonhealth #AI #Chatbot couldn’t get any worse…

I gave it the most basic of medical presentations; ‘I have a nose bleed’.

Sit back & watch 130 seconds of the most bizarre triage you will ever come across… #NoseBleed #GPatHand #FlawedAI pic.twitter.com/53YPoZIOUF

— Dr Murphy (@DrMurphy11) May 13, 2018

In the video, the AI is told ‘I have a nose bleed’ – a symptom which a medical professional would be able to diagnose quickly. What follows is over two minutes of bizarre questions resulting in the AI calling the symptoms ‘quite complex’ and failing to offer any possible causes.

The service will connect a patient to a qualified doctor for treatment options and prescriptions, but it’s expected to list potential causes for presented symptoms to save the inadvisable Googling most of us have tried.

AI in healthcare is supposed to improve efficiency and reduce the burden on primary care services. In this situation, the patient would have struggled with their nose bleed answering many questions for a failed diagnosis which should be trivial. In the end, they’d have to book a GP appointment anyway.

These services make their money off paid appointments, so there’s always going to be the concern a patient could be led on a futile journey only to be told they need to pay for an appointment. A person would be using the service because they’re concerned, so the potential for exploitation is high.

As of writing, there are five complaints raised against Babylon listed as ‘informally resolved’ with the Advertising Standards Authority (ASA) for potentially misleading claims. At least one, in correspondence seen by AI News, was that use of the chatbot is ’100% safe’ — a claim which has since been pulled from their advertising materials.

The study (PDF) behind the claim says it recruited 12 clinicians with at least four years of clinical experience — along with 17 nurses — to perform triage in a semi-naturalistic scenario. Actors performed mock consultations based on patient vignettes with both clinicians and nurses.

Here are the results:

Nurses — 73.5% accurate, 97% safe

Doctors — 77.5% accurate, 98% safe

Babylon Health — 90.2% accurate, 100% safe

All the sessions were timed and Babylon was fastest in 89 percent of cases; taking a median time of 01:09. On average, a consultation with a doctor took 03:12. A nurse took around 02:27.

The earlier video results in a failed diagnosis which can neither be classed as safe or unsafe advice. However, it doesn’t take even basic medical training to question the safety of the following advice:

Is this another negligent #Triage from the @babylonhealth #GPatHand #AI #Chatbot App?

48yr old obese 30/day male smoker develops sudden onset central chest pain & sweating….

I say call 999, the Babylon App says see your GP… pic.twitter.com/VcOqs9JOhP

— Dr Murphy (@DrMurphy11) March 22, 2018

Algorithms for all of the provided examples have since been updated, but you can see a collection of these problems here.

In MDDUS’ assessment, there are three key issues with remote consultations in general:

• • lack of prior knowledge of a patient

• • ensuring adequate consent
  • providing continuity of care

“Some systems for remote consulting use online forms based on algorithms in order to direct diagnostic questioning,” wrote the MDDUS. “Such systems present numerous inherent risks such as a lack of ‘relevant negatives’ – you only see what is on the form.”

“There is also increased potential for misunderstanding in regard to how the patient is interpreting questions, and barriers within the system to seek further clarification.”

Furthermore, the CQC (Care Quality Commission) recently criticised some online providers for failing to notify the patient’s regular GP when they issued prescriptions. This subsequently affects the ability for primary care services to have a full record of what medications a patient is using and how they responded.

Campaigning organisation Keep Our NHS Public also questioned the practices of some of these services.

In a poster (PDF), the campaigners claim GP at Hand “is using IT to hoover up NHS patients all round London, using NHS money.”

Even worse, they accuse it of ‘cherry-picking’ patients to reduce costs.

“GP at Hand seems to be deliberately targeting healthy young people. They won’t take you on if you’re pregnant, frail and elderly, or have a terminal illness. They don’t want patients with complex mental health problems, drug problems, dementia, a learning disability, or safeguarding needs. We think that’s because these patients are expensive.”

AI has a lot of potential in healthcare to improve efficiency, diagnosis, and treatment — but this serves as a reminder of how robust systems need to be for that potential to be reached. Otherwise, they can just be dangerous.

The post Opinion: The risks of AI in remote medical consulting appeared first on AI News.

AI News first covered GP at Hand in November last year.

“GP at Hand is a window into what the NHS of the future will look like,” said Dr Howard Freeman MBE, senior GP. “When innovative NHS GPs embrace Babylon’s technology to make life better for their patients, the sky is the limit.”

The service uses AI to determine if a patient’s symptoms require further attention before putting them in touch with a GP using video chat if necessary.

Where appropriate, prescriptions can be sent automatically to a pharmacy of choice — or a patient can be booked in for a physical examination at a practice.

Dr. Ali Parsa, Babylon’s Founder & CEO, says:

“Babylon’s mission is to make healthcare accessible and affordable and to put it into the hands of everyone on Earth. Samsung’s vision for empowering individuals and transforming healthcare, partnered with the company’s illustrious history of technological innovation, constant focus on customer satisfaction and truly global reach makes it a perfect fit with our values and mission.

It’s very exciting to know that millions of Samsung users will soon be able to better manage their health using Babylon’s services as we deliver personal health assessments and treatment advice via their Samsung Galaxy devices.”

Samsung will be integrating Babylon Health’s service into the built-in Samsung Health app on compatible Galaxy devices. The service will not be free: users can decide between a £50 per year subscription, or pay £25 for a one-off appointment.

Kyle Brown, Head of Technology and Services at Samsung UK, adds:

“We’re excited to be welcoming ‘Ask an Expert, powered by Babylon’ to the Samsung Health app. Now our customers will be able to look after their health from wherever they are – whether it’s checking a symptom or talking to a doctor – all within a few simple taps.

The availability of the Babylon service within the app is another milestone for Samsung as we move towards a more connected, healthy world.”

Health startup Babylon has been expanding rapidly as people look for alternatives to overburdened traditional health services.

Dame Barbara Hakin, Former GP and National Director in NHS England, comments:

“I know just how difficult times are for GPs these days and how busy they are. GP at Hand, in addition to being very convenient for patients, can help the service given the recruitment crisis we know is facing us.

This technology can take more of the strain and ensure the best information and insight is available ahead of consultations which will then relieve some of the pressure on hard-pressed clinicians.”

While the service is launching in the UK, Babylon Health is looking to expand its partnership with Samsung worldwide.

Babylon Health recently signed a deal with social giant WeChat in China to offer its services in the country; showing its desire to make healthcare more accessible to everyone around the world.

One day, it’s not hard to imagine a subscription to a service like GP at Hand being able to quickly connect patients with local doctors for advice and treatment even while travelling in other countries. That could offer a lot of peace of mind.

What are your thoughts on the partnership?

  deepgeniusai.com & ac

The post Samsung partners with Babylon Health to offer AI consultations appeared first on AI News.