The advent of artificial intelligence (AI) in healthcare is truly upon us. Used safely and responsibly, it has the potential to solve long-established challenges and create new opportunities to achieve improved patient outcomes and more frictionless experiences for patients and physicians, all at a lower cost.
However, while Optum’s Survey on AI in Healthcare for 2020 showed healthcare leaders to be optimistic about AI’s benefits, three out of four executives reported being wary about bias creeping into AI’s results. Considering the widespread awareness of AI’s safety challenges, the fact that even some of the world’s largest and most established organisations struggle to overcome them are testament to their complexities.
So, if big companies are struggling with AI safety, how are innovative digital health startups operating on the frontlines of technological discovery approaching the same challenges? In the below interview, Harvey Williams, Data Scientist at Cambridge-based, deep tech startup electronRx shares his experience.
electronRx’s work focuses on digitising the human body by sensing the physiological environment and applying state-of-the-art signal processing and machine learning (ML) techniques to uncover digital biomarkers that will help control diseases before their onset. The team is developing a democratised, scalable solution for remote vital signs monitoring using only a patient’s smartphone. Harvey’s efforts are currently focused on training the predictive algorithm for measuring blood pressure from a smartphone camera.
So, you’re in an interesting position; you used to work at a $20 billion eCommerce company as a data analyst and now you’re one of six members of the data science team at a deep tech startup. What has that transition been like?
In many ways, pretty drastic! The problems and challenges we are working on are evolving pretty rapidly. Being part of such a smaller team has taught me that startups work best when people work in a very multidisciplinary way, and I think it’s great that you can wear many hats while getting projects up and running. Rather than just focusing on, say, analysis or modelling, you need to understand and work to the needs of the clinician informing the project. You also need to get all the infrastructure set up in the cloud. It’s been incredibly interesting moving to a largely scientific role.
Ultimately, though, I think applying certain principles such as employing good software engineering practices and staying true to your customers (or patients!) is incredibly important no matter the size of the organisation. So it's been great to have experience on both ends of the spectrum.
And now that your role does involve working with data and algorithms that will be deployed in clinical settings and have real-world impact on people’s health, it must add another layer of complexity. Have you had to learn to work with data differently in the healthcare space?
Now that my end goal is creating useful products for clinicians and patients, it makes following good practices and maintaining scientific rigour all the more important. Issues such as class imbalance and dataset shift are commonplace in applying machine learning (ML) to healthcare, especially when the type of data you are after is sparse and ranging in quality.
Coming from a background in mathematics and analytics, I’ve learnt quickly how important fully understanding your dataset is when the data is physiological in nature. At first, simple questions would crop up like: What information should I leave out? Is this noise or real information about the body? It’s really great to be able to draw on domain experts in the rest of the team and build up my understanding of medicine and physiology and how these fields interact with data science.
It’s commonly accepted that AI holds great potential to secure the sustainability of our healthcare systems, but that we need to ensure it is safe, ethical and responsible in order to avoid any unintended negative consequences of implementing it. What are the main AI safety problems you encounter?
Thinking about the real-world implications of the tech we develop, I’ve realised how healthcare brings about its own set of problems in AI safety. The first and foremost challenge is ensuring your model performs well under distributional shift. What we mean by this is that demonstrating some predictive ability in one environment may not necessarily translate to another. The accuracy of your model can vary drastically when you introduce it to a new environment. Factors like what kinds of patients you’re dealing with or even which hospital they are in can affect this, so it’s key to ensure that what you are working on generalises.
Another important factor we have to consider is the black-box nature of some of the algorithms being used. Is it a useful model to a user if we can’t understand why it is making the prediction it is? Ideally, you want to know when signal quality is poor and your model has entered a regime in which it has failed, so you know when to disregard it!
And, finally, can you tell us a little bit about how you and the rest of the team are tackling these challenges?
Ultimately, I think it comes down to good scientific practice. From the start we’ve focused on being rigorous and meticulous when identifying the strengths and weaknesses of our approach. This, in practice, really has to come from the culture of the team and the wider company, which is easy for a startup like us that has a very solid, scientific foundation, but which might be tricky for more traditional, well-established companies to ingrain. Key to this is the proprietary tooling we have in place from graphing to evaluating predictive systems that helps enable this kind of analysis.
We are also readily aware of the limitations of ML and make sure to have the real-world impact in the back of our minds at every step of the way. Nailing down where data-driven technology can be incorporated and has a good chance of working feeds through to the product and planning for the future.
In general, AI safety is a super interesting and emerging area of research and we love engaging with the rest of our community to learn more about how others are tackling these issues.
