Design for AI in Healthcare

Artificial Intelligence has been around for a long time, decades. So why all the hype now? The confluence of cheap computing, abundant data, and consistent connectivity has provided the right ingredients for AI to become a practical reality. Data has always been a valuable asset in healthcare, demonstrating any therapy to be safe and effective has required data to prove it. However, a connected medical device presents the opportunity to access data in near real-time, potentially allowing the device to get smarter without a clinical study. An alternative way to look at it is that a device may be most valuable as a data generator, supercharging the next iteration of therapy development.  

The possibilities are exciting to say the least. Personalized medicine can leap forward with a device that is learning from the patient using it, providing a therapy that is better aligned with the individual patient’s needs. Alternatively, a population of diagnostic devices can share information to incrementally reduce the number of false positives over time. Effectively providing smarter devices, faster.

There is no shortage of excitement or literature discussing all of the possibilities that AI presents to healthcare. However, it is critical to understand the challenges and the system design considerations to ensure an AI initiative can be successful in healthcare.

The Challenges

Data Privacy: AI is powered by data, but do you have the right to use it? Under HIPAA, a patient has the right to control how their data is being used, additionally, 22 states have active or pending privacy legislation. What if a patient opts out after their data has already been included in an AI model? Do you need to revalidate the model after their data has been removed? It is hard to say exactly what the privacy landscape is going to look like in five years, but ignoring privacy use cases is likely to lead to technical debt in the short term.  

Diagnostics: The medical device industry has gotten pretty comfortable debugging code. Now the result is the combination of the coded algorithm and potentially large volumes of data. If the result is not what was expected, was it the code, or was it the injection of bad data? What is the best way to diagnose a bad result? The easy answer seems to be ensuring all of the data used is of the highest quality, but that does not represent the real world. Transparency into how a decision was made is required to be able to diagnose an unexpected result.     

Change Management: If you are using AI, there must be an expectation that it will evolve. Managing this evolution must be controlled if AI is making decisions in the delivery of patient care. Ideally, an AI-powered product is getting smarter over time, but how do you tell the difference between a product getting smarter vs just drifting? This scenario is even more complicated if the product is getting smarter for one population but degrading for another population. Many AI-powered devices are released with fixed algorithms, so they are deliberately not changing in the field, but you must decide how to release the next iteration of intelligence.  

Solutions

Privacy by Design: It is not just a catch phrase. If a core competency of your product is acquiring and utilizing patient health information to make care decisions, the product must have privacy use cases and requirements. In other words, the product must be designed explicitly to support privacy, just like any other key performance characteristic. For example, if a patient decides to “opt out” preventing the usage of their data for purposes beyond the delivery of their care, how does the system support this action. What if this patient’s data has already been included in a released model? There are high-tech and low-tech solutions to this particular use case, but the product does need a solution. 

“SDLC” for Data: There is a level of infrastructure needed to support any software development effort. A similar infrastructure is needed to support data, and the associated data models that power AI. Models must be versioned; data that is included in a model must be able to be traced back to the source.    

Architecture: The product’s software architecture must be adapted to support characteristics that may not have been considered pre-AI. The following table identifies a few of the characteristics that should be considered.   

The quest to inject healthcare with devices and systems that think like humans will continue. Yet, there will be no shortage of failed initiatives, particularly as privacy, cybersecurity, and regulatory landscapes evolve. Plan ahead; designing a system and associated processes to support these evolving conditions can be done.