17 Apr EU AI Act: WHAT IT MEANS FOR HEALTHCARE STARTUPS
Generative AI startups are emerging rapidly. Almost every tech startup has some level of AI integration in its digital product. But before jumping on the healthcare AI bandwagon, mind the EU AI Act. While designed to foster innovation and safeguard fundamental rights and values, this legal framework will bring increased complexity for health tech startups, says ethical AI expert Bart De Witte. Following the panel on the EU AI Act at DMEA 2024, we asked Bart to explain how early-stage startups can navigate the Act.
First things first: what is the European AI Act?
Bart explains, “The European AI Act is a pioneering risk-based legislation designed to regulate the deployment of artificial intelligence. It categorizes AI systems into different levels of risk, with the highest tier labeled ‘high-risk AI,’ which encompasses all AI applications in the healthcare sector, particularly those used clinically. This classification implies that developing an AI system will be as stringent as developing a medical product due to the regulatory requirements.”
What does that mean specifically?
Bart elaborates, “This classification means developers will need to implement a quality management system, ensure post-market surveillance, and meet other regulatory requirements typical of medical products. This presents a significant challenge for startups due to the substantial overhead. Additionally, AI is inherently dynamic, unlike traditional medical products. Those familiar with the rapid advancements from LLMs during the last 2 years can appreciate the pace of progress in AI technologies. Given the European regulation’s mandate for recertification with every update, startups focusing on proprietary AI developments face daunting hurdles in competing with larger established companies.”
So it will only provide benefits to the large players?
“Unfortunately, yes. Along with numerous open-source advocates, I’ve actively fought against this trend, especially during the period when the AI Act was being shaped. We still have an opportunity to keep AI open, but it will require a significant effort, akin to Meta’s LLAMA initiative, which kickstarted the medical open-source ecosystem. There’s no reason why we can’t publish certified open-source AI models and explore innovative collaborative methods to maintain, upgrade, and monitor them.”
What do you recommend to healthcare startups that work with AI?
“I recommend holding off on diving straight into developing your own AI products with the notion that they’ll be your main asset. Instead, consider the advantages of partnering with established, certified AI product startups. Start with a thorough analysis of what your users actually need. If AI can address these needs, explore if you are going to bet on closed or open source. For open-source engagement explore partnerships where you can jointly develop solutions, this will significantly reduce your R&C costs, and together, you can set open-source standards that you’ll jointly certify. This strategy not only diversifies your technological assets but also safeguards against future costs related to updates and recertifications, which could be substantial.
For example, consider the case of startups in the 3D printing orthopedics sector. These businesses once faced a significant challenge when a new printer, capable of producing at four times the speed and significantly lower costs, entered the market. Due to the steep €200,000 re-certification cost for their existing models, these startups found themselves unable to compete or upgrade. This situation highlights a critical flaw in regulatory frameworks, which often favor big corporations with deeper pockets and disadvantage smaller innovators.”
Could you tell our readers more about the European Health Data Space?
“The European Health Data Space is a legal framework aimed at promoting the safe exchange of health data across the EU. It aims to facilitate better healthcare delivery, allow personal access and control over one’s health data, and encourage innovation through broader use of health data for research & development. The EHDS emphasizes as well patient access to health data, the details of how this access is implemented and the extent of patient control can vary, and the implementation is still unfolding.
In January, I delivered a keynote at the inauguration of the Belgian Health Data Agency, and I was pleasantly surprised to learn that Belgium is a pioneer in this area. This health Data Agency is a health data access body, which is a component of the broader European Health Data Space. Each country establishes such a health data access body through which health data can be accessed for research and development purposes. The goal of the European Health Data Space is to increasingly make this data available and to enable the data to flow across borders.
I see a significant concern: the public sector, funded by our tax money and insurance premiums, has granted the industry access to our data. In itself, this is not an issue, as the proposal specifies that the data itself cannot be monetized directly. However, the problem arises with the privatization of data derivatives. Specifically, while society grants what could be considered solidarity access to our data, the insights and innovative derivatives derived from this data can be monopolized. The European Health Data Space has missed a crucial opportunity to require the industry to reciprocate by contributing data back into the ecosystem and to engage in developing open-source, knowledge-based systems. Currently, this is not happening, leading to a predominantly one-way transfer of value from the public to the economic sector. We must consider whether, over time, this will lead to significant disparities in information and power between the private and public sectors.”
In closing, is there one thing you think healthcare startups should think about?
“This might sound speculative, but I believe that within the next five years, we could see the emergence of the first unicorn company with only one employee. For many startups, AI provides the potential to create unprecedented value with minimal resources. However, I think that the adoption of an AI-first approach to optimize resource-heavy internal processes has not yet become widespread, despite its significant potential.”