by AI regulation has become an area of interstate competition. While the EU has just reached an agreement on the AI Act, the United States previously published a far-reaching executive order on AI and the United Kingdom convened political and industry leaders to the AI Safety Summit.
In many of these discussions, one risk is receiving more attention: AI-assisted bioterrorism, or the ability of individuals to cause catastrophe by using AI tools to gain access to a pandemic virus.
We recently showed that this is a risk worth considering. In an informal experiment, we tasked people with using a large open-source language model with safeguards removed to help them obtain a pathogen capable of causing a pandemic.
Within three hours, participants identified many of the steps necessary to initiate a potentially catastrophic outbreak.
Particularly concerning was that the model advised participants on how to access the viral DNA (the blueprint for creating the pathogen) while evading existing detection methods. It is still unclear to what extent current models aid bioterrorism by summarizing information that already exists online.
However, current capabilities aside, the findings suggest that, in the absence of strong safeguards, more advanced future models could provide malicious actors with simplified and accessible information on how to access, construct and release a pandemic virus.
A technology open to malicious actors
The DNA constructs needed to create a virus from scratch can be ordered online: many genetic synthesis providers will make a thousand base pairs of DNA for less than 200 euros, something that just a few decades ago required thousands of hours of research and deserved an award. Nobel. .
The advent of personalized gene synthesis is now a mainstay of life sciences, allowing researchers to rapidly iterate on the design of vaccines and mRNA-based therapies, among many other benefits. However, just as advances in gene synthesis enable new discoveries and treatments, malicious actors can also misuse the technology to access pandemic pathogens.
Many companies have taken the important step of screening orders for DNA from dangerous pathogens.
But not everyone does it. Language-based AI models can already advise people how to identify and exploit those gaps to obtain the DNA of pandemic-capable pathogens.
Echoing this concern, the CEO of Anthropic, a leading artificial intelligence company, recently warned US lawmakers that within two years, next-generation AI systems could enable large-scale bioterrorism unless establish appropriate barriers.
Yoshua Bengio, one of the “godfathers of AI,” expressed similar concerns. As EU policymakers think about appropriate reactions to rapid AI progress, one simple barrier deserves closer scrutiny: a legal requirement to examine all gene synthesis orders for dangerous sequences.
Fewer than a hundred highly specialized companies offer DNA synthesis services, offering an ideal policy lever: if the EU requires companies to check all orders against an updated database of known pandemic pathogens and apply know-your-customer requirements, bad Actors can no longer access the basic elements necessary to sow the next pandemic.
The case for mandatory genetic synthesis screening
The United States has already taken steps to ensure gene synthesis. The government’s recent Executive Order on AI requires federally funded entities to follow the government’s new biological synthesis screening guidelines.
The EU itself has not yet approved regulations on the subject. Given the growing risk of pandemic-level bioterrorism, a mandate for detection of gene synthesis would be a significant first step.
Suppliers representing more than 80% of the industry are in favor of the mandates. Others have legitimate concerns about the potential costs of gene synthesis analysis and worry about potential intellectual property issues, but the imminent release of free, privacy-preserving analysis tools should mitigate this issue.
A mandatory detection system must meet several criteria. Orders should be encrypted to preserve privacy and subject to checks that exclusively detect dangers with very few false alarms, while authorized laboratories should be offered a way to obtain permitted DNA sequences without delays.
Additionally, all detection systems must verifiably match orders against an immediately updated database of dangerous sequences to detect newly identified potential pandemic pathogens, with strong incentives to use the best detection tests as determined by detection efforts. “red team” testing how easily dangerous sequences can be obtained without detection.
Finally, new DNA synthesis and assembly devices, such as benchtop synthesizers, should incorporate integrated screening that meets the above criteria. Implementing rigorous screening protocols based on these criteria is vital to fully realize the broad benefits of biotechnology while protecting it from misuse.
Safeguarding the promise of biology
Advances in both biotechnology and artificial intelligence will drive revolutionary advances in life sciences and medicine.
Custom gene synthesis is a critical enabler of these remarkable benefits. But the damage caused by SARS-CoV-2, a single, historically mild pandemic virus, demonstrates that its misuse (made more likely by advances in generative AI) could cause harm on a scale that outweighs all of these benefits.
As the European Union experienced with the AI Law, finding a balance between encouraging innovation and reducing risks takes time.
But the specific risks of AI-powered bioterrorism, recognized by industry leaders and biosecurity professionals alike, can be addressed now.
By requiring testing of gene synthesis, the European Union can substantially reduce these risks, safeguarding the promise of ever-advancing biotechnology.
Kevin Esvelt is an associate professor at the Massachusetts Institute of Technology Media Lab, where he leads the Sculpting Evolution group. He co-founded the SecureDNA Foundation, a non-profit organization based in Switzerland. Ben Mueller is a research scientist at MIT and chief operating officer of SecureBio, an organization working on policies and technologies to protect against future pandemics.
At Euronews we believe that all opinions matter. Contact us at view@euronews.com to submit proposals or presentations and be part of the conversation.
