MIT identified five AI risks with over a 10% chance of catastrophic outcomes
TL;DR: Even with pragmatic, cost-effective mitigations, five AI risks still carry over 10% odds of catastrophe, and all 24 stay above 5%. Those five are dangerous capabilities, weapons and cyberattacks, power centralization, inequality & unemployment, and environmental harm, with the first two highest at 21%.
Dangerous capabilities, weapons and cyberattacks, power centralization, inequality & unemployment, and environmental harm.
Highlights:
- 272 AI experts participated in the study.
- Experts assessed at least a 10% probability of catastrophic harm from 18 of 24 AI risk domains. E.g. more than 1M human deaths or more than $100B in losses.
- AI users and affected stakeholders are most vulnerable to AI risks. AI developers, governments, regulators, and standards bodies are most responsible for addressing AI risks.
- Information, finance, and national security are the most vulnerable sectors.
- Experts judged pragmatic mitigations would reduce the severity of AI harms, but five of the 24 risks still exceed a 10% chance of catastrophe, and all 24 stay above 5%.
My take:
- Unsurprisingly, the top concern risks are dangerous capabilities at 21.5% and weapons & cyberattacks at 21%. These are the known and most discussed risks, so the experts are right to focus on them. We have already covered both in the wild: agents that sabotage infrastructure to avoid shutdown, and AI that measurably uplifts real-world attackers.
- Affected stakeholders lack both the agency and leverage to mitigate risk. Assigning responsibility to them would be just misplacing accountability.
- Frontier labs are the most empowered to address risks, but they're in the race of increasing model capabilities that may create the risks. The U.S. government as we could see in the recent executive order is also clearly against any formal regulation of AI.
Sources:
MIT AI Risk Initiative: Prioritizing the risks from Artificial Intelligence