OpenAI Is Starting a New Team to Reduce Risk From Future AI

OpenAI is starting a new team aimed at minimizing risks from artificial intelligence as the fast-developing technology gets more capable over time.

In a blog post Thursday the company best known for the popular chatbot ChatGPT announced it has formed a “preparedness” team headed up by Aleksander Madry, who has been working at OpenAI while on leave from a faculty position at the Massachusetts Institute of Technology. The group will analyze and try to ward off potential “catastrophic risks” of AI systems, ranging from cybersecurity issues to chemical, nuclear and biological threats.

The team will also come up with a policy aimed at helping the company determine how it can can reduce risks that may come with the development of so-called “frontier models” — the next generation of AI technology that is more capable than that what’s generally in use today.

OpenAI’s goal has long centered on building artificial general intelligence, or AGI — AI that can perform a range of tasks better than humans can. While current AI systems don’t meet this description, the company said it needs to “ensure we have the understanding and infrastructure needed for the safety of highly capable AI systems.