OpenAI recently announced sweeping changes to the company’s safety monitoring, including appointing a Safety Systems team tasked with ongoing evaluation of ChatGPT and the company’s other offerings.
This team joins two existing teams: Superalignment, which is focused on the potential of general AI and superintelligent systems, and Preparedness, which is focused on safety in what are called “frontier models” of AI.
The Safety Systems team has the power to veto decisions and even shut down releases of products that are deemed unsafe and can override corporate leadership, including CEO Sam Altman.
The changes come following turmoil among company leaders that resulted in the board attempting to fire Altman.
After company personnel and investors revolted against the ouster, Altman was reinstated.
While many factors reportedly led to the board’s attempted regime change, some of the key topics were the idea of safety in AI and Altman’s growing power to push the technology’s boundaries.
The new Safety Systems team will be split into four parts:
- Safety Engineering: focused on architecture design that facilitates security and enforcement at scale
- Model Safety Research: focused on creating safer LLMs that model the company’s core values
- Safety Reasoning Research: focused on building ethical standards to guide LLM learning
- Human-AI Interaction: focused on the development of human-built policies on which AI models can be trained
With this new structure, the company hopes to address prevalent fears about the future of AI and its products.
As stated on the company’s blog, “There is a delicate tradeoff between safe behavior and usefulness of the model.”
While the creation of the Safety Systems team is at least an acknowledgment of the risks, this delicate balance includes many ethical questions that remain unanswered and will continue to be a challenge in the years to come.