Artificial Intelligence & Machine Learning , Next-Generation Technologies & Secure Development
OpenAI Sets Up New Security Oversight Team
Committee Comes After Key Personnel Resign and Criticize the Company Over Safety
OpenAI on Tuesday set up a committee to make "critical" safety and security decisions for all of its projects, as the technology giant begins to train its next artificial intelligence model.
See Also: Solving business challenges with data & AI: 5 insights from C-suite leaders
CEO Sam Altman and board members Adam D'Angelo, Nicole Seligman and Bret Taylor will lead the committee, the company said. Over the next three months, the team will evaluate OpenAI's processes and safeguards and recommend improvements. OpenAI will share an update on which recommendations it chooses to adopt "in a manner that is consistent with safety and security."
The committee will also comprise technical and policy leads at the company, including Aleksander Mądry, head of preparedness; Lilian Weng, head of safety systems; John Schulman, head of alignment science; Matt Knight, head of security; and Jakub Pachocki, chief scientist. The company plans to consult external security and technical experts including former White House cybersecurity official Rob Joyce.
The committee's formation comes after OpenAI reportedly disbanded its "superalignment" security team dedicated to preventing AI systems from going rogue.
The superalignment safety team's leaders - OpenAI co-founder Ilya Sutskever and Jan Leike - quit the company over their misalignment on the approach to security, as did policy researcher Gretchen Kreuger.
Both Sutskever and Leike worked on addressing the long-term safety risks facing the company and the technology, and Leike criticized OpenAI's lack of support for the superalignment security team in a social media post. "Over the past years, safety culture and processes have taken a backseat to shiny products," Leike said. Sutskever was among the board members who in November removed Altman from OpenAI only to see him reinstated as CEO five days later. Krueger said she decided to resign a few hours before her other two colleagues did, as she shared their security concerns.
