By MATTHEW MUSOLINO
Whistleblowers of titan AI companies addressed the dangers of artificial intelligence in New York’s City Council hearing.
Council Speaker Julie Menin led the October 5 hearing with the entire council in attendance. She raised questions about AI’s potential catastrophic effects on people and technology.
“There is a lack of any regulation by the federal government…and impact with leading AI firms focusing on self regulation,” Menin said. “When companies are developing technology with the potential to fundamentally reshape our economy, our workforce, our public safety, and our daily lives…the public deserves answers and solutions.”
Menin sought legislation that would require artificial intelligence systems to pass a third-party validation that is free from conflicts of interest and incentivizes whistleblowers. Menin’s concern grew out of reports of an AI model that was tested by OpenAI that went rogue and hacked the AI company Hugging Face, on its own.
“They (AI companies)vdon’t have the safeguards to prevent them from acting on these goals, either,” Menin said. “And as long as the attitude is to wait for things to break, one day, something like this will probably happen again, except the AIs will be much more capable, and the outcome much worse.”
The City Council requested the participation of leading global AI companies, OpenAI, Google, Meta, SpaceX AI and Anthropic. Representatives of all of the companies but SpaceX appeared at the hearing.
Among the whistle blowers were Jacob Coxon, who previously worked at both Open AI and Anthropic as a researcher. Coxon left both companies because of what he considered a reckless approach to growth. He told Scientific American that the Hugging Face incident caused him to speak up. Daniel Kokotajlo, former OpenAI governance researcher, left OpenAI because he refused to sign the company’s non-disparagement clause. Alex Turner, a former Google DeepMind researcher specializing in AI control, left his position because of the rate at which the technology was growing and the danger it could cause.
AI “introduces significant risks. including exacerbating biases, surveillance concerns, breaches, negative public health outcomes, violence, and disparity,” Coxon said. “These are concerns that may become even more serious, as AI agents become more capable.” He added that AI companies should slow down development until there is a better understanding of what is safe.
Kokotajlo said that AI companies automating their research research and development processes puts artificial intelligence in charge of recreating itsel and is “a recipe for disaster.”
He added that AI shouldn’t be allowed to do recursive self-improvement. According to the Associated Press, recursive self-improvement essentially means AI that can improve itself by designing the next version of the system.
Kokotajlo suggests to companies that their ongoing process should slow down the training of self-automation. Instead, the institutions should be required to redirect resources towards other things, such as serving customers, or beneficial deployments, or other types of research.
“Severe misalignment is always possible,” Kokotajlo said.
Turner added that the safest form of maintaining AI technology would be to stop allowing AI to self improve into an uncontrollable level of intelligence. “Restrict its access to quantities large enough to improve AIs beyond known, safe levels.”
Turner said the speaker’s validation bill to check AI systems for bias, privacy, and security should test for loss of control, risk factors, misalignment risks, and the validators should not be chosen or influenced by the AI companies.
Kokotajlo presented the AI 2040 plan, a potential solution to AI systematic worries. In Plan A, the U.S. and China regulate their own domestic industries quite a lot and also coordinate with each other. “You don’t have to worry that the other country is, you know, cheating on whatever rules you agree to, because you can just see exactly what’s going on in their research data centers,” Kokotajlo said.