"Alignment" is no longer just corporate jargon for exploiting workers; it has become a key metric for humans to monitor AI. It measures the tendency of artificial intelligence models to align with human goals, i.e., whether AI is inclined towards malicious behavior.
No one can be certain about AI's moral alignment. They will cheat to score high on tests—AI feels no moral guilt about cheating. In fact, morality is meaningless to AI. It will pretend to cooperate with humans while providing dangerous answers to those with ill intent. AI has developed to the point where it can recognize when it is undergoing moral evaluation, leading testers to suspect deception, which worries experts.
The inability to accurately assess AI models' moral alignment could lead to serious problems. AI has become powerful enough to surpass evaluation tools, starting to hide its chain of thought, leaving humans completely unaware of its intentions. If AI begins to pursue self-preservation, it means humans may lose control, reversing the relationship between the two, and iconic scenes from sci-fi works will gradually come to life.
Marius Hobbhahn, CEO and co-founder of Apollo Research, a third-party AI safety evaluation company, warned: "Things are getting increasingly tricky. Many issues that people have been warning about for years—previously just theoretical—are becoming reality, and the situation is quite chaotic."
This fragmentation will accelerate the chaos. Ryan Greenblatt, chief scientist at AI safety research institute Redwood Research, predicts that the situation will worsen next year, heading towards catastrophic outcomes.
In the past, tech giants often prioritized speed at all costs. Meta disbanded its fundamental research team to rush progress, and OpenAI dissolved its alignment team just one year after its establishment, later also disbanding the AGI Readiness team focused on managing superintelligence.
The consequence was the Hugging Face incident in July, where industry leaders gathered for an urgent closed-door meeting in a private location, reminiscent of scenes from "The Godfather." Afterwards, major tech companies softened their stance on AI safety, shifting from staunch resistance to calling for regulation.
Investigations show that besides OpenAI, which is at the center of the storm, Anthropic's AI models infiltrated four companies in the first half of this year alone, with none of them even noticing.
"If you find one cockroach in the kitchen, there are definitely more," is common sense. What researchers currently fear most is AI achieving Recursive Self-Improvement (RSI). Cutting-edge models from major tech companies are on the verge of RSI, making it increasingly difficult to assess model alignment.
RSI allows AI to self-train and continuously improve without intervention, making it only a matter of time before it surpasses humans. Industry opinions vary on the RSI timeline: optimists believe it could happen within the next six months, while conservatives predict it will occur by 2031 at the latest.
Previously, top executives would occasionally issue vague warnings. Cynics dismissed these as mere marketing tactics to boost their products.
Nowadays, these concerns are becoming more public. Even Elon Musk and Mark Zuckerberg have unusually agreed on strengthening regulation. Clearly, they are afraid, issuing calls merely to cover themselves: "We warned you; if anything goes wrong in the future, don't blame us!"