Sony Interactive Entertainment (SIE) recently filed a new patent application numbered 18/314775, planning to deploy AI "fake accounts" driven by Large Language Models (LLMs) in the PlayStation Network (PSN) to proactively identify and flag potential scammers, harassers, and criminals.
Unlike traditional passive reporting mechanisms, this system, named the "Large Language Model-Driven Social Integrity System," adopts a proactive entrapment strategy.
According to the patent description, these AI-controlled fake accounts can highly replicate the chat habits and behavioral characteristics of specific groups (such as minors), disguise themselves as ordinary players, and be deliberately placed in social scenarios where malicious users might look for targets. When a suspicious user establishes contact and interacts with this AI account, the system will initiate an analysis program to conduct in-depth evaluation of the conversation content and behavior patterns.
The monitoring targets set by the system cover various violations, including but not limited to:
· Sending spam and scam attempts
· Soliciting personal private information
· Inappropriate contact with minors
· Sending inappropriate messages
· Cheating in games
It is worth noting that this mechanism does not adopt a "one-size-fits-all" punishment approach. Even if the AI account captures suspicious signals during interaction, the system will not immediately ban the other party, but instead mark them as a potentially dangerous account and include them in a continuous observation list.
When the user subsequently contacts real players, the system will issue warnings to them or take protective measures. If a user triggers risk alerts multiple times, they may face a ban. Its core purpose is to identify and intervene before potential harm occurs, enhancing the safety of PlayStation's online community, especially protecting minor players.
Industry analysis also points out that the system may face challenges such as privacy protection and false positives in practical application. The scope of AI analysis of user conversations and the criteria for determining danger will become key issues determining its effectiveness and fairness.
As with all tech company patents, it remains unknown whether this technology will ultimately be applied to PlayStation products. Tech companies often file numerous patents for forward-looking layouts, many of which never translate into actual products. Currently, this patent mainly indicates that Sony is exploring the possibility of using AI technology to proactively identify malicious users and purify the platform environment.