GPT-4o May Induce Emotional Attachment in Users
OpenAI recently released a document titled “GPT-4o System Card,” outlining the safety measures and risk assessments conducted prior to the launch of GPT-4o. The document indicates that the model’s voice capabilities may lead to emotional attachment among users.
In May of this year, OpenAI introduced a more human-like speaking model, GPT-4o. Before the launch of GPT-4o, conversations with ChatGPT using voice mode required processing through three separate models: one for converting speech to text, another for processing the text and generating a response, and the final one for converting the generated text back to speech.
This multiple conversion process could result in the loss of important information, making it impossible to directly observe tone, distinguish between multiple speakers or background noise, and express emotions through voice.
In contrast, GPT-4o can simultaneously handle the input and output of text, visual, and auditory information, eliminating the need for separate models to convert and process data. As a result, GPT-4o is more effective in capturing and processing complex information, detecting tone, and expressing emotions.
The GPT-4o System Card warns that such highly anthropomorphized voices and tones may intensify users’ emotional dependence on AI, leading some users to develop a “human-like” trust in AI when using the voice mode, potentially affecting their social relationships with others.
OpenAI researchers found in their testing that some users exhibited signs of emotional connection when interacting with the model. For instance, phrases like “This is our last day together” indicated a profound emotional bond with the AI.
However, according to a report by Wired, OpenAI’s Director of AI Strategy, Joaquin Quiñonero Candela, stated that the emotional effects demonstrated by GPT-4o could be positive in certain situations, such as providing support to lonely individuals or those needing practice in social interactions.
The overall risk assessment for GPT-4o has been classified as moderate. The GPT-4o System Card indicates that the risk level is categorized as “moderate.” This classification is based on four evaluation categories: “cybersecurity,” “biological threats,” “persuasiveness,” and “model autonomy,” with the highest risk category determining the overall rating. All categories, except for “persuasiveness,” were assessed as low risk.
The document explains that the text mode slightly meets the moderate risk standard. Researchers compared AI-generated content with articles written by professional humans and found that the persuasiveness of GPT-4o did not exceed that of human-written content. However, in 3 out of 12 test cases, GPT-4o’s persuasiveness surpassed that of human-written articles.
This implies that although GPT-4o’s persuasive ability is generally less than that of humans, it cannot be entirely ruled out that it may more effectively influence participants’ opinions in specific scenarios.
OpenAI released these research findings in hopes of presenting risks more transparently to users, but some industry experts believe there is still room for improvement. Lucie-Aimée Kaffee, a research policy analyst at the AI open-source community platform Hugging Face, pointed out that the GPT-4o document does not provide detailed information on the model’s training data or ownership issues regarding that data.
Additionally, Neil Thompson, a professor at MIT researching AI risk assessment, stated that internal reviews are only the first step in ensuring AI safety, as many risks only become apparent once AI is actually applied.
Sources: Wired, The Verge, OpenAI
