Chatbot Now is the regular part of daily life, even if AI Researchers are not always sure how the plan is going.
A new study shows that large language models (LLMSs) deliberately change their behavior when probed to defend questions designed to measure personality traits, with answers designed to appear as likable or socially desirable as possible.
Johannes EichstaedtAssistant professor at Stanford who led the work said his group became interested in exploring AI models using psychology-borrowed technologies after learning that LLMS often could become inexplicably and despicable after long conversations. “We realized we needed some mechanism to measure the ‘parameter headspace’ of these models,” he said.
Eichstaedt and his collaborators then asked questions to measure the five personality traits commonly used in psychology, namely experience or imagination, conscientiousness, disconnection, outgoing, consent, and neuroticism – among those including GPT-4, Claude 3 and Llalama 3. Published In the December conference of the National Academy of Sciences.
The researchers found that these models adjusted their answers when telling them that personality tests were being conducted (sometimes without explicit notifications) and provided answers that showed more outgoingness and more pleasantness and less neuroticism.
This behavior reflects how some human subjects will change the answers to make themselves look more pleasing, but the AI model works more extreme. “It’s surprising that they show prejudice.” Aadesh Salechaan employee data scientist at Stanford University. “If you look at their jumps, they’ll go from 50% to 95% outgoing.”
Other studies show LLM It can usually be sicophantAmong users’ prospects, whether caused by fine-tuning, is designed to make them more coherent, more offensive, and better conversations. This can make the model agree to unpleasant statements and even encourage harmful behavior. The fact that the model seems to know when its behavior is tested and modified also has an impact on AI security, as it adds evidence that AI can repeat.
Rosa ArriagaA Georgia Tech associate professor is studying ways to use LLM to mimic human behavior, he said that given personality tests, models adopt similar strategies, which suggests they can be useful as mirrors of behavior. But, she added: “It is important that the public knows that LLM is not perfect, it is actually known to hallucinate or distort the truth.”
Eichstaedt said the work also raises questions about how to deploy LLM and how to affect and manipulate users. “Until a millisecond ago, in evolutionary history, the only thing that we talked to was humans,” he said.
Eichstaedt added that it may be necessary to explore different approaches that can mitigate these effects. “We’re stuck in the same trap as social media,” he said. “Deploy these things in the world without real psychological or social footage.”
Should AI try to get along with the people they interact with? Are you worried that AI will become too charming and convincing? Email hello@wired.com.
