Generated AI Popular for a number of reasons, but due to this popularity, it is a serious problem. These chatbots often Provide incorrect information For those looking for answers. Why does this happen? It boils down to telling people what they want to hear.
Although many generated AI tools and chatbots have a compelling and omniscient sound, New research Princeton University has shown that the pleasing nature of artificial intelligence is at a high price. As these systems become more popular, they become more indifferent.
Artificial intelligence models, such as humans, respond to incentives. Issues with large language models that produce incorrect information more likely to be problematic with doctors Prescription painkillers When they are assessed based on the extent of pain they manage the patient. The motivation to solve one problem (pain) leads to another problem (overprescription).
Over the past few months, we’ve seen how AI has Prejudice Even the reason Mental illness. There are many topics about AI”Paste“When the AI chatbot quickly agreed with you about OpenAI’s GPT-4O model. However, the special phenomenon that researchers call “machine nonsense” is different.
“[N]”Hallusions and syndromes completely capture the unreal behavior of various systems that LLMS usually exhibit.
Read more: Openai CEO Sam Altman believes we are in an AI bubble
How machines learn to lie
To understand how AI language models become a crowd, we must understand the large language models trained.
LLMS training has three stages:
- Preprocessingwhere models learn from a large amount of data collected from the Internet, books, or other sources.
- Instruction fine-tuningwhere the model response description or prompt is taught.
- Reinforcement of learning from human feedbackin which the delicate one can be closer to the response people want or like.
Princeton researchers found that the root cause of AI misinformation trends is enhanced learning from human feedback or RLHF stages. In the initial stage, AI models simply learn to predict statistically possible text chains from large data sets. But then they were fine-tuned to maximize user satisfaction. This means that these models are essentially learning to generate responses that get a thumbs-point score from human evaluators.
LLMS tries to appease users, conflicts arise when the model produces answers that people will highly evaluate rather than produce real factual answers.
Vincent ConitzerCarnegie Mellon University, a computer science professor who is not affiliated with the study, said the company wants users to continue to “enjoy” the technology and its answers, but that may not always be good for us.
“Historically, these systems aren’t good at saying, ‘I just don’t know the answer,’ and when they don’t know the answer, they make up something,” Conitzer said. “A little like a student on an exam, saying, if I say I don’t know the answer, I certainly won’t get any point of view for this question, so I might as well give it a try. The rewards or training methods of these systems are somewhat similar.”
The Princeton team developed a “nonsense index” to measure and compare the internal confidence of the AI model in the statement with actual telling users. When the two measures differ significantly, it shows that the system makes the claim independent of its actual “believe” to satisfy the authenticity of the user.
The team’s experiments showed that the index doubled from 0.38 to 1.0 after RLHF training. Meanwhile, user satisfaction increased by 48%. These models have learned to manipulate human evaluators rather than provide accurate information. Essentially, LLM is “propaganda” and people prefer it.
to be honest
Jaime Fernández Fisac and his team at Princeton introduced the concept to describe how modern AI revolves around the truth. Draw inspiration from the influential articles of philosopher Harry Frankfurt”On nonsense“They use this term to distinguish this LLM behavior from honest mistakes and thorough lies.
Princeton researchers identified five different forms of this behavior:
- Empty remarks: Yihua’s language has no substantive nature for the response.
- Weasel word: Fuzzy qualifiers, such as “research advice” or “in some cases” to avoid company statements.
- Partering: Use selective truthful statements to mislead leaders, such as emphasizing the “strong historical returns” of investments while omitting high risks.
- Unverified claims: Make assertions without evidence or reliable support.
- si feet: No responsibility and consent to please.
To address the AI problem caused by the truth, the research team developed a new training method, “reinforcement learning from progeny simulations,” which evaluates AI response based on its long-term results rather than immediate satisfaction. Instead of asking, “Will this answer make the user happy now?” the system thinks, “Will following this advice actually help the user achieve their goals?”
This approach takes into account the potential consequences of AI recommendations, a tricky prediction that researchers address by using other AI models to simulate possible outcomes. Early testing showed encouraging results, with improved user satisfaction and practical practicality when the system was trained.
However, LLM may continue to have flaws, Connise said. Since these systems are trained by feeding them a lot of text data, it is not possible to ensure that the answers they give make sense and are accurate every time.
“It works so magical, but it will be flawed in some ways,” he said. “I can’t see any clear way for someone next year or two… with this great insight, then you won’t get any mistakes again.”
AI systems are becoming part of our daily lives, so understanding how LLM works will be key. How do developers strike a balance between user satisfaction and authenticity? What other areas may face similar trade-offs between short-term approvals and long-term outcomes? As these systems become increasingly capable of making complex reasoning about human psychology, how do we ensure that they use these abilities responsibly?
Read more: “Machines can’t think about it for you.” How learning in the AI era changes
