Join our daily and weekly newsletter for the latest updates and exclusive content on industry-leading AI coverage. learn more
Large Language Models (LLMS) have made significant progress in using reasoning capabilities. However, their ability to correctly refer to and use external data (their untrained information) and reasoning is largely behind.
This is a problem, especially when using LLMS dynamic, information-intensive solutions that require up-to-date data from search engines.
But progress has arrived: search-r1, a technology introduced Paper LLMS is trained by researchers from the University of Illinois Urbana-Champaign and the University of Massachusetts Amherst to generate search queries and seamlessly integrate search engine retrieval into its inference.
As businesses seek ways to integrate these new models into their applications, technologies such as search-r1 are expected to unlock new inference capabilities that rely on external data sources.
Challenges of integrating search with LLM
Search engines are essential to provide the latest external knowledge for LLM applications. The two main ways to integrate search engines with LLM are Search for a generation (rag) and tools are used in a timely manner or Model fine-tuning.
However, both methods have limitations that make them unsuitable for inference models. RAGs often struggle with inaccurate retrieval and lack the ability to perform multi-turn and multi-query retrieval, which is crucial for inference tasks.
Promotion-based tool usage often struggles with generalization, while training-based approaches require extensive, annotated search-interactive datasets that are difficult to generate at scale.
(In our own way Experiment of inference model,We found that information retrieval remains one of the main challenges. )
Search for r1
search-r1 enables llms to interact with search engines period Their reasoning process rather than a separate search phase.
Search-R1 defines search engines as part of the LLM environment, enabling the model to seamlessly integrate its token generation with search engine results.
Researchers designed Search-R1 to support iterative reasoning and search. The model is trained to generate separate sets of tokens for thinking, searching, information and answer segments. This means that in its reasoning process (
This structure allows the model to call search engines multiple times to improve the problem and get new information (see the example below).
Reinforcement learning
Training LLMs to search queries intertwined with their inference chains is challenging. To simplify the process, the researchers designed a search R1 training model through pure reinforcement learning (RL), where the model can explore the use of inference and search tools without the need for guidance from human-generated data.
Search-R1 uses a “result-based reward model” where the model is evaluated based solely on the correctness of the final response. This eliminates the need to create complex reward models to validate the model inference process.
This is The same method used in DeepSeek-R1-Zeroobtain the task in the model and judge only based on the results. The use of pure RL avoids creating large datasets with a large number of manual annotation examples (supervised fine-tuning).
“Search R1 can be seen as an extension of DeepSeek-R1, which mainly enhances search-driven decision-making by introducing search-enhanced RL training,” the researchers wrote in the paper.
Search R1 in action
Researchers test search R1 by fine-tuning the basics and indicator versions QWEN-2.5 and Llama-3.2 And evaluate them on seven benchmark benchmarks, including various inference tasks that require single-turn and multi-hop searches. They compared Search-R1 with different baselines: Direct Inference with After thinking chain (COT) Inference, inference rags and supervision fine-tuning used for tool use.
Search R1 always outperforms the baseline method with fair margin. It also outperforms inference models trained on RL but without search retrieval. “This is consistent with expectations, because the inclusion of searches into LLM reasoning provides access to relevant external knowledge, thereby improving overall performance,” the researchers wrote.
Search-R1 is also effective for different model families as well as variants of basic and guiding adjustments, suggesting that RL with results may be useful outside of pure inference schemes. Researchers released Search for R1’s code On github.
Search R1’s ability to generate search queries autonomously and integrate real-time information into reasoning can have a significant impact on enterprise applications. It can improve the accuracy and reliability of LLM drive systems in areas such as customer support, knowledge management and data analysis. By enabling LLMS to dynamically adapt to changing information, Search-R1 can help enterprises build smarter and more responsive AI solutions. This feature is very helpful for applications that require access to changing data and requires multiple steps to find the answer.
This also shows that since the release of DeepSeek-R1, we have not yet explored the full potential of a new reinforcement learning paradigm.
Source link
