LARGE LANGUAGE MODELS - Prompt Engineering
LLM ReAct Prompting
ReAct (Reason - Action) is a framework that uses advanced language models (LLMs) to generate both reasoning traces and task-specific actions in an interleaved manner. This allows the model to induce, track, and update action plans, handle exceptions, and leverage external sources to improve accuracy and performance on specific tasks. ReAct can retrieve information to support reasoning, while reasoning helps to target what to retrieve next. ReAct can be used to generate more personalized and relevant outputs from LLMs.
This is essentially a complicated way of describing the idea of drilling into a topic, and pretty much follows the way a human would naturally research a topic to learn it or understand it.
To demonstrate how ReAct prompting works, let's follow an example from the paper. The first step is to select cases from a training set (e.g., HotPotQA) and compose ReAct-format trajectories. These are used as few-shot exemplars in the prompts. The trajectories consist of multiple thought-action-observation steps as shown in the example below. The free-form thoughts are used to achieve different tasks such as decomposing questions, extracting information, performing commonsense/arithmetic reasoning, guide search formulation, and synthesizing final answer.
The HotPotQA example of ReAct prompting is a demonstration of how the ReAct framework can be used to answer complex questions that require reasoning and information retrieval from external sources. The example is taken from the paper "LLMs that Reason and Act" by Yao et al. (2023). The question is:
Aside from the Apple Remote, what other devices can control the program Apple Remote was originally designed to interact with?
The ReAct prompting consists of the following steps:
Thought 1: Decompose the question into two subquestions:
- What is the program Apple Remote was originally designed to interact with?
-
What other devices can control that program?
Act 1: Search Google for "Apple Remote" and observe the first result.
Obs 1: The first result is a Wikipedia page about Apple Remote, which says that it is a remote control device released in or after October 2005 by Apple Inc. for use with a number of its products which use infrared capabilities. The device was originally designed to interact with the Front Row media center program on the iMac G5 and other desktop Macintosh computers.
Thought 2: Extract the answer to subquestion (1) from Obs 1: The program Apple Remote was originally designed to interact with is Front Row.
Act 2: Search Google for "Front Row remote control" and observe the first result.
Obs 2: The first result is a Wikipedia page about Front Row, which says that it is a media center software application for Apple's Macintosh computers and Apple TV for navigating and viewing video, photos, podcasts, and music from a computer, optical disc, or the Internet through a 10-foot user interface. The software relies on iTunes and iPhoto and is controlled by an Apple Remote or the keyboard function keys.
Thought 3: Extract the answer to subquestion (2) from Obs 2: Other devices that can control Front Row are the keyboard function keys.
Thought 4: Synthesize the final answer based on Thought 2 and Thought 3: Aside from the Apple Remote, the keyboard function keys can control Front Row, the program Apple Remote was originally designed to interact with.
The final answer is:
Aside from the Apple Remote, the keyboard function keys can control Front Row, the program Apple Remote was originally designed to interact with.
If you want to learn more about ReAct and its applications, I recommend checking out this article on ...Prompt Engineering Guide... that provides a comprehensive overview of the topic with examples and use cases.
Next: Prompt Engineering - Reward Prompting
