ReasoningBank: Enabling agents to learn from experience

ReasoningBank is an agent-memory framework designed to help deployed agents learn continuously from both successful and failed task attempts. Rather than storing exhaustive action histories or only successful workflows, it distills reusable reasoning strategies, decision rationales, and preventative lessons. Evaluations on WebArena and SWE-Bench-Verified show higher success rates and fewer execution steps, especially when combined with memory-aware test-time scaling.

Distilling Generalizable Reasoning

• Each memory contains:
  • A concise title
  • A brief description
  • Detailed reasoning steps, rationales, or operational insights
• The agent retrieves relevant memories before acting.
• After completing a task, an LLM judge evaluates the trajectory and identifies useful success or failure signals.
• The agent converts those signals into new memories and appends them to the ReasoningBank.
• Failure analysis is central: mistakes become counterfactual guidance and strategic guardrails, such as verifying the current page before repeatedly clicking “Load More.”

Memory-Aware Test-Time Scaling

• Memory-aware test-time scaling (MaTTS) connects inference-time exploration with long-term memory.
• Parallel scaling: Multiple trajectories are generated and compared, allowing the agent to distinguish robust strategies from flawed reasoning.
• Sequential scaling: The agent progressively refines a single trajectory, preserving useful intermediate insights from trial and error.
• This creates a feedback loop: better memories guide exploration, while richer exploration produces better memories.

Benchmark Results and Strategic Maturity

• Against memory-free ReAct agents using Gemini-2.5-Flash:
  • Success rates improved by 8.3% on WebArena.
  • Success rates improved by 4.6% on SWE-Bench-Verified.
  • SWE-Bench-Verified tasks required nearly three fewer execution steps on average.
• Adding MaTTS with parallel scaling factor k=5 produced further gains:
  • A 3% success-rate increase over ReasoningBank alone on WebArena.
  • 0.4 fewer steps per task.
• Over repeated tasks, simple procedural checklists evolved into more sophisticated memories containing compositional and preventative logic.

ReasoningBank suggests that effective agent scaling requires more than additional inference compute or stored trajectories. Agents should systematically learn from both outcomes and mistakes, using structured reasoning memories to become more capable and efficient after deployment.