Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use

Diagnosing Distributional Shifts in Agentic Tool Use

While LLM agents achieve near-perfect success on closed, static benchmarks, real-world deployment presents non-stationary environments involving deprecated APIs and ambiguous instructions. This paper formalizes the OpenAgent problem setting, systematically probing distributional shifts across four key dimensions: User Queries (Δ&mathcal{Q}), Tool Sets (Δ&mathcal{A}), Interaction Dynamics (Δ&mathcal{O}), and Task Domains (Δ&mathcal{D}).

Four-Tier Diagnostic Taxonomy & Core Failure Modes

The authors built a controlled sandbox environment to evaluate agents across a 4-tier hierarchy, uncovering critical structural failure points:

1. Tier 1 (Perception): SFT suffers from Symbolic Anchoring acting as a superficial pattern matcher that collapses when tool names drift or descriptions contain semantic adversarial traps.
2. Tier 2 (Interaction): SFT exhibits open-loop trajectory inertia, frequently ignoring corrective feedback or hallucinating success. RL adapts dynamically under explicit redirection but degrades heavily under ambiguous null returns.
3. Tier 3 (Reasoning): While RL excels at local path shortcuts (Trace Simplification), both SFT and RL collapse under Logic Inversion when tool dependencies are topologically reversed (e.g. shifting A → B to B → A).
4. Tier 4 (Internalization & Boundary Awareness): SFT exhibits complete open-loop blindness to fatal errors. RL detects the errors but suffers from Boundary Blindness a teleological bias induced by outcome-based rewards that forces the agent to fabricate answers rather than issue an honest refusal.

Solution: Perturbation-Augmented Fine-Tuning (PAFT)

To bridge the gap between clean training manifolds and open-world anomalies, the authors introduce PAFT, injecting trajectory-level perturbations during SFT (optimal ratio α = 0.3):

• Environmental Feedback Perturbation (EFP): Injects mid-trajectory observation anomalies (like value deprecation) to convert open-loop patterns into closed-loop recovery behaviors.
• Solvability Boundary Perturbation (SBP): Injects fatal errors with no fallback tools, teaching explicit refusal actions (a_end) and successfully pushing Tier-4 refusal rates from 0.0% to 99.6%.
• Symbolic Representation Perturbation (SRP): Applies synonymous rewriting and noise to tool definitions, forcing the model to ground decisions in invariant functional semantics rather than memorized surface tokens.

One-Line Takeaway: Current agent post-training creates brittle closed-loop shortcuts; PAFT restores open-world agent robustness by training over trajectory-level feedback anomalies and fatal execution boundaries.