Research / Agent systems
When should a ReAct-style agent refresh its observations?
Refreshing evidence turns three invalid takes into two completions and one honest abstention. It still cannot make a later mutation atomic.
An inventory assistant sees one notebook in bin A and one in bin B. It plans to take the notebook from A. Another worker empties A before the action executes. The plan was valid when formed, but its evidence has expired. No additional eloquence in the plan changes that fact. The useful design question is where to place an observation boundary.
Inspect a case
Separate the world from the remembered world
ReAct interleaves language-based reasoning with environment actions and their observations. In the paper's formulation, reasoning updates the agent's context, while an external action can change the environment. The method does not make remembered observations automatically current. That distinction motivates this small state-machine study. ReAct, Section 2
Write the world state as s = (stock_A, stock_B), the remembered observation as o, and an action as take(bin). The action's precondition is stock_bin > 0. A scripted policy chooses A if its observation says A is available, otherwise B, otherwise it abstains. Crucially, the executor checks the actual state, not the remembered one. Mixing those two objects would hide the very failure being measured.
Work through six changes
Every episode begins with the same observation, (1, 1). Before the attempted take, the fixture substitutes one of six current states: (1, 1), (0, 1), (1, 0), (0, 0), (2, 0), or (0, 2). The cached policy always chooses A. It completes three episodes and issues three invalid actions. The refreshed policy rereads stock after the substitution. It completes five episodes and abstains once, with no invalid action.
The empty state is especially important. Counting only invalid actions would let a policy that never acts look perfect. Therefore the demonstration reports completions, invalid actions, abstentions, and observation counts together. Refreshing uses twelve observations across six episodes rather than six. Its additional information is not free, even though this local example assigns no monetary cost.
The code uses dictionaries copied at observation time. Without the copy, a mutable object could accidentally expose subsequent world changes to the supposedly cached policy. That would turn an implementation mistake into an apparent reliability improvement. Each episode also receives a fresh world, so a successful take cannot affect a later case.
Protect the transition as well as the decision
The deterministic policy already knows the correct precondition and how to choose a bin; this test isolates observation freshness rather than model reasoning. The experiment isolates whether it receives current evidence. A real model could misread the refreshed observation, choose the wrong action, or fail to recover after rejection.
There is also a remaining race: the world might change after the refresh but before execution. For consequential actions, a stronger tool contract can accept an expected revision and reject a request if the world revision differs. Observation refresh improves the decision; an atomic precondition check protects the transition. Neither should be confused with the other.
Refresh and commit are different boundaries
The second check takes every one of the six observed stock states and pairs it with every one of the six states at execution, producing 36 transitions. The refreshed policy chooses from the first state. A revision-checked executor rejects an attempted take when the recorded revision no longer matches. None of those guarded transitions attempts an invalid take. It also cancels some actions whose target still has stock: a revision protects consistency, not availability.
This distinction gives an interface designer two independent controls. Observation timing changes which plan is proposed. A compare-and-commit contract changes which proposal can mutate the world. A check implemented only in the caller cannot provide the latter guarantee, because another change may occur between the check and execution.
Read the tested Python and recorded output
"""When should a ReAct-style agent refresh its observations?
Deterministic, standard-library experiment. RESULTS contains the full data.
"""
STATES = [(1, 1), (0, 1), (1, 0), (0, 0), (2, 0), (0, 2)]
def choose(observation):
return next((b for b in ('A', 'B') if observation[b] > 0), None)
def run(refresh):
totals = dict(completed=0, invalid=0, abstained=0, observations=0)
for stocks in STATES:
world = dict(A=1, B=1)
observed = world.copy()
totals['observations'] += 1
world.update(zip(('A', 'B'), stocks))
if refresh:
observed = world.copy()
totals['observations'] += 1
target = choose(observed)
if target is None:
totals['abstained'] += 1
elif world[target] <= 0:
totals['invalid'] += 1
else:
world[target] -= 1
totals['completed'] += 1
assert sum(totals[k] for k in ('completed', 'invalid', 'abstained')) == 6
return totals
for policy in (False, True):
print('refresh=' + str(policy), run(policy))
assert run(False)['invalid'] == 3
assert run(True)['completed'] == 5
rows = []
for index, stocks in enumerate(STATES):
for policy in ('cached', 'refreshed'):
observed = (1, 1) if policy == 'cached' else stocks
choice = choose(dict(zip(('A', 'B'), observed)))
outcome = 'abstained' if choice is None else ('completed' if stocks[('A', 'B').index(choice)] > 0 else 'invalid')
rows.append(dict(case=index, stock=list(stocks), policy=policy, observation=list(observed), choice=choice, outcome=outcome))
races = []
for middle in STATES:
for final in STATES:
choice = choose(dict(zip(('A', 'B'), middle)))
changed = middle != final
unguarded = 'abstained' if choice is None else ('completed' if final[('A','B').index(choice)] > 0 else 'invalid')
guarded = 'abstained' if choice is None else ('canceled' if changed else unguarded)
races.append(dict(observed=list(middle), final=list(final), choice=choice, refresh_only=unguarded, revision_checked=guarded))
assert all(r['revision_checked'] != 'invalid' for r in races)
RESULTS = dict(cases=rows, totals={p:run(p == 'refreshed') for p in ('cached','refreshed')}, post_observation_changes=races)
print('Post-observation cross product:', len(races), 'cases; revision-check invalid actions:', sum(r['revision_checked']=='invalid' for r in races))
Recorded output
refresh=False {'completed': 3, 'invalid': 3, 'abstained': 0, 'observations': 6}
refresh=True {'completed': 5, 'invalid': 0, 'abstained': 1, 'observations': 12}
Post-observation cross product: 36 cases; revision-check invalid actions: 0