smolagents Execution Semantics: Transcript State, Python Heap, and Sandbox Boundaries

Abstract: CodeAgent gains composability through Python, but creates an unsaved second memory.

Snapshot: huggingface/smolagents at commit e3a5b8, analyzed on 2026-07-23. The repository is on 1.27.0.dev0; the latest stable GitHub and PyPI release is 1.26.0. The reset, timeout, and persistence details below are commit-specific.

A deterministic two-run experiment exposes the state boundary.

On the first run, a scripted model made CodeAgent execute:

x = 41
final_answer(x)

On the second run, the default reset=True remained enabled and the model executed only:

final_answer(x + 1)

The result was 42.

The second run's AgentMemory contained only its new task and action step. The first code block was gone. The Python executor, however, still held x.

That observation is more consequential than the usual description of smolagents as “an agent that writes Python.” CodeAgent has two kinds of runtime memory:

a typed, model-visible transcript;
a live Python namespace inside the executor.

They exchange observations, but they are not reset together, persisted together, or replayed together. Python is therefore not only an action format in smolagents. It is a second agent memory.

Scope and execution model

smolagents is Hugging Face's compact agent library. MultiStepAgent provides the ReAct-style lifecycle, typed memory steps, optional planning, monitoring, and callbacks. ToolCallingAgent asks a model for structured tool calls. CodeAgent asks for Python code and exposes tools as callable functions inside an executor.

It is not a durable workflow engine, and its local evaluator is not an untrusted-code platform. The LocalPythonExecutor docstring is explicit: it is “not a security sandbox.” Remote E2B, Modal, Blaxel, and Docker executors can provide a stronger process or container boundary, but isolation alone does not create checkpoints, transactions, or deterministic replay.

The README still describes the agent logic as roughly 1,000 lines. In this snapshot, src/smolagents/agents.py is 1,813 lines. The relevant form of minimalism is the small set of concepts—model, tool, memory step, executor, loop—not a line-count record.

The granularity problem with JSON tool calls.

Suppose an agent must fetch weather for five cities, normalize the temperatures, identify outliers, and fetch air quality only for those outliers.

A conventional tool-calling loop may look like:

model → weather(Beijing) → result
model → weather(Shanghai) → result
model → ...
model → calculate/filter → air-quality calls

The model may need another inference for every branch or iteration. A code action can submit the control flow in one turn:

rows = [get_weather(city) for city in cities]
average = sum(row["temp_c"] for row in rows) / len(rows)
outliers = [
    row for row in rows
    if abs(row["temp_c"] - average) > 8
]
details = [get_air_quality(row["city"]) for row in outliers]
final_answer({"average": average, "outliers": details})

Python acts as an action IR. Loops, branches, local bindings, structured objects, and function composition already have well-understood semantics, so the framework does not need to invent a large workflow DSL.

The CodeAct paper reported gains of up to 20% on some benchmark tasks. That is evidence for those experiments, not a universal production uplift for smolagents. The architecture-level benefit is narrower and more durable: one model turn can propose a small program rather than one isolated invocation.

One loop with two state lanes.

The CodeAgent path is easiest to understand as two coupled lanes:

MODEL-VISIBLE LANE
TaskStep / PlanningStep / ActionStep / error
→ to_messages()
→ model
→ Python action
→ logs + last value return to ActionStep

EXECUTOR LANE
persistent namespace + imports + tool objects
→ execute the same Python action
→ variables and objects remain alive

An ActionStep can hold:

model input messages and output;
the parsed code;
synthetic tool calls;
textual and image observations;
errors and the action output;
token usage and timing.

That is a useful event record. It is not a heap snapshot. A DataFrame, an imported module, a list accumulated across steps, or a complex value returned by a tool can remain in the executor. The next step can use it even when the model only sees the selected logs and final expression value.

This trade is central to the project. Avoiding repeated object-to-text-to-object conversion makes code actions expressive and efficient. It also means the transcript is no longer the sole source of truth about the run.

Request lifecycle and state experiments

Construction chooses an execution contract.

CodeAgent extends MultiStepAgent with:

authorized imports;
code block tags and parsing rules;
an executor type or injected PythonExecutor;
an executor-specific configuration;
a print-output limit.

create_python_executor() accepts local, blaxel, e2b, modal, or docker. Local execution constructs LocalPythonExecutor. A remote choice constructs the corresponding provider and currently rejects managed agents.

The system prompt receives tool descriptions, managed-agent descriptions, authorized imports, and the expected code syntax. This matters because the model is not generating an arbitrary standalone program. It is generating against a capability environment assembled by the host.

run() resets one lane, not both.

The relevant part of MultiStepAgent.run() is:

if reset:
    self.memory.reset()
    self.monitor.reset()

self.memory.steps.append(TaskStep(task=self.task))

if getattr(self, "python_executor", None):
    self.python_executor.send_variables(variables=self.state)
    self.python_executor.send_tools({**self.tools, **self.managed_agents})

There is no executor cleanup, reconstruction, or namespace clear on this path. For a local executor, self.state belongs to LocalPythonExecutor; for a remote executor, a stateful kernel remains behind the same interface.

The precise meaning of reset=True is therefore “start a fresh transcript,” not “start in a clean execution environment.” That distinction affects test isolation, multi-tenant use, and any application that treats a new task as a new security context.

Typed memory becomes provider-neutral messages.

_run_stream() starts at step one and optionally inserts a PlanningStep according to planning_interval. It creates an ActionStep with timing and images, then delegates to the subclass.

CodeAgent._step_stream() calls write_memory_to_messages(). System, task, planning, prior model output, code, observation, and error become provider-neutral chat messages. This is the state the next model call can reason about.

The model returns either text containing the configured code delimiters or an internal structured response. parse_code_blobs() extracts the action and fix_final_answer_code() normalizes final-answer syntax.

A whole program becomes one synthetic tool call.

The next lines define the execution boundary:

tool_call = ToolCall(
    name="python_interpreter",
    arguments=code_action,
    id=f"call_{len(self.memory.steps)}",
)
memory_step.tool_calls = [tool_call]

If the program invokes get_weather() five times, then calls get_air_quality() twice and save_report() once, the top-level ActionStep.tool_calls list still contains one python_interpreter event.

This is not an accidental omission. It is the observability consequence of treating a program as the action boundary. ToolCallingAgent, in contrast, records individual model tool calls and results and can dispatch independent calls in a thread pool.

The code action makes composition cheap, but an authorization or billing system cannot infer exact capability use from ActionStep.tool_calls alone. It needs instrumentation in tool wrappers, the executor, or the network and storage boundaries.

The executor owns the real effects.

LocalPythonExecutor.__call__() sends the code to an AST evaluator with its persistent self.state dictionary. The evaluator applies controls around imports, dangerous built-ins, dunder access, operations, loop counts, print size, and time. It returns:

CodeOutput(
  output,
  logs,
  is_final_answer
)

When the action does not call final_answer(), the captured logs and last value become ActionStep.observations; the model can repair or continue in the next iteration. Variables remain available in the executor namespace.

Remote executors preserve the same contract. They send tool source definitions and variables into a remote kernel. final_answer is patched into a special exception-like termination mechanism, serialized across the boundary, and decoded into a host-side CodeOutput.

That small interface is one of the project's strongest designs: the model loop does not need to know whether Python ran in-process, in a notebook-like kernel, or in a remote container.

Errors become observations after effects may already exist.

Parsing, unauthorized imports, interpreter failures, and other AgentError instances are attached to the ActionStep. The outer loop finalizes and appends the step, so the next model call can see the failure.

This is effective feedback, but it is append-only evidence. It does not undo the prefix of the program that already ran, nor any external side effect performed by a tool.

Checks and callbacks run after the action.

When the executor returns a final answer, _run_stream() evaluates final_answer_checks. In finally, _finalize_step() records timing and monitoring data and invokes step callbacks before the step is appended.

Those extension points are useful for postconditions, telemetry, and audit export. They are not a pre-tool approval mechanism for individual function calls hidden inside a code action.

Three state-semantics experiments.

I ran these against the pinned source snapshot with deterministic scripted models or the local executor. No network tool and no external side effect was involved.

Experiment A: reset clears the transcript, not the heap.

The two-run example from the opening produced:

run_1 = 41
executor_has_x = 41
run_2(reset=True) = 42
memory_after_run_2 = [TaskStep, ActionStep]
memory_contains_first_code = false
executor_x_after_run_2 = 41

The second model-visible history has no origin for x, while the executor can still resolve it. This is not ordinary conversational continuation. It is a split reset contract.

An application can handle it, but it must choose deliberately:

reconstruct the executor for each task;
explicitly clear selected variables;
keep a session-scoped heap and label reset as transcript-only;
or checkpoint both lanes with an application-defined serializer.

Experiment B: an error is not a rollback.

I evaluated:

ledger = []
ledger.append("charged")
1 / 0

The executor raised InterpreterError, but:

ledger_after_error = ["charged"]

The mutation before the exception survived. Replace the list append with an email, payment, ticket creation, or database write and the production implication is obvious.

Tools with material effects need their own safety protocol: idempotency keys, dry runs, confirmation, transactions, or compensating actions. “The model will see the exception and reflect” is not a recovery strategy.

Experiment C: the local timeout is not preemptive.

The current timeout decorator submits the evaluator to a ThreadPoolExecutor, then waits on:

future.result(timeout=timeout_seconds)

I configured timeout_seconds=0.2 and authorized time, then ran:

import time
time.sleep(1.2)
y = 7

The result was:

exception = ExecutionTimeoutError
wall_time = 1.21s
state_y = 7

The call returned around the natural end of the 1.2-second sleep, and the worker still modified shared state. Why? Leaving the with ThreadPoolExecutor(...) block waits for its worker, even after future.result() has raised a timeout.

Open issue #2197 describes the same behavior, and PR #2263 proposes avoiding the wait. The repository's focused custom-timeout test passes, but its sleep(2) with a one-second timeout takes about 2.01 seconds because the test asserts the exception type rather than the response deadline.

If a hard deadline matters, the host needs a killable process, kernel, or container. A timeout that stops waiting is not equivalent to a runtime that stops executing.

Sandbox, persistence, and deployment fit

The local evaluator's restrictions are valuable. They reduce accidental imports, runaway loops, excessive output, and several obvious introspection paths. They do not turn same-process execution into a security boundary.

The official secure-execution guide recommends isolated executors for untrusted generated code. Yet “remote” is not one property:

filesystem mounts determine what code can read or change;
network policy determines which services it can reach;
secret injection determines what credentials it can exfiltrate;
CPU, memory, and wall-clock quotas determine denial-of-service exposure;
image provenance determines the supply-chain boundary;
cleanup determines whether state survives between users or tasks.

Serialization is part of that protocol. Remote executors use SafeSerializer; allow_pickle=False is the default, permitting JSON-safe values. Turning pickle on supports more Python objects but reintroduces an arbitrary-code-execution surface during deserialization. The friction of a safe serialization boundary is a feature, not merely an inconvenience.

Remote kernels are also stateful. They move the heap behind an isolation boundary, but they do not automatically turn it into a durable, deterministic checkpoint. Security isolation and workflow persistence solve different problems.

Persistence saves configuration, not the run.

MultiStepAgent.to_dict() serializes the model, tools, managed agents, prompt templates, planning and run settings. CodeAgent.to_dict() adds authorized imports and executor settings. The source explicitly skips final_answer_checks and step_callbacks.

It does not serialize AgentMemory steps or the live executor heap. Open issue #1216 asks for save/load support for agent memory; related discussions propose governance callbacks, lifecycle hooks, and callback-based persistence.

That distinction should shape API naming. Saving an agent definition is not saving an execution. A resumable code agent needs, at minimum:

agent definition
+ model-visible transcript
+ executor state or a reproducible rebuild recipe
+ tool-side effect ledger
+ capability and secret policy
+ code / dependency / image versions

Even that does not guarantee deterministic replay if tools depend on live external systems.

Control-surface comparison.

| Approach | Primary strength | Primary cost | |---|---|---| | ToolCallingAgent | Individual calls are structured and easier to authorize or audit | Loops and composition can require more inference turns | | CodeAgent | One action can contain control flow and work with live objects | Internal calls collapse into one interpreter event; heap is hard to persist | | LangGraph | Checkpoints, threads, and interrupts have explicit semantics | A heavier graph and state schema must be designed | | Handwritten Python workflow | Most deterministic and testable | Poor fit for open-ended tasks whose control flow is unknown |

CodeAgent fits workloads in which the system must discover an algorithm: research, data exploration, document processing, or prototype automation. Once a high-value path becomes stable, it should be extracted into ordinary tested functions or a durable graph, leaving only genuinely open decisions to the agent.

Let the model discover the workflow. Let software engineering own the workflow once it is known.

Deployment fit.

Analysts, researchers, data engineers, and notebook-oriented developers are the obvious audience. Their work already consists of variables, tables, loops, filters, and intermediate objects. Python is closer to their natural working medium than a procession of isolated buttons.

CodeAgent is a strong fit when:

the task is exploratory or read-heavy;
failures are cheap and retryable;
intermediate objects are expensive to serialize into text;
the control flow is not known in advance;
fast iteration matters more than durable replay.

It needs a larger system around it when:

money, permissions, or regulated data are involved;
a run must resume across process or day boundaries;
exact replay and causal audit are required;
actions are irreversible;
multiple tenants must not share runtime residue.

The missing pieces are not “a better prompt.” They are durable state, tool-level telemetry, pre-action policy, idempotency, compensating actions, and a killable execution boundary.

Operational conclusion.

Code agents blur the boundary between automation and software generation. Historically, a person wrote a durable program and ran it repeatedly. Now a model synthesizes a tiny program that may live for only a few minutes inside one task.

As tool catalogs become commoditized, value moves toward the runtime around that ephemeral software: execution, state governance, identity, authorization, auditing, cost control, and recovery.

The developer's role changes with it. Prompting matters, but capability design is harder and more durable work:

Which functions should exist?
Which values should cross the executor boundary?
Which calls are safe to retry?
Which actions require a proposal and approval?
What compensates for a half-completed program?

The more freely an agent can write code, the more the platform resembles an operating system for temporary software.

This model also creates a new class of false confidence. The final answer looks correct, and the transcript looks clean, so the process appears explainable. In reality, decisive objects may exist only in the heap, while eight capability calls appear as one python_interpreter event.

Agent observability cannot stop at chat history. It needs the equivalent of distributed tracing: capability invocations, state diffs, resource budgets, and causal links across the model, executor, and external systems.

smolagents is valuable precisely because it exposes this tension in a small, readable runtime. Python gives an agent freedom. That freedom forces the engineer to account for what the code can remember, touch, and leave behind.

Conclusion.

smolagents does more than replace JSON tool calls with Python. It lets a model submit a composable policy in one turn and keep structured intermediate values alive across steps. In doing so, it creates two sources of runtime truth: the typed transcript and the executor heap.

As a research and prototyping runtime, its interfaces are thin, the call path is legible, and the action language is expressive. As a durable transaction engine, it still needs checkpointing, rollback semantics, per-capability governance, and hard deadlines.

The important question for a code agent is not only whether the model can write correct code. It is: who can see, persist, revoke, and repair the state that code leaves behind?

Source index and open questions

The complete version calibration, experiment output, and focused test record are in the research evidence file.

Open questions.

If reset=True clears the transcript but not the executor heap, is it a new conversation, a new task, or neither?
When one Python action invokes ten tools, should the audit unit be the program, each capability call, or every state diff?
Once an agent discovers a stable workflow, which decisions should remain open to the model and which should become ordinary tested software?