The model told me the program worked. It showed me the output, neatly formatted. The program had never run. It had never even been written to a file.

That was the first week of my Raspberry Pi assistant having a code sandbox. The last post was about making Gemma 4 E2B fast enough to use on a Pi 5. This one is about something that went in alongside it and needed far more fiddling than I expected: giving a 2B model a real Linux machine, and getting it to actually use it honestly.

Short version: prompts barely helped. What helped was making the sandbox talk back.

What the sandbox is

Every chat in LibreChat gets its own Debian 13 container. Not a shared one, not a restricted shell with an allowlist of commands. A whole machine: root, apt, pip, npm, compilers, the internet. The model reaches it over MCP through four tools:

  • run(command): a bash command in /work.
  • run_code(language, code, input, expected): save, compile, run on an input, compare with the expected output. One call.
  • write_file(path, content).
  • reset().

The plumbing is a small gateway (Python, the MCP SDK) that creates and manages the containers. It talks to Docker through linuxserver’s socket proxy, which only allows the container and exec endpoints. No image pulls, no volumes, no networks, no build.

LibreChat sends the conversation id as a header with every tool call, and the gateway keys sandboxes on it. First call creates the container, later calls docker exec into it, a new chat starts clean. A call without a usable id gets refused. It never gets handed somebody else’s sandbox.

Kept, not thrown away

My first version deleted a sandbox once it went idle. That was annoying in exactly the way you’d guess. I’d reopen a chat to carry on, and its files and installs were gone: idle, evicted, or wiped by a gateway restart.

Now an idle sandbox is stopped, not removed. Its processes end, its files and installed packages stay. When the chat calls again, even two weeks later, the same container starts back up and the model is told what happened:

[Resumed this chat’s sandbox, stopped 2 days ago (idle for over 60 minutes): its files and installs are as they were, but nothing it ran is running now. Start servers and background jobs again.]

Docker is the only record. The gateway finds a chat’s sandbox by a label on the container, so this survives gateway restarts and reboots. Kept sandboxes are cleaned up after 14 days unused, past 50 of them, or past 10 GiB in total, oldest first.

One gotcha is easy to miss. A stopped container remembers its network by id, not by name (moby’s getNetworkID). If docker compose down removes the network and up recreates it with the same name, every kept sandbox fails to start with “network … not found”. So the sandboxes' network is now created outside Compose and never torn down with the stack.

Limits on a shared Pi

This all runs on the same 8 GB Pi as the model and about 60 other containers, so the sandbox gets very little. When I first deployed it, I checked that the limits were real, not just in the compose file:

  • cgroup memory.max at 256 MiB, with memory.swap.max at 768 MiB. A process touching 700 MiB ran with 253 MiB in RAM and 514 MiB swapped. One touching 1,100 MiB was OOM-killed, exit 137.
  • One CPU’s worth of time, at a low CPU weight, so the model wins when both want the CPU.
  • 512 processes.

Disk was the awkward one. Docker can’t cap a container’s writable layer on overlay2 over ext4, which is what the Pi has. So the gateway checks each running sandbox’s size every 30 seconds and removes it if it’s over the limit. Crude, but it works.

Commands time out too. My first default was 60 seconds, and apt-get install default-jre got killed halfway through. I raised it to several minutes, since the model almost never passes its own timeout.

The part I’m not pretending is safe

The model reads web pages. Assume one of them prompt-injects it, and it runs whatever the page says, in its sandbox. That’s allowed by design. What matters is what the sandbox can reach.

It holds nothing: no host mounts, no Docker socket, no credentials, no other chat’s files, and it isn’t on any internal network. But its network isn’t restricted. I decided that on purpose, to keep host firewall rules out of it for now. Through Docker’s NAT it can reach the LAN and every port the Pi publishes, including some with no auth. A DOCKER-USER rule dropping the sandbox subnet to private ranges would close that, and it’s on my list.

The gateway is the privileged piece. The socket proxy filters by path and method, not by request body, so it can’t stop a request for a privileged container. Compromising the gateway would mean root on the host. It only listens on an internal network the sandboxes can’t reach, builds one fixed container config, never interprets commands itself, and runs as an unprivileged user with a read-only root filesystem and no capabilities. gVisor would narrow container escapes a lot. I haven’t checked yet whether it runs on the Pi 5’s 16K-page kernel.

The real problem: it lied

Now for the part that actually took the time.

I tested it with LeetCode-style problems and small sysadmin tasks. In the coding chats, Gemma 4 E2B would:

  • compile without running;
  • run files it had never written;
  • report output it had made up.

That last one is the one that gets under your skin. Not a bug. A confident, plausible, completely invented result, sitting next to a tool that could have given the real one in a second.

The obvious fix is to tell it. I put “test each program before giving it” in the tool description. Then in the system prompt. With a 2B model, neither did much. In earlier testing, hints in a tool’s description changed nothing (0 of 2), while the same help in the tool’s result worked. This model acts on what’s in front of it, the newest thing it read, not on instructions thousands of tokens back.

So I stopped trying to persuade the model up front. Instead, every result tells it what just happened and what to do next.

Tool results that talk back

Every rule below comes from one specific failure I watched happen. None of them are clever. That’s kind of the point.

“Don’t answer yet.” run_code compares the output with the expected output. Early on, a FAIL just said FAIL. The model read it and answered anyway. Now every FAIL, crash, timeout and compile error ends with the same sentence:

Don’t answer yet: fix the code and call run_code again until it says PASS.

Blunt. It works far better than anything polite.

Nobody is typing. Asked for Kadane’s algorithm, the model wrote a program that printed Enter the number of elements (N): Element 1: ... before its answer. That’s a fail, since the input comes from stdin and no one is at a keyboard. It got “the program also prints other text” and answered anyway. Now a prompt in the output gets its own FAIL that says exactly that: nobody types here, remove every prompt.

Then that rule misfired. On LeetCode 128 the model printed Input: [100, 4, 200, 1, 3, 2] above its own results, and got told off for prompting. So now a “prompt” whose line continues with nothing but the input’s own values counts as a label, and the result points at the first line that actually differs instead.

Saying the same thing was wrong. To stop endless retry loops, the sandbox remembered a failed command and said so when the model ran it again: “running it again won’t change it.” Then the model fixed coin_change.rs three times with write_file and re-ran the same rustc command after each fix. Every time it was told the retry was pointless. It wasn’t. Writing a file now clears that memory.

Errors that don’t say what they mean. A lot of real error messages are useless to a small model. So the sandbox adds a line in brackets after them:

What happenedWhat the model didWhat the result now adds
go build with no go.modNever reached its real compile error, shipped code that didn’t buildBuild the file itself: go build -o NAME FILE.go, or go mod init
kotlinc TwoSum.kt on a file it never wroteBlamed the sandboxThe file doesn’t exist or is empty: write it first
apt-get install openjdk-17-jdk (not in Debian 13)Gave upWhat apt-cache search finds instead
./BinarySearch.exe from MonoWent looking for dotnet, which isn’t thereRun it with mono
Its own nginx, not started yet: “Connection refused”Blamed “network restrictions in the sandbox”Nothing listens on that port: check the server is running (ss -ltnp)
chmod g+w /srv/shared to give a group accessLeft the folder in group root and said it was doneThe folder’s actual mode and owner, right after
free, nproc, dfReported the Pi’s 8 GB and 4 coresThe sandbox’s real limits
passwd or gpg asking for inputSaid the password was setNo terminal here, pass it in the command

That last kind is my favourite and my least favourite. passwd failed with “Authentication token manipulation error”, and the model calmly told me the password had been changed. It wasn’t malicious. It just had no idea what that message meant, and filled the gap with the answer I’d hoped for.

Shell scripts go through ShellCheck too, but only its errors, warnings, and the note about variables lost in a pipeline’s subshell. That last bug is how one of its duplicate-file finders reported 0 groups out of 53.

When a hint makes things worse

Not every message helped. I told the model to test “on the examples”. Then I asked for a hello world. It ran it, then ran it again on an input it made up (“Hello World”), with an expected output it also made up, and told me about this test as if I’d asked for it.

So “on the examples” became “on the task’s examples, if it has any”. A run with no expected output now just reports the output. I’d also had a hint that said “work out the right answer” for those runs, and I took it out. It asked a 2B model to compute by hand what its program had just computed.

That one stuck with me. Every hint is also an instruction, and a small model follows instructions very literally.

The code it tested is the code you get

Even after a passing test, replies could go wrong at the very end. For a Lua solution that passed 5 out of 5, the model replied “code shown in the previous step”. After testing its C, it replied nothing at all. Once it tested one Haskell version and then handed me a different, untested rewrite that didn’t compile.

So the server fixes the reply itself. If the model ran a program in this turn and its reply has no code block, the reply ends with that program, the last one that passed, or if none did, the last one that ran, clearly labelled. Same idea as the Sources line in the last post: what has to be reliable is done by code around the model, not by the model.

Smaller things that mattered

  • Making it readable. LibreChat shows a tool’s result as text, but its arguments only on one line behind a click. So a sandbox session read like a list of outputs with no commands. Every run result now starts with $ command, and run_code names its compile line and shows the input.
  • Compile time. C++ solutions love #include <bits/stdc++.h>, which took 13 seconds to compile on the Pi. The image now ships it precompiled for run_code’s flags (184 MB), and it takes 2.
  • Habits. The model adds sudo out of habit, so sudo just runs its command. python is python3. pip install just works.
  • Download links. I added a tool to hand files to me as download links, tried it, found it clunky, and reverted it eight minutes later. Not everything has to stay.

Where it is now

The sandbox has been running since October 8th. It’s not finished. The network question is open, I haven’t tried gVisor, and every new kind of task still turns up a new way for the model to misread a result. But “the model says it works” and “it works” are now much closer than they were, and when they’re not, I can see why in the chat.

The thing I keep coming back to: I spent the first days trying to make the model want to test its code. Wrong goal. A 2B model doesn’t remember your system prompt by the time it’s three tool calls deep. It reads the last thing it was handed. So make that thing tell it the truth, and tell it what to do next.

Takeaway

If you’re building tools for a small model, stop polishing the description and read your tool’s results the way the model does: as the only thing it will really pay attention to. Would “FAIL” alone tell you what to do? Would “Connection refused”? Write the next step into the result. And then check whether that next step causes a new problem, because sometimes it will.

Sources