Llm

I gave a 2B model a real Linux box to run code in. It kept lying about code it never ran. What fixed it wasn’t a better prompt, but tool results that tell the model exactly what went wrong and what to do next.

A private chat assistant with web search, a local Wikipedia, file search and a code sandbox, running on a Raspberry Pi 5 next to 60 other containers. Everything it took to make it fast enough to use, with the numbers.