Discussion

Search code, repositories, users, issues, pull requests...

eluded7: Personally I would probably always reach for a docker container if I want a sandboxed command that can run identically anywhere.I appreciate that alternate sandboxing tools can reduce some of the heavier parts of docker though (i.e. building or downloading the correct image)How would you compare this tool to say bubblewrap https://github.com/containers/

ebb_earl_co: The text says that it uses OS-level tools, specifically bubble wrap on Linux.

alyxya: Cool project, and I think there would be a lot of value in just logging all operations.

kimixa: For just logging would it really give any more info than a trace already does?

time0ut: Very interesting. I just started researching this topic yesterday to build something for adjacent use cases (sandboxing LLM authored programs). My initial prototype is using a wasm based sandbox, but I want something more robust and flexible.Some of my use cases are very latency sensitive. What sort of overhead are you seeing?

afshinmeh: I added a benchmark test (Apple M5) and on average I'm seeing 10ms overhead. I added a benchmark section to the repo as well https://github.com/afshinm/zerobox?tab=readme-ov-file#perfor...Also, I'm literally wrapping Claude with zerobox now! No latency issues at all.

volume_tech: the credential injection via MITM proxy is the most interesting part to me. the standard approach for agents is environment variables, which means the agent process can read them directly. having the sandbox intercept network calls and swap in credentials at the proxy layer means the agent code has a placeholder and never sees the real value -- useful when running less-trusted agent code or third-party tools.the deny-by-default network policy also matters specifically for agent use: without it there is nothing stopping a tool call from exfiltrating context window contents to an arbitrary endpoint. most sandboxes focus on filesystem isolation and treat network as an afterthought.

afshinmeh: Thanks and agreed! Zerobox uses the Deno sandboxing policy and also the same pattern for cred injection (placeholders as env vars, replaced at network call time).Real secrets are never readable by any processes inside the sandbox:```zerobox -- echo $OPENAI_API_KEYZEROBOX_SECRET_a1b2c3d4e5...```

simonw: This looks really good - the CLI interface design is solid, and I especially like the secrets / network proxy pattern - but the thing it needs most is copiously detailed documentation about exactly how the sandbox mechanism works - and how it was tested.There are dozens of projects like this emerging right now. They all share the same challenge: establishing credibility.I'm loathe to spend time evaluating them unless I've seen robust evidence that the architecture is well thought through and the tool has been extensively tested already.My ideal sandbox is one that's been used by hundreds of people in a high-stakes environment already. That's a tall order, but if I'm going to spend time evaluating one the next best thing is documentation that teaches me something about sandboxing and demonstrates to me how competent and thorough the process of building this one has been.

kjok: > There are dozens of projects like this emerging right now. They all share the same challenge: establishing credibility.Care to elaborate on the kind of "credibility" to be established here? All these bazillion sandboxing tools use the same underlying frameworks for isolation (e.g., ebpf, landlock, VMs, cgroups, namespaces) that are already credible.

jbverschoor: Again, it’s blacklisting so kind of impossible to get right. I’ve looked at this many times, but in order for things to properly work, you have to create a huge, huge, huge, huge sandbox file.Especially for your application that you any kind of Apple framework.

simonw: This doesn't look like it's blacklisting to me. It's an allowlist system: --allow-net=api.openai.com # Explicitly allow access to that host --allow-write=config.txt # Explicitly allow write to that file

afshinmeh: That's correct. The pattern is: reads allowed, write and network I/O blocked by default.```zerobox -- curl https://example.comCould not resolve host: example.com```

afshinmeh: That's right. It uses the same kernel mechanisms as Docker, the runtime is different though (bwrap on linux, seatbelt on mac, etc.)

simonw: Oh so it allows ALL file reads?I'd feel safer with default-deny on reads as well, but I know from past experience that this gets tricky fast - tools like Node.js and uv and Python all have a bunch of files they need to be able to read that you might not predict in advance.Might still be possible to do that in a DX-friendly way though, if you make it easy to manually approve reads the first time and use that to build a profile that can be reused on subsequent command invocations.

afshinmeh: I agree and you can deny all reads like this:```zerobox --deny-read=/ -- cat /etc/passwd```That being said, what the default DX shouldl be? What paths to deny by default? That's something I've been thinking about and I'd love to hear your thoughts.

simonw: That's a really tough question. I always worry about credentials that are tucked away in ~/.folders in my home directory like in ~/.aws - but you HAVE to provide access to some of those like ~/.claude because otherwise Claude Code won't work.That's why rather than a default set I'm interested in an option where I get to approve things on first run - maybe something like this: zerobox --build-profile claude-profile.txt -- claude The above command would create an empty claude-profile.txt file and then give me a bunch of interactive prompts every time Claude tried to access a file, maybe something like: claude wants to read ~/.claude/config.txt A) allow that file, D) allow full ~/.claude directory, X) exit You would then clatter through a bunch of those the first time you run Claude and your decisions would be written to claude-profile.txt - then once that file exists you can start Claude in the future like this: zerobox --profile claude-profile.txt -- claude (This is literally the first design I came up with after 30s of thought, I'm certain you could do much better.)

alyxya: Forgot about that, was mostly thinking about how AI agents with unrestricted permissions would ideally have some external logging and monitoring, so there would be a record of what it touched. A trace has all of the raw information, so some kind of wrapper around that would be useful.

simonw: Do you know if there's a widely shared name for this pattern? I've been collecting examples of it recently - it's a really good idea - but I'm not sure if there's good terminology. "Credential injection" is one option I've seen floating around.

afshinmeh: Not sure. I took this idea from the Deno sandboxing docs. They also do the exact same thing, different sandboxing mechanism though (I think Deno has it's own way of sandboxing subprocesses).

afshinmeh: Fantastic! I like that idea. I'm also exploring an option to define profiles, but also have predefines profiles that ships with the binary (e.g. Claude, then block all `.env` reads, etc.)

mdavid626: I trust sandbox-exec more, or Docker on Linux. Those come from the OS, well tested and known.MITM proxy is nice idea to avoid leaking secrets. Isn’t it very brittle though? Anthropic changes some URL-s and it’ll break.

afshinmeh: Thanks for sharing that. Zerobox _does_ use the native OS sandboxing mechanisms (e.g. seatbelt) under the hood. I'm not trying to reinvent the wheel when it comes to sandboxing.Re the URLs, I agree, that's why I added wildcard support, e.g. `*.openai.com` for secret injection as well as network call filtering.

mdavid626: How do you intercept network traffic on mac os? How do you fake certificates?

simonw: The problem is that those underlying frameworks can very easily be misconfigured. I need to know that the higher level sandboxing tools were written by people with a deep understanding of the primitives that they are building on, and a very robust approach to testing that their assumptions hold and they don't have any bugs in their layer that affect the security of the overall system.Most people are building on top of Apple's sandbox-exec which is itself almost entirely undocumented!

kjok: > The problem is that those underlying frameworks can very easily be misconfigured.Agreed. I'm sure a number of these sandboxing solutions are vibe-coded, which makes your concerns regarding misconfigurations even more relevant.