Discussion

Search code, repositories, users, issues, pull requests...

simianwords: Prompt injection?

nickphx: cool. if you choose to use a non-deterministic black box of bullshit, should you really be surprised when it shits all over your floor?

dataviz1000: Ok, people.Please ask Claude Code what is in `MEMORY.md`. After ask Claude Code what is in all `.claude/` directories. Remember to audit and prune them often.

BoorishBears: I was thinking surely scheduled tasks need to be explicitly invoked but nope: https://code.claude.com/docs/en/scheduled-tasks#set-a-one-ti...Some people are upset at my brave new world characterization, but yeah even as someone deriving value from Claude Code we've jumped the shark on AI in development.Either the industry will face that reality and recalibrate, or in 20 years we're going to look back on these days like the golden age of software reliability and just accept that software is significantly more broken than it was (we've been priming ourselves for that after all)

mhitza: People aren't upset about your characterization. Catch phrases, memes, or other low qualitative comments (with no context, elaboration or personal angle) are contrary to community ethos and down voted.

viccis: Feels like just yesterday that everyone agreed that critical code is read orders of magnitude more than written, so optimizing for quick writing is wrong.

jrvarela56: It’s a feature not a bug!

throw5: Isn't this a natural consequence of how these systems work?The model is probabilistic and sequences like `git reset --hard` are very common in training data, so they have some probability to appear in outputs.Whether such a command is appropriate depends on context that is not fully observable to the system, like whether a repository or changes are disposable or not. Because of that, the system cannot rely purely on fixed rules and has to figure intent from incomplete information, which is also probabilistic.With so many layers of probabilities, it seems expected that sometimes commands like this will be produced even if they are not appropriate in that specific situation. I guess, what I'm trying to say ... is this even a bug? Sounds like the model is doing exactly what it is designed to do.

simianwords: That's not how the systems work. Just by a thing being common in training data doesn't mean it will be produced.> I guess, what I'm trying to say ... is this even a bug? Sounds like the model is doing exactly what it is designed to do.False, it goes against the RL/HF and other post training goals.

throw5: > Just by a thing being common in training data doesn't mean it will be produced.That's not what I said at all. I never said it will be produced. I said there is some probability of it being produced.> False, it goes against the RL/HF and other post training goals.It is correct that frequency in training data alone does not determine outputs, and that post-training (RLHF, policies, etc.) is meant to steer the model away from undesirable behavior.But those mechanisms do not make such outputs impossible. They just make them less likely. The underlying system is still probabilistic and operating with incomplete context.I am not sure how you can be so confident that a probabilistic model would never produce `git reset --hard`. There is nothing inherent in how LLMs work that makes that sequence impossible to generate.

simianwords: It is meaningless to say that because the author was able to reproduce it multiple times.

TZubiri: tbf, that's claude's workspacedo not share a workspace with the llm, or with anybody for that matter.How would the llm even distinguish what was wrote by them and what was written by you ?

ghelmer: That is not my experience.

simianwords: I think this post potentially mischaracterises what may be a one off issue for a certain person as if it were a broader problem. I'm guessing some context has been corrupted?

claudiug: no more developers, all code is written alone /s

meander_water: Probably does it to reduce context for regex/git history searches

luxurytent: Not sure I understand, wouldn't permissions prevent this? The user runs with `--dangerously-skip-permissions` so they can expect wild behaviour. They should run with permissions and a ruleset.

throwaw12: you might be right, but consider the implications, if context can be corrupted in 0.1% cases and it starts showing another destructive behaviour, after creating 1000 tickets to agent, your data might be accidentally wiped off

throw5: > It is meaningless to say that because the author was able to reproduce it multiple times.I don't know how that refutes what I'm saying.The behaviour was reproduced multiple times, so it is clearly an observable outcome, not a one-off. It just shows that the probability of `git reset --hard` is > 0 even with RLHF and post-training.

fragmede: While that's obviously a bug which should be fixed, having stuff just sitting around uncommitted for days (which is much longer than 10 mins) is an anti-pattern (that I used to fall into).

gerdesj: Which is what?

bonoboTP: I agree that it's worrying that we're moving more and more towards implicit and opaque state. Hiding what exactly is getting edited, very limited tooling to check what the subagents are doing exactly, setting up scheduled and recurring tasks without it being obvious etc.It's tending more and more towards pushing the user to treat the whole thing as a pure chat interface magic black box, instead of a rich dashboard that allows you to keep precise track of what's going on and giving you affordances to intervene. So less a tool view and more magic agent, where the user is not supposed to even think about what the thing is even doing. Just trust the process. If you want to know what it did, just ask it. If you want to know if it deleted all the files, just ask it in the chat. Or don't. Caring about files is old school. Just care about the chat messages it sends you.

morganastra: the purpose of a system is what it does!

oelmgren: I'm curious how common this is or if this just affects this one user.

BoorishBears: This would be a more substantive comment if you also addressed the topic at hand as I did, rather than regurgitating the rules of the site.

pattilupone: I opened up Hacker News and I saw this right at the top, and I assumed it had started happening to everyone. I thought, good thing I'm not running Claude Code right now.

kccqzy: > Process monitoring at 0.1-second intervals found zero git processes around reset times.I don’t think this is a valid way of checking for spawned processes. Git commands are fast. 0.1-second intervals are not enough. I would replace the git on the $PATH by a wrapper that logs all operations and then execs the real git.

gerdesj: non sequitor.

jerukmangga: yes sir

SpicyLemonZest: Who knows whether permissions would prevent this? Anthropic's documentation on permissions (https://code.claude.com/docs/en/permissions) does not really describe how permissions are enforced; a slightly uncharitable reading of "How permissions interact with sandboxing" suggests that they are not really enforced and any prompt injection can circumvent them.

simianwords: If it reliably reproduces something undesirable with statistical significance, then it is a bug. It can be fixed with RLHF.

phyzome: It's an issue title. It means "this is what is happening for me".

gpm: The weird part is that it's "shitting over the floor" in quite a deterministic ma nner. Every 600seconds (+- less than 0.5 seconds) doing the exact same thing.

throw5: Yes, exactly. People often overlook that, even with guardrails, it is still probabilities all the way down.You can reduce the risk, but not drive it to zero, and at scale even very small failure rates will surface.

simianwords: I'm not sure what the argument is here.1. if the problem the post is suggesting is common enough, it is a bug and the extent needs to reduce (as you said)2. if it is not common and it happens only for this user, it is not a bug and should be mostly ignoredPoint is: the system is not something that is inherently a certain way that makes it unusable.

irishcoffee: I’m having this weird vision of a “the matrix 3” type machine crawling around inside Microsoft’s GitHub servers central repository and just wreaking havoc.This whole LLM thing is a blast, huh?

esafak: Stuff like this is why I think this existing sandboxing approaches fall short; this is the kind of stuff I worry about, not editing files that are already protected by git.

3eb7988a1663: It does make WH40k seem more plausible. Tech priests praying to the capricious machine spirit to just please do the thing.

ramses0: I'd been using cursor at work for a year or two now, figured I'd try it on a personal project. I got to the point where I needed to support env-vars, and my general pattern is `source ./source-me-local-auth` => `export SOME_TOKEN="$( passman read some-token.com/password )"` ...so I wrote up the little dummy script and it literally just says: "Hrm... I think I'll delete these untracked files from the working directory before committing!" ...and goes skipping merrily along it's way.Never had that experience in the whole time using cursor at work so I had to "take the agent to task" and ask it "WTF-mate? you'd better be able to repro that!" and then circle around the drain for a while getting an AGENTS.md written up. Not really a big deal, as the whole project was like 1k lines in and it's not like the code I'd hand-written there was "irreplaceable" but it lead to some interesting discussion w/ the AI like "Why should I have to tell you this? Shouldn't your baseline training data presume not to delete files that you didn't author? How do you think this affects my trust not just of this agent session, but all agent interactions in the future?"Overall, this is turning out to be quite interesting technology times we're living in.

jxcole: The obvious solution is to just copy paste it into Claude itself and ask it to fix. Works for almost any Claude problem

wswope: Sure looks to me like this whole case is Claude Code chasing its own tail, failing to debug, and offering to instead generate a bug report for the user when it can't figure out a better way forward. Maybe even submitting the bug report "agentically" without user input, if it's running on host without guardrails.

mememememememo: As a side note. Always configure remote to reject any kind of trunk push. And ideally any forced push on branches.

jeswin: It's not a one off issue - it has happened to me a few times. It has once even force pushed to github, which doesn't allow branch protection for private personal projects. Here's an example.1) claude will stash (despite clear instructions never to do so).2) claude will use sed to bulk replace (despite clear instructions never to do so). sed replacements make a mess and replaces far too many files.3) claude restores the stash. Finds a lot of conflicts. Nothing runs.4) claude decides it can't fix the problem and does a reset hard.I have this right at the top of my CLAUDE.md and it makes things better, but unlike codex, claude doesn't follow it to the letter. However, it has become a lot better now.NEVER USE sed TO BULK REPLACE.*NEVER USE FORCE PUSH OR DESTRUCTIVE GIT OPERATIONS*: `git push --force`, `git push --force-with-lease`, `git reset --hard`, `git clean -fd`, or any other destructive git operations are ABSOLUTELY FORBIDDEN. Use `git revert` to undo changes instead.

bschwindHN: When will you all learn that merely "telling" an LLM not to do something won't deterministically prevent it from doing that thing? If you truly want it to never use those commands, you better be prepared to sandbox it to the point where it is completely unable to do the things you're trying to stop.

throw5: Yes, if something is reproducible and undesirable, it is a bug and RLHF can reduce it. I'm not disupting that.My point is that fixing one bug does not eliminate the class of bugs. These systems are probabilistic and they can generate many different undesirable but plausible commands, each valid in some narrow context.So you fix one with RLHF, but others will still appear if you give it enough time. With git commands, there is not like a system like Lean that can formally reject invalid proofs. Really I think the mathematicians have got it easier with LLMs because a proof is either valid or invalid. It's not so clear cut with git commands. Almost any command can be valid in some narrow context, which makes it much harder to reject undesirable outputs entirely.Until the underlying probabilities of undesirable output become negligible so much that they become practically impossible, these kinds of issues will keep surfacing even if you address individual bugs.Will the probabilities become so low someday that these issues are practically impossible? Maybe. But we are not there yet. Until then, we should recalibrate our expectations and rely on deterministic safeguards outside the LLM.

kibwen: Let's focus on the real issue here, which is that HN has apparently normalized the double hyphen in the title to an en dash--yes, an en dash, not even an em dash.

rtpg: iOS keyboard autocomplete

chaos_emergent: Have you considered that Claude set up a crontab that does that programmatically? Every 10 mins seems awfully, idk, regular.

smallerize: But different projects are being reset at different times.

mtndew4brkfst: It has once even force pushed to github, which doesn't allow branch protection for private personal projects.This is only restricted for *fully free* accounts, but this feature only requires a minimum of a paid Pro account. That starts around $4 USD/month, which sounds worth it to prevent lost work from a runaway tool.

smallerize: Surely its copy and paste though?

jatora: With hooks you can enforce permissions much more concretely.

jatora: Reinforcing an avoidance tactic is nowhere near as effective as doing that PLUS enforcing a positive tactic. People with loads of 'DONT', 'STOP', etc. in their instructions have no clue what they're doing.In your own example you have all this huge emphasis on the negatives, and then the positive is a tiny un-emphasized afterthought.

refulgentis: I think you're generally correct, but certainly not definitively, and I worry the advice and tone isn't helpful in this instance with an outcome of this magnitude.(more loosely: I'm a big proponent of this too, but it's a helluva hot take, how one positively frames "don't blow away the effing repro" isn't intuitive at all)

DrewADesign: That’s right, because we’re not developers anymore— we orchestrate writhing piles of insane noobs that generally know how to code, but have absolutely no instinct or common sense. This is because it’s cheaper per pile of excreted code while this is all being heavily subsidized. This is the future and anyone not enthusiastically onboard is utterly foolish.

Jcampuzano2: You could prevent this even with --dangerously-skip-permissions with a simple pretooluse hook.

throw5: This! The safeguards need to be outside LLM and they need to be deterministic.Now I wish I could reject `git reset --hard` on my local system somehow.

namibj: Just fork git and patch that out? Can't be that hard just ask the agent for that patch. Don't need to update often either, so it's ok to rebase like twice a year.

bruce_one: eBPF is a great tool to use for debugging this kind of thing too, e.g. [bpftrace](https://bpftrace.org) has an [execsnoop](https://github.com/bpftrace/bpftrace/blob/master/tools/execs...) script for looking at everything being exec'd on the system :-)(No need to use bpftrace, just an easy example :-) )

byronsharman: I agree that it should be left as a double hyphen, but an en dash is far more appropriate considering the decades-long precedent set by LaTeX (and continued by Typst).

biglost: I use a script wrapper of git un muy path for claude, but as you correctly said, I'm not sure claude Will not ever use a new zsh with a differentPATH....

Jcampuzano2: Maybe stop using the CLAUDE.md to prevent it from running tools you don't want it to and just setup a hook for pretooluse that blocks any command you don't want.Its trivial to setup and you could literally ask claude to do it for you and never have any of these issues ever again.Any and all "I don't want it to ever run this command" issues are just skill issues.

namibj: That's a fee for not running a local git proxy with permissions enforcement that holds onto the GitHub credentials in place of Claude.

johnisgood: And it should be "--" to begin with, i.e. "--hard".