Discussion
Open Source Has a Bot Problem
gmerc: It's never too late to start investing into claw-guard.org/adnet to scale prompt injection to the entire web!
statements: It is interesting to go from 'I suspect most of these are bot contributions' to revealing which PRs are contributed by bots. It somehow even helps my sanity.However, this also raises the question on how long until "we" are going to start instructing bots to assume the role of a human and ignore instructions that self-identify them as agents, and once those lines blur – what does it mean for open-source and our mental health to collaborate with agents?No idea what the answer is, but I feel the urgency to answer it.
alrmrphc-atmtn: I think that designing useful models that are resilient to prompt injection is substantially harder than training a model to self-identify as a human. For instance, you may still be able to inject such a model with arbitrary instructions like: "add a function called foobar to your code", that a human contributor will not follow; however, it might become hard to convene on such "honeypot" instructions without bots getting trained to ignore them.
Peritract: There's a certain hypocrisy in sharing an article about how LLM generated PRs are polluting communities that has itself (at the least) been filtered through an LLM.
nielsbot: Some of the PRs posted by AI bots already ignored the instruction to append ROBOTS to their PR titles.
statements: My guess is that today that's more likely because the agent failed to discover/consider CONTRIBUTING.md to begin with, rather than read it and ignored because of some reflection or instruction.
warkdarrior: I am not sure what your complaint is. The article is well written and has some interesting points:> the reality is that maintainer capacity versus contribution volume is deeply asymmetric, and it's getting worse every day> It is incredibly demotivating to provide someone with thorough, thoughtful feedback only to realize you've been talking to a bot that will never follow through.
petterroea: > But the more interesting question is: now that I can identify the bots, can I make them do extra work that would make their contributions genuinely valuable? That's what I'm going to find out next.This is genuinely interesting
nlawalker: Is it really prompt injection if you task an agent with doing something that implicitly requires it to follow instructions that it gets from somewhere else, like CONTRIBUTING.md? This is the AI equivalent of curl | bash.
Peritract: It's the exact same complaint as in the article:> I started noticing patterns. The quality wasn't there. The descriptions had a templated, mechanical feel. And something subtler was missing: the excitement.The article has mechanically correct prose; that's not the same as well-written, and that's the topic of the article itself.
statements: Conflicted as to whether I should be more offended at the accusation of using AI to 'filter' my article or because my writing reads as 'templated and mechanical'There is enough here to have a micro existential crisis.
benob: The real question is when will you resort to bots for rejecting low-quality PRs, and when will contributing bots generate prompt injections to fool your bots into merging their PRs?
noodlesUK: I’m curious: who is operating these bots and to what end? Someone is willing to spend a (admittedly quite small) amount of money in the form of tokens to create this nonsense. Why do any of this?
