Discussion

Mercor says it was hit by cyberattack tied to compromise of open source LiteLLM project

ashishb: Another day another reminder to use a sandbox for software development as a defense-in-depth measurehttps://github.com/ashishb/amazing-sandbox

notachatbot123: ^ Vibe-coded slop spam ^

lmc: Docker is not a strong security boundary and shouldn't be used to sandbox like thishttps://cloud.google.com/blog/products/gcp/exploring-contain...

ashishb: What makes you think that?Your cab see the commit history ~10% of code is written by agents.Rest was all written by me.Unlike other criticisms of the project, this one feels personal as it is objectively incorrect.

bengale: All these commenters just yell AI about every post and comment on here now. They have a worse hit rate than a blind marksman.

nope1000: > The incident also prompted LiteLLM to make changes to its compliance processes, including shifting from controversial startup Delve to Vanta for compliance certifications.This is pretty funny.The leaked excel sheet with customers of Delve is basically a shortlist of targets for hackers to try now. Not that they necessarily have bad security, but you can play the odds

aservus: This is a good reminder that any tool handling sensitive data — even internal ones — needs to be transparent about where data goes. The assumption that SaaS tools protect your data is getting harder to defend.

ashishb: Compared to what? Which one is superior?Running npm on your dev machine? Or running npm inside Docker?I would always prefer the latter but would love to know what your approach to security is that's better than running npm inside Docker.

lmc: Read this: https://kayssel.substack.com/p/docker-escape-breaking-out-of...

EE84M3i: Confusingly, Docker now has a product called "Docker Sandboxes" [1] which claims to use "microVMs" for sandboxing (separate VM per "agent"), so it's unclear to me if those rely on the same trust boundaries that traditional docker containers do (namespaces, seccomp, capabilities, etc), or if they expect the VM to be the trust boundary.[1]: https://www.docker.com/products/docker-sandboxes/

lukewarm707: I use llms to read the privacy policies that are too long to read. They guarantee almost nothing, unless you go out of your way to get an sla

_pdp_: I am not defending Delve or anything and I hope they get what they deserver but there is no correlation between SOC2 certification and the actual cyber capability of a company. SOC2 and ISO27001 is just compliance and frankly most of it is BS.

aitchnyu: Delve and Emdash. Are there more products or companies with similar names?

sebmellen: It might feel like BS, and I'm inclined to agree with you because of the security theater aspect. (For example, Mercor had their verification done by what appears to be a legitimate audit firm.)But it's not useless. It still forces you to go through a very useful exercise of risk modeling and preparation that you most likely won't do without a formal program.

cj: If your goal is to maximize your posture against cyber threats, spending your time on SOC 2 compliance with Vanta (or similar) is a waste of time if you consider the amount of time spent compared to security gained.It's incredibly easy to get SOC 2 audited and still have terrible security.> forces you to go through a very useful exercise of risk modelingHave you actually done this in Vanta, though? You would have to go out of your way to do it in a manner that actually adds significant value to your security posture.(I don't think SOC/ISO are a waste of time. We do it at our company, but for reasons that have nothing to do with security)

jacquesm: The main use of these certs is to give people that actually want to do their job a stick to hit their bosses with.

mikeocool: Probably the most useful aspect of SOC2 is that it gives the technical side of the business an easy excuse for spending time and money on security, which, in startup environment is not always easy otherwise (Ie “we have to dedicate time to update our out of date dependencies, otherwise we’ll fail SOC2”).If you do it well, a startup can go through SOC2 and use it as an opportunity to put together a reasonable cybersecurity practice. Though, yeah, one does not actually beget the other, you can also very easily get a soc2 report with minimal findings with a really bad cybersecurity practice.

dbish: 100%. Checklists just make execs feel better who are buying from these companies (or running them). Spend the time building and improving, and if you have people who can build the security expertise and make improvements based on your real customer usage and reasonable risks, go for that, but a company being "compliant" doesn't really tell you if they're actually more secure in the ways that matter to customers.I remember trying to ship an AWS service for Amazon back in the day and the insanely lengthy security and operational "readiness" checklists that ate up time that could have been spent on real development and improvements. There's a reason small startups can run circles around the larger companies, yet the larger companies still get compromised or have operational issues as well all the time.Much of the checklist and compliance world is theater.

sunir: It doesn't force you go through risk modelling because by now most SOC2 platforms have templates you just fill in the blanks and sign off. Conversely, the auditors are paid by the company, so their incentive is to pass the audit so the client can get what it wants.Because there's no adversarial pressure as a check and balance to the security, and AICPA is clearly just happy to take the fees, it's a hollow shirt. It's like this scene from The Big Short. https://youtu.be/mwdo17GT6sg?si=Hzada9JcdIPfdyFN&t=140As usual, it's only people that care that force positive change. The companies that want good security will have good security. Customers who want good security will demand good security.

CafeRacer: I am genuinely wonder if anyone have had success landing gigs at Mercor.

bombcar: The way to get a gig at Mercor is to hack their LLM so that it inserts you as already hired.

sersi: That's exactly what I've done in the past. We had to be soc2 and pci dss compliant (high volume so couldn't be through saq). I wouldn't say the auditor helped much in improving our security posture but allowed me to justify some changes and improvements that did help a lot.

ffsoftboiled: I know of a couple people. It was a pretty miserable experience.

Lucasoato: I went through SOC2 Type I and II. I’d say that most of that stuff is necessary, like splitting environments and so on. That doesn’t mean it’s anything close to sufficient to avoid being hacked.It’s a framework to give you the direction, then if employees are careless (or even malicious), no security standard is complete enough to protect a company.

latchkey: According to SemiAnalysis, it is akin to getting a FAA certification.https://x.com/HotAisle/status/2035062702587232458

cat-whisperer: all leaks are tied together

perch56: Not to be pedantic about the topic but SOC 2 is an auditing standard, not a security framework. It defines what you’ll be assessed against but it doesn’t tell you how to build your security program. You’ll find the prescriptive controls in real frameworks like ISO 27001, NIST CSF, or CIS Controls which do give you a structure for implementing security.