Discussion

How Jennifer Aniston and Friends Cost Us 377GB and Broke ext4 Hardlinks

bravetraveler: As is always the case, short vs long term... but I think I'd put effort into migrating to a filesystem that is aware of duplication instead of trying to recreate one with links

replooda: In short: Deduplication efforts frustrated by hardlink limits per inode — and a solution compatible with different file systems.

UltraSane: The real problem is they aren't deduplicating at the filesystem level like sane people do.

UltraSane: This makes them look rather incompetent. Storing the exact same file 246,173 times is just stupid. Dedupe at the filesystem level and make your life easier.

ameliaquining: Discourse is self-hostable; they can't require their users to use a filesystem that supports deduplication. (Or, well, they could, but it would greatly complicate installation and maintenance and whatnot, and also there would need to be some kind of story for existing installations.)

bravetraveler: Now I'm confused, is this discourse backing up to their filesystems or a user backing up to theirs? Said another way: I don't see why one couldn't be expected to manage a filesystem well under their own use.So the 'fix' creates symlinks on user sites that are backed up elsewhere, outside of their control?edit: This is a nice preparation/consideration for them, I guess, but I still maintain a backup store not aware of duplication at the filesystem is a mistake.