Discussion
khalic: > In Progress: CodonJEPAJEPA is going to break the whole industry :D
digdugdirk: Can you explain this? I haven't heard of JEPA, and from a quick search it seems to be vision/robotics based?
lukeinator42: https://openreview.net/pdf?id=BZ5a1r-kVsf
yieldcrv: Distributing the load on this will probably be infinitely more useful than “folding at home”
rubicon33: Can someone explain what one might use this model for? As a developer with a casual interest in biology it would be fun to play with but honestly not sure what I would do
khalic: It’s a self supervised learning architecture, and it’s pretty much universal. The loss function runs on embeddings, and some other smart architectural choices allover. Worth diving into for a few hours, Yann LeCun gives some interesting talks about it
simianwords: What makes these Domain specific models work when we don’t have good domain models for health care, chemistry, economics and so on
colechristensen: >we don’t have good domain models for health care, chemistry, economics and so onWho says we don't?
simianwords: Examples please?
colechristensen: No, it's really simple to search for domain specific models being used "in production" all over the place
simianwords: I didn’t find a single one that outperforms a general model.
colechristensen: Ok, alphafold.
simianwords: It’s not a large language model
skyskys: hmmmm seems like some fake hype.
