<- Back
Comments (47)
- LercI'm not sure how I feel about the Platonic Representation Hypothesis.In some ways it seems like it should be profound, but part of me thinks it might just be saying 'There is an optimal way to perform dimensionality reduction and as things get bigger it is easier to appoximate'
- bigcat12345678All strands of scientific studies eventually are bound by "physical" reality. This assumption means all these research would gradually become more and more relevant, like math problems are highly relevant to physics, computer science, psychology is similar to experimental physics (observation and conjecture theorizing eventually crystalize into analytical theorems). The list goes on and on.One central observation is that we are well into the realms of no single human can coherently reason a nontrivial amount of knowledge our our collective. By that I meant it used to be that a single human being can appear "knowing all" by possessing a significant amount of knowledge through life time learning. That's no longer possible even with the smartest minds of our generation. One such example would be the related financial failure of renowned genius (Newtown Tesla etc).Due to our innate limitations of mutual exclusion, that is, humans are inherently conflicting with each other, not even the basic instinct of humans can bound 2 together for innovative collaboration. I mean, I never saw couples being better business partners, research partners, etc. AI becomes inevitable, as they are inherently mixable.Yes, I am conjecturing that the next weave of AI research would be neuronal level mixing or hybridinh between models to enable a new bread of intelligence where the intelligence is mechanically compounded to produce inherently better results.
- theptipI guess the fundamental question would be, for what domains can we pretrain with “good views”?Part of the magic of next token prediction is it works to extract intelligence from the unstructured web corpus.Can JEPA formulate a similar scale training set of “good views”?
- kooi> 33M-parameter text JEPA reaches .125 on the PRH paper’s benchmark, against .034 on Flickr captions, level with Qwen2.5-3B and -7B. They need fewer samples.Im bullish on jepa models. Success of autoregressive next token prediction feels like a stroke of luck.
- kaijiaI am sick of theory papers that adds human interpretation to connect a mathematical result to some fancy terms that everyone seeks after in real-world ML applications. Ironically, chatGPT is the most productive way to strip off those advertisement so the essence is better exposed. https://chatgpt.com/share/6ae78178-c2ec-83ea-9bc3-fd304c64ec...
- inshardDo memes have a concept of proto memes? Like primes are the proto numbers that all other numbers are made from? Could there be similar ontological primes for all concepts / semantic space? Base reality concepts that are irreducible so to speak.
- dominotwhow come all the examples in demo have wrong answershttps://huggingface.co/spaces/galilai-group/prh_demo> A man in a black shirt and hat with a yellow neckerchief has his hand raised.no hand raised> A white sheep sitting on top of a cake.no its just cake sheep . sheep is not sitting> A man is standing on the beach while holding his surfboard.Its not just a maninfact all the examples are wrong.
- suddenlybananasI thought the platonic representation hypothesis was much more ambitious: wasn't it supposed to apply for any model across modalities (e.g. including human minds), rather than just the same model but with different modalities? I'll have to actually read the paper but I suspect there to be very strong assumptions which defeat the interesting aspects of the platonic representation hypothesis.
- anonundefined
- junkaccountI built a ppt with claude for investors with the exact same phrase "AI infrastructure for proving and improving ... " on Oct 27 and sent it to investors. I suggested the title during the chat and claude agreed a lot. It ends up as a title of this paper 2 days later.