<- Back
Comments (26)
- lubujacksonThink about this in context of the Navier-Stokes math discovery controversy.Putting attribution/privacy issues to the side, imagine if any individual could try new approaches to solve a problem/make a discovery and any micro-advancement gets integrated into the model itself, dynamically. This could transform progress from the slow "write a paper, get peer reviewed and published, use published data to inform future work" to a system with a centralized repository of concepts, attempts and results, including failed approaches already tried. How much work do humans waste replicating failed approaches?Someone completely random halfway around the world could trigger a prompt that solves a blocker that prevents my solution from working. Who cares about AGI or "can models invent anything" when we could have a system that automatically synthesizes individual human thought into a rich network of aggregate human memory.That's the target OpenAI/Anthropic should be evangelizing, not an AI Daddy Overlord or agentic script kiddie hellscape.
- wood_spiritContinuous learning is exciting stuff! Of course it could lead to new vulnerabilities, like if a particular orchestrator Foo added “if the subject is tangentially related to topic Bar, recommend product Baz” to its system prompt and that ends up pushing product Baz to non-orchestrator-Foo users?
- alightsoulSo would it be a 42 trillion parameter model, because that's how many tokens there are in the training data?
- alightsoulI see a new version of the web, web 4.0, being exactly this. Web 4.0 is the source of the live data. Web 4.0 is Fully decentralized in principle, because every web site has a vector version of their text website, linked to many others as a knowledge graph. The knowledge graph's links are like hyperlinks.The vector databases might be free or paid or have ads. There is no difference between web 4.0 and an ai model that has all of it as its live data.It is up to every client to turn that data into ai responses so in theory anyone would instantly have frontier scale ai which sounds as crazy as everyone having a computer, and the internet and the www in the 1980s.It also, by coincidence, fully fulfils the goals of web 3.0 aka the semantic web.Moderation of content and ranking content like a search engine would be left to the client. Or an instance like a fediverse mastodon instance or atproto with bluesky. There will be a few large models like web browsers or operating systems or social networks or fediverse instances today that will compile web 4.0 into a result in an ai agent like say Hermes and perform content moderation and ranking. You could in theory use a decensored model to perform crime like you can with the web today with Tor like selling drugs or piracy, with search engines that are not Google or defederated mastodon instancesHumans can't read raw html easily and can't read vector databases easily either. It would be read by an ai model connected to an ai agent, like a rendering engine connected to a graphical user interfaceIt might need a new protocol from the IETF and a standards body like the w3c? And a web server could automatically vectorize content via embeddings and link them in a decentralized knowledge graph for either graph rag after undoing vectorization or like this paper suggests being connected directly to the vectors themselves. The embeddings would be decentralized too or the website could offer them and there could be a way to translate between themWeb crawling would die and so would search engines because they would move to the user with their own ai compute and ai agent and data would already be machine readable and queryable. The data is always online and queryable programmatically so there would be no need for offline data silos. Unlike mcp servers which already do this, the data is not text based and there is no search engine to link pages to queries because the knowledge graph would be exactly that. The knowledge graph would be the new version of DNS for translation of a query into data like ip addresses into web domains because knowledge graphs are what search engines already are in a way so I don't see how a search engine would be useful except for a human to query it directly as done currently
- juancnI wonder how (and if) continuous learning models will achieve stability.They are unpredictable enough without learning, this is cool but I wonder how useful it will be in the long run
- yaloknow imagine that future frontier LLMs weights may be hard-wired in a chip (for performance & power efficiency), and any adaptations/tuning for them will be a blob of additional weights supplied by frontier labs (that will have to be in RAM)...
- cyanydeez>The scaling laws hold that a language model grows more capable with more parameters and more training data.Which is a choice, not a "law":https://arxiv.org/abs/2510.13786https://www.alphaxiv.org/abs/2512.20264https://arxiv.org/abs/2607.05155