<- Back
Comments (29)
- rurbanThe leader boards are from the pre Fabrice Bellard days, btw. Neural network modeling helped finding better patterns in text.Also, you could say the same for the related data search problem. How to prepare data, so that it can most efficiently searched. Smallest encoding vs fastest search. Databases are mostly very, very stupid compared to more data-specific tuned algorithms. Like factor 1000 slower and bigger.
- dangRelated:Data Compression Explained (2011) - https://news.ycombinator.com/item?id=40631931 - June 2024 (1 comment)Data Compression Explained - https://news.ycombinator.com/item?id=5931493 - June 2013 (14 comments)Data Compression Explained by Matt Mahoney - https://news.ycombinator.com/item?id=1179242 - March 2010 (1 comment)
- BnjorogeI wonder what if anything has changed ever since this article. Is llm-based compression more mainstream?
- usernametaken29Isn’t the idea of AI precisely to find universal compression from arbitrary input data, at least with LLMs?
- firesteelrainMatt Mahoney is one of the best when it comes to compression. He is retired now.
- brownpointsI say transformers are the best compression systems
- blastroi was told that middle-out was best
- wpsThis is the guy who created Zpaq btw. Super interesting but niche backup/archive software.
- blobbersMatt is a great guy to explain this kind of stuff. He's very helpful.
- NooneAtAll3does anyone have any sources to read about ai-based compression?I remember hearing a lot about "compression is a lot about prediction", but I don't remember reading any practical result