<- Back
Comments (14)
- armcatThere has been extensive research into token-free LLMs, but for some reason we are still operating in a token domain, so there is something to that.Byte Latent Transformers (BLT): https://arxiv.org/abs/2412.09871Charformer: https://arxiv.org/abs/2106.12672
- mrkn1subword tokenizers were never causal in the first place, so BPE was peeking at future bytes all along! TIL
- anonundefined
- JonChesterfieldWeird paper. Models have had tokens for each byte for ages now. They can read and write individual bytes just fine, in addition to also having multibyte tokens.
- serioussecurityWow nature got rolled. Should have stayed closer to their expertise. They were already being hustled by a lot of the applied AI work they were accepting.
- BitProgram[flagged]
- anonundefined
- singularityisne[flagged]