<- Back
Comments (13)
- yobboLooks very nice, but I can't find numerical gradient checks, which is helpful when verifying that backward pass is correct:https://github.com/markusheimerl/gpt/blob/main/transformer/a...
- oakinnagbeNice implementation. Have you thought about supporting LoRA fine-tuning on top of this, or is the design too low-level for that kind of extension?
- Gred_papa_danceI need more info:* where is data (make data) how create new my own data, (questions for chat?) * how create a tokenizer (meybe separate) * how stop the code, how many memory need, how setup size of context etc. * how creating a LORA or learn with new data. * how quantize model?In my opinion this is great idea but making a Ruby extension will be goot way to increase users using this code.
- qqqqqlqq$make run -j 10CUDA error in attention.c:91: out of memoryCommand exited with non-zero status 11.38user 0.46system 0:00.75elapsed 246%CPU (0avgtext+0avgdata 226164maxresident)k0inputs+0outputs (0major+25414minor)pagefaults 0swapsmake: ** [Makefile:34: run] Błąd 1clang: warning: CUDA version 12.4 is only partially supported [-Wunknown-cuda-version](I have ubuntu and 8GB memory NVIDIA GeForce RTX 3050 876MiB / 8192MiB )
- anonundefined
- qqqqqlqqIt works on arm ?