Ai2 releases Llama-B 8B Stage 1 checkpoint, a byte-level Llama 3 8B variant

Ai2 has released allenai/Llama-B-8B-Stage1, a Llama 3 8B model retrofitted to operate over bytes instead of tokens through a short additional training procedure.

This Stage 1 checkpoint contains only Stage 1 training, with inner model parameters unchanged, and is licensed under the Llama 3 Community License for research and educational use.

It requires transformers 4.57.3 or later and the xlstm package, and is loaded with trust_remote_code.