GigaToken: ~1000x faster Language model tokenization

Language model tokenization at GB/s. Contribute to marcelroed/gigatoken development by creating an account on GitHub.