GitHub · C#
tryAGI/Tiktoken
High-performance .NET BPE tokenizer — up to 618 MiB/s, competitive with Rust. Zero-allocation counting, multilingual cache, o200k/cl100k/r50k/p50k encodings + HuggingFace tokenizer.json support.
- Stars
- 93
- Forks
- 7
- Stars gained in 24 hours
- 0
- Forks gained in 24 hours
- 0
Topics: csharp, openai, tiktoken, ai, bpe, dotnet, gpt4o, high-performance, huggingface, sdk
Latest data point: Nov 22, 2026 · 93 stars · 7 forks