GitHub · Python
NVIDIA/TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
- Stars
- 15,105
- Forks
- 2,944
- Stars gained in 24 hours
- +21
- Forks gained in 24 hours
- +6
Topics: blackwell, cuda, llm-serving, moe, pytorch
Latest data point: Nov 20, 2026 · 15,105 stars · 2,944 forks