Text Generation Inference (TGI)
⭐ 10.8k stars| Repository | huggingface/text-generation-inference |
| Category | infra |
| Difficulty | advanced |
| Status | active |
| Tags | serving inference huggingface rust |
| Website | https://huggingface.co/docs/text-generation-inference |
Review
Hugging Face's production-ready inference server for LLMs. Optimized with continuous batching, tensor parallelism, and quantization support. Best for Hugging Face ecosystem users deploying models in production.
Use Cases
- model-serving
- inference
- deployment