sgl-project/sglang

SGLang is a fast serving framework for large language models and vision language models.

Commits: 202

InternLM/lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Commits: 40

flashinfer-ai/flashinfer

FlashInfer: Kernel Library for LLM Serving

Commits: 16

vllm-project/vllm

Commits: 3

baidu/babylon

Commits: 1

trojan-gfw/trojan

An unidentifiable mechanism that helps you bypass GFW.

Commits: 1

chenshuo/muduo

Event-driven network library for multi-threaded Linux server in C++11

Commits: 1

FasterDecoding/Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

Commits: 1