I think there are 3 advantages in SGLang. It allows direct programing in python, it suuport RadixAttention to effeicient KVCache reuse, and it used compressed finite state machine to accelerate the structured output.
1. Runtime Programing
2. RadixAttention
Reuse the KVCache with the same prompts. The eviction policy is LRU. So Its main application scenarios are in long-context conversations and situations where prompts are shared accross requests.
3. Compressed finite state machine
The runtime analysis the adjacent singular transition edge into single edges as above graph to accelerate the decoding process.