Large Language Models

KVSwap: Disk-aware KV Cache Offloading for Long-Context On-device Inference

The new compiler stack: a survey on the synergy of LLMs and compilers