CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context
Scenarios
CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context
Scenarios
Large Language Models (LLMs) have been widely adopted to process long-context tasks. However, the large memory overhead of the key-value (KV) cache poses significant challenges in long-context scenarios. Existing training-free KV cache compression methods typically focus on quantization and token pruning, which have compression limits, and excessive sparsity can lead …