Skip to content

vllm.v1.kv_offload.tiering

Modules:

  • async_lookup –

    AsyncLookupManager: per-tier async lookup manager for secondary tier

  • base –

    Abstract interfaces and data types for the secondary tiering layer.

  • example –
  • fs –
  • manager –

    TieringOffloadingManager: Multi-tier KV cache offloading orchestrator.

  • obj –
  • spec –

    TieringOffloadingSpec: Spec for multi-tier KV cache offloading.