Harvesting Spare CPU Resources in Container Systems

Container platforms like Kubernetes are widely adopted for deploying latency-sensitive cloud services, and CPU resources for these containers are over-provisioned to ensure low 99th percentile tail latency under peak load. At the same time, cloud services exhibit bursty traffic patterns resulting in CPU usage variability that creates opportunity to harvest ephemerally unused CPU cores to run throughput-oriented containers. However, existing resource controls do not allow containers to share unused cores without compromising their low tail latency objectives. Prior research on performance isolation is inadequate for container systems because it requires modifying applications and system software, employs offline profiling, and does not account for interference from processing container networking interrupts. We present HarvestContainers, a system that protects latency-sensitive containers from all sources of interference while harvesting their spare CPU cores to run throughput-oriented containers. Our solution dynamically determines the safe number of CPU cores to harvest and does not require rewriting or recompiling applications or OS. We implement HarvestContainers integrated with Kubernetes and evaluate it experimentally. Our evaluation shows that latency-sensitive containers with microsecond-scale service level objectives can share up to 75% of their unused CPU cores while maintaining tail latency within 4% of standalone operation.