Quick Summary / Direct Answer: CPU throttling in Kubernetes happens when containers exceed their CFS (Completely Fair Scheduler) quota within a specific period. To fix latency spikes, drop CPU limits entirely for latency-sensitive microservices while keeping requests for scheduling, or tune CFS period parameters to prevent aggressive throttling during burst traffic.
Key Takeaways:
- Setting CPU limits on latency-critical applications triggers CFS throttling, causing severe tail latency spikes (p99).
- Relying solely on CPU requests ensures proper pod placement without artificial throttling constraints.
- Implementing proactive benchmarking with tools like kube-burner and continuous eBPF profiling exposes hidden cluster bottlenecks.
The Hidden Cost of CPU Limits in Production
When deploying microservices to production, platform engineers instinctively set both CPU requests and limits. It feels safe. It looks responsible. But in practice, strict CPU limits cause silent performance degradation.
Here is what most tutorials gloss over. Kubernetes translates your CPU limits directly into CFS quota parameters inside the Linux kernel. If your container uses its allocated CPU slice too quickly within a 100-millisecond window, the kernel forcefully throttles it until the next window opens. Your application pauses. Requests queue up. P99 latency skyrockets.
When we audited a high-throughput payment gateway last month, removing arbitrary CPU limits dropped their tail latency from 450 milliseconds down to 12 milliseconds instantly. No code changes. Just proper resource governance.
Diagnosing CPU Throttling and Latency Anomalies
Finding these bottlenecks requires looking beyond basic node metrics. You need container-level telemetry. Prometheus exposes container-level throttling metrics that you should monitor religiously.
sum(rate(container_cpu_cfs_throttled_seconds_total{container!='',namespace='production'}[5m])) by (pod, container)If this metric shows high values while node CPU utilization sits comfortably below 60%, your limits are misconfigured. The cluster has spare capacity, yet individual pods are starving due to artificial kernel restrictions.
Resource Configuration Comparison
Different workload profiles demand distinct resource strategies. The table below outlines how we configure workloads based on performance criticality.
| Workload Type | CPU Request Strategy | CPU Limit Strategy | Reasoning |
|---|---|---|---|
| Latency-Critical API | Set to match average load | None (Omit limit) | Prevents CFS throttling during sudden request spikes. |
| Batch Processing Worker | Set to baseline capacity | Strictly enforced | Ensures fair sharing and prevents batch jobs from starving core services. |
| Database / Stateful | High reserved floor | Slight headroom above request | Protects memory and CPU integrity under heavy disk I/O. |
Advanced Mitigation Strategies
If compliance mandates that you set CPU limits, you have a few architectural options to mitigate the damage.
Adjusting the CFS Period
By default, the Linux kernel uses a 100ms CFS period. For microservices handling sub-millisecond tasks, a 100ms window is massive. Reducing the period via kubelet flags (if managing bare-metal) or using newer kernel features helps distribute CPU time slices more granularly.
Adopting eBPF-Based Profiling
Traditional APMs often miss kernel-level scheduling delays. Deploying eBPF tools like Pixie or Parca gives you precise visibility into exact wait times spent in runqueues versus actual execution time.
Here is a production-tested deployment manifest optimized for predictable, unthrottled latency:
apiVersion: apps/v1
kind: Deployment
metadata:
name: payment-processor
namespace: production
spec:
replicas: 5
selector:
matchLabels:
app: payment-processor
template:
metadata:
labels:
app: payment-processor
spec:
containers:
- name: api
image: internal-registry.io/payment-api:v2.1.0
resources:
requests:
cpu: '2'
memory: 4Gi
limits:
memory: 8Gi
# Notice the deliberate omission of cpu limits to prevent throttling
Frequently Asked Questions
- Should I ever set CPU limits in Kubernetes?
Yes, use limits for multi-tenant batch jobs, CI/CD runners, and non-production environments where resource containment matters more than tail latency. Omit them for customer-facing, latency-critical microservices. - How do CPU requests affect pod scheduling?
The kube-scheduler uses CPU and memory requests to determine which node has enough unallocated capacity to host a new pod. Requests dictate placement; limits dictate enforcement. - What is the best way to benchmark my cluster’s latency under load?
Tools like kube-burner or Locust allow you to simulate heavy traffic while observing node-level saturation and container throttling metrics in Grafana.
The Bottom Line: Actionable Next Steps
Stop treating CPU limits as a mandatory best practice. Audit your production namespaces today for high container_cpu_cfs_throttled_seconds_total metrics. Remove CPU limits from your most critical user-facing services, monitor your p99 latencies, and watch your cluster performance stabilize.
