Kubernetes Resource Management

Kubernetes lets you set resource requests and limits for CPU, memory, ephemeral storage, and huge pages at both the container and Pod level. Requests guide the scheduler in placing Pods on suitable nodes, while limits are enforced by the kubelet and the kernel to prevent any single container from consuming more than its allotted share. CPU limits act as a hard ceiling enforced through throttling, whereas memory limits trigger OOM kills when exceeded. You can express CPU in whole or fractional cores (or millicores) and memory in bytes with standard suffixes. Since Kubernetes v1.34, Pod-level resource budgets are available (Beta), letting containers within a Pod share idle resources. After deployment, you can resize resources in place (Stable since v1.35) or by rolling replacement Pods. Properly configuring these values, combined with ResourceQuotas and LimitRanges at the namespace level, keeps clusters stable and prevents noisy-neighbor problems.

Resource Requests and Limits

When you define a Pod, you can optionally specify how much of each resource a container needs. The two most common are CPU and memory.

• Requests tell the kube-scheduler how much a container needs. The scheduler uses this to decide which node can host the Pod.
• Limits set a hard ceiling. The kubelet (and the kernel via cgroups) enforces them so a container cannot exceed the specified amount.

A container may use more than its request if the node has headroom, but it will never exceed its limit.

Resource Types

Clusters can also expose extended resources (custom names, typically via device plugins).

Resource Units

CPU

• 1 CPU = 1 physical core (or 1 vCPU on a VM).
• Fractional values are allowed: 0.5 CPU = half a core.
• 100m (millicpu) is equivalent to 0.1 CPU.
• CPU is always an absolute amount, not a percentage of the node.
• Minimum precision is 1m (0.001 CPU). Prefer the milli form (e.g., 5m) over decimals (e.g., 0.005) to avoid invalid values like 0.0005.

Memory

• Measured in bytes. Use suffixes E, P, T, G, M, k (decimal) or Ei, Pi, Ti, Gi, Mi, Ki (binary).
• M = megabytes; m = millibytes (1/1000 of a byte). Be careful with case.
• Examples of roughly equivalent values: 128974848, 129M, 123Mi.

Container-Level Resources

For each container, set requests and limits under spec.containers[].resources:

apiVersion: v1
kind: Pod
metadata:
  name: frontend
spec:
  containers:
  - name: app
    image: images.my-company.example/app:v4
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"
  - name: log-aggregator
    image: images.my-company.example/log-aggregator:v6
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

The Pod’s effective request is the sum of all container requests (0.5 CPU, 128 MiB), and its effective limit is the sum of all container limits (1 CPU, 256 MiB).

Pod-Level Resources

Feature state: Beta since Kubernetes v1.34; enabled by default.

With the PodLevelResources feature gate, you can declare an overall resource budget for the entire Pod. This is especially useful when a Pod has many containers and it is hard to gauge each one’s individual needs. Containers without explicit requests/limits simply share the Pod-level budget.

apiVersion: v1
kind: Pod
metadata:
  name: pod-resources-demo
  namespace: pod-resources-example
spec:
  resources:
    limits:
      cpu: "1"
      memory: "200Mi"
    requests:
      cpu: "1"
      memory: "100Mi"
  containers:
  - name: pod-resources-demo-ctr-1
    image: nginx
    resources:
      limits:
        cpu: "0.5"
        memory: "100Mi"
      requests:
        cpu: "0.5"
        memory: "50Mi"
  - name: pod-resources-demo-ctr-2
    image: fedora
    command:
    - sleep
    - inf

Here, pod-resources-demo-ctr-1 has its own explicit values, while pod-resources-demo-ctr-2 draws from the remaining Pod-level budget.

Scheduling and Enforcement

• The scheduler ensures the sum of all container requests on a node does not exceed the node’s allocatable capacity.
• The kubelet passes requests and limits to the container runtime, which configures kernel cgroups.
• CPU request acts as a scheduling weight; CPU limit is a hard throttle.
• Memory request is primarily a scheduling hint; memory limit triggers OOM kills.
• If a container exceeds its memory request and the node runs low on memory overall, the Pod may be evicted.
• Containers are never terminated for exceeding CPU limits.

Resizing Resources

In-place resize

Feature state: Stable since Kubernetes v1.35.

Update a running Pod’s CPU or memory via the /resize subresource without recreating the Pod. Use the resizePolicy field to control whether a container restart is required.

Replacement Pods

Update the Pod template in a Deployment or StatefulSet and let the controller roll new Pods with the updated resources. This works on any Kubernetes version.

The Vertical Pod Autoscaler can also generate resource recommendations automatically.

Memory-Backed emptyDir Volumes

A few cautions when using medium: Memory emptyDir volumes:

• If you do not set a sizeLimit, the volume can consume up to the Pod’s memory limit. Without a memory limit, it can consume all node memory.
• Files on a memory-backed volume are managed by your application; neither Kubernetes nor the OS will reclaim them under memory pressure.
• Memory is small and expensive compared to disk. Use these volumes judiciously.
• Set a ResourceQuota or LimitRange at the namespace level to cap total memory usage.

Extended Resources

Extended resources use fully-qualified names outside the kubernetes.io domain (e.g., example.com/foo). Cluster operators advertise them on nodes; users request them in Pod specs just like CPU or memory.

Key points:

• Quantities must be whole numbers (e.g., 3, 3000m). Fractional values like 0.5 are invalid.
• Extended resources cannot be overcommitted: request and limit must be equal if both are set.
• A Pod stays in PENDING until all resource requests (including extended resources) can be satisfied.

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: myimage
    resources:
      requests:
        cpu: 2
        example.com/foo: 1
      limits:
        example.com/foo: 1

Troubleshooting

Pod stuck in Pending (FailedScheduling)

Check events and node capacity:

kubectl describe pod frontend | grep -A 50 Events
kubectl describe nodes <node-name>

Common fixes:

• Add more nodes to the cluster.
• Terminate unneeded Pods to free capacity.
• Verify the Pod’s request does not exceed every node’s allocatable amount.
• Check for node taints that the Pod does not tolerate.

Container repeatedly terminated (OOMKilled)

kubectl describe pod <pod-name>

Look for Reason: OOMKilled and a high Restart Count. This means the container hit its memory limit. Investigate the application for a memory leak, or raise the memory limit (and possibly the request) if the workload legitimately needs more.

Namespace-Level Controls

To prevent any single team or workload from monopolizing cluster resources:

• ResourceQuota caps the total resources a namespace can consume.
• LimitRange sets default and per-container minimum/maximum values within a namespace.

Be mindful of access control: full write access to a namespace allows removal of the ResourceQuota itself.

References

• Kubernetes: Manage Compute Resources for Containers

• Technology
• How-To

• Kubernetes
• Troubleshooting