Resources and GPUs

---

CPU, memory, and GPU settings apply to sandboxes, scaling groups, and functions. All three take the same cpu, memory, and gpu parameters.

---

​

CPU and memory

The cpu and memory arguments (SDK) and the --cpu and --memory flags (scaling group CLI) set the resource requests and limits. Both the Kubernetes request and limit are set to the same value, guaranteeing your workload the resources it asks for, so it is not subject to noisy-neighbor throttling.

CPU values follow Kubernetes conventions: 1 means one full core, 500m means half a core. Memory values use standard suffixes: Mi (mebibytes) and Gi (gibibytes).

CPU must be 250m, 500m, or a power-of-two number of cores (1, 2, 4, 8, 16, 32, …). Memory must be at least 2Gi per core: 512Mi for 250m, 1Gi for 500m, 2Gi for 1, 4Gi for 2, and so on. If you set CPU and omit memory, the workload gets that minimum.

A sandbox:

from chalkcompute import Image, Sandbox

sandbox = Sandbox(
    image=Image.debian_slim(),
    cpu="4",
    memory="16Gi",
).run()

A function:

import chalkcompute

@chalkcompute.function(cpu="1", memory="2Gi")
def normalize_name(name: str) -> str:
    return " ".join(part.capitalize() for part in name.split())

normalize_name.wait_ready()
print(normalize_name("ada lovelace"))  # Ada Lovelace

A scaling group, from the CLI:

$ chalk scaling-group create \
    --image=my-registry/my-app:v1 \
    --name=my-service \
    --max-replicas=1 \
    --port=8080 \
    --cpu=2 \
    --memory=4Gi

If you omit CPU and memory, a sandbox gets 250m CPU and 512Mi memory. Each scaling group replica gets the same defaults, set as both the request and the limit.

A sandbox’s request must fit on a single host in a configured host pool. A sandbox that asks for more CPU or memory than any host provides fails to schedule with Unschedulable: no configured host pool can satisfy .... To run it, configure a host pool whose hosts are large enough for the request.

---

​

GPUs

The gpu argument (SDK) and the --gpu flag (scaling group CLI) take a type:count value, or just type or count:

Value	GPU type	GPUs
nvidia-l4:2	nvidia-l4	2
nvidia-l4	nvidia-l4	1
2	Any available	2

To build an image with GPU libraries, see GPU images.

​

Scaling groups and functions

A function runs on a scaling group, so the same rules apply to both.

import chalkcompute

img = chalkcompute.Image.debian_slim("3.12").entrypoint(
    ["python", "-m", "http.server", "8080"]
)

sg = chalkcompute.ScalingGroup(
    image=img,
    name="gpu-inference",
    port=8080,
    cpu="8",
    memory="32Gi",
    gpu="nvidia-l4:1",
    min_replicas=1,
    max_replicas=4,
).deploy()

$ chalk scaling-group create \
    --image=my-registry/my-gpu-model:v1 \
    --name=gpu-inference \
    --max-replicas=1 \
    --port=8080 \
    --gpu=nvidia-tesla-t4:1 \
    --cpu=4 \
    --memory=16Gi

The type portion selects the GPU node the replicas are scheduled on, and the count sets the number of GPUs each replica gets (the nvidia.com/gpu request and limit). Chalk adds the nvidia.com/gpu toleration automatically, so replicas can schedule onto GPU-tainted nodes.

Write type as a GKE accelerator name on both GCP and AWS:

• GCP: the value is used as the cloud.google.com/gke-accelerator node selector, so any accelerator in your cluster’s node pools works.
• AWS: Chalk maps the value to Karpenter’s karpenter.k8s.aws/instance-gpu-name label. The types in the table below are supported.

GPU type	GPU
nvidia-tesla-t4	NVIDIA T4
nvidia-a10g	NVIDIA A10G
nvidia-l4	NVIDIA L4
nvidia-l40s	NVIDIA L40S
nvidia-tesla-v100	NVIDIA V100
nvidia-tesla-a100	NVIDIA A100
nvidia-a100-80gb	NVIDIA A100 80GB
nvidia-h100-80gb	NVIDIA H100 80GB
nvidia-h100-mega-80gb	NVIDIA H100 Mega
nvidia-h200-141gb	NVIDIA H200
nvidia-rtx-pro-6000	NVIDIA RTX PRO 6000
nvidia-b200	NVIDIA B200

Which types you can schedule depends on the GPU node pools configured in your cluster.

To run on any available GPU node without selecting a type, pass only the count:

$ chalk scaling-group create \
    --image=my-registry/my-gpu-model:v1 \
    --name=gpu-inference \
    --max-replicas=1 \
    --port=8080 \
    --gpu=1

To request multiple GPUs, increase the count:

$ chalk scaling-group create \
    --image=my-registry/my-training-server:v1 \
    --name=multi-gpu \
    --max-replicas=1 \
    --port=8080 \
    --gpu=nvidia-tesla-a100:4

To scale on GPU utilization, see Autoscaling.

​

Sandboxes

A sandbox requests a GPU type at creation time. This example runs on the k8s compute class:

from chalkcompute import ComputeClass, Image, Sandbox

sandbox = Sandbox(
    image="nvcr.io/nvidia/pytorch:24.01-py3",
    gpu="nvidia-l4",
    cpu="8",
    memory="64Gi",
    compute_class=ComputeClass.K8S,
).run()

The Chalk scheduler matches the request to a node with the appropriate GPU hardware and configures the NVIDIA device plugin and driver mounts automatically.

A k8s sandbox does not support exec, so pass its workload as entrypoint when you create it. To run commands in a GPU sandbox with exec, use the host compute class.

On AWS and GKE, you can also run a GPU sandbox on the default host compute class when a host pool with the same GPU type is configured. Omit compute_class or set it to ComputeClass.HOST. Each GPU host pool host provides one GPU. GKE GPU host pools support the accelerator types listed in the host pool guide. On other clouds, use ComputeClass.K8S for GPU sandboxes.