Scaling Groups

---

A scaling group is a replicated, HTTP-fronted service managed by Chalk. Each group is backed by a Kubernetes Deployment with rolling updates, automatic service discovery, and autoscaling between configurable min and max replica counts, driven by CPU or GPU utilization or by schedules you set. A group can also scale to zero replicas and start again when a request arrives. Use scaling groups for ML inference servers, internal APIs, agent backends, and any long-lived service that needs to be reachable from outside the cluster.

For workloads that don’t need replication or HTTP fronting, use a Sandbox instead. For serverless, function-shaped invocations, use Functions.

For registered models served through Chalk’s Arrow inference contract, use the Model Deployments APIs. They expose typed deployment objects, readiness waiting, revision history, updates, and rollback.

You can drive scaling groups two ways: the Python SDK (chalkcompute.ScalingGroup) or the chalk scaling-group CLI. Both surfaces target the same underlying service, so groups created one way are visible and manageable from the other. For every CLI flag, see the chalk scaling-group create reference.

---

​

Quick start (SDK)

Save a small Flask app as app.py:

from flask import Flask

app = Flask(__name__)


@app.get("/health")
def health():
    return {"ok": True}


app.run(host="0.0.0.0", port=8080)

Then deploy it from a script in the same directory:

import chalkcompute

img = (
    chalkcompute.Image.debian_slim("3.12")
    .pip_install(["flask"])
    .add_local_file("./app.py", "/app/app.py")
    .entrypoint(["python", "/app/app.py"])
)

sg = chalkcompute.ScalingGroup(
    image=img,
    name="hello-api",
    port=8080,
    min_replicas=1,
    max_replicas=3,
    startup_probe=chalkcompute.StartupProbe.http("/health"),
).deploy()

resp = sg.call("/health", method="GET")
print(resp.status, resp.text())

sg.delete()

ScalingGroup(...).deploy() builds the image, uploads any local files declared via add_local_file / add_local_dir, creates the scaling group, and waits until at least one replica is ready and available. With startup_probe set, deploy() also waits until the group’s web_url responds at /health.

---

​

Quick start (CLI)

$ chalk scaling-group create \
    --image=my-registry/my-inference-server:v1 \
    --name=inference \
    --max-replicas=1 \
    --port=8080

This creates a Kubernetes Deployment with a single replica running your image, a headless Service, and an HTTPRoute that exposes the service over HTTPS.

You can tune replicas and resource limits at creation time:

$ chalk scaling-group create \
    --image=my-registry/my-inference-server:v1 \
    --name=inference \
    --port=8080 \
    --min-replicas=3 \
    --max-replicas=3 \
    --cpu=4 \
    --memory=8Gi

---

​

Full example

A GPU-accelerated inference service that loads model weights from a Chalk Volume, with custom resource limits, multiple replicas, autoscaling, and tags:

$ chalk scaling-group create \
    --image=my-registry/llm-serving:v3 \
    --name=llm-inference \
    --port=8080 \
    --min-replicas=2 \
    --max-replicas=4 \
    --gpu=nvidia-l4:1 \
    --cpu=8 \
    --memory=32Gi \
    --volume=chalk://llama3-weights=/models/llama3 \
    --tags="model=llama3,team=ml-platform" \
    --entrypoint="python,-m,vllm.entrypoints.openai.api_server,--model,/models/llama3"

my-registry/llm-serving:v3 stands for your own serving image, which must have vLLM installed. chalk://llama3-weights stands for a Chalk Volume holding the model weights; create it with chalk volume create and upload the weights with chalk volume cp --recursive.

After creation, the service is reachable at its assigned DNS name and can be referenced by resolvers or other services running in the same Chalk environment.

---

​

When to use a Scaling Group vs other compute primitives

See Choosing the right primitive on the Compute overview for the full decision table.