# Scaling Groups
source: https://docs.chalk.ai/docs/compute/scaling-groups

## Deploy autoscaling HTTP services with routing.

A scaling group is a replicated, HTTP-fronted service managed by Chalk. Each
group is backed by a Kubernetes Deployment with rolling updates, automatic
service discovery, and autoscaling between configurable min and max replica
counts, driven by CPU or GPU utilization or by schedules you set. A group can
also scale to zero replicas and start again when a request arrives. Use scaling groups for ML inference servers,
internal APIs, agent backends, and any long-lived service that needs to be
reachable from outside the cluster.

For workloads that don't need replication or HTTP fronting, use a
Sandbox instead. For serverless, function-shaped
invocations, use Functions.

For registered models served through Chalk's Arrow inference contract, use the
Model Deployments APIs. They expose typed deployment objects,
readiness waiting, revision history, updates, and rollback.

You can drive scaling groups two ways: the Python SDK (chalkcompute.ScalingGroup)
or the chalk scaling-group CLI. Both surfaces target the same underlying
service, so groups created one way are visible and manageable from the other.
For every CLI flag, see the
chalk scaling-group create reference.

### Quick start (SDK)

Save a small Flask app as app.py:

```
from flask import Flask

app = Flask(__name__)


@app.get("/health")
def health():
    return {"ok": True}


app.run(host="0.0.0.0", port=8080)
```

Then deploy it from a script in the same directory:

```
import chalkcompute

img = (
    chalkcompute.Image.debian_slim("3.12")
    .pip_install(["flask"])
    .add_local_file("./app.py", "/app/app.py")
    .entrypoint(["python", "/app/app.py"])
)

sg = chalkcompute.ScalingGroup(
    image=img,
    name="hello-api",
    port=8080,
    min_replicas=1,
    max_replicas=3,
    startup_probe=chalkcompute.StartupProbe.http("/health"),
).deploy()

resp = sg.call("/health", method="GET")
print(resp.status, resp.text())

sg.delete()
```

ScalingGroup(...).deploy() builds the image, uploads any local files declared
via add_local_file / add_local_dir, creates the scaling group, and waits
until at least one replica is ready and available. With startup_probe set,
deploy() also waits until the group's web_url responds at /health.

### Quick start (CLI)

```
$ chalk scaling-group create \
    --image=my-registry/my-inference-server:v1 \
    --name=inference \
    --max-replicas=1 \
    --port=8080
```

This creates a Kubernetes Deployment with a single replica running your image,
a headless Service, and an HTTPRoute that exposes the service over HTTPS.

You can tune replicas and resource limits at creation time:

```
$ chalk scaling-group create \
    --image=my-registry/my-inference-server:v1 \
    --name=inference \
    --port=8080 \
    --min-replicas=3 \
    --max-replicas=3 \
    --cpu=4 \
    --memory=8Gi
```

### Full example

A GPU-accelerated inference service that loads model weights from a Chalk
Volume, with custom resource limits, multiple replicas, autoscaling, and tags:

```
$ chalk scaling-group create \
    --image=my-registry/llm-serving:v3 \
    --name=llm-inference \
    --port=8080 \
    --min-replicas=2 \
    --max-replicas=4 \
    --gpu=nvidia-l4:1 \
    --cpu=8 \
    --memory=32Gi \
    --volume=chalk://llama3-weights=/models/llama3 \
    --tags="model=llama3,team=ml-platform" \
    --entrypoint="python,-m,vllm.entrypoints.openai.api_server,--model,/models/llama3"
```

my-registry/llm-serving:v3 stands for your own serving image, which must have
vLLM installed. chalk://llama3-weights stands for a Chalk Volume holding the
model weights; create it with chalk volume create and upload the weights with
chalk volume cp --recursive.

After creation, the service is reachable at its assigned DNS name and can be
referenced by resolvers or other services running in the same Chalk
environment.

### When to use a Scaling Group vs other compute primitives

See Choosing the right primitive
on the Compute overview for the full decision table.