Resource Configuration

​

Overview

Under Infrastructure > Resources, you will find the Cloud Resource Configuration page, where you can configure the resources allocated to each of your Chalk services and most other system-related settings.

​

Resource Groups and Services

Your Chalk environment consists of multiple Services, each of which serves a different purpose and represents a different kind of feature, compute, or background workload. For instance, the Query Server (also referred to as the engine) is responsible both for serving online inference queries and also for serving dataplane data requests, the Job Queue Consumer is responsible for pulling jobs like offline queries (backfills, dataset ingestions, etc.) from a queue and running them, and the Streaming Server handles execution and coordination of Stream Resolvers.

Chalk also provides a higher-level isolation concept called a Resource Group. Each resource group provides its own independently-configurable set of Chalk Services, and is backed by logically-independent Kubernetes deployments. This provides isolation between different kinds of workloads so that they don’t interfere with each other. For instance, you may want to run low-latency high-volume IO-bound inference queries against the Query Server in one resource group, and slower CPU-bound analytics queries against the Query Server in a separate resource group. You may also want to configure several different resource groups of different sizes so that you can target long-running offline workloads against machines of different sizes.

All Chalk environments will have a Default resource group. New resource groups can be added and configured independently from each other in the Cloud Resource Configuration pane.

For online inference queries, each resource group exposes its own URL for the query client to use, and for offline workloads like Offline Query, a resource group can be specified in the resource_request parameter of the ChalkClient.offline_query() method and other related methods.

​

Configuring Service Resources

Each Chalk Service can be independently configured to consume different amounts of compute resources, depending on your needs. The primary way to configure resource consumption for a Service is to select a Chalk Machine Type from the dropdown selector on the Cloud Resource Configuration page when viewing the individual Service configuration pane. This guarantees that each replica of the Service will have full access to the listed CPU and memory amounts, and will run isolated on its own cloud instance, ensuring that separate services cannot interfere with each other. Different machine types are available for online, offline, and other kinds of workloads, since these workloads tend to have different performance requirements.

In addition to configuring the resources allocated to each replica of a Service, you can configure the Service’s scaling properties. Chalk supports CPU-utilization-based autoscaling as the default, as well as time-based pre-scaling and offset/backlog-based scaling for stream resolver execution. You can configure the min and max instance counts as well as the target cpu % for each Service independently for each Resource Group, so different workloads can scale independently.

More information about Chalk Machine Types can be found on the detailed docs page.

​

Autoscaling

Chalk supports autoscaling of resources in your environment in order to optimize resource usage.

For the branch server, you can set the Auto Shutdown Period, which will determine the duration of inactivity (no queries or deploys) after which the branch server pod will be automatically shut down.

For the Query Server and gRPC Query Server which are used to serve your production traffic, you can configure autoscaling based on a CPU utilization target or you can set time-based pre-scaling.

For stream resolvers, you can also set an autoscaling policy based on the stream queue message backlog. The underlying system used to define scaling policies is KEDA.

​

CPU-Based Autoscaling

To enable CPU-based autoscaling, you can set the Target CPU % in the Cloud Resource Configuration. Chalk will then monitor the CPU utilization of the service and scale the number of instances up or down based on the target CPU percentage. In order for CPU-based autoscaling to work, you must have the Min Instances, Max Instances, and Target CPU% all set for the service.

​

Scheduled Autoscaling

To enable scheduled autoscaling, you can define scaling triggers for each service configuration in the Cloud Resource Configuration page. For each service in a resource group that you would like to autoscale, define the Min Instances and Max Instances under Scaling. Then, under the ADVANCED section, under Time-based Scaling, you can define one or more Scaling Triggers. For each Scaling Trigger, you can define the desired number of replicas, the start time, the end time, and the timezone for the schedule. The start time and end time are defined using standard cron syntax. The underlying implementation of time-based scaling is the KEDA Cron scaler.

​

Handling Predictable Traffic Bursts

Scheduled capacity and CPU-based autoscaling work best together: the schedule handles known demand, while CPU autoscaling provides additional capacity for unplanned or longer-lived demand. They do not replace capacity planning. If traffic can arrive as a large synchronized fan-out, consider spreading it over a short interval with batching or jitter where your application permits. This reduces the amount of capacity that must be available at the exact start of the burst.

CPU-based autoscaling is well-suited to sustained or unexpected changes in demand, but it is reactive. When traffic rises abruptly, a new pod must still be created, scheduled, and pass readiness checks before it can serve requests. If the cluster needs to provision a node as well, this can take longer. Requests that arrive at the start of a burst may therefore see increased latency before autoscaling adds capacity.

Allow several minutes of lead time for a scheduled increase when additional nodes might be needed. The actual time depends on signal evaluation, pod scheduling, node provisioning, image startup, and readiness checks. Exact times vary by environment, so use the p95 time from a scaling signal to a ready replica to choose the schedule’s lead time.

When your production traffic has a recurring peak (for example, a daily batch, campaign, or scheduled workflow), use scheduled autoscaling to make capacity available before the peak begins:

1. Measure the burst. Review request rate at a short interval, end-to-end latency, client-visible errors or timeouts, CPU utilization, and ready replica count. A five-minute average can hide a burst that lasts only seconds or minutes.
2. Choose a scheduled baseline. Set the desired replica count to the capacity required at the start of the peak, rather than the normal idle baseline. Establish this count with a representative load test or prior peak data.
3. Start early enough for readiness. Schedule the increase with enough lead time for pod scheduling, image startup, and readiness checks. Include extra time if a new node may need to be provisioned.
4. Keep overflow headroom. Set Max Instances above the scheduled baseline so CPU-based autoscaling can still respond to demand beyond the expected peak. The maximum should be tested against the resource requests, nodepool capacity, and any downstream dependencies.
5. Scale down after the peak. End the schedule once traffic has returned to its normal level to avoid paying for unused capacity.

​

Programmatic management

The same resource-group settings exposed in Infrastructure > Resource Configuration can be read and edited from the CLI. This is useful for codifying environment configuration in source control, scripting per-environment changes (image bumps, replica counts, autoscaler overrides), and applying targeted edits without leaving the terminal.

The full configuration lives on the environment record under spec_config_json. A typical shape looks like:

{
  "id": "tm-abc123",
  "name": "production",
  "spec_config_json": {
    "services": {
      "engine-grpc": {
        "min_instances": 2,
        "max_instances": 10,
        "target_cpu_utilization_percentage": 50,
        "chalk_machine_type": "small"
      },
      "branch": { "chalk_machine_type": "large" }
    },
    "resource_groups": [
      {
        "resource_group_name": "high-throughput",
        "services": {
          "engine-grpc": {
            "min_instances": 5,
            "chalk_machine_type": "large-highmem",
            "engine_image_uri_override": "gcr.io/chalk/engine:sha-abc123"
          }
        }
      }
    ]
  }
}

The fields directly under spec_config_json (alongside resource_groups) make up the default resource group. Each entry in resource_groups is a named override group keyed by resource_group_name.

There are three CLI commands you’ll use:

environment is aliased to env, so chalk env config, chalk env update, and chalk env patch all work.

​

Inspecting the current configuration

Use chalk environment config --json to dump the active environment. With --json the command emits the full environment proto as JSON; without it you get a YAML rendering.

$ chalk environment config --json > env.json

# Inspect just the resource groups
$ chalk environment config --json | jq '.spec_config_json.resource_groups'

# Find a specific group
$ chalk environment config --json \
    | jq '.spec_config_json.resource_groups[] | select(.resource_group_name=="high-throughput")'

# Grab the environment id (needed by `chalk environment update`)
$ chalk environment config --json | jq -r '.id'

The active environment is determined by your CLI auth context — switch with chalk environment <name> before running the command, or use --env-id to scope individual operations.

​

Bulk updates with chalk environment update

chalk environment update overwrites whole top-level fields on the environment from a YAML config file. Use this when you want to replace the resource-group configuration wholesale (for example, deploying a checked-in environment.yaml from a repo).

$ chalk environment update \
    --env-id env_abc123 \
    --config-file environment.yaml

The YAML file may contain any subset of the following keys; only keys that are present get sent in the update mask, so omitted keys are left untouched on the server:

A minimal environment.yaml that edits only resource groups looks like:

specs_config:
  services:
    engine-grpc:
      min_instances: 2
      max_instances: 10
      target_cpu_utilization_percentage: 50
      chalk_machine_type: small
  resource_groups:
    - resource_group_name: high-throughput
      services:
        engine-grpc:
          min_instances: 5
          max_instances: 20
          engine_image_uri_override: gcr.io/chalk/engine:sha-abc123

specs_config is a wholesale replacement: anything you omit under it is removed. The typical workflow is therefore “fetch → edit → apply”:

$ chalk environment config --json | jq '.spec_config_json' > spec.json
# edit spec.json (or convert to YAML and edit)
$ yq -P spec.json > environment.yaml   # wrap under `specs_config:` first
$ chalk environment update --env-id $(chalk environment config --json | jq -r .id) \
    --config-file environment.yaml

For per-field edits without round-tripping the whole config, prefer chalk environment patch.

​

Targeted patches with chalk environment patch

chalk environment patch applies a kubectl-style patch to a single resource group within the active environment’s spec_config_json. The rest of the spec config is preserved as-is, which makes it safe to script narrow changes (image bumps, replica overrides, autoscaler tweaks) without touching unrelated services.

The command operates on the environment selected in your CLI auth context — there is no --env-id flag. Switch with chalk environment <name> first if needed.

There are two input modes.

​

Sugar form: --service + --set

The fastest way to bump a single field on a single service inside a resource group:

$ chalk environment patch \
    --resource-group high-throughput \
    --service engine-grpc \
    --set engine_image_uri_override=gcr.io/chalk/engine:sha-abc123

Each --set key=value entry nests under services.<service> of the resource group. You can pass --set multiple times. Values parse as JSON when they look like JSON (true, 42, "quoted", [1,2,3]); otherwise they’re treated as strings. Dotted keys build nested objects, so:

$ chalk environment patch \
    --resource-group high-throughput \
    --service engine-grpc \
    --set min_instances=5 \
    --set max_instances=20 \
    --set resources.requests.cpu=500m \
    --set resources.requests.memory=1Gi

is equivalent to a merge patch of:

{
  "services": {
    "engine-grpc": {
      "min_instances": 5,
      "max_instances": 20,
      "resources": { "requests": { "cpu": "500m", "memory": "1Gi" } }
    }
  }
}

​

Raw form: --patch / --patch-file

For anything the sugar can’t express (multiple services in one shot, deletes, array operations) supply a raw patch body. The --patch-type flag selects the format:

• merge (default) — RFC 7396 JSON merge patch. Setting a field to null deletes it.
• json — RFC 6902 JSON patch (an array of {op, path, value} operations).

# Inline merge patch
$ chalk environment patch --resource-group high-throughput \
    --patch-type merge \
    --patch '{"services":{"engine-grpc":{"engine_image_uri_override":"gcr.io/chalk/engine:sha-abc123"}}}'

# Delete a field by setting it to null in a merge patch
$ chalk environment patch --resource-group high-throughput \
    --patch '{"services":{"engine-grpc":{"engine_image_uri_override":null}}}'

# RFC 6902 patch from a file
$ chalk environment patch --resource-group high-throughput \
    --patch-type json --patch-file replace-image.json

# Read the patch from stdin
$ jq -n '{services:{"engine-grpc":{min_instances:5}}}' \
    | chalk environment patch --resource-group high-throughput --patch-file -

Sugar (--service/--set) and raw (--patch/--patch-file) modes are mutually exclusive.

​

Patching the default resource group

The fields directly under spec_config_json (the ones that live alongside resource_groups) are treated as the default resource group. Pass --resource-group default to patch them:

$ chalk environment patch --resource-group default \
    --service branch \
    --set request.cpu=2 \
    --set request.memory=6Gi

Patches targeting default are not allowed to touch the resource_groups key — patch each named group by name instead.

​

Previewing and confirming changes

Every patch prints a JSON diff of the resource group before vs. after, then prompts for confirmation before applying. Use --dry-run to see the diff and exit without applying, and --yes (or -y) to skip the prompt in scripts:

# Preview only
$ chalk environment patch --resource-group high-throughput \
    --service engine-grpc --set min_instances=5 --dry-run

# Apply non-interactively (CI)
$ chalk environment patch --resource-group high-throughput \
    --service engine-grpc --set min_instances=5 --yes

The patch fails fast if the named resource group doesn’t exist, if the patch body is invalid JSON, if the merge would change resource_group_name, or if a default patch would mutate resource_groups.

​

Shared Resources

Under Settings > Shared Resources, environment admins can view and manage shared resources across different services in your environment, including the Metrics Database configuration, Gateway configuration, and Background Persistence configurations. You can adjust these configurations here if, for example, you need to scale up your Metrics Database to handle larger volumes of data, or if you drastically increase the volume of data being written to the online or offline stores.

For enabling autoscaling of Background Persistence workers to handle variations in data being persisted online and offline, you can set the Horizontal Pod Autoscaling (HPA) Settings in the Background Persistence Configuration tab. In order to enable autoscaling, it is required to set

• Max replicas
• Min replicas
• Target average value (the approximate target undelivered message count in queue per replica)
• Pubsub Subscription ID

​

Engine Configuration Variables

Under Integrations > Config Variables, you can view and edit global configuration variables for data plane services like the query server or offline query consumer for your environment. Please reach out to the Chalk team if you have any questions about which configuration variables to set for your use case. Directly setting configuration variables is an advanced operation designed for enabling experimental or advanced features, and using the existing Resources pane in Infrastructure > Resources is preferable in most cases.

Configuration variables are directly injected into the query server or offline query jobs (the Chalk execution engine runtime) for the purpose of configuration. Values that you would like to make available as environment variables in your resolver runtime should be set under Integrations > Secrets instead.

​

(Legacy) Custom Resources Configuration

The legacy resources configuration system allows for extremely flexible configuration of your Chalk workloads, but can also lead to issues provisioning resources that can be hard to debug. It remains available but should be treated as an advanced form of configuration and should only be used when necessary for special cases not covered by Chalk Machine Types.

The primary controls that are available for configuring individual Chalk Services in this system are effectively passthroughs for the underlying Kubernetes API: requests and limits. Each of these allows specifying amounts of an available resource: cpu, memory, and ephemeral storage. Since the underlying system is the Kubernetes Resource API, specifying a request reserves the specified amount of that resource; any node that the service schedules on must have at least that much of that resource available; the available amount is calculated by adding up all the other resource requests (reservations) on that machine along with Kubernetes overhead and subtracting that from the available amount of the resource on the machine.

For example, if there is an 8-cpu machine (node) in the cluster, and there are 3 pods running on the node requesting 2 cpu each, the node will have (8 - 2*3 - overhead)=~1.5cpu available. A new pod with a 1-cpu request may be able to schedule on the node, but a new pod with a 2-cpu request will not, because of the overhead. An interesting consequence of the overhead that is always present is that a request for 2cpu or 4Gi of memory can never schedule on a node of that size; typically it will end up scheduling on a node twice the size (4cpu or 8Gi of memory).

This is just a reservation system for ensuring that too many pods don’t schedule on the same node; nothing about the requests actually requires the pod running your workload to use only the requested amount, and this applies to all pods running on a node. Setting limits will instruct Kubernetes to reach in and terminate a pod that is using “too much” memory, or throttle a pod using “too much” CPU. With Chalk Machine Types, requests and limits are set automatically, so you’ll never have to worry about fine-tuning resource requests to make sure your workload can schedule.

​

(Legacy) Isolating your workloads

The legacy resources configuration system contains a few different techniques for isolating your workloads, depending on the cloud provider you’re using:

• AWS/Azure (Karpenter): Your services can explicitly select an Instance Type and/or a Nodepool
• GCP: Your services can explicitly select a GCP Nodepool or a GCP Machine Family.
• All Clouds: Your services can isolate from each other with the Isolate from other Chalk Services checkbox

AWS and Azure clusters use Karpenter, a flexible node provisioning system that creates nodes on-demand. Karpenter provides the Nodepool as an available tool for isolation and configuration; Nodepools can be configured with an arbitrary and flexible set of constraints that determine what kinds of nodes they can produce. This is very flexible but can lead to internally-unsatisfiable configuration that prevents workloads from scheduling. In clusters using Karpenter, you can explicitly select a Nodepool via the Nodepool selector and you can independently select an Instance Type with the Instance Type selector. Using the Instance Type selector ensures that you will schedule on exactly that type of instance.

GCP Nodepools are different; each nodepool can only produce one Instance Type, so you can’t independently select Instance Type and Nodepool for your Services. If your cluster is set up with NAP then you can also select a GCP Machine Family, although this doesn’t give you as much control.

In all clusters, you can use the Isolate from other Chalk Services checkbox to set an anti-affinity that prevents your selected Service from running on the same node alongside other services; this is recommended for production configurations in order to avoid cross-service interference.

As you can see, this system is very flexible, but it can also be hard to debug: it’s possible to set various kinds of unsatisfiable configuration. For instance, you can configure a Karpenter Nodepool that only makes m6a.2xlarge instances, then explicitly request that nodepool, explicitly request an instance type of r8i.4xlarge, and independently configure a cpu request of 32. These configurations are all mutually incompatible: the Nodepool can’t make instances of the required type, and the 32-cpu request can’t fit on either of the specified instance types (it can’t fit on the larger machine because of overhead).

When you use a Chalk Machine Type, all of these settings are chosen automatically, including requests, limits, the Instance Type, the Nodepool, and the isolation setting. For this reason, selecting a Chalk Machine Type is recommended unless custom configuration is required. More information is available on the detailed docs page.

​

(Legacy) Nodepools

Configuring and using custom nodepools in AWS, GCP, and Azure environments is a legacy feature that will eventually be deprecated. Selecting a Chalk Machine Type instead of using custom resource configuration ensures that your workloads will always run isolated from each other on an appropriately sized cloud instance.

Under Infrastructure > Nodepools, you can view and configure the nodepools used to run your Chalk environment. A NodePool defines constraints on the Kubernetes nodes that can be scheduled in your Kubernetes cluster. Notably, you can select the machine families that you would like to use in your deployment, as different machine families are optimized for different workloads. You can also set a CPU limit on your NodePool. This page will allow you to provision the resources and machine families that are best suited for your workload and usage requirements.

When configuring a nodepool, you’ll notice two checkboxes, “Isolate this nodepool” and “Restrict to Chalk workloads only”.

“Isolate this nodepool” being set means that workloads cannot be scheduled in the nodepool unless they are specifically configured to do so under Infrastructure > Resource Configuration, which can be useful to minimize interference or reserve certain instance types for specific workloads. This causes the nodepool to have the Isolated label.

“Restrict to Chalk workloads only” being set means that unrelated non-Chalk workloads in the cluster will not be allowed to schedule on this nodepool, which can be useful for environments deployed into pre-existing clusters that also run unrelated workloads. This ensures that those non-Chalk workloads do not inadvertently consume Chalk credits. This causes the nodepool to have the Restricted label.

You may also notice that the nodepools you create through the Chalk dashboard automatically have the Chalk label. This indicates that the nodepool makes Chalk-managed nodes, whose uptime contributes to credit usage. All nodepools created through the Chalk dashboard automatically have this label, although the dashboard will also display nodepools without it. Chalk workloads are only allowed to schedule on nodes from nodepools with the Chalk label.

More details about these labels can be found in the Billing Documentation.

Once you have configured your Nodepools, under Infrastructure > Resource Configuration, you can select the Nodepool to use for each service in each resource group in your environment, as well as the Pod Disruption Budget.