# Resource Configuration source: https://docs.chalk.ai/docs/resource-configuration ## Configure your Chalk Kubernetes cluster ### Overview This page covers resources for Chalk's own platform services (the Query Server, Job Queue Consumer, Streaming Server, and similar) via the dashboard. To size a Sandbox, Scaling Group, or Function that you deploy yourself through Chalk Compute, set the cpu, memory, and gpu string parameters in code. See the Compute overview instead of this page. Go to Infrastructure > Resources to open the Cloud Resource Configuration page, where you can configure the resources allocated to each of your Chalk services and most other system-related settings. ### Resource groups and services Your Chalk environment consists of multiple Services, each of which serves a different purpose and represents a different kind of feature, compute, or background workload. For instance, the Query Server (also referred to as the engine) is responsible both for serving online inference queries and also for serving dataplane data requests, the Job Queue Consumer is responsible for pulling jobs like offline queries (backfills, dataset ingestions, and so on) from a queue and running them, and the Streaming Server handles execution and coordination of Stream Resolvers. Chalk also provides a higher-level isolation concept called a Resource Group. Each resource group provides its own independently-configurable set of Chalk Services, and is backed by logically-independent Kubernetes deployments. This provides isolation between different kinds of workloads so that they do not interfere with each other. For instance, you may want to run low-latency high-volume IO-bound inference queries against the Query Server in one resource group, and slower CPU-bound analytics queries against the Query Server in a separate resource group. You may also want to configure several different resource groups of different sizes so that you can target long-running offline workloads against machines of different sizes. Every Chalk environment has a Default resource group. You can add new resource groups and configure each one independently in the Cloud Resource Configuration pane. For online inference queries, each resource group exposes its own URL for the query client to use, and for offline workloads like Offline Query, a resource group can be specified in the resource_request parameter of the ChalkClient.offline_query() method and other related methods. ### Configuring service resources Each Chalk Service can be independently configured to consume different amounts of compute resources, depending on your needs. The primary way to configure resource consumption for a Service is to select a Chalk Machine Type from the dropdown selector on the Cloud Resource Configuration page when viewing the individual Service configuration pane. This guarantees that each replica of the Service will have full access to the listed CPU and memory amounts, and will run isolated on its own cloud instance, ensuring that separate services cannot interfere with each other. Different machine types are available for online, offline, and other kinds of workloads, since these workloads tend to have different performance requirements. In addition to configuring the resources allocated to each replica of a Service, you can configure the Service's scaling properties. Chalk autoscales on CPU utilization by default. The Query Server and gRPC Query Server can also scale on a schedule, and the Streaming Server can scale on stream backlog. You configure the min and max instance counts and the target cpu % for each Service independently in each Resource Group, so different workloads scale independently. More information about Chalk Machine Types can be found on the detailed docs page. ### Autoscaling Chalk supports autoscaling of resources in your environment in order to optimize resource usage. For the branch server, set the Auto Shutdown Period to the duration of inactivity after which Chalk automatically shuts down the branch server pod. The default is 1 hour. See The branch server. For the Query Server and gRPC Query Server, which serve your production traffic, you can configure autoscaling based on a CPU utilization target or set time-based pre-scaling. For stream resolvers, you can also set an autoscaling policy based on the stream queue message backlog. Chalk defines scaling policies with KEDA. See stream source autoscaling for supported sources, requirements, and consumer group configuration. ### CPU-based autoscaling To enable CPU-based autoscaling, set Target CPU % in the Cloud Resource Configuration. Chalk then monitors the CPU utilization of the service and scales the number of instances up or down based on the target CPU percentage. CPU-based autoscaling requires Min Instances, Max Instances, and Target CPU % to all be set for the service. ### Scheduled autoscaling Time-based scaling is available for the Query Server and gRPC Query Server. To autoscale on a schedule, define scaling triggers for a service on the Cloud Resource Configuration page: - Under Scaling, set Min Instances and Max Instances for the service. Time-based scaling is available only when the minimum is lower than the maximum. - Open Advanced, then open Time-based Scaling. - Select a timezone, then select Add Scaling Trigger. - For each trigger, enter the number of instances and the start and end times. Write the start and end times in standard cron syntax. The service scales to that number of instances between the start and end times. A trigger's instance count should fall between the service's minimum and maximum. The form shows a warning when it does not. Select Preview Schedule to chart the scheduled instance count for the previous and the next day. The underlying implementation of time-based scaling is the KEDA Cron scaler. ### Handling predictable traffic bursts Scheduled capacity and CPU-based autoscaling work best together: the schedule handles known demand, while CPU autoscaling provides additional capacity for unplanned or longer-lived demand. They do not replace capacity planning. If traffic can arrive as a large synchronized fan-out, consider spreading it over a short interval with batching or jitter where your application permits. This reduces the amount of capacity that must be available at the exact start of the burst. CPU-based autoscaling suits sustained or unexpected changes in demand, but it reacts after demand rises. When traffic rises abruptly, a new pod must still be created, scheduled, and pass readiness checks before it can serve requests. If the cluster needs to provision a node as well, this can take longer. Requests that arrive at the start of a burst may therefore see increased latency before autoscaling adds capacity. Allow several minutes of lead time for a scheduled increase when additional nodes might be needed. The actual time depends on signal evaluation, pod scheduling, node provisioning, image startup, and readiness checks. Exact times vary by environment, so use the p95 time from a scaling signal to a ready replica to choose the schedule's lead time. When your production traffic has a recurring peak (for example, a daily batch, campaign, or scheduled workflow), use scheduled autoscaling to make capacity available before the peak begins: - Measure the burst. Review request rate at a short interval, end-to-end latency, client-visible errors or timeouts, CPU utilization, and ready replica count. A five-minute average can hide a burst that lasts only seconds or minutes. - Choose a scheduled baseline. Set the desired replica count to the capacity required at the start of the peak, rather than the normal idle baseline. Establish this count with a representative load test or prior peak data. - Start early enough for readiness. Schedule the increase with enough lead time for pod scheduling, image startup, and readiness checks. Include extra time if a new node may need to be provisioned. - Keep overflow headroom. Set Max Instances above the scheduled baseline so CPU-based autoscaling can still respond to demand beyond the expected peak. The maximum should be tested against the resource requests, nodepool capacity, and any downstream dependencies. - Scale down after the peak. End the schedule once traffic has returned to its normal level to avoid paying for unused capacity. ### Programmatic management You can read and edit the same resource-group settings that Infrastructure > Resources exposes from the CLI. This is useful for codifying environment configuration in source control, scripting per-environment changes (image bumps, replica counts, autoscaler overrides), and applying targeted edits without leaving the terminal. The full configuration lives on the environment record under spec_config_json. A typical shape looks like: ``` { "id": "tm-abc123", "name": "production", "spec_config_json": { "services": { "engine-grpc": { "min_instances": 2, "max_instances": 10, "target_cpu_utilization_percentage": 50, "chalk_machine_type": "small" }, "branch": { "chalk_machine_type": "large" } }, "resource_groups": [ { "resource_group_name": "high-throughput", "services": { "engine-grpc": { "min_instances": 5, "chalk_machine_type": "large-highmem", "engine_image_uri_override": "gcr.io/chalk/engine:sha-abc123" } } } ] } } ``` The fields directly under spec_config_json (alongside resource_groups) make up the default resource group. Each entry in resource_groups is a named override group keyed by resource_group_name. Three CLI commands manage this configuration: | Command | What it does | | --------------------------------- | ------------------------------------------------------------------------- | | `chalk environment config --json` | Print the active environment, including `spec_config_json`, as JSON. | | `chalk environment update` | Replace whole top-level fields of an environment from a YAML file. | | `chalk environment patch` | Apply a targeted JSON-merge or RFC 6902 patch to a single resource group. | environment is aliased to env, so chalk env config, chalk env update, and chalk env patch all work. ### Inspecting the current configuration Use chalk environment config --json to dump the active environment. With --json the command emits the full environment proto as JSON; without it you get a YAML rendering. ``` $ chalk environment config --json > env.json # Inspect just the resource groups $ chalk environment config --json | jq '.spec_config_json.resource_groups' # Find a specific group $ chalk environment config --json \ | jq '.spec_config_json.resource_groups[] | select(.resource_group_name=="high-throughput")' # Grab the environment id (needed by `chalk environment update`) $ chalk environment config --json | jq -r '.id' ``` Your CLI auth context determines the active environment. Switch with chalk environment before running the command, or use --env-id to scope individual operations. ### Bulk updates with chalk environment update chalk environment update overwrites whole top-level fields on the environment from a YAML config file. Use this when you want to replace the resource-group configuration wholesale (for example, deploying a checked-in environment.yaml from a repo). ``` $ chalk environment update \ --env-id env_abc123 \ --config-file environment.yaml ``` The YAML file may contain any subset of the following keys; only keys that are present get sent in the update mask, so omitted keys are left untouched on the server: | YAML key | Effect | | -------------------------- | --------------------------------------------------------------------------------------- | | `specs_config` | Replaces the whole `spec_config_json` blob (resource groups, services, autoscalers, …). | | `additional_env_vars` | Replaces the environment-level env-var map (engine configuration variables) | | `private_pip_repositories` | Sets the private PyPI URL string. | | `online_store_secret` | Sets the secret name for the online store. | | `feature_store_secret` | Sets the secret name for the feature/offline store. | A minimal environment.yaml that edits only resource groups looks like: ``` specs_config: services: engine-grpc: min_instances: 2 max_instances: 10 target_cpu_utilization_percentage: 50 chalk_machine_type: small resource_groups: - resource_group_name: high-throughput services: engine-grpc: min_instances: 5 max_instances: 20 engine_image_uri_override: gcr.io/chalk/engine:sha-abc123 ``` specs_config is a wholesale replacement: anything you omit under it is removed. The typical workflow is therefore "fetch → edit → apply": ``` $ chalk environment config --json | jq '.spec_config_json' > spec.json # edit spec.json (or convert to YAML and edit) $ yq -P spec.json > environment.yaml # wrap under `specs_config:` first $ chalk environment update --env-id $(chalk environment config --json | jq -r .id) \ --config-file environment.yaml ``` For per-field edits without round-tripping the whole config, prefer chalk environment patch. ### Targeted patches with chalk environment patch chalk environment patch applies a kubectl-style patch to a single resource group within the active environment's spec_config_json. The rest of the spec config is preserved as-is, which makes it safe to script narrow changes (image bumps, replica overrides, autoscaler tweaks) without touching unrelated services. The command operates on the environment selected in your CLI auth context, and it has no --env-id flag. Switch with chalk environment first if needed. There are two input modes. ### Sugar form: --service + --set Use this form to change a single field on a single service inside a resource group: ``` $ chalk environment patch \ --resource-group high-throughput \ --service engine-grpc \ --set engine_image_uri_override=gcr.io/chalk/engine:sha-abc123 ``` Each --set key=value entry nests under services. of the resource group. You can pass --set multiple times. Values parse as JSON when they look like JSON (true, 42, "quoted", [1,2,3]); otherwise they are treated as strings. Dotted keys build nested objects, so: ``` $ chalk environment patch \ --resource-group high-throughput \ --service engine-grpc \ --set min_instances=5 \ --set max_instances=20 \ --set resources.requests.cpu=500m \ --set resources.requests.memory=1Gi ``` is equivalent to a merge patch of: ``` { "services": { "engine-grpc": { "min_instances": 5, "max_instances": 20, "resources": { "requests": { "cpu": "500m", "memory": "1Gi" } } } } } ``` ### Raw form: --patch / --patch-file For anything the sugar form cannot express (multiple services in one shot, deletes, array operations), supply a raw patch body. The --patch-type flag selects the format: - merge (default): An RFC 7396 JSON merge patch. Setting a field to null deletes it. - json: An RFC 6902 JSON patch (an array of {op, path, value} operations). ``` # Inline merge patch $ chalk environment patch --resource-group high-throughput \ --patch-type merge \ --patch '{"services":{"engine-grpc":{"engine_image_uri_override":"gcr.io/chalk/engine:sha-abc123"}}}' # Delete a field by setting it to null in a merge patch $ chalk environment patch --resource-group high-throughput \ --patch '{"services":{"engine-grpc":{"engine_image_uri_override":null}}}' # RFC 6902 patch from a file $ chalk environment patch --resource-group high-throughput \ --patch-type json --patch-file replace-image.json # Read the patch from stdin $ jq -n '{services:{"engine-grpc":{min_instances:5}}}' \ | chalk environment patch --resource-group high-throughput --patch-file - ``` Sugar (--service/--set) and raw (--patch/--patch-file) modes are mutually exclusive. ### Patching the default resource group The fields directly under spec_config_json (the ones that live alongside resource_groups) are treated as the default resource group. Pass --resource-group default to patch them: ``` $ chalk environment patch --resource-group default \ --service branch \ --set request.cpu=2 \ --set request.memory=6Gi ``` Patches that target default cannot touch the resource_groups key. Patch each named group by name instead. ### Previewing and confirming changes Every patch prints a JSON diff of the resource group before and after the change, then prompts for confirmation before applying. Use --dry-run to see the diff and exit without applying, and --yes (or -y) to skip the prompt in scripts: ``` # Preview only $ chalk environment patch --resource-group high-throughput \ --service engine-grpc --set min_instances=5 --dry-run # Apply non-interactively (CI) $ chalk environment patch --resource-group high-throughput \ --service engine-grpc --set min_instances=5 --yes ``` The patch fails immediately if the named resource group does not exist, if the patch body is invalid JSON, if the merge would change resource_group_name, or if a default patch would mutate resource_groups. ### Shared resources In Settings > Shared resources, environment admins can view and manage shared resources across different services in your environment, including the Metrics Database configuration, Gateway configuration, and Background Persistence configurations. You can adjust these configurations here if, for example, you need to scale up your Metrics Database to handle larger volumes of data, or if you drastically increase the volume of data being written to the online or offline stores. To autoscale Background Persistence workers as the volume of data persisted online and offline varies, set the Horizontal Pod Autoscaling (HPA) Settings in the Background Persistence Configuration tab. Autoscaling requires you to set: - Max replicas - Min replicas - Target average value (the approximate target undelivered message count in queue per replica) - Pub/Sub Subscription ID ### Engine configuration variables In Integrations > Config variables, you can view and edit global configuration variables for data plane services like the query server or offline query consumer for your environment. Contact the Chalk team with any questions about which configuration variables to set for your use case. Setting configuration variables directly is an advanced operation for enabling experimental or advanced features. In most cases, use the Resources pane in Infrastructure > Resources instead. Configuration variables are directly injected into the query server or offline query jobs (the Chalk execution engine runtime) for the purpose of configuration. To make values available as environment variables in your resolver runtime, set them in Integrations > Secrets & variables instead. ### (Legacy) Custom resources configuration The legacy resources configuration system allows flexible configuration of your Chalk workloads, but it can also cause resource provisioning issues that are hard to debug. It remains available as an advanced form of configuration. Use it only for special cases that Chalk Machine Types do not cover. The primary controls that are available for configuring individual Chalk Services in this system are effectively passthroughs for the underlying Kubernetes API: requests and limits. Each of these allows specifying amounts of an available resource: cpu, memory, and ephemeral storage. Since the underlying system is the Kubernetes Resource API, specifying a request reserves the specified amount of that resource; any node that the service schedules on must have at least that much of that resource available; the available amount is calculated by adding up all the other resource requests (reservations) on that machine along with Kubernetes overhead and subtracting that from the available amount of the resource on the machine. For example, if there is an 8-CPU machine (node) in the cluster, and there are 3 pods running on the node requesting 2 CPU each, the node will have (8 - 2*3 - overhead)=~1.5cpu available. A new pod with a 1-CPU request may be able to schedule on the node, but a new pod with a 2-CPU request will not, because of the overhead. Because this overhead is always present, a request for 2 CPU or 4Gi of memory can never schedule on a node of that size. It typically schedules on a node twice the size (4 CPU or 8Gi of memory). Requests reserve capacity so that too many pods do not schedule on the same node. They do not require the pod running your workload to use only the requested amount, and this applies to all pods running on a node. Setting limits instructs Kubernetes to terminate a pod that uses too much memory, or to throttle a pod that uses too much CPU. With Chalk Machine Types, Chalk sets requests and limits automatically, so you do not need to fine-tune resource requests to make sure your workload can schedule. ### (Legacy) Isolating your workloads The legacy resources configuration system contains a few different techniques for isolating your workloads, depending on your cloud provider: - AWS/Azure (Karpenter): Your services can explicitly select an Instance Type and/or a Nodepool. - GCP: Your services can explicitly select a GCP Nodepool or a GCP Machine Family. - All clouds: Your services can isolate from each other with the Isolate from other Chalk Services checkbox. AWS and Azure clusters use Karpenter, a flexible node provisioning system that creates nodes on-demand. Karpenter provides the Nodepool as an available tool for isolation and configuration; Nodepools can be configured with an arbitrary and flexible set of constraints that determine what kinds of nodes they can produce. This flexibility can lead to internally unsatisfiable configuration that prevents workloads from scheduling. In clusters using Karpenter, you can explicitly select a Nodepool via the Nodepool selector and you can independently select an Instance Type with the Instance Type selector. Using the Instance Type selector ensures that you will schedule on exactly that type of instance. In GCP, each Nodepool can produce only one Instance Type, so you cannot independently select Instance Type and Nodepool for your Services. If your cluster is set up with NAP then you can also select a GCP Machine Family, although this gives you less control. In all clusters, you can use the Isolate from other Chalk Services checkbox to set an anti-affinity that prevents your selected Service from running on the same node alongside other services; this is recommended for production configurations in order to avoid cross-service interference. This system is flexible, and it is possible to set various kinds of unsatisfiable configuration, which makes problems hard to debug. For instance, you can configure a Karpenter Nodepool that only makes m6a.2xlarge instances, then explicitly request that nodepool, explicitly request an instance type of r8i.4xlarge, and independently configure a CPU request of 32. These configurations are all mutually incompatible: the Nodepool cannot make instances of the required type, and the 32-CPU request cannot fit on either of the specified instance types (it cannot fit on the larger machine because of overhead). When you use a Chalk Machine Type, all of these settings are chosen automatically, including requests, limits, the Instance Type, the Nodepool, and the isolation setting. For this reason, we recommend selecting a Chalk Machine Type unless you need custom configuration. More information is available on the detailed docs page. ### (Legacy) nodepools Configuring and using custom nodepools in AWS, GCP, and Azure environments is a legacy feature that will eventually be deprecated. Selecting a Chalk Machine Type instead of using custom resource configuration ensures that your workloads always run isolated from each other on an appropriately sized cloud instance. In Infrastructure > Nodepools, you can view and configure the nodepools used to run your Chalk environment. A NodePool defines constraints on the Kubernetes nodes that can be scheduled in your Kubernetes cluster. You can select the machine families to use in your deployment, as different machine families are optimized for different workloads. You can also set a CPU limit on your NodePool. Use this page to provision the resources and machine families that best suit your workload and usage requirements. Configuring a nodepool includes two checkboxes, "Isolate this nodepool" and "Restrict to Chalk workloads only". When "Isolate this nodepool" is set, workloads cannot be scheduled in the nodepool unless they are specifically configured to do so under Infrastructure > Resources, which can be useful to minimize interference or reserve certain instance types for specific workloads. This gives the nodepool the Isolated label. When "Restrict to Chalk workloads only" is set, unrelated non-Chalk workloads in the cluster cannot schedule on this nodepool, which can be useful for environments deployed into pre-existing clusters that also run unrelated workloads. This prevents those non-Chalk workloads from inadvertently consuming Chalk credits. This gives the nodepool the Restricted label. Nodepools that you create through the Chalk dashboard automatically have the Chalk label. This label indicates that the nodepool makes Chalk-managed nodes, whose uptime contributes to credit usage. The dashboard also displays nodepools without the label. Chalk workloads can only schedule on nodes from nodepools with the Chalk label. More details about these labels can be found in the billing documentation. After you configure your Nodepools, go to Infrastructure > Resources to select the Nodepool to use for each service in each resource group in your environment, as well as the Pod Disruption Budget.