# Batching
source: https://docs.chalk.ai/docs/functions/batching

## Have a Chalk Function receive incoming items in batches with max_buffer_duration and max_batching_size.

When your handler is more efficient processing multiple items at once (e.g.
GPU batch inference), enable batching with max_buffer_duration and/or
max_batching_size. Chalk will accumulate incoming items and invoke the
handler in chunks:

```
import chalkcompute

@chalkcompute.function(
    max_buffer_duration=1000,  # buffer up to 1000 ms
    max_batching_size=32,      # or up to 32 items
    gpu="nvidia-l4",
)
def batch_embed(texts: list[str]) -> list[list[float]]:
    from sentence_transformers import SentenceTransformer
    model = SentenceTransformer("all-MiniLM-L6-v2")
    return model.encode(texts).tolist()
```

When batching is enabled, handler arguments are delivered as lists or a single
pyarrow table. Defaults apply when either parameter is set:

| Parameter             | Default (when batching enabled) |
| --------------------- | ------------------------------- |
| `max_buffer_duration` | `1000` ms                       |
| `max_batching_size`   | `10` items                      |