Snowflake Unloading with GCS

Run the SQL snippets against your Snowflake instance (as a user with the ACCOUNTADMIN role); the IAM steps are performed in your Google Cloud project.

For background, see the Snowflake documentation:

• Unloading into Google Cloud Storage
• Configuring a Snowflake storage integration for GCS

​

Create Storage Integration

Run against your Snowflake instance. This creates the storage integration object that Snowflake will use to authenticate to GCS:

-- Set variables (customize these for your environment)
SET INTEGRATION_NAME='GCS_INTEGRATION_CHALK_DATA_BUCKET';
SET GCS_BUCKET='gcs://chalk-data-bucket/';

-- Create storage integration if it doesn't exist
CREATE STORAGE INTEGRATION IF NOT EXISTS IDENTIFIER($INTEGRATION_NAME)
  TYPE = EXTERNAL_STAGE
  STORAGE_PROVIDER = 'GCS'
  ENABLED = TRUE
  STORAGE_ALLOWED_LOCATIONS = ($GCS_BUCKET);

-- Verify integration and retrieve the Snowflake-managed service account
DESCRIBE INTEGRATION IDENTIFIER($INTEGRATION_NAME);

From the DESCRIBE INTEGRATION output, record the STORAGE_GCP_SERVICE_ACCOUNT value — an email like service-account-id@project.iam.gserviceaccount.com. You will grant this service account access to your bucket in the next step.

​

Grant the Service Account Access to Your GCS Bucket

These steps are performed in your Google Cloud project, using the service account email from Step 1.

1. In the Google Cloud Console, go to IAM & Admin → Roles and Create Role with the permissions required for unloading (read and write) on the GCS bucket or prefix:

   • storage.buckets.get
   • storage.objects.create
   • storage.objects.delete
   • storage.objects.get
   • storage.objects.list

   Alternatively, you can assign the predefined Storage Object Admin (roles/storage.objectAdmin) role instead of a custom role.

2. Go to Cloud Storage → Buckets and select your bucket.

3. Open Permissions → Grant access, paste the STORAGE_GCP_SERVICE_ACCOUNT email as the principal, and assign the role from the previous step.

4. If your bucket is encrypted with Cloud KMS, also grant the service account the Cloud KMS CryptoKey Encryptor/Decryptor role on the relevant key.

The Chalk workload service account also needs access to the same unload bucket or prefix so it can read the files Snowflake writes and clean them up. Grant it storage.buckets.get, storage.objects.get, storage.objects.list, and storage.objects.delete. Include storage.objects.create as well if that workload identity is also used for direct writes to this prefix.

​

Create External Stage

Run against your Snowflake instance. The stage points at a location inside your GCS bucket and writes through the storage integration created in Step 1:

-- Set variables (customize these for your environment)
SET STAGE_NAME='CHALK_UNLOAD_STAGE_GCS';
SET INTEGRATION_NAME='GCS_INTEGRATION_CHALK_DATA_BUCKET';
SET GCS_URL='gcs://chalk-data-bucket/unload/';

-- Create external stage using storage integration
CREATE STAGE IF NOT EXISTS IDENTIFIER($STAGE_NAME)
  STORAGE_INTEGRATION = <INTEGRATION_NAME>
  URL = $GCS_URL;

​

Grant Permissions

Run against your Snowflake instance. This grants the Chalk role access to the storage integration and stage so it can perform the unload:

-- Variables for reference (example values; use the same names as the earlier steps)
SET ROLE_NAME='CHALK_ROLE';
SET INTEGRATION_NAME='GCS_INTEGRATION_CHALK_DATA_BUCKET';
SET STAGE_NAME='CHALK_UNLOAD_STAGE_GCS';

-- Grant usage on storage integration
GRANT USAGE ON INTEGRATION <INTEGRATION_NAME> TO ROLE <ROLE_NAME>;

-- Grant stage permissions
GRANT USAGE ON STAGE <STAGE_NAME> TO ROLE <ROLE_NAME>;

​

Validate

You can optionally validate the integration end-to-end using

SELECT SYSTEM$VALIDATE_STORAGE_INTEGRATION('GCS_INTEGRATION_CHALK_DATA_BUCKET', 'gcs://chalk-data-bucket/unload/', 'test.csv', 'WRITE');

After completing these steps, continue to Configure Chalk environment.