Google BigQuery Destination

1. Use your existing Cloud Storage Bucket or create one if needed.

   • Follow the steps in the Cloud Storage documentation to create one.

2. Create a Custom IAM Role.

   • Use the following permissions and follow the IAM Role documentation.

3. Assign BigQuery and GCS Roles to your service account. Ensure your service account has the necessary roles to run the BigQuery load job.

   • BigQuery Roles:

     • BigQuery Data Editor: Grants permissions to read, create, update, and delete tables.

     • BigQuery Job User: Grants permissions to run jobs within the project.

       # Assign the BigQuery Data Editor role
       gcloud projects add-iam-policy-binding your-project-id \
         --member=user:your-email@example.com \
         --role=roles/bigquery.dataEditor
       
       # Assign the BigQuery Job User role
       gcloud projects add-iam-policy-binding your-project-id \
         --member=user:your-email@example.com \
         --role=roles/bigquery.jobUser

4. Create a new Service Account and assign Roles.

   • Follow the steps in the Service Account documentation to create a new service account and assign the roles from the previous step.

5. Grant Permissions to the Bucket.

   • Assign the required roles to the service account for the specific bucket by following the Bucket Permissions documentation.

6. Record Credentials.

   • Securely download and record the JSON key file for the new service account.

Permissions Summary

BigQuery Permissions

• bigquery.jobs.create

• bigquery.tables.update

• bigquery.tables.create

• bigquery.tables.get

• bigquery.tables.delete

Google Cloud Storage Permissions

• storage.buckets.get

• storage.buckets.list

• storage.objects.get

• storage.objects.create

• storage.objects.delete