...
Step 7: Execute the Script
- Open BigQuery Console
- Navigate to your project
- Paste the complete script in the query editor: https://gitlab.syensqo.com/syensqo-connected-research/toolkit/toolkit/-/blob/master/SQL/gemini_column_description.sql?ref_type=heads
- Update the configuration variables at the top
- Run the script
...
"Permission denied" errors
- Verify all required APIs are enabled
- Check IAM roles are correctly assigned
- Ensure connection service account has Vertex AI access
"Model not found" errors
- Verify the Vertex AI connection was created successfully
- Check the model name matches exactly
- Ensure the connection region matches your dataset location
"No tables found" errors
- Verify source dataset exists and contains tables
- Check dataset names in the configuration variables
- Ensure the service account has access to source datasets
Quota exceeded errors
- Check Vertex AI quotas in the Cloud Console
- Consider reducing sample_size or processing fewer tables at once
Monitoring and Costs
- Monitor usage: Go to Cloud Console > Vertex AI > Quotas to track API usage
- Cost estimation: Gemini AI pricing varies by model and token usage
- BigQuery costs: Query processing and storage costs apply
Security Best Practices
- Use least privilege: Grant only the minimum required permissions
- Service accounts: Consider using dedicated service accounts for production
- Data access: Implement row-level security for sensitive datasets
- Audit logging: Enable Cloud Audit Logs for compliance
- Network security: Use VPC Service Controls for additional isolation
Next Steps
Once installed, you can:
- Schedule the script to run periodically using Cloud Scheduler
- Integrate with data catalog tools for metadata management
- Customize prompts for domain-specific descriptions
- Extend to other Vertex AI models for different use cases
Support
Documentation
For issues specific to:
- BigQuery: BigQuery Documentation
- Vertex AI:Vertex AI DocumentationIAM: IAM Documentation
...