BigQuery Logging and Monitoring Google Professional Data Engineer GCP

  1. Home
  2. BigQuery Logging and Monitoring Google Professional Data Engineer GCP
  • use Cloud Monitoring to monitor BigQuery project:
  • Display the metrics collected by Cloud Monitoring in own charts and dashboards.
  • Create an alert for any thresholds.

Useful Metrics –

Name Units
Scanned bytes Bytes per minute
Scanned bytes billed Bytes per minute
Query time
– 5th percentile
– 50th percentile
– 95th percentile
– 99th percentile
seconds
Slots allocated slots
Slots allocated by job type slots
Slots available slots
Stored bytes Bytes
Tables Tables
Uploaded billed bytes Bytes per minute
Uploaded bytes Bytes per minute
Uploaded rows Rows per minute
  • Use Cloud Audit Logs for audit log messages
  • BigQuery service has 3 distinct messages:
    • AuditData
    • BigQueryAuditMetadata
    • AuditLog
  • Go to the Logs Viewer page in the Cloud Console for viewing logs for policy tag events
Menu