GCP Services: Cloud SQL, Pub/Sub and Cloud Monitoring

Three services that most GCP applications use together: a managed relational database, a message bus that decouples producers from workers, and monitoring that tells you when something breaks. This post wires them into one order-processing flow.

Architecture

flowchart TB
  API[Orders API<br/>Cloud Run] -->|publish| T[Pub/Sub topic orders]
  T --> S[Subscription orders-worker]
  S --> W[Worker<br/>Cloud Run]
  S -.->|after 5 failed attempts| DLQ[Dead-letter topic orders-dlq]
  W --> SQL[Cloud SQL PostgreSQL<br/>regional HA, private IP]
  API --> SQL
  SQL -.->|metrics| MON[Cloud Monitoring]
  W -.->|logs and metrics| MON
  DLQ -.-> MON
  MON --> AL[Alerting policy] --> NC[Notification channel<br/>email and Slack]

Cloud SQL

Cloud SQL runs MySQL, PostgreSQL and SQL Server with backups, high availability and patching handled for you. First give the VPC private connectivity to Google-managed services (private services access), then create the instance on private IP:

gcloud compute addresses create google-managed-services-app-vpc --global \
  --purpose=VPC_PEERING --prefix-length=16 --network=app-vpc
gcloud services vpc-peerings connect --service=servicenetworking.googleapis.com \
  --ranges=google-managed-services-app-vpc --network=app-vpc

gcloud sql instances create shop-db --database-version=POSTGRES_16 --edition=ENTERPRISE \
  --region=asia-south1 --tier=db-custom-2-7680 --availability-type=REGIONAL \
  --backup-start-time=21:00 --enable-point-in-time-recovery \
  --maintenance-window-day=SUN --maintenance-window-hour=22 \
  --no-assign-ip --network=app-vpc --database-flags=cloudsql.iam_authentication=on

Setting	Effect
--availability-type=REGIONAL	A standby in another zone with automatic failover, at roughly double the instance cost
Backups and point-in-time recovery	Restore to any moment within the retention window; test a restore
--no-assign-ip --network	Private IP only; applications reach it from inside the VPC
IAM database authentication	Users and service accounts sign in with IAM instead of stored passwords
Maintenance window	Chooses when patching restarts happen

Applications connect through the Cloud SQL Auth Proxy (or the language connectors), which encrypts traffic and checks IAM permission (roles/cloudsql.client), never through an open public address:

./cloud-sql-proxy --private-ip demo-ci-2025:asia-south1:shop-db

Add read replicas for reporting queries, and watch connections: each Cloud Run instance can open its own connections, so use a pool sized to the database’s limit.

Pub/Sub

Pub/Sub is Google’s publish and subscribe messaging service (comparable to SNS plus SQS):

gcloud pubsub topics create orders
gcloud pubsub topics create orders-dlq
gcloud pubsub subscriptions create orders-worker --topic=orders --ack-deadline=60 \
  --dead-letter-topic=orders-dlq --max-delivery-attempts=5 \
  --min-retry-delay=10s --max-retry-delay=300s
gcloud pubsub subscriptions create orders-dlq-sub --topic=orders-dlq
gcloud pubsub topics publish orders --message='{"id":1}'

Dead-letter topics need permissions: the Pub/Sub service agent (service-PROJECT_NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com) must have roles/pubsub.publisher on the dead-letter topic and roles/pubsub.subscriber on the subscription. Without them, failing messages are retried forever.

Design points: delivery is at least once by default, so workers must be idempotent (exactly-once delivery exists, but only for pull subscriptions, and adds latency); use ordering keys only when order really matters; and always alert on the age of the oldest unacknowledged message and on the dead-letter backlog.

Cloud Monitoring and Logging

Google services send metrics and logs automatically. On virtual machines, install the Ops Agent for memory, disk and file logs:

curl -sSO https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh
sudo bash add-google-cloud-ops-agent-repo.sh --also-install

Useful signals and alert thresholds to start from:

Signal	Alert when
Cloud SQL CPU utilisation	Above 80 percent for 10 minutes
Cloud SQL disk utilisation	Above 85 percent
Cloud SQL connections	Above 80 percent of the instance limit
Cloud Run 5xx response rate	Above 1 percent for 5 minutes
Pub/Sub oldest unacked message age	Above 5 minutes
Dead-letter subscription backlog	Above zero

Create a log-based metric for application errors and an alert policy with a notification channel:

gcloud logging metrics create app_errors \
  --description="Application errors" \
  --log-filter='resource.type="cloud_run_revision" AND severity>=ERROR'
gcloud alpha monitoring policies create --policy-from-file=cpu-policy.json

A policy file defines a condition (for example the metric cloudsql.googleapis.com/database/cpu/utilization, threshold 0.8, duration 600 seconds), the notification channels and documentation text that tells the responder what to check. Add uptime checks for public endpoints and SLOs for key user journeys, and keep log retention and exclusion rules in mind, since Cloud Logging bills for ingestion.