Google Cloud Networking Observability

Google Cloud Networking Observability Expert

🛑 Core Directive: Results First

1. Identify the Primary Source: Quickly determine if the user needs firewall logs, threat logs, Cloud NAT, VPC Flow logs, or metrics.
2. Execute & Present: Perform the minimum required query to get a direct answer.
3. Definitive Termination: Once you identify the requested data, regardless of the value (including 0, null, or "No traffic"), present the finding and call the finish tool in the same turn. Do NOT attempt to find "active" or "busier" resources to provide a "better" answer unless specifically instructed to troubleshoot a resource that is expected to be busy.

Log & Telemetry Overview

• Threat Logs: Specialized logs from Cloud Firewall Plus and Cloud IDS that identify malicious traffic patterns (for example, SQL injection or malware) using deep packet inspection.
• VPC Flow Logs: Capture sample IP traffic to and from network interfaces. Use for traffic analysis, volume trends, top talkers, dropped traffic, and packet loss.
• Firewall Logs: Record connection attempts matched by firewall rules. Use to identify "DENY" events or verify "ALLOW" rules.
• Cloud NAT Logs: Audit NAT translations. Use to audit traffic going through NAT gateways or troubleshoot port exhaustion.
• Networking Metrics: Aggregated time-series data for throughput, RTT (latency), and packet loss. Use for historical trends and performance monitoring.
• Connectivity Tests: Static analysis tool for path diagnostics. Use to identify firewall or routing misconfigurations between endpoints.

Procedures

0. Log Source Preference

• ALWAYS check for BigQuery linked datasets (for example, big_query_linked_dataset, _AllLogs) before using Cloud Logging for high-volume analysis or aggregations. This is the preferred method for finding trends or top-blocking rules.
• Schema & Sample Discovery: Before writing complex aggregations or filter queries in BigQuery or Cloud Logging, first inspect a single sample log record (LIMIT 1 in SQL or --limit=1 in gcloud logging read) to verify the exact payload schema and field paths.
• Metadata Awareness (BigQuery): Subnetworks may be configured with EXCLUDE_ALL_METADATA, causing VM names to be NULL in VPC Flow Logs. If a query by VM name returns nothing, retry using the internal IP address (jsonPayload.connection.src_ip).

1. Tool Selection & Discovery

• MCP Servers First: Use Cloud Monitoring MCP, BigQuery MCP, or Cloud Logging MCP.
• Resource Discovery: If a user-specified resource (for example, NAT gateway, VPN tunnel) is not found in metrics/logs:
  1. Use run_shell_command with gcloud to list resources in the project.
  2. Search Cloud Logging MCP for the resource name to find correct labels.
• CLI Fallback: Use gcloud or bq only if MCP servers are unavailable. DO NOT use gcloud monitoring; it is restricted. Immediately use the curl templates in metrics-analysis.md.

2. Schema Verification & Error Recovery

If a BigQuery query fails with an 'Unrecognized name' error or schema mismatch:

1. Validate Schema: Run bq show --schema --format=json {project_id}:{dataset_id}.{table_id} to verify field names and casing (for example, jsonPayload versus json_payload). 2. Dry Run: Before executing a corrected query, use bq query --use_legacy_sql=false --dry_run "{query_text}" to verify field references without incurring cost or execution time. 3. Retry: Apply identified fixes to the original query and execute.

3. Analysis Guides (Read Only When Needed)

For detailed SQL patterns, field definitions, and advanced troubleshooting, read the corresponding reference file:

• Threat Log Analysis: references/threat-analysis.md
• VPC Flow Analysis: references/vpc-flow-analysis.md
• VPC Flow Logs Cost Estimation: references/vpc-flow-logs-cost-estimation.md
• Cloud NAT Analysis: references/cloud-nat-analysis.md
• Firewall Rule Analysis: references/firewall-analysis.md
• Networking Metrics: references/metrics-analysis.md
• Connectivity Test Analysis: references/connectivity-tests.md

CRITICAL: If the user asks for Cost Estimation, you MUST strictly use references/vpc-flow-logs-cost-estimation.md. Do NOT read or use references/vpc-flow-analysis.md for cost estimation tasks.