check_telemetry.py / gather_agent_info.py)
create_online_monitor.py /
provisioning)
pip install -r scripts/requirements.txt
gather_agent_info.py to automatically identify agent runtime, verify
telemetry, metric scopes, linked datasets, and more. This script covers
most of the manual verifications listed in subsequent steps.
python3 scripts/gather_agent_info.py --project-id {project_id} --agent-name {agent_name}gcloud beta monitoring metrics-scopes list projects/{project_id}. If a scoping project is returned, you MUST
deploy policies there.google_monitoring_monitored_project resources to extract the
scoping project.reasoning_engine_id or gen_ai_agent_name). Use
scan_duplicates.py to verify.references/ with names ending in _alert_policies.md to learn how to
configure alert policies based on type. By default you MUST configure all of
the following alert types UNLESS the user requests to generate explicit
alert policies and/or types. Follow their tables of content to help you find
the reference sections you need to read:| Alert Type | Reference File |
|---|---|
| Reliability | reliability_alert_policies.md [blocked] |
| Quality | quality_alert_policies.md [blocked] |
| Cost | cost_alert_policies.md [blocked] |
| Safety | safety_alert_policies.md [blocked] |
| Security | security_alert_policies.md [blocked] |
.tf) files (such as alerts.tf, variables.tf).condition_sql requires the provider version >= 6.0.0 (or late 5.x
versions supporting the feature).{gen_ai_agent_name="{agent_name}"} or
metric.labels.agent_resource_name="{agent_name}") in alerting conditions
unless explicitly requested (for example, "ONLY for this agent"). Merely mentioning
a specific agent name or ID in the request does NOT constitute an explicit
request to pin/filter; you MUST still default to dynamic grouping to cover
all agents. To cover all active agents in the project dynamically:sum(rate(workload_googleapis_com:gen_ai_invoke_agent_duration_count{monitored_resource="generic_node"}[5m])) by (gen_ai_agent_name)
sum(rate(workload_googleapis_com:gen_ai_invoke_agent_duration_count{monitored_resource="generic_node", gen_ai_agent_name="support-bot"}[5m]))
gen_ai_agent_name (for example, by (gen_ai_agent_name)). Avoid filtering to a single ID/Name unless
requested.agent_resource_name filter entirely. Configure the condition filter to
only target the monitored resource type
(aiplatform.googleapis.com/OnlineEvaluator) and metric type
(aiplatform.googleapis.com/online_evaluator/scores) globally for the
project.SELECT
JSON_VALUE(resource.attributes, '$."cloud.resource_id"') as agent_id,
...
FROM ...
GROUP BY agent_id
SELECT ...
FROM ...
WHERE JSON_VALUE(resource.attributes, '$."cloud.resource_id"') = 'support-bot'
ENDS_WITH filter
targeting a specific agent name. Instead, extract the agent identifier
(for example, JSON_VALUE(resource.attributes, '$."cloud.resource_id"')) and
add it to the GROUP BY clause alongside the model or tool name.monitoring/, ops/, sre/). Use tools to locate where
alert policies or state pointers exist in the project, rather than blindly
writing to the root.manage_task tool with action kill).Tooling Scripts section below.python3 scripts/gather_agent_info.py --project-id {project_id} --agent-name {agent_name}python3 scripts/scan_duplicates.py {target_tf_dir} --engine-var '${var.gen_ai_agent_name}'python3 scripts/lint_syntax.py {path_to_tf_file}lint_syntax.py
validation. Repeat this loop until the validation script passes
successfully.scan_duplicates.py exiting with code 1: Parse the JSON
output for duplicate resource targets. Perform in-place upgrade edits,
then re-check until it passes with 0.gather_agent_info.py
successfully returns the Trace or Log table names (or writes them to
variables file), do NOT redundantly call
list_trace_scope_table_names.py or list_log_scope_table_names.py.
These scripts are run internally by gather_agent_info.py and are
provided as external Fallbacks only.gather_agent_info.py, check_telemetry.py,
create_online_monitor.py, analyze_traffic.py,
list_log_scope_table_names.py, or list_trace_scope_table_names.py)
fails unexpectedly, you MUST read and inspect the stdout/stderr logs or
error output. Analyze the error message and attempt to dynamically
correct parameters and retry execution before escalating or
falling back to manual plans. Consult the relevant domain-specific
reference file for detailed troubleshooting steps for specific scripts.ALIGN_MEAN cannot be
applied to DELTA distribution metrics like online_evaluator/scores. You
MUST use percentile-based aligners (like ALIGN_PERCENTILE_50) to reduce
the score distribution into a comparable numeric stream.ls -R, find ., or raw recursive grep) from
the repository root if it contains a very large number of files, as this
will freeze your session. Always target specific subdirectories.Start with one job and grow from there.