Sunday, November 15, 2026

Top 5 This Week

Related Posts

Google Cloud expands AI agent observability and controls


Google deploys a unified framework to track, govern and evaluate AI agents in BigQuery and Looker. Some building blocks are generally available, others in preview, with privacy guardrails, cost caps, and open source assessment tools for native and custom agents.

Evaluate before deployment and reconcile production logs

The open source Prism tool allows you to test conversation analytics agents via the API, whether they rely on BigQuery tables or Looker explorations. It supports reusable test suites and regex assertions, as well as multi-region agent discovery behind an access layer such as Cloud IAP. Prism can stream benchmark runs into day-partitioned BigQuery tables, placing pre-deployment benchmark scores alongside production agent_events logs for joint analysis in BigQuery and Looker. Meanwhile, first-party agent assessments in BigQuery are available for preview via the API and a guided Colab notebook, leveraging the Gemini Enterprise Agent Platform Assessment service with support for a customer-specific service account.

Protect data while tracing agent reasoning

Administrators can enable Cloud Monitoring and Cloud Trace on a project or organization scale. Cloud Monitoring provides aggregated metrics such as request volumes, latency, and token usage, while Cloud Trace provides step-by-step spans for each request, with observability scopes to aggregate multiple projects. By default, sanitized 1P metrics and spans exclude prompts, responses, internal thoughts, and generated SQL, allowing operational telemetry and content to be separated. If finer-grained auditing is required, conversation logging can be enabled on a per-agent basis: prompts, responses, and per-round telemetry are streamed to the BigQuery agent_events table via the BigQuery Storage Write API, inheriting IAM permissions, CMEK keys, and VPC Service Controls. Guardrails apply a fail‑closed check every round and skip logging in the presence of per-row security, per-column policy tags, or dynamic masking; a fail‑open execution maintains sessions in the event of an error.

A unified telemetry framework covers data and custom agents

A unified telemetry framework now covers 1P data agents and custom agents. Conversation analytics agents automatically emit telemetry, and agent-to-agent flows to BigQuery agents in Gemini Enterprise write Cloud Monitoring metrics, Cloud Trace spans, and chat logs to the respective project. Agents built with ADK or LangGraph can, in one line of instrumentation, stream the same structured telemetry to BigQuery. Prompts and responses from 1P logging are stored in customer-owned BigQuery tables. The Conversation Analytics API is generally available, with administrative audit logs always active and read audit logs optionally enableable, while providing compatibility with VPC Service Controls, CMEK, and regional or multi-regional data residency. BigQuery Studio offers predefined dashboards covering usage, active sessions, knowledge sources, and latency. In Looker (Google Cloud core and original), generally available system activity charts track token engagement and usage, while the Replies & Feedback tab previews user reviews and query reviews. It is also possible to create reports with an open source Data Studio model or deploy a ready-to-use Looker Block.

The AI ​​brief that professionals read

We test and decipher the new AI tools every evening, in 5 minutes. Free.

Included upon registration: our selection of the best AI guides & comparisons.

Choose your pace

Free · No spam · Unsubscribe in 1 click

Token FinOps: Detailed Tracking, Native Tables, and Active Caps

Promotional pricing offers concern conversational analytics, thus allowing the implementation of safeguards. During each conversation round, BigQuery logs the number of input, output, thought, cache tokens and total, in addition to latency. System activity charts in Looker provide estimated input and output tokens for BI workloads, while BigQuery Studio’s monitoring tab allows you to visualize expected usage and engagement. agent_events data can be mined in SQL and presented graphically using a seven-page open source Data Studio template or via a user-hosted Streamlit application. To avoid unexpected charges, big_query_max_billed_bytes limits per query, based on dry‑runs, allow you to block excessively large scans without incurring on-demand billing. In overview, daily input and output token quotas apply per project, user or agent, configurable in the Google Cloud console and via gcloud CLI.

Warehouse diagnostics and custom agent instrumentation

Developers of custom agents on ADK or LangGraph can use the open source BigQueryAgentAnalyticsPlugin plugin; A single-line configuration allows you to stream full execution trees, temporal telemetry, and multimodal loads to BigQuery. Since logs land natively in agent_events, no additional exports are required to diagnose failures. BigQuery AI functions like AI.GENERATE or AI.EMBED with VECTOR_SEARCH are used for error summarization and incident grouping, while the bigquery-agent-analytics Python SDK and a Colab notebook facilitate post-hoc semantic drift detection and sentiment analysis. Google recommends adding the SDK to agents, launching the open source Data Studio reporting template or the Streamlit app, and exploring analytics in the warehouse with a Colab notebook. He also recommends running automated accuracy suites with Prism, then setting up scan caps and daily token quotas. For 1P agents, enabling Cloud Monitoring and Cloud Trace, opting in to conversation logging, and tracking BI token consumption via Looker are indicated.

Shredder Smith
Shredder Smith
Shredder Smith is the lead curator and digital persona behind topaitools4you.com, an AI directory dedicated to "shredding" through industry hype to identify high-utility software for everyday users. Smith positions himself as a blunt, no-nonsense reviewer who vets thousands of emerging applications to filter out overpriced "wrappers" in favor of tools that offer genuine ROI and practical productivity. The site serves as a watchdog for the AI gold rush, providing categorized rankings and transparent reviews designed to help small businesses and creators navigate the crowded tech landscape without wasting money on low-value tools.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Popular Articles