cxas trace¶

cxas trace is the observability and debugging surface for past conversations. It composes the Conversational Agents API, Cloud Logging, GCS, and Gemini into a single, scriptable workflow so you can debug a flaky live conversation, audit an eval failure, replay a conversation against the current agent, or flag a platform bug — all from your terminal.

All cxas trace subcommands share a few flags:

Subcommands¶

Examples¶

List the 10 most recent audio conversations from the last day:

cxas trace list \
  --app-name projects/p/locations/l/apps/a \
  --time-filter 24h --channel AUDIO --limit 10

Build a Markdown trace report with merged Cloud Logs, downloaded audio, and Gemini audio + transcript analysis — written to a file:

cxas trace get conv-id-1 \
  --app-name projects/p/locations/l/apps/a \
  --format md --with-logs --with-audio --with-analysis --with-triage \
  --out trace.md

Compare a deployed conversation against the current agent:

cxas trace replay conv-id-1 --app-name projects/p/locations/l/apps/a --diff

Generate a 7-day stats report grouped by source, written as Markdown:

cxas trace stats --time-filter 7d --source LIVE \
  --app-name projects/p/locations/l/apps/a --out stats.md

Flag a conversation as a platform bug with reason and severity:

cxas trace bug-report conv-id-1 \
  --app-name projects/p/locations/l/apps/a \
  --reason "agent hallucinated the refund amount" --severity high

Audio Transcription, WER Evaluation & BigQuery Reprocessing¶

The cxas trace audio transcribe (or cxas trace transcribe-audio) subcommand enables end-to-end audio speech-to-text transcription auditing using Gemini Flash/Flash-Lite multimodal models.

Capabilities¶

1. GCS Audio Turn Discovery: Automatically locates user turn recordings (user-turn-*.wav) for a conversation in the app's configured GCS audio bucket.
2. Gemini STT Transcription: Transcribes user speech verbatim using GeminiGenerate (gemini-3.5-flash, gemini-2.5-flash-lite, etc.) with zero temperature.
3. Word Error Rate (WER) Metrics: Aligns baseline CES real-time transcripts with Gemini transcription ground-truth using dynamic programming Levenshtein distance, reporting Substitutions ($S$), Deletions ($D$), Insertions ($I$), and overall WER ($WER = \frac{S + D + I}{N}$).
4. Multilingual & Non-English Turn Filtering: Pass --only-non-english to filter and reprocess only user turns containing non-ASCII / foreign characters.
5. Append-Only BigQuery Updates Table: Safely appends only the reprocessed/updated turns into a target BigQuery table (reprocessed_transcripts), auto-creating the table schema if it does not exist. This design avoids DML locks and table overwrite conflicts when running multiple parallel CLI jobs simultaneously.
6. Parallel Concurrency: Fully parallelized across user turns and BigQuery row inserts using --max-workers.

Command Flags¶

Examples¶

1. Transcribe a conversation and view WER metrics (Dry Run)¶

cxas trace transcribe-audio conv-12345 \
  --app-name projects/my-project/locations/us/apps/my-app \
  --model gemini-3.5-flash \
  --dry-run

2. Transcribe only non-English turns and output as Markdown¶

cxas trace audio transcribe conv-12345 \
  --app-name projects/my-project/locations/us/apps/my-app \
  --only-non-english \
  --format md \
  --out transcription_wer_report.md

3. Reprocess turns into a shared BigQuery updates table in parallel¶

cxas trace transcribe-audio conv-12345 \
  --app-name projects/my-project/locations/us/apps/my-app \
  --table my_dataset.reprocessed_transcripts \
  --max-workers 16

Configuration: ./.cxas/trace.yaml¶

cxas trace is fully configurable. Drop a trace.yaml next to your agent in ./.cxas/trace.yaml (or globally at ~/.cxas/trace.yaml) to override prompts, log filters, audio behavior, and the bug-report destination. Every field has a sensible default, so the file is optional.

audio:
  bucket_override: null            # leave null to use app.json's gcsBucket
  uri_pattern: "{bucket}/{conversation_id}.wav"
  download_dir: ./.cxas/audio
  mime_type: audio/wav

cloud_logging:
  default_level: WARNING
  time_padding_seconds: 30
  filter_template: |
    severity >= "{level}"
    AND timestamp >= "{start_time}" AND timestamp <= "{end_time}"
    AND (jsonPayload.conversation_id="{conversation_id}"
         OR labels.conversation_id="{conversation_id}")

gemini:
  model: gemini-2.5-flash
  # Audio analyses come from `cxas_scrapi.utils.audio_analysis.ANALYSIS_REGISTRY`
  # (5 built-ins: agent_voice_consistency, no_long_pauses,
  # agent_having_trouble, agent_looping, agent_cutoff). Each declares which
  # files it needs (e.g. `agent-turn-*.wav` vs `full-session.wav`) and a
  # default prompt. Override a prompt per-project by listing the metric here:
  audio_metrics: {}
  # Example override:
  #   agent_cutoff:
  #     prompt: "Custom cutoff prompt for our agent..."
  triage_metrics:
    hallucination:        { prompt: "..." }
    off_topic:            { prompt: "..." }
    failed_understanding: { prompt: "..." }

ui:
  ces_console_base: https://ces.cloud.google.com
  ccai_insights_base: https://ccai.cloud.google.com/insights

bug_report:
  bucket: gs://cxas-platform-bugs
  path_template: "{model_version}/{date}/{user}/{severity}/{conversation_id}/"
  include: [transcript, logs, audio, gemini_analysis, environment]

App-side discovery¶

cxas trace reads the same app.json and environment.json files that cxas pull writes — no extra setup required. From app.loggingSettings:

• audioRecordingConfig.gcsBucket → audio download source
• cloudLoggingSettings.enableCloudLogging → gates --with-logs
• bigqueryExportSettings → surfaced in metadata (informational)

$env_var placeholders are resolved against the chosen environment.json — the same convention used by cxas push.