Skip to content

cxas ci-test

cxas ci-test runs the complete CI test lifecycle — it pushes your local agent to a temporary app, runs tool tests and platform evaluations, and exits with a clear pass/fail signal so your pipeline knows exactly what happened.

Usage

cxas ci-test [--app-dir DIR]
             --project-id PROJECT
             --location LOCATION
             [--display-name NAME]
             [--env-file FILE]

Options

Option Required Default Description
--app-dir DIR No . (current directory) Path to the agent directory to push and test.
--project-id PROJECT Yes — GCP project ID.
--location LOCATION Yes — GCP location (e.g., global, us-central1).
--display-name NAME No [CI] PR Test <uuid> Deterministic display name for the temporary app. If an app with this name already exists it is reused and updated rather than creating a duplicate. Use a name tied to your PR number (e.g., [CI] PR-123) to keep things tidy.
--env-file FILE No — Path to a JSON environment file to inject as environment.json. Useful for passing secrets that shouldn't live in source control.

What the Lifecycle Does

cxas ci-test orchestrates several steps automatically:

  1. Push — Uploads the agent directory to a temporary app (creating or updating it based on --display-name).
  2. Tool Tests — If tests/tool_tests.yaml exists inside the agent directory, runs cxas test-tools against the temp app.
  3. Evaluations — Discovers all evaluations in the temp app and runs each one via cxas run --wait --filter-auto-metrics.
  4. Result — Exits 0 if everything passed, 1 if anything failed.

The temporary app is left running after the lifecycle completes so you can inspect it in the Console if needed. Use cxas delete to clean it up manually, or rely on the cleanup job generated by cxas init-github-action.

Examples

Run the CI lifecycle from the current directory:

cxas ci-test \
  --project-id my-gcp-project \
  --location us-central1

Run for a specific PR with a deterministic app name:

cxas ci-test \
  --app-dir ./my-agent \
  --project-id my-gcp-project \
  --location us-central1 \
  --display-name "[CI] PR-42"

Run in a GitHub Actions workflow step:

- name: Run CI Tests
  run: |
    cxas ci-test \
      --app-dir ./my-agent \
      --project-id ${{ vars.GCP_PROJECT_ID }} \
      --location ${{ vars.GCP_LOCATION }} \
      --display-name "[CI] ${{ github.ref_name }}" \
      --env-file ./secrets/env.json

Use with a secrets-based environment file:

cxas ci-test \
  --app-dir ./my-agent \
  --project-id my-gcp-project \
  --location us-central1 \
  --display-name "[CI] PR-42" \
  --env-file /secrets/environment.json