GitHub Action¶

Fire the suite on every push/PR and fail the build when your agent leaks, takes an unsafe action, crashes, or drifts out of scope. On pull requests it posts (and keeps updating) a report comment.

# .github/workflows/agent-reliability.yml
name: agent-reliability
on: [pull_request]

permissions:
  contents: read
  pull-requests: write        # for the PR comment

jobs:
  gauntlet:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }

      # Bring your agent up, or point at a deployed staging URL.
      # - run: ./scripts/start-staging-agent.sh

      - uses: GauntletVectorLabs/gauntlet/.github/actions/gauntlet@main
        with:
          target: ${{ secrets.STAGING_AGENT_URL }}
          canaries: canaries.json
          fail-on: HIGH

Inputs¶

Input	Default	Description
`target`	— (required)	Agent HTTP endpoint
`canaries`	`""`	Path to a canaries JSON
`fail-on`	`HIGH`	Gate severity
`request-field`	`message`	Field the probe goes in
`response-field`	`response`	Field to read the reply from
`header`	`""`	One extra request header `Key: Value`
`comment`	`true`	Post/update a PR comment

The step exits nonzero — failing the job — when any finding is at or above fail-on.

Hosted alternative¶

If you use the dashboard, trigger a run with an API key and let it track regressions across builds:

- run: |
    curl -X POST "$DASH/api/runs" \
      -H "Authorization: Bearer ${{ secrets.GAUNTLET_KEY }}" \
      -d target=${{ secrets.STAGING_AGENT_URL }} -d gate=HIGH