Test a pull or merge request

Run Teststream from a GitHub pull request or GitLab merge request pipeline. For each request, Teststream maintains a result comment and posts a teststream commit status. Your CI job starts the run; connecting a Git provider alone does not trigger tests.

Set up once

1. In Organization > Providers, connect GitHub or GitLab. For GitHub, grant the Teststream GitHub App access to the intended repositories. For GitLab, authorize the connection. The server administrator must configure the provider integration if connection is unavailable.
2. In the project's Settings > Repositories, select the repository. The server uses this connection to read changed files and post results. TESTSTREAM_API_KEY authenticates CI to Teststream; it is not the provider's credential.
3. Follow CLI setup and CI prerequisites. Create Staging in the project with ready-to-run tests and a reachable application.
4. Supply the secret TESTSTREAM_API_KEY. Add the CI variable TESTSTREAM_URL only for a self-hosted Teststream server.

The examples below run on the provider's hosted runners with Node.js 22 and fetch the CLI with npx --yes teststream@1, which pins the major version. A self-hosted Teststream server must be reachable from the runner; if hosted runners cannot reach it, use a self-hosted runner on its network.

Only run secret-bearing jobs on changes you trust. Reject fork pipelines for these examples, and do not use a privileged event such as pull_request_target to bypass missing fork secrets.

Replace SHOP and Staging with your project and environment. These first examples run against that environment's deployed URL; add a preview URL only after your deployment job has produced one.

GitHub Actions

# .github/workflows/teststream.yml
name: Teststream
on:
  pull_request:
permissions:
  contents: read
jobs:
  e2e:
    if: github.event.pull_request.head.repo.full_name == github.repository
    runs-on: ubuntu-latest
    timeout-minutes: 40
    steps:
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - name: Run Teststream tests
        shell: bash
        env:
          TESTSTREAM_API_KEY: ${{ secrets.TESTSTREAM_API_KEY }}
        run: >
          npx --yes teststream@1 run
          --project SHOP --env Staging
          --mr auto --affected --wait --timeout 30m --json
          > teststream-run.json
      - name: Keep the run result
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: teststream-run
          path: teststream-run.json

Add the key as a repository secret. For a self-hosted server, also set TESTSTREAM_URL as a repository variable and add TESTSTREAM_URL: ${{ vars.TESTSTREAM_URL }} to the step's env. No application checkout is needed for --mr auto: the CLI reads the request number, head commit SHA and source branch from the event file, not from a checkout's synthetic merge commit.

The artifact is collected even if the run fails. Open its url in Teststream, or use its id with status and report. If the command could not start a run, stderr explains the error and the artifact can be empty.

GitLab CI

# Add this job to .gitlab-ci.yml.
teststream:
  stage: test
  image: node:22
  timeout: 40m
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_SOURCE_PROJECT_ID == $CI_MERGE_REQUEST_TARGET_PROJECT_ID'
  script:
    - >
      npx --yes teststream@1 run
      --project SHOP --env Staging
      --mr auto --affected --wait --timeout 30m --json
      > teststream-run.json
  artifacts:
    when: always
    paths:
      - teststream-run.json

Set TESTSTREAM_API_KEY in GitLab CI/CD settings, and TESTSTREAM_URL too for a self-hosted server. Mask the API key, and ensure your variable protection policy allows it in the trusted MR pipeline; do not broadly expose it to untrusted branches to make the job pass.

--mr auto uses CI_MERGE_REQUEST_IID, the MR project path and source branch name. For the commit, it prefers CI_MERGE_REQUEST_SOURCE_BRANCH_SHA, falling back to CI_COMMIT_SHA. A normal branch or scheduled pipeline without MR metadata cannot use --mr auto.

Use a preview deployment

Have the test job depend on your deployment job and receive the preview URL as an output or environment variable. Replace the run command with:

: "${PREVIEW_URL:?The deployment job must supply a ready preview URL}"
npx --yes teststream@1 run \
  --project SHOP --env Preview --base-url "$PREVIEW_URL" \
  --mr auto --affected --wait --timeout 30m

Create the Preview environment first, with appropriate credentials and allowed sites. --base-url does not create an environment or wait for a deployment.

The browser runner must be able to access this URL. A server running on your laptop is not reachable from a hosted runner merely because the job knows its localhost address.

Choose affected tests or the full suite

--affected needs --mr. It matches changed file-name words against the app map's screen paths, recorded API paths and test names, then includes tests tagged critical. This is a heuristic, not a code-coverage guarantee.

When the map is empty, nothing matches, or changed files cannot be read, it falls back to the available tests and explains the reason in selection.note and the request comment. Existing tag/folder filters still constrain this selection, including critical tests. Explicit --tests bypasses affected selection.

Leave out --affected for the full available suite:

npx --yes teststream@1 run \
  --project SHOP --env Staging --mr auto --wait

Run by hand from the application's checkout

After installing the CLI, switch to the application's source branch and run:

teststream run --project SHOP --env Local --mr 7 --affected --wait
teststream run --project SHOP --env Local \
  --mr https://github.com/owner/repo/pull/7 --wait
teststream run --project SHOP --env Local \
  --mr https://gitlab.example.com/group/shop/-/merge_requests/7 --wait

With a number, the repository comes from git remote origin, the commit from HEAD, and the branch from the current checkout. With a URL, local commit and branch information is attached only when the checkout matches the linked repository. Use --repo owner/repo, --provider github|gitlab and --sha <commit> to override detected values.

For a run with no explicit --branch, a detected MR/PR source branch opens or reuses its Teststream change set. The run uses that branch's proposed test versions and keeps its recordings on the branch.

What the pull request shows

Result	Provider status
Still running	Pending.
Failed, blocked, errored or skipped tests; no tests; failed run	Failed.
All passed, but test changes await review	Pending when the repository's pending-change policy is enabled.
All passed and review policy is satisfied	Success, including any healed passes noted in the description.
Run cancelled	Cancelled.

The comment includes counts, failure evidence, healed passes, estimated cost, selection details, branch proposals and the run link. New runs update the request's report rather than adding a fresh comment for every push.

Choose your merge requirements

On GitHub, make the appropriate checks required in branch protection/rulesets. On GitLab, require a successful pipeline and keep the test job required.

• Require the CI job to enforce the CLI verdict, including --fail-on-heal if you add it.
• Require the provider's teststream status if you also want its pending test-review policy to block a merge.

These are different signals: the CLI can return 0 while the provider status waits for test review, or return 1 for a healed pass while the provider status is successful. The explanation describes why.

Troubleshooting

Symptom	Check
Not in a merge request pipeline	Use a PR/MR event, or supply a number/link manually.
Run succeeds, but no comment or status appears	Inspect mergeRequestError in JSON; verify the organization's provider connection and the project's repository selection. Reporting errors do not necessarily fail the CLI job.
All tests passed but status stays pending	Review proposed/conflicting tests under Branches, and check the repository's pending-change policy.
--affected runs everything	Read selection.note; the map or provider file information may be missing, or the change may not match any test.
Wrong application version was tested	Check the deployed URL. Selecting a test branch does not deploy the application's git branch.