# Testing

This is the screen for checking the state of testing ('Testing and feedback' in the sidebar). It is split into two tabs: **'Scenario tasks'** and **'Standalone tasks'**.

![PM on Rails test results. Pass/fail and evidence per scenario. Before: 'it works' was verbal. After: the result against acceptance criteria stays written.](https://pmonrails.com/docs-img/test-results.png)

## Scenario tasks (pass or fail for scenarios)

This tab records and collects whether scenarios ([user stories and scenarios](https://pmonrails.com/en/docs/stories.md)) **passed or failed** in the actual tests.

- Choose **story → scenario** in the tree on the left to see that scenario's result (**pass / fail / skipped / not run**).
- **Coverage (the proportion tested) and pass counts at the top are aggregated for the whole project** when you open the screen (you do not need to expand the tree for the numbers to appear). Pick a scenario in the tree to inspect individual results.
- You can filter by state (all / pass / fail / not run).
- Results can be recorded by hand ('pass / fail / skipped') or taken in from the development side (CI and the like). The run history is kept as well. Over MCP, you record with `create_test_result` against the `task_id` of the scenario task that was generated automatically when the scenario was created.
- **Record from CI automatically** — when a workflow in a repository connected through the GitHub App leaves a per-scenario pass/fail report as an artifact, PM on Rails reads it and records the results on the scenario tasks. No PM on Rails token or secret is needed in the repository. See "Record from CI automatically" below for the setup.
- Every run in the history carries its **source** (CI automated test / manual record / agent) and can be filtered by it. CI rows show the workflow name and a link to the run, so 'the tests pass' and 'a person confirmed the behaviour' stay distinguishable. The origin of a CI row cannot be rewritten later, and a manual record cannot be turned into a CI one.
- **Only a pass on the current version counts as verified.** A pass recorded against an older version (the scenario has since been edited) or without a recorded version is treated as unverified in project health and similar summaries.
- **Evidence images and video** — For manual records you can attach files or **paste from the clipboard** (Cmd/Ctrl+V), including in the [Mac desktop app](https://pmonrails.com/en/docs/desktop.md). Paste is not captured while another input field has focus.
- From a failing scenario you can **'raise a bug'** on the spot and send it to the board to be dealt with.

> Editing the scenario definition itself happens on the [user stories](https://pmonrails.com/en/docs/stories.md) screen (this tab concentrates on recording results). `create_scenario` does not pre-create TestResult rows.

## Record from CI automatically

The results of a GitHub Actions run can be recorded as pass/fail per scenario. Nobody has to press "pass": scenarios whose tests pass and scenarios whose tests fail show up on the test screen by themselves.

### Prerequisites

1. **Connect the project and the repository through the GitHub App** ([GitHub integration](https://pmonrails.com/en/docs/github.md); click 'Add to GitHub' in Workspace settings > Integrations > GitHub, then choose the repository in Project settings > Integrations > GitHub).
2. The GitHub App must have the **Actions: Read** permission (used to list and download artifacts). Without it the import is left in the sync event history as "cannot fetch the artifact (insufficient permission)".

### Steps

**1. Put the scenario id in the test name**

Tests are bound to the id of the SDLCScenario (not the id of the scenario task). You can find the id with "Copy Scenario ID" on a scenario in the stories screen, or with `list_scenarios` / `read_scenario` over MCP. Include that id in the test's name as `[pmr:<Scenario ID>]` (for example `it('logs in [pmr:3f2a9c1e-…]')`). If your runner cannot put it in the name, a JUnit `<property name="pmr_scenario" value="<Scenario ID>"/>` works too.

Bind only tests whose **assertions verify the scenario's Then as written**. Binding a test that checks only part of it turns "verified" into a lie.

You do not copy the scenario name, Gherkin or version into the repository. Every version of a scenario has its own id, so the id alone pins down which version was verified.

**2. Emit your runner's JUnit XML**

There is no dedicated report format or converter. Use the JUnit XML your runner already writes: pytest `--junitxml`, PHPUnit / Pest `--log-junit`, Jest / Vitest `jest-junit`, go `gotestsum --junitfile`, and so on.

When several tests are bound to the same scenario they fold into one result: any failure means fail, passes without failures mean pass, only skips mean skipped.

**3. Upload the JUnit XML as an artifact in the workflow**

The artifact name **must start with `pmr-scenario-report`**. When jobs are split (unit / integration, say) distinguish them with a suffix such as `pmr-scenario-report-unit`.

```yaml
- name: Run tests
  run: vendor/bin/pest --log-junit /tmp/junit.xml

- name: Upload scenario report
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: pmr-scenario-report
    path: /tmp/junit.xml
    if-no-files-found: ignore
```

With `if: always()` a run whose tests fail still records "fail".

**4. Check the results**

- When the workflow completes, PM on Rails reads the artifact in the background and records the results on the scenario tasks. They appear in the run history on the test screen with the source "CI automated test", the workflow name and a link to the run.
- The progress and reasons of the import appear in the **sync event history** under "Settings > GitHub". A missing artifact, a missing permission or a malformed report is also explained there.

### Scenarios that are not recorded

Only **agreed versions (approved / active)** are recorded. The following are not recorded, so that no false "verified" is created; the reason is kept in the sync event history.

- The version behind the reported id is still a draft / in review (agree on it, then rerun)
- The reported id is a superseded version (an agreed scenario was edited and became a new version, for instance). The history shows **the current version's Scenario ID**; move the test to that id. If the new version is still a draft, the reason says it is waiting for agreement
- The version is rejected / deprecated / archived
- The run was triggered by a pull request from a forked repository (so nobody outside can mark arbitrary scenarios as passing)
- The same run was delivered twice (a run is recorded only once)
- One of the run's `pmr-scenario-report*` artifacts is over the import limit (32 MB) (the whole run is skipped so that only the readable passes are not recorded; the history names the oversized artifact)

Editing an agreed scenario marks the old version superseded and the new one draft, so **CI recording pauses until the new version is approved**. After fixing the source of truth, align the test with the new version, switch the id and rerun; the next run is recorded.

### Re-import

After adding the permission, or when a webhook was missed, the same run can be imported again with `POST /api/github/workflow-runs/{run_id}/scenario-report?project_id=<project id>` (editor or above). A run that was already imported is not recorded twice. Re-import does not apply the webhook fork gate; not re-importing a fork run is the caller's responsibility.

## Standalone tasks (checking tasks that belong to no story)

This tab lists the tasks that are not linked to a user story and shows the results of checking them.

- You can filter by pass / fail and by your own check status.
- Select a task to open its content and its check results.

## Related bugs

When you select a scenario, 'Related bugs' appears on the right.

- 'File a bug' files a defect linked to that scenario.
- You can see the number of unresolved bugs, and each bug's severity, status and reporter.
- From 'Fix proposal' on each bug you can propose, approve and apply a fix ([Bugs and change management](https://pmonrails.com/en/docs/bugs-changes.md)).

## How it ties into development

Because 'the bar for completion (scenarios)', 'the actual test results' and 'the defects found' connect in the same place, you can see **how much is genuinely finished** in one continuous line from requirement to implementation.

---

https://pmonrails.com/en/docs/test-results