# CI: PR Validation on `qa/*` and `main`

How pull requests into `qa/*` branches (and docs-only PRs into `main`) are validated (they ran on `imp/blazor-hybrid` until the
branching strategy changed on 2026-10-04; `dev` has no PR validation by design), why the trigger lives in a branch
policy rather than in the pipeline YAML, and how to run the checks by hand.

Last verified: 2026-10-05 (policy scopes read from Azure DevOps).

## The three validators

| Pipeline | Id | Agent | What it checks |
| --- | --- | --- | --- |
| ALPA Mobile PR Validation - Android | 98 | `windows-2022` | `dotnet build -f net10.0-android -c Debug` (compile only, no signing), then the unit + architecture suite with `--filter "TestCategory!=Docs"` |
| ALPA Mobile PR Validation - iOS | 99 | `macOS-15`, Xcode 26.3 | `dotnet build -f net10.0-ios -c Debug -p:EnableCodeSigning=false`, then the same filtered test run |
| ALPA Mobile PR Validation - Docs | 100 | `ubuntu-latest` | `DocsNavIntegrityTests` (`TestCategory=Docs`), `dotnet run docs/build-nav.cs -- --check`, and `markdownlint-cli2 "docs/**/*.md"` |

The Android and iOS runs exclude the `Docs` category on purpose. Those tests assert
documentation state — no app change can break them and no build depends on them — but while
they lived in the app runs they redded app PRs over a stale nav in a doc the PR never
touched (twice on 2026-08-14), and each was answered with a SKIP bypass. Bypasses are how a
real failure eventually gets waved through, so the docs checks were split into pipeline 100.
A PR touching code *and* docs trips both path filters and runs both.

`docs/validate-doc-timestamps.sh` is deliberately **not** a PR gate. It asserts a stamp
matches *today*, which is only meaningful at the moment of the edit — a PR re-validated the
next day carries yesterday's stamp and is not wrong. It stays a pre-commit hook.

## Why the trigger is a branch policy, not `pr:` in the YAML

All three YAML files declare:

```yaml
trigger: none
pr:
  branches:
    include: [ qa/*, main ]
  paths:
    include: [ ... ]
```

**Azure Repos Git ignores the `pr:` block.** It is a GitHub-only trigger. On Azure Repos, PR
validation fires only from a **Build Validation branch policy**. With `trigger: none` and an
ignored `pr:`, these pipelines had no trigger at all and only ever ran when queued by hand.

That was the state from their creation (2026-08-14) until 2026-08-22: every PR into
the then integration branch, `imp/blazor-hybrid`, merged with no compile check, no unit tests and no nav validation, gated
only by "Minimum number of reviewers". Pipeline 100 had never executed once.

Two consequences of the trigger living in the policy:

- The `paths:` filter in the YAML is ignored along with `pr:`. Path scoping has to be
  expressed as `--path-filter` on the policy, or every PR runs every validator.
- Editing the YAML cannot change when a pipeline runs. Only the policy can.

## The policies

Created 2026-08-22 on `imp/blazor-hybrid` and moved to the `qa/` folder prefix on 2026-10-04 (ids
below are the `qa/` policies; see the [Branching Strategy](detail/BRANCHING-STRATEGY.html)), repository id from `az repos show --repository "ALPA Mobile" --query id -o tsv` (shown as `$REPO_ID` below):

| Policy | Id | Build definition | Blocking | Path filter |
| --- | --- | --- | --- | --- |
| PR Validation - Android | 22 | 98 | yes | `/ALPAMobile`, `/ALPAMobile.Domain`, `/ALPAMobile.Application`, `/ALPAMobile.Infrastructure`, `/ALPAMobile.Presentation`, `/UnitTest` |
| PR Validation - iOS | 23 | 99 | yes | same as Android |
| PR Validation - Docs | 24 | 100 | yes | `/docs`, `/.githooks` |
| PR Validation - Docs (on `main`) | 34 | 100 | yes | `/docs`, `/.githooks` |

The path filters are what actually implements the docs/code split. Without them all three
validators run on every PR — a docs-only PR queues a Windows and a macOS agent to install
the MAUI workload and build the app, which is the delay the split exists to remove. This was
observed on PR 2160 (docs-only) before the filters were added: all three queued.

The `main` row is the same docs check scoped to docs-only fixes that go straight into `main` (the docs site is
published from `main`); code changes reach `main` only by a release merge, which the `qa/` builds have already
validated. On `qa/` all three are **blocking**: a PR into a `qa/*` branch cannot complete until they pass. (On
`imp/blazor-hybrid` they were optional while the two defects under "Known defects" were open; those
policies, ids 18 to 20, remain on the retired branch.) A validator that blocks on a known-bad check
trains everyone to bypass it, so keep both defects fixed.

To recreate one:

```bash
az repos policy build create \
  --org https://dev.azure.com/airlinepilotsassociation --project "ALPA Mobile" \
  --repository-id "$REPO_ID" \
  --branch qa/ --branch-match-type prefix \
  --build-definition-id 98 --display-name "PR Validation - Android" \
  --blocking false --enabled true \
  --queue-on-source-update-only true --manual-queue-only false --valid-duration 0 \
  --path-filter "/ALPAMobile;/ALPAMobile.Domain;/ALPAMobile.Application;/ALPAMobile.Infrastructure;/ALPAMobile.Presentation;/UnitTest"
```

Settings worth understanding:

- `--queue-on-source-update-only true` — re-runs only when the PR head moves, not when a
  comment is added or the target branch advances. Without it, every comment on a PR costs
  three CI runs.
- `--valid-duration 0` — the result never expires, so a stale green cannot age into a gate.
- `--blocking false` — optional. Change to `true` to require it.

To make one blocking later:

```bash
az repos policy build update --org "$ORG" --project "ALPA Mobile" \
  --policy-id <id> --blocking true --enabled true \
  --repository-id "$REPO_ID" \
  --branch qa/ --branch-match-type prefix --build-definition-id <98|99|100> \
  --queue-on-source-update-only true --manual-queue-only false --valid-duration 0
```

## Running the validators by hand

The policies fire on PRs. To validate an arbitrary branch or the current tip:

```bash
ORG=https://dev.azure.com/airlinepilotsassociation
for id in 98 99 100; do
  az pipelines run --org "$ORG" --project "ALPA Mobile" \
    --id "$id" --branch dev -o json | jq -r '"queued run \(.id) — \(.definition.name)"'
done
```

Watch them:

```bash
az pipelines runs list --org "$ORG" --project "ALPA Mobile" \
  --pipeline-ids 98 99 100 --top 6 \
  --query "[].{run:id,name:definition.name,status:status,result:result}" -o table
```

Read why one failed — the failing task's log, not the summary:

```bash
TOKEN=$(az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 \
        --query accessToken -o tsv)
API=https://dev.azure.com/airlinepilotsassociation/ALPA%20Mobile/_apis
BUILD=<run id>
curl -sf -H "Authorization: Bearer $TOKEN" "$API/build/builds/$BUILD/timeline?api-version=7.1" \
  | jq -r '.records[] | select(.result=="failed" and .type=="Task") | "\(.name) -> log \(.log.id)"'
curl -sf -H "Authorization: Bearer $TOKEN" "$API/build/builds/$BUILD/logs/<log id>?api-version=7.1" \
  | grep -iE "Failed |error" | head
```

Queueing the same pipeline twice starts two runs; cancel the extra rather than leave it to
occupy an agent:

```bash
curl -s -X PATCH -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"status":"cancelling"}' "$API/build/builds/<run id>?api-version=7.1"
```

## Last full validation

Run by hand on 2026-08-22 against `b796f55e`, before the policies existed:

| Pipeline | Run | Result |
| --- | --- | --- |
| PR Validation - Android | 4649 | succeeded |
| PR Validation - iOS | 4650 | succeeded |
| PR Validation - Docs | 4651 | **failed** |

## Known defects

### 1. Pipeline 100 (Docs) could not pass — fixed 2026-08-22

Run 4651 — its first execution ever — failed before running a single test:

```text
error NETSDK1147: To build this project, the following workloads must be
installed: maui-tizen  [.../ALPAMobile.csproj]
Dotnet command failed ... : [ '.../UnitTest/UnitTest.csproj' ]
```

`UnitTest.csproj` references `ALPAMobile.csproj`, which needs the MAUI workload. The docs
pipeline installs no workload by design, so it could not compile the test project it was
trying to filter.

**Fix:** the documentation tests moved to a standalone `DocsTests` project with **no
`ProjectReference` entries**. They only ever used BCL file and regex APIs, so nothing was
lost. The pipeline now runs `dotnet test DocsTests/DocsTests.csproj` and needs only the .NET
SDK.

Two things to keep true, or this breaks again in exactly the same way:

- **Never add a `ProjectReference` to `DocsTests.csproj`.** One reference to an app project
  reintroduces the workload dependency.
- **Never point the docs pipeline back at `UnitTest/UnitTest.csproj`.**

The Android and iOS validators keep their `--filter "TestCategory!=Docs"`. It is a no-op now
that the file has moved, and is left deliberately: if a `Docs`-category test is ever added to
`UnitTest`, it stays out of the app runs.

The pre-commit hook's docs step (5b) was missed by the move: until 2026-10-15 it still ran
`--filter "TestCategory=Docs"` against `UnitTest/UnitTest.csproj`, which matched no test,
exited 0 and printed "Docs nav tests passed" on every docs commit. It now runs
`DocsTests/DocsTests.csproj` with `--configuration Release` and no filter, as this pipeline
does, and fails the commit when the `.trx` records zero tests.

### 1b. markdownlint debt — cleared 2026-10-05

**Resolved:** on 2026-10-05 `markdownlint-cli2 "docs/**/*.md"` reported 0 issues in 47 files, so
pipeline 100 no longer fails on lint. The history below explains why the debt mattered.

With the workload blocker gone, pipeline 100 gets far enough to run all three of its steps.
Two now pass — `DocsTests` (4/4) and `build-nav --check` — and the third does not:
`markdownlint-cli2 "docs/**/*.md"` reports **90 violations across 9 files**, all pre-existing
and none of them in files this change touches.

| File | Violations |
| --- | --- |
| `docs/component-specifications/dynamic-feed/ComponentView-payload-map.md` | 29 |
| `docs/FTDT/ground-duty-nonfdp-refactor-design.md` | 23 |
| `docs/FTDT-DATA-MIGRATION-PLAN.md` | 15 |
| `docs/FTDT/domain-model-validation-plan.md` | 6 |
| `docs/FTDT-POCO-SPLIT-PLAN.md` | 5 |
| four more FTDT handoff/design docs | 12 |

Roughly 72 are mechanically auto-fixable (`markdownlint-cli2 --fix` — blank lines around
headings/lists/fences, emphasis style, ordered-list prefixes). The remaining ~18 need a
judgement call: untagged code fences, one duplicate heading, one malformed table.

The debt survived this long because the pre-commit hook lints **staged files only**, so a
file that was already non-compliant never got re-checked.

**Related config bug:** `.markdownlint-cli2.yaml` sets `MD013: {line_length: 120}`,
`MD033: false` and `MD036: false`, but a root `.markdownlint.json` also exists and takes
precedence per directory. It sets only `MD013: false` and `MD060: false`, so the yaml's
`MD033`/`MD036`/`MD029` settings are dead text and those rules fire anyway. Aligning the two
files removes 3 of the 90 on its own.

This is deliberately **not** fixed here: rewriting nine unrelated design documents inside a
pipeline fix is not a reviewable change. The docs policy was non-blocking then, so the pipeline
reported real debt at no cost until it was addressed.

### 2. `Reopening_IsNotCollapsedByThePreviousCountdown` is flaky

`UnitTest/WebViewClusterAutoCollapseTests.cs` waits 90 ms against a 120 ms countdown — a
30 ms margin on a shared CI agent:

```csharp
var vm = Make(120);
vm.IsClusterExpanded = true;
await Task.Delay(60);
vm.IsClusterExpanded = false;
vm.IsClusterExpanded = true;   // restarts the countdown
await Task.Delay(90);
Assert.That(vm.IsClusterExpanded, Is.True);
```

It failed `ALPA-Mobile-Packages` three times on 2026-08-22 (runs 4638, 4641, 4645) and passed
on 4646 — flaky, not fixed. The passing run merely added new tests; nothing repaired this one.
Because `ALPA-Mobile-Packages` gates on "only green code packs", this intermittently blocks
package publishing to `ALPASharedLibrary-Feed`.

Windows timer granularity is roughly 15.6 ms, so the Android validator (a Windows agent) has
the least headroom. The fix is to wait on the state change rather than race a wall clock, or
to widen the margin substantially.

## What is still not gated

`dev` has no build validation policy by design: the automated PR checks were a bottleneck
on integration, so they run on promotion PRs into `qa/*` and the developer who merges into
`dev` builds the merged tip. Everything above applies to `qa/*` only. The retired
`imp/blazor-hybrid` and `DotNet10` branches are not validated.

---

Tracking: AB#2548