TL;DR: How to run ALPA Mobile's Appium/NUnit UI test suites —./run-ui-tests.shfor everything, or per-platform viadotnet test UITests.iOS/UITests.iOS.csproj/UITests.Android— with logs landing intest-logs/. Also covers the beta testing groups: TestFlight join link for iOS, Google Play Internal Testing for Android, and how testers report issues.
This guide consolidates app and UI test workflows for ALPA Mobile.
UnitTest/: Unit and architecture tests (no device required)UITests.Shared/: Shared Appium and NUnit test logicUITests.iOS/: iOS test projectUITests.Android/: Android test projectDocsNavTests/: Browser click-through tests for the docs site navigationUnitTest/UnitTest.csproj (~92 test files) runs without a simulator or device:
dotnet test UnitTest/UnitTest.csprojAlongside the service/ViewModel unit tests, UnitTest/Architecture/ holds repo-wide integrity suites:
LayerDependencyRulesTests — enforces the n-tier dependency direction (Domain ← Application ← Infrastructure/Presentation)DocsNavIntegrityTests — validates the generated docs nav blocks against docs/build-nav.csMojibakeTests — catches encoding corruption in docs and sourceOrphanedCssTests / UndefinedCssClassTests — CSS classes defined-but-unused / used-but-undefined (baselines in UnitTest/Architecture/)RazorComponentResolutionTests — every referenced Razor component resolvesDocsNavTests/NavClickThroughTests.cs complements the static nav checks by clicking through the docs navigation in a real browser.
uiautomator2 for Android, xcuitest for iOSInstall Appium and drivers:
npm install -g appium
appium driver install uiautomator2
appium driver install xcuitestFor Android, ensure ANDROID_HOME is configured.
Use the repository test runner script:
./run-ui-tests.shCommon variants:
# iOS only
./run-ui-tests.sh ios
# Android only
./run-ui-tests.sh android
# Verbose output
./run-ui-tests.sh --verbose
# Reuse existing Appium server
./run-ui-tests.sh --no-appium
# Record the iOS simulator during the run (video saved to test-logs/)
./run-ui-tests.sh ios --recordCredentials prerequisite: login-dependent tests need member credentials. Set
ALPA_UITEST_VAULT and the runner hydrates every stored member from Key Vault — see
UI-test credentials below for storing, updating and validating them:
ALPA_UITEST_VAULT=ALPAMobile-UITEST-VAULT ./run-ui-tests.sh iosThe team vault is ALPAMobile-UITEST-VAULT (resource group ALPA-DevOps, RBAC
authorization; you need Key Vault Secrets User to read, Secrets Officer to store). One
secret pair per role, and for members the role is the environment plus the MEC:
uitest-member-<env>-<mec>-username
uitest-member-<env>-<mec>-password env = prod | qa mec = aca, ajt, ala, …Names are case-insensitive in Key Vault and the hydration lower-cases the role, so
…-prod-CJT-… and …-prod-cjt-… are the same secret. Tags are inventory only —
lookup is always by name. Secret values are set by the project lead and are never committed, logged or echoed.
Replace the three placeholders; the same command run again creates a new version of the secret, which the next hydration reads:
MEC=aca; USER='<member number>'; PASS='<password>'; \
az keyvault secret set --vault-name ALPAMobile-UITEST-VAULT --name uitest-member-prod-$MEC-username --value "$USER" --tags role=member env=prod mec=$MEC purpose=uitest -o none && \
az keyvault secret set --vault-name ALPAMobile-UITEST-VAULT --name uitest-member-prod-$MEC-password --value "$PASS" --tags role=member env=prod mec=$MEC purpose=uitest -o none && \
echo "prod-$MEC stored"MEC=aca; PASS='<new password>'; \
az keyvault secret set --vault-name ALPAMobile-UITEST-VAULT --name uitest-member-prod-$MEC-password --value "$PASS" -o none && echo "prod-$MEC password updated"Tag the username secret with known-failure=<reason>. The API check reports it as
SKIP with the reason and the hydration leaves it out, so the Appium smoke never attempts it (repeated failed
attempts lock the account). set-attributes replaces the whole tag set, so repeat the inventory tags:
MEC=ati; WHY='account rejected by auth API 2026-08-22, AB#…'; \
az keyvault secret set-attributes --vault-name ALPAMobile-UITEST-VAULT --name uitest-member-prod-$MEC-username --tags known-failure="$WHY" role=member env=prod mec=$MEC purpose=uitest -o none && echo "prod-$MEC marked known failure"Clear it the same way without the known-failure tag once the account works again.
az keyvault secret list --vault-name ALPAMobile-UITEST-VAULT --query "[].{name:name,tags:tags}" -o tableMemberCredentialSmokeTests discovers each hydrated member, signs in through the real member log-in
screen, confirms the session on Auth Status and signs out; a failure names the role and quotes the screen's
rejection message (“Invalid username or password…”), so you know which secret to fix. Run it
alone, against a simulator you own:
export ALPA_UITEST_VAULT=ALPAMobile-UITEST-VAULT && . scripts/uitest-credentials.sh && \
APPIUM_UDID=<simulator udid> dotnet test UITests.iOS/UITests.iOS.csproj --filter "FullyQualifiedName~MemberCredentialSmokeTests"Source the script directly as shown — piping it (. script | grep …) runs it in a subshell and
the exported variables never reach the test host. To check a single MEC after fixing it:
dotnet test UITests.iOS/UITests.iOS.csproj --filter "Name~member-prod-aca"scripts/auth-credential-check.sh discovers every uitest-member-<env>-<mec>
pair in the vault and POSTs the app’s own log-in request (/api/token/auth, id canonicalised like
AlpaIdHelper) to that environment’s auth host — prod authsvc2.alpa.org, qa
authapi.alpa.org (override with AUTH_URL_PROD / AUTH_URL_QA). It prints a
per-MEC table, writes JUnit, and exits 1 if any pair fails. Newly ingested MECs are checked automatically.
ENVS=prod,qa bash scripts/auth-credential-check.shThe same script backs the on-demand pipeline ALPA Mobile UI-test Credential Check
(AzureDevOps/azdo_credential_check.yml; parameters: service connection, environments, vault). Run it
from Pipelines → Run pipeline after ingesting or fixing credentials; failures appear on the run’s Tests
tab named member-<env>-<mec>. One-time prerequisite: an Azure Resource Manager service
connection to subscription AzureSimple ALPA named ALPA-Mobile-UITest-Vault (workload identity
federation), with its identity granted Key Vault Secrets User on the vault.
scripts/gateway-readiness-check.sh runs every gateway route the app calls
(scripts/gateway-readiness/endpoints.json — the 29 RestService routes, the FTDT / Jumpseat /
PilotComms / MobileMenu NSwag routes, the theme route and the hardcoded dynamic-content host) against
gateway.alpa.org (baseline) and gatewayapiqa.alpa.org, as the anonymous identity plus one stored
member per MEC (default DAL, UAL, FDX), and reports per cell whether status, schema and — for config-like routes
— values match production. Reads always run; the eight server-mutating routes run only as named round-trip
flows under WRITES=, against WRITE_HOSTS (default prod,qa — the vault members are test accounts). Procedure,
write-flow side effects and the baseline findings: GATEWAY-QA-READINESS-CHECK.md.
HOSTS=prod,qa,dev ./scripts/gateway-readiness-check.sh # reads, 4 identities, 3 hosts
WRITES=log,favorites,subscription,push,ftdt ./scripts/gateway-readiness-check.sh # + write flows on prod and qa
python3 scripts/gateway-readiness-check.py --catalog-check # catalog covers Postman + RestService.cs?Pipeline: ALPA Mobile Gateway Readiness Check (AzureDevOps/azdo_gateway_readiness.yml, on demand;
parameters: hosts, MECs, write flows, write hosts, fail-on verdicts). Results land on the run’s Tests tab as
<route>[<mec>] prod-vs-<host> and the full report, summary JSON and raw responses are published as
the gateway-readiness artifact.
DrawerMecContextTests signs in as each MEC’s vault member, opens the ☰ drawer and asserts the
MY MEC hero reads <MEC> MEC, links to the MEC tab route and agrees with the tab bar’s MEC slot;
a last case checks the hero is withheld signed out. Default MECs are dal,ual,fdx; a MEC without a
stored credential skips by name. For a one-off:
ALPA_UITEST_DRAWER_MECS=aca dotnet test UITests.iOS/UITests.iOS.csproj --filter "FullyQualifiedName~DrawerMecContextTests"Every hydrated pair becomes ALPA_UITEST_<ROLE>_USERNAME/PASSWORD (role upper-cased, dashes
to underscores). The plain ALPA_UITEST_USERNAME/PASSWORD alias points at
ALPA_UITEST_DEFAULT_ROLE (member-prod-aca unless overridden), so tests that do not care
which member they are keep working unchanged. A test that needs a particular MEC asks for it:
if (CredentialsFor("member-prod-cjt") is not { } creds)
Assert.Ignore(MissingCredentialsMessage("member-prod-cjt"));# Android
dotnet test UITests.Android/UITests.Android.csproj
# iOS
dotnet test UITests.iOS/UITests.iOS.csprojUITests.Android/PushSubscriptionIntegrationTests.cs validates the Flight Finder push pipeline
end to end against the real notification hub: bell tap → per-leg fs- tag on the
device's hub installation → a targeted send reaches this device → tapping the OS-shade banner opens the
flight the alert is about → unsubscribe cleans up. It is the only automated coverage of what a tapped
push actually does, because no amount of tapping inside the app reproduces it.
It is [Explicit], so ./run-ui-tests.sh never runs it. Invoke it deliberately:
dotnet test UITests.Android/UITests.Android.csproj \
--filter FullyQualifiedName~PushSubscriptionIntegrationProduction-safe by construction: sends are constrained to
(fs-tag && $InstallationId:{this device}), so no real member's device can match. Never send to a
bare fs- tag — real members subscribe to the same flights.
Prerequisites that have actually blocked runs (the fixture's own remarks carry the detail):
ALPA_UITEST_VAULT as above.ALPA_FSNH_KEY or a logged-in az CLI. Missing
credentials skip rather than fail.correlationKey case needs a device whose alert feed already holds real
alerts, since it borrows a real alert id — a synthetic one proves nothing. On a device with no
subscriptions it reports inconclusive, and it must be run on its own (a populated feed is the opposite
of the subscribe step's clean-slate precondition).-t:Install can fail with InsufficientSpaceException even with
~1 GB free; adb install -r of the -p:EmbedAssembliesIntoApk=true APK works. Do not
uninstall to free space unless you mean to re-do the login it takes with it.files/.__override__ present
(fast-deploy assemblies shadow an APK on Debug) and dumpsys package's
lastUpdateTime matching the APK.UITests.Shared/BlazorWalkthroughTests.cs automates the framework-validatable checks of the
tester guide walkthroughs — the Page Readiness index, the member login screen's
control inventory and rejection messages, the non-member member-credential prompt, promo banner render/dismiss,
and the logged-out Favorites redirect. Checks needing real credentials or an app restart stay manual (see the
walkthrough docs).
# Run only the Blazor walkthrough suite (iOS)
dotnet test UITests.iOS/UITests.iOS.csproj --filter "FullyQualifiedName~BlazorWalkthroughTests"Blazor DOM elements carry id attributes following pagename.semanticIdN —
a page prefix, a name descriptive of the control, and a number that increments through the page so a failing id
pinpoints where on the page the break happened. Examples: login.memberNumberField2,
login.membersLoginButton4, ftdtdash.addDutyPeriodButton9,
authstatus.logOutButton2. Dynamic collections use a stable discriminator in place of the number
(e.g. the readiness rows: devdash.row-login-non-members from the route). Shared components take an
optional Id parameter (ActionButton) so pages can label their instance.
How the tests reach the DOM: XCUITest drives the native shell (the Blazor Dev tile's AutomationId is the literal
AutomationId.Blazor\nDev — HomePage.xaml binds AutomationId.{Title} and the title is two
lines), then switches to Appium's WEBVIEW context and selects by CSS attribute
([id='login.membersLoginButton4'] — the dots rule out #id selectors). The WEBVIEW context
only exists on debug builds: iOS 16.4+ hides WKWebViews from the remote inspector unless
WKWebView.Inspectable is set, which BlazorDevPage does under #if DEBUG.
On release builds the suite self-skips via Assert.Ignore. Fills go through the native value setter +
input/change events (BaseTest.FillWeb) because raw key events can race
Blazor's @bind. Android: WEBVIEW context attachment works as of 2026-08-12 — the session requests
chromedriverAutodownload (UITests.Android/AppiumSetup.cs) and
AppiumServerHelper starts the server with
--allow-insecure uiautomator2:chromedriver_autodownload so a chromedriver matching the device's
WebView is fetched on first use. Caveat: chromedriver binds the context to the first page in the app
process, so a long-lived process with leaked hidden webview pages (e.g. after push-tap deep links) polls a dead
page forever — force-stop and relaunch the app for a deterministic single-page state
(PushSubscriptionIntegrationTests does this in its setup).
UITests.Android/PushSubscriptionIntegrationTests.cs validates the Flight Finder notify-me pipeline
end to end against the real notification hub: bell tap in the Blazor UI → per-leg
fs- tag on the device's hub installation (asserted server-side) → a hub template send targeted at
that tag delivers a device notification → delete unsubscribes and the tag disappears from the hub. It is
[Explicit] — excluded from every normal run — and meant as an occasional validation step:
dotnet test UITests.Android/UITests.Android.csproj --filter "FullyQualifiedName~PushSubscriptionIntegration"Prerequisites: a booted emulator with Play services, an authenticated app session,
notification permission granted, and hub send credentials — export ALPA_FSNH_KEY
(the hub's DefaultFullSharedAccessSignature primary key) or just be logged into Azure CLI and the
test fetches it itself. Missing credentials skip the test rather than fail it.
Production-safety contract: per the AB#1892 scope decision the test runs against
FlightSearchProduction (override with ALPA_FSNH_HUB), so every synthetic send is
constrained to (fs-tag && $InstallationId:{test device}) — it can never match a real member's
device. Never loosen that expression to a bare fs- tag: real members may be subscribed to the same
flight. Sends target tomorrow's ATL→DEN nonstops (a same-day search legitimately returns zero flights late in
the evening).
The script writes logs under test-logs/:
appium.logios-tests.logandroid-tests.logPlatform test runs also emit TRX results in their respective output locations.
Join URL: https://testflight.apple.com/join/bjhOj9We
Join URL: https://play.google.com/apps/testing/com.accella.alpa