Azure Notification Hub Integration Testing Strategy

Updated: 2026-07-07 15:49 ET · Audited: 2026-06-26 · Version: 1.0 · Draft date: 2026-02-20 · Status: AB#1892 — Backlog

TL;DR: Proposed testing strategy for the Azure Notification Hub push integration (AB#1892 — Backlog, not yet implemented). Basic push already works via Azure NH; the gap is that no comprehensive integration-testing or validation framework exists. The plan covers device registration (APNS/FCM), tag-based targeting, templates, error handling, performance/scale targets, and monitoring, phased over a 7-week roadmap. The payload contract itself lives in the push notification payload reference.

Comprehensive testing strategy for Azure Notification Hub (ANH) integration in ALPA Mobile (.NET MAUI 10). Covers device registration, tag-based targeting, template notifications, error handling, performance testing, and monitoring across iOS (APNS) and Android (FCM) platforms.

Current State: Basic push notifications functional via Azure NH
Gap: No comprehensive integration testing or validation framework
Goal: Establish robust testing coverage to ensure reliable push notification delivery

1. Testing Strategy

1.1 Device Registration Testing

iOS (APNS) Registration Test Cases

• REG-iOS-001: Fresh install — verify APNS token, Azure NH registration, installation ID, tag assignment
• REG-iOS-002: App reinstall — verify registration cleanup, new registration, tag reassignment
• REG-iOS-003: Token refresh — simulate APNS token rotation, verify Azure NH update, no duplicates
• REG-iOS-004: Permission states — test denied→allowed and allowed→denied flows

Android (FCM) Registration Test Cases

• REG-Android-001: Fresh install registration — FCM token, Azure NH registration, installation ID
• REG-Android-002: App reinstall — registration cleanup, new registration, tag reassignment
• REG-Android-003: Token refresh — simulate FCM token rotation, verify Azure NH update
• REG-Android-004: Multiple device scenarios — same user on multiple Android devices, independent registrations

Cross-Platform Registration

• REG-Cross-001: Multi-device user — same user on iOS and Android, independent registrations, cross-platform tag targeting
• REG-Cross-002: Registration data consistency — Azure NH stores platform-specific data, installation ID uniqueness

1.2 Tag-Based Targeting

User Segmentation Tags

• TAG-Segment-001: Member status tags — member:active, member:inactive, member:retired
• TAG-Segment-002: Base tags — base:ORD, base:ATL — multi-base OR logic, base change updates tags
• TAG-Segment-003: Role-based tags — role:captain, role:fo, role:committee

Tag Expressions

• TAG-Expr-001: Simple expressions — single tag, OR (base:ORD || base:DEN), AND (member:active && role:captain)
• TAG-Expr-002: Complex expressions — nested logic, NOT, combinations
• TAG-Expr-003: Tag limit validation — Azure NH 20-tag limit per device, error handling for exceeding limits

1.3 Template Notifications

• TPL-Structure: Platform-specific templates, personalization ({{firstName}}, {{baseCode}}), multi-language templates
• TPL-Payload: Custom data (deep link URLs, action identifiers), payload size limits (iOS 4KB, Android 4KB), rich notifications

1.4 Scheduled and Delayed Notifications

• SCH-Schedule: Future scheduled send (±1 min tolerance), timezone handling, cancellation
• SCH-Delay: Device offline scenarios, Azure NH queuing and TTL, background app state

1.5 Multi-Platform Delivery Verification

• MPD-Consistency: Same notification across both platforms within 5-second window
• MPD-Confirm: Azure NH delivery status API, client-side telemetry, failed delivery tracking

1.6 Error Handling

• ERR-Token: Expired APNS token (410 Gone), invalid FCM token, token feedback loop
• ERR-Network: Registration during offline, Azure NH outage (retry logic), partial delivery failure
• ERR-Perm: Notification permission denied, permission revoked mid-session, iOS provisional permission

1.7 Performance and Scale Testing

1.8 Azure NH SDK Integration Tests

• SDK-Install: NuGet package installation, platform-specific setup (iOS AppDelegate, Android MainActivity + FirebaseMessagingService)
• SDK-API: Registration, tag management, unregistration APIs
• SDK-Telemetry: Event tracking, Sentry integration

1.9 Monitoring and Diagnostics

Azure Portal alerts configured for: failed notifications >5%, registration errors >10/hour, service unavailability, quota usage >80%.

Custom Sentry breadcrumbs trace the full push flow: registration → token → Azure NH → delivery.

2. Implementation Roadmap

CI/CD Integration

# Pseudo-pipeline config
stages:
  - stage: UnitTest
    # Run on every PR
    - dotnet test **/ALPA.Mobile.Tests.Unit.csproj --filter Category=Push
  - stage: IntegrationTest
    condition: develop branch merges
    - dotnet test **/ALPA.Mobile.Tests.Integration.csproj --filter Category=AzureNH
  - stage: E2ETest
    condition: main branch
    pool: macOS-latest (for iOS)
    - npm run test:e2e:push

3. Tools and Setup

3.1 Azure NH REST API

Base URL: https://<namespace>.servicebus.windows.net/<notificationHub>

Key endpoints: Create/Update Registration (PUT /registrations/{id}), Query Registrations (GET /registrations), Send Notification (POST /messages), Get Outcome (GET /messages/{id}).

3.2 Test Harness Requirements

• iOS: 2 physical devices (iPhone iOS 16+, one iPad)
• Android: 2 physical devices (different manufacturers, Android 10+)
• Staging Azure NH namespace: alpa-mobile-nh-staging — isolated from production
• Backend Staging API: https://staging-api.alpa.org with test database

3.3 Mock Setup (Unit Tests)

public interface IAzureNotificationHubClient
{
    Task<RegistrationDescription> CreateOrUpdateRegistrationAsync(
        string deviceToken, IEnumerable<string> tags, CancellationToken ct = default);
    Task DeleteRegistrationAsync(string registrationId, CancellationToken ct = default);
    Task<NotificationOutcome> SendNotificationAsync(
        Notification notification, string tagExpression, CancellationToken ct = default);
}

3.4 Daily Smoke Test Script

Automated script runs daily at 6 AM UTC: registers test device → sends test notification → verifies outcome → cleans up registration. Runtime ~5 minutes.

4. Test Data Management

Test user accounts for staging environment (captain at ORD, FO at DEN, retired at ATL). Test device inventory tracked in spreadsheet (device ID, platform, OS version, registration ID, tags, last tested). Pre-defined notification templates: basic-alert, personalized-alert (with {{firstName}}, {{baseCode}}), deep-link.

5. Success Criteria

6. Risk Assessment

7. Appendix: Azure NH Configuration Checklist

• Staging and production namespaces created
• APNS certificate uploaded (sandbox + production)
• FCM API key configured
• Shared Access Policies configured (DefaultFullSharedAccessSignature, DefaultListenSharedAccessSignature)
• Application Insights connected
• Alert rules configured
• Cost management alerts set (quota usage)

Useful Azure NH CLI Commands

# List registrations for a specific tag
az notification-hub notification list \
  --resource-group alpa-mobile-rg \
  --namespace-name alpa-mobile-nh-staging \
  --notification-hub-name alpa-mobile-hub-staging \
  --filter "tags contains 'base:ORD'"

# Send test notification via CLI
az notification-hub notification send \
  --resource-group alpa-mobile-rg \
  --namespace-name alpa-mobile-nh-staging \
  --notification-hub-name alpa-mobile-hub-staging \
  --payload '{"aps":{"alert":"CLI test"}}' \
  --apple

Glossary

• APNS — Apple Push Notification Service
• FCM — Firebase Cloud Messaging (Google's push service)
• ANH — Azure Notification Hub
• PNS — Platform Notification Service (generic term for APNS/FCM)
• Device Token — Unique identifier for a device issued by APNS/FCM
• Registration ID — Azure NH's identifier for a device registration
• Tag — Metadata label for targeting notifications (e.g., base:ORD)
• Template — Notification payload format with variable substitution
• TTL — Time-to-Live (how long Azure NH queues a notification for offline devices)

Related Documentation

• Push notification admin console
• Push notification payload reference
• Engineering overview