Skip to main content

Overview

StatusAggregator is a .NET 4.7.2 console application (run via NuGet.Jobs.JobRunner) that continuously polls an internal incident API, parses and aggregates incidents into a three-level hierarchy (Incident → IncidentGroup → Event), and serializes the resulting service health state to Azure Blob Storage as status.json. It also supports operator-driven manual overrides — adding, editing, or deleting events and messages — stored in Azure Table Storage. The job runs on a schedule and each execution:
  1. Processes manual status change entries from Azure Table Storage (primary and secondary accounts).
  2. Fetches new incidents from the incident API since the last cursor position.
  3. Updates all active events and their associated component states.
  4. Exports the full service component tree plus recent events to status.json.

Incident Ingestion

Pulls raw incidents from the internal NuGet incident API, authenticated via an X.509 certificate stored in Azure Key Vault.

Status Page Feed

Writes status.json to Azure Blob Storage, which is the data source for the public NuGet status page at status.nuget.org.

Manual Override Support

Operators can write ManualStatusChangeEntity records to Azure Table Storage to inject, edit, or remove events and messages without a code deploy.

Dual-Region Resilience

Reads and writes to both a primary and a secondary Azure Storage account, ensuring status data is kept consistent across regions.

Key Files and Classes

Dependencies

Internal Project References

Key NuGet / Framework Packages (resolved transitively)

Notable Patterns and Implementation Details

The job registers two named storage accounts (“Primary” and “Secondary”) in Autofac using Named<T> registrations. The primary account is registered last so it becomes the default unnamed binding. Both accounts get their own ManualStatusChangeCollectorProcessor, meaning manual overrides written to either account are processed.
Incidents are aggregated through a three-level hierarchy: raw IncidentEntity records are grouped into IncidentGroupEntity records (same component path), which are further grouped into EventEntity records. AggregationStrategy<TChild, TAgg> enforces that a child can only be linked to an aggregation that has existing children (preventing reuse of manually-created or broken aggregations) and that is still active if the child is active.
The cursor pattern (ICursor backed by Azure Table Storage) gives each collector a named timestamp watermark. The incident collector advances the cursor only on success. The StatusUpdater wraps incident ingestion in a try/catch so a transient API failure does not block the export step — the last successfully exported status remains live.
Certificate loading in Job.GetCertificateFromConfiguration supports two legacy formats: a JSON object with Base64 Data + Password (older Key Vault certificates) and a plain Base64 string (newer Key Vault certificates). If neither parses successfully the job will throw at startup. Ensure Key Vault secrets are in one of these two forms.
The target framework is net472. All Azure SDK calls use async/await but the entry point calls .GetAwaiter().GetResult() — this is intentional for the JobRunner pattern used across NuGet background jobs. Do not migrate to net6+ without verifying JobRunner compatibility.
Configuration delays (EventStartMessageDelayMinutes, EventEndDelayMinutes) default to 15 minutes each. EventVisibilityPeriodDays defaults to 10 days. These can be tuned in the StatusAggregator config section without a code change, which is useful during incidents when faster message cadence is desired.

Data Flow Summary