Skip to main content

​
Overview

NuGet.Services.Revalidate is a .NET Framework 4.7.2 console application (run as an Azure WebJob) responsible for enqueuing existing NuGet packages into the validation pipeline so they can receive repository signatures that were not applied at original publish time. This was necessary because repository signing was introduced after many packages already existed in the gallery; this job backfills the signing validation for all of them. The job operates in three sequential phases that must be completed in order. The first phase (Build Preinstalled Packages) is a one-time developer task that scans local Visual Studio and .NET SDK installation directories to produce an embedded JSON manifest of preinstalled package IDs. The second phase (Initialization) populates the PackageRevalidations database table with an ordered list of all packages requiring revalidation, grouped and prioritized by importance. The third phase (Revalidation) continuously dequeues batches from that table and sends Service Bus messages to the validation pipeline, dynamically throttling its rate to stay within a configurable event budget shared with live gallery traffic. A key design goal is to never destabilize the NuGet ingestion pipeline. The throttler computes a real-time quota by querying the Application Insights REST API for the count of push, list, and unlist events in the past hour and subtracting that from a dynamically increasing desired rate ceiling. If the pipeline status blob in Azure Blob Storage shows any component as degraded, the job pauses and resets its desired rate back to the configured minimum. The desired rate increases incrementally with each successful batch and is clamped between MinPackageEventRate and MaxPackageEventRate to prevent both starvation and overload.

​
Role in System

Priority-Ordered Initialization

Packages are categorized into four priority sets — Microsoft-owned, preinstalled by VS/.NET SDK, transitive dependencies of those sets, and all remaining packages — and inserted into the queue in descending download-count order within each set.

Pipeline-Aware Throttling

Before each batch the job calculates a revalidation quota as DesiredRate - RecentGalleryEvents - RecentRevalidations. If quota is exhausted or the pipeline status component is not Up, the batch is deferred and the desired rate is reset to its minimum.

Adaptive Rate Control

The desired package event rate starts at MinPackageEventRate and increases by MaxBatchSize per successful iteration up to MaxPackageEventRate. An unhealthy pipeline resets it to the minimum, preventing runaway throughput after an outage clears.

Lazy Skip Logic

Packages that are already repository-signed or are no longer available (hard-deleted or status Deleted) are detected at dequeue time and marked completed without sending a Service Bus message, avoiding wasteful validation work.

​
Key Files and Classes

​
Dependencies

​
NuGet Package References

The project has no explicit <PackageReference> entries in its csproj; all NuGet dependencies flow through the three internal project references below (which in turn bring in Autofac, Microsoft.Extensions.*, Azure Service Bus, Application Insights, Entity Framework, and the NuGet Jobs framework).

​
Internal Project References

​
Notable Patterns and Implementation Details

The MaxPackageCreationDate cutoff in InitializationConfiguration is central to correctness. Only packages with a Created timestamp strictly before this date are included in revalidation, because packages published after repository signing was enabled already have the correct signatures and do not need retroactive validation.
Initialization uses SqlBulkCopy via PackageRevalidationInserter rather than EF SaveChanges because potentially millions of rows need to be inserted. The rows are ordered by download count descending within each priority set so that the most-used packages are processed first during the revalidation phase.
SingletonService.IsSingletonAsync() always returns true and contains a // TODO comment. There is no actual distributed lock preventing two job instances from running concurrently. If two instances run simultaneously, they will both dequeue and enqueue the same packages, causing duplicate validation messages.
The killswitch is checked twice inside RevalidationStarter.CanStartRevalidationAsync() — once before the throttle check and once after the health check. This is an intentional defensive pattern: the health check involves an async I/O call, so a killswitch activated during that window would otherwise be missed until the next iteration.
The throttler enforces a minimum sleep of 5 seconds between batches regardless of the calculated delay, preventing a tight spin loop when the desired rate is very high relative to the batch size and the batch completes nearly instantaneously.
The PackageFinder.FindDependencyPackages method performs a breadth-first traversal of the PackageDependencies table using iterative SQL queries rather than a recursive CTE, ensuring it works within EF’s LINQ-to-SQL translation constraints while still capturing the full transitive dependency graph of Microsoft and preinstalled packages.