Skip to main content

​
Overview

ArchivePackages is a headless .NET Framework 4.7.2 console job that provides incremental backup of NuGet package binaries (.nupkg files). On each execution it:
  1. Reads a cursor timestamp from a cursor.json blob stored in the destination container.
  2. Queries the Gallery SQL database for all packages whose Published or LastEdited timestamp is newer than the cursor.
  3. For each such package, triggers a server-side blob copy from the primary packages container into an ng-backups container on the destination storage account.
  4. Advances the cursor to max(LastEdited, Published) across the batch.
The job optionally replicates the same set of packages to a secondary destination account, providing a redundant offsite backup.
The job only initiates the blob copy — it calls StartCopyAsync and does not wait for Azure Storage to complete the server-side transfer. Any process relying on a fully-consistent backup must account for in-flight copies.

​
Role in System

Source

Azure Blob Storage — packages container. Blob name format: {id}.{version}.nupkg (lowercased).

Destination

Azure Blob Storage — ng-backups container. Blob name format: packages/{id}/{version}/{url-encoded-hash}.nupkg.

Cursor

cursor.json stored in the destination container. Contains a single ISO-8601 cursorDateTime key that advances after each successful batch.

Deployment

Packaged as a NuGet package via .nuspec; installed as a Windows Service using NSSM via Octopus Deploy PowerShell scripts.

​
Key Files and Classes

​
Dependencies

​
NuGet Package References

​
Internal Project References

​
Transitive / Shared Dependencies

​
Configuration Reference

​
Notable Patterns and Implementation Details

Cursor-based incremental processing. The cursor is a single cursorDateTime value persisted inside cursor.json in the destination container itself. A freshly created destination container requires a manually seeded cursor.json before the job will run successfully.
Parallel tuple construction, sequential copy dispatch. The Dapper result list is projected into (sourceBlobName, destBlobName) tuples using AsParallel().Select(...), but actual ArchivePackage calls are dispatched sequentially. The parallelism only benefits name-string computation, not I/O throughput.
Secondary destination re-queries the database independently. Each destination triggers a full separate Archive() call that re-reads the cursor and re-queries SQL. If packages are published between the two calls, the two archives may cover slightly different sets of packages.
No copy-completion verification. ArchivePackage calls StartCopyAsync(...) and returns immediately. Azure Storage executes the copy asynchronously. A storage outage or throttling event can leave copies in a pending state with no job-level alerting.
Blob name hash encoding. Destination blob names URL-encode the package hash (WebUtility.UrlEncode(hash)). This is necessary because SHA-512 hashes are Base64 strings containing +, /, and = characters that are not safe in Azure blob path segments.
NSSM-based Windows Service deployment. nssm.exe (Non-Sucking Service Manager) is bundled in the NuGet package. PostDeploy.ps1 uses it to register the job with automatic restart-on-failure, ensuring the job recovers from transient crashes without manual intervention.