Skip to main content

​
Overview

Stats.CollectAzureChinaCDNLogs is a .NET Framework 4.7.2 console application that runs as a scheduled NuGet job. Its sole purpose is to bridge the gap between the China-region Azure Front Door CDN log format (newline-delimited JSON) and the tab-separated OutputLogLine format consumed by the rest of the NuGet download-statistics pipeline. Each execution leases up to 4 blob files from a China Azure Blob Storage container, parses each line as an AfdLogLine JSON object, maps the AFD-specific fields to canonical stats columns, and writes gzip-compressed output blobs to a global (non-China) Azure Blob destination.
The job defaults to a 4-hour execution timeout (14400 seconds). A secondary “force-stop” cancellation fires 60 seconds after the primary timeout to ensure the process terminates even if StreamReader/StreamWriter calls do not honour the cancellation token (a known limitation of .NET Framework—fixable on .NET 6+).

​
Role in the NuGetGallery System

Stats Pipeline Ingestion

Feeds China CDN download events into the global stats pipeline so that NuGet.org package download counts include traffic served through the Azure China sovereign cloud.

Cross-Cloud Bridge

Authenticates to China storage via SAS token and to global destination storage via Managed Identity, acting as a secure cross-cloud data relay.

Format Normalisation

Converts AFD JSON log lines into the standard W3C-style TSV OutputLogLine format shared by all other CDN log collectors in the stats subsystem.

Windows Service Deployment

Packaged as a NuGet .nuspec and installed as a Windows service via NSSM (Non-Sucking Service Manager) using PowerShell pre/post-deploy scripts.

​
Key Files and Classes

​
Dependencies

​
NuGet Packages (resolved transitively via project references)

​
External Assembly Reference (not on NuGet)

​
Internal Project References

​
Configuration Reference

​
Notable Patterns and Quirks

Azure SDK SAS token bug workaround. Both Job.cs and ValidateAzureBlobServiceClient contain an explicit string replacement of SharedAccessSignature=? → SharedAccessSignature= to work around Azure SDK issue #44373. The fix appears in two places in the same method scope.
Dual authentication strategy. The source (China) always uses a SAS token connection string because Managed Identity is not available in the Azure China sovereign cloud. The destination (global) uses Managed Identity when UseManagedIdentity = true, falling back to SAS tokens otherwise. In DEBUG builds, DefaultAzureCredential is substituted to support developer workstations without a full MSI setup.
Status code format difference. Global CDN logs emit cacheStatus/httpStatusCode as a single combined field. China AFD logs provide these separately, so ChinaStatsCollector constructs the combined string manually: CacheStatus + "/" + HttpStatusCode. Several downstream stats fields (filesize, sport, rsbytes, timetaken) have no AFD equivalent and are hard-coded to "0" or "na".
Stream verification before processing. Before consuming a blob, ChinaStatsCollector.VerifyStreamAsync reads up to the first 10 non-empty lines and attempts JSON deserialization. If none parse successfully the blob is rejected, preventing corrupt or misrouted files from polluting the output pipeline.
TLS metadata packed into xeccustom1. SSL protocol, cipher, and elliptic curve are concatenated into the xeccustom1 output column as "SSL-Protocol: ... SSL-Cipher: ... SSL-Curves: ...", mirroring the convention used by the global CDN collectors in Stats.AzureCdnLogs.Common.
CancellationToken propagation limitation. The Run() method documents that StreamReader.ReadLineAsync and StreamWriter.WriteLineAsync in .NET Framework 4.x do not accept a CancellationToken, making graceful mid-stream cancellation unreliable. The force-stop CTS (+60 s) exists as a safety net. This is acknowledged tech debt that would be resolved by migrating to .NET 6+.