Dogfooding at scale: migrating cdnjs to Cloudflare’s Developer Platform (opens in new tab)

cdnjs now runs entirely on Cloudflare’s Developer Platform after a migration intended to improve maintainability rather than performance. Despite the rise of bundlers and modern JavaScript tooling, cdnjs still serves about 9 billion requests per day because it is free, familiar, immutable, auditable, and widely used by both developers and AI coding assistants. The migration replaces a fragmented GCP, GitHub, VM, and Cloudflare setup with a unified architecture built around Workers, R2, Workflows, Queues, D1, KV, Cache, and Containers.

cdnjs’s Scale and Continued Relevance

• cdnjs serves roughly:
  • 108,000 requests per second
  • 9 billion requests per day
  • Traffic across more than 330 Cloudflare data centers
  • A 98.6% cache-hit rate
• It is used by approximately 12% of websites and holds a 48.3% share of the JavaScript CDN market.
• Its simple <script>-tag model remains popular because:
  • URLs and versions are consistent and immutable.
  • Libraries are available without accounts, API keys, or rate limits.
  • Files include Subresource Integrity hashes.
  • The project is open source and community-driven.
• AI assistants frequently generate cdnjs URLs because they appear throughout years of tutorials, documentation, GitHub repositories, and Stack Overflow answers.

Why the Existing Architecture Became a Problem

• Cloudflare moved cdnjs file serving to Workers and KV in 2020, improving resilience and enabling pre-compressed Brotli and gzip assets.
• The publishing pipeline remained on GCP because Cloudflare previously lacked suitable tools for:
  • Fetching large package archives
  • Running CPU-intensive processing
  • Coordinating multi-step jobs over hours
• The old pipeline combined GCP Functions, Google Cloud Storage, Pub/Sub, a git-sync VM, GitHub, Workers KV, and a bare-metal origin.
• New features and bug fixes required coordinating deployments across multiple platforms, while observability required manually stitching together unrelated logs.

Problems with the Legacy Pipeline

• No shared tracing

  • Package updates could pass through several systems without a common correlation ID.
  • Partial failures could leave KV updated while GitHub remained stale, with no alert indicating the divergence.

• Split-brain storage

  • File content existed both in Workers KV and a GitHub repository.
  • Neither system was cleanly authoritative, making reconciliation difficult.

• Storage-driven orchestration

  • GCP Cloud Functions triggered one another through object-created events.
  • Storage effectively acted as a message queue without dead-letter handling, backlog visibility, or reliable replay.

• Operational fragmentation

  • npm polling required 26 separately deployed Cloud Functions, one for each alphabetic shard.
  • Health monitoring required checking all 26 deployments and their logs.

• An oversized GitHub repository

  • The repository exceeded 1.1 TB of packed storage.
  • GitHub could no longer generate archive downloads reliably.
  • Cloning and forking became impractical.
  • A 274-entry .gitignore accumulated to exclude releases the pipeline could not reject properly.

• Security overhead

  • Cloud Functions, a VM, container images, storage buckets, and service-account credentials all required patching, auditing, and protection.
  • Retiring these components reduced the attack surface and eliminated recently exposed vulnerabilities.

The New Cloudflare-Based Architecture

• The rebuilt system uses Cloudflare’s Developer Platform end to end.
• R2 becomes the single source of truth for file content.
  • It can store large assets that previously did not fit comfortably in KV, including source maps, large bundles, and font packages.
  • Its S3-compatible API makes the catalog accessible to external tools and mirrors.
• The broader platform combines:
  • Workers for request handling
  • Workflows for orchestration
  • Queues for reliable asynchronous processing
  • R2 for durable object storage
  • D1, KV, Workers Cache, and Containers for supporting services
• Centralizing the pipeline should make processing state observable, reduce deployment complexity, and eliminate inconsistencies between edge storage and the GitHub repository.

Practical Conclusion

The cdnjs migration demonstrates that a globally critical, high-volume open-source service can evolve from a collection of legacy systems into a unified serverless platform. Its continued value comes not only from speed, but from being free, predictable, immutable, and easy for both humans and automated tools to consume.