Nodejitsu

Save time managing and deploying your node.js app. Code faster with jitsu and npm

Scaling npm, December 2013

About the author

Name
Location
Worldwide
nodejitsu nodejitsu

The public npm registy has been busy in the last few weeks. In fact: it has grown 7.2% since #scalenpm launched on November 26th! This is the first in a series of monthly posts from Nodejitsu to keep the community up-to-date with how we are keeping the public npm registry awesome!

So ... what now? Like we said we've been focused on battle-hardening high-availability via horizontally scaled multi-master clusters:

  • New consistent SSD-backed hardware: On December 19th, two new consistent machines went live in SoftLayer's Dallas datacenter. Each of these new machines comes equipped with an 800GB SSD, which leaves the public npm registry room to grow on disk while we begin regular compaction cycles.

  • Monitor & audit replication: All replication audits, conflict resolution, are all monitored automatically via new modules from Jason Smith and Jarrett Cruger. These modules will be made Open Source early in the new year after they have run for a few weeks against the production public npm registry.

  • Atomic publishes! Want to help scale npm? Update your npm client to v1.3.19 or above right now!

[sudo] npm update -g npm

Why? Because npm@1.3.19 added atomic publishes. That is: a package can be published to the npm registry in a single HTTP request. This is important because previously the same publish operation required multiple writes to CouchDB. Behind a multi-master configuration (like the one used by the public registry), this can cause conflicts in CouchDB.

If you're interested in learning more about atomic publishes:

  1. Jan Lehnardt wrote up a fantastic explaination about how this situation arises recently which we highly suggest you read if you're interested in how CouchDB works.
  2. You can read through the changes in the npm-registry-client by isaacs that made it possible.

That's it for December. Keep checking back for more #scalenpm updates!