Nodejitsu

Save time managing and deploying your node.js app

Package.json dependencies done right

About the author

Name
Location
Worldwide
nodejitsu nodejitsu

Other popular posts

- Scaling Isomorphic Javascript Code - Keep a node.js server up with Forever - Package.json dependencies done right
- npm cheat sheet - 6 Must Have Node.js Modules
Sign up to our platform for free - get $20 usage

Adding public npm modules to your project is easy, but what if you want to add private modules? Code that contains sensitive data or logic which you do not want to share with the public.

Let us help you and manage your private modules with our private npm service, sign up now.

This is part one of a three part post on npm

Dependency management. Need I say more? Entire developer ecosystems live and die by how dependency management works. Ask a Ruby developer what they would do without RubyGems, or ask (most) Java developers how to get things done without Maven. Most importantly: ask a node.js developer how to get things done without npm.

Npm recently received a pretty serious makeover in Version 1.0. If you haven't read up on what changed between 0.3.18 and 1.0, you should read Isaac Schlueter's (or simply @isaacs), the author of npm, blog post now.

Npm installs modules and applications based on an interpretation of the package.json specification. The most important part of this is how it determines the which version(s) of your dependencies to download. In this article I will examine:


  1. How npm works with dependencies in a package.json manifest.
  2. Best practices for specifying modules and node.js dependencies.


tldr;? Jump down to Best Practices

Semantic versioning in npm

There are two separate (but equally important) entities that must have specified versions in a package.json manifest: modules dependencies and node version. All of these version numbers are interpreted using a slightly modified semantic version (or semver) specification.

What is slightly modified about it? It accepts the '-' character as a valid character in build strings. So 0.2.0-1 is valid in npm semantic versioning, but not in the official semver spec. Personally, I like this flavor of semver and hope that it will get pulled in up-stream.

Ok, so npm uses semver, awesome. Now what? Lets take a look at a naive example:

{
  "name": "naive",
  "description": "A package using naive versioning",
  "author": "A confused individual <iam@confused.com>",
  "dependencies": {
    "express": ">= 1.2.0",
    "optimist": ">= 0.1.0"
  },
  "engine": "node 0.4.1"
}

The versions for both the dependencies and node properties in the example above are interpreted using node-semver, a library created and maintained by isaacs for working with semantic version strings.

Now suppose that we install the above example using npm:

  1. The node version installed on the current system is compared against the version specified in the package.json. If the node version does not satisfy the target version, the installation fails.
  2. For each dependency specified in the dependencies property, the highest version which satisfies the specified semantic version string is downloaded and placed into the node_modules folder in the target directory. If there is no version available in the npm registry which satisfies the target version, the installation fails.

Dependencies: Best Practices

"Best practices" is a term that is often misused and thus can be ambiguous in this context. For such reasons, it is important to make clear the goals behind this particular versioning scheme:

  1. Strive to be forwards compatible with node.js (within reason)
  2. Keep the dependencies of multiple installations of the same app cohesive
  3. Reduce the surface area of potential bugs due to breaking changes

Here is a sample package.json that uses the best practices we will discuss:

{
  "name": "best-practices",
  "description": "A package using versioning best-practices",
  "author": "Charlie Robbins <charlie@nodejitsu.com>",
  "dependencies": {
    "colors": "0.x.x",
    "express": "2.3.x",
    "optimist": "0.2.x"
  },
  "devDependencies": {
    "vows": "0.5.x"
  },
  "engine": "node >= 0.4.1"
}

When specifying modules dependencies: use 1.0.x syntax

Until recently, I was guilty of not following this guideline: I continued to use the >= 0.2.0 syntax illustrated above in the naive.package.json example. At first glance there doesn't seem to be anything wrong with that style. You're saying "If there are changes in the future I want them."

The problem arises because authors are conveying meaning in their versions. Not every version will be completely backwards compatible with the particular version you were using when you wrote your application. This is conveyed in the version string:

e.g.: 0.3.18

  • Major Version (0)
  • Minor Version (3)
  • Patch Version (18)

Changes to the major and minor parts of the version mean that changes have happened, although there is no convention to convey they are breaking. Changes to patch versions are used to express that a fix has been made and it is (usually) safe to upgrade.

Conversely, when using the 0.2.x syntax you're saying: "If there are patch changes in the future I want them, but no minor or major versions." Given the description of the meaning conveyed by each of the version components above this means you won't be tearing your hair out over breaking changes in a module you depend on.



When specifying node versions in modules: use >= 0.4.0 or 0.4.x | 0.5.x syntax

Don't write modules and publish them on npm? Skip this and jump down to specifying node versions in apps.

As a module author it is your responsibility to consciously follow anything you take as a dependency and ensure that you update your module(s) accordingly. Beyond that, you must also realize that (if you're lucky) your module will be used on the bleeding edge. As such, you must accommodate these valid scenarios. At the end of the day it is the choice of the user to decide on their target node version and you should not limit their choices unless you are explicitly aware of a problem in future versions of node.js core.




When specifying node versions in apps: use 0.4.x or 0.4.0 syntax

As with any sufficiently important problem, there are nuances to dependency management you should be aware of. One such nuance is the difference between specifying dependencies in an application vs. modules.

Module authors encounter a much wider surface area of use cases, and as such should be more liberal in the versions of node.js they accept. On the other hand, application developers typically have no upstream dependencies and should be more explicit about what version of node.js they are expecting on the target system.

For example, at Nodejitsu we will use the highest version of node.js which satisfies the node specified version. So if you use 0.4.x, but are really expecting 0.4.5, you should indicate this in your package.json or else you may encounter unexpected behavior.



Take advantage of devDependencies

The package.json specification used by npm has an additional property for separately listing dependencies which are only required for development purposes. Why separate you might ask? Besides not having to download these dependencies from the registry when running in production, npm it is required if you wish to take advantage of a couple of neat npm features such as test:

  $ pwd
  /Users/Charlie/GitHub/nconf

  $ npm test
  npm info it worked if it ends with ok
  npm info using npm@1.0.6
  npm info using node@v0.4.8
  npm info pretest nconf@0.1.9
  npm info test nconf@0.1.9

  > nconf@0.1.9 test /Users/Charlie/GitHub/nconf
  > vows test/*-test.js --spec

  (...)

If I had not specified my test library, vows, in the devDependencies section of the package.json for nconf I would have received:

  $ npm test
  npm info it worked if it ends with ok
  npm info using npm@1.0.6
  npm info using node@v0.4.8
  npm info pretest nconf@0.1.9
  npm info test nconf@0.1.9

  > nconf@0.1.9 test /Users/Charlie/GitHub/nconf
  > vows test/*-test.js --spec

  sh: vows: command not found
  npm info nconf@0.1.9 Failed to exec test script
  npm ERR! nconf@0.1.9 test: `vows test/*-test.js --spec`
  npm ERR! `sh "-c" "vows test/*-test.js --spec"` failed with 127
  npm ERR! 
  npm ERR! Failed at the nconf@0.1.9 test script.

  (...)


Conclusion

Node.js is evolving rapidly: it has been less than a month since npm 1.0 was released which dramatically changed the way in which dependencies are handled. More importantly: the modules on-top of node.js critical to building scalable web applications are evolving even faster, making the best practices outlined here that much more important if you want to cope.

As with any complex ecosystem, this is a process and these practices may change over time as the goals outlined change. We'll keep you posted at Nodejitsu.