Nodejitsu

Save time managing and deploying your node.js app. Code faster with jitsu and npm

Introducing Flatiron

About the author

Name
Location
Worldwide
nodejitsu nodejitsu

Web development is all about choices. What language do you want to use? What database? What libraries? No matter how you develop web applications these choices have to be made. When using large frameworks in other languages such as Ruby or PHP most of these choices are made for you. In Node.js we have npm, which has allowed a diverse ecosystem of modules to develop. This has fostered a philosophy of creative experimentation, but not made it any easier to make these choices.

This is the motivation behind flatiron. At it's core flatiron is two things:


An initiative to build a collection of decoupled tools with the same standard of quality and performance that you would expect from anything built by Nodejitsu.

A full-stack web application development framework which packages these tools together to make isomorphic and stream-based application development easier.

This article will introduce you to the components that make-up flatiron as well as how the core flatiron project.


A Sum of its Parts

At Nodejitsu, we have and continue to believe in a simple mantra: build the best tools and the best systems will follow. Tools and their distribution among members of an organization evolve organically overtime. To put it another way: if you use Node.js you are probably already using a component of flatiron. We have simply moved these tools to the Flatiron Github Organization

  • api-easy: Fluent (i.e. chainable) syntax for generating vows tests against RESTful APIs.
  • complete: Custom command line tab completion for nodejs CLI apps.
  • nconf: Hierarchical node.js configuration with files, environment variables, command-line arguments, and atomic object merging.
  • neuron: The simplest possible event driven job manager, FIFO queue, and 'task based cache' in node.js.
  • node-prompt: A beautiful command-line prompt for node.js.
  • sugarskull: A depth-first search based routing library for the browser and node.js.
  • winston: A multi-transport async logging library for Node.js.

In addition to migrating existing libraries we have been working on a number of new libraries to round out these tools.

  • broadway: Lightweight feature reflection and application composition for node.js and the browser.
  • flatiron: An unobtrusive full-stack web framework for node.js and the browser.
  • plates: Unobtrusive, thin and fast anti-templating
  • revalidator: A cross-browser / node.js validator used by resourceful and flatiron.
  • resourceful: A resource-oriented ODM with a high-level API for routing and validation.
  • union: A hybrid buffered / streaming middleware kernel backwards compatible with connect.
  • utile: A drop-in replacement for util with some additional advantageous functions.

A lot of effort has been made to ensure that each component stands on it's own and that each Github repository clearly explains the purpose, use case, and stability of the library. There are a few particularly interesting highlights among these libraries that we'll explore below.


A quick tour of Flatiron

Application composition

Thanks to Javascript you never have to worry about types again (mostly), but that doesn't mean that we can't learn something from design patterns more prevalent is statically typed languages. Namely: inversion of control and dependency injection. Don't be scared! Promise you won't have to register anything, ever.

Broadway exposes a simple "plugin" API which allows the application developer to extend the top-level application object easily. This model translates well between the browser and the server because developers can write different plugins to suite the needs of the different environments. Broadway includes several plugins by default which can be overridden if you don't agree with the particular module or library choice we've made:



Figure 1: app.use() in Broadway


Doesn't remind you of dependency injection? It shouldn't! Because of the loosely typed nature of Javascript it isn't necessary to bring along all of the heavyweight type registration APIs that are so common in most dependency injection frameworks. Instead, plugins must be responsible and properly implement three methods

  • .attach(options): This synchronous method is called immediately on app.use(somePlugin).
  • .init(done): This asynchronous method is called once when the application initializes and before it starts.
  • .detach(): This synchronous method is called on app.unuse(somePlugin).



Figure 2: Plugin Lifecycle in Broadway


All of these functions are evaluated in the scope of the app (i.e. this inside of a plugin is the app itself). Lets take a look at a quick sample from broadway itself:

exports.attach = function (options) {  
  //
  // Here setting `this.config` will expose the `.config` 
  // property on the application itself
  //
  options = options  || {};
  this.config = new nconf.Provider(options);
};

exports.init = function (done) {  
  //
  // Later on we can initialize the configuration 
  // we attached earlier.
  //
  this.config.load(function (err) {
    return err ? done(err) : done();
  });
};

Ok. So what? Application extensibility allows us to reuse more components across different kinds of applications while writing less code. In almost every express application, you will see a server.js file which is essentially wiring up the same libraries to the same methods. If you write a couple of small applications it might not seem like a problem, but when building a product like Nodejitsu it simply isn't DRY enough.

Streaming or not streaming? No problem!

One of the current problems in node.js core is that there is no good way to buffer a stream when not closing the underlying file descriptor. Don't worry though; the geniuses behind node.js are on it and already have a solution on the drawing board. In the meantime though, something needs to fill the gap and that is union and sugarskull.

The problem is that 90% of the time you actually want to buffer a small amount of request data (< 10k) into memory then parse it and attach it to the http.ServerRequest object. Think: a POST with some JSON attached to it. Occasionally though, you may want to take advantage of a stream. Think: a large file upload to Amazon S3. In this case, buffering is cancer and .pipe() is king.

The solution is two-fold:

  • Pipe the concrete http.ServerRequest to a BufferedStream implementation by default.
  • Attach metadata to routes indicating if the route requires a stream or not. If a stream is not required, attempt to parse the entire body before continuing to the Router.
var fs = require('fs'),  
    union = require('../lib'),
    sugarskull = require('sugarskull'),
    favicon = require('./middleware/favicon');

var router = new sugarskull.http.Router();

var server = union.createServer({  
  before: [
    favicon('./favicon.png'),
    function (req, res) {
      var found = router.dispatch(req, res);
      if (!found) {
        res.emit('next');
      }
    }
  ]
});

router.get(/foo/, function () {  
  this.res.writeHead(200, { 'Content-Type': 'text/plain' })
  this.res.end('hello world\n');
});

router.post(/foo/, { stream: true }, function () {  
  var req = this.req,
      res = this.res,
      writeStream;

  writeStream = fs.createWriteStream(Date.now() + '-foo.txt');
  req.pipe(writeStream);

  writeStream.on('close', function () {
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end('wrote to a stream!');
  });
});

server.listen(8080);  
console.log('union with sugarskull running on 8080');  

As the above sample illustrates, not much changes from the API you may already know with a couple of key differences:

  • The request, req, and response, res, are set on the this argument of the route function.
  • Optional metadata can be passed to Router methods like .get. To enable the stream API just set { stream: true }.

You may be yelling "what happened to function (req, res, next)!?!" but that's too much to go into here. Rest assured there are good reasons. If you're thinking about


Routing, templating and validation: anywhere, anytime

Flatiron comes with a highly performant router, a robust validation library, and a fast unobtrusive templating library which all work on the client and the server. Currently there is no packaged browser-build of all of these components together, but that will be coming soon through a collaboration with the jQuery UI team.

There are other components which are on the Roadmap for isomorphic (i.e. browser-server) compatibility: broadway and resourceful. Here's a quick compatibility matrix of all the modules we've listed above:



Figure 3: Flatiron Compatibility Matrix


A community effort

We have not been operating in a vacuum at Nodejitsu on Flatiron. We have taken feedback from the Node.js community for all of our existing libraries. Thanks to you hundreds of issues have been found and closed. For the components we are releasing today, many of them are a result of your feedback. Fast unobtrusive templating? Done. Streaming + non-streaming. Done.

In the coming weeks we will be announcing new libraries to fulfill our roadmap with help from other companies who are as exciting about Node.js and isomorphic Javascript applications as we are.


What's next?

There are still pieces missing, but the Flatiron team will be working hard over the coming weeks and months to finish up our roadmap:

  • A browser-isomorphic (i.e. client-server) Presenter layer.
  • Make resourceful and broadway browser-isomorphic.
  • A pluggable View layer.
  • A full-stack browser solution.


Want more? Check back tomorrow.

This is just the beginning.