Getting Started With Node and Express

JavaScript | Node 13 13 Comments

Update

“Learn JavaScript with Eric Elliott”, a new series of JavaScript training courses is currently crowdfunding on Kickstarter. Right now you can lock in crazy discounts, and special rewards exclusively for backers. What are you waiting for? Pick your reward now. This post will still be here when you get back.

/Update

Programming JavaScript Applications

This post is an excerpt from my new book, “Programming JavaScript Applications”. Enjoy!

Update – you can watch a recorded O’Reilly webcast presenting this information: “Getting Started With Node and Express”.

Node is a server-side JavaScript environment with many attractive features:

  • A fast JavaScript engine (built on V8).
  • Asynchronous by default philosophy (nothing should block).
  • Event-loop design (much like the browser environment).
  • Networking as a first class citizen (create production capable servers with few lines of code).
  • A highly usable streams API.
  • A large, rapidly growing developer community.
  • A simple, CommonJS-based module solution that guarantees module encapsulation (your var declarations are limited to module scope).
  • A developer friendly package management system with thousands of open-source packages to choose
    from.

Some of these features might take some getting used to if you are accustomed to server-side environments that allow features such as blocking IO and a single thread per connection (for convenient state management). However, you’ll find that the incredible performance boost achieved by non-blocking request/response cycles is well worth the learning effort.

Don’t underestimate the value of the asynchronous by default philosophy. That is the key to Node’s incredible performance in production environments.

Where other environments force users to wait in line while files load or network operations take place, Node fires off the request and keeps accepting new connections and executing other code paths while the asynchronous event does its work in the background.

Processes can spend an incredible amount of time waiting for file reads and network requests, especially if they encounter an error. Node just keeps cruising along. It’s like getting out of congested city streets with stop lights at every block and on to an open freeway. Node isn’t fast simply because of the performance of the V8 JavaScript engine (though that does help). It’s fast because it doesn’t waste time waiting around for things to happen.

There are other platforms that share some of JavaScript’s performance characteristics: Twisted Python and Tornado spring to mind. They’re fast for the same reason. However, even though they are more mature, they can’t compete with the active membership of the JavaScript developer community.

Node comes packaged with a module management solution called npm. It gives you access to a package registry stocked with thousands of open-source packages, and makes it very easy for you to contribute your own, or use a private git repository for proprietary work. Of course, it’s easy to mix and match open-source and proprietary packages in a single application.

Installing Node and Express

First, make sure you have node installed. There are installers available from the Node homepage, but I like to use nvm so that I can easily switch between different versions of node. To install node with nvm:

For more on nvm, check out the docs on the Github repository.

With node installed, you’ll need to create a new directory for your project:

Then initialize your project:

Express is currently the most popular application framework for Node. It’s easy to learn and use, and it has a vibrant developer community. If you’re going to build applications in Node, chances are you’ll eventually use express. There’s no time like the present to get started. Install express:

That’s it. You’re ready to get started!

Node Tips

If this is your first time using Node and Express, it might be helpful to see what some of the community believes are the current set of best practices. Node Bootstrap aims to show new users some common practices in the Node / Express community, using Twitter Bootstrap. Among other things, there’s an example of using the cluster module to manage multiple instances of the server (utilizing all available CPU cores).

Organizing Files in Node

It’s a good idea to follow the emerging file organization trends in existing, popular Node repositories. That way, anybody familiar with Node should be able to find their way around your repository. Here are some common file locations:

  • Main ./index.js, ./server.js, or ./yourentryfile.js in the root.
  • Supporting files in ./lib/
  • Static http files in ./public/
  • Views or templates in ./views/
  • Command-line executables in ./bin/
  • Tests in ./test/ (or ./spec/ if you`re a jasmine cool-aid drinker)
  • Npm Scripts in ./scripts/
  • Config in ./config/
  • Documentation in ./doc/
  • Examples in ./examples/
  • Performance analysis in ./benchmarks/
  • Native c / c++ source in ./source/

The npm repository serves as a good example.

Node Libraries

Some of my favorite Node libraries include:

  • Mout Like Underscore / LoDash. Stuff that should probably be included in JavaScript.
  • Express Web application framework.
  • Qconf Application config.
  • Hogan Mustache for express.
  • Superagent Communicate with APIs.
  • Socket.io Realtime communications (websockets).
  • Q Promises.
  • Async Asynchronous functional utilities.
  • Bunyan Logging.
  • Tape Testing.
  • Cuid Better than guid/uuid for web applications.
  • Credential Easy password hashing and verification.
  • Sails Rapid application prototyping with socket.io and MVC on Express.
  • Node-http-proxy Proxy your service APIs.

Configuration

Don’t include configuration data in your app repository (including secrets, paths to file locations, server hostnames, etc..).

Instead, set up environment files with examples for sane defaults. Check in the examples, but don’t check in the actual configuration. Following this rule of thumb will make deployment / ops support for the app a lot easier.

Check an example file into your app repo:

s3.env.example

Then copy it and fill in the real values when you install the app:

Use a package like nconf to make the environment variables available in your app.

Make sure that the real environment files get added to .gitignore so that you don’t accidentally check them into your repository.

Warning About State

One of the first stumbling blocks you might run into moving from browsers to Node is that you can’t rely on your closure state to be reserved for a single user. You have a single instance of the app, with a single pool of memory, and a potentially unbounded number of incoming connections.

State needs to be kept in a database, or passed as parameters through function calls. For example, each request in an Express application will have corresponding request and response objects. That may be a good place to store in-memory state for a single request/response cycle.

Likewise, singletons are a good way to store state that will be shared for all requests, such as your application configuration, but otherwise, they’re usually an anti-pattern in Node applications.

Express

There are a lot of application frameworks available for Node. One popular framework that I find particularly useful is Express. It’s basically an HTTP server built on top of Node’s http module and Connect middleware.

Create your app

To create an express app instance, you’ll need to require express, and call the function that gets returned:

Routing

Express has a built-in app router. It’s pretty simple to use. First, request method names correspond to the methods you call to set up your route. GET is .get(). POST is .post() and so on. To create a route that will handle any request type, use .all().

Pass the route as the first parameter, and a function as the second parameter:

Routes have easy parameter matching:

A route can be a regular expression:

Middleware

Middleware is software that takes an incoming request, processes it, and passes it on to the next piece of middleware in the chain. Express middleware takes the form:

Here’s how it works in the context of an express server:

Point a browser at the new server, or just use curl:

Handling errors is just as simple. Again, you’ll use middleware:

You can clean up after a lot of errors. In fact, sometimes an error is an expected probability. For example, there’s a chance that sometimes remote services won’t be available, and you can recover from that condition and try again later. However, sometimes you just won’t get the answer you’re looking for and there’s nothing you can do to recover. You don’t want to keep your server running with undefined state. In the case of errors that you can’t easily recover from, it’s important to shut down the process as quickly as possible.

Let it Crash

Processes crash. Like all things, your server’s runtime will expire. Don’t sweat it. Log the error, shut down the server, and launch a new instance. You can use Node’s cluster module, forever (a Node module available on npm), or a wide range of other server monitor utilities to detect crashes and repair the service in order to keep things running smoothly, even in the face of unexpected exceptions.

Templates

Express comes with some built-in handling of templates, but it must be configured. You have to tell Express which view engine to use in order to process templates, and where to find the views. First, you’ll want to require your template engine. For Mustache templates, you can use Hogan:

Most of the settings for express are specified with app.set(). You’ll need to use it to configure express to use the template engine of your choice. There are four options that you should be aware of:

Remember to define a route that uses your new view. Assuming you’ve used your middleware to build a data object on the request object called req.data (see Middleware, above):

You can leave off the callback parameter and any errors will be internally passed via next(err) for your generic error handlers to catch. If you pass the callback, that automatic error handling will not occur, and you should handle the error explicitly.

Next Steps

Of course, you want to do a lot more with your app than return a hard-coded message to your users. The good news is that there are drivers for just about any database you can dream of. You can use a variety of template libraries, and of course, serve static files. I encourage you to dive into the Node module playground and take a look around.

For starters, here’s a simple static file server example using the built-in static middleware:

Have a look at the Express guide and API reference for a lot more useful examples, and the Node Manual for Node API documentation. There are lots of useful gems that you’ll want to learn more about.


  1. Nam Nguyen - May 6, 2013

    Very nice and definitely useful info for node beginners. There are many other templates are commonly used with Express such as: Halm, Jade, CoffeeKup, jQuery Templates for node… would you be able to provide some samples how to integrate node with one of these template engines?

    • Eric Elliott - May 7, 2013

      Yes – thank you for bringing this up. I’ll definitely add a section about how templates integrate with Express applications.

  2. Gilles Ruppert - May 7, 2013

    Is there a reason you’re using setTimeout instead of process.nextTick in line 52 of the 4th middleware example? The latter is much more efficient according to the documentation (http://nodejs.org/api/process.html#process_process_nexttick_callback). 

    • Eric Elliott - May 10, 2013

      Yes.

      Node 0.10 changed the way process.nextTick() works so that it always returns before any IO. In this case, we want to give our logging / graceful shutdown processes time to do their thing (this is the whole point of setTimeout() here).

      That said, I haven’t taken a deep dive into the call stack and examined it both ways to really wrap my head around the nuances. A lot of Node developers are taking advantage of the new setImmediate() function instead of process.nextTick(), but I want this code to work with ~0.8.x as well.

  3. You wrote that ./public will contain static http files. Does this mean, it will contain images, static web page content, videos etc..?

  4. Nam Nguyen - September 12, 2013

    Eric, I guess you might need to update your sample to handle content-length issue if you have a chance. Reference Connect’s issue: https://github.com/senchalabs/connect/issues/889 and stackoverflow: http://stackoverflow.com/questions/18692580/node-js-post-causes-error-socket-hang-up-code-econnreset

  5. Nam Nguyen - September 12, 2013

    I also recommend to use express-resource or namespace (from same express author) to organize routes. It’s pretty cool idea. https://github.com/visionmedia/express-resource

  6. I think that 
      app.use( app.router );

    should come before
      app.use( express.static(__dirname + ‘/public’) );

    Regards,
    Wilson 

    • Eric Elliott - September 29, 2013

      That depends on which handler you want to take precedence. You could, for example, stick a static HTML page in your /public directory that will override a route if you have static before router. That configuration is sometimes useful. It depends on the app, and your needs for the app.

      • Wilson - October 1, 2013

        oh I see thanks for the reply by the way I have reinforced this concept with stackoverflow link: http://stackoverflow.com/questions/12695591/node-js-express-js-how-does-app-router-work which has a more detail information. Thanks Eric

  7. By the way thank your for the article I found very useful and I liked it.
    keep it up with great posts like this one.

Add Comment Register



Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">