Taming CPU Intensive Node.js Streams

Streams are a fundamental software concept, allowing us to read data from a source (faucet) and funnel it over out to a destination (sink), while potentially running some transformations in between. They are perfectly analogous to water running through pipes, thus the terminology of “stream” and “pipe”.

Node.js of course does not fall short when it comes to this paradigm, and offers the following stream types:

1. Readable – reads from a data source
2. Writable – writes to a data destination
3. Transform – used to apply transformations to the data stream
4. Duplex – used for both write and read operations, or for two parallel streams flowing in both directions

Let’s take a look at a trivial example – compressing a file on the fly:

const fs = require('fs'), gzip = require('zlib').createGzip();

// create a readable stream from file
var source = fs.createReadStream('big_file.log'); 

// create a writable stream to file
var destination = fs.createWriteStream('compressed.gz'); 

source // read big_file.log from the filesystem
 .pipe(gzip) // perform compression
 .pipe(destination) // write compressed file to filesystem
 .on('finish', () => { console.log('completed!'); });

Let’s reiterate the example above: we have created Readable file stream, piped it via a gzip Transform stream, and out to a file through a Writable stream.

The thing to note here is that while both the Readable and Writable stream operations are I/O bound, the interim gzip Transform stream is CPU bound. While there is nothing wrong with it, those familiar with node.js know that being a single threaded (single core) framework, its #1 drawback is I/O bound operations.

There are of course perfectly valid use cases when we knowingly use node.js even when intensive CPU bound operations are involved, for example when running gulp/grunt tasks or even performing image manipulation with GraphicsMagick, as long as we know what we’re doing.

However sometimes the consequences can be harsh on our production environment, because such CPU bound transformations can easily shoot the core’s CPU to 100% for a long time (as long as the stream is flowing) and starve other processes on that machine. Oh, and it doesn’t require heavy computation either; in fact, even a simple string calculation/manipulation can do that to your nodejs app.

One possible solution for not hogging the CPU is to allow node.js’ libuv engine a few ticks / cycles between each intensive calculation, averaging the overall calculation over a longer period of time. This yields a lower CPU footprint for our node.js app (e.g. ~30% CPU instead of ~100%), so that other processes can also run on the core and are not starved.

Here is an example of a simple Transform stream which does exactly that:

// Transform stream - a progress monitor & throttler
var progress = new stream.Transform();
progress._transform = (data, enc, cb) => {
 for (let d of data) {
     if (d === 10) { lineCount++; }
 }

 setTimeout(() => { // allow a 10ms delay before invoking the callback
 cb(null, data);
 }, 10);
};

// create read & write streams
let readStream = fs.createReadStream('big_xml_file.xml');
let saxStream = sax.createStream(false, {});

// hook up the streams: input file => progress (and throttle) => XML SAX parser
readStream.pipe(progress).pipe(saxStream);

The magic happens on the setTimeout line. Instead of immediately invoking the Transform stream’s callback, we’re pending it for 10ms. Because node.js streams are Pull and not Push based (meaning that the receiver pumps data by calling stream._read() every time it can accommodate more data), the readStream is slowed down as well, and the entire application throughput is reduced, leading to reduced CPU usage.

There are of course other solutions, couple of other viable options are:

1. Throttle the input stream by periodically calling stream.pause() and then stream.resume()
2. Use an external tool to limit node’s cpu usage (example cpulimit)

Having said that, the solution proposed above does do the trick and is extremely easy to implement.

– Zacky