Handling Consumer Failures in Highland.js

Highland.js is a library that allows for easy stream handling in Node.js or the browser. I recently had to research it for an application that needed to gather an unknown number of events from the browser and push them in batches of 10 to the server, while also making sure ALL of them were pushed successfully.

So I created a Proof of Concept in JSFiddle to experiment a bit. The solution I came up with simply retries the POST requests that failed until they’re successful – in a real-word application you’d probably want to retry a few times and then desist. Unfortunately I couldn’t find many articles about Highland and the documentation is good but needs more complete examples in my opinion. So anyway, here’s what I came up with:

Open Fiddle

Of course this is a work in progress and I still have a long way to go. For example I’d like to experiment if I can re-append failed elements to the end of the stream.

If you used Highland before and know of a better way to do this please feel free to share your experience in the comments!