I am a big fan of node.js but I have a big problem with the core team. No need to hide it and pretend everything is OK. The fibers vs. callback war erupted again this week on the mailing list, with new ballistic benchmark missiles fired from both sides and showers of sarcasms.
This is ridiculous. Time to sit down and blog about it!
Node also comes with a good package system that manages dependencies in a very decoupled way and a simple tool to publish, install and update packages. I also like the fact that most components are published under the very liberal MIT license and that Windows, which is important for us, is supported as a first class platform.
Definitely, node.js is an an awesome runtime. I’m not regretting one single day to have chosen it for one of our projects.
So the core team does a great job on the runtime but it seems invested with another mission: evangelizing a new way of programming, based on callbacks, streams and pipes.
It is not hard, you just need to learn callbacks.
Also, some people may tell you that they have found ways to ease your pain, but you should not listen to them; they are just heretics who are trying to divert you from the true and only way to node.
You *have* to learn callbacks!
So you will learn callbacks! And you’ll probably find out that they are not so hard after all. It is mostly a question of knowing a few patterns and applying them. It is more painful than hard. But it is also more error prone; it forces you to write more code; and the code is more difficult to read and modify because of all the “pattern noise”.
You’ll probably think that all this might be fine for a personal project but that it doesn’t scale well to large projects. Costs will rise at all levels: training costs to get people up to speed with callbacks, development costs because of extra code to write and of more difficult code reviews, quality costs because of more fragile code, maintenance costs, etc.
So, you may come back to the mailing list with a question like:
Fine, now I understand callbacks but I still have problems. Isn’t there a better way?
And you’ll get the same answer:
No. Callbacks are perfectly fine! You just need to refactor your logic and you should try to reformulate your problem with streams and pipes.
And don’t listen to the sirens who say that that they have solutions for you and that you shouldn’t bother with callbacks.
You *have* to learn callbacks and streams!
Someone might add:
Your application is probably not a good fit for node. You should have chosen PHP or Ruby instead.
I don’t know what you’ll do at this point. One possibility is that you’ll follow the party line: you will write a stream. It might not solve the problem you were trying to solve in the first place but you’ll be able to post your success story on the mailing list and you’ll get plenty of kudos from the core team.
The worst is that this is hardly a caricature. Node is not just a runtime; it comes with a gospel:
You have to program with callbacks and you have to rethink your application as streams and pipes.
The gospel is wrong!
Asynchronous !== Callbacks
Part of the problem comes from an unfortunate identification between asynchronous programming and callbacks. Asynchronous programming and callbacks are seen by many as one and the same thing. For them, programming asynchronously means programming with callbacks.
This is just plain wrong: asynchronism is the essence of node while callbacks are just an accident of node. They belong to different levels.
Asynchronism is a behavior. It is essential in node because node is all about performance and asynchronous I/O. Synchronous (blocking) I/O is a disaster for node.
And, BTW, callbacks are probably not the best artifact to express asynchronism because they are *not* essentially asynchronous. Just consider the callbacks passed to
Array.prototype.forEach: those are invoked synchronously.
IMO, the artifact that I have introduced in streamline.js (the
_ marker), and that has been the target of so many sarcastic comments on the mailing list, is a better artifact because it captures the essence of asynchronism in code (the points where execution yields). So, in a sense, streamline.js provides a cleaner model for asynchronous programming than callbacks.
Every piece of software is not a proxy
The nirvana of node’s gospel is a program built entirely with streams and pipes. This comes from a vision that every piece of software can be thought of as being some kind of proxy and that systems can be built by connecting streams with pipes. This is a very appealing vision and it would just be so great if it would apply to every domain.
It does not!
The streams and pipes vision will likely work well in domains that are naturally event driven. This is the case of most front-end systems. But what about back-end systems? Most of the time these are more goal driven than event driven: they don’t push much data to other layers; instead, they respond to requests/questions by pulling data from other systems. I don’t see the streams and pipes model fitting too well in this context (but maybe it is just my imagination which is limited).
Does this mean that node.js should be ruled out for back-end systems and that I am just plain wrong because I’m trying to use it in areas where it does not fit? I don’t think so. Back-end systems pull data. This means I/O. And the best way to do I/O is the asynchronous way. So why not node.js?
I wrote all this because I was getting really fed up with the childish attitude on the mailing list and the difficulty to get into interesting discussions.
As I said in the introduction, I’m a big fan of node.js. It is a great runtime and I’ll continue to use it.
I don’t agree with the gospel and I think that it is totally counterproductive.
The simple fact that the “callback hell” question comes back so regularly on the mailing list should raise a red flag. There is a reality behind this question and this reality cannot be ignored forever.
There is a big opportunity for node.js to compete with PHP and the likes but it won’t succeed if it keeps the bar so high.
It is also counterproductive because people don’t use the tools that would allow them to develop 100% asynchronous modules without pain. So, what they end up doing instead is bury a few
Sync calls in their modules, thinking that “well, this is just a little bit of file I/O; it won’t hurt” (I hit two cases like this recently and I had to fork and streamline them to eliminate the
And it is counterproductive because it pollutes the discussions, blurs the message and upsets everyone.