Making Sense of Node.JS Callbacks

Shifting from a straightforward programming language like C++ to an asynchronous, free-for-all one like Node.JS is hard. Not only do you have to learn a new syntax, but you also have to completely change the way you think and code. The hardest part of the transition for me was to understand callbacks — it took me quite a while to figure them out. So in this article, I’m going to delve into what I hope is a simple, intuitive explanation of the topic without any overly technical terms getting in the way.

Most languages like C++ or Java are sequentially or synchronously written. Things in the code happen in a fixed order. Here’s a simple example:

fileContents = readFile (pathToFile);
print (fileContents);

In the above example, the execution pauses at the first line and waits until the file read is complete. Only then does it move on to the next line to print the contents of the file. In other words, the program simply stops whenever something needs to be done, and waits for it to finish. While this may be fine for CPU intensive applications that are executed locally, this is an extremely poor approach to designing real-time applications like servers and webpages. For example, if each element on the NYTimes website had to be loaded one after the other, we’d be waiting for minutes for the whole thing to finish rendering.

So, how do we fix this?

Check out the following piece of code.

readFile (pathToFile, print);

Here, not only am I passing the file path to the read function, but I’m also passing the print function itself! So, when the file read is complete, the contents of the file are sent to the callback function, which in this case is the print function. Think about this for a second. Instead of executing the print function here in the main code, we pass it off to the readFile function to be executed at a later time. But is our problem of waiting around for something to finish solved? Not really — let’s take a look at one definition of the readFile function.

function readFile (path, callBack) {
    //code to read file from path and transfer it to fileContents
    //code to call the callBack function and send fileContents to it
}

This is still synchronous! When I call the readFile function, the body of the function is still executed synchronously (in order), so we haven’t solved our problem yet. We still have to wait for the read to complete, after which the print (callback) function is called. Only then can we move to the next line in the main code.

The lesson we take from this example is that simply using a callback function doesn’t make our code run out of order (asynchronously) and more efficiently. A callback function is simply that, a function that is called later — “Hey, I’ll call you back later.” So, what then is the deal with callback functions? Why are they so special? Let’s find out.

Consider a small restaurant called ‘Single threaded Async’, on a bad day. There’s only one waiter in the entire restaurant — no cooks, no dishwashers, no servers — he has to do everything himself. Our example above is similar to this restaurant, it doesn’t matter if the waiter shuffles things around — regardless of what order things happen in, he still has to complete each task by himself. The customers have to wait for a long time and are left dissatisfied even though the waiter works as efficiently as he possibly can by himself.

Let’s look at another restaurant called ‘Multi-threaded Async’ where there’s a cook, a dishwasher, and other staff members. In this restaurant, the waiter delegates his work. He moves around the restaurant taking orders from the customers and conveys it to the cooks and assistants, who operate behind the scenes in the kitchen. The waiter can thus serve his customers in real-time, responding to their requests immediately and keeping them happy.

The second restaurant is pretty much how Node.JS works. While it is still single threaded (one waiter), it has a behind-the-scenes worker thread pool that handles file input/output and network connections — the hard, time intensive work. The main thread of Node.JS smartly delegates its work to its worker threads so that it remains free to handle real-time requests. And this is the magic of Node.JS and the reason behind why it’s used so widely. It can handle asynchronous programming at a high level and abstracts away the yucky code that’s needed to make it happen under the hood. And as a bonus, since Node.JS only has a single main thread, we don’t even have to worry about the host of complex problems that come with having to manage multiple threads at the development level.

Coming back to our question, why are callbacks special? Because they act as reminders when the main thread sends away something to the worker threads. A callback is like the bell that a cook rings when a dish is done, it basically tells the main thread (the waiter) that the operation is over and that the output is ready to be served.

The main takeaway here is that when an asynchronous code is implemented on a multi-threaded system, the program doesn’t wait around for one process to be complete before moving to the next. Multiple things can happen at the same time. And the way that the main thread manages not to lose track of all the operations is by using callback functions, which we can think of as reminders.

A few points to note:

A call back function is passed as an argument to another function, but this doesn’t automatically imply execution of the callback. To successfully implement a callback, it has to be called inside the initial definition of the function it’s being passed to. For example:

firstThis (data, callBack) {
    data = 1;
    callBack (data);
}

firstThis (x, anyFunction);

An API is an interface between an application and a program. The main thread can offload operations to an API too. APIs are offered not only by services on the Internet like Google Maps or Instagram but also locally by the browser itself. Chrome for example, offers APIs to handle everything from storage of user data to configuring WiFi connections.
Node.JS is not a programming language. It’s a runtime environment (RTE) which means that it’s everything we need to run a program. It uses the Javascript language and runs on Chrome’s V8 Javascript engine. Think of Javascript as a train — it can only run in a special environment like a browser (train tracks) and is primarily built to add dynamic behaviour to webpages. Node.JS on the other hand is like a train that lays its own tracks. It can exist outside of the browser and be used for general purpose programming applications.
The main advantage of Node.JS comes from its asynchronous nature. Running a CPU intensive program using Node effectively takes away that advantage because the main thread gets blocked. Node is ideally best suited for real-time applications where low latency (response delay) and high throughput (data/second) is of the utmost importance.
I’m just reiterating a point here that I repeatedly missed myself: Synchronicity has nothing to do with threading. We confuse using multiple threads with asynchronous programming because multi-threading is the best way to achieve asynchronous execution, but they are still different concepts.
When people call Node.JS single threaded, they are referring to the main thread that handles the tasks. The thread pool that handles file I/O and the network requests is comprised of more worker threads (implemented under-the-hood of Node, using the ‘libuv’ C library.)

Useful links and videos:

Overview of Blocking vs Non-Blocking | Node.js
nodejs.orgHow Node and Javascript handle asynchronous functions
During the short time that I’ve been studying Javascript, I’ve been confused about some (many) things. What is a…medium.comJavaScript Callbacks Explained Using Minions
Callbacks. Asynchronous. Non-blocking.medium.freecodecamp.org