Introduction to NodeJs

Introduction

When trying to learn about web

development, we usually find that front-end is significantly more accessible than back-end. There are many reasons for this, particularly the feeling of instant feedback that comes from changing a certain element of a page in the code, and noticing the change being applied to the website. This feedback is often helpful for beginners, because it enables them to adjust their code and learn from their mistakes. Unfortunately, it is not the case with the backend: oftentimes, a significant amount of work is dedicated to setting up the environment upfront, and installing the dependencies needed in order to get a simple “Hello World" message to appear on the terminal. Luckily, a lot of progress is constantly being made in the open-source community towards facilitating the development process of frameworks, and improving the developer's experience. NodeJs is a good example of this. This framework is a very capable technology that makes writing server-side code in Javascript convenient, and offers a variety of built-in tools and features that differentiate it from its competitors. In this article, we will explore NodeJs and its ecosystem, with a hands-on approach, building a fully functional project.

What will we build?

ToDo applications are a go-to project for beginners learning front-end development. This is why we decided to build a Todo list API. This will enable us to add data persistence to our interface, and give us the ability to manipulate this data (by adding, updating, deleting todos, etc….).

The final code can be found here.

Our tools

We will use a simplified tech stack for this project. It can be considered as a minimal version of a lot of tools you will find in real-life projects, the reason being that the higher-level ideas are the same. The details of implementation and choice of a specific tool over another are not important to get started.

• NodeJs, as we mentioned is one of the most popular Javascript frameworks for building server-side applications.

• ExpressJs is a minimal Javascript framework used on top of NodeJS. It speeds up the development process by using many built-in features. It is also used as a way to standardize the development practices in NodeJS projects in order to facilitate its use for engineers.

• LowDB is a simple in-memory database. Its simplicity allows us to showcase how to interact with a database in a NodeJs project, without dealing with more advanced subjects like deployments and configurations.

Now that we identified all the tools that we will use, let's get to our keyboards and start coding!

Installation

Node is available on all platforms. All installation guides can be found on the official website. Windows users should make sure to add node path to environment variables so it can be used on the command line.

We will also need npm installed. Npm is the standard package manager for NodeJs. It will enable us to manage our project dependencies. The installation guide can be found here.

Project initialization

Head over to the link and clone the starter project:

This is a simple starter repository for our project. It contains all the dependencies we will use along with the project file structure. We will explain each element once reached. Open your terminal, navigate to the path of the project and run the command:

npm install

This will install all the dependencies of the project specified in the package.json file. package.json is the file found at the root of any Javascript/NodeJs project, it contains metadata about the latter and is used to manage all project dependencies, scripts, and versions.

After all dependencies are installed, we can start our application:

npm run start

“start” is a script that we specified in the package. json file. It specifies the entry file to our application, which in our case is app.js.

The following message should now appear in your terminal:

This means that our server started successfully, and is listening for any requests sent to port 3000. Let's look at app.js and explain what is happening here:

App.js is our project entry file (and the only one at this point). We instantiate an express application named app, specify that all requests that have the http method “GET” and the subpath ‘/’ will be handled by this route, pass in a function called a middleware, which takes in the request and response object as parameters. This is crucial, since the request contains all the information needed for it to be handled (parameters, request body, request headers, etc..), and the response object is the one that will be returned to the client. We start by just send the “Hello world” message. After that we make our application listen to any incoming requests to the port specified (in our case 3000), and log the message “Listening to port 3000” to indicate that our application is up and running and ready to receive requests.

Open your terminal and in the link bar type “localhost:3000/”, and hit Enter. This is the specified path that we can use to reach our server locally. You will receive the following message:

Database configuration

Lowdb is an open-source database that’s easy to use, and requires no specific setup. The general idea behind it is to store all data in a local json file. After LowDB is installed (which has been done when we installed all the dependencies), we can add the following code to db.js:

This code is quite similar to the one found on LowDB's official documentation. We tweaked it a little bit for our own use case. Let's explain it line by line:

The first few lines are for importing the necessary dependencies. “join” is a utility function available in the “path” module. It is one of the core modules of NodeJs that offers a lot of methods for dealing with, and handling paths. “Low” and “JSONFile'' are the two classes exposed by LowDB. The first one creates the json file instance that will contain our data. The second one creates the actual database instance that will act upon it. Finally, “lodash” is one of the most used javascript libraries offering a wide variety of utility functions for common programming tasks. We add it to our database instance to enable us to use its advanced methods to handle our data.

First, we specify the path for the db.json file. It's the file that will contain our data, and be passed to LowDB. If the file is not found at the specified path, LowDB will create one.

We then pass the file path to the LowDB adapter, and pass it to our new LowDB database instance. The “db” variable can then be used to communicate with our database. Once the database instance is created, we read from the json file by using db.read(). This will set the “data” field in our database instance so we can access the database content. Notice that we preceded this line with “await”. This is to specify that this instruction may take an unknown amount of time to resolve, and that the NodeJs process must wait for its execution before proceeding with the rest of the code. We do this because the reading operation requires memory access to the specified file, and the time to execute this kind of operation depends on your machine specifications.

Now that we have access to the data field, we set it as an object containing an empty array of posts, or rather, we check if the file contains any prior data and set the empty array if it is not the case.

Finally, we execute db.write() to apply the modifications we made to the data, and export the database instance so it can be used in other files in our project.

General Request/Response workflow

Consider the following diagram:

It shows the general architecture applied in a plethora of backend applications built with NodeJs/Express. Understanding the general workflow behind handling a request will not only allow you to build and structure NodeJs applications, but also enable you to transfer these concepts into practically any technical stack of your choice. We will explore the different layers that interfere with this process, and explain their roles:

## HTTP Request Layer

This is the first layer in our application, imagine it as a gateway that receives a wide range of different requests coming from different clients, each request is then analyzed and forwarded to the dedicated part of the application for it to be handled.

• Routers: here we are referring to Express routers, but this concept can be found in many backend frameworks. Routers are a way to apply the logical distribution in our business logic to our code, meaning that each set of elements that share similar features are handled by the same entry, and can be separated from the rest of the sets. This has the benefit of making each component of the code independent from the others, and easier to maintain and extend. More specifically, and as an example, all requests that fulfill conditions of the shared url path “/posts” will be handled by the same router. Depending on their http method (GET, POST, etc.. ), a different controller will be used.

• Controllers: a controller receives filtered requests from routers, applies additional processing, and calls the suitable service methods.

Business Logic Layer

This layer is unique depending on the specific use cases of the application, and the business logic behind it.

• Services: Services are a set of methods that contain the core logic of the application. They also interact with the database through the use of ORMs/ODMs.).

• Third-party services: many modern applications choose to delegate a part of the application’s logic to dedicated services accessible through an API. Services of this sort can be payment handling services, static files storage, notifications, and others.

• ODM/ORM: ORMs and ODMs act as middlemen between the services and the database. Their role is to provide a high-level abstraction upon a database that allows a developer to write code in the programming language of their choice instead of dedicated database languages, such as SQL.

Data Persistence Layer

• Databases: as we mentioned earlier, almost all applications need some form of data persistence. This part is handled by databases, and depending on the nature of the data, the business logic, and many other considerations, the choice of a certain database over another is considered crucial for the efficiency and the scalability of the application.

Example: Adding a Post

Now that we understand the general idea behind the architecture, let’s apply it to our simple example. We will implement the feature of adding a todo post to our application. Let's assume that any post has a unique id that will enable us to identify it later in our database, a title that is a string, and an order that is of type integer. Following our diagram, we will start by implementing the router. Add following code to the index.js file:

This is our router file. We import express and the “addPost” method from our controller (we will implement this one shortly), create an instance of express router, and bind the addPost method to our router - meaning that for each request that has the root path and the http method “POST”, the “addPost” method will be called to handle it.

Before we implement our method in the controller, we reference the new router in our main app.js file, and specify its path as “/posts”: All routes with the specified paths will be forwarded to this router, so it can be handled by the different controller methods:

We import the router and name it as “posts”. app.use(“/posts”,..) means that all requests with the subpath “/posts”, no matter their http method, will be routed to the specified router.

Other changes to app.js include importing the database configuration file in order for it to be executed, and using the express.json() as a middleware to enable us to access the request body object.

Now that our routes are set, we can add the “addPost” method in the controller.js file:

“addPost” is a middleware function that takes as parameters the request, response objects, and the next function. When the next function is called, the process will move to the next middleware in the chain, or end the request. In the method’s code, we extract the title and the order from the request body, and pass those as parameters to the service function “createPost”. This function takes the post attributes, creates a new post, and returns it. Once the new post is created, we return it to the client along with the 200 status code, meaning that the request has been successful. You may notice that our code is put inside a try/catch block in order to catch any unexpected error, and pass it to the next middleware. It is considered best practice to attach to all routers an error-handling middleware that extracts the error, and returns a meaningful error message to the client.

All that’s left now is to implement the “createPost” function in service.js:

As we mentioned earlier when explaining the different layers of the architecture, the service layer interacts with the data storage solution through the use of ORM/ODM. However, in our example, we won’t need to use a separate ORM, since Lowdb comes with built-in support for Javascript. All details about its syntax can be found in the documentation.

The “createPost” method receives title and order as parameters, and uses them to create the post object. For the unique id, we use a dedicated library called “nanoid”, which generates a unique sequence of characters. We add the new post into the posts array in the database, and write these changes; the new post is then returned by the function.

Now that “createPost” is ready, the adding posts feature is now finished and up and running. We test it using Postman, a popular tool for testing APIs:

We select “POST” as the http method for the request along with the specified url path “localhost:3000/posts”. We add the title and the order as json format in the body section, and send the request. As shown above, we receive the 200 OK status along with the newly created post.

Conclusion

A lot of concepts and ideas have been explored in this project: We covered how to install and set up our project environment, learned how to configure LowDB for local data persistence, explored the general architecture of NodeJS/Express backend applications, and saw how to apply it in a simple example. Finally, we tested our application using Postman.

The intent here was to expose a simplified version of all that goes into building modern backend applications. As we saw earlier, NodeJs is a powerful tool that enables us to build simple and complex APIs. Combined with its rich ecosystem of frameworks, such as express and a plethora of tools and libraries for just about any use case, it is a legit solution for modern backend development - a solution that we recommend to learn and master.