What will we build?
ToDo applications are a go-to project for beginners learning front-end development. This is why we decided to build a Todo list API. This will enable us to add data persistence to our interface, and give us the ability to manipulate this data (by adding, updating, deleting todos, etc….).
The final code can be found here.
We will use a simplified tech stack for this project. It can be considered as a minimal version of a lot of tools you will find in real-life projects, the reason being that the higher-level ideas are the same. The details of implementation and choice of a specific tool over another are not important to get started.
LowDB is a simple in-memory database. Its simplicity allows us to showcase how to interact with a database in a NodeJs project, without dealing with more advanced subjects like deployments and configurations.
Now that we identified all the tools that we will use, let's get to our keyboards and start coding!
Node is available on all platforms. All installation guides can be found on the official website. Windows users should make sure to add node path to environment variables so it can be used on the command line.
We will also need npm installed. Npm is the standard package manager for NodeJs. It will enable us to manage our project dependencies. The installation guide can be found here.
Head over to the link and clone the starter project:
This is a simple starter repository for our project. It contains all the dependencies we will use along with the project file structure. We will explain each element once reached. Open your terminal, navigate to the path of the project and run the command:
After all dependencies are installed, we can start our application:
npm run start
“start” is a script that we specified in the package. json file. It specifies the entry file to our application, which in our case is app.js. The following message should now appear in your terminal:
This means that our server started successfully, and is listening for any requests sent to port 3000. Let's look at app.js and explain what is happening here:
App.js is our project entry file (and the only one at this point). We instantiate an express application named app, specify that all requests that have the http method “GET” and the subpath ‘/’ will be handled by this route, pass in a function called a middleware, which takes in the request and response object as parameters. This is crucial, since the request contains all the information needed for it to be handled (parameters, request body, request headers, etc..), and the response object is the one that will be returned to the client. We start by just send the “Hello world” message. After that we make our application listen to any incoming requests to the port specified (in our case 3000), and log the message “Listening to port 3000” to indicate that our application is up and running and ready to receive requests.
Open your terminal and in the link bar type “localhost:3000/”, and hit Enter. This is the specified path that we can use to reach our server locally. You will receive the following message:
Lowdb is an open-source database that’s easy to use, and requires no specific setup. The general idea behind it is to store all data in a local json file. After LowDB is installed (which has been done when we installed all the dependencies), we can add the following code to db.js:
This code is quite similar to the one found on LowDB's official documentation. We tweaked it a little bit for our own use case. Let's explain it line by line:
First, we specify the path for the db.json file. It's the file that will contain our data, and be passed to LowDB. If the file is not found at the specified path, LowDB will create one. We then pass the file path to the LowDB adapter, and pass it to our new LowDB database instance. The “db” variable can then be used to communicate with our database. Once the database instance is created, we read from the json file by using db.read(). This will set the “data” field in our database instance so we can access the database content. Notice that we preceded this line with “await”. This is to specify that this instruction may take an unknown amount of time to resolve, and that the NodeJs process must wait for its execution before proceeding with the rest of the code. We do this because the reading operation requires memory access to the specified file, and the time to execute this kind of operation depends on your machine specifications. Now that we have access to the data field, we set it as an object containing an empty array of posts, or rather, we check if the file contains any prior data and set the empty array if it is not the case. Finally, we execute db.write() to apply the modifications we made to the data, and export the database instance so it can be used in other files in our project.
General Request/Response workflow
Consider the following diagram:
It shows the general architecture applied in a plethora of backend applications built with NodeJs/Express. Understanding the general workflow behind handling a request will not only allow you to build and structure NodeJs applications, but also enable you to transfer these concepts into practically any technical stack of your choice. We will explore the different layers that interfere with this process, and explain their roles:
## HTTP Request Layer
This is the first layer in our application, imagine it as a gateway that receives a wide range of different requests coming from different clients, each request is then analyzed and forwarded to the dedicated part of the application for it to be handled.
- Routers: here we are referring to Express routers, but this concept can be found in many backend frameworks. Routers are a way to apply the logical distribution in our business logic to our code, meaning that each set of elements that share similar features are handled by the same entry, and can be separated from the rest of the sets. This has the benefit of making each component of the code independent from the others, and easier to maintain and extend. More specifically, and as an example, all requests that fulfill conditions of the shared url path “/posts” will be handled by the same router. Depending on their http method (GET, POST, etc.. ), a different controller will be used.
- Controllers: a controller receives filtered requests from routers, applies additional processing, and calls the suitable service methods.
Business Logic Layer
This layer is unique depending on the specific use cases of the application, and the business logic behind it.
Services: Services are a set of methods that contain the core logic of the application. They also interact with the database through the use of ORMs/ODMs.).
Third-party services: many modern applications choose to delegate a part of the application’s logic to dedicated services accessible through an API. Services of this sort can be payment handling services, static files storage, notifications, and others.
ODM/ORM: ORMs and ODMs act as middlemen between the services and the database. Their role is to provide a high-level abstraction upon a database that allows a developer to write code in the programming language of their choice instead of dedicated database languages, such as SQL.
## Data Persistence Layer
- Databases: as we mentioned earlier, almost all applications need some form of data persistence. This part is handled by databases, and depending on the nature of the data, the business logic, and many other considerations, the choice of a certain database over another is considered crucial for the efficiency and the scalability of the application.
## Example: Adding a Post
Now that we understand the general idea behind the architecture, let’s apply it to our simple example. We will implement the feature of adding a todo post to our application. Let's assume that any post has a unique id that will enable us to identify it later in our database, a title that is a string, and an order that is of type integer. Following our diagram, we will start by implementing the router. Add following code to the index.js file:
This is our router file. We import express and the “addPost” method from our controller (we will implement this one shortly), create an instance of express router, and bind the addPost method to our router - meaning that for each request that has the root path and the http method “POST”, the “addPost” method will be called to handle it.
Before we implement our method in the controller, we reference the new router in our main app.js file, and specify its path as “/posts”: All routes with the specified paths will be forwarded to this router, so it can be handled by the different controller methods:
We import the router and name it as “posts”. app.use(“/posts”,..) means that all requests with the subpath “/posts”, no matter their http method, will be routed to the specified router. Other changes to app.js include importing the database configuration file in order for it to be executed, and using the express.json() as a middleware to enable us to access the request body object. Now that our routes are set, we can add the “addPost” method in the controller.js file:
“addPost” is a middleware function that takes as parameters the request, response objects, and the next function. When the next function is called, the process will move to the next middleware in the chain, or end the request. In the method’s code, we extract the title and the order from the request body, and pass those as parameters to the service function “createPost”. This function takes the post attributes, creates a new post, and returns it. Once the new post is created, we return it to the client along with the 200 status code, meaning that the request has been successful. You may notice that our code is put inside a try/catch block in order to catch any unexpected error, and pass it to the next middleware. It is considered best practice to attach to all routers an error-handling middleware that extracts the error, and returns a meaningful error message to the client. All that’s left now is to implement the “createPost” function in service.js:
The “createPost” method receives title and order as parameters, and uses them to create the post object. For the unique id, we use a dedicated library called “nanoid”, which generates a unique sequence of characters. We add the new post into the posts array in the database, and write these changes; the new post is then returned by the function.
Now that “createPost” is ready, the adding posts feature is now finished and up and running. We test it using Postman, a popular tool for testing APIs:
We select “POST” as the http method for the request along with the specified url path “localhost:3000/posts”. We add the title and the order as json format in the body section, and send the request. As shown above, we receive the 200 OK status along with the newly created post.
A lot of concepts and ideas have been explored in this project: We covered how to install and set up our project environment, learned how to configure LowDB for local data persistence, explored the general architecture of NodeJS/Express backend applications, and saw how to apply it in a simple example. Finally, we tested our application using Postman.
The intent here was to expose a simplified version of all that goes into building modern backend applications. As we saw earlier, NodeJs is a powerful tool that enables us to build simple and complex APIs. Combined with its rich ecosystem of frameworks, such as express and a plethora of tools and libraries for just about any use case, it is a legit solution for modern backend development - a solution that we recommend to learn and master.