Installing and Using NVM (Node Version Manager)

Handling asynchronous tasks efficiently is crucial in Node.js applications, especially when dealing with time-intensive operations like sending emails, processing images, or performing complex calculations. Without proper management, these tasks can block the event loop, leading to poor performance and a subpar user experience. This is where BullMQ comes into play. BullMQ is a powerful Node.js package that offers a reliable and scalable queuing system powered by Redis. It enables developers to transfer heavy operations to a queue in the background, keeping the main application responsive. With BullMQ you can successfully manage async queues, plan processes, and easily keep an eye on their progress. This tutorial will show you how to manage asynchronous tasks with Node.js and BullMQ. The process involves setting up a project folder, performing a time-intensive task without using BullMQ, and enhancing the application by incorporating BullMQ for running tasks in parallel. Prerequisites Before you begin, ensure you: Set up a Linux server. Set up Node.js on your server. Set up Redis on your server, as BullMQ depends on Redis for managing queues. Setting Up the Project Directory Before you can use Node.js and BullMQ for asynchronous tasks, it is necessary to establish your project directory. Set up and customize your Node.js application using these guidelines. Create a New Directory Open your terminal and go to the location of your project. Create a fresh folder and navigate into it: mkdir bullmq-demo && cd bullmq-demo Initialize a New Node.js Project Set up a Node.js project using npm. It generates a package.json file containing the default configurations: npm init -y Install Required Dependencies Set up the required packages for your application: npm install express bullmq ioredis Here's what each package does: express: A fast Node.js web framework commonly used for server creation. bullmq: An excellent tool for handling queues within Node.js programs. ioredis: A Redis client for Node.js that BullMQ needs in order to establish a connection with Redis. Create the Main Application File Create an index.js file as the primary access point for your application: touch index.js Alternatively, you have the option to generate this file by using your code editor. Set Up a Basic Express Server To set up a simple Express server, include this code in your index.js file: const express = require('express'); const app = express(); const port = 3000; app.use(express.json()); app.listen(port, () => { console.log(`Server is running on port ${port}`); }); This code initiates an Express app on port 3000 which handles requests using JSON middleware. Verify the Server Setup Start the server by running: node index.js The below message should appear: Open up your internet browser and go to either http://your_server_ip:3000 or http://localhost:3000. You will receive a "Cannot GET /" message as there are no routes set up, as anticipated. When ready to proceed, you can terminate the server using Ctrl + C. Implementing a Time-Intensive Task Without BullMQ This part describes how to include a route in your Express app that performs a time-consuming task in a synchronous way. This will demonstrate how specific tasks can block the event loop and negatively affect your application's performance. Define a Time-Intensive Function Create a function in the index.js file that simulates a computationally intensive task: // Function to simulate a heavy computation function heavyComputation() { const start = Date.now(); // Run a loop for 5 seconds while (Date.now() - start < 5000) { // Perform a CPU-intensive task Math.sqrt(Math.random()); } } The function runs a loop for about five seconds, performing math operations to mimic a CPU-heavy task. Create a Route to Handle the Task Create a fresh route in your Express application that calls the heavyComputation function: app.get('/heavy-task', (req, res) => { heavyComputation(); res.send('Heavy computation finished'); }); This route is set up to receive GET requests specifically at the /heavy-task endpoint. After receiving a request, it carries out the specified intensive computation and then provides a response. Start the Server To restart your server, execute the following command: node index.js Confirm the server is functioning before moving on to the next stage. Test the Heavy Task Route Open your internet browser and type in either http://your_server_ip:3000/heavy-task or http://localhost:3000/heavy-task to access the webpage.  The following message should be displayed: It is important to observe that the response time is approximately five seconds. The delay is a result of the synchronous execution of the intensive computation process. Observe Blocking Behavior After the server is up and running, open a new tab on your internet browser and go to http://your_server_ip:3000/. The response to this request may not be immediate. The system delays taking action until the extensive processing of the previous step. This happens when the time-consuming task is blocking the Node.js event loop, stopping the server from processing additional incoming requests. When the server performs a task that takes a lot of time in a synchronous manner, it is unable to respond to additional requests. The act of blocking could result in a suboptimal user experience, particularly in apps that need to be highly responsive. Executing Time-Intensive Tasks Asynchronously with BullMQ We saw in the last section how synchronous execution of time-consuming operations can severely affect your application's performance by slowing down the event loop. This section explains how to implement a high-performance asynchronous queue into your application using BullMQ. Modify index.js to Use BullMQ Make changes to the index.js file to include BullMQ in your application. Import BullMQ and ioredis At the top of your index.js file, you should include the following import statements: const { Queue, Worker } = require('bullmq'); const Redis = require('ioredis'); Create a Redis Connection Next, set up a connection with Redis: const connection = new Redis(); Redis has been programmed to run on port 6379 and the localhost interface by default. To create a connection to a remote Redis server that has a different port, please enter the appropriate host address and port number: const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); Initialize a BullMQ Queue Create a new queue called heavyTaskQueue: const heavyTaskQueue = new Queue('heavyTaskQueue', { connection }); Add a Route to Enqueue Tasks Change the heavy-task route to add a job to the queue instead of running the task right away: app.get('/heavy-task', async (req, res) => { await heavyTaskQueue.add('heavyComputation', {}); res.send('Heavy computation job added to the queue'); }); The application will respond after a lengthy process has completed, handling requests asynchronously, when the /heavy-task route is accessed. Remove the Worker Code from index.js The worker must be implemented in a separate file. This is essential to ensure that the worker does not coexist with the Express server process. A worker's use of the heavyComputation function during execution won't interfere with the event loop of the main application. The index.js file is structured in the following way: const express = require('express'); const app = express(); const port = 3000; app.use(express.json()); const { Queue } = require('bullmq'); const Redis = require('ioredis'); const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); const heavyTaskQueue = new Queue('heavyTaskQueue', { connection }); app.get('/heavy-task', async (req, res) => { await heavyTaskQueue.add('heavyComputation', {}); res.send('Heavy computation job added to the queue'); }); app.listen(port, () => { console.log(`Server is running on port ${port}`); }); Create a Separate Worker File Generate a fresh file and name it worker.js. The file is intended for executing the worker code in charge of handling tasks obtained from the queue. Create the worker.js file: touch worker.js Add Worker Code to worker.js: const { Worker } = require('bullmq'); const Redis = require('ioredis'); const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); // Function to simulate a heavy computation function heavyComputation() { const start = Date.now(); // Run a loop for 5 seconds while (Date.now() - start < 5000) { // Perform a CPU-intensive task Math.sqrt(Math.random()); } } const worker = new Worker( 'heavyTaskQueue', async job => { // Time-intensive task here heavyComputation(); console.log('Heavy computation completed'); }, { connection } ); worker.on('completed', job => { console.log(`Job ${job.id} has completed`); }); worker.on('failed', (job, err) => { console.log(`Job ${job.id} has failed with error ${err.message}`); }); Run the Worker in a Separate Process You must now execute worker.js as an independent Node.js process. Start the Worker Process Open a new terminal window or tab, navigate to your project folder, and run the specified command: node worker.js Start the Express Server Initiate the Express server in your original terminal window: node index.js Test the Application with BullMQ Proceed to conduct testing of the application utilizing BullMQ.  Make a Request to /heavy-task:Open your internet browser and type in either http://your_server_ip:3000/heavy-task or http://localhost:3000/heavy-task in the URL bar. The following message should be displayed: Heavy computation job added to the queue. The rapid response time suggests that there is no blockage in the main thread. Adding a Dashboard to Monitor BullMQ Queues Monitoring your application's queues and jobs is essential for ensuring they are functioning properly and for troubleshooting purposes. BullMQ comes with a functionality called Bull Board, which offers a visual interface for overseeing your queues. This part explains how to incorporate a dashboard into your application. Install Bull Board Use npm to install the @bull-board/express package: npm install @bull-board/express Set Up Bull Board in Your Application In order to set up the bull board application, follow these steps: Import Bull Board Modules Insert the code provided at the top of your index.js file: const { createBullBoard } = require('@bull-board/api'); const { BullMQAdapter } = require('@bull-board/api/bullMQAdapter'); const { ExpressAdapter } = require('@bull-board/express'); Create an Express Adapter for the Dashboard Initialize the Express adapter: const serverAdapter = new ExpressAdapter(); serverAdapter.setBasePath('/admin/queues'); Set Up Bull Board with Your Queue Create the Bull Board instance and pass your queue: createBullBoard({ queues: [new BullMQAdapter(heavyTaskQueue)], serverAdapter: serverAdapter, }); Use the Dashboard in Your Express App Add the following line to mount the dashboard at /admin/queues: app.use('/admin/queues', serverAdapter.getRouter()); Make sure to include this line following the setup of your queue and worker. The final index.js file looks like below: // Import Express and Initialize App const express = require('express'); const app = express(); const port = 3000; app.use(express.json()); // Import BullMQ and Redis const { Queue } = require('bullmq'); const Redis = require('ioredis'); // Redis Connection const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); // Initialize Queue const heavyTaskQueue = new Queue('heavyTaskQueue', { connection }); // Define Route to Add Job to Queue app.get('/heavy-task', async (req, res) => { await heavyTaskQueue.add('heavyComputation', {}); res.send('Heavy computation job added to the queue'); }); // Import Bull Board and Set Up Dashboard const { createBullBoard } = require('@bull-board/api'); const { BullMQAdapter } = require('@bull-board/api/bullMQAdapter'); const { ExpressAdapter } = require('@bull-board/express'); const serverAdapter = new ExpressAdapter(); serverAdapter.setBasePath('/admin/queues'); createBullBoard({ queues: [new BullMQAdapter(heavyTaskQueue)], serverAdapter: serverAdapter, }); app.use('/admin/queues', serverAdapter.getRouter()); // Start the Server app.listen(port, () => { console.log(`Server is running on port ${port}`); }); Access the Dashboard To access the dashboard, follow the steps listed below: Restart Your Server node index.js Navigate to the Dashboard Open your browser and go to http://your_server_ip:3000/admin/queues. Explore the Dashboard: Queue Overview: See the list of queues and their status. Jobs List: View active, completed, failed, and delayed jobs. Job Details: Click on a job to see its data, logs, and stack trace if it failed. You can easily manage your BullMQ queues by integrating Bull Board into your application. It is much easier to keep an eye on progress and identify issues when you can view your queues and tasks on the dashboard in real-time. Conclusion You have now learned how to use BullMQ with Node.js to manage asynchronous processes. Your application's responsiveness and efficiency have been enhanced by moving time-consuming operations to a separate queue. Your Node.js app is now much more capable of handling heavy demands thanks to the usage of queues.

The popular JavaScript runtime Node.js enables server-side programming with JavaScript. NPM, a package manager for Node.js projects, helps with dependency management. This guide will show how to install NPM and Node.js on Ubuntu 24.04. Prerequisites System running in Ubuntu 24.04 Root access or user with sudo privileges Installing Node.js and npm from the Default Ubuntu Repository Update the package lists to ensure to have the most recent information on package versions and dependencies. Run the command below:  sudo apt update && sudo apt upgrade -y Node.js is normally available from Ubuntu's default repository. Install it by running the following command: sudo apt install nodejs npm  -y Installing Node.js and npm via the NodeSource Repository Add the NodeSource repository for Node.js:  curl -fsSL https://deb.nodesource.com/setup_20.x | sudo bash -  Replace setup_20.x with the desired version. Different version can be found on nodesource.com. Use the following command to install Node.js after adding the NodeSource repository: sudo apt install nodejs -y Verifying the Node.js and npm Installation Verify the following versions of Node.js and npm to make sure they were installed correctly. Run the below command. node -v npm version Installing Specific Node.js Versions with NVM  With the help of the robust utility Node Version Manager (NVM), devops may easily manage several Node.js versions on a single machine. This is very helpful when switching between several project needs. To install NVM, download and run the installation script from the NVM repository using the following command: curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash After running the scripts, source the user profile and add NVM to the shell session. Add the following lines to the user's home directory (~/.bashrc, ~/.zshrc, or the corresponding shell profile script). Create it using nano editor: nano ~/.bashrc 3. Add the following content: export NVM_DIR="$HOME/.nvm"[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" Run the command below so changes will take effect. source ~/.bashrc With NVM installed, install the specific versions of Node.js. In this case, to install Node.js version 16, run the command below: nvm install 16 Switch to a specific version of Node.js that is installed, using the command below. nvm use 16 Managing Node.js Projects Several essential procedures and best practices are involved in managing Node.js projects in order to ensure the effectiveness, maintainability, and scalability of the application. This is a tutorial to help to efficiently manage your Node.js projects. Launch the terminal, navigate to the project creation path, and make a folder named after the project you are creating. mkdir my_project Initiate the Node project by running the command npm init. Provide the required details (marked in red). All of the answers in this example will be default. The file package.json will result from this. npm init Install any required dependencies, such as nodemon and express. The package-lock.json file and the node_modules folder will be created as a result. npm i express nodemon To initialize git in the project, use the git init command. This will include the file .gitignore. git init Make a file called Readme.md that will have all of the project's information. touch Readme.md Make a file with the .env extension that will hold the project's credentials and sensitive data. touch process.env To launch the program, create a file with the name app.js or index.js. touch app.js Make two folders: Public (which contains resources and static files) and src (which contains controllers, models, routes, and views). mkdir Public src Check each and every folder and file that was generated. This is how a typical structure might look like. For the NODE JS application, it is best practice to establish a project structure, divide files based on their function, and place those files in the appropriate directories. To make it simple to confirm the existence and logic of any given file or folder, unify the application's naming conventions and include business logic in the controllers folder, for instance. ls -lrt Best Practices for Node JS Project Structure For production systems, set up logging and monitoring with tools like Datadog or New Relic. Plan routine maintenance activities including performance reviews, security audits, and dependency updates. Put in place a backup plan for important configurations and data. Check for security flaws in your dependencies and code on a regular basis. Troubleshooting Common Issues There are some frequent problems that a user could run into when installing npm and Node.js. These troubleshooting instructions should help you to address the majority of typical problems that arise when installing npm and Node.js. The steps for troubleshooting these issues are listed below: When attempting to install Node.js or npm globally (i.e., using sudo), users get permission-related issues that prevent them from finishing the installation process. After installing nvm, the command is not recognized. The error nvm Command Not Found will be encountered. Make sure that the shell's configuration file (.bashrc, .bash_profile, .zshrc, etc.) has nvm sourced, and then the command source ~/.bashrc has been use to reload it. The npm version is out of date or does not correspond with the Node.js version after installing Node.js. Use nvm install <version> to install a particular Node.js version, which will include the matching npm version, and manually update npm by running npm install -g npm.  Conclusion In conclusion, an important initial step in creating new web applications and utilizing server-side JavaScript is installing Node.js and npm. Although installing software is usually simple, there are a few frequent problems that can arise, such as permissions conflicts, environment setup problems, or version mismatches. One can successfully overcome these problems by configuring npm to be compatible with your network environment, modifying system settings for global installations, and managing Node.js versions with tools like nvm. Do not forget to update npm and Node.js frequently to take advantage of the newest features and security updates. It will have a strong base for developing and implementing Node.js-powered, scalable applications with correct setup and troubleshooting.

PostgreSQL and Node.js are two powerful technologies that, when integrated, can form the backbone of highly efficient, scalable, and robust web applications.  One of the most popular open-source relational database systems, PostgreSQL is well known for its sophisticated features, dependability, and adherence to SQL standards. It provides sophisticated querying capabilities, broad data integrity, and a wide range of functional extensions. Because of its robust indexing, full-text search capabilities, support for ACID transactions, and capacity to manage massive data volumes, PostgreSQL is a popular database management system. Hostman provides pre-configured and ready-to-use cloud databases, including cloud PostgreSQL. Designed to create scalable network applications, Node.js is a JavaScript runtime based on the V8 JavaScript engine found in Chrome. Its lightweight and efficient I/O model is event-driven and non-blocking. Applications requiring real-time functionality and the ability to manage numerous concurrent connections with little overhead are especially well suited for Node.js. The JSON-centric approach of Node.js is enhanced by PostgreSQL's support for JSON and JSONB data types, which facilitates seamless data interchange and manipulation. PostgreSQL's advanced features, like full-text search, spatial data support with PostGIS, and custom data types, can greatly improve Node.js applications. Comprehensive security features are offered by both technologies. Strong data integrity, access control, and encryption are provided by PostgreSQL, and a range of security techniques and plugins are available in Node.js to safeguard application data. Importance of database access in applications Accessing a PostgreSQL database from a Node.js application is a key aspect of modern web and mobile application development. Through this integration, developers may take advantage of the strengths of both platforms to create scalable, secure, and effective applications. Here’s how to access PostgreSQL database from Node.js application:  Requirements A cloud server or a local machine running Ubuntu Root access or user with sudo privilege Node.js and npm installation After all requirements are met, proceed with the Node.js and npm installation. There are several ways to install Node.js and npm on Ubuntu. In this tutorial, the Node.js and npm will be installed on Ubuntu using the Ubuntu apt repository. Login to the terminal and run the command below. sudo apt update && apt upgrade  Run the command below to install Node.js and npm. sudo apt install nodejs npm After successful installation, verify the version by running the commands below respectively: nodejs --version npm --version PostgreSQL setup Install PostgreSQL and configure the database using the instructions below. Install the postgresql package along with its additional utilities. sudo apt install postgresql postgresql-contrib Use the systemctl start command to make sure the service is up and running. sudo systemctl start postgresql.service Verify the status by running the command below. sudo systemctl status postgresql.service Once the package is installed and up, login to PostgreSQL database using the default postgres user.  sudo -u postgres psql Create a new database.   CREATE DATABASE mydatabase; Enter your database name instead of mydatabase. Switch to the new database that has been created: \c mydatabase Create additional user to be used in the new database.  CREATE USER myuser WITH PASSWORD 'mypassword'; To enable the newly created user 'myuser' to be utilized, create an equivalent OS user. Execute the following command and respond to the prompts accordingly. adduser myuser Lastly, test the connection of the database created using the new user. Run the commands below respectively: sudo su - myuser psql -d mydatabase Creating a Node.js Project After successfully installing and configuring the PostgreSQL database, proceed to create the Node.js project. This project will facilitate the creation of a PostgreSQL database and interaction with npm configuration files. Create directory by running the command below: mkdir nodejs-pg-mydir Go to the newly created directory named nodejs-pg-mydir. Run the command below: cd nodejs-pg-mydir Initialize the directory by running the command below: npm init -y Installing Necessary Packages (pg) After creating the Node.js project, it's time to install the node-postgres module using the following command. npm install pg   Configuration Details To access the database using the application, follow these steps: Before verifying the connection, create a table on the database by running the command below. CREATE TABLE mytable( id SERIAL PRIMARY KEY, name VARCHAR(50) NOT NULL, color VARCHAR(50) NOT NULL);   Testing the Example Connection Exit the database. The command prompt will go back to the project directory. Create a file called database.js in the nodejs-pg-mydir folder using nano virtual editor. Run the command nano database.js and paste the following code into it then save and exit. const { Pool } = require("pg"); const pool = new Pool({ user: "myuser", database: "mydatabase", password: "mypassword", port: 5432, host: "localhost", }); module.exports = { pool }; Note: Make sure to change 'user', 'database', and 'password' with the real database credentials. The module.exports = { pool }; line at the end of the file means that the pool variable can be used anywhere in the app. Test the connection by inserting data into the PostgreSQL database using the connection pool database.js, that was created on the previous step. Create a file datainsert.js and copy the below code. Save the file and exit. nano datainsert.js const { pool } = require("./database"); async function datainsert() { const [name, color] = process.argv.slice(2); const res = await pool.query( "INSERT INTO mytable (name, color) VALUES ($1, $2)", [name, color] ); console.log(`Added a mytable with the name ${name}`); } datainsert() Use the node command below to execute the file datainsert.js with argument name and color. node datainsert.js jcbrian brown The connection was successful if the output is similar to above. This means that the application can access the database with the table name mytable. To perform another test, try to retrieve data from the Postgres database and log them into the console. Create a file name dataretrieve.js and put the following code. Save and exit. nano dataretrieve.js const { pool } = require("./database"); async function dataretrieve() { try { const res = await pool.query("SELECT * FROM mytable"); console.log(res.rows); } catch (error) { console.error(error); } } dataretrieve() Run the command below to query on the database using Node.js. node dataretrieve.js  An output similar to above indicates that the query is completed without any error. Conclusion In conclusion, developers have an extensive number of options when integrating a PostgreSQL database with a Node.js application. Robust and scalable web applications can be developed because of the efficient data storage, retrieval, and manipulation made possible by the smooth connection between these technologies. Developers can leverage the flexibility of Node.js and the powerful features of PostgreSQL to create dynamic, data-driven apps that fit the needs of modern software development by following the methods suggested in this article. Accessing PostgreSQL via Node.js offers a significant benefit in developing high-performance applications when the appropriate tools and skills are applied.