Adding and Removing Packages with npm and Yarn

Handling asynchronous tasks efficiently is crucial in Node.js applications, especially when dealing with time-intensive operations like sending emails, processing images, or performing complex calculations. Without proper management, these tasks can block the event loop, leading to poor performance and a subpar user experience. This is where BullMQ comes into play. BullMQ is a powerful Node.js package that offers a reliable and scalable queuing system powered by Redis. It enables developers to transfer heavy operations to a queue in the background, keeping the main application responsive. With BullMQ you can successfully manage async queues, plan processes, and easily keep an eye on their progress. This tutorial will show you how to manage asynchronous tasks with Node.js and BullMQ. The process involves setting up a project folder, performing a time-intensive task without using BullMQ, and enhancing the application by incorporating BullMQ for running tasks in parallel. Prerequisites Before you begin, ensure you: Set up a Linux server. Set up Node.js on your server. Set up Redis on your server, as BullMQ depends on Redis for managing queues. Setting Up the Project Directory Before you can use Node.js and BullMQ for asynchronous tasks, it is necessary to establish your project directory. Set up and customize your Node.js application using these guidelines. Create a New Directory Open your terminal and go to the location of your project. Create a fresh folder and navigate into it: mkdir bullmq-demo && cd bullmq-demo Initialize a New Node.js Project Set up a Node.js project using npm. It generates a package.json file containing the default configurations: npm init -y Install Required Dependencies Set up the required packages for your application: npm install express bullmq ioredis Here's what each package does: express: A fast Node.js web framework commonly used for server creation. bullmq: An excellent tool for handling queues within Node.js programs. ioredis: A Redis client for Node.js that BullMQ needs in order to establish a connection with Redis. Create the Main Application File Create an index.js file as the primary access point for your application: touch index.js Alternatively, you have the option to generate this file by using your code editor. Set Up a Basic Express Server To set up a simple Express server, include this code in your index.js file: const express = require('express'); const app = express(); const port = 3000; app.use(express.json()); app.listen(port, () => { console.log(`Server is running on port ${port}`); }); This code initiates an Express app on port 3000 which handles requests using JSON middleware. Verify the Server Setup Start the server by running: node index.js The below message should appear: Open up your internet browser and go to either http://your_server_ip:3000 or http://localhost:3000. You will receive a "Cannot GET /" message as there are no routes set up, as anticipated. When ready to proceed, you can terminate the server using Ctrl + C. Implementing a Time-Intensive Task Without BullMQ This part describes how to include a route in your Express app that performs a time-consuming task in a synchronous way. This will demonstrate how specific tasks can block the event loop and negatively affect your application's performance. Define a Time-Intensive Function Create a function in the index.js file that simulates a computationally intensive task: // Function to simulate a heavy computation function heavyComputation() { const start = Date.now(); // Run a loop for 5 seconds while (Date.now() - start < 5000) { // Perform a CPU-intensive task Math.sqrt(Math.random()); } } The function runs a loop for about five seconds, performing math operations to mimic a CPU-heavy task. Create a Route to Handle the Task Create a fresh route in your Express application that calls the heavyComputation function: app.get('/heavy-task', (req, res) => { heavyComputation(); res.send('Heavy computation finished'); }); This route is set up to receive GET requests specifically at the /heavy-task endpoint. After receiving a request, it carries out the specified intensive computation and then provides a response. Start the Server To restart your server, execute the following command: node index.js Confirm the server is functioning before moving on to the next stage. Test the Heavy Task Route Open your internet browser and type in either http://your_server_ip:3000/heavy-task or http://localhost:3000/heavy-task to access the webpage.  The following message should be displayed: It is important to observe that the response time is approximately five seconds. The delay is a result of the synchronous execution of the intensive computation process. Observe Blocking Behavior After the server is up and running, open a new tab on your internet browser and go to http://your_server_ip:3000/. The response to this request may not be immediate. The system delays taking action until the extensive processing of the previous step. This happens when the time-consuming task is blocking the Node.js event loop, stopping the server from processing additional incoming requests. When the server performs a task that takes a lot of time in a synchronous manner, it is unable to respond to additional requests. The act of blocking could result in a suboptimal user experience, particularly in apps that need to be highly responsive. Executing Time-Intensive Tasks Asynchronously with BullMQ We saw in the last section how synchronous execution of time-consuming operations can severely affect your application's performance by slowing down the event loop. This section explains how to implement a high-performance asynchronous queue into your application using BullMQ. Modify index.js to Use BullMQ Make changes to the index.js file to include BullMQ in your application. Import BullMQ and ioredis At the top of your index.js file, you should include the following import statements: const { Queue, Worker } = require('bullmq'); const Redis = require('ioredis'); Create a Redis Connection Next, set up a connection with Redis: const connection = new Redis(); Redis has been programmed to run on port 6379 and the localhost interface by default. To create a connection to a remote Redis server that has a different port, please enter the appropriate host address and port number: const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); Initialize a BullMQ Queue Create a new queue called heavyTaskQueue: const heavyTaskQueue = new Queue('heavyTaskQueue', { connection }); Add a Route to Enqueue Tasks Change the heavy-task route to add a job to the queue instead of running the task right away: app.get('/heavy-task', async (req, res) => { await heavyTaskQueue.add('heavyComputation', {}); res.send('Heavy computation job added to the queue'); }); The application will respond after a lengthy process has completed, handling requests asynchronously, when the /heavy-task route is accessed. Remove the Worker Code from index.js The worker must be implemented in a separate file. This is essential to ensure that the worker does not coexist with the Express server process. A worker's use of the heavyComputation function during execution won't interfere with the event loop of the main application. The index.js file is structured in the following way: const express = require('express'); const app = express(); const port = 3000; app.use(express.json()); const { Queue } = require('bullmq'); const Redis = require('ioredis'); const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); const heavyTaskQueue = new Queue('heavyTaskQueue', { connection }); app.get('/heavy-task', async (req, res) => { await heavyTaskQueue.add('heavyComputation', {}); res.send('Heavy computation job added to the queue'); }); app.listen(port, () => { console.log(`Server is running on port ${port}`); }); Create a Separate Worker File Generate a fresh file and name it worker.js. The file is intended for executing the worker code in charge of handling tasks obtained from the queue. Create the worker.js file: touch worker.js Add Worker Code to worker.js: const { Worker } = require('bullmq'); const Redis = require('ioredis'); const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); // Function to simulate a heavy computation function heavyComputation() { const start = Date.now(); // Run a loop for 5 seconds while (Date.now() - start < 5000) { // Perform a CPU-intensive task Math.sqrt(Math.random()); } } const worker = new Worker( 'heavyTaskQueue', async job => { // Time-intensive task here heavyComputation(); console.log('Heavy computation completed'); }, { connection } ); worker.on('completed', job => { console.log(`Job ${job.id} has completed`); }); worker.on('failed', (job, err) => { console.log(`Job ${job.id} has failed with error ${err.message}`); }); Run the Worker in a Separate Process You must now execute worker.js as an independent Node.js process. Start the Worker Process Open a new terminal window or tab, navigate to your project folder, and run the specified command: node worker.js Start the Express Server Initiate the Express server in your original terminal window: node index.js Test the Application with BullMQ Proceed to conduct testing of the application utilizing BullMQ.  Make a Request to /heavy-task:Open your internet browser and type in either http://your_server_ip:3000/heavy-task or http://localhost:3000/heavy-task in the URL bar. The following message should be displayed: Heavy computation job added to the queue. The rapid response time suggests that there is no blockage in the main thread. Adding a Dashboard to Monitor BullMQ Queues Monitoring your application's queues and jobs is essential for ensuring they are functioning properly and for troubleshooting purposes. BullMQ comes with a functionality called Bull Board, which offers a visual interface for overseeing your queues. This part explains how to incorporate a dashboard into your application. Install Bull Board Use npm to install the @bull-board/express package: npm install @bull-board/express Set Up Bull Board in Your Application In order to set up the bull board application, follow these steps: Import Bull Board Modules Insert the code provided at the top of your index.js file: const { createBullBoard } = require('@bull-board/api'); const { BullMQAdapter } = require('@bull-board/api/bullMQAdapter'); const { ExpressAdapter } = require('@bull-board/express'); Create an Express Adapter for the Dashboard Initialize the Express adapter: const serverAdapter = new ExpressAdapter(); serverAdapter.setBasePath('/admin/queues'); Set Up Bull Board with Your Queue Create the Bull Board instance and pass your queue: createBullBoard({ queues: [new BullMQAdapter(heavyTaskQueue)], serverAdapter: serverAdapter, }); Use the Dashboard in Your Express App Add the following line to mount the dashboard at /admin/queues: app.use('/admin/queues', serverAdapter.getRouter()); Make sure to include this line following the setup of your queue and worker. The final index.js file looks like below: // Import Express and Initialize App const express = require('express'); const app = express(); const port = 3000; app.use(express.json()); // Import BullMQ and Redis const { Queue } = require('bullmq'); const Redis = require('ioredis'); // Redis Connection const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); // Initialize Queue const heavyTaskQueue = new Queue('heavyTaskQueue', { connection }); // Define Route to Add Job to Queue app.get('/heavy-task', async (req, res) => { await heavyTaskQueue.add('heavyComputation', {}); res.send('Heavy computation job added to the queue'); }); // Import Bull Board and Set Up Dashboard const { createBullBoard } = require('@bull-board/api'); const { BullMQAdapter } = require('@bull-board/api/bullMQAdapter'); const { ExpressAdapter } = require('@bull-board/express'); const serverAdapter = new ExpressAdapter(); serverAdapter.setBasePath('/admin/queues'); createBullBoard({ queues: [new BullMQAdapter(heavyTaskQueue)], serverAdapter: serverAdapter, }); app.use('/admin/queues', serverAdapter.getRouter()); // Start the Server app.listen(port, () => { console.log(`Server is running on port ${port}`); }); Access the Dashboard To access the dashboard, follow the steps listed below: Restart Your Server node index.js Navigate to the Dashboard Open your browser and go to http://your_server_ip:3000/admin/queues. Explore the Dashboard: Queue Overview: See the list of queues and their status. Jobs List: View active, completed, failed, and delayed jobs. Job Details: Click on a job to see its data, logs, and stack trace if it failed. You can easily manage your BullMQ queues by integrating Bull Board into your application. It is much easier to keep an eye on progress and identify issues when you can view your queues and tasks on the dashboard in real-time. Conclusion You have now learned how to use BullMQ with Node.js to manage asynchronous processes. Your application's responsiveness and efficiency have been enhanced by moving time-consuming operations to a separate queue. Your Node.js app is now much more capable of handling heavy demands thanks to the usage of queues.

The popular JavaScript runtime Node.js enables server-side programming with JavaScript. NPM, a package manager for Node.js projects, helps with dependency management. This guide will show how to install NPM and Node.js on Ubuntu 24.04. Prerequisites System running in Ubuntu 24.04 Root access or user with sudo privileges Installing Node.js and npm from the Default Ubuntu Repository Update the package lists to ensure to have the most recent information on package versions and dependencies. Run the command below:  sudo apt update && sudo apt upgrade -y Node.js is normally available from Ubuntu's default repository. Install it by running the following command: sudo apt install nodejs npm  -y Installing Node.js and npm via the NodeSource Repository Add the NodeSource repository for Node.js:  curl -fsSL https://deb.nodesource.com/setup_20.x | sudo bash -  Replace setup_20.x with the desired version. Different version can be found on nodesource.com. Use the following command to install Node.js after adding the NodeSource repository: sudo apt install nodejs -y Verifying the Node.js and npm Installation Verify the following versions of Node.js and npm to make sure they were installed correctly. Run the below command. node -v npm version Installing Specific Node.js Versions with NVM  With the help of the robust utility Node Version Manager (NVM), devops may easily manage several Node.js versions on a single machine. This is very helpful when switching between several project needs. To install NVM, download and run the installation script from the NVM repository using the following command: curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash After running the scripts, source the user profile and add NVM to the shell session. Add the following lines to the user's home directory (~/.bashrc, ~/.zshrc, or the corresponding shell profile script). Create it using nano editor: nano ~/.bashrc 3. Add the following content: export NVM_DIR="$HOME/.nvm"[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" Run the command below so changes will take effect. source ~/.bashrc With NVM installed, install the specific versions of Node.js. In this case, to install Node.js version 16, run the command below: nvm install 16 Switch to a specific version of Node.js that is installed, using the command below. nvm use 16 Managing Node.js Projects Several essential procedures and best practices are involved in managing Node.js projects in order to ensure the effectiveness, maintainability, and scalability of the application. This is a tutorial to help to efficiently manage your Node.js projects. Launch the terminal, navigate to the project creation path, and make a folder named after the project you are creating. mkdir my_project Initiate the Node project by running the command npm init. Provide the required details (marked in red). All of the answers in this example will be default. The file package.json will result from this. npm init Install any required dependencies, such as nodemon and express. The package-lock.json file and the node_modules folder will be created as a result. npm i express nodemon To initialize git in the project, use the git init command. This will include the file .gitignore. git init Make a file called Readme.md that will have all of the project's information. touch Readme.md Make a file with the .env extension that will hold the project's credentials and sensitive data. touch process.env To launch the program, create a file with the name app.js or index.js. touch app.js Make two folders: Public (which contains resources and static files) and src (which contains controllers, models, routes, and views). mkdir Public src Check each and every folder and file that was generated. This is how a typical structure might look like. For the NODE JS application, it is best practice to establish a project structure, divide files based on their function, and place those files in the appropriate directories. To make it simple to confirm the existence and logic of any given file or folder, unify the application's naming conventions and include business logic in the controllers folder, for instance. ls -lrt Best Practices for Node JS Project Structure For production systems, set up logging and monitoring with tools like Datadog or New Relic. Plan routine maintenance activities including performance reviews, security audits, and dependency updates. Put in place a backup plan for important configurations and data. Check for security flaws in your dependencies and code on a regular basis. Troubleshooting Common Issues There are some frequent problems that a user could run into when installing npm and Node.js. These troubleshooting instructions should help you to address the majority of typical problems that arise when installing npm and Node.js. The steps for troubleshooting these issues are listed below: When attempting to install Node.js or npm globally (i.e., using sudo), users get permission-related issues that prevent them from finishing the installation process. After installing nvm, the command is not recognized. The error nvm Command Not Found will be encountered. Make sure that the shell's configuration file (.bashrc, .bash_profile, .zshrc, etc.) has nvm sourced, and then the command source ~/.bashrc has been use to reload it. The npm version is out of date or does not correspond with the Node.js version after installing Node.js. Use nvm install <version> to install a particular Node.js version, which will include the matching npm version, and manually update npm by running npm install -g npm.  Conclusion In conclusion, an important initial step in creating new web applications and utilizing server-side JavaScript is installing Node.js and npm. Although installing software is usually simple, there are a few frequent problems that can arise, such as permissions conflicts, environment setup problems, or version mismatches. One can successfully overcome these problems by configuring npm to be compatible with your network environment, modifying system settings for global installations, and managing Node.js versions with tools like nvm. Do not forget to update npm and Node.js frequently to take advantage of the newest features and security updates. It will have a strong base for developing and implementing Node.js-powered, scalable applications with correct setup and troubleshooting.

Node Version Manager (NVM) is a powerful tool designed to manage multiple versions of Node.js on a single machine. It simplifies the process of installing, updating, and switching between different versions of Node.js, making it an essential tool for developers working on various projects with different Node.js requirements. Benefits of Using NVM Version Management: Easily install and switch between different versions of Node.js. Environment Isolation: Maintain separate Node.js versions for different projects. Convenience: Simplifies testing and development by providing quick version switching. Installing NVM To install NVM, follow these steps: 1. Download and Install Script Open a terminal and run the following command to download and install NVM: curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash Output: 2. Load NVM After the installation, you need to add NVM to your shell profile. Run the following command: export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"   [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" 3. Verify Installation Ensure that NVM is installed correctly by checking its version: nvm --version Output: Installing Node.js Versions with NVM With NVM installed, you can now install different versions of Node.js. 1. Install a Specific Version To install a specific version of Node.js, use the following command: nvm install <version> Replace <version> with the desired Node.js version number (e.g., 14.17.0). Output: 2. Install the Latest LTS Version To install the latest Long-Term Support (LTS) version of Node.js, run: nvm install --lts Output: Switching Between Node.js Versions NVM allows you to switch between installed Node.js versions easily. 1. List Installed Versions To view all installed Node.js versions, use: nvm ls Output: 2. Switch to a Specific Version To switch to a specific version, run: nvm use <version> Replace <version> with the version number you want to use. Setting a Default Node.js Version You can set a default Node.js version that will be used in all new shell sessions. Set Default Version To set a default Node.js version, use: nvm alias default <version> Replace <version> with the desired Node.js version number. Uninstalling Node.js Versions with NVM NVM also provides an easy way to uninstall Node.js versions that are no longer needed. Uninstall a Specific Version To uninstall a specific Node.js version, run: nvm uninstall <version> Replace <version> with the version number you want to uninstall. Conclusion Using NVM (Node Version Manager) simplifies the management of multiple Node.js versions, providing a convenient and efficient workflow for developers. By following the steps outlined in this tutorial, you can easily install, switch, and manage different Node.js versions on your system. This tutorial provides a comprehensive guide to installing and using NVM, ensuring a smooth and efficient Node.js version management experience.