Sign In
Sign In

How to Handle Asynchronous Tasks with Node.js and BullMQ

How to Handle Asynchronous Tasks with Node.js and BullMQ
Emmanuel Oyibo
Technical writer
Node.js
28.11.2024
Reading time: 11 min

Handling asynchronous tasks efficiently is crucial in Node.js applications, especially when dealing with time-intensive operations like sending emails, processing images, or performing complex calculations. Without proper management, these tasks can block the event loop, leading to poor performance and a subpar user experience. This is where BullMQ comes into play.

BullMQ is a powerful Node.js package that offers a reliable and scalable queuing system powered by Redis. It enables developers to transfer heavy operations to a queue in the background, keeping the main application responsive. With BullMQ you can successfully manage async queues, plan processes, and easily keep an eye on their progress.

This tutorial will show you how to manage asynchronous tasks with Node.js and BullMQ. The process involves setting up a project folder, performing a time-intensive task without using BullMQ, and enhancing the application by incorporating BullMQ for running tasks in parallel.

Prerequisites

Before you begin, ensure you:

Setting Up the Project Directory

Before you can use Node.js and BullMQ for asynchronous tasks, it is necessary to establish your project directory. Set up and customize your Node.js application using these guidelines.

Create a New Directory

Open your terminal and go to the location of your project. Create a fresh folder and navigate into it:

mkdir bullmq-demo && cd bullmq-demo

Initialize a New Node.js Project

Set up a Node.js project using npm. It generates a package.json file containing the default configurations:

npm init -y

Image3

Install Required Dependencies

Set up the required packages for your application:

npm install express bullmq ioredis

Here's what each package does:

  • express: A fast Node.js web framework commonly used for server creation.
  • bullmq: An excellent tool for handling queues within Node.js programs.
  • ioredis: A Redis client for Node.js that BullMQ needs in order to establish a connection with Redis.

Create the Main Application File

Create an index.js file as the primary access point for your application:

touch index.js

Alternatively, you have the option to generate this file by using your code editor.

Set Up a Basic Express Server

To set up a simple Express server, include this code in your index.js file:

const express = require('express');
const app = express();
const port = 3000;

app.use(express.json());

app.listen(port, () => {
  console.log(`Server is running on port ${port}`);
});

This code initiates an Express app on port 3000 which handles requests using JSON middleware.

Verify the Server Setup

Start the server by running:

node index.js

The below message should appear:

Image4

Open up your internet browser and go to either http://your_server_ip:3000 or http://localhost:3000. You will receive a "Cannot GET /" message as there are no routes set up, as anticipated.

Image2

When ready to proceed, you can terminate the server using Ctrl + C.

Implementing a Time-Intensive Task Without BullMQ

This part describes how to include a route in your Express app that performs a time-consuming task in a synchronous way. This will demonstrate how specific tasks can block the event loop and negatively affect your application's performance.

Define a Time-Intensive Function

Create a function in the index.js file that simulates a computationally intensive task:

// Function to simulate a heavy computation
function heavyComputation() {
  const start = Date.now();
  // Run a loop for 5 seconds
  while (Date.now() - start < 5000) {
    // Perform a CPU-intensive task
    Math.sqrt(Math.random());
  }
}

The function runs a loop for about five seconds, performing math operations to mimic a CPU-heavy task.

Create a Route to Handle the Task

Create a fresh route in your Express application that calls the heavyComputation function:

app.get('/heavy-task', (req, res) => {
  heavyComputation();
  res.send('Heavy computation finished');
});

This route is set up to receive GET requests specifically at the /heavy-task endpoint. After receiving a request, it carries out the specified intensive computation and then provides a response.

Start the Server

To restart your server, execute the following command:

node index.js

Confirm the server is functioning before moving on to the next stage.

Test the Heavy Task Route

Open your internet browser and type in either http://your_server_ip:3000/heavy-task or http://localhost:3000/heavy-task to access the webpage. 

The following message should be displayed:

Image1

It is important to observe that the response time is approximately five seconds. The delay is a result of the synchronous execution of the intensive computation process.

Observe Blocking Behavior

After the server is up and running, open a new tab on your internet browser and go to http://your_server_ip:3000/. The response to this request may not be immediate. The system delays taking action until the extensive processing of the previous step.

This happens when the time-consuming task is blocking the Node.js event loop, stopping the server from processing additional incoming requests.

When the server performs a task that takes a lot of time in a synchronous manner, it is unable to respond to additional requests. The act of blocking could result in a suboptimal user experience, particularly in apps that need to be highly responsive.

Executing Time-Intensive Tasks Asynchronously with BullMQ

We saw in the last section how synchronous execution of time-consuming operations can severely affect your application's performance by slowing down the event loop.

This section explains how to implement a high-performance asynchronous queue into your application using BullMQ.

Modify index.js to Use BullMQ

Make changes to the index.js file to include BullMQ in your application.

Import BullMQ and ioredis

At the top of your index.js file, you should include the following import statements:

const { Queue, Worker } = require('bullmq');
const Redis = require('ioredis');

Create a Redis Connection

Next, set up a connection with Redis:

const connection = new Redis();

Redis has been programmed to run on port 6379 and the localhost interface by default. To create a connection to a remote Redis server that has a different port, please enter the appropriate host address and port number:

const connection = new Redis({
  host: '127.0.0.1',
  port: 6379,
  maxRetriesPerRequest: null,
});

Initialize a BullMQ Queue

Create a new queue called heavyTaskQueue:

const heavyTaskQueue = new Queue('heavyTaskQueue', { connection });

Add a Route to Enqueue Tasks

Change the heavy-task route to add a job to the queue instead of running the task right away:

app.get('/heavy-task', async (req, res) => {
  await heavyTaskQueue.add('heavyComputation', {});
  res.send('Heavy computation job added to the queue');
});

The application will respond after a lengthy process has completed, handling requests asynchronously, when the /heavy-task route is accessed.

Remove the Worker Code from index.js

The worker must be implemented in a separate file. This is essential to ensure that the worker does not coexist with the Express server process. A worker's use of the heavyComputation function during execution won't interfere with the event loop of the main application.

The index.js file is structured in the following way:

const express = require('express');
const app = express();
const port = 3000;
app.use(express.json());

const { Queue } = require('bullmq');
const Redis = require('ioredis');

const connection = new Redis({
  host: '127.0.0.1',
  port: 6379,
  maxRetriesPerRequest: null,
});

const heavyTaskQueue = new Queue('heavyTaskQueue', { connection });

app.get('/heavy-task', async (req, res) => {
  await heavyTaskQueue.add('heavyComputation', {});
  res.send('Heavy computation job added to the queue');
});

app.listen(port, () => {
  console.log(`Server is running on port ${port}`);
});

Create a Separate Worker File

Generate a fresh file and name it worker.js. The file is intended for executing the worker code in charge of handling tasks obtained from the queue.

Create the worker.js file:

touch worker.js

Add Worker Code to worker.js:

const { Worker } = require('bullmq');
const Redis = require('ioredis');

const connection = new Redis({
  host: '127.0.0.1',
  port: 6379,
  maxRetriesPerRequest: null,
});

// Function to simulate a heavy computation
function heavyComputation() {
  const start = Date.now();
  // Run a loop for 5 seconds
  while (Date.now() - start < 5000) {
    // Perform a CPU-intensive task
    Math.sqrt(Math.random());
  }
}

const worker = new Worker(
  'heavyTaskQueue',
  async job => {
    // Time-intensive task here
    heavyComputation();
    console.log('Heavy computation completed');
  },
  { connection }
);

worker.on('completed', job => {
  console.log(`Job ${job.id} has completed`);
});

worker.on('failed', (job, err) => {
  console.log(`Job ${job.id} has failed with error ${err.message}`);
});

Run the Worker in a Separate Process

You must now execute worker.js as an independent Node.js process.

Start the Worker Process

Open a new terminal window or tab, navigate to your project folder, and run the specified command:

node worker.js

Start the Express Server

Initiate the Express server in your original terminal window:

node index.js

Test the Application with BullMQ

Proceed to conduct testing of the application utilizing BullMQ. 

Make a Request to /heavy-task:
Open your internet browser and type in either http://your_server_ip:3000/heavy-task or http://localhost:3000/heavy-task in the URL bar. The following message should be displayed:

Heavy computation job added to the queue.

The rapid response time suggests that there is no blockage in the main thread.

Adding a Dashboard to Monitor BullMQ Queues

Monitoring your application's queues and jobs is essential for ensuring they are functioning properly and for troubleshooting purposes. BullMQ comes with a functionality called Bull Board, which offers a visual interface for overseeing your queues.

This part explains how to incorporate a dashboard into your application.

Install Bull Board

Use npm to install the @bull-board/express package:

npm install @bull-board/express

Set Up Bull Board in Your Application

In order to set up the bull board application, follow these steps:

Import Bull Board Modules

Insert the code provided at the top of your index.js file:

const { createBullBoard } = require('@bull-board/api');
const { BullMQAdapter } = require('@bull-board/api/bullMQAdapter');
const { ExpressAdapter } = require('@bull-board/express');

Create an Express Adapter for the Dashboard

Initialize the Express adapter:

const serverAdapter = new ExpressAdapter();
serverAdapter.setBasePath('/admin/queues');

Set Up Bull Board with Your Queue

Create the Bull Board instance and pass your queue:

 createBullBoard({
  queues: [new BullMQAdapter(heavyTaskQueue)],
  serverAdapter: serverAdapter,
});

Use the Dashboard in Your Express App

Add the following line to mount the dashboard at /admin/queues:

app.use('/admin/queues', serverAdapter.getRouter());

Make sure to include this line following the setup of your queue and worker.

The final index.js file looks like below:

 // Import Express and Initialize App
const express = require('express');
const app = express();
const port = 3000;
app.use(express.json());

// Import BullMQ and Redis
const { Queue } = require('bullmq');
const Redis = require('ioredis');

// Redis Connection
const connection = new Redis({
  host: '127.0.0.1',
  port: 6379,
  maxRetriesPerRequest: null,
});

// Initialize Queue
const heavyTaskQueue = new Queue('heavyTaskQueue', { connection });

// Define Route to Add Job to Queue
app.get('/heavy-task', async (req, res) => {
  await heavyTaskQueue.add('heavyComputation', {});
  res.send('Heavy computation job added to the queue');
});

// Import Bull Board and Set Up Dashboard
const { createBullBoard } = require('@bull-board/api');
const { BullMQAdapter } = require('@bull-board/api/bullMQAdapter');
const { ExpressAdapter } = require('@bull-board/express');

const serverAdapter = new ExpressAdapter();
serverAdapter.setBasePath('/admin/queues');

createBullBoard({
  queues: [new BullMQAdapter(heavyTaskQueue)],
  serverAdapter: serverAdapter,
});

app.use('/admin/queues', serverAdapter.getRouter());

// Start the Server
app.listen(port, () => {
  console.log(`Server is running on port ${port}`);
});

Access the Dashboard

To access the dashboard, follow the steps listed below:

Restart Your Server

node index.js

Navigate to the Dashboard

Open your browser and go to http://your_server_ip:3000/admin/queues.

Image6

Explore the Dashboard:

  • Queue Overview: See the list of queues and their status.
  • Jobs List: View active, completed, failed, and delayed jobs.
  • Job Details: Click on a job to see its data, logs, and stack trace if it failed.

Image5

You can easily manage your BullMQ queues by integrating Bull Board into your application. It is much easier to keep an eye on progress and identify issues when you can view your queues and tasks on the dashboard in real-time.

Conclusion

You have now learned how to use BullMQ with Node.js to manage asynchronous processes. Your application's responsiveness and efficiency have been enhanced by moving time-consuming operations to a separate queue.

Your Node.js app is now much more capable of handling heavy demands thanks to the usage of queues.

Node.js
28.11.2024
Reading time: 11 min

Similar

Node.js

Using node-cron to Automate Tasks in Node.js

In many projects, there is a need to automate the execution of functions or scripts at specific times. To address this need in Node.js, you can use the node-cron library. In this article, we’ll cover how to install the package, explore best practices, build a simple project, and deploy it to the cloud. What Are Cron and node-cron? Cron is a task scheduler used in Unix-like operating systems (such as Linux) that allows you to automatically run commands or scripts on a schedule. The schedule is written in crontab format, where each line describes the time and command to be executed. node-cron is a library for Node.js that implements cron functionality directly in JavaScript applications. It allows you to create tasks that run on a given schedule in real-time in a selected time zone, just like classic cron in Unix systems. Key Advantages of node-cron: Easy to integrate into existing Node.js projects Dynamic control over tasks Supports the same scheduling format as the classic Cron node-cron Syntax The syntax of node-cron is similar to traditional cron: Valid field values: Field Values Seconds 0–59 Minutes 0–59 Hours 0–23 Day of Month 1–31 Month 1–12 (or names) Day of Week 0–7 (or names, 0 or 7 = Sun) Using Multiple Values const cron = require('node-cron'); cron.schedule('1,2,4,5 * * * *', () => { console.log('Runs at minute 1, 2, 4, and 5'); }); Using Ranges const cron = require('node-cron'); cron.schedule('1-5 * * * *', () => { console.log('Runs every minute from 1 to 5'); }); Using Step Values Step values can be used with ranges or asterisks by adding / and a number. Example: 1-10/2 is the same as 2, 4, 6, 8, 10. You can also use it after *, e.g. */2 to run every 2 minutes. const cron = require('node-cron'); cron.schedule('*/2 * * * *', () => { console.log('Runs every 2 minutes'); }); Using Names for Months and Days You can use full names for months and days of the week: const cron = require('node-cron'); cron.schedule('* * * January,September Sunday', () => { console.log('Runs on Sundays in January and September'); }); Or abbreviated names: const cron = require('node-cron'); cron.schedule('* * * Jan,Sep Sun', () => { console.log('Runs on Sundays in January and September'); }); cron.schedule Method The main method in node-cron is schedule(), which is used to set up a task. It takes a cron expression, the task function, and an optional configuration object: scheduled: whether the task is started automatically (Boolean) timezone: the time zone the cron will follow (String) Example: const cron = require('node-cron'); cron.schedule('0 1 * * *', () => { console.log('Will run at 01:00 Cyprus time'); }, { scheduled: true, timezone: "Europe/Cyprus" }); ScheduledTask Methods You can manage the state of a scheduled task using: start() — starts a stopped task stop() — stops a running task Starting a task: const cron = require('node-cron'); const task = cron.schedule('* * * * *', () => { console.log('Stopped task is now running'); }, { scheduled: false }); task.start(); Stopping a task: const cron = require('node-cron'); const task = cron.schedule('* * * * *', () => { console.log('Will run every minute until stopped'); }); task.stop(); Setting Up the Working Environment Let’s set up our environment for working with Node.js and node-cron. Installing Node.js and npm To begin local development, you need to install a recent version of Node.js (we recommend v22.14.0 LTS). This will install npm (Node Package Manager). For Windows: Go to the official website and download the installer. Run it and follow the installation instructions. For Linux / macOS: In the terminal, run: curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash \. "$HOME/.nvm/nvm.sh" nvm install 22 After installation, verify everything with: node -v && npm -v Make sure the versions of Node.js and npm display correctly. Setting Up the Project Directory Create a new directory for your project and navigate into it: mkdir node-cron-project && cd node-cron-project Initialize the project: npm init -y Install node-cron: npm install --save node-cron Basic Example of Using node-cron Let’s build a simple but interesting project using the node-cron library. Our task is to automatically fetch the USD to EUR exchange rate and save the data to a file. This project is simple, yet it effectively demonstrates how to use node-cron for scheduled tasks. Installing Additional Libraries Install additional dependencies for the project: npm install axios fs axios — for making HTTP requests (to fetch exchange rate data) fs — built-in module to work with the file system (to write to files) Writing the Project Our app will do the following: Create a task to fetch the current exchange rate every minute Write the data to a file called exchange_rates.txt Log activity to the console Create a file named index.js and paste in the following code: const cron = require('node-cron'); const axios = require('axios'); const fs = require('fs'); // Create file if it doesn't exist if (!fs.existsSync('exchange_rates.txt')) { fs.writeFileSync('exchange_rates.txt', ''); } // Function to fetch exchange rate async function getExchangeRate() { try { const response = await axios.get('https://open.er-api.com/v6/latest/USD'); const rate = response.data.rates.EUR; return rate; } catch (error) { console.error('Error fetching exchange rate:', error); return null; } } // Function to save data to file function saveData(rate) { const currentTime = new Date().toLocaleString(); const data = { time: currentTime, rate }; fs.appendFileSync('exchange_rates.txt', `${JSON.stringify(data)}\n`); console.log(`Rate saved: ${currentTime} - ${rate} EUR`); } // Cron job running every minute cron.schedule('* * * * *', async () => { const rate = await getExchangeRate(); if (rate !== null) { saveData(rate); } }); console.log('Data collection started...'); Let’s explain what exactly this code does: if (!fs.existsSync(...)) — Checks if the file exchange_rates.txt exists; if not, it creates it. getExchangeRate() — Fetches the USD to EUR exchange rate using a public API (in this case, open.er-api.com). saveData() — Saves the retrieved rate and current timestamp to the file. cron.schedule('* * * * *', ...) — Sets up a cron job that runs every minute to get and save the latest exchange rate. Testing the Project To run your project, execute: node index.js You will see this message in the console: Data collection started... And a little later you’ll see logs like: Rate saved: 4/9/2025, 12:00:00 PM - 0.92 EUR And the exchange_rates.txt file will contain entries with the date, time, and exchange rate. Using node-cron in a Real Project Let’s apply node-cron in a practical task. We’ll write a script that automatically sends emails. Companies often use this case to send various promotional content. It’s simple to implement but quite functional. Getting an App Password First, we need to obtain a token for your Gmail account: Log in to your Google Account. Go to the Security section. Enable Two-Step Verification. You'll be asked to confirm your identity, for example, via a code sent by SMS. Once enabled, proceed to the next step. Go to App Passwords to generate a new app password. Give your app a name (e.g., "nodemailer") and create it. A modal window will appear with the password. Copy this password and use it in your code. Writing the Code First, install the required libraries. Since node-cron is already installed, we only need to install nodemailer: npm install nodemailer Now create a file called app.js and write the following code: const nodemailer = require('nodemailer'); const cron = require('node-cron'); const recipients = [ 'recipient1@gmail.com', 'recipient2@outlook.com' ]; let transporter = nodemailer.createTransport({ service: 'gmail', auth: { user: 'sender@example.com', pass: 'PASSWORD' } }); function sendEmail(recipient) { let mailOptions = { to: recipient, subject: 'Scheduled Email', text: 'This email was sent automatically on a schedule using node-cron.', html: '<b>This email was sent automatically on a schedule using node-cron.</b>' }; transporter.sendMail(mailOptions, function(error, info){ if (error) { console.log(`Error sending email to ${recipient}:`, error); } else { console.log(`Email successfully sent to ${recipient}:`, info.response); } }); } cron.schedule('* * * * *', () => { console.log('Running cron job...'); recipients.forEach((recipient) => { sendEmail(recipient); }); }); Explanation: The recipients array contains the list of email recipients. The transporter variable holds the authentication info for the sender. Replace user with your Gmail address and pass with the generated app password. sendEmail() is a function that takes a recipient's address and sends an email. mailOptions holds the subject, plain text, and HTML content. The cron.schedule('* * * * *') task runs every minute, calling sendEmail() for each recipient. Testing the Application To run the file, use the command: node app.js After a couple of minutes, you’ll see output in the console confirming the emails have been sent. Check your inbox, and you should see the emails arriving. Deploying the Project on a Cloud Server (Hostman) After development, we’ll deploy the app to the cloud. For this lightweight mailer, a minimal server setup is sufficient. 1. Install Node.js on your server: curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash \. "$HOME/.nvm/nvm.sh" nvm install 22 Check the installation: node -v && npm -v 2. Create the project directory: cd /home && mkdir nodemailer 3. Upload your files (app.js and package.json) On Windows, use FileZilla. On Linux/macOS, use: rsync -av --exclude="node_modules" ./ root@166.1.227.189:/home/nodemailer Explanation: --exclude="node_modules" — skip uploading installed libraries ./ — source directory root@166.1.227.189:/home/nodemailer — target path on the server 4. SSH into the server and verify the files: cd /home/nodemailer && ls 5. Install dependencies: npm install 6. Run the script: node app.js Check if the emails are being sent correctly. If there’s an issue, make sure port 465 (SMTP) is open on the server. If not, contact support to open it. To keep the app running even after closing the terminal, create a systemd unit file: sudo nano /etc/systemd/system/nodemailer.service Paste the following content: [Unit] Description=NodeMailer Service After=network.target [Service] User=root WorkingDirectory=/home/nodemailer ExecStart=/root/.nvm/versions/node/v22.14.0/bin/node /home/nodemailer/app.js Restart=always RestartSec=5 [Install] WantedBy=multi-user.target Note: Adjust WorkingDirectory and ExecStart paths if necessary. Enable and start the service: sudo systemctl daemon-reload sudo systemctl enable nodemailer.service sudo systemctl start nodemailer.service Check status and logs: sudo systemctl status nodemailer.service sudo journalctl -u nodemailer.service -f You should see active (running) if everything is working properly. Service Management Commands Restart the service: sudo systemctl restart nodemailer.service Stop the service: sudo systemctl stop nodemailer.service Delete the service: sudo systemctl disable nodemailer.service sudo rm /etc/systemd/system/nodemailer.service sudo systemctl daemon-reload Conclusion The node-cron library is a powerful tool for automating tasks on the Node.js platform. In this article, we created a simple app that retrieves USD to EUR exchange rates and writes them to a file, and we also explored a real-world use case: automatically sending scheduled emails. We’ve seen how easily node-cron enables you to schedule recurring jobs, from data collection to user interactions. It’s a great choice for developers looking for a reliable and user-friendly scheduling system in Node.js projects. Its flexibility and ease of use make node-cron an essential tool in any modern backend developer’s toolkit.
15 April 2025 · 10 min to read
Node.js

How to Install and Use Yarn Package Manager for Node.js

Yarn is an efficient tool for managing dependencies in Node.js-based projects. It is known for its high speed, security, and ease of use. What is Yarn and Why Use It? Yarn is an alternative to the standard npm (Node Package Manager). It is designed to handle packages and projects built on Node.js. Yarn offers several advantages over npm: Speed: Yarn downloads packages in parallel, significantly reducing installation time. Security: The use of a yarn.lock file helps prevent version conflicts. Deterministic Builds: Ensures identical package versions across different machines. User-Friendly Interface: Cleaner command syntax and additional tools for dependency management. If your project involves working with many packages and dependencies, using Yarn can greatly simplify the task. It allows for faster and more secure package installations while making dependency management more predictable — a valuable benefit for team-based projects. Comparison of Yarn and npm Yarn's advantages make it particularly appealing for developers, especially in large-scale projects. Feature Yarn npm Installation Speed Faster thanks to caching Slower Dependency Handling Deterministic builds Potential version conflicts Lock File yarn.lock package-lock.json Ease of Use Simplified syntax More standard interface Installing Yarn Before installing Yarn, ensure that Node.js and npm are installed: Open the terminal or command prompt. Run the following commands to check the versions of Node.js and npm: node -vnpm -v If Node.js or npm is not installed, download them from the official Node.js website. You may also find our installation guide helpful. To install Yarn globally, run: npm install -g yarn Check if Yarn was installed successfully: yarn --version If the command returns the version number, Yarn has been installed correctly. Yarn Commands Yarn's intuitive syntax makes it easy to manage your project dependencies efficiently. Project Initialization To get started with Yarn, initialize your project to create a package.json file containing project and dependency information. Navigate to your project directory: cd your-project-directory Run the following command and follow the prompts: yarn init This will generate a package.json file with basic project settings. Installing Packages To install a single package: yarn add <package-name> This adds the specified package to your project. To install a package as a development dependency: yarn add <package-name> --dev This is useful for packages required only during development. To install a specific version of a package: yarn add <package-name>@<version> This allows you to select the desired package version. Installing All Dependencies If the project already contains a package.json or yarn.lock, run: yarn install This is helpful when cloning a project from a repository to quickly set up the environment. Removing Packages To remove a package from your project and update package.json, use: yarn remove <package-name> Updating Dependencies To upgrade packages to their latest versions, run: yarn upgrade This ensures your project uses the most current versions. Dependency Security Audit To identify vulnerabilities in your project dependencies: yarn audit This helps detect and address potential security threats. Caching Yarn leverages caching to speed up subsequent package installations. To clear the cache: yarn cache clean This command can be useful if you encounter issues during package installation. Conclusion Yarn is a modern tool for managing dependencies in Node.js projects. Its speed, security features, and intuitive interface make it an excellent choice for developers.
10 February 2025 · 3 min to read
Node.js

Difference Between Polling and Webhook in Telegram Bots

When developing Telegram bots using Node.js, there are two main methods for receiving user messages: Polling and Webhook. Both serve the purpose of handling incoming requests, but each has its unique features, making them suitable for different scenarios. What is Polling? Polling is a method of fetching updates from the Telegram server by periodically sending requests. The bot sends requests at specific time intervals to check for new messages or events. There are two types of polling: Long Polling and Short Polling. Long Polling In Long Polling, the bot sends a request to the server and waits for a response. If there are no new messages, the server holds the request open until a new message arrives or the timeout period ends. Once the bot receives a response, it immediately sends a new request. Here’s an example where the bot is configured to poll the Telegram server every 3 seconds, with a timeout of 10 seconds: const TelegramBot = require('node-telegram-bot-api'); const token = 'TOKEN'; // Create a bot instance with Long Polling enabled const bot = new TelegramBot(token, { polling: { interval: 3000, // Interval between requests (3 seconds) autoStart: true, // Automatically start polling params: { timeout: 10 // Request timeout (10 seconds) } } }); bot.on('message', (msg) => { const chatId = msg.chat.id; const text = msg.text; // Respond to the received message bot.sendMessage(chatId, `You wrote: ${text}`); }); bot.onText(/\/start/, (msg) => { const chatId = msg.chat.id; bot.sendMessage(chatId, 'Hello! I am a bot using Long Polling.'); }); Short Polling In Short Polling, the bot sends requests to the server at short intervals, regardless of whether new messages are available. This method is less efficient because it generates more network requests and consumes more resources. In this case, the bot constantly requests updates from the server without keeping the connection open for a long time. This can lead to high network usage, especially with heavy traffic. Here’s an example of a bot using Short Polling: const TelegramBot = require('node-telegram-bot-api'); const token = 'TOKEN'; // Create a bot instance with Short Polling enabled const bot = new TelegramBot(token, { polling: true }); bot.on('message', (msg) => { const chatId = msg.chat.id; const text = msg.text; bot.sendMessage(chatId, `You wrote: ${text}`); }); bot.onText(/\/start/, (msg) => { const chatId = msg.chat.id; bot.sendMessage(chatId, 'Hello! I am a bot using Short Polling.'); }); What is Webhook? Webhook is a method that allows a bot to receive updates automatically. Instead of periodically polling the Telegram server, the bot provides Telegram with a URL, where POST requests will be sent whenever new updates arrive. This approach helps to use resources more efficiently and minimizes latency. In the following example, the bot receives requests from Telegram via Webhook, eliminating the need for frequent server polling. This reduces server load and ensures instant message handling. const TelegramBot = require('node-telegram-bot-api'); const express = require('express'); const bodyParser = require('body-parser'); const token = 'TOKEN'; // Your server URL const url = 'https://your-server.com'; const port = 3000; // Create a bot instance without automatic polling const bot = new TelegramBot(token, { webHook: true }); // Set the Webhook URL for your server bot.setWebHook(`${url}/bot${token}`); // Configure the Express server const app = express(); app.use(bodyParser.json()); // Request handler for incoming updates from Telegram app.post(`/bot${token}`, (req, res) => { bot.processUpdate(req.body); res.sendStatus(200); }); bot.on('message', (msg) => { const chatId = msg.chat.id; bot.sendMessage(chatId, `You wrote: ${msg.text}`); }); // Start the server app.listen(port, () => { console.log(`Server running on port ${port}`); }); To run the code and start the bot, install the required libraries: npm install node-telegram-bot-api express Server Setup We need to set up a server to work with Webhook. We'll use Hostman for this. Step 1: Set Up a Cloud Server Log in to your Hostman control panel and start by creating a new project. Next, create a cloud server. During the server creation process, select the Marketplace tab and choose Node.js. When the server starts, Node.js will automatically be installed. Choose the nearest region with the lowest ping. You can choose the configuration according to your needs, but for testing purposes, the minimum configuration will suffice. In the Network settings, make sure to assign a public IP. In the Authorization and Cloud-init settings, leave them unchanged.  In the server's information, specify the server name and description, and select the project created earlier. Once all settings are configured, click on the Order button. The server will start, and you will receive a free domain. Step 2: Install SSL Certificate Since Telegram's API only works with HTTPS, you need to install an SSL certificate. For this, you will need a registered domain name. To set up the web server and install the certificate, execute the following commands sequentially: Update available package lists: sudo apt update Create and open the Nginx configuration file: sudo nano /etc/nginx/sites-available/your_domain Inside this file, add the following configuration: server { listen 80; server_name your_domain; location / { proxy_pass http://localhost:3000; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; } } Replace your_domain with your actual domain name in this file and throughout the console. Create a symbolic link to the file: sudo ln -s /etc/nginx/sites-available/your_domain /etc/nginx/sites-enabled/ Restart Nginx: sudo systemctl restart nginx Install certbot to create SSL certificates: sudo apt install certbot python3-certbot-nginx Use certbot to configure the SSL certificate: sudo certbot --nginx -d your_domain Replace your_domain with your actual domain name. Examples of Using Polling and Webhook Before choosing a method for receiving messages, it is important to consider the characteristics of each approach and its applicability in different situations. Polling: Local Development: When developing and testing a bot on a local machine, using Long Polling allows for easy updates without the need to set up a server. Small Projects: If you are creating a bot for a small group of users or for personal use, and you do not have strict requirements for response time, Polling will be sufficient. Low Traffic Projects: If your bot is not expecting a large number of messages, using Short Polling can be appropriate as it is simple to implement. Webhook: Production Applications: For bots working in a production environment where immediate responses to events are important, Webhook is the preferred choice. For example, bots that handle payments or respond to user queries in real time should use Webhook to ensure high performance. High Traffic Systems: If you're developing a bot that will serve a large number of users, Webhook will be more efficient since it reduces server load by eliminating continuous requests. Systems with Long Operations: If your bot performs long operations (such as generating reports or processing data), Webhook can be used to notify users once these operations are complete. Comparison of Polling and Webhook To better understand the differences between the two approaches, here is a comparison table of their characteristics: Characteristic Polling Webhook Method of Data Retrieval Periodic requests to the Telegram server Automatic sending of updates to a specified URL Setup Simple setup, no additional resources required Requires HTTPS server setup and SSL certificate Response Speed May have slight delays due to polling intervals Near-instant message reception Resource Usage Continuously requests updates, taxing the server More resource-efficient since updates come automatically Infrastructure Requirements Does not require a public server Requires a public HTTPS server Reliability Does not depend on the availability of an external server Can be unavailable if there are issues with the HTTPS server Setup Issues in Local Environment Can be used locally for testing Difficult to use without public access Conclusion The choice between Polling and Webhook depends on the specific needs of your project. Polling is a simple and quick way to develop, especially in the early stages, while Webhook offers more efficient message processing for production environments.
31 January 2025 · 7 min to read

Do you have questions,
comments, or concerns?

Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
Email us
Hostman's Support