Sign In
Sign In

Difference Between Polling and Webhook in Telegram Bots

Difference Between Polling and Webhook in Telegram Bots
Hostman Team
Technical writer
Node.js
31.01.2025
Reading time: 7 min

When developing Telegram bots using Node.js, there are two main methods for receiving user messages: Polling and Webhook. Both serve the purpose of handling incoming requests, but each has its unique features, making them suitable for different scenarios.

What is Polling?

Polling is a method of fetching updates from the Telegram server by periodically sending requests. The bot sends requests at specific time intervals to check for new messages or events.

There are two types of polling: Long Polling and Short Polling.

Long Polling

In Long Polling, the bot sends a request to the server and waits for a response. If there are no new messages, the server holds the request open until a new message arrives or the timeout period ends. Once the bot receives a response, it immediately sends a new request.

Here’s an example where the bot is configured to poll the Telegram server every 3 seconds, with a timeout of 10 seconds:

const TelegramBot = require('node-telegram-bot-api');

const token = 'TOKEN';

// Create a bot instance with Long Polling enabled
const bot = new TelegramBot(token, {
    polling: {
        interval: 3000,   // Interval between requests (3 seconds)
        autoStart: true,  // Automatically start polling
        params: {
            timeout: 10   // Request timeout (10 seconds)
        }
    }
});

bot.on('message', (msg) => {
    const chatId = msg.chat.id;
    const text = msg.text;

    // Respond to the received message
    bot.sendMessage(chatId, `You wrote: ${text}`);
});

bot.onText(/\/start/, (msg) => {
    const chatId = msg.chat.id;
    bot.sendMessage(chatId, 'Hello! I am a bot using Long Polling.');
});

Short Polling

In Short Polling, the bot sends requests to the server at short intervals, regardless of whether new messages are available. This method is less efficient because it generates more network requests and consumes more resources.

In this case, the bot constantly requests updates from the server without keeping the connection open for a long time. This can lead to high network usage, especially with heavy traffic.

Here’s an example of a bot using Short Polling:

const TelegramBot = require('node-telegram-bot-api');

const token = 'TOKEN';

// Create a bot instance with Short Polling enabled
const bot = new TelegramBot(token, { polling: true });

bot.on('message', (msg) => {
    const chatId = msg.chat.id;
    const text = msg.text;
    bot.sendMessage(chatId, `You wrote: ${text}`);
});

bot.onText(/\/start/, (msg) => {
    const chatId = msg.chat.id;
    bot.sendMessage(chatId, 'Hello! I am a bot using Short Polling.');
});

What is Webhook?

Webhook is a method that allows a bot to receive updates automatically. Instead of periodically polling the Telegram server, the bot provides Telegram with a URL, where POST requests will be sent whenever new updates arrive. This approach helps to use resources more efficiently and minimizes latency.

In the following example, the bot receives requests from Telegram via Webhook, eliminating the need for frequent server polling. This reduces server load and ensures instant message handling.

const TelegramBot = require('node-telegram-bot-api');
const express = require('express');
const bodyParser = require('body-parser');

const token = 'TOKEN';

// Your server URL
const url = 'https://your-server.com';
const port = 3000;

// Create a bot instance without automatic polling
const bot = new TelegramBot(token, { webHook: true });

// Set the Webhook URL for your server
bot.setWebHook(`${url}/bot${token}`);

// Configure the Express server
const app = express();
app.use(bodyParser.json());

// Request handler for incoming updates from Telegram
app.post(`/bot${token}`, (req, res) => {
    bot.processUpdate(req.body);
    res.sendStatus(200);
});

bot.on('message', (msg) => {
    const chatId = msg.chat.id;
    bot.sendMessage(chatId, `You wrote: ${msg.text}`);
});

// Start the server
app.listen(port, () => {
    console.log(`Server running on port ${port}`);
});

To run the code and start the bot, install the required libraries:

npm install node-telegram-bot-api express

Server Setup

We need to set up a server to work with Webhook. We'll use Hostman for this.

Step 1: Set Up a Cloud Server

  1. Log in to your Hostman control panel and start by creating a new project.
  2. Next, create a cloud server. During the server creation process, select the Marketplace tab and choose Node.js. When the server starts, Node.js will automatically be installed.
  3. Choose the nearest region with the lowest ping.
  4. You can choose the configuration according to your needs, but for testing purposes, the minimum configuration will suffice.
  5. In the Network settings, make sure to assign a public IP.
  6. In the Authorization and Cloud-init settings, leave them unchanged. 
  7. In the server's information, specify the server name and description, and select the project created earlier.
  8. Once all settings are configured, click on the Order button. The server will start, and you will receive a free domain.

Step 2: Install SSL Certificate

Since Telegram's API only works with HTTPS, you need to install an SSL certificate. For this, you will need a registered domain name. To set up the web server and install the certificate, execute the following commands sequentially:

  1. Update available package lists:

sudo apt update
  1. Create and open the Nginx configuration file:

sudo nano /etc/nginx/sites-available/your_domain
  1. Inside this file, add the following configuration:

server {
    listen 80;
    server_name your_domain;

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Replace your_domain with your actual domain name in this file and throughout the console.

  1. Create a symbolic link to the file:

sudo ln -s /etc/nginx/sites-available/your_domain /etc/nginx/sites-enabled/
  1. Restart Nginx:

sudo systemctl restart nginx
  1. Install certbot to create SSL certificates:

sudo apt install certbot python3-certbot-nginx
  1. Use certbot to configure the SSL certificate:

sudo certbot --nginx -d your_domain

Replace your_domain with your actual domain name.

Examples of Using Polling and Webhook

Before choosing a method for receiving messages, it is important to consider the characteristics of each approach and its applicability in different situations.

Polling:

  • Local Development: When developing and testing a bot on a local machine, using Long Polling allows for easy updates without the need to set up a server.

  • Small Projects: If you are creating a bot for a small group of users or for personal use, and you do not have strict requirements for response time, Polling will be sufficient.

  • Low Traffic Projects: If your bot is not expecting a large number of messages, using Short Polling can be appropriate as it is simple to implement.

Webhook:

  • Production Applications: For bots working in a production environment where immediate responses to events are important, Webhook is the preferred choice. For example, bots that handle payments or respond to user queries in real time should use Webhook to ensure high performance.

  • High Traffic Systems: If you're developing a bot that will serve a large number of users, Webhook will be more efficient since it reduces server load by eliminating continuous requests.

  • Systems with Long Operations: If your bot performs long operations (such as generating reports or processing data), Webhook can be used to notify users once these operations are complete.

Comparison of Polling and Webhook

To better understand the differences between the two approaches, here is a comparison table of their characteristics:

Characteristic

Polling

Webhook

Method of Data Retrieval

Periodic requests to the Telegram server

Automatic sending of updates to a specified URL

Setup

Simple setup, no additional resources required

Requires HTTPS server setup and SSL certificate

Response Speed

May have slight delays due to polling intervals

Near-instant message reception

Resource Usage

Continuously requests updates, taxing the server

More resource-efficient since updates come automatically

Infrastructure Requirements

Does not require a public server

Requires a public HTTPS server

Reliability

Does not depend on the availability of an external server

Can be unavailable if there are issues with the HTTPS server

Setup Issues in Local Environment

Can be used locally for testing

Difficult to use without public access

Conclusion

The choice between Polling and Webhook depends on the specific needs of your project. Polling is a simple and quick way to develop, especially in the early stages, while Webhook offers more efficient message processing for production environments.

Node.js
31.01.2025
Reading time: 7 min

Similar

Node.js

How to Create a Telegram Bot Using Node.js

Telegram bots have become an integral part of this messenger: every day, hundreds of thousands of people use them—and for good reason. Telegram bots are easy for users to interact with, and developers can quickly and comfortably create them thanks to the constantly evolving Telegram API, which aims to improve daily. The main idea behind Telegram bots is task automation and extending the messenger’s functionality. Bots can serve as simple assistants performing commands or as complex systems with full-fledged business logic. From sending out news updates to implementing intricate game mechanics—the possibilities for building bots are nearly limitless. With Node.js, you can implement almost any functionality for a Telegram bot, thanks to its ecosystem of libraries and frameworks. Node.js, as a platform with asynchronous request handling, is ideal for building bots that need to work in real-time and interact with thousands of users simultaneously. Here are some capabilities that can be implemented: Basic Functionality Responding to commands Inline bots Buttons Integration with External Services APIs and databases Webhooks Notifications Sending scheduled notifications or alerts when certain events occur Automatically sending news updates from sources every N seconds Analytics Collecting various statistics Creating a Telegram Bot First, you need to create a bot within Telegram. Use the official BotFather bot to register your bot. Click the "Start" button (or if you’ve already interacted with the bot, send the command /start). In BotFather’s response, find and select the /newbot command. BotFather will ask you to provide a bot name and then a username. The username must end with the word bot. For example, if your bot’s name is Tetris, the username should be one of the following: TetrisBot Tetris_bot Tetrisbot Tetris_Bot If everything is entered correctly, your bot will be created. BotFather will also give you a unique bot token, which you must keep private. Development We will create a bot that sends various quizzes in the form of Telegram polls. The quiz topics will be school subjects. The bot will have two commands: one for sending questions and another for selecting quiz topics. Preparing the Environment Before starting development, ensure that Node.js and npm are installed on your PC. You can download Node.js from the official website, and npm will be installed automatically along with Node.js. If you are using Linux, you can install npm by following this guide. Once Node.js is installed, you can begin developing the bot. First, create a new private repository on GitHub and select Node under the Add .gitignore section. Now, clone this repository to your PC using the terminal. If you want the project to be on your desktop, enter: cd Desktop Then enter: git clone https://github.com/username/School-Quiz Replace username with your actual GitHub username. You can also replace School-Quiz with any other project name. After cloning the repository, without closing the terminal, enter: cd School-Quiz Replace School-Quiz with the actual name of the folder where your project was cloned from GitHub. To initialize the project, run the following command: npm init You will be prompted to enter the package name, version, description, default entry file, test command, Git repository, keywords, author, and license. You can press "Enter" to accept the default values. Now, let’s install the library that will be used to write the bot’s code. Enter the following command in the terminal (ensuring that you are in the project folder): npm install node-telegram-bot-api Writing Code for the Quiz After the installation is complete, you can start writing the code. Open the package.json file and find the scripts section. Inside it, above the test command, add the following line: "start": "node index.js", This allows you to start the project by simply entering npm start in the terminal instead of typing node followed by the file name. Now, create a file called index.js and add the following code: const TelegramBot = require('node-telegram-bot-api'); const fs = require('fs'); const bot = new TelegramBot('TOKEN', { polling: true }); // Replace 'TOKEN' with the actual token provided by BotFather const ADMIN_ID = '1402655980'; let awaitingSupportMessage = {}; // Stores information about users waiting for support // Stores selected topics for users let userTopics = {}; // Topics and their respective question files const topics = { math: { name: 'Math', file: 'questions/math.json' }, spanish: { name: 'Spanish', file: 'questions/spanish.json' }, history: { name: 'History', file: 'questions/history.json' } }; // Function to retrieve questions based on selected topics function getQuestionsByTopics(userId) { const selectedTopics = userTopics[userId] || Object.keys(topics); let allQuestions = []; selectedTopics.forEach(topic => { const questions = JSON.parse(fs.readFileSync(topics[topic].file, 'utf8')); allQuestions = allQuestions.concat(questions); }); return allQuestions; } function getRandomQuestion(userId) { const questions = getQuestionsByTopics(userId); const randomIndex = Math.floor(Math.random() * questions.length); return questions[randomIndex]; } bot.onText(/\/quiz/, (msg) => { const chatId = msg.chat.id; const userId = msg.from.id; // Retrieve a random question const questionData = getRandomQuestion(userId); // Send the poll as a quiz bot.sendPoll( chatId, questionData.question, // The question text questionData.options, // Answer options { type: 'quiz', // Quiz type correct_option_id: questionData.correct_option_id, // Correct answer is_anonymous: false // The quiz won't be anonymous } ).then(pollMessage => { // Handle poll results bot.on('poll_answer', (answer) => { if (answer.poll_id === pollMessage.poll.id) { const selectedOption = answer.option_ids[0]; // Check if the answer is correct if (selectedOption !== questionData.correct_option_id) { bot.sendMessage(chatId, questionData.explanation); } } }); }); }); bot.onText(/\/settopic/, (msg) => { const chatId = msg.chat.id; const userId = msg.from.id; const keyboard = Object.keys(topics).map(topicKey => ({ text: `${(userTopics[userId] || []).includes(topicKey) ? '✅ ' : ''}${topics[topicKey].name}`, callback_data: topicKey })); bot.sendMessage(chatId, 'Select the topics for questions:', { reply_markup: { inline_keyboard: [keyboard] } }); }); // Topic selection handler bot.on('callback_query', (callbackQuery) => { const message = callbackQuery.message; const userId = callbackQuery.from.id; const topicKey = callbackQuery.data; // Initialize selected topics for the user if they don't exist if (!userTopics[userId]) { userTopics[userId] = Object.keys(topics); } // Add or remove the selected topic if (userTopics[userId].includes(topicKey)) { userTopics[userId] = userTopics[userId].filter(t => t !== topicKey); } else { userTopics[userId].push(topicKey); } // Update the message with buttons const keyboard = Object.keys(topics).map(topicKey => ({ text: `${userTopics[userId].includes(topicKey) ? '✅ ' : ''}${topics[topicKey].name}`, callback_data: topicKey })); bot.editMessageReplyMarkup({ inline_keyboard: [keyboard] }, { chat_id: message.chat.id, message_id: message.message_id }); }); bot.onText(/\/start/, (msg) => { const chatId = msg.chat.id; bot.sendMessage(chatId, "Hello! Type /quiz to start a quiz. Use /settopic to choose topics."); }); console.log('Bot is running.'); Quiz Questions Files Now, create a folder named questions inside your project. Within this folder, create three JSON files: spanish.json [ { "question": "How do you say 'I' in Spanish?", "options": ["Yo", "Tú", "Nosotros"], "correct_option_id": 0, "explanation": "The correct answer is: Yo." }, { "question": "What does the verb 'correr' mean?", "options": ["to run", "to walk", "to stand"], "correct_option_id": 0, "explanation": "The correct answer is: to run." }, { "question": "How do you say 'she' in Spanish?", "options": ["Tú", "Ella", "Vosotros"], "correct_option_id": 1, "explanation": "The correct answer is: Ella." } ] history.json [ { "question": "In which year did World War II begin?", "options": ["1939", "1941", "1914"], "correct_option_id": 0, "explanation": "The correct answer is: 1939." }, { "question": "Who was the first president of the United States?", "options": ["Abraham Lincoln", "George Washington", "Franklin Roosevelt"], "correct_option_id": 1, "explanation": "The correct answer is: George Washington." }, { "question": "Which country was the first to send a human into space?", "options": ["USA", "USSR", "China"], "correct_option_id": 1, "explanation": "The correct answer is: USSR." } ] math.json [ { "question": "What is 2 + 2?", "options": ["3", "4", "5"], "correct_option_id": 1, "explanation": "The correct answer is: 4." }, { "question": "What is 5 * 5?", "options": ["10", "20", "25"], "correct_option_id": 2, "explanation": "The correct answer is: 25." }, { "question": "What is 10 / 2?", "options": ["4", "5", "6"], "correct_option_id": 1, "explanation": "The correct answer is: 5." } ] Each JSON file contains the question, answer options, the index of the correct answer, and an explanation that will be sent if the user selects the wrong answer. Telegram Stars Recently, Telegram introduced an internal currency called Telegram Stars, along with an API update allowing bots to support donations in Stars. Let’s add a /donate command to the index.js file. When users send this command, the bot will generate a payment invoice. Add the following code inside index.js: bot.onText(/\/donate/, (msg) => { const chatId = msg.chat.id; bot.sendInvoice(chatId, 'Donation', 'Support the project with a donation', 'unique_payload', '', // Empty provider_token for Stars Payments 'XTR', // Currency "XTR" [{ label: 'Donation', amount: 1 }] // Amount: 1 Star ); }); Support Command Let’s add another command called /support. This command allows a large number of users to contact you without creating multiple unnecessary chats. Users will be able to send text, photos, and videos, and the bot will forward these messages directly to the admin (in this case, you). Place the following code inside index.js. At the beginning of the file, add: const ADMIN_ID = 'ID'; let awaitingSupportMessage = {}; // Stores information about users waiting for support The ADMIN_ID tells the bot where to forward the user’s message. To find your ID, you can use the Get My ID bot by simply sending the /start command to it. At the end of the file, add the following code: bot.onText(/\/support/, (msg) => { const chatId = msg.chat.id; const userId = msg.from.id; // Inform the user that we are waiting for their message bot.sendMessage(chatId, "Please send your message in a single message, including text, photos, or videos!"); // Mark the user as currently composing a support message awaitingSupportMessage[userId] = true; }); Handling All Messages This section processes all incoming messages and checks if they are part of a support request. Add the following code to handle different types of user content: bot.on('message', (msg) => { const userId = msg.from.id; // Check if the user is sending a message after the /support command if (awaitingSupportMessage[userId]) { const chatId = msg.chat.id; const caption = msg.caption || ''; // Include caption if present // Check the type of message and forward the corresponding content to the admin if (msg.text) { // If the message contains text bot.sendMessage(ADMIN_ID, `New support request from @${msg.from.username || msg.from.first_name} (ID: ${userId}):\n\n${msg.text}`); } else if (msg.photo) { // If the message contains a photo const photo = msg.photo[msg.photo.length - 1].file_id; // Select the highest resolution photo bot.sendPhoto(ADMIN_ID, photo, { caption: `New support request from @${msg.from.username || msg.from.first_name} (ID: ${userId})\n\n${caption}` }); } else if (msg.video) { // If the message contains a video const video = msg.video.file_id; bot.sendVideo(ADMIN_ID, video, { caption: `New support request from @${msg.from.username || msg.from.first_name} (ID: ${userId})\n\n${caption}` }); } else { // If the message type is unsupported bot.sendMessage(msg.chat.id, "Sorry, this type of message is not supported."); } // Confirm to the user that their message has been sent bot.sendMessage(chatId, "Your message has been sent. The administrator will contact you soon."); // Remove the user from the list of those composing a support message delete awaitingSupportMessage[userId]; } }); Deployment on a Server For our bot to operate continuously, we must upload and run it on a server. For deployment, we will use Hostman cloud servers. Uploading to GitHub Before launching the bot on the server, you first need to upload the project files to GitHub. Run the following commands in the console in sequence: Add all changes in the current directory to the next commit: git add . Create a commit with the message "first commit", recording all changes added with git add: git commit -m "first commit" Push the changes to GitHub: git push Server Setup Go to your Hostman control panel and: Create a New Project (optional): Specify an icon, a name, a description, and add users if necessary. Create a Cloud Server: Either from your project or from the Cloud servers page start creating a new cloud server. Select the Region: Choose the region that is closest to you or where the lowest ping is available. Go to the Marketplace tab in the second step and select Node.js. Set the Ubuntu version to the latest one. This ensures that Node.js will already be installed on the server when it starts, so you won’t need to install it manually. Choose Configuration: Select the configuration according to your needs. For running the project, the minimum configuration is sufficient. If the project requires more resources in the future, you can upgrade the server without disrupting its operation. Network Settings: Ensure that you assign a public IP for the server. Configure any additional services as needed. Authorization and Cloud-init: In the Authorization step, you can add your SSH key to the server. However, it’s optional, and you can leave these settings as they are. Server Information: Provide the server’s name and description, and select the project to which you want to add the server. Once everything is set up, click the Order button. After a short while, the server will be up and running, and you can proceed with the next steps. Launching the Bot After creating the server, go to the Dashboard tab, copy the Root password, and open the Console tab. Enter the username root and press Enter. Next, paste the password you copied and press Enter again. When typing or pasting the password, it will not be visible! If everything is correct, you will see a welcome message. Now, run the following command to get the latest updates: sudo apt-get update Create a new folder where you will place the bot. Enter these commands in sequence: cd /sudo mkdir Botcd Bot You can replace the folder name "Bot" with any other name you choose. To ensure Git is installed on the server (it is usually pre-installed by default), check the version using: git --version Next, set up global Git settings to link it to your GitHub profile: git config --global user.name "your GitHub username"git config --global user.email "email used during registration" After this, clone the repository by entering the following command with your repository URL: git clone https://github.com/username/School-Quiz During cloning, you will be prompted to enter your username and then your password. If you have two-factor authentication (2FA) enabled on your GitHub account, entering your regular password will result in an error saying the password is incorrect. To clone a repository with 2FA enabled, you need to create a personal access token. Click your profile picture in the top-right corner and select “Settings”. In the left-hand menu, click “Developer settings”. Under the “Personal access tokens” section, select “Tokens (classic)” and click “Generate new token”. Set token parameters: In the “Note” field, provide a description for the token. Set the expiration date for the token in the “Expiration” field. Under “Select scopes”, choose the necessary permissions for the token. For example, to work with repositories, select repo. Click “Generate token”. Copy the generated token and store it in a secure place. Note that you won’t be able to view the token again after closing the page. Once you have the personal access token, use it instead of your password when prompted during the repository cloning process. Navigate to your project folder using the following command: cd School-Quiz Replace School-Quiz with the actual name of your project. To install the project dependencies, run: npm install Once the packages are installed, you can start the project by running: npm start In the console, you should see the message “Bot is running”. However, there is one issue—if you restart the server or close the console, the bot will stop working! To ensure the bot runs continuously and automatically starts after a server reboot, you need to install a process manager like pm2. Install pm2 globally using the following command: sudo npm install pm2 -g Next, start the Node.js server using pm2: sudo pm2 start index.js --name "bot-quiz" --watch In this example, the process is named bot-quiz, but you can use any name you prefer. Set up automatic startup on server reboot: sudo pm2 startup Save all the changes made: sudo pm2 save Conclusion In this guide, we covered the entire process of creating a Telegram bot using Node.js, from registering the bot via BotFather to deploying the finished solution on a server.
31 January 2025 · 15 min to read
Node.js

How to Update Node.js Version

Node.js stands as a robust JavaScript runtime environment powered by Chrome's V8 engine. With its capabilities, developers can construct scalable network applications with simplicity. Renowned for its event-driven, non-blocking architecture, it’s perfect for creating real-time applications. Regularly refreshing your tools ensures access to the newest features, security fixes, and performance gains. Updating Node.js regularly is crucial for the stability and security of projects, regardless of their scale. This all-inclusive guide will navigate you through diverse methods to update Node.js, covering everything from package managers to hands-on manual installations. Method 1: Via NVM Node Version Manager, abbreviated as NVM, is extensively employed for seamless handling of various Node.js versions. Its flexibility and user-friendly interface make it particularly popular among developers. This tool facilitates easy switching between node versions, perfect for projects that demand particular Node.js versions. Why Use NVM? Flexibility: Easily shift from one node version to another. Convenience: Handle installations, updates, and management of various versions effortlessly. Isolation: Isolates different versions to minimize conflicts. Step-by-Step Guide Adhere to these guidelines to set up and utilize NVM for node version management. Install NVM Initiate a terminal session and input: curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash Then configure NVM in your shell profile like Bash: export NVM_DIR="$HOME/.nvm" [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm [ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion Next, confirm the NVM installation with: nvm --version Update Version List available versions first, then proceed with installation: nvm ls-remote Subsequently, install the latest release via: nvm install node Then, set the newly installed version as the default: nvm alias default node Validate the installation through: node -v Update npm To guarantee superior performance and safety, keep npm up-to-date alongside node: npm install -g npm Lastly, validate the updated npm version for confirmation via: npm -v Switching Node Versions First, list out the installed versions through: nvm ls Next, switch to another version: nvm use <version> Insert the required version number in place of <version>. Method 2: Via Node.js Binary Packages Direct installation from Node.js official binaries is hassle-free and secures the specific release straight from the source. Why Use Binary Packages? Simplicity: Achieve direct installation without using additional tools. Consistency: Backed by the official Node.js development team. Reliability: Guarantees the exact release comes from the official source. Step-by-Step Guide Adopt these guidelines to update Node.js through binary packages. Obtain the Binary Package Access the Node.js website and grab the binary package compatible for your particular OS. Install Node.js Finish the installation by adhering to the guidelines specific to your OS: Windows: Run the installer and finish by following the on-screen steps. macOS: Execute the .pkg file and continue via the setup process. Linux: Unpack the tarball and transfer the contents to /usr/local. tar -xvf node-v<version>-linux-x64.tar.xz sudo cp -r node-v<version>-linux-x64/bin /usr/local/ Next, access the .bashrc file: nano ~/.bashrc Then, insert the lines below: export PATH="/usr/local/bin:$PATH" Save the file and use source to update settings: source ~/.bashrc Verify Installation Validate the installation via: node -v Check npm release: npm -v Method 3: Via Package Managers Homebrew and apt facilitate Node.js installation and updates by handling dependency and version management. Why Use Package Managers? Ease of Use: Simplifies installation and updates. Automation: Handles dependencies and version management. Integration: Easily integrates with other software and systems. Step-by-Step Guide Apply these procedures to upgrade Node.js via package managers. Via Homebrew (macOS) Employ the instructions on the Homebrew website if not already installed. Enter the subsequent command: brew install node For upgrading the existing installation: brew upgrade node Validate the current installation: node -v Via apt (Ubuntu/Debian) Include the node’s PPA in your system setup: curl -fsSL https://deb.nodesource.com/setup_23.x | sudo -E bash - Update 23.x to the desired release. Enter the subsequent command: sudo apt install nodejs Check that the node release is correct: node -v Method 4: Via Node.js Version Manager (n) The n package offers an efficient method for managing Node.js versions, serving as an alternative to NVM. This tool transforms the way you install, switch, and maintain different node versions on your system. Why Use n? Efficiency: Fast and lightweight. Simplicity: User-friendly and requires minimal commands. Control: Complete control over the versions set up. Step-by-Step Guide Here's how to configure and use the n package for managing node versions. Install n Run npm to globally install the n package: npm install -g n Install or Update Node.js Install the latest node release: sudo n latest Install a specific release: sudo n <version> Change <version> to the required release number. Verify Installation Confirm version: node -v Method 5: Manual Installation Manual installation suits advanced users seeking complete control over the setup. Why Use Manual Installation? Control: Complete authority over the installation workflow. Customization: Customize the build and installation settings. Step-by-Step Guide Adhere to these guidelines for manual installation: Install Dependencies Install essential dependencies first via: sudo apt install build-essential gcc g++ make Download Source Code Head over to the official website and obtain the source code. wget https://nodejs.org/download/release/v23.6.0/node-v23.6.0.tar.gz Build and Install Extract the source code: tar -xvf node-v<version>.tar.gz Replace <version> with the version number. Navigate to the extracted directory: cd node-v<version> Configure and compile the source code: ./configure make If you encounter dependency errors while running the above command, use your default package manager to install them. Install the tool: sudo make install Verify Installation Confirm version: node -v Additional Resources For in-depth information on Node.js updates and management, consider these resources: Documentation: Comprehensive resource for all Node.js things. NVM GitHub Repository: Extensive support and information for NVM. n GitHub Repository: Information on applying the n package for node version control. Release Notes: Remain informed about the newest enhancements and updates. Best Practices Regular Checks: Make it a habit to regularly check for node updates to ensure your environment is always up-to-date. Backup: Always create backups of your projects and important data before updating to safeguard against any loss during the upgrade. Testing: Upon completing the update, thoroughly test your applications to verify they function correctly with the new node release. Documentation: Keep your project documentation in sync with the latest Node.js release and note any changes from the update. Fixing Common Problems Running into problems while updating Node.js? Discover frequent issues and effective fixes: Issue 1: Version Not Changing  In case the node version stays the same after updating, attempt the following: Clear npm cache: npm cache clean -f Reinstall node using the desired method. Issue 2: Permission Errors  If permission problems occur, use sudo for elevated command execution: sudo npm install -g n Issue 3: Dependency Conflicts  Sometimes, updating the application can result in conflicts with dependencies in your current projects.  To resolve this: Utilize nvm to pick the necessary release for each project. Ensure dependencies in your project are updated to align with the new Node.js release. Conclusion There are various methods available to update Node.js, each suited to specific needs and preferences. Whether it's NVM, binary packages, package managers, the n package, or manual installation, you can keep your dev environment up-to-date with the newest features and security fixes. Frequent updates are always a key factor to maintaining top performance and safety. Consistently update Node.js to benefit from the latest features, security enhancements, and performance boosts for robust and secure applications. Using this in-depth guide, you can expertly handle and update your node installations according to your specific needs and preferences. In addition, check out our platform as a service to deploy various Node.js frameworks, such as Express, Fastify, Hapi and Nest.
23 January 2025 · 7 min to read
Node.js

How to Handle Asynchronous Tasks with Node.js and BullMQ

Handling asynchronous tasks efficiently is crucial in Node.js applications, especially when dealing with time-intensive operations like sending emails, processing images, or performing complex calculations. Without proper management, these tasks can block the event loop, leading to poor performance and a subpar user experience. This is where BullMQ comes into play. BullMQ is a powerful Node.js package that offers a reliable and scalable queuing system powered by Redis. It enables developers to transfer heavy operations to a queue in the background, keeping the main application responsive. With BullMQ you can successfully manage async queues, plan processes, and easily keep an eye on their progress. This tutorial will show you how to manage asynchronous tasks with Node.js and BullMQ. The process involves setting up a project folder, performing a time-intensive task without using BullMQ, and enhancing the application by incorporating BullMQ for running tasks in parallel. Prerequisites Before you begin, ensure you: Set up a Linux VPS server. Set up Node.js on your server. Set up Redis on your server, as BullMQ depends on Redis for managing queues. Setting Up the Project Directory Before you can use Node.js and BullMQ for asynchronous tasks, it is necessary to establish your project directory. Set up and customize your Node.js application using these guidelines. Create a New Directory Open your terminal and go to the location of your project. Create a fresh folder and navigate into it: mkdir bullmq-demo && cd bullmq-demo Initialize a New Node.js Project Set up a Node.js project using npm. It generates a package.json file containing the default configurations: npm init -y Install Required Dependencies Set up the required packages for your application: npm install express bullmq ioredis Here's what each package does: express: A fast Node.js web framework commonly used for server creation. bullmq: An excellent tool for handling queues within Node.js programs. ioredis: A Redis client for Node.js that BullMQ needs in order to establish a connection with Redis. Create the Main Application File Create an index.js file as the primary access point for your application: touch index.js Alternatively, you have the option to generate this file by using your code editor. Set Up a Basic Express Server To set up a simple Express server, include this code in your index.js file: const express = require('express'); const app = express(); const port = 3000; app.use(express.json()); app.listen(port, () => { console.log(`Server is running on port ${port}`); }); This code initiates an Express app on port 3000 which handles requests using JSON middleware. Verify the Server Setup Start the server by running: node index.js The below message should appear: Open up your internet browser and go to either http://your_server_ip:3000 or http://localhost:3000. You will receive a "Cannot GET /" message as there are no routes set up, as anticipated. When ready to proceed, you can terminate the server using Ctrl + C. Implementing a Time-Intensive Task Without BullMQ This part describes how to include a route in your Express app that performs a time-consuming task in a synchronous way. This will demonstrate how specific tasks can block the event loop and negatively affect your application's performance. Define a Time-Intensive Function Create a function in the index.js file that simulates a computationally intensive task: // Function to simulate a heavy computation function heavyComputation() { const start = Date.now(); // Run a loop for 5 seconds while (Date.now() - start < 5000) { // Perform a CPU-intensive task Math.sqrt(Math.random()); } } The function runs a loop for about five seconds, performing math operations to mimic a CPU-heavy task. Create a Route to Handle the Task Create a fresh route in your Express application that calls the heavyComputation function: app.get('/heavy-task', (req, res) => { heavyComputation(); res.send('Heavy computation finished'); }); This route is set up to receive GET requests specifically at the /heavy-task endpoint. After receiving a request, it carries out the specified intensive computation and then provides a response. Start the Server To restart your server, execute the following command: node index.js Confirm the server is functioning before moving on to the next stage. Test the Heavy Task Route Open your internet browser and type in either http://your_server_ip:3000/heavy-task or http://localhost:3000/heavy-task to access the webpage.  The following message should be displayed: It is important to observe that the response time is approximately five seconds. The delay is a result of the synchronous execution of the intensive computation process. Observe Blocking Behavior After the server is up and running, open a new tab on your internet browser and go to http://your_server_ip:3000/. The response to this request may not be immediate. The system delays taking action until the extensive processing of the previous step. This happens when the time-consuming task is blocking the Node.js event loop, stopping the server from processing additional incoming requests. When the server performs a task that takes a lot of time in a synchronous manner, it is unable to respond to additional requests. The act of blocking could result in a suboptimal user experience, particularly in apps that need to be highly responsive. Executing Time-Intensive Tasks Asynchronously with BullMQ We saw in the last section how synchronous execution of time-consuming operations can severely affect your application's performance by slowing down the event loop. This section explains how to implement a high-performance asynchronous queue into your application using BullMQ. Modify index.js to Use BullMQ Make changes to the index.js file to include BullMQ in your application. Import BullMQ and ioredis At the top of your index.js file, you should include the following import statements: const { Queue, Worker } = require('bullmq'); const Redis = require('ioredis'); Create a Redis Connection Next, set up a connection with Redis: const connection = new Redis(); Redis has been programmed to run on port 6379 and the localhost interface by default. To create a connection to a remote Redis server that has a different port, please enter the appropriate host address and port number: const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); Initialize a BullMQ Queue Create a new queue called heavyTaskQueue: const heavyTaskQueue = new Queue('heavyTaskQueue', { connection }); Add a Route to Enqueue Tasks Change the heavy-task route to add a job to the queue instead of running the task right away: app.get('/heavy-task', async (req, res) => { await heavyTaskQueue.add('heavyComputation', {}); res.send('Heavy computation job added to the queue'); }); The application will respond after a lengthy process has completed, handling requests asynchronously, when the /heavy-task route is accessed. Remove the Worker Code from index.js The worker must be implemented in a separate file. This is essential to ensure that the worker does not coexist with the Express server process. A worker's use of the heavyComputation function during execution won't interfere with the event loop of the main application. The index.js file is structured in the following way: const express = require('express'); const app = express(); const port = 3000; app.use(express.json()); const { Queue } = require('bullmq'); const Redis = require('ioredis'); const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); const heavyTaskQueue = new Queue('heavyTaskQueue', { connection }); app.get('/heavy-task', async (req, res) => { await heavyTaskQueue.add('heavyComputation', {}); res.send('Heavy computation job added to the queue'); }); app.listen(port, () => { console.log(`Server is running on port ${port}`); }); Create a Separate Worker File Generate a fresh file and name it worker.js. The file is intended for executing the worker code in charge of handling tasks obtained from the queue. Create the worker.js file: touch worker.js Add Worker Code to worker.js: const { Worker } = require('bullmq'); const Redis = require('ioredis'); const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); // Function to simulate a heavy computation function heavyComputation() { const start = Date.now(); // Run a loop for 5 seconds while (Date.now() - start < 5000) { // Perform a CPU-intensive task Math.sqrt(Math.random()); } } const worker = new Worker( 'heavyTaskQueue', async job => { // Time-intensive task here heavyComputation(); console.log('Heavy computation completed'); }, { connection } ); worker.on('completed', job => { console.log(`Job ${job.id} has completed`); }); worker.on('failed', (job, err) => { console.log(`Job ${job.id} has failed with error ${err.message}`); }); Run the Worker in a Separate Process You must now execute worker.js as an independent Node.js process. Start the Worker Process Open a new terminal window or tab, navigate to your project folder, and run the specified command: node worker.js Start the Express Server Initiate the Express server in your original terminal window: node index.js Test the Application with BullMQ Proceed to conduct testing of the application utilizing BullMQ.  Make a Request to /heavy-task:Open your internet browser and type in either http://your_server_ip:3000/heavy-task or http://localhost:3000/heavy-task in the URL bar. The following message should be displayed: Heavy computation job added to the queue. The rapid response time suggests that there is no blockage in the main thread. Adding a Dashboard to Monitor BullMQ Queues Monitoring your application's queues and jobs is essential for ensuring they are functioning properly and for troubleshooting purposes. BullMQ comes with a functionality called Bull Board, which offers a visual interface for overseeing your queues. This part explains how to incorporate a dashboard into your application. Install Bull Board Use npm to install the @bull-board/express package: npm install @bull-board/express Set Up Bull Board in Your Application In order to set up the bull board application, follow these steps: Import Bull Board Modules Insert the code provided at the top of your index.js file: const { createBullBoard } = require('@bull-board/api'); const { BullMQAdapter } = require('@bull-board/api/bullMQAdapter'); const { ExpressAdapter } = require('@bull-board/express'); Create an Express Adapter for the Dashboard Initialize the Express adapter: const serverAdapter = new ExpressAdapter(); serverAdapter.setBasePath('/admin/queues'); Set Up Bull Board with Your Queue Create the Bull Board instance and pass your queue: createBullBoard({ queues: [new BullMQAdapter(heavyTaskQueue)], serverAdapter: serverAdapter, }); Use the Dashboard in Your Express App Add the following line to mount the dashboard at /admin/queues: app.use('/admin/queues', serverAdapter.getRouter()); Make sure to include this line following the setup of your queue and worker. The final index.js file looks like below: // Import Express and Initialize App const express = require('express'); const app = express(); const port = 3000; app.use(express.json()); // Import BullMQ and Redis const { Queue } = require('bullmq'); const Redis = require('ioredis'); // Redis Connection const connection = new Redis({ host: '127.0.0.1', port: 6379, maxRetriesPerRequest: null, }); // Initialize Queue const heavyTaskQueue = new Queue('heavyTaskQueue', { connection }); // Define Route to Add Job to Queue app.get('/heavy-task', async (req, res) => { await heavyTaskQueue.add('heavyComputation', {}); res.send('Heavy computation job added to the queue'); }); // Import Bull Board and Set Up Dashboard const { createBullBoard } = require('@bull-board/api'); const { BullMQAdapter } = require('@bull-board/api/bullMQAdapter'); const { ExpressAdapter } = require('@bull-board/express'); const serverAdapter = new ExpressAdapter(); serverAdapter.setBasePath('/admin/queues'); createBullBoard({ queues: [new BullMQAdapter(heavyTaskQueue)], serverAdapter: serverAdapter, }); app.use('/admin/queues', serverAdapter.getRouter()); // Start the Server app.listen(port, () => { console.log(`Server is running on port ${port}`); }); Access the Dashboard To access the dashboard, follow the steps listed below: Restart Your Server node index.js Navigate to the Dashboard Open your browser and go to http://your_server_ip:3000/admin/queues. Explore the Dashboard: Queue Overview: See the list of queues and their status. Jobs List: View active, completed, failed, and delayed jobs. Job Details: Click on a job to see its data, logs, and stack trace if it failed. You can easily manage your BullMQ queues by integrating Bull Board into your application. It is much easier to keep an eye on progress and identify issues when you can view your queues and tasks on the dashboard in real-time. Conclusion You have now learned how to use BullMQ with Node.js to manage asynchronous processes. Your application's responsiveness and efficiency have been enhanced by moving time-consuming operations to a separate queue. Your Node.js app is now much more capable of handling heavy demands thanks to the usage of queues.
28 November 2024 · 11 min to read

Do you have questions,
comments, or concerns?

Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
Email us
Hostman's Support