Sign In
Sign In

Deploying Python Applications with Gunicorn

Deploying Python Applications with Gunicorn
Hostman Team
Technical writer
Python
23.10.2024
Reading time: 7 min

In this article, we’ll show how to set up an Ubuntu 20.04 server and install and configure the components required for deploying Python applications. We’ll configure the WSGI server Gunicorn to interact with our application. Gunicorn will serve as an interface that converts client requests via the HTTP protocol into Python function calls executed by the application. Then, we will configure Nginx as a reverse proxy server for Gunicorn, which will forward requests to the Gunicorn server. Additionally, we will cover securing HTTP connections with an SSL certificate or using other features like load balancing, caching, etc. These details can be helpful when working with cloud services like those provided by Hostman.

Creating a Python Virtual Environment

To begin, we need to update all packages:

sudo apt update

Ubuntu provides the latest version of the Python interpreter by default. Let’s check the installed version using the following command:

python3 --version

Example output:

Python 3.10.12

We’ll set up a virtual environment to ensure that our project has its own dependencies, separate from other projects. First, install the virtualenv package, which allows you to create virtual environments:

sudo apt-get install python3-venv python3-dev

Next, create a folder for your project and navigate into it:

mkdir myapp
cd myapp

Now, create a virtual environment:

python3 -m venv venv

And create a folder for your project:

mkdir app

Your project directory should now contain two items: app and venv.

You can verify this using the standard Linux command to list directory contents:

ls

Expected output:

myapp venv

You need to activate the virtual environment so that all subsequent components are installed locally for the project:

source venv/bin/activate

Installing and Configuring Gunicorn

Gunicorn (Green Unicorn) is a Python WSGI HTTP server for UNIX. It is compatible with various web frameworks, fast, easy to implement, and uses minimal server resources.

To install Gunicorn, run the following command:

pip install gunicorn

WSGI and Python

WSGI (Web Server Gateway Interface) is the standard interface between a Python application running on the server side and the web server itself, such as Nginx. A WSGI server interacts with the application, allowing you to run code when handling requests. Typically, the application is provided as an object named application in a Python module, which is made available to the server.

In the standard wsgi.py file, there is usually a callable application. For example, let’s create such a file using the nano text editor:

nano wsgi.py

Add the following simple code to the file:

from aiohttp import web

async def index(request):
    return web.Response(text="Welcome home!")

app = web.Application()
app.router.add_get('/', index)

In the code above, we import aiohttp, a library that provides an asynchronous HTTP client built on top of asyncio. HTTP requests are a classic example of where asynchronous handling is ideal, as they involve waiting for server responses, during which other code can execute efficiently. This library allows sequential requests to be made without waiting for the first response before sending a new one. It’s common to run aiohttp servers behind Nginx.

Running the Gunicorn Server

You can launch the server using the following command template:

gunicorn [OPTIONS] [WSGI_APP]

Here, [WSGI_APP] consists of $(MODULE_NAME):$(VARIABLE_NAME) and [OPTIONS] is a set of parameters for configuring Gunicorn.

A simple command would look like this:

gunicorn wsgi:app

To restart Gunicorn, you can use:

sudo systemctl restart gunicorn

Systemd Integration

systemd is a system and service manager that allows for strict control over processes, resources, and permissions. We’ll create a socket that systemd will listen to, automatically starting Gunicorn in response to traffic.

Configuring the Gunicorn Service and Socket

First, create the service configuration file:

sudo nano /etc/systemd/system/gunicorn.service

Add the following content to the file:

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
Type=notify
User=someuser
Group=someuser
RuntimeDirectory=gunicorn
WorkingDirectory=/home/someuser/myapp
ExecStart=/path/to/venv/bin/gunicorn wsgi:app
ExecReload=/bin/kill -s HUP $MAINPID
KillMode=mixed
TimeoutStopSec=5
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Make sure to replace /path/to/venv/bin/gunicorn with the actual path to the Gunicorn executable within your virtual environment. It will likely look something like this: /home/someuser/myapp/venv/bin/gunicorn.

Next, create the socket configuration file:

sudo nano /etc/systemd/system/gunicorn.socket

Add the following content:

[Unit]
Description=gunicorn socket

[Socket]
ListenStream=/run/gunicorn.sock
SocketUser=www-data

[Install]
WantedBy=sockets.target

Enable and start the socket with:

systemctl enable --now gunicorn.socket

Configuring Gunicorn

Let's review some useful parameters for Gunicorn in Python 3. You can find all possible parameters in the official documentation.

Sockets

  • -b BIND, --bind=BIND — Specifies the server socket. You can use formats like: $(HOST), $(HOST):$(PORT).

Example:

gunicorn --bind=127.0.0.1:8080 wsgi:app

This command will run your application locally on port 8080.

Worker Processes

  • -w WORKERS, --workers=WORKERS — Sets the number of worker processes. Typically, this number should be between 2 to 4 per server core.

Example:

gunicorn --workers=2 wsgi:app

Process Type

  • -k WORKERCLASS, --worker-class=WORKERCLASS — Specifies the type of worker process to run.

By default, Gunicorn uses the sync worker type, which is a simple synchronous worker that handles one request at a time. Other worker types may require additional dependencies.

Asynchronous worker processes are available using Greenlets (via Eventlet or Gevent). Greenlets are a cooperative multitasking implementation for Python. The corresponding parameters are eventlet and gevent.

We will use an asynchronous worker type compatible with aiohttp:

gunicorn wsgi:app --bind localhost:8080 --worker-class aiohttp.GunicornWebWorker

Access Logging

You can enable access logging using the --access-logfile flag.

Example:

gunicorn wsgi:app --access-logfile access.log

Error Logging

To specify an error log file, use the following command:

gunicorn wsgi:app --error-logfile error.log

You can also set the verbosity level of the error log output using the --log-level flag. Available log levels in Gunicorn are:

  • debug

  • info

  • warning

  • error

  • critical

By default, the info level is set, which omits debug-level information.

Installing and Configuring Nginx

First, install Nginx with the command:

sudo apt install nginx

Let’s check if the Nginx service can connect to the socket created earlier:

sudo -u www-data curl --unix-socket /run/gunicorn.sock http

If successful, Gunicorn will automatically start, and you'll see the HTML code from the server in the terminal.

Nginx configuration involves adding config files for virtual hosts. Each proxy configuration should be stored in the /etc/nginx/sites-available directory.

To enable each proxy server, create a symbolic link to it in /etc/nginx/sites-enabled. When Nginx starts, it automatically loads all proxy servers in this directory.

Create a new configuration file:

sudo nano /etc/nginx/sites-available/myconfig.conf

Then create a symbolic link with the command:

sudo ln -s /etc/nginx/sites-available/myconfig.conf /etc/nginx/sites-enabled

Nginx must be restarted after any changes to the configuration file to apply the new settings.

First, check the syntax of the configuration file:

nginx -t

Then reload Nginx:

nginx -s reload

Conclusion

Gunicorn is a robust and versatile WSGI server for deploying Python applications, offering flexibility with various worker types and integration options like Nginx for load balancing and reverse proxying. Its ease of installation and configuration, combined with detailed logging and scaling options, make it an excellent choice for production environments. By utilizing Gunicorn with frameworks like aiohttp and integrating it with Nginx, you can efficiently serve Python applications with improved performance and resource management.

Python
23.10.2024
Reading time: 7 min

Similar

Python

How to Use the numpy.where() Method in Python

The numpy.where() method in Python is one of the most powerful and frequently used tools in the NumPy library for the conditional selection of elements from arrays. It provides flexible options for processing and analyzing large datasets, replacing traditional if-else conditional operators and significantly speeding up code execution. This method allows you to replace elements in an array that meet a certain condition with specified values while leaving other elements unchanged. Unlike regular loops, which can slow down execution when working with large datasets, numpy.where() uses vectorization, making operations faster and more efficient. Syntax of the where() Method The numpy.where() method has the following syntax: numpy.where(condition[, x, y]) Where: condition: the condition or array of conditions to be checked. x: values returned if the condition is True. y: values returned if the condition is False. If the arguments x and y are not specified, the method will return the indices of the elements that satisfy the condition. Main Usage Approaches Let's move on to practical examples. Finding Element Indices It is often necessary to determine the positions of elements that satisfy a certain condition. numpy.where() makes this easy to achieve: import numpy as np arr = np.array([1, 2, 3, 4, 5]) indices = np.where(arr > 3) print(indices) In this example, we create an array [1, 2, 3, 4, 5]. Then, we use the np.where() function to find the indices of elements greater than 3. Running the code yields (array([3, 4]),), indicating the positions of the numbers 4 and 5 in the original array, as only these numbers satisfy the condition arr > 3. In this case, the method returns a tuple containing an array of indices for elements greater than 3. Conditional Element Replacement The numpy.where() method is widely used for conditionally replacing elements in an array: import numpy as np arr = np.array([1, 2, 3, 4, 5]) result = np.where(arr > 3, 100, arr) print(result) This code starts by creating an array [1, 2, 3, 4, 5]. The np.where() function is then used to find elements greater than 3. The additional parameter 100 allows these elements to be replaced with the specified value. The resulting output is [1, 2, 3, 100, 100], where the elements 4 and 5 have been replaced with 100 because they satisfy the condition arr > 3. In this case, np.where() replaces all elements meeting the condition with the specified value. Working with Multidimensional Arrays The numpy.where() method also works effectively with multidimensional arrays: import numpy as np matrix = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) result = np.where(matrix % 2 == 0, 'even', 'odd') print(result) This example creates a matrix [[1, 2, 3], [4, 5, 6], [7, 8, 9]]. The np.where() function is applied to replace elements based on the condition: if the number is even (divisible by 2 without a remainder), it is replaced with the string 'even'; otherwise, it is replaced with 'odd'. The resulting matrix is printed as: [['odd' 'even' 'odd'] ['even' 'odd' 'even'] ['odd' 'even' 'odd']] In this example, the method returns an updated matrix with strings instead of numbers. Applying Multiple Conditions By using logical operators, numpy.where() can handle more complex conditions: import numpy as np arr = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9]) result = np.where((arr > 3) & (arr < 7), arr * 2, arr) print(result) In this example, an array [1, 2, 3, 4, 5, 6, 7, 8, 9] is created. The np.where() function is used with a combined condition: if the number is greater than 3 and less than 7, it is multiplied by 2; otherwise, it remains unchanged. The output is: [1, 2, 3, 8, 10, 12, 7, 8, 9] The numbers 4, 5, and 6 are multiplied by 2 as they meet the condition. In this case, the method returns a new array with updated values based on the condition. Practical Examples Working with Temperature Data Suppose we have an array of temperatures in Celsius, and we want to classify them as hot" or "comfortable": import numpy as np temperatures = np.array([23, 25, 28, 32, 35, 29]) status = np.where(temperatures > 30, 'hot', 'comfortable') print(status) In this example, the temperature array [23, 25, 28, 32, 35, 29] is created. The np.where() function is applied to determine comfort levels: if the temperature exceeds 30 degrees, it is labeled as 'hot'; otherwise, it is 'comfortable'.  The output is:  ['comfortable' 'comfortable' 'comfortable' 'hot' 'hot' 'comfortable']  Temperatures 32 and 35 degrees are marked as 'hot' because they exceed the threshold.  This method returns a new array with string values reflecting the temperature evaluation. Handling Missing Values In datasets, missing values often need to be replaced or handled: import numpy as np data = np.array([1, np.nan, 3, np.nan, 5]) cleaned_data = np.where(np.isnan(data), 0, data) print(cleaned_data) Here, we create an array with missing values [1, np.nan, 3, np.nan, 5]. The np.where() function is combined with np.isnan() to replace missing values (NaN) with 0.  The result is: [1. 0. 3. 0. 5.] The NaN values are replaced with 0, while other elements remain unchanged.  This example demonstrates how to clean data by handling missing values. Method Comparison Table Characteristic numpy.where() Loops List Comprehension Speed High Low Medium Memory Usage Medium High Medium Readability High Medium High Vectorization Yes No Partially Flexibility High High High As the table shows, numpy.where() outperforms traditional loops and list comprehensions in terms of speed and memory efficiency, while maintaining high readability and flexibility. Conclusion The numpy.where() method is an indispensable tool for efficient data processing and analysis in Python. Its use allows developers to write more performant, clean, and readable code, especially when working with large datasets and complex conditions. This method simplifies tasks related to replacing array elements based on specified conditions and eliminates the need for bulky loops and checks, making the code more compact and faster. numpy.where() is particularly useful for handling large datasets where high performance and simple conditional operations are crucial. Loops remain a better choice for complex data processing logic or step-by-step operations, especially when working with smaller datasets. On the other hand, list comprehensions are suitable for compact and readable code when dealing with small to medium datasets, provided the operations are not overly complex. Understanding the syntax and capabilities of numpy.where() opens up new approaches for solving various problems in areas such as data analysis, image processing, and financial analysis. The method enables efficient handling of large data volumes and significantly accelerates operations through vectorization, which is particularly important for tasks requiring high performance. Using techniques like vectorization and masks in combination with NumPy functions helps developers optimize code and achieve fast and accurate results. Regardless of your level of experience in Python programming, mastering numpy.where() and understanding its advantages will be a crucial step toward more efficient data handling, improving program performance, and implementing optimal solutions in analytics and information processing.
06 February 2025 · 6 min to read
Python

How to Set Up Visual Studio Code for Python

Creating and debugging programs in Python is easier when using a specialized Integrated Development Environment (IDE). With an IDE, you can quickly and efficiently develop, test, and debug programs. Visual Studio Code (VS Code) for Python provides full support for the language and offers a wide range of plugins and extensions. In this article, we will install Visual Studio Code on three operating systems (Windows, macOS, Linux) and set it up for Python programming, including the use of popular plugins. Prerequisites To install and set up Visual Studio Code for Python, we will need the following: A personal or work computer with Windows 10/11, macOS, or Ubuntu Linux distribution version 24.04 pre-installed. Alternatively, you can rent a dedicated server or a virtual machine with Windows Server 2016/2019/2022. If using regular versions of Windows, you can download your own ISO image in advance. You can also rent a server with Ubuntu. Installing the Python Interpreter Before installing VS Code, we need to install the Python interpreter on all three operating systems — Windows, macOS, and Linux. On Windows Go to the official Python website and download the installer file. In this case, we will be installing Python version 3.13.1. Run the installer file. You will have two installation options: Install Now — This performs a full installation, including documentation files, the package manager pip, the tcl/tk library for graphical interface support, and standard libraries. Customize Installation — This option allows you to choose which components to install. We will use the full installation. Make sure to check the box next to the option Add python.exe to PATH and click Install Now. The installation process will begin. Once the installation is complete, the program will notify you that it has finished. On macOS On macOS, the Python interpreter is pre-installed by default. You can verify this by running the following command in the terminal: python3 --version However, the installed version may be outdated. If necessary, you can install a newer version. To do this, we will use the Homebrew package manager. First, if Homebrew is not installed on your system, you can install it by running the following command: /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" Next, you need to check which versions of Python are available for installation. Use the following command: brew search python In our case, several versions of Python are available: Install the latest available version, Python 3.13, by running the following command: brew install [email protected] Check the Python version again: python3 --versionpython3.13 --version As shown in the screenshot above, running the python3 --version still shows the old version (Python 3.9.6). However, the newly installed version (3.13) can be accessed using the command python3.13 --version. If needed, you can change the default Python version to the newly installed one. To do this, first get the full path to the newly installed Python interpreter using the following command: brew --prefix [email protected] Then, check which shell you are using: echo $SHELL Depending on the shell used, open the corresponding file for editing: For bash or sh: nano ~/.bashrc For zsh: nano ~/.zshrc Add the following line at the end of the file: export PATH="/opt/homebrew/opt/[email protected]/bin:$PATH" Save the changes and reload the file: source ~/.zshrc Now, when you check the Python version, it will display the latest installed version: python3 --version On Ubuntu By default, Python is pre-installed on almost all Linux distributions, including Ubuntu. In the latest supported versions of Ubuntu, the current version of Python is installed: python3 --version However, if Python is not installed for any reason, you can install it by running the following command: apt -y install python3 Installing Visual Studio Code You can install Visual Studio Code on your personal computer. You can also rent a dedicated or cloud server with Windows Server or one of the available Linux distributions. If the required distribution is not available in the list of offered images, you can upload your own. We will cover the installation of Visual Studio Code on three operating systems: Windows, macOS, and Linux (Ubuntu 24.04 distribution). For Windows Visual Studio Code supports installation on Windows 10 and Windows 11. It also supports Windows Server distributions, from version 2016 to 2022. We will install it on Windows 10.  Go to the official website and download the installer. This will download an .exe installation file. Run the installer file.  On the first step, accept the license agreement by selecting the option "I accept the agreement". Next, the installer will prompt you to choose an installation location. You can choose the default path suggested by the installer or specify your own. If necessary, you can create a shortcut for the program in the Windows menu. If you don’t want to create a shortcut, select the option "Don’t create a Start Menu folder" at the bottom: The next step lets you configure additional options by ticking the corresponding checkboxes: Create a desktop icon — creates a shortcut on the desktop for quick access to the program. Add “Open with Code” action to Windows Explorer file context menu — adds the "Open with Code" option to the context menu when right-clicking on a file. This option allows you to quickly open any file directly in Visual Studio Code. Add “Open with Code” action to Windows Explorer directory file context menu — similar to the above option but adds the "Open with Code" option to the context menu of directories (folders). Register Code as an editor for supported file types — makes Visual Studio Code the default editor for certain file types (e.g., .c, .cpp, .py, .java, .js, .html files). Add to PATH (require shell restart) — adds Visual Studio Code to the system’s PATH variable so it can be launched from the command line (cmd). Once all necessary options are set, Visual Studio Code is ready for installation. Click Install. After the installation is complete, you can launch the program immediately: For macOS Go to the official website and download the installer: After downloading, you will have a ZIP archive. Inside the archive, you will find the executable file, which you need to extract to the "Applications" directory. On the first launch, the system will notify you that this file was downloaded from the internet and may not have vulnerabilities. Click Open to continue: For Linux (Ubuntu) Visual Studio Code supports installation on Linux distributions such as Ubuntu, Debian, Red Hat, Fedora, and SUSE. You need a graphical desktop environment to install Visual Studio Code on Linux (GNOME, KDE, Xfce, etc.). Let’s consider the installation of Visual Studio Code on Ubuntu 24.04 with the Xfce desktop environment. You can also install Visual Studio Code using Snapcraft. Go to the official website and download the installer for your Linux distribution. In our case, we need the .deb installer: Once the file is downloaded, open a terminal (console) and navigate to the directory where the file was downloaded (e.g., /root). To install, run the following command where code_1.96.2-1734607745_amd64.deb is the name of the downloaded file: dpkg -i code_1.96.2-1734607745_amd64.deb During installation, a message will prompt you to add Microsoft repositories to the system. Select <Yes> and press Enter: Wait for the installation to complete. Once the installation is finished, you can launch the program from the applications menu (for distributions using Xfce, Visual Studio Code is available in the menu: Applications → Development): Adding Python Interpreter to PATH Variable in Windows If you haven't checked the Add python.exe to PATH checkbox during the Python installation on Windows, you need to manually add the full path to the interpreter to run Python from the command line. To do this: Press Win+R, type sysdm.cpl in the Run window, and press Enter. In the window that opens, go to the Advanced tab and click on the Environment Variables button. To add a user-level variable, select the Path variable under User variables and click on Edit. Double-click on an empty field or click the New button. Enter the full path to the Python interpreter file. By default, the Python interpreter is located at the following path: C:\Users\<Username>\AppData\Local\Programs\Python\Python313 For example: C:\Users\Administrator\AppData\Local\Programs\Python\Python313 After entering the path, click OK to save the changes. To verify, open the command prompt and type python. If the path to the interpreter is correctly specified, the Python console will open. Setting Up Python Interpreter in Visual Studio Code In Windows Once Python is installed, you need to connect it to Visual Studio Code. To do this: Open Visual Studio Code and click the New File... button on the home page to create a new file.  Alternatively, you can create a Python project in Visual Studio Code by clicking on Open Folder…, where you can select the entire project folder containing the files. Type any name for the file, use the .py extension, and press Enter. Save the file in any location. Ensure that the file name ends with the .py extension. Once the file is saved, the interface in Visual Studio Code will display a prompt at the bottom right, suggesting you install the recommended Python extension. To run Python in Visual Studio Code, you first need to select the Python interpreter. A button will appear at the bottom of the panel with a warning: Select Interpreter. Click on it. In the menu that appears, select Enter interpreter path… and press Enter. Specify the full path to the Python interpreter. By default, it is located at: C:\Users\Administrator\AppData\Local\Programs\Python\Python313 where Administrator is your user account name. Select the file named python and click on Select Interpreter. To test it, write a simple program that calculates the square root of a number: import math num = float(input("Enter a number to find its square root: ")) if num >= 0: sqrt_result = math.sqrt(num) print(f"The square root of {num} is {sqrt_result}") else: print("Square root of a negative number is not real.") In Visual Studio Code, to run the Python code, click the Run Python File button at the top-right. If the interpreter is set up correctly, the program will run successfully. In macOS On macOS operating systems, the Python interpreter is automatically recognized. Simply create a new .py file as described in the Windows section above and run the program directly. In Ubuntu Similarly to macOS, in the Ubuntu distribution, VS Code automatically detects the installed Python interpreter in the system. All you need to do is create a new .py file and run the program directly. Recommended Extensions for Python in Visual Studio Code VS Code offers a wide range of Python extensions (plugins) that simplify the development process. Here are some of the most popular ones: Pylance Pylance provides code analysis, autocompletion, and IntelliSense support, making Python development more efficient and user-friendly. Key features include fast autocompletion, type checking, and IntelliSense support. Jupyter The Jupyter extension is a powerful tool for working with interactive notebooks directly in the editor.  It’s especially useful for data analysis, machine learning, visualization, and interactive programming tasks. autoDocstring — Python Docstring Generator autoDocstring is a popular extension that helps automatically generate docstrings for Python functions, methods, and classes. Docstrings improve code readability and serve as built-in documentation. isort isort is a tool for automatically sorting and organizing imports in Python code. You can configure it in Visual Studio Code to make working with imports easier and to improve code readability. Conclusion This article covered the installation and setup of Visual Studio Code for Python development. Visual Studio Code offers full support for Python and provides the ability to extend its functionality through various plugins, making the coding process easier and more efficient.
05 February 2025 · 10 min to read
Python

How to Merge Lists in Python

Python offers numerous data types for storing and manipulating information. Lists, tuples, sets, and dictionaries are among the most frequently used. List: An unordered collection of data that can contain duplicate elements. Tuple: An ordered collection where the order cannot be changed. Dictionaries are similar to sets but organized as key-value pairs, allowing for efficient value retrieval based on keys. Sets: Collections of unique, unordered elements. Lists, however, are simple ordered collections of elements, allowing for flexible additions and deletions as needed. They are particularly useful for dynamically tracking multiple elements. In this guide, we’ll explore how to merge lists in Python 3.11, providing examples to demonstrate their functionality. How to Run Examples from This Guide If you're new to Python, here’s how to run examples from this tutorial to practice list merging: Open any text editor and create a file, e.g., main.py. Copy the code from one of the examples into this file and save it. On Windows, open the Command Prompt; on Linux/macOS, open the terminal. Navigate to the directory where your file is located using the cd command, e.g.: cd C:\Users\ Execute the following command to run your script: python main.py Or: python3 main.py The result of the program execution will be displayed in the console. Method 1: The + Operator The + operator can be used to merge two lists in Python. It appends one list to the end of another, resulting in a new list. a1 = [1, 12, 5, 49, 56] a2 = [27, 36, 42] list= a1 + a2 print(list) Output: [1, 12, 5, 49, 56, 27, 36, 42] Let’s look at another example, where a Python program generates three lists with random numbers and combines them into a single list: import random def generate_and_combine_lists(length): if length <= 0: raise ValueError("List length must be a positive number") list1 = [random.randint(1, 10) for _ in range(length)] list2 = [random.randint(1, 100) for _ in range(length)] list3 = [random.randint(1, 1000) for _ in range(length)] try: combined_list = list1 + list2 + list3 return list1, list2, list3, combined_list except TypeError as e: print(f"Error combining lists: {e}") return None list_length = 5 list1, list2, list3, combined_list = generate_and_combine_lists(list_length) if combined_list: print(f"List 1: {list1}") print(f"List 2: {list2}") print(f"List 3: {list3}") print(f"Combined List: {combined_list}") Output: List 1: [4, 7, 3, 2, 10] List 2: [43, 73, 5, 61, 39] List 3: [500, 315, 935, 980, 224] Combined List: [4, 7, 3, 2, 10, 43, 73, 5, 61, 39, 500, 315, 935, 980, 224] Method 2: The * Operator The * operator can easily combine lists in Python by unpacking the elements of collections into indexed positions. If you have two lists, for example: list1 = [1, 12, 5, 49, 56]list2 = [27, 36, 42] Using the * operator replaces the list with its individual elements at the specified index positions, effectively "unpacking" the list contents. list1 = [1, 12, 5, 49, 56]list2 = [27, 36, 42]combined_list = [*list1, *list2]print(str(combined_list)) Output: [1, 12, 5, 49, 56, 27, 36, 42] Below is another example where randomly generated Python lists are combined using the * operator: import random def generate_and_combine_lists(length): if length <= 0: raise ValueError("List length must be a positive number") list1 = [random.randint(1, 100) for _ in range(length)] list2 = [random.randint(1, 100) for _ in range(length)] list3 = [random.randint(1, 100) for _ in range(length)] return list1, list2, list3, *list1, *list2, *list3 list_length = 5 list1, list2, list3, *combined_list = generate_and_combine_lists(list_length) print(f"List 1: {list1}") print(f"List 2: {list2}") print(f"List 3: {list3}") print(f"Combined List: {combined_list}") Output: List 1: [10, 43, 17, 74, 99] List 2: [65, 91, 56, 37, 37] List 3: [33, 39, 87, 27, 82] Combined List: [10, 43, 17, 74, 99, 65, 91, 56, 37, 37, 33, 39, 87, 27, 82] The * operator efficiently merges the contents of list1, list2, and list3 into a single combined_list. Method 3: Using a for Loop In this method, we use a for loop to iterate over the second list. Each element from the second list is added to the first list using the append() method. The result is a new list that combines the elements of both lists. list1 = [6, 11, 32, 71, 3] list2 = [18, 54, 42] print("Original List 1:", str(list1)) for x in list2: list1.append(x) print("Combined List:", str(list1)) Output: Original List 1: [6, 11, 32, 71, 3] Combined List: [6, 11, 32, 71, 3, 18, 54, 42] Method 4: List Comprehension We can also use list comprehensions in Python to combine lists efficiently. A list comprehension is a concise way to generate a new list based on an iterable. list1 = [5, 73, 232, 1, 8, 19] list2 = [84, 56, 7, 10, 20, 30] combined_list = [j for i in [list1, list2] for j in i] print("Combined List:", str(combined_list)) Output: [5, 73, 232, 1, 8, 19, 84, 56, 7, 10, 20, 30]   Method 5: Using the extend() Method The extend() method in Python iterates over the elements of the provided list and appends them to the current list, effectively merging both lists. import random list1 = [random.randint(10, 20) for _ in range(5)] list2 = [random.randint(20, 50) for _ in range(3)] print("First List:", str(list1)) list1.extend(list2) print("Combined List:", str(list1)) Output: First List: [19, 19, 16, 17, 16]Combined List: [19, 19, 16, 17, 16, 47, 21, 31] In this approach, all elements from list2 are added to list1, updating list1 directly with the combined contents. Method 6: Using itertools.chain() The itertools module in Python provides various functions for working with iterators, which can be used to efficiently generate lists. It is particularly useful for generating large lists created with complex rules, as it avoids creating the entire list in memory at once, which can lead to memory overflow for very large datasets. We can also use the itertools.chain() function from the itertools module to combine lists in Python. import itertools list_of_lists = [[1, 5], [3, 4], [7, 12]] chained_list = list(itertools.chain(*list_of_lists)) print(chained_list) Output: [1, 5, 3, 4, 7, 12] Let's consider a case where we generate letters and combine them into a list. import itertools import string def generate_letter_range(start, stop): for letter in string.ascii_lowercase[start:stop]: yield letter list1 = generate_letter_range(0, 3) list2 = generate_letter_range(7, 16) combined_list = list(itertools.chain(list1, list2)) print(combined_list) Output: ['a', 'b', 'c', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p'] We can also combine lists of numbers using itertools.chain(). import itertools list1 = [5, 73, 232, 1, 8] list2 = [19, 84, 56, 7] list3 = [10, 20, 30] combined_list = list(itertools.chain(list1, list2, list3)) print(combined_list) Output: [5, 73, 232, 1, 8, 19, 84, 56, 7, 10, 20, 30] Let's generate random letters in two lists, with one list containing 3 letters and the other containing 7, and then combine them. import itertools import random import string def generate_letter_list(num_letters): for i in range(num_letters): yield random.choice(string.ascii_letters) num_list1 = 3 num_list2 = 7 list1 = generate_letter_list(num_list1) list2 = generate_letter_list(num_list2) combined_list = list(itertools.chain(list1, list2)) print(combined_list) Output: ['d', 'e', 'O', 'M', 'q', 'i', 'N', 'V', 'd', 'C'] Conclusion Each of these methods for merging lists in Python has its own particularities, and the choice of which one to use depends on what you need to accomplish, the amount of data you have, and how quickly you want to get the result. Understanding these methods will help you to work more efficiently with data in your Python projects. Choose the method that suits your needs, and don't hesitate to try different approaches to get the best result!
05 February 2025 · 7 min to read

Do you have questions,
comments, or concerns?

Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
Email us
Hostman's Support