Sign In
Sign In

Working with the pathlib Module in Python 3

Working with the pathlib Module in Python 3
Hostman Team
Technical writer
Python
23.10.2024
Reading time: 8 min

Manipulating file system paths in Python 3 is handled by several libraries, one of which is pathlib. In this article, we'll show you how to use the pathlib module and solve practical tasks with it.

File System Basics

Before diving into the practical use of the pathlib library, we need to refresh some foundational concepts about the file system and its terminology.

Let's consider a file example.exe located in C:\Program Files (Windows file system). This file has four main characteristics from the perspective of the file system:

  • Path: This is the identifier of the file, determining its location in the file system based on a sequence of directory names. Paths can be divided into two types:

    • Absolute path: A full path that starts from the root directory, e.g., C:\Program Files\example.exe.

    • Relative path: A path relative to another directory, e.g., Program Files\example.exe.

  • Directory (C:\Program Files): This is an object in the file system used to organize files hierarchically. It's often referred to as a directory or folder. The term "folder" perfectly describes this object’s practical use — for convenient storage and quick retrieval when needed.

  • Extension (.exe): This indicates the file type. Extensions help users or programs determine the type of data stored in the file (video, text, music, etc.).

Path Instances

Let's import the pathlib module to manipulate file system paths:

from pathlib import *

The classes in this module can be divided into two types: Pure (or "abstract") and Concrete. The Pure classes are used for abstract computational work with paths, where no actual interaction with the OS file system occurs. The Concrete classes, on the other hand, allow for direct interaction with the file system (e.g., creating and deleting directories, reading files, etc.).

If you're unsure which class to use, the Path class is likely the right choice. It automatically adapts to the OS and allows interaction with the file system without restrictions. In this article, we'll cover all classes but focus specifically on Path.

Let's start small by creating a variable of type Path:

>>> PathExample = Path('Hostman', 'Cloud', 'Pathlib')
>>> PathExample
WindowsPath('Hostman/Cloud/Pathlib')

As we can see, the module automatically adapts to the operating system — in this case, Windows 10. The constructor Path(args) creates a new Path instance, accepting directories, files, or other paths as arguments.

The PathExample is a relative path. We need to add the root directory to make it an absolute path.

Using the Path.home() and Path.cwd() methods, we can get the current user's home directory and the current working directory:

>>> Path.home()
WindowsPath('C:/Users/Blog')
>>> Path.cwd()
WindowsPath('C:/Users/Blog/AppData/Local/Programs/Python/Python310')

Let's make PathExample an absolute path and explore other aspects of working with pathlib:

>>> PathExample = Path.home() / PathExample
>>> PathExample
WindowsPath('C:/Users/Blog/Hostman/Cloud/Pathlib')

Using the / operator, we can concatenate paths to create new ones.

Path Attributes

The Python pathlib module provides various methods and properties to retrieve information about file paths. For illustration, let's create a new variable AttributeExample and append a file to the path:

>>> AttributeExample = PathExample / 'file.txt'
>>> AttributeExample
WindowsPath('C:/Users/Blog/Hostman/Cloud/Pathlib/file.txt')

Disk

To retrieve the drive letter or name, use the .drive property:

>>> AttributeExample.drive
'C:'

Parent Directories

You can get parent directories using two properties: .parent and .parents[n].

The .parent property returns the immediate parent directory:

>>> AttributeExample.parent
WindowsPath('C:/Users/Blog/Hostman/Cloud/Pathlib')

To get higher-level parent directories, you can either call .parent multiple times:

>>> AttributeExample.parent.parent
WindowsPath('C:/Users/Blog/Hostman/Cloud')

Or use .parents[n] to retrieve the nth ancestor:

>>> AttributeExample.parents[3]
WindowsPath('C:/Users/Blog')

Name

To get the file name, use the .name property:

>>> AttributeExample.name
'file.txt'

Extension

To get the file extension, use the .suffix property (or .suffixes for multiple extensions, such as .tar.gz):

>>> AttributeExample.suffix
'.txt'
>>> Path('file.tar.gz').suffixes
['.tar', '.gz']

Absolute or Relative Path

To check if the path is absolute, use the .is_absolute() method:

>>> AttributeExample.is_absolute()
True

Path Components

To split the path into its individual components, use the .parts property:

>>> AttributeExample.parts
('C:\\', 'Users', 'Blog', 'Hostman', 'Cloud', 'Pathlib', 'file.txt')

Paths Comparison

You can compare paths using both comparison operators and various methods.

Comparison Operators

You can check if two paths are the same:

>>> Path('hostman') == Path('HOSTMAN')
True

Note: On UNIX-based systems, case sensitivity matters, so the result would be False:

>>> PurePosixPath('hostman') == PurePosixPath('HOSTMAN')
False

This is because Windows file systems are case-insensitive, unlike UNIX-based systems.

Comparison Methods

You can check if one path is part of another using .is_relative_to():

>>> CompareExample = AttributeExample
>>> CompareExample.is_relative_to('C:')
True

>>> CompareExample.is_relative_to('D:/')
False

You can also use patterns for matching with the .match() method:

>>> CompareExample.match('*.txt')
True

Creating and Deleting Folders and Files

To create directories using the pathlib module, you can use the .mkdir(parents=True/False, exist_ok=True/False) method. This method takes two boolean parameters in addition to the path where the folder is to be created:

  • parents: If True, it creates any necessary parent directories. If False, it raises an error if they don't exist.

  • exist_ok: Determines whether an error should be raised if the directory already exists.

Let's create a folder CreateExample, but first, check if such a directory already exists using the .is_dir() method:

>>> CreateExample = CompareExample
>>> CreateExample.is_dir()
False

Now, let's create the folder:

>>> CreateExample.mkdir(parents=True, exist_ok=True)
>>> CreateExample.is_dir()
True

If you check the result in Windows Explorer, you will see that a directory (not a file) was created. To create an empty file, use the .touch() method. But first, let's remove the file.txt directory using the .rmdir() method:

>>> CreateExample.rmdir()
>>> CreateExample.touch()
>>> CreateExample.is_file()
True

To delete files, use the .unlink() method.

Searching for Files

Let's create a more complex directory structure based on the existing folder:

>>> SearchingExample = CreateExample

>>> Hosting = Path(SearchingExample.parents[2], 'hosting/host.txt')
>>> Hosting.parent.mkdir(parents=True, exist_ok=True)
>>> Hosting.touch()

>>> Docker = Path(SearchingExample.parents[1], 'Docker/desk.txt')
>>> Docker.parent.mkdir(parents=True, exist_ok=True)
>>> Docker.touch()

We now have the following structure (starting from C:\Users\Blog\Hostman):

Cloud
|— Pathlib
|    `— file.txt
`— Docker
        `— desk.txt
Hosting
`— host.txt

To search for files using pathlib, you can use a for loop along with the .glob() method:

>>> for file_cloud in SearchingExample.parents[2].glob('*.txt'):
   print(file_cloud)

This code doesn't find anything because it doesn't search in subfolders. To search recursively in subdirectories, modify it as follows:

>>> for file_cloud in SearchingExample.parents[2].glob('**/*.txt'):
    print(file_cloud)
...
C:\Users\Blog\Hostman\Cloud\Docker\desk.txt
C:\Users\Blog\Hostman\Cloud\Pathlib\file.txt
C:\Users\Blog\Hostman\hosting\host.txt

Reading and Writing to Files

You can perform both reading and writing operations in text or binary mode. We'll focus on text mode. The pathlib module provides four methods:

  • Reading: .read_text() and .read_bytes()

  • Writing: .write_text() and .write_bytes()

Let's write some important information to a file, for example, "Hostman offers really cool cloud servers!". This is definitely something worth saving:

>>> WRExample = SearchingExample

>>> WRExample.is_file()
True
>>> WRExample.write_text('Hostman offers really cool cloud servers!')
55  # Length of the message
>>> WRExample.read_text()
'Hostman offers really cool cloud servers!'

Conclusion

In conclusion, the pathlib module is an essential tool for efficient file system management in Python. It allows developers to easily create, manipulate, and query file paths, regardless of the operating system. pathlib simplifies tasks such as file creation, deletion, and searching by supporting both absolute and relative paths and providing detailed metadata access. Its flexibility and ease of use make it a preferred choice for modern Python development, allowing for cleaner, more maintainable code. Embracing pathlib can significantly enhance productivity in any file-related project.

Check out our app platform to find Python applications, such as Celery, Django, FastAPI and Flask. 

Python
23.10.2024
Reading time: 8 min

Similar

Python

How to Use the numpy.where() Method in Python

The numpy.where() method in Python is one of the most powerful and frequently used tools in the NumPy library for the conditional selection of elements from arrays. It provides flexible options for processing and analyzing large datasets, replacing traditional if-else conditional operators and significantly speeding up code execution. This method allows you to replace elements in an array that meet a certain condition with specified values while leaving other elements unchanged. Unlike regular loops, which can slow down execution when working with large datasets, numpy.where() uses vectorization, making operations faster and more efficient. Syntax of the where() Method The numpy.where() method has the following syntax: numpy.where(condition[, x, y]) Where: condition: the condition or array of conditions to be checked. x: values returned if the condition is True. y: values returned if the condition is False. If the arguments x and y are not specified, the method will return the indices of the elements that satisfy the condition. Main Usage Approaches Let's move on to practical examples. Finding Element Indices It is often necessary to determine the positions of elements that satisfy a certain condition. numpy.where() makes this easy to achieve: import numpy as np arr = np.array([1, 2, 3, 4, 5]) indices = np.where(arr > 3) print(indices) In this example, we create an array [1, 2, 3, 4, 5]. Then, we use the np.where() function to find the indices of elements greater than 3. Running the code yields (array([3, 4]),), indicating the positions of the numbers 4 and 5 in the original array, as only these numbers satisfy the condition arr > 3. In this case, the method returns a tuple containing an array of indices for elements greater than 3. Conditional Element Replacement The numpy.where() method is widely used for conditionally replacing elements in an array: import numpy as np arr = np.array([1, 2, 3, 4, 5]) result = np.where(arr > 3, 100, arr) print(result) This code starts by creating an array [1, 2, 3, 4, 5]. The np.where() function is then used to find elements greater than 3. The additional parameter 100 allows these elements to be replaced with the specified value. The resulting output is [1, 2, 3, 100, 100], where the elements 4 and 5 have been replaced with 100 because they satisfy the condition arr > 3. In this case, np.where() replaces all elements meeting the condition with the specified value. Working with Multidimensional Arrays The numpy.where() method also works effectively with multidimensional arrays: import numpy as np matrix = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) result = np.where(matrix % 2 == 0, 'even', 'odd') print(result) This example creates a matrix [[1, 2, 3], [4, 5, 6], [7, 8, 9]]. The np.where() function is applied to replace elements based on the condition: if the number is even (divisible by 2 without a remainder), it is replaced with the string 'even'; otherwise, it is replaced with 'odd'. The resulting matrix is printed as: [['odd' 'even' 'odd'] ['even' 'odd' 'even'] ['odd' 'even' 'odd']] In this example, the method returns an updated matrix with strings instead of numbers. Applying Multiple Conditions By using logical operators, numpy.where() can handle more complex conditions: import numpy as np arr = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9]) result = np.where((arr > 3) & (arr < 7), arr * 2, arr) print(result) In this example, an array [1, 2, 3, 4, 5, 6, 7, 8, 9] is created. The np.where() function is used with a combined condition: if the number is greater than 3 and less than 7, it is multiplied by 2; otherwise, it remains unchanged. The output is: [1, 2, 3, 8, 10, 12, 7, 8, 9] The numbers 4, 5, and 6 are multiplied by 2 as they meet the condition. In this case, the method returns a new array with updated values based on the condition. Practical Examples Working with Temperature Data Suppose we have an array of temperatures in Celsius, and we want to classify them as hot" or "comfortable": import numpy as np temperatures = np.array([23, 25, 28, 32, 35, 29]) status = np.where(temperatures > 30, 'hot', 'comfortable') print(status) In this example, the temperature array [23, 25, 28, 32, 35, 29] is created. The np.where() function is applied to determine comfort levels: if the temperature exceeds 30 degrees, it is labeled as 'hot'; otherwise, it is 'comfortable'.  The output is:  ['comfortable' 'comfortable' 'comfortable' 'hot' 'hot' 'comfortable']  Temperatures 32 and 35 degrees are marked as 'hot' because they exceed the threshold.  This method returns a new array with string values reflecting the temperature evaluation. Handling Missing Values In datasets, missing values often need to be replaced or handled: import numpy as np data = np.array([1, np.nan, 3, np.nan, 5]) cleaned_data = np.where(np.isnan(data), 0, data) print(cleaned_data) Here, we create an array with missing values [1, np.nan, 3, np.nan, 5]. The np.where() function is combined with np.isnan() to replace missing values (NaN) with 0.  The result is: [1. 0. 3. 0. 5.] The NaN values are replaced with 0, while other elements remain unchanged.  This example demonstrates how to clean data by handling missing values. Method Comparison Table Characteristic numpy.where() Loops List Comprehension Speed High Low Medium Memory Usage Medium High Medium Readability High Medium High Vectorization Yes No Partially Flexibility High High High As the table shows, numpy.where() outperforms traditional loops and list comprehensions in terms of speed and memory efficiency, while maintaining high readability and flexibility. Conclusion The numpy.where() method is an indispensable tool for efficient data processing and analysis in Python. Its use allows developers to write more performant, clean, and readable code, especially when working with large datasets and complex conditions. This method simplifies tasks related to replacing array elements based on specified conditions and eliminates the need for bulky loops and checks, making the code more compact and faster. numpy.where() is particularly useful for handling large datasets where high performance and simple conditional operations are crucial. Loops remain a better choice for complex data processing logic or step-by-step operations, especially when working with smaller datasets. On the other hand, list comprehensions are suitable for compact and readable code when dealing with small to medium datasets, provided the operations are not overly complex. Understanding the syntax and capabilities of numpy.where() opens up new approaches for solving various problems in areas such as data analysis, image processing, and financial analysis. The method enables efficient handling of large data volumes and significantly accelerates operations through vectorization, which is particularly important for tasks requiring high performance. Using techniques like vectorization and masks in combination with NumPy functions helps developers optimize code and achieve fast and accurate results. Regardless of your level of experience in Python programming, mastering numpy.where() and understanding its advantages will be a crucial step toward more efficient data handling, improving program performance, and implementing optimal solutions in analytics and information processing.
06 February 2025 · 6 min to read
Python

How to Set Up Visual Studio Code for Python

Creating and debugging programs in Python is easier when using a specialized Integrated Development Environment (IDE). With an IDE, you can quickly and efficiently develop, test, and debug programs. Visual Studio Code (VS Code) for Python provides full support for the language and offers a wide range of plugins and extensions. In this article, we will install Visual Studio Code on three operating systems (Windows, macOS, Linux) and set it up for Python programming, including the use of popular plugins. Prerequisites To install and set up Visual Studio Code for Python, we will need the following: A personal or work computer with Windows 10/11, macOS, or Ubuntu Linux distribution version 24.04 pre-installed. Alternatively, you can rent a dedicated server or a virtual machine with Windows Server 2016/2019/2022. If using regular versions of Windows, you can download your own ISO image in advance. You can also rent a server with Ubuntu. Installing the Python Interpreter Before installing VS Code, we need to install the Python interpreter on all three operating systems — Windows, macOS, and Linux. On Windows Go to the official Python website and download the installer file. In this case, we will be installing Python version 3.13.1. Run the installer file. You will have two installation options: Install Now — This performs a full installation, including documentation files, the package manager pip, the tcl/tk library for graphical interface support, and standard libraries. Customize Installation — This option allows you to choose which components to install. We will use the full installation. Make sure to check the box next to the option Add python.exe to PATH and click Install Now. The installation process will begin. Once the installation is complete, the program will notify you that it has finished. On macOS On macOS, the Python interpreter is pre-installed by default. You can verify this by running the following command in the terminal: python3 --version However, the installed version may be outdated. If necessary, you can install a newer version. To do this, we will use the Homebrew package manager. First, if Homebrew is not installed on your system, you can install it by running the following command: /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" Next, you need to check which versions of Python are available for installation. Use the following command: brew search python In our case, several versions of Python are available: Install the latest available version, Python 3.13, by running the following command: brew install [email protected] Check the Python version again: python3 --versionpython3.13 --version As shown in the screenshot above, running the python3 --version still shows the old version (Python 3.9.6). However, the newly installed version (3.13) can be accessed using the command python3.13 --version. If needed, you can change the default Python version to the newly installed one. To do this, first get the full path to the newly installed Python interpreter using the following command: brew --prefix [email protected] Then, check which shell you are using: echo $SHELL Depending on the shell used, open the corresponding file for editing: For bash or sh: nano ~/.bashrc For zsh: nano ~/.zshrc Add the following line at the end of the file: export PATH="/opt/homebrew/opt/[email protected]/bin:$PATH" Save the changes and reload the file: source ~/.zshrc Now, when you check the Python version, it will display the latest installed version: python3 --version On Ubuntu By default, Python is pre-installed on almost all Linux distributions, including Ubuntu. In the latest supported versions of Ubuntu, the current version of Python is installed: python3 --version However, if Python is not installed for any reason, you can install it by running the following command: apt -y install python3 Installing Visual Studio Code You can install Visual Studio Code on your personal computer. You can also rent a dedicated or cloud server with Windows Server or one of the available Linux distributions. If the required distribution is not available in the list of offered images, you can upload your own. We will cover the installation of Visual Studio Code on three operating systems: Windows, macOS, and Linux (Ubuntu 24.04 distribution). For Windows Visual Studio Code supports installation on Windows 10 and Windows 11. It also supports Windows Server distributions, from version 2016 to 2022. We will install it on Windows 10.  Go to the official website and download the installer. This will download an .exe installation file. Run the installer file.  On the first step, accept the license agreement by selecting the option "I accept the agreement". Next, the installer will prompt you to choose an installation location. You can choose the default path suggested by the installer or specify your own. If necessary, you can create a shortcut for the program in the Windows menu. If you don’t want to create a shortcut, select the option "Don’t create a Start Menu folder" at the bottom: The next step lets you configure additional options by ticking the corresponding checkboxes: Create a desktop icon — creates a shortcut on the desktop for quick access to the program. Add “Open with Code” action to Windows Explorer file context menu — adds the "Open with Code" option to the context menu when right-clicking on a file. This option allows you to quickly open any file directly in Visual Studio Code. Add “Open with Code” action to Windows Explorer directory file context menu — similar to the above option but adds the "Open with Code" option to the context menu of directories (folders). Register Code as an editor for supported file types — makes Visual Studio Code the default editor for certain file types (e.g., .c, .cpp, .py, .java, .js, .html files). Add to PATH (require shell restart) — adds Visual Studio Code to the system’s PATH variable so it can be launched from the command line (cmd). Once all necessary options are set, Visual Studio Code is ready for installation. Click Install. After the installation is complete, you can launch the program immediately: For macOS Go to the official website and download the installer: After downloading, you will have a ZIP archive. Inside the archive, you will find the executable file, which you need to extract to the "Applications" directory. On the first launch, the system will notify you that this file was downloaded from the internet and may not have vulnerabilities. Click Open to continue: For Linux (Ubuntu) Visual Studio Code supports installation on Linux distributions such as Ubuntu, Debian, Red Hat, Fedora, and SUSE. You need a graphical desktop environment to install Visual Studio Code on Linux (GNOME, KDE, Xfce, etc.). Let’s consider the installation of Visual Studio Code on Ubuntu 24.04 with the Xfce desktop environment. You can also install Visual Studio Code using Snapcraft. Go to the official website and download the installer for your Linux distribution. In our case, we need the .deb installer: Once the file is downloaded, open a terminal (console) and navigate to the directory where the file was downloaded (e.g., /root). To install, run the following command where code_1.96.2-1734607745_amd64.deb is the name of the downloaded file: dpkg -i code_1.96.2-1734607745_amd64.deb During installation, a message will prompt you to add Microsoft repositories to the system. Select <Yes> and press Enter: Wait for the installation to complete. Once the installation is finished, you can launch the program from the applications menu (for distributions using Xfce, Visual Studio Code is available in the menu: Applications → Development): Adding Python Interpreter to PATH Variable in Windows If you haven't checked the Add python.exe to PATH checkbox during the Python installation on Windows, you need to manually add the full path to the interpreter to run Python from the command line. To do this: Press Win+R, type sysdm.cpl in the Run window, and press Enter. In the window that opens, go to the Advanced tab and click on the Environment Variables button. To add a user-level variable, select the Path variable under User variables and click on Edit. Double-click on an empty field or click the New button. Enter the full path to the Python interpreter file. By default, the Python interpreter is located at the following path: C:\Users\<Username>\AppData\Local\Programs\Python\Python313 For example: C:\Users\Administrator\AppData\Local\Programs\Python\Python313 After entering the path, click OK to save the changes. To verify, open the command prompt and type python. If the path to the interpreter is correctly specified, the Python console will open. Setting Up Python Interpreter in Visual Studio Code In Windows Once Python is installed, you need to connect it to Visual Studio Code. To do this: Open Visual Studio Code and click the New File... button on the home page to create a new file.  Alternatively, you can create a Python project in Visual Studio Code by clicking on Open Folder…, where you can select the entire project folder containing the files. Type any name for the file, use the .py extension, and press Enter. Save the file in any location. Ensure that the file name ends with the .py extension. Once the file is saved, the interface in Visual Studio Code will display a prompt at the bottom right, suggesting you install the recommended Python extension. To run Python in Visual Studio Code, you first need to select the Python interpreter. A button will appear at the bottom of the panel with a warning: Select Interpreter. Click on it. In the menu that appears, select Enter interpreter path… and press Enter. Specify the full path to the Python interpreter. By default, it is located at: C:\Users\Administrator\AppData\Local\Programs\Python\Python313 where Administrator is your user account name. Select the file named python and click on Select Interpreter. To test it, write a simple program that calculates the square root of a number: import math num = float(input("Enter a number to find its square root: ")) if num >= 0: sqrt_result = math.sqrt(num) print(f"The square root of {num} is {sqrt_result}") else: print("Square root of a negative number is not real.") In Visual Studio Code, to run the Python code, click the Run Python File button at the top-right. If the interpreter is set up correctly, the program will run successfully. In macOS On macOS operating systems, the Python interpreter is automatically recognized. Simply create a new .py file as described in the Windows section above and run the program directly. In Ubuntu Similarly to macOS, in the Ubuntu distribution, VS Code automatically detects the installed Python interpreter in the system. All you need to do is create a new .py file and run the program directly. Recommended Extensions for Python in Visual Studio Code VS Code offers a wide range of Python extensions (plugins) that simplify the development process. Here are some of the most popular ones: Pylance Pylance provides code analysis, autocompletion, and IntelliSense support, making Python development more efficient and user-friendly. Key features include fast autocompletion, type checking, and IntelliSense support. Jupyter The Jupyter extension is a powerful tool for working with interactive notebooks directly in the editor.  It’s especially useful for data analysis, machine learning, visualization, and interactive programming tasks. autoDocstring — Python Docstring Generator autoDocstring is a popular extension that helps automatically generate docstrings for Python functions, methods, and classes. Docstrings improve code readability and serve as built-in documentation. isort isort is a tool for automatically sorting and organizing imports in Python code. You can configure it in Visual Studio Code to make working with imports easier and to improve code readability. Conclusion This article covered the installation and setup of Visual Studio Code for Python development. Visual Studio Code offers full support for Python and provides the ability to extend its functionality through various plugins, making the coding process easier and more efficient.
05 February 2025 · 10 min to read
Python

How to Merge Lists in Python

Python offers numerous data types for storing and manipulating information. Lists, tuples, sets, and dictionaries are among the most frequently used. List: An unordered collection of data that can contain duplicate elements. Tuple: An ordered collection where the order cannot be changed. Dictionaries are similar to sets but organized as key-value pairs, allowing for efficient value retrieval based on keys. Sets: Collections of unique, unordered elements. Lists, however, are simple ordered collections of elements, allowing for flexible additions and deletions as needed. They are particularly useful for dynamically tracking multiple elements. In this guide, we’ll explore how to merge lists in Python 3.11, providing examples to demonstrate their functionality. How to Run Examples from This Guide If you're new to Python, here’s how to run examples from this tutorial to practice list merging: Open any text editor and create a file, e.g., main.py. Copy the code from one of the examples into this file and save it. On Windows, open the Command Prompt; on Linux/macOS, open the terminal. Navigate to the directory where your file is located using the cd command, e.g.: cd C:\Users\ Execute the following command to run your script: python main.py Or: python3 main.py The result of the program execution will be displayed in the console. Method 1: The + Operator The + operator can be used to merge two lists in Python. It appends one list to the end of another, resulting in a new list. a1 = [1, 12, 5, 49, 56] a2 = [27, 36, 42] list= a1 + a2 print(list) Output: [1, 12, 5, 49, 56, 27, 36, 42] Let’s look at another example, where a Python program generates three lists with random numbers and combines them into a single list: import random def generate_and_combine_lists(length): if length <= 0: raise ValueError("List length must be a positive number") list1 = [random.randint(1, 10) for _ in range(length)] list2 = [random.randint(1, 100) for _ in range(length)] list3 = [random.randint(1, 1000) for _ in range(length)] try: combined_list = list1 + list2 + list3 return list1, list2, list3, combined_list except TypeError as e: print(f"Error combining lists: {e}") return None list_length = 5 list1, list2, list3, combined_list = generate_and_combine_lists(list_length) if combined_list: print(f"List 1: {list1}") print(f"List 2: {list2}") print(f"List 3: {list3}") print(f"Combined List: {combined_list}") Output: List 1: [4, 7, 3, 2, 10] List 2: [43, 73, 5, 61, 39] List 3: [500, 315, 935, 980, 224] Combined List: [4, 7, 3, 2, 10, 43, 73, 5, 61, 39, 500, 315, 935, 980, 224] Method 2: The * Operator The * operator can easily combine lists in Python by unpacking the elements of collections into indexed positions. If you have two lists, for example: list1 = [1, 12, 5, 49, 56]list2 = [27, 36, 42] Using the * operator replaces the list with its individual elements at the specified index positions, effectively "unpacking" the list contents. list1 = [1, 12, 5, 49, 56]list2 = [27, 36, 42]combined_list = [*list1, *list2]print(str(combined_list)) Output: [1, 12, 5, 49, 56, 27, 36, 42] Below is another example where randomly generated Python lists are combined using the * operator: import random def generate_and_combine_lists(length): if length <= 0: raise ValueError("List length must be a positive number") list1 = [random.randint(1, 100) for _ in range(length)] list2 = [random.randint(1, 100) for _ in range(length)] list3 = [random.randint(1, 100) for _ in range(length)] return list1, list2, list3, *list1, *list2, *list3 list_length = 5 list1, list2, list3, *combined_list = generate_and_combine_lists(list_length) print(f"List 1: {list1}") print(f"List 2: {list2}") print(f"List 3: {list3}") print(f"Combined List: {combined_list}") Output: List 1: [10, 43, 17, 74, 99] List 2: [65, 91, 56, 37, 37] List 3: [33, 39, 87, 27, 82] Combined List: [10, 43, 17, 74, 99, 65, 91, 56, 37, 37, 33, 39, 87, 27, 82] The * operator efficiently merges the contents of list1, list2, and list3 into a single combined_list. Method 3: Using a for Loop In this method, we use a for loop to iterate over the second list. Each element from the second list is added to the first list using the append() method. The result is a new list that combines the elements of both lists. list1 = [6, 11, 32, 71, 3] list2 = [18, 54, 42] print("Original List 1:", str(list1)) for x in list2: list1.append(x) print("Combined List:", str(list1)) Output: Original List 1: [6, 11, 32, 71, 3] Combined List: [6, 11, 32, 71, 3, 18, 54, 42] Method 4: List Comprehension We can also use list comprehensions in Python to combine lists efficiently. A list comprehension is a concise way to generate a new list based on an iterable. list1 = [5, 73, 232, 1, 8, 19] list2 = [84, 56, 7, 10, 20, 30] combined_list = [j for i in [list1, list2] for j in i] print("Combined List:", str(combined_list)) Output: [5, 73, 232, 1, 8, 19, 84, 56, 7, 10, 20, 30]   Method 5: Using the extend() Method The extend() method in Python iterates over the elements of the provided list and appends them to the current list, effectively merging both lists. import random list1 = [random.randint(10, 20) for _ in range(5)] list2 = [random.randint(20, 50) for _ in range(3)] print("First List:", str(list1)) list1.extend(list2) print("Combined List:", str(list1)) Output: First List: [19, 19, 16, 17, 16]Combined List: [19, 19, 16, 17, 16, 47, 21, 31] In this approach, all elements from list2 are added to list1, updating list1 directly with the combined contents. Method 6: Using itertools.chain() The itertools module in Python provides various functions for working with iterators, which can be used to efficiently generate lists. It is particularly useful for generating large lists created with complex rules, as it avoids creating the entire list in memory at once, which can lead to memory overflow for very large datasets. We can also use the itertools.chain() function from the itertools module to combine lists in Python. import itertools list_of_lists = [[1, 5], [3, 4], [7, 12]] chained_list = list(itertools.chain(*list_of_lists)) print(chained_list) Output: [1, 5, 3, 4, 7, 12] Let's consider a case where we generate letters and combine them into a list. import itertools import string def generate_letter_range(start, stop): for letter in string.ascii_lowercase[start:stop]: yield letter list1 = generate_letter_range(0, 3) list2 = generate_letter_range(7, 16) combined_list = list(itertools.chain(list1, list2)) print(combined_list) Output: ['a', 'b', 'c', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p'] We can also combine lists of numbers using itertools.chain(). import itertools list1 = [5, 73, 232, 1, 8] list2 = [19, 84, 56, 7] list3 = [10, 20, 30] combined_list = list(itertools.chain(list1, list2, list3)) print(combined_list) Output: [5, 73, 232, 1, 8, 19, 84, 56, 7, 10, 20, 30] Let's generate random letters in two lists, with one list containing 3 letters and the other containing 7, and then combine them. import itertools import random import string def generate_letter_list(num_letters): for i in range(num_letters): yield random.choice(string.ascii_letters) num_list1 = 3 num_list2 = 7 list1 = generate_letter_list(num_list1) list2 = generate_letter_list(num_list2) combined_list = list(itertools.chain(list1, list2)) print(combined_list) Output: ['d', 'e', 'O', 'M', 'q', 'i', 'N', 'V', 'd', 'C'] Conclusion Each of these methods for merging lists in Python has its own particularities, and the choice of which one to use depends on what you need to accomplish, the amount of data you have, and how quickly you want to get the result. Understanding these methods will help you to work more efficiently with data in your Python projects. Choose the method that suits your needs, and don't hesitate to try different approaches to get the best result!
05 February 2025 · 7 min to read

Do you have questions,
comments, or concerns?

Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
Email us
Hostman's Support