Sign In
Sign In

Introduction to Requests Library in Python

Introduction to Requests Library in Python
Hostman Team
Technical writer
Python
18.10.2024
Reading time: 8 min

Requests is a Python library that helps your local script interact with web resources and the global network. It provides developers with a wide range of functions for working with all types of HTTP requests. With this library, you can obtain weather forecasts, translate text, and download movies or images without a browser within your script.

This guide provides introductory information about the Python Requests library, which will be sufficient for using it in your scripts.

How to Install the requests Library in Python

For the purpose of this guide, we will be working in the PyCharm IDE. requests is an external library in Python, so it needs to be installed before use. First, create a project and open the terminal. Run the following command:

pip3 install requests

If you are using a virtual environment like Pipenv, you can install the Requests library in Python 3 with the following command:

pipenv install requests

After running these commands, the module will begin downloading. You can use the pip freeze command to check which modules have been installed:

pip3 freeze

Example output:

certifi==2022.9.24
charset-normalizer==2.1.1
idna==3.4
requests==2.28.1
urllib3==1.26.12

Here's a brief explanation of what these modules do:

  • certifi: A package of certificates for ensuring secure connections.
  • charset-normalizer: A library that automatically detects text encodings, useful since not all sites and services use UTF-8.
  • idna: A library that supports internationalized domain names.
  • requests: The actual Requests module.
  • urllib3: A module that provides functions and classes for working with URLs.

First Requests

In this section, we will write code to retrieve information from web resources and learn the basics of the Requests library. We will explore all aspects in more detail later.

To start working with the Requests library, we need to import it:

import requests

Let's request google.com:

import requests as rq
response = rq.get('https://google.com')
print(response)

Output:

<Response [200]>

Here, we use an HTTP GET request, which is similar to visiting a website via a URL in a browser. In return, we get a Response object with a status code 200, meaning the request was successful. If we try to access a non-existent section of the site, we will receive a different response:

import requests as rq
response = rq.get('https://google.com/hostman')
print(response)

Output:

<Response [404]>

Now, let's discuss the types of HTTP requests and how to work with them.

HTTP Requests

The primary HTTP request type is GET, which allows you to view the content of a resource without modifying it. However, other requests may be needed for full interaction with web resources. Not all servers support all requests. Here are the seven types of requests supported by the Requests library:

  • GET
  • POST
  • OPTIONS
  • HEAD
  • PUT
  • PATCH
  • DELETE

To test the Requests library, the creators have designed the site httpbin.org, which you can use to practice.

GET

A GET request sends information to the site via the URL. It is suitable for cases where the transmitted data is not sensitive, such as searching for a product online. You shouldn't use it to send passwords or other confidential information. Data is appended to the URL after a ? symbol, followed by key-value pairs, like so:

https://serverurl.com/get?param1=value1&param2=value2

Where:

  • https://serverurl.com/get is the URL.
  • param1=value1&param2=value2 are the parameters, separated by &.

In Requests, the GET method syntax is:

requests.get('URL', {key: value}, additional_arguments)

Where:

  • URL: The URL of the resource.
  • {key: value}: Optional parameters for the request.
  • additional_arguments: Optional arguments like timeout.

Example:

import requests as rq
GetParams = {'param1': 'value1', 'param2': 'value2'}
response = rq.get('https://google.com', GetParams)
print(response.url)

Output:

https://www.google.com/?param1=value1&param2=value2

POST

The POST request is used to send data to a website in the request body. This data does not appear in the URL and is ideal for sending confidential information.

In Requests, the POST method syntax is:

requests.post('URL', {key: value}, additional_arguments)

Where:

  • URL: The URL of the resource.
  • {key: value}: Optional parameters for the request body.

Example:

import requests as rq
PostParams = {'param1': 'value1', 'param2': 'value2'}
response = rq.post('https://httpbin.org/post', PostParams, timeout=2)
print(response.json()['form'])

Output:

{'param1': 'value1', 'param2': 'value2'}

Here, we used the json() method to retrieve the request body. If you make a similar request to google.com, you will get an error:

import requests as rq
PostParams = {'param1': 'value1', 'param2': 'value2'}
response = rq.post('https://google.com', PostParams, timeout=2)
print(response)

Output:

<Response [405]>

Error 405 (Method Not Allowed) means that the resource does not support this type of request. To check which requests are supported by a resource, you can use the OPTIONS method.

OPTIONS

The OPTIONS method helps determine which requests the resource will not block.

Syntax for OPTIONS:

requests.options('URL', optional_arguments)

Let's send an OPTIONS request to google.com to find out which methods it supports:

import requests as rq
response = rq.options('https://google.com', timeout=2)
print(response.headers['Allow'])

Output:

GET, HEAD

HEAD

A HEAD request returns only the HTTP headers from the server. This is useful when you need metadata about a file or resource without downloading its content. It can also be used for testing purposes.

Syntax for HEAD:

requests.head('URL', optional_arguments)
  • URL: The resource's address.

  • optional_arguments: Additional arguments, such as a timeout.

Example with google.com:

import requests as rq
response = rq.head('https://google.com', timeout=2)
print(response.headers)

This will return a list of headers from the server.

PUT

PUT is used to create a new object or replace an existing one. It is similar to POST but is idempotent, meaning repeated calls with the same data will not change the result.

To clarify, imagine a database of usernames and passwords. If a user wants to change their password:

  • POST might add a new record (if no validation occurs).
  • PUT would update the existing record.

Syntax for PUT:

requests.put('URL', {key: value}, optional_arguments)
  • URL: The resource's address.

  • {key: value}: Parameters, which are automatically included in the request body.

  • optional_arguments: Additional parameters like timeout.

Example with httpbin.org:

import requests as rq
PutParams = {'param1': 'value2', 'param2': 'value2'}
response = rq.put('https://httpbin.org/put', data=PutParams, timeout=2)
print(response.status_code)

Output:

200

PATCH

PATCH is used for partial updates to a resource, such as changing a token or specific piece of data.

Syntax for PATCH:

requests.patch('URL', {key: value}, optional_arguments)

Example with httpbin.org:

import requests as rq
PatchParams = {'param1': 'value2', 'param2': 'value2'}
response = rq.patch('https://httpbin.org/patch', data=PatchParams, timeout=2)
print(response.status_code)

Output:

200

DELETE

The DELETE method is used to remove an object from the resource.

Syntax for DELETE:

requests.delete('URL', {key: value})
  • URL: The resource's address.
  • {key: value}: Parameters representing the data to be deleted.

Example with httpbin.org:

import requests as rq
DelParams = {'param1': 'value2', 'param2': 'value2'}
response = rq.delete('https://httpbin.org/delete', data=DelParams, timeout=2)
print(response.status_code)

Output:

200

Response Object

As shown in the examples, the Response object in the Python Requests library has many methods and properties. Below is a list of them with brief descriptions:

  • apparent_encoding — The encoding detected by charset-normalizer.

  • close() — Closes the connection to the server.

  • content — Returns the received data in bytes.

  • cookies — Returns cookies.

  • elapsed — Returns the elapsed time of the request.

  • encoding — Specifies the encoding for decoding the response content.

  • headers — Returns the headers of the resource.

  • history — Returns the redirection history.

  • is_permanent_redirect — Checks for permanent redirects.

  • is_redirect — Checks if a redirect occurred.

  • iter_content() — Returns the content in chunks.

  • iter_lines() — Returns the content line by line.

  • json() — Returns the response in JSON format.

  • links — Returns the links in the response headers.

  • next — Returns the next PreparedRequest.

  • ok — Returns True if the request was successful, False if not.

  • raise_for_status() — Raises an HTTPError exception if the request failed.

  • reason — Returns a textual explanation of the response.

  • request — Returns the PreparedRequest object.

  • status_code — Returns the HTTP status code.

  • text — Returns the response in Unicode.

  • url — Returns the URL of the resource.

Conclusion

In this material, we explored the basic elements of the Requests library. If you want to learn more, the official documentation of the Python Requests library provides all the necessary details. And to test your scripts, you can use Hostman VPS.

Python
18.10.2024
Reading time: 8 min

Similar

Python

How to Delete Characters from a String in Python

When writing Python code, developers often need to modify string data. Common string modifications include: Removing specific characters from a sequence Replacing characters with others Changing letter case Joining substrings into a single sequence In this guide, we will focus on the first transformation—deleting characters from a string in Python. It’s important to note that strings in Python are immutable, meaning that any method or function that modifies a string will return a new string object with the changes applied. Methods for Deleting Characters from a String This section covers the main methods in Python used for deleting characters from a string. We will explore the following methods: replace() translate() re.sub() For each method, we will explain the syntax and provide practical examples. replace() The first Pyhton method we will discuss is replace(). It is used to replace specific characters in a string with others. Since strings are immutable, replace() returns a new string object with the modifications applied. Syntax: original_string.replace(old, new[, count]) Where: original_string – The string where modifications will take place old – The substring to be replaced new – The substring that will replace old count (optional) – The number of occurrences to replace (if omitted, all occurrences will be replaced) First, let’s remove all spaces from the string "H o s t m a n": example_str = "H o s t m a n" result_str = example_str.replace(" ", "") print(result_str) Output: Hostman We can also use the replace() method to remove newline characters (\n). example_str = "\nHostman\nVPS" print(f'Original string: {example_str}') result_str = example_str.replace("\n", " ") print(f'String after adjustments: {result_str}') Output: Original string: Hostman VPS String after adjustments: Hostman VPS The replace() method has an optional third argument, which specifies the number of replacements to perform. example_str = "Hostman VPS Hostman VPS Hostman VPS" print(f'Original string: {example_str}') result_str = example_str.replace("Hostman VPS", "", 2) print(f'String after adjustments: {result_str}') Output: Original string: Hostman VPS Hostman VPS Hostman VPS String after adjustments: Hostman VPS Here, only two occurrences of "Hostman VPS" were removed, while the third occurrence remained unchanged. We have now explored the replace() method and demonstrated its usage in different situations. Next, let’s see how we can delete and modify characters in a string using translate(). translate( The Python translate() method functions similarly to replace() but with additional flexibility. Instead of replacing characters one at a time, it allows mapping multiple characters using a dictionary or translation table. The method returns a new string object with the modifications applied. Syntax: original_string.translate(mapping_table) In the first example, let’s remove all occurrences of the $ symbol in a string and replace them with spaces: example_str = "Hostman$Cloud$—$Cloud$Service$Provider." print(f'Original string: {example_str}') result_str = example_str.translate({ord('$'): ' '}) print(f'String after adjustments: {result_str}') Output: Original string: Hostman$Cloud$—$Cloud$Service$Provider. String after adjustments: Hostman Cloud — Cloud Service Provider. To improve code readability, we can define the mapping table before calling translate(). This is useful when dealing with multiple replacements: example_str = "\nHostman%Cloud$—$Cloud$Service$Provider.\n" print(f'Original string: {example_str}') # Define translation table example_table = {ord('\n'): None, ord('$'): ' ', ord('%'): ' '} result_str = example_str.translate(example_table) print(f'String after adjustments: {result_str}') Output: Original string: Hostman%Cloud$—$Cloud$Service$Provider. String after adjustments: Hostman Cloud — Cloud Service Provider. re.sub() In addition to replace() and translate(), we can use regular expressions for more advanced character removal and replacement. Python's built-in re module provides the sub() method, which searches for a pattern in a string and replaces it. Syntax: re.sub(pattern, replacement, original_string [, count=0, flags=0]) pattern – The regular expression pattern to match replacement – The string or character that will replace the matched pattern original_string – The string where modifications will take place count (optional) – Limits the number of replacements (default is 0, meaning replace all occurrences) flags (optional) – Used to modify the behavior of the regex search Let's remove all whitespace characters (\s) using the sub() method from the re module: import re example_str = "H o s t m a n" print(f'Original string: {example_str}') result_str = re.sub('\s', '', example_str) print(f'String after adjustments: {result_str}') Output: Original string: H o s t m a nString after adjustments: Hostman Using Slices to Remove Characters In addition to using various methods to delete characters, Python also allows the use of slices. As we know, slices extract a sequence of characters from a string. To delete characters from a string by index in Python, we can use the following slice: example_str = "\nHostman \nVPS" print(f'Original string: {example_str}') result_str = example_str[1:9] + example_str[10:] print(f'String after adjustments: {result_str}') In this example, we used slices to remove newline characters. The output of the code: Original string:HostmanVPSString after adjustments: Hostman VPS Apart from using two slice parameters, you can also use a third one, which specifies the step size for index increments. For example, if we set the step to 2, it will remove every odd-indexed character in the string. Keep in mind that indexing starts at 0. Example: example_str = "Hostman Cloud" print(f'Original string: {example_str}') result_str = example_str[::2] print(f'String after adjustments: {result_str}') Output: Original string: Hostman CloudString after adjustments: HsmnCod Conclusion In this guide, we learned how to delete characters from a string in Python using different methods, including regular expressions and slices. The choice of method depends on the specific task. For example, the replace() method is suitable for simpler cases, while re.sub() is better for more complex situations.
23 August 2025 · 5 min to read
Python

Command-Line Option and Argument Parsing using argparse in Python

Command-line interfaces (CLIs) are one of the quickest and most effective means of interacting with software. They enable you to provide commands directly which leads to quicker execution and enhanced features. Developers often build CLIs using Python for several applications, utilities, and automation scripts, ensuring they can dynamically process user input. This is where the Python argparse module steps in. The argparse Python module streamlines the process of managing command-line inputs, enabling developers to create interactive and user-friendly utilities. As part of the standard library, it allows programmers to define, process, and validate inputs seamlessly without the need for complex logic. This article will discuss some of the most important concepts, useful examples, and advanced features of the argparse module so that you can start building solid command-line tools right away. How to Use Python argparse for Command-Line Interfaces This is how to use argparse in your Python script: Step 1: Import Module First import the module into your Python parser script: import argparse This inclusion enables parsing .py arg inputs from the command line. Step 2: Create an ArgumentParser Object The ArgumentParser class is the most minimal class of the Python argumentparser module's API. To use it, begin by creating an instance of the class: parser = argparse.ArgumentParser(description="A Hostman tutorial on Python argparse.") Here: description describes what the program does and will be displayed when someone runs --help. Step 3: Add Inputs and Options Define the parameters and features your program accepts via add_argument() function: parser.add_argument('filename', type=str, help="Name of the file to process") parser.add_argument('--verbose', action='store_true', help="Enable verbose mode") Here: filename is a mandatory option. --verbose is optional, to allow you to set the flag to make it verbose. Step 4: Parse User Inputs Process the user-provided inputs by invoking the parse_args() Python method: args = parser.parse_args() This stores the command-line values as attributes of the args object for further use in your Python script.  Step 5: Access Processed Data Access the inputs and options for further use in your program: For example: print(f"File to process: {args.filename}") if args.verbose:     print("Verbose mode enabled") else:     print("Verbose mode disabled") Example CLI Usage Here are some scenarios to run this script: File Processing Without Verbose Mode python3 file.py example.txt File Processing With Verbose Mode python3 file.py example.txt --verbose Display Help If you need to see what arguments the script accepts or their description, use the --help argument: python3 file.py --help Common Examples of argparse Usage Let's explore a few practical examples of the module. Example 1: Adding Default Values Sometimes, optional inputs in command-line interfaces need predefined values for smoother execution. With this module, you can set a default value that applies when someone doesn’t provide input. This script sets a default timeout of 30 seconds if you don’t specify the --timeout parameter. import argparse # Create the argument parser parser = argparse.ArgumentParser(description="Demonstrating default argument values.") # Pass an optional argument with a default value parser.add_argument('--timeout', type=int, default=30, help="Timeout in seconds (default: 30)") # Interpret the arguments args = parser.parse_args() # Retrieve and print the timeout value print(f"Timeout value: {args.timeout} seconds") Explanation Importing Module: Importing the argparse module. Creating the ArgumentParser Instance: An ArgumentParser object is created with a description so that a short description of the program purpose is provided. This description is displayed when the user runs the program via the --help option. Including --timeout: The --timeout option is not obligatory (indicated by the -- prefix). The type=int makes the argument for --timeout an integer. The default=30 is provided so that in case the user does not enter a value, then the timeout would be 30 seconds. The help parameter adds a description to the argument, and it will also appear in the help documentation. Parsing Process: The parse_args() function processes user inputs and makes them accessible as attributes of the args object. In our example, we access args.timeout and print out its value. Case 1: Default Value Used If the --timeout option is not specified, the default value of 30 seconds is used: python file.py Case 2: Custom Value Provided For a custom value for --timeout (e.g., 60 seconds), apply: python file.py --timeout 60 Example 2: Utilizing Choices The argparse choices parameter allows you to restrict an argument to a set of beforehand known valid values. This is useful if your program features some specific modes, options, or settings to check. Here, we will specify a --mode option with two default values: basic and advanced. import argparse # Creating argument parser parser = argparse.ArgumentParser(description="Demonstrating the use of choices in argparse.") # Adding the --mode argument with predefined choices parser.add_argument('--mode', choices=['basic', 'advanced'], help="Choose the mode of operation") # Parse the arguments args = parser.parse_args() # Access and display the selected mode if args.mode: print(f"Mode selected: {args.mode}") else: print("No mode selected. Please choose 'basic' or 'advanced'.") Adding --mode: The choices argument indicates that valid options for the --mode are basic and advanced. The application will fail when the user supplies an input other than in choices. Help Text: The help parameter gives valuable information when the --help command is executed. Case 1: Valid Input To specify a valid value for --mode, utilize: python3 file.py --mode basic Case 2: No Input Provided For running the program without specifying a mode: python3 file.py Case 3: Invalid Input If a value is provided that is not in the predefined choices: python3 file.py --mode intermediate Example 3: Handling Multiple Values The nargs option causes an argument to accept more than one input. This is useful whenever your program requires a list of values for processing, i.e., numbers, filenames, or options. Here we will show how to use nargs='+' to accept a --numbers option that can take multiple integers. import argparse # Create an ArgumentParser object parser = argparse.ArgumentParser(description="Demonstrating how to handle multiple values using argparse.") # Add the --numbers argument with nargs='+' parser.add_argument('--numbers', nargs='+', type=int, help="List of numbers to process") # Parse the arguments args = parser.parse_args() # Access and display the numbers if args.numbers: print(f"Numbers provided: {args.numbers}") print(f"Sum of numbers: {sum(args.numbers)}") else: print("No numbers provided. Please use --numbers followed by a list of integers.") Adding the --numbers Option: The user can provide a list of values as arguments for --numbers. type=int interprets the input as an integer. If a non-integer input is provided, the program raises an exception. The help parameter gives the information.  Parsing Phase: After parsing the arguments, the input to --numbers is stored in the form of a list in args.numbers. Utilizing the Input: You just need to iterate over the list, calculate statistics (e.g., sum, mean), or any other calculation on the input. Case 1: Providing Multiple Numbers To specify multiple integers for the --numbers parameter, execute: python3 file.py --numbers 10 20 30 Case 2: Providing a Single Number If just one integer is specified, run: python3 file.py --numbers 5 Case 3: No Input Provided If the script is run without --numbers: python3 file.py Case 4: Invalid Input In case of inputting a non-integer value: python3 file.py --numbers 10 abc 20 Example 4: Required Optional Arguments Optional arguments (those that begin with the --) are not mandatory by default. But there are times when you would like them to be mandatory for your script to work properly. You can achieve this by passing the required=True parameter when defining the argument. In this script, --config specifies a path to a configuration file. By leveraging required=True, the script enforces that a value for --config must be provided. If omitted, the program will throw an error. import argparse # Create an ArgumentParser object parser = argparse.ArgumentParser(description="Demonstrating required optional arguments in argparse.") # Add the --config argument parser.add_argument('--config', required=True, help="Path to the configuration file") # Parse the arguments args = parser.parse_args() # Access and display the provided configuration file path print(f"Configuration file path: {args.config}") Adding the --config Option: --config is considered optional since it starts with --. However, thanks to the required=True parameter, users must include it when they run the script. The help parameter clarifies what this parameter does, and you'll see this information in the help message when you use --help. Parsing: The parse_args() method takes care of processing the arguments. If someone forgets to include --config, the program will stop and show a clear error message. Accessing the Input: The value you provide for --config gets stored in args.config. You can then use this in your script to work with the configuration file. Case 1: Valid Input For providing a valid path to the configuration file, use: python3 file.py --config settings.json Case 2: Missing the Required Argument For running the script without specifying --config, apply: python3 file.py Advanced Features  While argparse excels at handling basic command-line arguments, it also provides advanced features that enhance the functionality and usability of your CLIs. These features ensure your scripts are scalable, readable, and easy to maintain. Below are some advanced capabilities you can leverage. Handling Boolean Flags Boolean flags allow toggling features (on/off) without requiring user input. Use the action='store_true' or action='store_false' parameters to implement these flags. parser.add_argument('--debug', action='store_true', help="Enable debugging mode") Including --debug enables debugging mode, useful for many Python argparse examples. Grouping Related Arguments Use add_argument_group() to organize related arguments, improving readability in complex CLIs. group = parser.add_argument_group('File Operations') group.add_argument('--input', type=str, help="Input file") group.add_argument('--output', type=str, help="Output file") Grouped arguments appear under their own section in the --help documentation. Mutually Exclusive Arguments To ensure users select only one of several conflicting options, use the add_mutually_exclusive_group() method. group = parser.add_mutually_exclusive_group() group.add_argument('--json', action='store_true', help="Output in JSON format") group.add_argument('--xml', action='store_true', help="Output in XML format") This ensures one can choose either JSON or XML, but not both. Conclusion The argparse Python module simplifies creating reliable CLIs for handling Python program command line arguments. From the most basic option of just providing an input to more complex ones like setting choices and nargs, developers can build user-friendly and robust CLIs. Following the best practices of giving proper names to arguments and writing good docstrings would help you in making your scripts user-friendly and easier to maintain.
21 July 2025 · 10 min to read
Python

How to Get the Length of a List in Python

Lists in Python are used almost everywhere. In this tutorial we will look at four ways to find the length of a Python list: by using built‑in functions, recursion, and a loop. Knowing the length of a list is most often required to iterate through it and perform various operations on it. len() function len() is a built‑in Python function for finding the length of a list. It takes one argument—the list itself—and returns an integer equal to the list’s length. The same function also works with other iterable objects, such as strings. Country_list = ["The United States of America", "Cyprus", "Netherlands", "Germany"] count = len(Country_list) print("There are", count, "countries") Output: There are 4 countries Finding the Length of a List with a Loop You can determine a list’s length in Python with a for loop. The idea is to traverse the entire list while incrementing a counter by  1 on each iteration. Let’s wrap this in a separate function: def list_length(list): counter = 0 for i in list: counter = counter + 1 return counter Country_list = ["The United States of America", "Cyprus", "Netherlands", "Germany", "Japan"] count = list_length(Country_list) print("There are", count, "countries") Output: There are 5 countries Finding the Length of a List with Recursion The same task can be solved with recursion: def list_length_recursive(list): if not list: return 0 return 1 + list_length_recursive(list[1:]) Country_list = ["The United States of America", "Cyprus", "Netherlands","Germany", "Japan", "Poland"] count = list_length_recursive(Country_list) print("There are", count, "countries") Output: There are 6 countries How it works. The function list_length_recursive() receives a list as input. If the list is empty, it returns 0—the length of an empty list. Otherwise it calls itself recursively with the argument list[1:], a slice of the original list starting from index 1 (i.e., the list without the element at index 0). The result of that call is added to 1. With each recursive step the returned value grows by one while the list shrinks by one element. length_hint() function The length_hint() function lives in the operator module. That module contains functions analogous to Python’s internal operators: addition, subtraction, comparison, and so on. length_hint() returns the length of iterable objects such as strings, tuples, dictionaries, and lists. It works similarly to len(): from operator import length_hint Country_list = ["The United States of America", "Cyprus", "Netherlands","Germany", "Japan", "Poland", "Sweden"] count = length_hint(Country_list) print("There are", count, "countries") Output: There are 7 countries Note that length_hint() must be imported before use. Conclusion In this guide we covered four ways to determine the length of a list in Python. Under equal conditions the most efficient method is len(). The other approaches are justified mainly when you are implementing custom classes similar to list.
17 July 2025 · 3 min to read

Do you have questions,
comments, or concerns?

Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
Email us
Hostman's Support