Introduction to Requests Library in Python

Requests is a Python library that helps your local script interact with web resources and the global network. It provides developers with a wide range of functions for working with all types of HTTP requests. With this library, you can obtain weather forecasts, translate text, and download movies or images without a browser within your script.

This guide provides introductory information about the Python Requests library, which will be sufficient for using it in your scripts.

How to Install the requests Library in Python

For the purpose of this guide, we will be working in the PyCharm IDE. requests is an external library in Python, so it needs to be installed before use. First, create a project and open the terminal. Run the following command:

pip3 install requests

If you are using a virtual environment like Pipenv, you can install the Requests library in Python 3 with the following command:

pipenv install requests

After running these commands, the module will begin downloading. You can use the pip freeze command to check which modules have been installed:

pip3 freeze

Example output:

certifi==2022.9.24
charset-normalizer==2.1.1
idna==3.4
requests==2.28.1
urllib3==1.26.12

Here's a brief explanation of what these modules do:

• certifi: A package of certificates for ensuring secure connections.
• charset-normalizer: A library that automatically detects text encodings, useful since not all sites and services use UTF-8.
• idna: A library that supports internationalized domain names.
• requests: The actual Requests module.
• urllib3: A module that provides functions and classes for working with URLs.

First Requests

In this section, we will write code to retrieve information from web resources and learn the basics of the Requests library. We will explore all aspects in more detail later.

To start working with the Requests library, we need to import it:

import requests

Let's request google.com:

import requests as rq
response = rq.get('https://google.com')
print(response)

Output:

<Response [200]>

Here, we use an HTTP GET request, which is similar to visiting a website via a URL in a browser. In return, we get a Response object with a status code 200, meaning the request was successful. If we try to access a non-existent section of the site, we will receive a different response:

import requests as rq
response = rq.get('https://google.com/hostman')
print(response)

Output:

<Response [404]>

Now, let's discuss the types of HTTP requests and how to work with them.

HTTP Requests

The primary HTTP request type is GET, which allows you to view the content of a resource without modifying it. However, other requests may be needed for full interaction with web resources. Not all servers support all requests. Here are the seven types of requests supported by the Requests library:

• GET
• POST
• OPTIONS
• HEAD
• PUT
• PATCH
• DELETE

To test the Requests library, the creators have designed the site httpbin.org, which you can use to practice.

GET

A GET request sends information to the site via the URL. It is suitable for cases where the transmitted data is not sensitive, such as searching for a product online. You shouldn't use it to send passwords or other confidential information. Data is appended to the URL after a ? symbol, followed by key-value pairs, like so:

https://serverurl.com/get?param1=value1&param2=value2

Where:

• https://serverurl.com/get is the URL.
• param1=value1&param2=value2 are the parameters, separated by &.

In Requests, the GET method syntax is:

requests.get('URL', {key: value}, additional_arguments)

Where:

• URL: The URL of the resource.
• {key: value}: Optional parameters for the request.
• additional_arguments: Optional arguments like timeout.

Example:

import requests as rq
GetParams = {'param1': 'value1', 'param2': 'value2'}
response = rq.get('https://google.com', GetParams)
print(response.url)

Output:

https://www.google.com/?param1=value1&param2=value2

POST

The POST request is used to send data to a website in the request body. This data does not appear in the URL and is ideal for sending confidential information.

In Requests, the POST method syntax is:

requests.post('URL', {key: value}, additional_arguments)

Where:

• URL: The URL of the resource.
• {key: value}: Optional parameters for the request body.

Example:

import requests as rq
PostParams = {'param1': 'value1', 'param2': 'value2'}
response = rq.post('https://httpbin.org/post', PostParams, timeout=2)
print(response.json()['form'])

Output:

{'param1': 'value1', 'param2': 'value2'}

Here, we used the json() method to retrieve the request body. If you make a similar request to google.com, you will get an error:

import requests as rq
PostParams = {'param1': 'value1', 'param2': 'value2'}
response = rq.post('https://google.com', PostParams, timeout=2)
print(response)

Output:

<Response [405]>

Error 405 (Method Not Allowed) means that the resource does not support this type of request. To check which requests are supported by a resource, you can use the OPTIONS method.

OPTIONS

The OPTIONS method helps determine which requests the resource will not block.

Syntax for OPTIONS:

requests.options('URL', optional_arguments)

Let's send an OPTIONS request to google.com to find out which methods it supports:

import requests as rq
response = rq.options('https://google.com', timeout=2)
print(response.headers['Allow'])

Output:

GET, HEAD

HEAD

A HEAD request returns only the HTTP headers from the server. This is useful when you need metadata about a file or resource without downloading its content. It can also be used for testing purposes.

Syntax for HEAD:

requests.head('URL', optional_arguments)

• URL: The resource's address.

• optional_arguments: Additional arguments, such as a timeout.

Example with google.com:

import requests as rq
response = rq.head('https://google.com', timeout=2)
print(response.headers)

This will return a list of headers from the server.

PUT

PUT is used to create a new object or replace an existing one. It is similar to POST but is idempotent, meaning repeated calls with the same data will not change the result.

To clarify, imagine a database of usernames and passwords. If a user wants to change their password:

• POST might add a new record (if no validation occurs).
• PUT would update the existing record.

Syntax for PUT:

requests.put('URL', {key: value}, optional_arguments)

• URL: The resource's address.

• {key: value}: Parameters, which are automatically included in the request body.

• optional_arguments: Additional parameters like timeout.

Example with httpbin.org:

import requests as rq
PutParams = {'param1': 'value2', 'param2': 'value2'}
response = rq.put('https://httpbin.org/put', data=PutParams, timeout=2)
print(response.status_code)

Output:

200

PATCH

PATCH is used for partial updates to a resource, such as changing a token or specific piece of data.

Syntax for PATCH:

requests.patch('URL', {key: value}, optional_arguments)

Example with httpbin.org:

import requests as rq
PatchParams = {'param1': 'value2', 'param2': 'value2'}
response = rq.patch('https://httpbin.org/patch', data=PatchParams, timeout=2)
print(response.status_code)

Output:

200

DELETE

The DELETE method is used to remove an object from the resource.

Syntax for DELETE:

requests.delete('URL', {key: value})

• URL: The resource's address.
• {key: value}: Parameters representing the data to be deleted.

Example with httpbin.org:

import requests as rq
DelParams = {'param1': 'value2', 'param2': 'value2'}
response = rq.delete('https://httpbin.org/delete', data=DelParams, timeout=2)
print(response.status_code)

Output:

200

Response Object

As shown in the examples, the Response object in the Python Requests library has many methods and properties. Below is a list of them with brief descriptions:

• apparent_encoding — The encoding detected by charset-normalizer.

• close() — Closes the connection to the server.

• content — Returns the received data in bytes.

• cookies — Returns cookies.

• elapsed — Returns the elapsed time of the request.

• encoding — Specifies the encoding for decoding the response content.

• headers — Returns the headers of the resource.

• history — Returns the redirection history.

• is_permanent_redirect — Checks for permanent redirects.

• is_redirect — Checks if a redirect occurred.

• iter_content() — Returns the content in chunks.

• iter_lines() — Returns the content line by line.

• json() — Returns the response in JSON format.

• links — Returns the links in the response headers.

• next — Returns the next PreparedRequest.

• ok — Returns True if the request was successful, False if not.

• raise_for_status() — Raises an HTTPError exception if the request failed.

• reason — Returns a textual explanation of the response.

• request — Returns the PreparedRequest object.

• status_code — Returns the HTTP status code.

• text — Returns the response in Unicode.

• url — Returns the URL of the resource.

Conclusion

In this material, we explored the basic elements of the Requests library. If you want to learn more, the official documentation of the Python Requests library provides all the necessary details. And to test your scripts, you can use Hostman VPS.