Sign In
Sign In

Working with HTTP Requests in Python

Working with HTTP Requests in Python
Hostman Team
Technical writer
Python
23.10.2024
Reading time: 8 min

The entire web operates using the HTTP protocol. To open this page, you sent an HTTP request via your browser to the server hosting Hostman tutorials. The server analyzed your request and responded by returning this post so you could learn about the requests library in Python.

The requests library allows you to send HTTP requests. With it, you can write requests in Python to solve a wide range of tasks.

What You Need to Know to Work with requests

Sending HTTP requests and using the responses isn’t rocket science. However, some basic knowledge is still required. At the very least, you need to know the primary methods you'll be sending to servers:

  • GET — retrieve a resource.

  • POST — create a resource.

  • PUT — update a resource.

  • DELETE — delete a resource.

These are not all the methods. There are also HEAD, PATCH, and others. But for our practice, we’ll only need GET, the most common method. Users constantly request servers to give them something: pages, posts, images. Processing HTTP requests in Python with other methods works similarly.

In response to requests, you’ll receive status codes. These range from 1xx to 5xx. Every internet user encounters them, though not everyone knows what they are. Error 404 is status code 404. It indicates a client-side error: the user is trying to access a page that doesn’t exist.

The first digit of a status code indicates the type of response:

  • 1xx — informational codes.

  • 2xx — success.

  • 3xx — redirection.

  • 4xx — client-side error.

  • 5xx — server-side error.

These basic concepts are enough to make a few interesting requests and correctly interpret the responses. Now, let’s see how all this behind-the-scenes web magic is handled with the requests library.

Installing the Library

To work with the library, you need Python installed on your computer. You’ll also need the package manager pip to install requests.

To install Python requests using pip:

python -m pip install requests

The library’s code is hosted on GitHub. You can clone it to your computer:

git clone git://github.com/psf/requests.git

For more details, check the library’s documentation.

Usage Examples

Now, for the most exciting part — practice. 

Let's start with a simple request. We'll give the Python requests library a task to connect to a page containing information about Hostman cloud servers.

Create a file called script.py. Inside it, write the following script:

import requests
result = requests.get('https://hostman.con/cloud-servers')
print(result)

The first line imports the requests library. Then, you define a variable result and use the GET method to get a response from the server, specifying the URI you're interested in. The last line prints the response in the terminal.

To run this short program, open any terminal, navigate to the folder where the script.py file is located, and execute it with the command: 

python script.py

The response should return status code 200, meaning the requested page is available. You can continue working with it.

As an example, let's try using a non-existent URI. You should get a 404 error code.

Change the page address in your script. You can make up your own invalid link or just alter any working one:

import requests
result = requests.get('https://hostman.com/badlink')
print(result)

The response will return a 404 status code.

From Python’s perspective, every response is an object. This object has its own methods and functions that help retrieve additional information or normalize data.

Let's revert to the correct page address and try to get its content. To do this, simply add the text method to the print command. Here’s what the script should look like after editing:

import requests
result = requests.get('https://hostman.com/cloud-servers')
print(result.text)

Running the script will display the entire HTML code of the requested page in the terminal. You can see the markup, linked style sheets, scripts, and content. This is public information, which can also be viewed in a browser using the "View Page Source" option.

Instead of the text method, you can use the url method, which will display the URL you requested. You can also use the status_code method to print the request's status code.

This was just the beginning of working with requests in Python. Now, let’s look at some more interesting examples.

Basic Authentication

Authentication helps an application understand who you are and check if there’s any information about you in the database. Typically, you provide your credentials to the server by passing them through the Authorization header.

One example of an API that requires user authentication is the GitHub Authenticated User API. This service provides information about the profile of an authenticated user. To make a request to the Authenticated User API, you must pass your GitHub username and password as a tuple, using the get() method:

from getpass import getpass
requests.get('https://api.github.com/user', auth=('username', getpass()))

If you passed valid credentials in the auth tuple, you’ll receive a success message in response:

<Response [200]>

If you try to make the same request without valid credentials, you’ll get a 401 Unauthorized status code:

>>> requests.get('https://api.github.com/user')
<Response [401]>

When you pass the username and password in a tuple, the credentials are applied using the basic access authentication scheme. You can make the same request using HTTPBasicAuth:

from requests.auth import HTTPBasicAuth
from getpass import getpass
requests.get(
   'https://api.github.com/user',
   auth=HTTPBasicAuth('username', getpass())
)
<Response [200]>

The requests library also provides other authentication methods, such as HTTPDigestAuth and HTTPProxyAuth. You can also implement your own method. However, keep in mind that poorly designed authentication mechanisms can introduce security vulnerabilities. Unless there's a strong need to create your own solution, using a proven authentication scheme like Basic or OAuth is better.

SSL Verification

Working with requests in Python also allows you to solve another important task — checking if a website has an SSL certificate.

When handling sensitive data, security is crucial. You can protect it during transmission over HTTP by setting up an encrypted connection using an SSL certificate.

To ensure the safety of a site, you need to verify the SSL certificate of the target server. The good news is that the requests library does this by default. However, in some cases, you might want to change this behavior.

If you want to disable SSL certificate verification, set the verify parameter to False:

requests.get('https://api.github.com', verify=False)

This will return a warning:

InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised.
<Response [200]>

requests even warns you when making a request without SSL verification to help keep your data secure.

Fetching the Weather Forecast

Finally, let’s look at a small project example. We’ll create a utility that fetches the weather forecast and displays the result in the terminal. The data source will be OpenWeather, where you can create a free API key for access.

Create a file called weather.py and write the following:

import requests
result = requests.get('https://api.openweathermap.org/data/2.5/weather?q=Warsaw,pl&appid=someidappkey0ff7889240be2&units=metric')
weather = result.text
print(weather)

First, you import the requests library. Then, you make an API request to the weather service. After the q parameter, you specify the city and country; appid is the key that you need to create on OpenWeather. The units=metric parameter indicates that the metric system should be used, so the temperature will be displayed in Celsius.

By default, the response comes in JSON format, but you can choose XML or HTML by adding the mode parameter to the request with the corresponding value.

Here’s an example of a response in JSON format:

{
   "coord": {"lon": 21.0118, "lat": 52.2298},
   "weather": [
       {
           "id": 800,
           "main": "Clear",
           "description": "clear sky",
           "icon": "01d"
       }
   ],
   "base": "stations",
   "main": {
       "temp": 15.35,
       "feels_like": 14.28,
       "temp_min": 14.78,
       "temp_max": 16.12,
       "pressure": 1021,
       "humidity": 63
   },
   "visibility": 10000,
   "wind": {
       "speed": 3.09,
       "deg": 90
   },
   "clouds": {
       "all": 0
   },
   "dt": 1697721600,
   "sys": {
       "type": 2,
       "id": 2038500,
       "country": "PL",
       "sunrise": 1697689772,
       "sunset": 1697726778
   },
   "timezone": 7200,
   "id": 756135,
   "name": "Warsaw",
   "cod": 200
}

Even without additional data processing, it’s easy to see that at the time of checking, the temperature in Warsaw was slightly above 7 degrees Celsius, with light wind. For example, you can use this information to create a web interface for a more user-friendly display.

Conclusion

You’ve learned the basics of the requests library in Python. Now you can make requests using HTTP methods, perform user authentication, check data, and retrieve information from various sources.

On our app platform you can find Python applications, such as Celery, Django, FastAPI and Flask. 

Python
23.10.2024
Reading time: 8 min

Similar

Python

How to Get the Length of a List in Python

Lists in Python are used almost everywhere. In this tutorial we will look at four ways to find the length of a Python list: by using built‑in functions, recursion, and a loop. Knowing the length of a list is most often required to iterate through it and perform various operations on it. len() function len() is a built‑in Python function for finding the length of a list. It takes one argument—the list itself—and returns an integer equal to the list’s length. The same function also works with other iterable objects, such as strings. Country_list = ["The United States of America", "Cyprus", "Netherlands", "Germany"] count = len(Country_list) print("There are", count, "countries") Output: There are 4 countries Finding the Length of a List with a Loop You can determine a list’s length in Python with a for loop. The idea is to traverse the entire list while incrementing a counter by  1 on each iteration. Let’s wrap this in a separate function: def list_length(list): counter = 0 for i in list: counter = counter + 1 return counter Country_list = ["The United States of America", "Cyprus", "Netherlands", "Germany", "Japan"] count = list_length(Country_list) print("There are", count, "countries") Output: There are 5 countries Finding the Length of a List with Recursion The same task can be solved with recursion: def list_length_recursive(list): if not list: return 0 return 1 + list_length_recursive(list[1:]) Country_list = ["The United States of America", "Cyprus", "Netherlands","Germany", "Japan", "Poland"] count = list_length_recursive(Country_list) print("There are", count, "countries") Output: There are 6 countries How it works. The function list_length_recursive() receives a list as input. If the list is empty, it returns 0—the length of an empty list. Otherwise it calls itself recursively with the argument list[1:], a slice of the original list starting from index 1 (i.e., the list without the element at index 0). The result of that call is added to 1. With each recursive step the returned value grows by one while the list shrinks by one element. length_hint() function The length_hint() function lives in the operator module. That module contains functions analogous to Python’s internal operators: addition, subtraction, comparison, and so on. length_hint() returns the length of iterable objects such as strings, tuples, dictionaries, and lists. It works similarly to len(): from operator import length_hint Country_list = ["The United States of America", "Cyprus", "Netherlands","Germany", "Japan", "Poland", "Sweden"] count = length_hint(Country_list) print("There are", count, "countries") Output: There are 7 countries Note that length_hint() must be imported before use. Conclusion In this guide we covered four ways to determine the length of a list in Python. Under equal conditions the most efficient method is len(). The other approaches are justified mainly when you are implementing custom classes similar to list.
17 July 2025 · 3 min to read
Python

Understanding the main() Function in Python

In any complex program, it’s crucial to organize the code properly: define a starting point and separate its logical components. In Python, modules can be executed on their own or imported into other modules, so a well‑designed program must detect the execution context and adjust its behavior accordingly.  Separating run‑time code from import‑time code prevents premature execution, and having a single entry point makes it easier to configure launch parameters, pass command‑line arguments, and set up tests. When all important logic is gathered in one place, adding automated tests and rolling out new features becomes much more convenient.  For exactly these reasons it is common in Python to create a dedicated function that is called only when the script is run directly. Thanks to it, the code stays clean, modular, and controllable. That function, usually named main(), is the focus of this article. All examples were executed with Python 3.10.12 on a Hostman cloud server running Ubuntu 22.04. Each script was placed in a separate .py file (e.g., script.py) and started with: python script.py The scripts are written so they can be run just as easily in any online Python compiler for quick demonstrations. What Is the main() Function in Python The simplest Python code might look like: print("Hello, world!")  # direct execution Or a script might execute statements in sequence at file level: print("Hello, world!")       # action #1 print("How are you, world?") # action #2 print("Good‑bye, world...")  # action #3 That trivial arrangement works only for the simplest scripts. As a program grows, the logic quickly becomes tangled and demands re‑organization: # function containing the program’s main logic (entry point) def main():     print("Hello, world!") # launch the main logic if __name__ == "__main__":     main()                    # call the function with the main logic With more actions the code might look like: def main(): print("Hello, world!") print("How are you, world?") print("Good‑bye, world...") if __name__ == "__main__": main() This implementation has several important aspects, discussed below. The main() Function The core program logic lives inside a separate function. Although the name can be anything, developers usually choose main, mirroring C, C++, Java, and other languages.  Both helper code and the main logic are encapsulated: nothing sits “naked” at file scope. # greeting helper def greet(name): print(f"Hello, {name}!") # program logic def main(): name = input("Enter your name: ") greet(name) # launch the program if __name__ == "__main__": main() Thus main() acts as the entry point just as in many other languages. The if __name__ == "__main__" Check Before calling main() comes the somewhat odd construct if __name__ == "__main__":.  Its purpose is to split running from importing logic: If the script runs directly, the code inside the if block executes. If the script is imported, the block is skipped. Inside that block, you can put any code—not only the main() call: if __name__ == "__main__":     print("Any code can live here, not only main()") __name__ is one of Python’s built‑in “dunder” (double‑underscore) variables, often called magic or special. All dunder objects are defined and used internally by Python, but regular users can read them too. Depending on the context, __name__ holds: "__main__" when the module runs as a standalone script. The module’s own name when it is imported elsewhere. This lets a module discover its execution context. Advantages of Using  main() Organization Helper functions and classes, as well as the main function, are wrapped separately, making them easy to find and read. Global code is minimal—only initialization stays at file scope: def process_data(data): return [d * 2 for d in data] def main(): raw = [1, 2, 3, 4] result = process_data(raw) print("Result:", result) if __name__ == "__main__": main() A consistent style means no data manipulation happens at the file level. Even in a large script you can quickly locate the start of execution and any auxiliary sections. Isolation When code is written directly at the module level, every temporary variable, file handle, or connection lives in the global namespace, which can be painful for debugging and testing. Importing such a module pollutes the importer’s globals: # executes immediately on import values = [2, 4, 6] doubles = [] for v in values: doubles.append(v * 2) print("Doubled values:", doubles) With main() everything is local; when the function returns, its variables vanish: def double_list(items): return [x * 2 for x in items] # create a new list with doubled elements def main(): values = [2, 4, 6] result = double_list(values) print("Doubled values:", result) if __name__ == "__main__": main() That’s invaluable for unit testing, where you might run specific functions (including  main()) without triggering the whole program. Safety Without the __name__ check, top‑level code runs even on import—usually undesirable and potentially harmful. some.py: print("This code will execute even on import!") def useful_function(): return 42 main.py: import some print("The logic of the imported module executed itself...") Console: This code will execute even on import! The logic of the imported module executed itself... The safer some.py: def useful_function():     return 42 def main():     print("This code will not run on import") main() plus the __name__ check guard against accidental execution. Inside main() you can also verify user permissions or environment variables. How to Write main() in Python Remember: main() is not a language construct, just a regular function promoted to “entry point.” To ensure it runs only when the script starts directly: Tools – define helper functions with business logic. Logic – assemble them inside main() in the desired order. Check – add the if __name__ == "__main__" guard.  This template yields structured, import‑safe, test‑friendly code—excellent practice for any sizable Python project. Example Python Program Using main() # import the standard counter from collections import Counter # runs no matter how the program starts print("The text‑analysis program is active") # text‑analysis helper def analyze_text(text): words = text.split() # split text into words total = len(words) # total word count unique = len(set(words)) # unique word count avg_len = sum(len(w) for w in words) / total if total else 0 freq = Counter(words) # build frequency counter top3 = freq.most_common(3) # top three words return { 'total': total, 'unique': unique, 'avg_len': avg_len, 'top3': top3 } # program’s main logic def main(): print("Enter text (multiple lines). Press Enter on an empty line to finish:") lines = [] while True: line = input() if not line: break lines.append(line) text = ' '.join(lines) stats = analyze_text(text) print(f"\nTotal number of words: {stats['total']}") print(f"Unique words: {stats['unique']}") print(f"Average word length: {stats['avg_len']:.2f}") print("Top‑3 most frequent words:") for word, count in stats['top3']: print(f" {word!r}: {count} time(s)") # launch program if __name__ == "__main__": main() Running the script prints a prompt: Enter text (multiple lines). Press Enter on an empty line to finish: Input first line: Star cruiser Orion glided silently through the darkness of intergalactic space. Second line: Signals of unknown life‑forms flashed on the onboard sensors where the nebula glowed with a phosphorescent light. Third line: The cruiser checked the sensors, then the cruiser activated the defense system, and the cruiser returned to its course. Console output: The text‑analysis program is active Total number of words: 47 Unique words: 37 Average word length: 5.68 Top‑3 most frequent words: 'the': 7 time(s) 'cruiser': 4 time(s) 'of': 2 time(s) If you import this program (file program.py) elsewhere: import program         # importing program.py Only the code outside main() runs: The text‑analysis program is active So, a moderately complex text‑analysis utility achieves clear logic separation and context detection. When to Use main() and When Not To Use  main() (almost always appropriate) when: Medium/large scripts – significant code with non‑trivial logic, multiple functions/classes. Libraries or CLI utilities – you want parts of the module importable without side effects. Autotests – you need to test pure logic without extra boilerplate. You can skip main() when: Tiny one‑off scripts – trivial logic for a quick data tweak. Educational snippets – short examples illustrating a few syntax features. In short, if your Python program is a standalone utility or app with multiple processing stages, command‑line arguments, and external resources—introduce  main(). If it’s a small throw‑away script, omitting main() keeps things concise. Conclusion The  main() function in Python serves two critical purposes: Isolates the program’s core logic from the global namespace. Separates standalone‑execution logic from import logic. Thus, a Python file evolves from a straightforward script of sequential actions into a fully‑fledged program with an entry point, encapsulated logic, and the ability to detect its runtime environment.
14 July 2025 · 8 min to read
Python

Python Static Method

A static method in Python is bound to the class itself rather than any instance of that class. So, you can call it without first creating an object and without access to instance data (self).  To create a static method we need to use a decorator, specifically @staticmethod. It will tell Python to call the method on the class rather than an instance. Static methods are excellent for utility or helper functions that are logically connected to the class but don't need to access any of its properties.  When To Use & Not to Use a Python Static Method Static methods are frequently used in real-world code for tasks like input validation, data formatting, and calculations—especially when that logic naturally belongs with a class but doesn't need its state. Here's an example from a User class that checks email format: class User: @staticmethod def is_valid_email(email): return "@" in email and "." in email This method doesn't depend on any part of the User instance, but conceptually belongs in the class. It can be used anywhere as User.is_valid_email(email), keeping your code cleaner and more organized. If the logic requires access to or modification of instance attributes or class-level data, avoid using a static method as it won't help here. For instance, if you are working with user settings or need to monitor object creation, you will require a class method or an instance method instead. class Counter: count = 0 @classmethod def increment(cls): cls.count += 1 In this example, using a static method would prevent access to cls.count, making it useless for this kind of task. Python Static Method vs Class Method Though they look similar, class and static methods in Python have different uses; so, let's now quickly review their differences. Defined inside a class, a class method is connected to that class rather than an instance. Conventionally called cls, the class itself is the first parameter; so, it can access and change class-level data. Factory patterns, alternate constructors, or any activity applicable to the class as a whole and not individual instances are often implemented via class methods. Conversely, a static method is defined within a class but does not start with either self or cls parameters. It is just a regular function that “lives” inside a class but doesn’t interact with the class or its instances. For utility tasks that are conceptually related to the class but don’t depend on its state, static methods are perfect. Here's a quick breakdown of the Python class/static methods differences: Feature Class Method Static Method Binding Bound to the class Not bound to class or instance First parameter cls (class itself) None (no self or cls) Access to class/instance data Yes No Common use cases Factory methods, class-level behavior Utility/helper functions Decorator @classmethod @staticmethod Python Static Method vs Regular Functions You might ask: why not just define a function outside the class instead of using a static method? The answer is structure. A static method keeps related logic grouped within the class, even if it doesn't interact with the class or its instances. # Regular function def is_even(n): return n % 2 == 0 # Static method inside a class class NumberUtils: @staticmethod def is_even(n): return n % 2 == 0 Both functions do the same thing, but placing is_even inside NumberUtils helps keep utility logic organized and easier to find later. Let’s proceed to the hands-on Python static method examples. Example #1 Imagine that we have a MathUtils class that contains a static method for calculating the factorial: class MathUtils: @staticmethod def factorial(n): if n == 0: return 1 else: return n * MathUtils.factorial(n-1) Next, let's enter: print(MathUtils.factorial(5))120 We get the factorial of 5, which is 120. Here, the factorial static method does not use any attributes of the class instance, only the input argument n. And we called it using the MathUtils.factorial(n) syntax without creating an instance of the MathUtils class. In Python, static methods apply not only in classes but also in modules and packages. The @staticmethod decorator marks a function you define inside a class if it does not interact with instance-specific data. The function exists on its own; it is related to the class logically but is independent of its internal state. Managed solution for Backend development Example #2 Let's say we're working with a StringUtils module with a static method for checking if a string is a palindrome. The code will be: def is_palindrome(string):    return string == string[::-1] This function doesn't rely on any instance-specific data — it simply performs a check on the input. That makes it a good candidate for a static method. To organize it within a class and signal that it doesn't depend on the class state, we can use the @staticmethod decorator like this: class StringUtils:    @staticmethod    def is_palindrome(string):       return string == string[::-1] Let's enter for verification: print(StringUtils.is_palindrome("deed"))True print(StringUtils.is_palindrome("deer"))False That's correct, the first word is a palindrome, so the interpreter outputs True, but the second word is not, and we get False. So, we can call the is_palindrome method through the StringUtils class using the StringUtils.is_palindrome(string) syntax instead of importing the is_palindrome function and calling it directly. - Python static method and class instance also differ in that the static cannot affect the state of an instance. Since they do not have access to the instance, they cannot alter attribute values, which makes sense. Instance methods are how one may modify the instance state of a class. Example #3 Let's look at another example. Suppose we have a Person class that has an age attribute and a static is_adult method that checks the value against the age of majority: class Person:    def __init__(self, age):        self.age = age    @staticmethod    def is_adult(age):       return age >= 21 Next, let's create an age variable with a value of 24, call the is_adult static method from the Person class with this value and store its result in the is_adult variable, like this: age = 24is_adult = Person.is_adult(age) Now to test this, let's enter: print(is_adult)True Since the age matches the condition specified in the static method, we get True. In the example, the is_adult static method serves as an auxiliary tool—a helper function—accepting the age argument but without access to the age attribute of the Person class instance. Conclusion Static methods improve code readability and make it possible to reuse it. They are also more convenient when compared to standard Python functions. Static methods are convenient as, unlike functions, they do not call for a separate import. Therefore, applying Python class static methods can help you streamline and work with your code greatly. And, as you've probably seen from the examples above, they are quite easy to master. On our app platform you can find Python applications, such as Celery, Django, FastAPI and Flask. 
16 April 2025 · 6 min to read

Do you have questions,
comments, or concerns?

Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
Email us
Hostman's Support