Cloud servers from $4/mo - Grab the Deal! Let’s start ->
English
Community
Sign In
Sign In

How to Use Grep and Regular Expressions in Linux

How to Use Grep and Regular Expressions in Linux
Hostman Team
Technical writer
Linux Linux-System Sistema Linux O sistema Linux
11.02.2025
Reading time: 16 min
Share

GREP (short for "global regular expression print") is one of the most popular utilities in the Linux operating system.

With it, you can search for phrases (sequences of characters) in multiple files simultaneously using regular expressions and filter the output of other commands, keeping only the necessary information.

This guide will cover how to search for specific expressions in a set of text files with various contents using the GREP utility.

All examples shown were run on a cloud server hosted by Hostman running Ubuntu version 22.04.

How Does GREP Work

The GREP command follows this structure:

grep [OPTIONS] [PATTERN] [SOURCES]

Where:

  • OPTIONS: Special parameters (flags) that activate certain mechanisms in the utility related to searching for expressions and displaying results.

  • PATTERN: A regular expression (or plain string) containing the phrase (pattern, template, sequence of characters) you want to find.

  • SOURCES: The path to the files where we will search for the specified expression.

If the GREP command is used to filter the output of another command, its structure looks a bit different:

[COMMAND] | grep [OPTIONS] [PATTERN]

Thus:

  • COMMAND: An arbitrary command with its own set of parameters whose output needs to be filtered.

  • The "pipe" symbol (|) is necessary to create a command pipeline, redirecting streams so that the output of an arbitrary command becomes the input for the GREP command.

Preparation

To understand the nuances of using GREP, it's best to start with small examples of searching for specific phrases. Therefore, we will first create a few text files and then test the GREP command on them.

Let’s first prepare a separate directory where the search will take place:

mkdir texts

Next, create the first file:

nano texts/poem

It will contain one of Langston Hughes's poems:

Hold fast to dreams  
For if dreams die  
Life is a broken-winged bird  
That cannot fly.  
Hold fast to dreams  
For when dreams go  
Life is a barren field  
Frozen with snow.

Now, create the second file:

nano texts/code.py

It will contain a simple Python script:

from datetime import date

dateNow = date.today()
print("Current time:", dateNow)

Finally, create the third file:

nano texts/page.html

This one will have simple HTML markup:

<html>
	<head>
		<title>Some Title</title>
	</head>

	<body>
		<div class="block">
			<p>There's gold here</p>
		</div>

		<div class="block">
			<p>A mixture of wax and clouds</p>
		</div>

		<div class="block block_special">
			<p>Today there's nothing</p>
		</div>
	</body>
</html>

By using files of different formats, we can better understand what the GREP command does by utilizing the full range of the utility's features.

Regular Expressions

Regular expressions are the foundation of the GREP command. Unlike a regular string, regular expressions contain special characters that allow you to specify phrases with a certain degree of variability.

When using the GREP utility, regular expressions are placed within single quotes:

'^date[[:alpha:]]*'

Thus, the full command can look like this:

grep '^date[[:alpha:]]*' texts/*

In this case, the console output will be:

texts/code.py:dateNow = date.today()

However, using double quotes allows you to pass various system data into the expression. For example, you can first create an environment variable with the search expression:

PATTERN="^date[[:alpha:]]*"

And then use it in the GREP command:

grep "$PATTERN" ./texts/*

Additionally, using single backticks allows you to use bash subprocess commands within the GREP command. For example, you can extract a regular expression from a pre-prepared file:

grep `cat somefile` ./texts/*

Note that with the asterisk symbol (wildcard), you can specify all the files in the directory at once. However, the GREP command also allows you to specify just one file: 

grep '^date[[:alpha:]]' texts/code.py 

Because regular expressions are a universal language used in many operating systems and programming languages, their study is a separate vast topic. 

However, it makes sense to briefly cover the main special characters and their functions. It’s important to note that regular expressions in Linux can work in two modes: basic (Basic Regular Expression, BRE) and extended (Extended Regular Expression, ERE). The extended mode is activated with the additional flag -E. The difference between the two modes lies in the number of available special characters and, consequently, the breadth of available functionality.

Basic Syntax

Basic syntax allows you to define only general formal constructs without considering the specific configuration of their characters.

Start of a line — ^

The caret symbol indicates that the sought sequence of characters must be at the beginning of the line:

grep '^Hold' texts/*

The console output will be as follows:

texts/poem:Hold fast to dreams
texts/poem:Hold fast to dreams

End of a line — $

The dollar sign indicates that the sought sequence of characters must be at the end of the line:

grep '</p>$' texts/*

Output:

texts/page.html:                        <p>There's gold here</p>
texts/page.html:                        <p>A mixture of wax and clouds</p>
texts/page.html:                        <p>Today there's nothing</p>

Note that the console output preserves the original representation of the found lines as they appear in the files.

Start of a word — \<

The backslash and less-than symbol indicate that the sought phrase must be at the beginning of a word:

grep '\<br' texts/*

Output:

texts/poem:Life is a broken-winged bird

End of a word — \>

The backslash and greater-than symbol indicate that the sought sequence of characters must be at the end of a word:

grep 'en\>' texts/*

Output:

texts/poem:Life is a broken-winged bird
texts/poem:For when dreams go
texts/poem:Life is a barren field
texts/poem:Frozen with snow.

Start or end of a word — \b

You can specify the start or end of a word using the more universal sequence of characters — backslash and the letter b.

For example, this marks the beginning:

grep '\bdie' texts/*

Output:

texts/poem:For if dreams die

And this marks the end:

grep '<div\b' texts/*

In this case, the console terminal output will be as follows:

texts/page.html:                <div class="block">
texts/page.html:                <div class="block">
texts/page.html:                <div class="block block_special">

Any character — .

Certain characters in the sought phrases can be left unspecified using the dot symbol:

grep '..ere' texts/*

Output:

texts/page.html:                        <p>There's gold here</p>
texts/page.html:                        <p>Today there's nothing</p>

Extended Syntax

Unlike basic syntax, extended syntax allows you to specify the exact number of characters in the sought phrases, thus expanding the range of possible matches.

Combining patterns — |

To avoid running the GREP command multiple times, you can specify several patterns in a single regular expression:

grep -E '^Hold|</p>$' texts/*

The result of running this command will be a combined console output containing the search results for the two separate regular expressions shown earlier.

texts/page.html:                        <p>There's gold here</p>
texts/page.html:                        <p>A mixture of wax and clouds</p>
texts/page.html:                        <p>Today there's nothing</p>
texts/poem:Hold fast to dreams
texts/poem:Hold fast to dreams

Repetition range — {n, d}

In some cases, certain characters in the sought phrase may vary in quantity. Therefore, in the regular expression, you can specify a range of the allowed number of specific characters.

grep -E 'en{1,2}' texts/*

Output:

texts/code.py:print("Current time:", dateNow)
texts/poem:Life is a broken-winged bird
texts/poem:For when dreams go
texts/poem:Life is a barren field
texts/poem:Frozen with snow.

However, frequently used repetition intervals are more conveniently written as special characters, thus simplifying the appearance of the regular expression.

One or more repetitions — +

A repetition interval from one to infinity can be expressed using the plus sign:

grep -E 'en+' texts/*

In this case, the console output will not differ from the previous example.

texts/code.py:print("Current time:", dateNow)
texts/poem:Life is a broken-winged bird
texts/poem:For when dreams go
texts/poem:Life is a barren field
texts/poem:Frozen with snow.

Zero or one repetition — ?

A repetition interval from 0 to 1 can be expressed using the question mark:

grep -E 'ss?' texts/*

As a result, this command will produce the following output in the console terminal:

texts/page.html:                <div class="block">
texts/page.html:                        <p>There's gold here</p>
texts/page.html:                <div class="block">
texts/page.html:                        <p>A mixture of wax and clouds</p>
texts/page.html:                <div class="block block_special">
texts/page.html:                        <p>Today there's nothing</p>
texts/poem:Hold fast to dreams
texts/poem:For if dreams die
texts/poem:Life is a broken-winged bird
texts/poem:Hold fast to dreams
texts/poem:For when dreams go
texts/poem:Life is a barren field
texts/poem:Frozen with snow.

Character set — [abc]

Instead of one specific character, you can specify an entire set enclosed in square brackets:

grep -E '[Hh]o[Ll]' texts/*

Output:

texts/poem:Hold fast to dreams
texts/poem:Hold fast to dreams

Character range — [a-z]

We can replace a large set of allowed characters with a range written using a hyphen:

grep -E 'h[a-z]+' texts/*

Output:

texts/page.html:<html>
texts/page.html:        <head>
texts/page.html:        </head>
texts/page.html:                        <p>There's gold here</p>
texts/page.html:                        <p>Today there's nothing</p>
texts/page.html:</html>
texts/poem:That cannot fly.
texts/poem:For when dreams go

Moreover, character sets and ranges can be combined:

grep -E 'h[abcd-z]+' texts/*

Each range is implicitly transformed into a set of characters:

  • [a-e] into [abcde]
  • [0-6] into [0123456]
  • [a-eA-F] into [abcdeABCDEF]
  • [A-Fa-e] into [ABCDEFabcde]
  • [A-Fa-e0-9] into [ABCDEFabcde0123456789]
  • [a-dA-CE-G] into [abcdABCEFG]
  • [acegi-l5-9] into [acegijkl56789]

Character type — [:alpha:]

Frequently used ranges can be replaced with predefined character types, whose names are specified in square brackets with colons:

[:lower:]

characters from a to z in lowercase

[:upper:]

characters from A to Z in uppercase

[:alpha:]

all alphabetic characters

[:digit:]

all digit characters

[:alnum:]

all alphabetic characters and digits

It is important to understand that the character type is a separate syntactic construct. This means that it must be enclosed in square brackets, which denote a set or range of characters:

grep -E '[[:alpha:]]+ere' texts/*

Output:

texts/page.html:                        <p>There's gold here</p>
texts/page.html:                        <p>Today there's nothing</p>

Filtering Output

To filter the output of another command, you need to write a pipe symbol after it, followed by the standard call to the GREP utility, but without specifying the files to search:

cat texts/code.py | grep 'import'

Like when searching in regular files, the console output will contain the lines with the matches of the specified phrases:

from datetime import date

In this case, the cat command extracts the file content and passes it to the input stream of the GREP utility.

Search Options

In addition to regular expressions, you can specify additional keys for the GREP command, which are special options in flag format that refine the search.

Extended Regular Expressions (-E)

Activates the extended regular expressions mode, allowing the use of more special characters.

Case Insensitivity (-i)

Performs a search for a regular expression without considering the case of characters:

grep -E -i 'b[ar]' texts/*

The console output corresponding to this command will be:

texts/poem:Life is a broken-winged bird
texts/poem:Life is a barren field

You can also specify flags together in a single string:

grep -Ei 'b[ar]' texts/*

Whole Word (-w)

Performs a search so that the specified regular expression is a complete word (not just a substring) in the found line:

grep -w and texts/*

Note that quotes are not required when specifying a regular string without special characters.

The result of this command will be:

texts/page.html: <p>A mixture of wax and clouds</p>

Multiple Expressions (-e)

To avoid running the command multiple times, you can specify several expressions at once:

grep -e 'Hold' -e 'html' texts/*

The result of this command will be identical to this one:

grep -E 'Hold|html' texts/*

In both cases, the console terminal will display the following output:

texts/page.html:<html>
texts/page.html:</html>
texts/poem:Hold fast to dreams
texts/poem:Hold fast to dreams

Recursive Search (-r)

Performs a recursive search in the specified directory to the maximum depth of nesting:

grep -r '[Ff]ilesystem' /root

The console terminal will display output containing file paths at different nesting levels relative to the specified directory:

/root/parser/parser/settings.py:#HTTPCACHE_STORAGE = "scrapy.extensions.httpcache.FilesystemCacheStorage"
/root/resize.log:Resizing the filesystem on /dev/vda1 to 3931904 (4k) blocks.
/root/resize.log:The filesystem on /dev/vda1 is now 3931904 (4k) blocks long.

Search for Special Characters (-F)

Allows the use of special characters as the characters of the search phrase:

grep -F '[' texts/*

Without this flag, you would encounter an error in the console terminal:

grep: Invalid regular expression

An alternative to this flag would be using the escape character in the form of a backslash (\):

grep '\[' texts/*

Including Files (--include)

Allows limiting the search to the specified files only:

grep --include='*.py' 'date' texts/*

The console output will be:

texts/code.py:from datetime import date
texts/code.py:dateNow = date.today()
texts/code.py:print("Current time:", dateNow)

We can also write this command without the wildcard by using an additional recursive search flag:

grep -r --include='*.py' 'date' texts

Excluding Files (--exclude)

Selectively excludes certain files from the list of search sources:

grep --exclude='*.py' 'th' texts/*

The console output will be:

texts/page.html: <p>Today there's nothing</p>
texts/poem:Frozen with snow.

Output Options

Some parameters of the GREP command affect only the output of search results, improving their informativeness and clarity.

Line Numbers (-n)

To increase the informativeness of the GREP results, you can add the line numbers where the search phrases were found:

grep -n '</p>$' texts/*

Each line in the output will be supplemented with the corresponding line number:

texts/page.html:8:                      <p>There's gold here</p>
texts/page.html:12:                     <p>A mixture of wax and clouds</p>
texts/page.html:16:                     <p>Today there's nothing</p>

Lines Before (-B)

Displays a specified number of lines before the lines with found matches:

grep -B3 'mix' texts/*

After the flag, you specify the number of previous lines to be displayed in the console terminal:

texts/page.html-                </div>
texts/page.html-
texts/page.html-                <div class="block">
texts/page.html:                        <p>A mixture of wax and clouds</p>

Lines After (-A)

Displays a specified number of lines after the lines with found matches:

grep -A3 'mix' texts/*

After the flag, you specify the number of subsequent lines to be displayed in the console terminal:

texts/page.html:                        <p>A mixture of wax and clouds</p>
texts/page.html-                </div>
texts/page.html-
texts/page.html-                <div class="block block_special">

Lines Before and After (-C)

Displays a specified number of lines both before and after the lines with found matches:

grep -C3 'mix' texts/*

After the flag, you specify the number of preceding and following lines to be displayed in the console terminal:

texts/page.html-                </div>
texts/page.html-
texts/page.html-                <div class="block">
texts/page.html:                        <p>A mixture of wax and clouds</p>
texts/page.html-                </div>
texts/page.html-
texts/page.html-                <div class="block block_special">

Line Count (-c)

Instead of listing the found lines, the GREP command will output only the number of matches:

grep -c 't' texts/*

The console output will contain the count of matches found in all specified files:

texts/code.py:3
texts/page.html:5
texts/poem:4

If only one file is specified as the source:

grep -c 't' texts/block

The console output will contain only the number:

4

File Names (-l)

This flag allows you to output only the names of the files in which matches were found:

grep -l 't' texts/*

The console output will be as follows:

texts/code.py
texts/page.html
texts/poem

Limit Output (-m)

Limits the number of lines output to the console terminal to the number specified next to the flag:

grep -m2 't' texts/*

The console output will be:

texts/code.py:from datetime import date
texts/code.py:dateNow = date.today()
texts/page.html:<html>
texts/page.html:                <title>Some Title</title>
texts/poem:Hold fast to dreams
texts/poem:That cannot fly.

As you can see, the limiting number affects not the entire output but the lines of each file.

Exact Match of Whole Line (-x)

Searches for an exact match of the entire line with no variability:

grep -x 'Life is a broken-winged bird' texts/*

The console output will be:

texts/poem:Life is a broken-winged bird

Conclusion

The GREP command in Linux is the most flexible and precise tool for searching expressions in large volumes of text data.

When using the command, you need to specify the following elements:

  • A specific set of options (flags) that configure the search and output mechanisms.
  • One or more regular expressions that describe the search phrase.
  • A list of sources (files and directories) where the search will be performed.

Additionally, the utility is used to filter the output of other commands by redirecting input and output streams.

The core of the GREP command is regular expressions. Unlike a simple string, they allow you to define a phrase with a certain degree of variability, making it match multiple similar entries.

There are two modes of operation for regular expressions:

  • Basic Mode: A limited set of special characters that allow you to formalize expressions only in general terms.
  • Extended Mode: A full set of special characters that allows you to formalize expressions with precision down to each character.

The extended mode provides complete flexibility and accuracy when working with regular expressions.

In rare cases where you only need to find matches for trivial patterns, you can limit yourself to the basic mode.

Linux Linux-System Sistema Linux O sistema Linux
11.02.2025
Reading time: 16 min
Share

Similar

Linux

How to Open a Port on Linux

Opening ports in Linux is an important task that allows certain services or applications to exchange data over the network. Ports act as communication gateways, allowing access to authorized services while blocking unauthorized connections. Managing ports is key to secure access, smooth app functionality, and reliable performance. Understanding Ports and Their Purpose Ports are the logical endpoints of network communication, where devices can send and receive information. HTTP uses port 80, HTTPS uses port 443, and SSH uses port 22. An open port means the service that listens for incoming network traffic is associated with it. A closed port, on the other hand, stops communication via that gateway. Maintaining availability and security requires proper management of Linux open ports. Check Existing Open Ports on Linux Before opening a port, check the open ports in Linux to see which ones are currently active. You may achieve this using several Linux commands. netstat To display open ports, run: netstat -tuln The netstat utility provides a real-time view of active network connections, displaying all listening endpoints. The -tuln flags refine the output to show only TCP and UDP ports without resolving hostnames. Note: In case netstat isn’t installed, install it via: sudo apt install net-tools ss The ss utility can also be utilized to check ports: ss -tuln Compared to netstat, the ss command is more recent and fast. It shows the ports that are in use as well as socket information. nmap For a detailed analysis of Linux open ports, use: nmap localhost The nmap utility scans the given host (localhost in this case) for open ports. This is useful for finding ports exposed to public networks. Note: You can install nmap on Linux via: sudo apt install nmap Opening Ports on Linux Firewall modification is required to grant access through a chosen endpoint. Linux provides several options for handling these tasks, including iptables, ufw, and firewalld. Here are the methods to open ports with these utilities. Method 1: Via iptables Iptables is a robust and lower level firewall utility that grants fine-grained control over network traffic. To open a port with iptables, take these steps: Add a Rule to Allow Traffic from a Specific Port  Enable HTTP access on port 8080 with this command: sudo iptables -A INPUT -p tcp --dport 8080 -j ACCEPT sudo: Execute the command as superuser. iptables: Refers to the firewall utility. -A INPUT: Inserts a rule in the input chain, controlling incoming traffic. -p tcp: Shows that the rule is for TCP traffic. --dport 8080: Points to port 8080 for the rule. ACCEPT: Specifies that incoming traffic matching the rule is accepted. This permits incoming TCP on port 8080. However, iptables changes are volatile and will be undone after reboot. Note: The iptables can be installed with persistent packages using: sudo apt install iptables iptables-persistent Save the Configuration For making the rule permanent and remain even after a system restart, store iptables rules via: sudo netfilter-persistent save This directive preserves current iptables or nftables rules such that they are preserved during reboots. Reload Changes Reload the firewall configuration as needed with: sudo netfilter-persistent reload Method 2: Via UFW Ufw (Uncomplicated Firewall) is a minimal front-end for managing iptables rules. It allows you to easily open ports with simple commands. This is how you can do it: Enable Ufw  First, ensure the ufw firewall is activated: sudo ufw enable Executing this command allows UFW to modify firewall settings. Note: UFW can be installed with: sudo apt install ufw Allow Traffic Via Specific Port  For instance, to open port 22 for SSH, use: sudo ufw allow 22/tcp sudo: Grants superuser privileges. ufw allow: Adds a rule to permit traffic. 22/tcp: Sets port 22 for communication while restricting the rule to TCP protocol. This permits access on port 22, enabling remote SSH connections. Verify the Firewall Status  To ensure the port is accessible and the rule is active, execute: sudo ufw status The status command displays all active rules, including the allowed ports. Method 3: Via Firewalld Firewalld is a dynamic firewall daemon present on Linux. It is simpler to customize the firewall rules compared to using iptables. Here’s how to enable port access via firewalld: Add a Permanent Rule for the Desired Port  To enable HTTPS access on port 443, run: sudo firewall-cmd --permanent --add-port=443/tcp firewall-cmd: Invokes the firewalld command. --permanent: Ensures the rule stays active after the firewall reloads or the system boots. --add-port=443/tcp: Opens port 443 to accept incoming TCP traffic. Note: Install firewalld on Linux via: sudo apt install firewalld Once installed, you should activate and run it: sudo systemctl enable firewalld sudo systemctl start firewalld Reload the Firewall  Finalize the settings to enable the newly defined policy: sudo firewall-cmd --reload Applying firewall modifications makes recent policy updates functional without rebooting. Verification Check whether the port is opened successfully: sudo firewall-cmd --list-all The --list-all command provides a complete list of rules, helping you determine if port 443 is open. Testing the Newly Opened Port Always check if the newly opened port is available for incoming connections. Here’s how: Using telnet Test the port opening via: telnet localhost port_number Successful access means the port is open and responsive. Using nmap Analyze the host to verify if the specified endpoint is accessible.: nmap -p port_number localhost The -p flag specifies the port to scan. Using curl Check HTTP service availability: curl localhost:port_number A successful response confirms the service is running on the opened port. Troubleshooting Common Issues Ports opening may occasionally fail due to configuration errors or conflicting software settings. Follow these tips: Verify Firewall Rules: Run iptables -L or ufw status to assess firewall restrictions and permissions. Check Service Status: Check if the assigned service is active with systemctl status <service-name>. Opening Specific Ports Based on Protocol Understanding the protocol used by the service can help configure ports more effectively. For instance, web traffic typically uses TCP (Transmission Control Protocol) for stable communication, while certain gaming services may require UDP (User Datagram Protocol) for faster packet transmission. Opening a TCP Port To access port 3306 for MySQL traffic: sudo ufw allow 3306/tcp This explicitly permits TCP traffic through port 3306, ensuring stable communication for database queries. Opening a UDP Port To access port 161 for SNMP (Simple Network Management Protocol), run: sudo ufw allow 161/udp UDP provides faster, connectionless communication, ideal for monitoring tools like SNMP. Managing Port Accessibility Once a port is opened, controlling its visibility ensures security and prevents unauthorized access. Restricting Access to Specific IPs To limit port access to a specific IP address (e.g., 192.168.1.100): sudo ufw allow from 192.168.1.100 to any port 22 This allows SSH access via port 22 only from the specified IP address, enhancing security. Closing Ports To revoke access to port 80: sudo ufw deny 80/tcp This denies incoming traffic on port 80, effectively closing it for HTTP services. Conclusion Confirming open ports in Linux is a key step for optimizing network functionality and deploying services effectively. With the use of utilities such as iptables, ufw, or firewalld, you can control traffic securely for your apps. You need to test and debug in order to confirm the port is open and working as expected. From web servers to SSH access, to other network services, port management skills ensure smooth operations and better security.
01 July 2025 · 7 min to read
Linux

NATS Installation, Configuration, and Usage Guide

NATS is a simple, fast, and lightweight message broker written in the Go programming language. NATS has several data organization features: Key-Value: Data within NATS is stored in "key-value" format, where each key corresponds to a specific value. Subjects: Data within NATS is organized into so-called "Subjects," which are named channels for message transmission. Subjects can be divided into segments with hierarchical structures. Publish/Subscribe (Pub/Sub): Data within NATS is transmitted through a model where "Publishers" send messages to "Subjects," and "Subscribers" can subscribe to these "Subjects" to receive messages. Unlike many other message brokers (such as Apache Kafka or RabbitMQ), NATS has several significant advantages: Simplicity and Performance: Messages are transmitted through a simple and fast Pub/Sub protocol. When a message is sent to a subject, all subscribers immediately receive it. This minimizes delays and other overhead costs. Stateless: Information about the state of messages transmitted through the broker is not stored within it, nor is data about subject subscribers. The absence of complex state synchronization allows NATS to scale easily. No Default Queues: In standard configuration, NATS does not form message queues. This is important in cases where data timeliness is more important than persistence. It also eliminates queue management overhead. Reliable Protocol: Messages within the broker are transmitted using the "at-most-once delivery" method. This means a subscriber either receives a message once or not at all. This increases communication reliability and prevents duplicate responses to forwarded messages. Thus, NATS enables building fast and reliable communication between multiple different services. In this guide, we will thoroughly examine how to install, configure, and correctly use NATS in projects running on Ubuntu 22.04. Downloading NATS Package Updates Before installation, it's recommended to update the list of available repositories in the system: sudo apt update Downloading the Archive Next, you need to manually download the ZIP archive with NATS from its official GitHub repository: wget https://github.com/nats-io/nats-server/releases/download/v2.10.22/nats-server-v2.10.22-linux-amd64.zip After the download is complete, you can check the file list: ls Among them will be the NATS archive: nats-server-v2.10.22-linux-amd64.zip  resize.log  snap Extracting the Archive Next, install the package that performs ZIP archive extraction: sudo apt install unzip -y The -y flag is added so that the installer automatically answers 'yes' to all questions. Now extract the NATS archive using the installed extractor: unzip nats-server-v2.10.22-linux-amd64.zip Check the file list: ls As you can see, a new folder with the archive contents has appeared: nats-server-v2.10.22-linux-amd64  nats-server-v2.10.22-linux-amd64.zip  resize.log  snap We no longer need the archive, so delete it: rm nats-server-v2.10.22-linux-amd64.zip Installing NATS Server Installation Let's look at the contents of the created folder: ls nats-server-v2.10.22-linux-amd64 Inside it is the main directory with the NATS server: LICENSE  nats-server  README.md This is what we need to copy to the system catalog with binary files: sudo mv nats-server-v2.10.22-linux-amd64/nats-server /usr/local/bin/ After copying, you need to set the appropriate access permissions: sudo chmod +x /usr/local/bin/nats-server The folder with NATS contents, like the archive, can now also be deleted: rm nats-server-v2.10.22-linux-amd64 -R Server Verification Let's verify that the NATS server is installed by requesting its version: nats-server -v A similar output should appear in the console terminal: nats-server: v2.10.22 However, this command doesn't start the server; it only returns its version. You can start the server as follows: nats-server [3704] 2024/11/07 02:59:53.908362 [INF] Starting nats-server [3704] 2024/11/07 02:59:53.908623 [INF] Version: 2.10.22 [3704] 2024/11/07 02:59:53.908669 [INF] Git: [240e9a4] [3704] 2024/11/07 02:59:53.908701 [INF] Name: NC253DIPURNIY4HUXYQYC5LLAFA6UZEBKUIWTBLLPSMICFH3E2FMSXB7 [3704] 2024/11/07 02:59:53.908725 [INF] ID: NC253DIPURNIY4HUXYQYC5LLAFA6UZEBKUIWTBLLPSMICFH3E2FMSXB7 [3704] 2024/11/07 02:59:53.909430 [INF] Listening for client connections on 0.0.0.0:4222 [3704] 2024/11/07 02:59:53.909679 [INF] Server is ready In this case, the server starts with binding to the console terminal, not as a background service. Therefore, to return to command input mode, you need to press Ctrl + C. NATS Configuration Creating a Configuration File After the broker server is started, you can create a separate directory for the NATS configuration file: mkdir /etc/nats And then create the configuration file itself: sudo nano /etc/nats/nats-server.conf Its contents will be as follows: cluster { name: "test-nats" } store_dir: "/var/lib/nats" listen: "0.0.0.0:4222" Specifically in this configuration, the most basic parameters are set: name: Server name within the NATS cluster store_dir: Path to the directory where working data will be stored listen: IP address and port that the NATS server will occupy Creating a Separate User For all directories related to NATS, you need to create a separate user: useradd -r -c 'NATS service' nats Now create the directories specified in the configuration file: mkdir /var/log/nats /var/lib/nats For each directory, assign appropriate access permissions to the previously created user: chown nats:nats /var/log/nats /var/lib/nats Creating a Background Service Earlier we started the NATS server with binding to the console terminal. In this case, when exiting the console, the server will stop working. To prevent this, you need to create a file for the systemd service: sudo nano /etc/systemd/system/nats-server.service Its contents will be: [Unit] Description=NATS message broker server After=syslog.target network.target [Service] Type=simple ExecStart=/usr/local/bin/nats-server -c /etc/nats/nats-server.conf User=nats Group=nats LimitNOFILE=65536 ExecReload=/bin/kill -HUP $MAINPID Restart=on-failure [Install] WantedBy=multi-user.target This file contains several key parameters: Description: Short description of the service ExecStart: NATS server startup command with the configuration file explicitly specified User: Name of the user created for NATS Now we need to set up the service to start up at boot:  systemctl enable nats-server --now The --now flag immediately starts the specified service. The corresponding message will appear in the console: Created symlink /etc/systemd/system/multi-user.target.wants/nats-server.service → /etc/systemd/system/nats-server.service. Now check the status of the running service: systemctl status nats-server If the NATS server service started successfully, the corresponding message will be among the console output: ... Active: active (running) ... Connecting to NATS You can connect to the NATS server through the console terminal and thus perform message broker testing. For example, publish messages or subscribe to subjects. Client Installation To manage the NATS server, you need to install the natscli client. You can download it from the official GitHub repository: wget https://github.com/nats-io/natscli/releases/download/v0.1.5/nats-0.1.5-amd64.deb After this, the downloaded archive can be extracted and installed: dpkg -i nats-0.1.5-amd64.deb The archive itself can be deleted as it's no longer needed: rm nats-0.1.5-amd64.deb Sending Messages Now you can send a message to the message broker: nats pub -s 127.0.0.1 "someSubject" "Some message" In this command, we send the message "Some message" to the subject "someSubject" to the message broker running on IP address 127.0.0.1 and located on the standard NATS port - 4222. After this, information about the sent data will appear in the console terminal: 10:59:51 Published 12 bytes to "someSubject" Reading Messages Currently, no one will see this message since there's no agent subscribed to the specified subject. We can simulate a service subscribed to the subject and reading messages using another SSH session. To do this, you need to open another console terminal, connect to the remote machine, and subscribe to the previously specified subject: nats sub -s 127.0.0.1 "someSubject" A message about successful subscription will appear in the terminal: 11:11:10 Subscribing on someSubject Now repeat sending the message from the first terminal: nats pub -s 127.0.0.1 "someSubject" "Some message" Information about the new message will appear in the second terminal: [#1] Received on "someSubject" Some message Let's send another message from the first terminal: nats pub -s 127.0.0.1 "someSubject" "Some message again" The corresponding notification will appear in the second terminal: [#2] Received on "someSubject" Some message again Note that the console output of received messages has numbering in square brackets. Go Program + NATS Let's create a small program in the Golang programming language using the NATS message broker. Installing Go First, you need to ensure that the Go compiler is installed in the system: go version If the following message appears in the console terminal, then Go is not yet installed: Command 'go' not found, but can be installed with: snap install go # version 1.23.2, or apt install golang-go # version 2:1.18~0ubuntu2 apt install gccgo-go # version 2:1.18~0ubuntu2 See 'snap info go' for additional versions. In this case, you need to download it as an archive from the official website: wget https://go.dev/dl/go1.23.3.linux-amd64.tar.gz -O go.tar.gz And then extracted: sudo tar -xzvf go.tar.gz -C /usr/local As we no longer need the downloaded archive, we can delete it: rm go.tar.gz Next, you need to add the Go compiler to the PATH variable so it can be called from the console terminal: echo export PATH=$HOME/go/bin:/usr/local/go/bin:$PATH >> ~/.profile Then apply the changes: source ~/.profile Verify that Go is installed successfully by requesting its version: go version You will see a similar output: go version go1.23.3 linux/amd64 Creating a Project Let's create a separate folder for the Golang program: mkdir nats_go Then navigate to it: cd nats_go And initialize the Go project: go mod init nats_go Installing the Module After project initialization, you need to install the NATS client from the official GitHub repository. You don't need to download anything manually; it's enough to use the built-in Golang function: go get github.com/nats-io/nats.go/ Writing Code Now you can create a file with the program code: nano nats_go.go Its contents will be: package main import ( "fmt" // module for working with console "os" // module for working with system functions "time" // module for working with time "github.com/nats-io/nats.go" // module for working with NATS server ) func main() { // get NATS server address from environment variable url := os.Getenv("NATS_URL") // if there's no address in environment variable, use default address if url == "" { url = nats.DefaultURL } // connect to NATS server nc, _ := nats.Connect(url) // defer message broker cleanup until main() function completion defer nc.Drain() // send message to subject without subscribers to ensure it disappears nc.Publish("people.philosophers", []byte("Hello, Socrates!")) // subscribe to all sub-subjects in "people" subject sub, _ := nc.SubscribeSync("people.*") // extract message msg, _ := sub.NextMsg(10 * time.Millisecond) // output message status (it's not there because it was sent before subscribing to subjects) fmt.Printf("No message? Answer: %v\n", msg == nil) // send message to "philosophers" sub-subject of "people" subject nc.Publish("people.philosophers", []byte("Hello, Socrates!")) // send message to "physicists" sub-subject of "people" subject nc.Publish("people.physicists", []byte("Hello, Feynman!")) // extract message and output to console msg, _ = sub.NextMsg(10 * time.Millisecond) fmt.Printf("Message: %q in subject %q\n", string(msg.Data), msg.Subject) // extract message and output to console msg, _ = sub.NextMsg(10 * time.Millisecond) fmt.Printf("Message: %q in subject %q\n", string(msg.Data), msg.Subject) // send message to "biologists" sub-subject of "people" subject nc.Publish("people.biologists", []byte("Hello, Darwin!")) // extract message and output to console msg, _ = sub.NextMsg(10 * time.Millisecond) fmt.Printf("Message: %q in subject %q\n", string(msg.Data), msg.Subject) } Now you can run the created program: go run . The program's output will appear in the console terminal: No message? Answer: true Message: "Hello, Socrates!" in subject "people.philosophers" Message: "Hello, Feynman!" in subject "people.physicists" Message: "Hello, Darwin!" in subject "people.biologists" Python Program + NATS As another example, let's consider using the NATS message broker in the Python programming language. First, you need to ensure that the Python interpreter is installed in the system by requesting its version: python --version The corresponding message will appear in the console: Python 3.10.12 Note that this guide uses Python version 3.10.12. Installing PIP To download the NATS client for Python, you first need to install the PIP package manager: apt install python3-pip -y The -y flag helps automatically answer positively to all questions during installation. Installing the Client Now you can install the NATS client for Python: pip install nats-py Creating a Project For the Python program, let's create a separate directory: mkdir nats_python And navigate to it: cd nats_python Writing Code Let's create a file with the program code: nano nats_python.py Its contents will be: import os import asyncio # import NATS client import nats from nats.errors import TimeoutError # get environment variable containing NATS server address servers = os.environ.get("NATS_URL", "nats://localhost:4222").split(",") async def main(): # connect to NATS server nc = await nats.connect(servers=servers) # send message to subject without subscribers to ensure it disappears await nc.publish("people.philosophers", "Hello, Socrates!".encode()) # subscribe to all sub-subjects in "people" subject sub = await nc.subscribe("people.*") try: # extract message msg = await sub.next_msg(timeout=0.1) except TimeoutError: pass # send message to "philosophers" sub-subject of "people" subject await nc.publish("people.philosophers", "Hello, Socrates!".encode()) # send message to "physicists" sub-subject of "people" subject await nc.publish("people.physicists", "Hello, Feynman!".encode()) # extract message and output to console msg = await sub.next_msg(timeout=0.1) print(f"{msg.data.decode('utf-8')} in subject {msg.subject}") # extract message and output to console msg = await sub.next_msg(timeout=0.1) print(f"{msg.data.decode('utf-8')} in subject {msg.subject}") # send message to "biologists" sub-subject of "people" subject await nc.publish("people.biologists", "Hello, Darwin!".encode()) # extract message and output to console msg = await sub.next_msg(timeout=0.1) print(f"{msg.data.decode('utf-8')} in subject {msg.subject}") # unsubscribe from subjects await sub.unsubscribe() # clean up message broker await nc.drain() if __name__ == '__main__': asyncio.run(main()) Now you can run the created script: python nats_python.py The result of its operation will be the following output in the console terminal: Hello, Socrates! in subject people.philosophers Hello, Feynman! in subject people.physicists Hello, Darwin! in subject people.biologists As you can notice, the logic of this Python program doesn't differ from the logic of the Go program. The difference is only in the syntactic constructions of the specific programming language. Conclusion This guide examined the use of the NATS message broker in sequential stages: Downloading and installing NATS from the official GitHub repository Minimal NATS server configuration Managing the NATS server through the console terminal client Using NATS in a Golang program Using NATS in a Python program We downloaded all NATS clients used in this guide (for terminal, Go, and Python) from the official NATS repository on GitHub, which hosts modules and libraries for all programming languages supported by NATS. You can find more detailed information about configuring and using NATS in the official documentation. There are also many examples of using NATS in different programming languages.
24 June 2025 · 13 min to read
Linux

Listing and Deleting Iptables Firewall Rules

The iptables application is a firewall essential for securely working with network resources on the Linux platform. While there is extensive material dedicated to configuring iptables, we will focus on a few specific tasks: how to view rule lists, delete unnecessary rules, flush chains, and clear the packet count and byte size counters.  We do not recommend modifying the SSH connection on port 22 unless you are absolutely sure of your actions, as you might accidentally block remote access to your test host. In this guide, we will use a Hostman cloud server running Ubuntu. The setup process will be similar on CentOS and Debian. Before proceeding, make sure you have a user with sudo privileges. Viewing Rules In iptables, you can view the rules set by default or by a previous administrator. Execute the command: sudo iptables -S The result will be displayed like this: -P INPUT DROP -P FORWARD DROP -P OUTPUT ACCEPT -N ICMP -N TCP -N UDP -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -m conntrack --ctstate INVALID -j DROP -A INPUT -p udp -m conntrack --ctstate NEW -j UDP -A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j TCP -A INPUT -p icmp -m conntrack --ctstate NEW -j ICMP -A INPUT -p udp -j REJECT --reject-with icmp-port-unreachable -A INPUT -p tcp -j REJECT --reject-with tcp-reset -A INPUT -j REJECT --reject-with icmp-proto-unreachable -A TCP -p tcp -m tcp --dport 22 -j ACCEPT Viewing a Specific Chain This function is used when you want to exclude a specific chain (e.g., INPUT, OUTPUT, TCP, etc.) from the general output. Specify the chain name after the -S option. Example: sudo iptables -S TCP The result: -N TCP -A TCP -p tcp -m tcp --dport 22 -j ACCEPT View Rules as a Table This method is convenient for comparing different rules. The tabular format is built into the utility and is activated using the -L option. Enter: sudo iptables -L You can also limit the output to a specific chain: sudo iptables -L INPUT Sample output: Chain INPUT (policy DROP) target prot opt source destination ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ACCEPT all -- anywhere anywhere DROP all -- anywhere anywhere ctstate INVALID UDP udp -- anywhere anywhere ctstate NEW TCP tcp -- anywhere anywhere tcp flags:FIN,SYN,RST,ACK/SYN ctstate NEW ICMP icmp -- anywhere anywhere ctstate NEW REJECT udp -- anywhere anywhere reject-with icmp-port-unreachable REJECT tcp -- anywhere anywhere reject-with tcp-reset REJECT all -- anywhere anywhere reject-with icmp-proto-unreachable Explanation: target – action taken when a packet matches the rule (e.g., ACCEPT, DROP, redirect to another chain). prot – protocol used (UDP, TCP, ALL). opt – IP options, if any. source – source IP/subnet (e.g., "anywhere" = from anywhere). destination – destination IP/subnet. The last column (without a header) contains additional rule parameters like port numbers or connection states. Viewing Packet and Byte Counters You can also display the packet and total byte count per rule. This is useful for estimating traffic by rule. Available with -L and -v: sudo iptables -L INPUT -v Sample output: Chain INPUT (policy DROP 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 284K 42M ACCEPT all -- any any anywhere anywhere ctstate RELATED,ESTABLISHED 0 0 ACCEPT all -- lo any anywhere anywhere 0 0 DROP all -- any any anywhere anywhere ctstate INVALID 396 63275 UDP udp -- any any anywhere anywhere ctstate NEW 17067 1005K TCP tcp -- any any anywhere anywhere tcp flags:FIN,SYN,RST,ACK/SYN ctstate NEW 2410 154K ICMP icmp -- any any anywhere anywhere ctstate NEW 396 63275 REJECT udp -- any any anywhere anywhere reject-with icmp-port-unreachable 2916 179K REJECT all -- any any anywhere anywhere reject-with icmp-proto-unreachable 0 0 ACCEPT tcp -- any any anywhere anywhere tcp dpt:ssh ctstate NEW,ESTABLISHED Compare this to previous output and you’ll see two new columns: pkts and bytes. Resetting Packet and Byte Counters You can reset these counters using the -Z option. This happens automatically on reboot, but can also be done manually to test for new traffic: sudo iptables -Z To reset a specific chain: sudo iptables -Z OUTPUT To reset a specific rule in a chain by number: sudo iptables -Z OUTPUT 2 Deleting Rules Deleting by Specification Use -D followed by the full rule specification. View existing rules first. For example, to remove the rule that drops invalid outgoing traffic: sudo iptables -D OUTPUT -m conntrack --ctstate INVALID -j DROP No need to use -A when deleting. Deleting by Rule Number Use --line-numbers to get rule numbers: sudo iptables -L --line-numbers Sample output: Chain INPUT (policy DROP) num target prot opt source destination 1 ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED 2 ACCEPT all -- anywhere anywhere 3 DROP all -- anywhere anywhere ctstate INVALID ... Then delete by number: sudo iptables -D INPUT 3 Flushing Chains Be cautious when flushing chains; you could block your SSH connection. Flush a Single Chain sudo iptables -F INPUT Flush All Chains sudo iptables -F This command allows all traffic (inbound, outbound, forwarded), essentially disabling the firewall. If you run it on a production system, you’ll need to reconfigure the firewall from scratch. Always back up your current rules: iptables-save > iptables_backup.txt Restore them later with: iptables-restore < iptables_backup.txt Before flushing, set the default policy to ACCEPT to avoid losing SSH access: sudo iptables -P INPUT ACCEPT sudo iptables -P FORWARD ACCEPT sudo iptables -P OUTPUT ACCEPT Then flush everything: sudo iptables -t nat -F sudo iptables -t mangle -F sudo iptables -F sudo iptables -X This allows all traffic. If you list rules after this, only the default chains (INPUT, FORWARD, OUTPUT) will be present. Conclusion This tutorial provides practical guidance on how to view, reset, and delete iptables firewall rules and perform similar actions on specific chains. Keep in mind that any changes will be lost upon server reboot unless saved.
23 June 2025 · 6 min to read

Do you have questions,
comments, or concerns?

Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
Email us
Hostman's Support