Sign In
Sign In

Bash Regular Expressions

Bash Regular Expressions
Hostman Team
Technical writer
Linux
01.11.2024
Reading time: 10 min

One of the core principles of Unix systems is the extensive use of text data: configuration files, as well as input and output data in *nix systems, are often organized as plain text. Regular expressions are a powerful tool for manipulating text data. This guide delves into the intricacies of using regular expressions in Bash, helping you fully harness the power of the command line and scripts in Linux.

What Are Regular Expressions?

Regular expressions are specially formatted strings used to search for character patterns in text. They resemble shell wildcards in some ways, but their capabilities are much broader. Many text-processing utilities in Linux and programming languages include a regular expression engine. However, different programs and languages often employ different regular expression dialects. This article focuses on the POSIX standard to which most Linux utilities adhere.

The grep Utrequires at least one match of theility

The grep program is the primary tool for working with regular expressions. grep reads data from standard input, searches for matches to a specified pattern, and outputs all matching lines. grep is typically pre-installed on most distributions.

You can try the commands in a virtual machine or a VPS to practice using regular expressions.

The syntax of grep is as follows:

grep [options] regular_expression [file...]

The simplest use case for grep is finding lines that contain a fixed substring. In the example below, grep outputs all lines that contain the sequence nologin:

grep nologin /etc/passwd

Output:

daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
games:x:5:60:games:/usr/games:/usr/sbin/nologin
...

grep has many options, which are detailed in the documentation. Here are some useful options for working with regular expressions:

  • -v — Inverts the match criteria. With this option, grep outputs lines that do not contain matches:

ls /bin | grep -v zip

# Output:
411toppm 7z 7za 7zr ...
  • -i — Ignores case.

  • -o — Outputs only the matches, not the entire lines:

ls /bin | grep -o zip

# Output:
zip zip zip zip ...
  • -w — Searches for lines containing whole words matching the pattern.

ls /bin | grep -w zip

# Output:
gpg-zip
zip

For comparison, the same command without the -w option also includes lines where the pattern appears as a substring within a word.

ls /bin | grep zip

# Output:
bunzip2 bzip2 bzip2recover funzip

Basic Regular Expressions (BRE)

As previously mentioned, there are multiple dialects of regular expressions. The POSIX standard defines two main types of implementations: Basic Regular Expressions (BRE), which are supported by almost all POSIX-compliant programs, and Extended Regular Expressions (ERE), which allow for more complex patterns but aren't supported by all utilities. We'll start by exploring the features of BRE.

Metacharacters and Literals

We've already encountered simple regular expressions. For example, the expression “zip” represents a string with the following criteria: it must contain at least three characters; it includes the characters “z”, “i”, and “p” in that exact order; and there are no other characters in between. Characters that match themselves (like “z”, “i”, and “p”) are called literals. Another category is metacharacters, which are used to define various search criteria. Metacharacters in BRE include:

^ $ . [ ] * \ -

To use a metacharacter as a literal, you need to escape it with a backslash (\). Note that some metacharacters have special meanings in the shell, so enclose it in quotes when passing a regular expression as a command argument.

Any Character

The dot (.) metacharacter matches any character in that position. For example:

ls /bin | grep '.zip'

Output:

bunzip2
bzip2
bzip2recover
funzip
gpg-zip
gunzip
gzip
mzip
p7zip
pbzip2
preunzip
prezip
prezip-bin
streamzip
unzip
unzipsfx

One important detail: the zip program itself isn’t included in the output because the dot (.) metacharacter increases the required match length to four characters.

Anchors

The caret (^) and dollar sign ($) in regular expressions serve as anchors. This means that, when included, a match can only occur at the start of a line (^) or at the end ($).

ls /bin | grep '^zip'

# Output:
zip zipcloak zipdetails zipgrep …


ls /bin | grep 'zip$'

# Output:
funzip gpg-zip gunzip ...


ls /bin | grep '^zip$'

# Output:
zip

The regular expression ^$ matches empty lines.

Character Sets

Besides matching any character in a given position (.), regular expressions allow for matching a character from a specific set. This is done with square brackets. The following example searches for strings matching bzip or gzip:

ls /bin | grep '[bg]zip'

# Output:
bzip2
bzip2recover
gzip

All metacharacters lose their special meaning within square brackets, except two.

If a caret (^) is placed immediately after the opening bracket, the characters in the set are treated as excluded from that position. For example:

ls /bin | grep '[^bg]zip'

Output:

bunzip2
funzip
gpg-zip
gunzip
mzip
p7zip
preunzip
prezip
prezip-bin
streamzip
unzip
unzipsfx

With negation, we get a list of filenames containing zip but preceded by any character other than b or g. Note that zip is not included here; the negation requires the presence of some character in that position. The caret serves as a negation only if it appears immediately after the opening bracket; otherwise, it loses its special meaning.

Using a hyphen (-), you can specify character ranges. This lets you match a range of characters or even multiple ranges. For instance, to find all filenames that start with a letter or a number:

ls ~ | grep '^[A-Za-z0-9]'

Output:

backup
bin
Books
Desktop
docker
Documents
Downloads
GNS3
...

POSIX Character Classes

When using character ranges, one challenge is that ranges can be interpreted differently based on locale settings. For instance, the range [A-Z] may sometimes be interpreted lexicographically, potentially excluding lowercase a. To address this, the POSIX standard provides several classes that represent various character sets. Some of these classes include:

  • [:alnum:] — Alphanumeric characters; equivalent to [A-Za-z0-9] in ASCII.

  • [:alpha:] — Alphabetic characters; equivalent to [A-Za-z] in ASCII.

  • [:digit:] — Digits from 0 to 9.

  • [:lower:] and [:upper:] — Lowercase and uppercase letters, respectively.

  • [:space:] — Whitespace characters, including space, tab, carriage return, newline, vertical tab, and form feed.

Character classes don’t provide an easy way to express partial ranges, like [A-M]. Here’s an example of using a character class:

ls ~ | grep '[[:upper:]].*'

Output:

Books
Desktop
Documents
Downloads
GNS3
GOG Games
Learning
Music
...

Extended Regular Expressions (ERE)

Most POSIX-compliant applications and those using BRE (such as grep and the stream editor sed) support the features discussed above. The POSIX ERE standard allows for more expressive regular expressions, though not all programs support it. The egrep program traditionally supported the ERE dialect, but the GNU version of grep also supports ERE when run with the -E option.

In ERE, the set of metacharacters is expanded to include:

( ) { } ? + |

Alternation

Alternation allows for a match with one of multiple expressions. Similar to square brackets that allow a character to match one of several characters, alternation allows for matching one of multiple strings or regular expressions. Alternation is represented by the pipe (|):

echo "AAA" | grep -E 'AAA|BBB'

# Output:
AAA
echo "BBB" | grep -E 'AAA|BBB'

# Output:
BBB
echo "CCC" | grep -E 'AAA|BBB'

# Output: (no match)

Grouping

You can group elements of regular expressions and treat them as a single unit using parentheses. The following expression matches filenames starting with bz, gz, or zip. Without the parentheses, the regular expression would change meaning to match filenames starting with bz or containing gz or zip.

ls /bin | grep -E '^(bz|gz|zip)'

Output:

bzcat
bzgrep
bzip2
bzip2recover
bzless
bzmore
gzexe
gzip
zip
zipdetails
zipgrep
zipinfo
zipsplit

Quantifiers

Quantifiers specify the number of times an element can occur. BRE supports several quantifiers:

  • ? — Matches the preceding element zero or one time, meaning the previous element is optional:

echo "tet" | grep -E 'tes?t'

# Output:
tet
echo "test" | grep -E 'tes?t'

# Output:
test
echo "tesst" | grep -E 'tes?t'

# Output: (no match)
  • * — Matches the preceding element zero or more times. Unlike ?, this element can appear any number of times:

echo "tet" | grep -E 'tes*t'

# Output:
tet
echo "test" | grep -E 'tes*t'

# Output:
test
echo "tesst" | grep -E 'tes*t'

# Output:
tesst
  • + — Similar to *, but requires at least one match of the preceding element:

echo "tet" | grep -E 'tes+t'

# Output: (no match)
echo "test" | grep -E 'tes+t'

# Output:
test
echo "tesst" | grep -E 'tes+t'

# Output:
tesst

In BRE, special metacharacters { and } allow you to specify minimum and maximum match counts for the preceding element in four possible ways:

  • {n} — Matches if the preceding element occurs exactly n times.

  • {n,m} — Matches if the preceding element occurs at least n and at most m times.

  • {n,} — Matches if the preceding element occurs n or more times.

  • {,m} — Matches if the preceding element occurs no more than m times.

Example:

echo "tet" | grep -E "tes{1,3}t"

# Output: (no match)
echo "test" | grep -E "tes{1,3}t"

# Output:
test
echo "tesst" | grep -E "tes{1,3}t"

# Output:
tesst
echo "tessst" | grep -E "tes{1,3}t"

# Output:
tessst
echo "tesssst" | grep -E "tes{1,3}t"

# Output: (no match)

Only the lines where s appears one, two, or three times match the pattern.

Regular Expressions in Practice

To conclude, let’s look at a couple of practical examples of how regular expressions can be applied.

Validating Phone Numbers

Suppose we have a list of phone numbers where the correct format is (nnn) nnn-nnnn. Out of a list of 10 numbers, three are incorrectly formatted.

cat phonenumbers.txt

Output:

(185) 136-1035
(95) 213-1874
(37) 207-2639
(285) 227-1602
(275) 298-1043
(107) 204-2197
(799) 240-1839
(218) 750-7390
(114) 776-2276
(7012) 219-3089

The task is to identify the incorrect numbers. We can use the following command:

grep -Ev '^\([0-9]{3}\) [0-9]{3}-[0-9]{4}$' phonenumbers.txt

Output:

(95) 213-1874
(37) 207-2639
(7012) 219-3089

Here, we used the -v option to invert the match and output only lines that do not match the specified format. Since parentheses are considered metacharacters in ERE, we escaped them with backslashes to treat them as literals.

Finding Improper File Names

The find command supports checking paths with regular expressions. It’s important to note that, unlike grep which matches parts of lines, find requires the whole path to match. Suppose we want to identify files and directories containing spaces or potentially problematic characters.

find . -regex '.*[^-_./0-9a-zA-Z].*'

The .* sequences at the beginning and end represent any number of any characters, which is necessary because find expects the entire path to match. Inside the square brackets, we use negation to exclude valid filename characters, meaning any file or directory name containing characters other than hyphens, underscores, digits, or Latin letters will appear in the output.

Conclusion

This article has covered a few practical examples of Bash regular expressions. Creating complex regular expressions might seem challenging at first. But over time, you’ll gain experience and skill in using regular expressions for searches across various applications that support them.

Linux
01.11.2024
Reading time: 10 min

Similar

Linux

NATS Installation, Configuration, and Usage Guide

NATS is a simple, fast, and lightweight message broker written in the Go programming language. NATS has several data organization features: Key-Value: Data within NATS is stored in "key-value" format, where each key corresponds to a specific value. Subjects: Data within NATS is organized into so-called "Subjects," which are named channels for message transmission. Subjects can be divided into segments with hierarchical structures. Publish/Subscribe (Pub/Sub): Data within NATS is transmitted through a model where "Publishers" send messages to "Subjects," and "Subscribers" can subscribe to these "Subjects" to receive messages. Unlike many other message brokers (such as Apache Kafka or RabbitMQ), NATS has several significant advantages: Simplicity and Performance: Messages are transmitted through a simple and fast Pub/Sub protocol. When a message is sent to a subject, all subscribers immediately receive it. This minimizes delays and other overhead costs. Stateless: Information about the state of messages transmitted through the broker is not stored within it, nor is data about subject subscribers. The absence of complex state synchronization allows NATS to scale easily. No Default Queues: In standard configuration, NATS does not form message queues. This is important in cases where data timeliness is more important than persistence. It also eliminates queue management overhead. Reliable Protocol: Messages within the broker are transmitted using the "at-most-once delivery" method. This means a subscriber either receives a message once or not at all. This increases communication reliability and prevents duplicate responses to forwarded messages. Thus, NATS enables building fast and reliable communication between multiple different services. In this guide, we will thoroughly examine how to install, configure, and correctly use NATS in projects running on Ubuntu 22.04. Downloading NATS Package Updates Before installation, it's recommended to update the list of available repositories in the system: sudo apt update Downloading the Archive Next, you need to manually download the ZIP archive with NATS from its official GitHub repository: wget https://github.com/nats-io/nats-server/releases/download/v2.10.22/nats-server-v2.10.22-linux-amd64.zip After the download is complete, you can check the file list: ls Among them will be the NATS archive: nats-server-v2.10.22-linux-amd64.zip  resize.log  snap Extracting the Archive Next, install the package that performs ZIP archive extraction: sudo apt install unzip -y The -y flag is added so that the installer automatically answers 'yes' to all questions. Now extract the NATS archive using the installed extractor: unzip nats-server-v2.10.22-linux-amd64.zip Check the file list: ls As you can see, a new folder with the archive contents has appeared: nats-server-v2.10.22-linux-amd64  nats-server-v2.10.22-linux-amd64.zip  resize.log  snap We no longer need the archive, so delete it: rm nats-server-v2.10.22-linux-amd64.zip Installing NATS Server Installation Let's look at the contents of the created folder: ls nats-server-v2.10.22-linux-amd64 Inside it is the main directory with the NATS server: LICENSE  nats-server  README.md This is what we need to copy to the system catalog with binary files: sudo mv nats-server-v2.10.22-linux-amd64/nats-server /usr/local/bin/ After copying, you need to set the appropriate access permissions: sudo chmod +x /usr/local/bin/nats-server The folder with NATS contents, like the archive, can now also be deleted: rm nats-server-v2.10.22-linux-amd64 -R Server Verification Let's verify that the NATS server is installed by requesting its version: nats-server -v A similar output should appear in the console terminal: nats-server: v2.10.22 However, this command doesn't start the server; it only returns its version. You can start the server as follows: nats-server [3704] 2024/11/07 02:59:53.908362 [INF] Starting nats-server [3704] 2024/11/07 02:59:53.908623 [INF] Version: 2.10.22 [3704] 2024/11/07 02:59:53.908669 [INF] Git: [240e9a4] [3704] 2024/11/07 02:59:53.908701 [INF] Name: NC253DIPURNIY4HUXYQYC5LLAFA6UZEBKUIWTBLLPSMICFH3E2FMSXB7 [3704] 2024/11/07 02:59:53.908725 [INF] ID: NC253DIPURNIY4HUXYQYC5LLAFA6UZEBKUIWTBLLPSMICFH3E2FMSXB7 [3704] 2024/11/07 02:59:53.909430 [INF] Listening for client connections on 0.0.0.0:4222 [3704] 2024/11/07 02:59:53.909679 [INF] Server is ready In this case, the server starts with binding to the console terminal, not as a background service. Therefore, to return to command input mode, you need to press Ctrl + C. NATS Configuration Creating a Configuration File After the broker server is started, you can create a separate directory for the NATS configuration file: mkdir /etc/nats And then create the configuration file itself: sudo nano /etc/nats/nats-server.conf Its contents will be as follows: cluster { name: "test-nats" } store_dir: "/var/lib/nats" listen: "0.0.0.0:4222" Specifically in this configuration, the most basic parameters are set: name: Server name within the NATS cluster store_dir: Path to the directory where working data will be stored listen: IP address and port that the NATS server will occupy Creating a Separate User For all directories related to NATS, you need to create a separate user: useradd -r -c 'NATS service' nats Now create the directories specified in the configuration file: mkdir /var/log/nats /var/lib/nats For each directory, assign appropriate access permissions to the previously created user: chown nats:nats /var/log/nats /var/lib/nats Creating a Background Service Earlier we started the NATS server with binding to the console terminal. In this case, when exiting the console, the server will stop working. To prevent this, you need to create a file for the systemd service: sudo nano /etc/systemd/system/nats-server.service Its contents will be: [Unit] Description=NATS message broker server After=syslog.target network.target [Service] Type=simple ExecStart=/usr/local/bin/nats-server -c /etc/nats/nats-server.conf User=nats Group=nats LimitNOFILE=65536 ExecReload=/bin/kill -HUP $MAINPID Restart=on-failure [Install] WantedBy=multi-user.target This file contains several key parameters: Description: Short description of the service ExecStart: NATS server startup command with the configuration file explicitly specified User: Name of the user created for NATS Now we need to set up the service to start up at boot:  systemctl enable nats-server --now The --now flag immediately starts the specified service. The corresponding message will appear in the console: Created symlink /etc/systemd/system/multi-user.target.wants/nats-server.service → /etc/systemd/system/nats-server.service. Now check the status of the running service: systemctl status nats-server If the NATS server service started successfully, the corresponding message will be among the console output: ... Active: active (running) ... Connecting to NATS You can connect to the NATS server through the console terminal and thus perform message broker testing. For example, publish messages or subscribe to subjects. Client Installation To manage the NATS server, you need to install the natscli client. You can download it from the official GitHub repository: wget https://github.com/nats-io/natscli/releases/download/v0.1.5/nats-0.1.5-amd64.deb After this, the downloaded archive can be extracted and installed: dpkg -i nats-0.1.5-amd64.deb The archive itself can be deleted as it's no longer needed: rm nats-0.1.5-amd64.deb Sending Messages Now you can send a message to the message broker: nats pub -s 127.0.0.1 "someSubject" "Some message" In this command, we send the message "Some message" to the subject "someSubject" to the message broker running on IP address 127.0.0.1 and located on the standard NATS port - 4222. After this, information about the sent data will appear in the console terminal: 10:59:51 Published 12 bytes to "someSubject" Reading Messages Currently, no one will see this message since there's no agent subscribed to the specified subject. We can simulate a service subscribed to the subject and reading messages using another SSH session. To do this, you need to open another console terminal, connect to the remote machine, and subscribe to the previously specified subject: nats sub -s 127.0.0.1 "someSubject" A message about successful subscription will appear in the terminal: 11:11:10 Subscribing on someSubject Now repeat sending the message from the first terminal: nats pub -s 127.0.0.1 "someSubject" "Some message" Information about the new message will appear in the second terminal: [#1] Received on "someSubject" Some message Let's send another message from the first terminal: nats pub -s 127.0.0.1 "someSubject" "Some message again" The corresponding notification will appear in the second terminal: [#2] Received on "someSubject" Some message again Note that the console output of received messages has numbering in square brackets. Go Program + NATS Let's create a small program in the Golang programming language using the NATS message broker. Installing Go First, you need to ensure that the Go compiler is installed in the system: go version If the following message appears in the console terminal, then Go is not yet installed: Command 'go' not found, but can be installed with: snap install go # version 1.23.2, or apt install golang-go # version 2:1.18~0ubuntu2 apt install gccgo-go # version 2:1.18~0ubuntu2 See 'snap info go' for additional versions. In this case, you need to download it as an archive from the official website: wget https://go.dev/dl/go1.23.3.linux-amd64.tar.gz -O go.tar.gz And then extracted: sudo tar -xzvf go.tar.gz -C /usr/local As we no longer need the downloaded archive, we can delete it: rm go.tar.gz Next, you need to add the Go compiler to the PATH variable so it can be called from the console terminal: echo export PATH=$HOME/go/bin:/usr/local/go/bin:$PATH >> ~/.profile Then apply the changes: source ~/.profile Verify that Go is installed successfully by requesting its version: go version You will see a similar output: go version go1.23.3 linux/amd64 Creating a Project Let's create a separate folder for the Golang program: mkdir nats_go Then navigate to it: cd nats_go And initialize the Go project: go mod init nats_go Installing the Module After project initialization, you need to install the NATS client from the official GitHub repository. You don't need to download anything manually; it's enough to use the built-in Golang function: go get github.com/nats-io/nats.go/ Writing Code Now you can create a file with the program code: nano nats_go.go Its contents will be: package main import ( "fmt" // module for working with console "os" // module for working with system functions "time" // module for working with time "github.com/nats-io/nats.go" // module for working with NATS server ) func main() { // get NATS server address from environment variable url := os.Getenv("NATS_URL") // if there's no address in environment variable, use default address if url == "" { url = nats.DefaultURL } // connect to NATS server nc, _ := nats.Connect(url) // defer message broker cleanup until main() function completion defer nc.Drain() // send message to subject without subscribers to ensure it disappears nc.Publish("people.philosophers", []byte("Hello, Socrates!")) // subscribe to all sub-subjects in "people" subject sub, _ := nc.SubscribeSync("people.*") // extract message msg, _ := sub.NextMsg(10 * time.Millisecond) // output message status (it's not there because it was sent before subscribing to subjects) fmt.Printf("No message? Answer: %v\n", msg == nil) // send message to "philosophers" sub-subject of "people" subject nc.Publish("people.philosophers", []byte("Hello, Socrates!")) // send message to "physicists" sub-subject of "people" subject nc.Publish("people.physicists", []byte("Hello, Feynman!")) // extract message and output to console msg, _ = sub.NextMsg(10 * time.Millisecond) fmt.Printf("Message: %q in subject %q\n", string(msg.Data), msg.Subject) // extract message and output to console msg, _ = sub.NextMsg(10 * time.Millisecond) fmt.Printf("Message: %q in subject %q\n", string(msg.Data), msg.Subject) // send message to "biologists" sub-subject of "people" subject nc.Publish("people.biologists", []byte("Hello, Darwin!")) // extract message and output to console msg, _ = sub.NextMsg(10 * time.Millisecond) fmt.Printf("Message: %q in subject %q\n", string(msg.Data), msg.Subject) } Now you can run the created program: go run . The program's output will appear in the console terminal: No message? Answer: true Message: "Hello, Socrates!" in subject "people.philosophers" Message: "Hello, Feynman!" in subject "people.physicists" Message: "Hello, Darwin!" in subject "people.biologists" Python Program + NATS As another example, let's consider using the NATS message broker in the Python programming language. First, you need to ensure that the Python interpreter is installed in the system by requesting its version: python --version The corresponding message will appear in the console: Python 3.10.12 Note that this guide uses Python version 3.10.12. Installing PIP To download the NATS client for Python, you first need to install the PIP package manager: apt install python3-pip -y The -y flag helps automatically answer positively to all questions during installation. Installing the Client Now you can install the NATS client for Python: pip install nats-py Creating a Project For the Python program, let's create a separate directory: mkdir nats_python And navigate to it: cd nats_python Writing Code Let's create a file with the program code: nano nats_python.py Its contents will be: import os import asyncio # import NATS client import nats from nats.errors import TimeoutError # get environment variable containing NATS server address servers = os.environ.get("NATS_URL", "nats://localhost:4222").split(",") async def main(): # connect to NATS server nc = await nats.connect(servers=servers) # send message to subject without subscribers to ensure it disappears await nc.publish("people.philosophers", "Hello, Socrates!".encode()) # subscribe to all sub-subjects in "people" subject sub = await nc.subscribe("people.*") try: # extract message msg = await sub.next_msg(timeout=0.1) except TimeoutError: pass # send message to "philosophers" sub-subject of "people" subject await nc.publish("people.philosophers", "Hello, Socrates!".encode()) # send message to "physicists" sub-subject of "people" subject await nc.publish("people.physicists", "Hello, Feynman!".encode()) # extract message and output to console msg = await sub.next_msg(timeout=0.1) print(f"{msg.data.decode('utf-8')} in subject {msg.subject}") # extract message and output to console msg = await sub.next_msg(timeout=0.1) print(f"{msg.data.decode('utf-8')} in subject {msg.subject}") # send message to "biologists" sub-subject of "people" subject await nc.publish("people.biologists", "Hello, Darwin!".encode()) # extract message and output to console msg = await sub.next_msg(timeout=0.1) print(f"{msg.data.decode('utf-8')} in subject {msg.subject}") # unsubscribe from subjects await sub.unsubscribe() # clean up message broker await nc.drain() if __name__ == '__main__': asyncio.run(main()) Now you can run the created script: python nats_python.py The result of its operation will be the following output in the console terminal: Hello, Socrates! in subject people.philosophers Hello, Feynman! in subject people.physicists Hello, Darwin! in subject people.biologists As you can notice, the logic of this Python program doesn't differ from the logic of the Go program. The difference is only in the syntactic constructions of the specific programming language. Conclusion This guide examined the use of the NATS message broker in sequential stages: Downloading and installing NATS from the official GitHub repository Minimal NATS server configuration Managing the NATS server through the console terminal client Using NATS in a Golang program Using NATS in a Python program We downloaded all NATS clients used in this guide (for terminal, Go, and Python) from the official NATS repository on GitHub, which hosts modules and libraries for all programming languages supported by NATS. You can find more detailed information about configuring and using NATS in the official documentation. There are also many examples of using NATS in different programming languages.
24 June 2025 · 13 min to read
Linux

Listing and Deleting Iptables Firewall Rules

The iptables application is a firewall essential for securely working with network resources on the Linux platform. While there is extensive material dedicated to configuring iptables, we will focus on a few specific tasks: how to view rule lists, delete unnecessary rules, flush chains, and clear the packet count and byte size counters.  We do not recommend modifying the SSH connection on port 22 unless you are absolutely sure of your actions, as you might accidentally block remote access to your test host. In this guide, we will use a Hostman cloud server running Ubuntu. The setup process will be similar on CentOS and Debian. Before proceeding, make sure you have a user with sudo privileges. Viewing Rules In iptables, you can view the rules set by default or by a previous administrator. Execute the command: sudo iptables -S The result will be displayed like this: -P INPUT DROP -P FORWARD DROP -P OUTPUT ACCEPT -N ICMP -N TCP -N UDP -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -m conntrack --ctstate INVALID -j DROP -A INPUT -p udp -m conntrack --ctstate NEW -j UDP -A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j TCP -A INPUT -p icmp -m conntrack --ctstate NEW -j ICMP -A INPUT -p udp -j REJECT --reject-with icmp-port-unreachable -A INPUT -p tcp -j REJECT --reject-with tcp-reset -A INPUT -j REJECT --reject-with icmp-proto-unreachable -A TCP -p tcp -m tcp --dport 22 -j ACCEPT Viewing a Specific Chain This function is used when you want to exclude a specific chain (e.g., INPUT, OUTPUT, TCP, etc.) from the general output. Specify the chain name after the -S option. Example: sudo iptables -S TCP The result: -N TCP -A TCP -p tcp -m tcp --dport 22 -j ACCEPT View Rules as a Table This method is convenient for comparing different rules. The tabular format is built into the utility and is activated using the -L option. Enter: sudo iptables -L You can also limit the output to a specific chain: sudo iptables -L INPUT Sample output: Chain INPUT (policy DROP) target prot opt source destination ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ACCEPT all -- anywhere anywhere DROP all -- anywhere anywhere ctstate INVALID UDP udp -- anywhere anywhere ctstate NEW TCP tcp -- anywhere anywhere tcp flags:FIN,SYN,RST,ACK/SYN ctstate NEW ICMP icmp -- anywhere anywhere ctstate NEW REJECT udp -- anywhere anywhere reject-with icmp-port-unreachable REJECT tcp -- anywhere anywhere reject-with tcp-reset REJECT all -- anywhere anywhere reject-with icmp-proto-unreachable Explanation: target – action taken when a packet matches the rule (e.g., ACCEPT, DROP, redirect to another chain). prot – protocol used (UDP, TCP, ALL). opt – IP options, if any. source – source IP/subnet (e.g., "anywhere" = from anywhere). destination – destination IP/subnet. The last column (without a header) contains additional rule parameters like port numbers or connection states. Viewing Packet and Byte Counters You can also display the packet and total byte count per rule. This is useful for estimating traffic by rule. Available with -L and -v: sudo iptables -L INPUT -v Sample output: Chain INPUT (policy DROP 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 284K 42M ACCEPT all -- any any anywhere anywhere ctstate RELATED,ESTABLISHED 0 0 ACCEPT all -- lo any anywhere anywhere 0 0 DROP all -- any any anywhere anywhere ctstate INVALID 396 63275 UDP udp -- any any anywhere anywhere ctstate NEW 17067 1005K TCP tcp -- any any anywhere anywhere tcp flags:FIN,SYN,RST,ACK/SYN ctstate NEW 2410 154K ICMP icmp -- any any anywhere anywhere ctstate NEW 396 63275 REJECT udp -- any any anywhere anywhere reject-with icmp-port-unreachable 2916 179K REJECT all -- any any anywhere anywhere reject-with icmp-proto-unreachable 0 0 ACCEPT tcp -- any any anywhere anywhere tcp dpt:ssh ctstate NEW,ESTABLISHED Compare this to previous output and you’ll see two new columns: pkts and bytes. Resetting Packet and Byte Counters You can reset these counters using the -Z option. This happens automatically on reboot, but can also be done manually to test for new traffic: sudo iptables -Z To reset a specific chain: sudo iptables -Z OUTPUT To reset a specific rule in a chain by number: sudo iptables -Z OUTPUT 2 Deleting Rules Deleting by Specification Use -D followed by the full rule specification. View existing rules first. For example, to remove the rule that drops invalid outgoing traffic: sudo iptables -D OUTPUT -m conntrack --ctstate INVALID -j DROP No need to use -A when deleting. Deleting by Rule Number Use --line-numbers to get rule numbers: sudo iptables -L --line-numbers Sample output: Chain INPUT (policy DROP) num target prot opt source destination 1 ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED 2 ACCEPT all -- anywhere anywhere 3 DROP all -- anywhere anywhere ctstate INVALID ... Then delete by number: sudo iptables -D INPUT 3 Flushing Chains Be cautious when flushing chains; you could block your SSH connection. Flush a Single Chain sudo iptables -F INPUT Flush All Chains sudo iptables -F This command allows all traffic (inbound, outbound, forwarded), essentially disabling the firewall. If you run it on a production system, you’ll need to reconfigure the firewall from scratch. Always back up your current rules: iptables-save > iptables_backup.txt Restore them later with: iptables-restore < iptables_backup.txt Before flushing, set the default policy to ACCEPT to avoid losing SSH access: sudo iptables -P INPUT ACCEPT sudo iptables -P FORWARD ACCEPT sudo iptables -P OUTPUT ACCEPT Then flush everything: sudo iptables -t nat -F sudo iptables -t mangle -F sudo iptables -F sudo iptables -X This allows all traffic. If you list rules after this, only the default chains (INPUT, FORWARD, OUTPUT) will be present. Conclusion This tutorial provides practical guidance on how to view, reset, and delete iptables firewall rules and perform similar actions on specific chains. Keep in mind that any changes will be lost upon server reboot unless saved.
23 June 2025 · 6 min to read
Linux

How to Mount an SMB Share in Linux

The Server Message Block (SMB) protocol facilitates network file sharing, allowing applications to read and write to files and request services from server programs. This protocol is pivotal for seamless communication between different devices in a network, particularly in mixed OS environments like Windows and Linux. Mounting an SMB share in Linux enables users to access files on a Windows server or another SMB-enabled device directly from their Linux system. This tutorial will guide you through the process of mounting an SMB share on Linux, ensuring smooth file sharing and network communication. Prerequisites for Mounting SMB Shares Before mounting an SMB share, ensure the following prerequisites are met: A Linux system, such as a Hostman cheap cloud server, with root or sudo privileges. The cifs-utils package installed on your Linux system. Access credentials (username and password) for the SMB share. Network connectivity between your Linux system and the SMB server. Installing Necessary Packages The cifs-utils package is essential for mounting SMB shares on Linux. Additionally, the psmisc package provides the fuser command, which helps manage and monitor file usage. Update Package List and Upgrade System First, update your package list and upgrade your system: sudo apt update Install cifs-utils and psmisc Install the necessary packages: sudo apt install cifs-utils psmisc Verify Installation Verify the installation of cifs-utils and availability of the fuser command: mount -t cifsfuser Finding SMB Share Details Identify the SMB share details, including the server name or IP address and the share name. You might need to consult your network administrator or check the server configuration. Example: Server: smbserver.example.com Share: sharedfolder Mounting SMB Shares Using the mount Command To mount the SMB share, use the mount command with the -t cifs option, specifying the SMB protocol. Create a directory to serve as the mount point: sudo mkdir /mnt/smb_share Mount the SMB share using the following command: sudo mount -t cifs -o username=your_username,password=your_password //192.0.2.17/SharedFiles /mnt/smb_share Replace your_username and your_password with your actual username and password. Ensure /mnt/smb_share is an existing directory. Verifying the Mount To confirm that the SMB share is successfully mounted, use the mount command: mount -t cifs Navigate to the mount point and list the files: cd /mnt/smb_sharels Creating a Credentials File To avoid entering credentials each time, create a credentials file. This file should be hidden and secured. Use a text editor to create the file: nano ~/.smbcredentials Add the following content, replacing with your actual credentials: username=your_usernamepassword=your_password Set appropriate permissions for the file: sudo chown your_username: ~/.smbcredentialssudo chmod 600 ~/.smbcredentials Mount Using the Credentials File Mount the SMB share using the credentials file: sudo mount -t cifs -o credentials=~/.smbcredentials //192.168.2.12/SharedFiles /mnt/smb_share Automating SMB Share Mounts To automate the mounting process, add an entry to the /etc/fstab file. This will ensure the SMB share is mounted at boot. 1. Open /etc/fstab for editing: sudo nano /etc/fstab 2. Add the following line: //smbserver.example.com/sharedfolder /mnt/smbshare cifs username=johndoe,password=securepassword,iocharset=utf8,sec=ntlm 0 0 3. Save and close the file. 4. Test the fstab entry: sudo mount -a Ensure no errors are displayed. Troubleshooting Common Issues Permission Denied Check your credentials and permissions on the SMB server. No Such File or Directory Ensure the server IP, share path, and mount point are correct. Mount Error 13 = Permission Denied Double-check your username and password. Mount Error 112 = Host is Down Verify network connectivity and server availability. Unmounting an SMB Share To unmount the SMB share, use the umount command followed by the mount point: sudo umount /mnt/smb_share Conclusion Mounting an SMB share in Linux is a straightforward process that enhances file sharing capabilities across different operating systems. By following this tutorial, you can efficiently set up and troubleshoot SMB share mounts, facilitating seamless network communication and file access. Don't forget to check how to configure server image on Lunix! Frequently Asked Questions What is Samba in Linux and how does it relate to SMB? Samba is an open-source implementation of the SMB/CIFS protocol in Linux. It allows Linux systems to share files and printers with Windows devices over a network. What is the command to mount a Windows share in Linux? Use mount -t cifs //server/share /mnt/share -o username=your_user. How can I auto-mount an SMB share on boot in Linux? Add the mount configuration to /etc/fstab using proper credentials. Do I need root access to mount an SMB share? For traditional mounting, yes. But user-space tools like gio mount can be used without root.
16 June 2025 · 4 min to read

Do you have questions,
comments, or concerns?

Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
Email us
Hostman's Support