How to Use Grep and Regular Expressions in Linux
Linux
11.02.2025
Reading time: 16 min
Share
GREP (short for "global regular expression print") is one of the most popular utilities in the Linux operating system.
With it, you can search for phrases (sequences of characters) in multiple files simultaneously using regular expressions and filter the output of other commands, keeping only the necessary information.
This guide will cover how to search for specific expressions in a set of text files with various contents using the GREP utility.
All examples shown were run on a cloud server hosted by Hostman running Ubuntu version 22.04.
How Does GREP Work
The GREP command follows this structure:
grep [OPTIONS] [PATTERN] [SOURCES]Where:
-
OPTIONS: Special parameters (flags) that activate certain mechanisms in the utility related to searching for expressions and displaying results. -
PATTERN: A regular expression (or plain string) containing the phrase (pattern, template, sequence of characters) you want to find. -
SOURCES: The path to the files where we will search for the specified expression.
If the GREP command is used to filter the output of another command, its structure looks a bit different:
[COMMAND] | grep [OPTIONS] [PATTERN]Thus:
-
COMMAND: An arbitrary command with its own set of parameters whose output needs to be filtered. -
The "pipe" symbol (
|) is necessary to create a command pipeline, redirecting streams so that the output of an arbitrary command becomes the input for the GREP command.
Preparation
To understand the nuances of using GREP, it's best to start with small examples of searching for specific phrases. Therefore, we will first create a few text files and then test the GREP command on them.
Let’s first prepare a separate directory where the search will take place:
mkdir textsNext, create the first file:
nano texts/poemIt will contain one of Langston Hughes's poems:
Hold fast to dreams
For if dreams die
Life is a broken-winged bird
That cannot fly.
Hold fast to dreams
For when dreams go
Life is a barren field
Frozen with snow.
Now, create the second file:
nano texts/code.pyIt will contain a simple Python script:
from datetime import date
dateNow = date.today()
print("Current time:", dateNow)
Finally, create the third file:
nano texts/page.htmlThis one will have simple HTML markup:
<html>
<head>
<title>Some Title</title>
</head>
<body>
<div class="block">
<p>There's gold here</p>
</div>
<div class="block">
<p>A mixture of wax and clouds</p>
</div>
<div class="block block_special">
<p>Today there's nothing</p>
</div>
</body>
</html>
By using files of different formats, we can better understand what the GREP command does by utilizing the full range of the utility's features.
Regular Expressions
Regular expressions are the foundation of the GREP command. Unlike a regular string, regular expressions contain special characters that allow you to specify phrases with a certain degree of variability.
When using the GREP utility, regular expressions are placed within single quotes:
'^date[[:alpha:]]*'Thus, the full command can look like this:
grep '^date[[:alpha:]]*' texts/*In this case, the console output will be:
texts/code.py:dateNow = date.today()However, using double quotes allows you to pass various system data into the expression. For example, you can first create an environment variable with the search expression:
PATTERN="^date[[:alpha:]]*"And then use it in the GREP command:
grep "$PATTERN" ./texts/*Additionally, using single backticks allows you to use bash subprocess commands within the GREP command. For example, you can extract a regular expression from a pre-prepared file:
grep `cat somefile` ./texts/*Note that with the asterisk symbol (wildcard), you can specify all the files in the directory at once. However, the GREP command also allows you to specify just one file:
grep '^date[[:alpha:]]' texts/code.py Because regular expressions are a universal language used in many operating systems and programming languages, their study is a separate vast topic.
However, it makes sense to briefly cover the main special characters and their functions. It’s important to note that regular expressions in Linux can work in two modes: basic (Basic Regular Expression, BRE) and extended (Extended Regular Expression, ERE). The extended mode is activated with the additional flag -E. The difference between the two modes lies in the number of available special characters and, consequently, the breadth of available functionality.
Basic Syntax
Basic syntax allows you to define only general formal constructs without considering the specific configuration of their characters.
Start of a line — ^
The caret symbol indicates that the sought sequence of characters must be at the beginning of the line:
grep '^Hold' texts/*The console output will be as follows:
texts/poem:Hold fast to dreams
texts/poem:Hold fast to dreamsEnd of a line — $
The dollar sign indicates that the sought sequence of characters must be at the end of the line:
grep '</p>$' texts/*Output:
texts/page.html: <p>There's gold here</p>
texts/page.html: <p>A mixture of wax and clouds</p>
texts/page.html: <p>Today there's nothing</p>
Note that the console output preserves the original representation of the found lines as they appear in the files.
Start of a word — \<
The backslash and less-than symbol indicate that the sought phrase must be at the beginning of a word:
grep '\<br' texts/*Output:
texts/poem:Life is a broken-winged birdEnd of a word — \>
The backslash and greater-than symbol indicate that the sought sequence of characters must be at the end of a word:
grep 'en\>' texts/*Output:
texts/poem:Life is a broken-winged bird
texts/poem:For when dreams go
texts/poem:Life is a barren field
texts/poem:Frozen with snow.Start or end of a word — \b
You can specify the start or end of a word using the more universal sequence of characters — backslash and the letter b.
For example, this marks the beginning:
grep '\bdie' texts/*Output:
texts/poem:For if dreams dieAnd this marks the end:
grep '<div\b' texts/*In this case, the console terminal output will be as follows:
texts/page.html: <div class="block">
texts/page.html: <div class="block">
texts/page.html: <div class="block block_special">
Any character — .
Certain characters in the sought phrases can be left unspecified using the dot symbol:
grep '..ere' texts/*Output:
texts/page.html: <p>There's gold here</p>
texts/page.html: <p>Today there's nothing</p>
Extended Syntax
Unlike basic syntax, extended syntax allows you to specify the exact number of characters in the sought phrases, thus expanding the range of possible matches.
Combining patterns — |
To avoid running the GREP command multiple times, you can specify several patterns in a single regular expression:
grep -E '^Hold|</p>$' texts/*The result of running this command will be a combined console output containing the search results for the two separate regular expressions shown earlier.
texts/page.html: <p>There's gold here</p>
texts/page.html: <p>A mixture of wax and clouds</p>
texts/page.html: <p>Today there's nothing</p>
texts/poem:Hold fast to dreams
texts/poem:Hold fast to dreams
Repetition range — {n, d}
In some cases, certain characters in the sought phrase may vary in quantity. Therefore, in the regular expression, you can specify a range of the allowed number of specific characters.
grep -E 'en{1,2}' texts/*Output:
texts/code.py:print("Current time:", dateNow)
texts/poem:Life is a broken-winged bird
texts/poem:For when dreams go
texts/poem:Life is a barren field
texts/poem:Frozen with snow.
However, frequently used repetition intervals are more conveniently written as special characters, thus simplifying the appearance of the regular expression.
One or more repetitions — +
A repetition interval from one to infinity can be expressed using the plus sign:
grep -E 'en+' texts/*In this case, the console output will not differ from the previous example.
texts/code.py:print("Current time:", dateNow)
texts/poem:Life is a broken-winged bird
texts/poem:For when dreams go
texts/poem:Life is a barren field
texts/poem:Frozen with snow.
Zero or one repetition — ?
A repetition interval from 0 to 1 can be expressed using the question mark:
grep -E 'ss?' texts/*As a result, this command will produce the following output in the console terminal:
texts/page.html: <div class="block">
texts/page.html: <p>There's gold here</p>
texts/page.html: <div class="block">
texts/page.html: <p>A mixture of wax and clouds</p>
texts/page.html: <div class="block block_special">
texts/page.html: <p>Today there's nothing</p>
texts/poem:Hold fast to dreams
texts/poem:For if dreams die
texts/poem:Life is a broken-winged bird
texts/poem:Hold fast to dreams
texts/poem:For when dreams go
texts/poem:Life is a barren field
texts/poem:Frozen with snow.
Character set — [abc]
Instead of one specific character, you can specify an entire set enclosed in square brackets:
grep -E '[Hh]o[Ll]' texts/*Output:
texts/poem:Hold fast to dreams
texts/poem:Hold fast to dreamsCharacter range — [a-z]
We can replace a large set of allowed characters with a range written using a hyphen:
grep -E 'h[a-z]+' texts/*Output:
texts/page.html:<html>
texts/page.html: <head>
texts/page.html: </head>
texts/page.html: <p>There's gold here</p>
texts/page.html: <p>Today there's nothing</p>
texts/page.html:</html>
texts/poem:That cannot fly.
texts/poem:For when dreams go
Moreover, character sets and ranges can be combined:
grep -E 'h[abcd-z]+' texts/*Each range is implicitly transformed into a set of characters:
[a-e]into[abcde][0-6]into[0123456][a-eA-F]into[abcdeABCDEF][A-Fa-e]into[ABCDEFabcde][A-Fa-e0-9]into[ABCDEFabcde0123456789][a-dA-CE-G]into[abcdABCEFG][acegi-l5-9]into[acegijkl56789]
Character type — [:alpha:]
Frequently used ranges can be replaced with predefined character types, whose names are specified in square brackets with colons:
|
|
characters from a to z in lowercase |
|
|
characters from A to Z in uppercase |
|
|
all alphabetic characters |
|
|
all digit characters |
|
|
all alphabetic characters and digits |
It is important to understand that the character type is a separate syntactic construct. This means that it must be enclosed in square brackets, which denote a set or range of characters:
grep -E '[[:alpha:]]+ere' texts/*Output:
texts/page.html: <p>There's gold here</p>
texts/page.html: <p>Today there's nothing</p>
Filtering Output
To filter the output of another command, you need to write a pipe symbol after it, followed by the standard call to the GREP utility, but without specifying the files to search:
cat texts/code.py | grep 'import'Like when searching in regular files, the console output will contain the lines with the matches of the specified phrases:
from datetime import dateIn this case, the cat command extracts the file content and passes it to the input stream of the GREP utility.
Search Options
In addition to regular expressions, you can specify additional keys for the GREP command, which are special options in flag format that refine the search.
Extended Regular Expressions (-E)
Activates the extended regular expressions mode, allowing the use of more special characters.
Case Insensitivity (-i)
Performs a search for a regular expression without considering the case of characters:
grep -E -i 'b[ar]' texts/*The console output corresponding to this command will be:
texts/poem:Life is a broken-winged bird
texts/poem:Life is a barren fieldYou can also specify flags together in a single string:
grep -Ei 'b[ar]' texts/*Whole Word (-w)
Performs a search so that the specified regular expression is a complete word (not just a substring) in the found line:
grep -w and texts/*Note that quotes are not required when specifying a regular string without special characters.
The result of this command will be:
texts/page.html: <p>A mixture of wax and clouds</p>Multiple Expressions (-e)
To avoid running the command multiple times, you can specify several expressions at once:
grep -e 'Hold' -e 'html' texts/*The result of this command will be identical to this one:
grep -E 'Hold|html' texts/*In both cases, the console terminal will display the following output:
texts/page.html:<html>
texts/page.html:</html>
texts/poem:Hold fast to dreams
texts/poem:Hold fast to dreams
Recursive Search (-r)
Performs a recursive search in the specified directory to the maximum depth of nesting:
grep -r '[Ff]ilesystem' /rootThe console terminal will display output containing file paths at different nesting levels relative to the specified directory:
/root/parser/parser/settings.py:#HTTPCACHE_STORAGE = "scrapy.extensions.httpcache.FilesystemCacheStorage"
/root/resize.log:Resizing the filesystem on /dev/vda1 to 3931904 (4k) blocks.
/root/resize.log:The filesystem on /dev/vda1 is now 3931904 (4k) blocks long.Search for Special Characters (-F)
Allows the use of special characters as the characters of the search phrase:
grep -F '[' texts/*Without this flag, you would encounter an error in the console terminal:
grep: Invalid regular expressionAn alternative to this flag would be using the escape character in the form of a backslash (\):
grep '\[' texts/*Including Files (--include)
Allows limiting the search to the specified files only:
grep --include='*.py' 'date' texts/*The console output will be:
texts/code.py:from datetime import date
texts/code.py:dateNow = date.today()
texts/code.py:print("Current time:", dateNow)We can also write this command without the wildcard by using an additional recursive search flag:
grep -r --include='*.py' 'date' textsExcluding Files (--exclude)
Selectively excludes certain files from the list of search sources:
grep --exclude='*.py' 'th' texts/*The console output will be:
texts/page.html: <p>Today there's nothing</p>
texts/poem:Frozen with snow.Output Options
Some parameters of the GREP command affect only the output of search results, improving their informativeness and clarity.
Line Numbers (-n)
To increase the informativeness of the GREP results, you can add the line numbers where the search phrases were found:
grep -n '</p>$' texts/*Each line in the output will be supplemented with the corresponding line number:
texts/page.html:8: <p>There's gold here</p>
texts/page.html:12: <p>A mixture of wax and clouds</p>
texts/page.html:16: <p>Today there's nothing</p>
Lines Before (-B)
Displays a specified number of lines before the lines with found matches:
grep -B3 'mix' texts/*After the flag, you specify the number of previous lines to be displayed in the console terminal:
texts/page.html- </div>
texts/page.html-
texts/page.html- <div class="block">
texts/page.html: <p>A mixture of wax and clouds</p>
Lines After (-A)
Displays a specified number of lines after the lines with found matches:
grep -A3 'mix' texts/*After the flag, you specify the number of subsequent lines to be displayed in the console terminal:
texts/page.html: <p>A mixture of wax and clouds</p>
texts/page.html- </div>
texts/page.html-
texts/page.html- <div class="block block_special">
Lines Before and After (-C)
Displays a specified number of lines both before and after the lines with found matches:
grep -C3 'mix' texts/*After the flag, you specify the number of preceding and following lines to be displayed in the console terminal:
texts/page.html- </div>
texts/page.html-
texts/page.html- <div class="block">
texts/page.html: <p>A mixture of wax and clouds</p>
texts/page.html- </div>
texts/page.html-
texts/page.html- <div class="block block_special">
Line Count (-c)
Instead of listing the found lines, the GREP command will output only the number of matches:
grep -c 't' texts/*The console output will contain the count of matches found in all specified files:
texts/code.py:3
texts/page.html:5
texts/poem:4If only one file is specified as the source:
grep -c 't' texts/blockThe console output will contain only the number:
4File Names (-l)
This flag allows you to output only the names of the files in which matches were found:
grep -l 't' texts/*The console output will be as follows:
texts/code.py
texts/page.html
texts/poemLimit Output (-m)
Limits the number of lines output to the console terminal to the number specified next to the flag:
grep -m2 't' texts/*The console output will be:
texts/code.py:from datetime import date
texts/code.py:dateNow = date.today()
texts/page.html:<html>
texts/page.html: <title>Some Title</title>
texts/poem:Hold fast to dreams
texts/poem:That cannot fly.
As you can see, the limiting number affects not the entire output but the lines of each file.
Exact Match of Whole Line (-x)
Searches for an exact match of the entire line with no variability:
grep -x 'Life is a broken-winged bird' texts/*The console output will be:
texts/poem:Life is a broken-winged birdConclusion
The GREP command in Linux is the most flexible and precise tool for searching expressions in large volumes of text data.
When using the command, you need to specify the following elements:
- A specific set of options (flags) that configure the search and output mechanisms.
- One or more regular expressions that describe the search phrase.
- A list of sources (files and directories) where the search will be performed.
Additionally, the utility is used to filter the output of other commands by redirecting input and output streams.
The core of the GREP command is regular expressions. Unlike a simple string, they allow you to define a phrase with a certain degree of variability, making it match multiple similar entries.
There are two modes of operation for regular expressions:
- Basic Mode: A limited set of special characters that allow you to formalize expressions only in general terms.
- Extended Mode: A full set of special characters that allows you to formalize expressions with precision down to each character.
The extended mode provides complete flexibility and accuracy when working with regular expressions.
In rare cases where you only need to find matches for trivial patterns, you can limit yourself to the basic mode.
Linux
11.02.2025
Reading time: 16 min
Share
Similar
Linux
How to Set Up Backup with Bacula
Bacula is a cross-platform client-server open source backup software that enables you to back up files, directories, databases, mail server data (Postfix, Exim, Sendmail, Dovecot), system images, and entire operating systems.
In this guide, we’ll walk you through the process of installing and configuring Bacula on Linux, as well as creating backups and restoring user data.
To get started with Bacula, you’ll need a server or virtual machine running any Linux distribution. In this tutorial, we’ll be using a cloud server from Hostman with Debian 12.
Bacula Architecture
Bacula’s architecture consists of the following components:
Director (Bacula Director)
The core component responsible for managing all backup, restore, and verification operations. The Director schedules jobs, sends commands to other components, and writes information to the database.
Storage Daemon (Bacula Storage)
Handles communication with storage devices such as disks, cloud storage, etc. The Storage Daemon receives data from the File Daemon and writes it to the configured storage medium.
File Daemon (Bacula File)
The agent installed on client machines to perform the actual backup operations.
Catalog
A database (MySQL, PostgreSQL, or SQLite) used by Bacula to store information about completed jobs, such as backup metadata, file lists, and restore history.
Console (Bacula Console, bconsole)
A command-line utility for interacting with Bacula. The Console allows administrators to control the Director via a CLI. GUI tools such as Bacula Web and Baculum are also available.
Monitor (Optional)
A component for monitoring the Bacula system status. It tracks job statuses, daemon states, and storage device conditions.
Creating Test Data for Backup
Let’s create some test files to use in our backup.
Create a test directory and navigate into it:
mkdir /root/test_backups && cd /root/test_backups
Now create six sequential files:
touch file{1..6}.txt
Also, create a directory in advance for storing restored files:
mkdir /root/restored-files
Installing Bacula
In this tutorial, we will install all Bacula components on a single server. However, Bacula also supports a distributed setup where components such as the Director, Storage Daemon, Client, and database can be installed on separate servers. This decentralized setup is suitable for backing up multiple systems without overloading a single server.
We'll be using Debian 12 and installing PostgreSQL (version 15) as the backend database.
Update the package index and install Bacula (server and client components):
apt update && apt -y install bacula-server bacula-client
PostgreSQL 15 will also be installed during this process.
During installation:
When prompted with: “Configure database for bacula-director-pgsql with dbconfig-common?”, press ENTER.
When asked to choose the database host, select localhost, since we are installing everything on one server.
When prompted with: “PostgreSQL application password for bacula-director-pgsql”, set a password for the Bacula database.
Do not leave this field empty, or a random password will be generated.
Re-enter the password when asked to confirm.
The installation will then continue normally.
After the installation is complete, verify the status of Bacula components and PostgreSQL.
Check the status of the Bacula Director:
systemctl status bacula-director
Check the Storage Daemon:
systemctl status bacula-sd
Check the File Daemon:
systemctl status bacula-fd
Check PostgreSQL:
systemctl status postgresql
If all components display a status of active, then Bacula has been successfully installed and is running.
Bacula Configuration
Bacula is configured by editing the configuration files of the program components. By default, all Bacula configuration files are located in the /etc/bacula directory.
Next, we will configure each Bacula component individually.
Configuring Bacula Director
Using any text editor, open the bacula-dir.conf configuration file for editing:
nano /etc/bacula/bacula-dir.conf
Let’s start with the Director block, which sets the main configuration parameters for the Director component:
Director {
Name = 4142939-bi08079-dir
DIRport = 9101
QueryFile = "/etc/bacula/scripts/query.sql"
WorkingDirectory = "/var/lib/bacula"
PidDirectory = "/run/bacula"
Maximum Concurrent Jobs = 20
Password = "ohzb29XNWSFISd6qN6fG2urERzxOl9w68"
Messages = Daemon
DirAddress = 127.0.0.1
}
Explanation of parameters:
Name: The name of the Director component. This is a unique identifier used to connect with other components like the File Daemon and Storage Daemon. By default, it includes the server's hostname and the -dir suffix. Example: 4142939-bi08079-dir.
DIRport: The port that Bacula Director listens to for incoming connections from the management console (bconsole). Default is 9101.
QueryFile: Path to the SQL script file used to run queries on the database. It contains predefined SQL queries for job management, verification, data restoration, etc. Default: /etc/bacula/scripts/query.sql.
WorkingDirectory: The working directory where Bacula Director temporarily saves files during job execution.
PidDirectory: The directory where the Director saves its PID file (process identifier). This is used to track if the process is running.
Maximum Concurrent Jobs: The maximum number of jobs that can run simultaneously. The default is 20.
Password: Password used for authenticating the management console (bconsole) with the Director. Must match the one specified in the console’s configuration.
Messages: Specifies the name of the message resource that determines how messages (errors, warnings, events) are handled. Common values: Daemon, Standard, Custom.
DirAddress: The IP address the Director listens on. This can be 127.0.0.1 for local connections or an external IP.
Catalog Configuration
By default, Bacula comes with its own PostgreSQL instance on the same host, and in that case, database connection settings don’t need changes. But if you're deploying the database separately (recommended for production), the address, username, and password must be specified in the Catalog block:
Catalog {
Name = MyCatalog
dbname = "bacula"; DB Address = "localhost"; dbuser = "bacula"; dbpassword = "StrongPassword4747563"
}
Explanation of parameters:
dbname: The name of the database used by Bacula (default is bacula). The database must already exist (when deployed separately).
DB Address: Host address where the DBMS is deployed. Use IP or a domain name. For local setup: localhost or 127.0.0.1.
dbuser: The user Bacula will use to connect to the database.
dbpassword: Password for the specified database user. Must be preconfigured.
Restore Job Configuration
Locate the Job block named RestoreFiles, responsible for file restoration. Set the Where parameter to specify the directory where restored files will be saved. Earlier, we created /root/restored-files, which we’ll use here:
Job {
Name = "RestoreFiles"
Type = Restore
Client=4244027-bi08079-fd
Storage = File1
# The FileSet and Pool directives are not used by Restore Jobs
# but must not be removed
FileSet="Full Set"
Pool = File
Messages = Standard
Where = /root/restored-files
}
Backup Schedule Configuration
Next, we set up the Schedule block that defines when backups are created.
We create:
A full backup every Monday at 00:01.
A differential backup every Sunday (2nd to 5th week) at 23:05.
An incremental backup daily at 23:00:
Schedule {
Name = "WeeklyCycle"
Run = Full 1st mon at 00:01
Run = Differential 2nd-5th sun at 23:05
Run = Incremental mon-sun at 23:00
}
FileSet Configuration
Now, we specify which files and directories will be backed up. This is defined in the FileSet block. Earlier we created /root/test_backups with six files. We’ll specify that path:
FileSet {
Name = "Full Set"
Include {
Options {
signature = MD5
}
File = /root/test_backups
}
}
Explanation of parameters:
Name: The name of the FileSet block, used for identification in configuration.
Options: Settings that apply to all files listed under Include.
signature = MD5: Specifies the checksum algorithm used to verify file integrity. MD5 generates a 128-bit hash to track file changes.
Exclude Configuration (Optional)
The Exclude block is used to specify files or directories that should not be backed up. This block is placed inside the FileSet definition and acts on files included via Include.
Exclude {
File = /var/lib/bacula
...
}
Pool Configuration
The Pool block defines a group of volumes (storage units) used for backup. Pools help manage how data is stored, rotated, and deleted.
Pool {
Name = Default
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Retention = 7 days
Maximum Volume Bytes = 10G
Maximum Volumes = 2
}
Explanation of parameters:
Name: The pool's name, here it's Default.
Pool Type: Defines the pool's function:
Backup: Regular backups.
Archive: Long-term storage.
Cloning: Data duplication.
Recycle: Indicates whether volumes can be reused once they're no longer needed (yes or no).
AutoPrune: Enables automatic cleanup of expired volumes.
Volume Retention: How long (in days) to retain data on a volume. After 7 days, the volume becomes eligible for reuse.
Maximum Volume Bytes: The max size for a volume. If it exceeds 10 GB, a new volume is created (if allowed).
Maximum Volumes: Limits the number of volumes in the pool. Here, it's 2. Older volumes are recycled when the limit is hit (if Recycle = yes).
Validating Configuration and Restarting Bacula
After making all changes, check the bacula-dir.conf file for syntax errors:
/usr/sbin/bacula-dir -t -c /etc/bacula/bacula-dir.conf
If the command output is empty, there are no syntax errors. If there are errors, the output will specify the line number and error description.
Restart the Bacula Director service:
systemctl restart bacula-director
Configuring Bacula Storage
The next step is configuring Bacula Storage, where the backup files will be stored.
Using any text editor, open the configuration file bacula-sd.conf for editing:
nano /etc/bacula/bacula-sd.conf
We'll start with the Storage block, which defines the storage daemon responsible for physically saving backup files:
Storage {
Name = 4149195-bi08079-sd
SDPort = 9103
WorkingDirectory = "/var/lib/bacula"
Pid Directory = "/run/bacula"
Plugin Directory = "/usr/lib/bacula"
Maximum Concurrent Jobs = 20
SDAddress = 127.0.0.1
}
Here’s what each parameter means:
Name: Name of the storage daemon instance, used to identify it uniquely.
SDPort: Port number the Storage Daemon listens on. The default is 9103.
WorkingDirectory: Working directory for temporary files. Default: /var/lib/bacula.
Pid Directory: Directory to store the PID file (process ID) for the storage daemon. Default: /run/bacula.
Plugin Directory: Path where Bacula’s plugins for the storage daemon are located. These plugins can provide extra features such as encryption or cloud integration.
Maximum Concurrent Jobs: Maximum number of jobs the storage daemon can handle simultaneously.
SDAddress: IP address the Storage Daemon is available at. This can be an IP or a domain name. Since in our case the Storage Daemon runs on the same server as the Director, we use localhost.
The next block to configure is Device, which defines the storage device where backups will be written.
The device can be physical (e.g., a tape drive) or logical (e.g., a directory on disk). For testing, one Device block will suffice. By default, bacula-sd.conf may contain more than one Device block, including a Virtual Autochanger — a mechanism that emulates a physical autochanger (used for managing tapes or other media). It lets you manage multiple virtual volumes (typically as disk files) just like real tapes in a tape library.
Locate the Autochanger block and remove the FileChgr1-Dev2 value from the Device parameter:
Autochanger {
Name = FileChgr1
Device = FileChgr1-Dev1
Changer Command = ""
Changer Device = /dev/null
}
Next, in the Device block below, specify the full path to the directory we previously created for storing backup files (/srv/backup) in the Archive Device parameter:
Device {
Name = FileChgr1-Dev1
Media Type = File1
Archive Device = /srv/backup
LabelMedia = yes;
Random Access = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 5
}
Any blocks referencing FileChgr2 and FileChgr1-dev2 should be deleted:
Explanation of the parameters:
Autochanger Block:
Name: Identifier for the autochanger (you can have multiple).
Device: Name of the device linked to this autochanger—must match the Device block name.
Changer Command: Script or command used to manage the changer. An empty value ("") means none is used—suitable for virtual changers or simple setups.
Changer Device: Refers to the device tied to the autochanger, typically for physical devices.
Device Block:
Name: Identifier for the device.
Media Type: Media type associated with the device. Must match the Pool block media type.
Archive Device: Full path to the device or directory for storing backups; /srv/backup in this case.
LabelMedia: Whether Bacula should auto-label new media.
Random Access: Whether random access is supported.
AutomaticMount: Whether to auto-mount the device when used.
RemovableMedia: Specifies if the media is removable.
AlwaysOpen: Whether the device should always stay open.
Maximum Concurrent Jobs: Maximum number of simultaneous jobs using this device.
Since we previously specified the directory for backup storage, create it:
mkdir -p /srv/backup
Set the ownership to the bacula user:
chown bacula:bacula /srv/backup
Next, check the config file for syntax errors:
/usr/sbin/bacula-sd -t -c /etc/bacula/bacula-sd.conf
If there are no syntax errors, the output will be empty. Otherwise, it will indicate the line number and description of any error.
Restart the storage daemon:
systemctl restart bacula-sd
Creating a Backup
Backups in Bacula are created using the bconsole command-line tool. Launch the utility:
bconsole
If it connects to the Director component successfully, it will display 1000 OK.
Before running a backup, you can check the status of all components by entering the command:
status
This will display a list of the five Bacula system components. To check them all, enter 6.
To initiate a backup, enter the command:
run
From the list, choose the BackupClient1 option (your client name might differ based on previous config), by typing 1.
After selecting the option, you’ll see detailed info about the backup operation.
You’ll then be prompted with three choices:
yes — start the backup process;
mod — modify parameters before starting;
no — cancel the backup.
If you enter mod, you’ll be able to edit up to 9 parameters.
To proceed with the backup, type yes.
To view all backup and restore jobs and their statuses:
list jobs
In our case, a backup with Job ID 1 was created:
list jobid=1
If the status is T, the backup was successful.
Possible statuses in the "Terminated Jobs" column:
T (Success) — Job completed successfully.
E (Error) — Job ended with an error.
A (Canceled) — Job was canceled by the user.
F (Fatal) — Job ended due to a critical error.
R (Running) → Terminated — Job completed (may be successful or not).
You can also monitor backup activity and errors via the log file:
cat /var/log/bacula/bacula.log
Once the backup finishes, the file will be saved in the specified directory.
file Vol-0001
Restoring Files from Backup
Earlier, we backed up the /root/test_backups directory, which contained six .txt files. Suppose these files were lost or deleted. Let’s restore them:
Launch the Bacula console:
bconsole
Start the restore process:
restore
You’ll see 12 available restore options.
We’ll use option 3. Type 3.
Earlier we used Job ID 1 for our backup. Enter 1.
You’ll enter a file selection mode. Since our files were in the root/test_backups directory, navigate there.
All previously saved files should be visible.
To restore the whole directory, go up one level:
cd ..
Then mark the whole test_backups folder:
mark test_backups/
Finish selection:
done
The system will display a final summary showing which data will be restored and the target directory (in our case: /root/restored-files).
To start the restore, enter yes.
Finally, verify that the files have been successfully restored.
Conclusion
We’ve now reviewed the installation and configuration of Bacula, a client-server backup solution. Bacula isn’t limited to backing up regular files—thanks to its plugin support, it can also handle backups of virtual machines, OS images, and more.
18 July 2025 · 14 min to read
Linux
Installing Arch Linux in a Cloud Environment
Arch Linux is a lightweight and flexible Linux distribution that provides users with extensive opportunities for customizing and optimizing their systems. It includes a minimal amount of preinstalled software and offers a console-based interface. In most cases, it is used by experienced users: professional developers, system administrators, or hackers. This is due to the complexity of its installation and subsequent configuration, which involves adding the required packages and components to the system. However, these difficulties are justified, because in the end the user gets exactly the system and services they need.
In this article, we will explain how to install Arch Linux on your cloud server and perform its basic configuration.
Advantages of Arch Linux
It is worth noting that Arch Linux is ideally suited as an OS for a cloud server due to its low resource requirements. This distribution also has several other advantages:
System UpdatesArch Linux updates automatically when a new OS version is released.
Software InstallationPackages can be downloaded both over the network and from a local disk. In addition, the installed software does not need to be specifically compatible with Arch Linux.
Rich RepositoriesArch Linux offers a wide variety of packages. Today, there are over 12,000 packages in the official repositories alone. In the community repository, there are even more — over 83,000.
Up-to-date DocumentationThe official Arch Linux documentation is actively updated to reflect the latest changes and innovations. This ensures accurate and relevant system information.
Active CommunityThis distribution has an active user community ready to help and share their experience. There are many forums, wikis, and repositories where you can find detailed instructions and guides for installation, configuration, and troubleshooting.
1. Preparing for Installation
To follow this guide and install Arch Linux, you will need:
A cloud server with any operating system (in our case, Debian 11);
A link to the Arch Linux image from an official source;
An additional disk, which you can attach under the Plan tab in the control panel.
Step 1. To install Arch Linux on the server, you must first upload its installation image from an official source in .iso format. For example:
wget https://mirror.rackspace.com/archlinux/iso/2025.06.01/archlinux-2025.06.01-x86_64.iso
Step 2. Next, add a new disk where the installation image will be stored. It will appear in the system as /dev/sdb. You can specify the minimum disk size.
Step 3. Write the installation image to the new disk:
dd if=archlinux-2025.06.01-x86_64.iso of=/dev/sdb
The writing process will take some time. When finished, verify it with the following command:
fdisk -l
In the output, you will see that the installation image has been written to the new disk, creating two necessary partitions.
Step 4. After writing the installation image, proceed to boot from it. To do this, go to the Access tab and boot the server from the recovery disk. Open the console in the control panel.
Step 5. In the console window, go to the Boot existing OS menu item and press Tab on your keyboard. This will allow you to edit the text at the bottom of the screen. Here, you need to manually replace hd0 with hd1, as shown in the figure below.
After that, press Enter to launch the installation program.
Step 6. In the system bootloader that appears, select the first option.
2. Partitioning the Disk
Now we can partition the main disk (sda). In our case, there will be 3 partitions: a 300 MB UEFI partition (type EFI), a 700 MB swap partition (type Linux swap), and a main filesystem partition taking up all remaining space (type Linux). In your own installation, the number and size of partitions may differ depending on your requirements.
Make sure there are no important files on the server’s disk, because it will be formatted later. You may also wish to back it up to preserve important data.
Step 1. First, check whether there are any files on the disk you need to save:
lsblk
The screenshot below shows the list. For creating the described partitions, we will use a 25 GB disk — sda. It currently has Debian 11 installed, which does not contain important files.
Step 2. To partition the disk, enter the following command:
cfdisk /dev/sda
Step 3. In the window that opens, you need to delete all existing partitions. To do this, select a partition and use the Delete button in the lower menu.
Step 4. Next, select the New button in the lower menu to create a new partition.
Step 5. Then specify the size of the partition to be created. In our case, this is 300 MB for UEFI.
Step 6. In the next window, choose Primary.
Step 7. The partition is now created, and you need to specify its type. Go to the Type menu and select EFI.
Step 8. Now move to the Free space and create 2 more partitions, repeating steps 4 through 7. Partition details were listed at the beginning of this chapter.
Step 9. Once all partitions have been created, go to the Write button and select it. To confirm, type yes in the field that appears.
Step 10. Partitioning is now complete. To exit the tool, select the Quit button in the lower menu.
Step 11. You can verify your work using the lsblk command again.
Check in the output that all changes have been successfully applied.
3. Formatting and Mounting the Created Partitions
At this stage, the created partitions will be formatted and mounted. Remember, all data will be erased in this process!
Step 1. For the first partition, format it using the following command:
mkfs.fat -F32 /dev/sda1
This command will create a FAT32 filesystem, which is the recommended format for the UEFI partition.
Step 2. Next, assign it a mount point:
mkdir /mnt/efi
mount /dev/sda1 /mnt/efi
Step 3. For the second partition, perform special formatting:
mkswap /dev/sda2
Step 4. Then activate the swap partition:
swapon /dev/sda2
Step 5. Finally, format the system’s root partition:
mkfs.ext4 /dev/sda3
Step 6. After formatting, create its mount point:
mount /dev/sda3 /mnt
After completing the formatting and mounting, your partitions will be ready for installing and configuring Arch Linux and its main components.
4. Installing the Main Arch Linux Components
Step 1. First, let’s install the OS and its core components:
pacstrap /mnt base linux grub openssh nano dhcpcd
Step 2. Once the installation finishes, you need to generate the fstab file:
genfstab -U /mnt >> /mnt/etc/fstab
Generating the fstab file makes partition mounting management easier and ensures automatic and consistent mounting at system startup.
5. System Configuration
Step 1. To configure Arch Linux after installation, you need to chroot into the OS without rebooting:
arch-chroot /mnt
Step 2. First, install the nano text editor:
pacman -S nano
Step 3. Uncomment the encoding for English in the relevant file (you would edit locale.gen):
nano /etc/locale.gen
Uncomment the line for en_US.UTF-8.
After this, save the changes and exit nano, then generate the locales:
locale-gen
To enable the English language, execute:
echo "LANG=en_US.UTF-8" > /etc/locale.conf
Step 4. At this step, set up the system clock. For example:
ln -sf /usr/share/zoneinfo/Europe/Nicosia /etc/localtime
The region is set. Now synchronize the hardware clock:
hwclock --systohc
Step 5. Next, set the hostname for your system:
echo "hostname" > /etc/hostname
Step 6. As the second-to-last step, set the root password. Run:
passwd
You will be prompted to enter and confirm the password.
Step 7. Lastly, set up the previously installed GRUB bootloader to boot the server:
grub-install --target=i386-pc /dev/sda
Then create the GRUB configuration file:
grub-mkconfig -o /boot/grub/grub.cfg
This command will automatically configure GRUB.
Step 8. Arch Linux is now successfully installed. Exit the chroot:
exit
Then go to the Access tab in your control panel and switch the server to standard boot mode. After that, click Save and Reboot.
You can remove the additional disk after this step.
Step 9. The system will boot, but it is not ready for use yet. First, connect to the server and enable the DHCP client daemon:
systemctl enable dhcpcd
Then start it:
systemctl start dhcpcd
Make sure the service shows the status active (running).
Step 10. Next, configure the SSH connection. First, create a backup of the sshd configuration:
cp /etc/ssh/sshd_config /etc/ssh/backup.sshdconf
Then set PermitRootLogin to Yes in the /etc/ssh/sshd_config file:
nano /etc/ssh/sshd_config
Finally, enable the SSH daemon:
systemctl enable sshd
And start it:
systemctl start sshd
When checking with systemctl status sshd, the service should show active (running) status.
Don’t forget to add and configure SSH keys before connecting to the server.
6. Additional Configuration
The installation is complete, but you can also perform additional system configuration by reviewing the official Arch Linux setup documentation.
To install packages, use the command:
pacman -S package_name
To update the system, use:
pacman -Suy
Conclusion
In this guide, we reviewed the process of installing Arch Linux on your cloud server and performed its basic configuration. We used a temporary Debian 11 OS and an additional disk for the installation image. By following these steps, you can create a powerful and flexible virtual environment for developing, testing, and running applications based on Arch Linux.
03 July 2025 · 8 min to read
Linux
How to Open a Port on Linux
Opening ports in Linux is an important task that allows certain services or applications to exchange data over the network. Ports act as communication gateways, allowing access to authorized services while blocking unauthorized connections. Managing ports is key to secure access, smooth app functionality, and reliable performance.
Understanding Ports and Their Purpose
Ports are the logical endpoints of network communication, where devices can send and receive information. HTTP uses port 80, HTTPS uses port 443, and SSH uses port 22. An open port means the service that listens for incoming network traffic is associated with it. A closed port, on the other hand, stops communication via that gateway. Maintaining availability and security requires proper management of Linux open ports.
Check Existing Open Ports on Linux
Before opening a port, check the open ports in Linux to see which ones are currently active. You may achieve this using several Linux commands.
netstat
To display open ports, run:
netstat -tuln
The netstat utility provides a real-time view of active network connections, displaying all listening endpoints. The -tuln flags refine the output to show only TCP and UDP ports without resolving hostnames.
Note: In case netstat isn’t installed, install it via:
sudo apt install net-tools
ss
The ss utility can also be utilized to check ports:
ss -tuln
Compared to netstat, the ss command is more recent and fast. It shows the ports that are in use as well as socket information.
nmap
For a detailed analysis of Linux open ports, use:
nmap localhost
The nmap utility scans the given host (localhost in this case) for open ports. This is useful for finding ports exposed to public networks.
Note: You can install nmap on Linux via:
sudo apt install nmap
Opening Ports on Linux
Firewall modification is required to grant access through a chosen endpoint. Linux provides several options for handling these tasks, including iptables, ufw, and firewalld.
Here are the methods to open ports with these utilities.
Method 1: Via iptables
Iptables is a robust and lower level firewall utility that grants fine-grained control over network traffic. To open a port with iptables, take these steps:
Add a Rule to Allow Traffic from a Specific Port
Enable HTTP access on port 8080 with this command:
sudo iptables -A INPUT -p tcp --dport 8080 -j ACCEPT
sudo: Execute the command as superuser.
iptables: Refers to the firewall utility.
-A INPUT: Inserts a rule in the input chain, controlling incoming traffic.
-p tcp: Shows that the rule is for TCP traffic.
--dport 8080: Points to port 8080 for the rule.
ACCEPT: Specifies that incoming traffic matching the rule is accepted.
This permits incoming TCP on port 8080. However, iptables changes are volatile and will be undone after reboot.
Note: The iptables can be installed with persistent packages using:
sudo apt install iptables iptables-persistent
Save the Configuration
For making the rule permanent and remain even after a system restart, store iptables rules via:
sudo netfilter-persistent save
This directive preserves current iptables or nftables rules such that they are preserved during reboots.
Reload Changes
Reload the firewall configuration as needed with:
sudo netfilter-persistent reload
Method 2: Via UFW
Ufw (Uncomplicated Firewall) is a minimal front-end for managing iptables rules. It allows you to easily open ports with simple commands. This is how you can do it:
Enable Ufw
First, ensure the ufw firewall is activated:
sudo ufw enable
Executing this command allows UFW to modify firewall settings.
Note: UFW can be installed with:
sudo apt install ufw
Allow Traffic Via Specific Port
For instance, to open port 22 for SSH, use:
sudo ufw allow 22/tcp
sudo: Grants superuser privileges.
ufw allow: Adds a rule to permit traffic.
22/tcp: Sets port 22 for communication while restricting the rule to TCP protocol.
This permits access on port 22, enabling remote SSH connections.
Verify the Firewall Status
To ensure the port is accessible and the rule is active, execute:
sudo ufw status
The status command displays all active rules, including the allowed ports.
Method 3: Via Firewalld
Firewalld is a dynamic firewall daemon present on Linux. It is simpler to customize the firewall rules compared to using iptables. Here’s how to enable port access via firewalld:
Add a Permanent Rule for the Desired Port
To enable HTTPS access on port 443, run:
sudo firewall-cmd --permanent --add-port=443/tcp
firewall-cmd: Invokes the firewalld command.
--permanent: Ensures the rule stays active after the firewall reloads or the system boots.
--add-port=443/tcp: Opens port 443 to accept incoming TCP traffic.
Note: Install firewalld on Linux via:
sudo apt install firewalld
Once installed, you should activate and run it:
sudo systemctl enable firewalld
sudo systemctl start firewalld
Reload the Firewall
Finalize the settings to enable the newly defined policy:
sudo firewall-cmd --reload
Applying firewall modifications makes recent policy updates functional without rebooting.
Verification
Check whether the port is opened successfully:
sudo firewall-cmd --list-all
The --list-all command provides a complete list of rules, helping you determine if port 443 is open.
Testing the Newly Opened Port
Always check if the newly opened port is available for incoming connections. Here’s how:
Using telnet
Test the port opening via:
telnet localhost port_number
Successful access means the port is open and responsive.
Using nmap
Analyze the host to verify if the specified endpoint is accessible.:
nmap -p port_number localhost
The -p flag specifies the port to scan.
Using curl
Check HTTP service availability:
curl localhost:port_number
A successful response confirms the service is running on the opened port.
Troubleshooting Common Issues
Ports opening may occasionally fail due to configuration errors or conflicting software settings. Follow these tips:
Verify Firewall Rules: Run iptables -L or ufw status to assess firewall restrictions and permissions.
Check Service Status: Check if the assigned service is active with systemctl status <service-name>.
Opening Specific Ports Based on Protocol
Understanding the protocol used by the service can help configure ports more effectively. For instance, web traffic typically uses TCP (Transmission Control Protocol) for stable communication, while certain gaming services may require UDP (User Datagram Protocol) for faster packet transmission.
Opening a TCP Port
To access port 3306 for MySQL traffic:
sudo ufw allow 3306/tcp
This explicitly permits TCP traffic through port 3306, ensuring stable communication for database queries.
Opening a UDP Port
To access port 161 for SNMP (Simple Network Management Protocol), run:
sudo ufw allow 161/udp
UDP provides faster, connectionless communication, ideal for monitoring tools like SNMP.
Managing Port Accessibility
Once a port is opened, controlling its visibility ensures security and prevents unauthorized access.
Restricting Access to Specific IPs
To limit port access to a specific IP address (e.g., 192.168.1.100):
sudo ufw allow from 192.168.1.100 to any port 22
This allows SSH access via port 22 only from the specified IP address, enhancing security.
Closing Ports
To revoke access to port 80:
sudo ufw deny 80/tcp
This denies incoming traffic on port 80, effectively closing it for HTTP services.
Conclusion
Confirming open ports in Linux is a key step for optimizing network functionality and deploying services effectively. With the use of utilities such as iptables, ufw, or firewalld, you can control traffic securely for your apps. You need to test and debug in order to confirm the port is open and working as expected. From web servers to SSH access, to other network services, port management skills ensure smooth operations and better security.
01 July 2025 · 7 min to read
Do you have questions,
comments, or concerns?
Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
whether you need help or are just unsure of where to start.
