How to Use the diff Command in Linux

One of the core principles of Unix systems is the extensive use of text data: configuration files, as well as input and output data in *nix systems, are often organized as plain text. Regular expressions are a powerful tool for manipulating text data. This guide delves into the intricacies of using regular expressions in Bash, helping you fully harness the power of the command line and scripts in Linux. What Are Regular Expressions? Regular expressions are specially formatted strings used to search for character patterns in text. They resemble shell wildcards in some ways, but their capabilities are much broader. Many text-processing utilities in Linux and programming languages include a regular expression engine. However, different programs and languages often employ different regular expression dialects. This article focuses on the POSIX standard to which most Linux utilities adhere. The grep Utrequires at least one match of theility The grep program is the primary tool for working with regular expressions. grep reads data from standard input, searches for matches to a specified pattern, and outputs all matching lines. grep is typically pre-installed on most distributions. You can try the commands in a virtual machine or a VPS to practice using regular expressions. The syntax of grep is as follows: grep [options] regular_expression [file...] The simplest use case for grep is finding lines that contain a fixed substring. In the example below, grep outputs all lines that contain the sequence nologin: grep nologin /etc/passwd Output: daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin bin:x:2:2:bin:/bin:/usr/sbin/nologin sys:x:3:3:sys:/dev:/usr/sbin/nologin games:x:5:60:games:/usr/games:/usr/sbin/nologin ... grep has many options, which are detailed in the documentation. Here are some useful options for working with regular expressions: -v — Inverts the match criteria. With this option, grep outputs lines that do not contain matches: ls /bin | grep -v zip# Output:411toppm 7z 7za 7zr ... -i — Ignores case. -o — Outputs only the matches, not the entire lines: ls /bin | grep -o zip# Output:zip zip zip zip ... -w — Searches for lines containing whole words matching the pattern. ls /bin | grep -w zip# Output:gpg-zipzip For comparison, the same command without the -w option also includes lines where the pattern appears as a substring within a word. ls /bin | grep zip# Output:bunzip2 bzip2 bzip2recover funzip Basic Regular Expressions (BRE) As previously mentioned, there are multiple dialects of regular expressions. The POSIX standard defines two main types of implementations: Basic Regular Expressions (BRE), which are supported by almost all POSIX-compliant programs, and Extended Regular Expressions (ERE), which allow for more complex patterns but aren't supported by all utilities. We'll start by exploring the features of BRE. Metacharacters and Literals We've already encountered simple regular expressions. For example, the expression “zip” represents a string with the following criteria: it must contain at least three characters; it includes the characters “z”, “i”, and “p” in that exact order; and there are no other characters in between. Characters that match themselves (like “z”, “i”, and “p”) are called literals. Another category is metacharacters, which are used to define various search criteria. Metacharacters in BRE include: ^ $ . [ ] * \ - To use a metacharacter as a literal, you need to escape it with a backslash (\). Note that some metacharacters have special meanings in the shell, so enclose it in quotes when passing a regular expression as a command argument. Any Character The dot (.) metacharacter matches any character in that position. For example: ls /bin | grep '.zip' Output: bunzip2 bzip2 bzip2recover funzip gpg-zip gunzip gzip mzip p7zip pbzip2 preunzip prezip prezip-bin streamzip unzip unzipsfx One important detail: the zip program itself isn’t included in the output because the dot (.) metacharacter increases the required match length to four characters. Anchors The caret (^) and dollar sign ($) in regular expressions serve as anchors. This means that, when included, a match can only occur at the start of a line (^) or at the end ($). ls /bin | grep '^zip'# Output:zip zipcloak zipdetails zipgrep …ls /bin | grep 'zip$'# Output:funzip gpg-zip gunzip ...ls /bin | grep '^zip$'# Output:zip The regular expression ^$ matches empty lines. Character Sets Besides matching any character in a given position (.), regular expressions allow for matching a character from a specific set. This is done with square brackets. The following example searches for strings matching bzip or gzip: ls /bin | grep '[bg]zip'# Output:bzip2bzip2recovergzip All metacharacters lose their special meaning within square brackets, except two. If a caret (^) is placed immediately after the opening bracket, the characters in the set are treated as excluded from that position. For example: ls /bin | grep '[^bg]zip' Output: bunzip2 funzip gpg-zip gunzip mzip p7zip preunzip prezip prezip-bin streamzip unzip unzipsfx With negation, we get a list of filenames containing zip but preceded by any character other than b or g. Note that zip is not included here; the negation requires the presence of some character in that position. The caret serves as a negation only if it appears immediately after the opening bracket; otherwise, it loses its special meaning. Using a hyphen (-), you can specify character ranges. This lets you match a range of characters or even multiple ranges. For instance, to find all filenames that start with a letter or a number: ls ~ | grep '^[A-Za-z0-9]' Output: backup bin Books Desktop docker Documents Downloads GNS3 ... POSIX Character Classes When using character ranges, one challenge is that ranges can be interpreted differently based on locale settings. For instance, the range [A-Z] may sometimes be interpreted lexicographically, potentially excluding lowercase a. To address this, the POSIX standard provides several classes that represent various character sets. Some of these classes include: [:alnum:] — Alphanumeric characters; equivalent to [A-Za-z0-9] in ASCII. [:alpha:] — Alphabetic characters; equivalent to [A-Za-z] in ASCII. [:digit:] — Digits from 0 to 9. [:lower:] and [:upper:] — Lowercase and uppercase letters, respectively. [:space:] — Whitespace characters, including space, tab, carriage return, newline, vertical tab, and form feed. Character classes don’t provide an easy way to express partial ranges, like [A-M]. Here’s an example of using a character class: ls ~ | grep '[[:upper:]].*' Output: Books Desktop Documents Downloads GNS3 GOG Games Learning Music ... Extended Regular Expressions (ERE) Most POSIX-compliant applications and those using BRE (such as grep and the stream editor sed) support the features discussed above. The POSIX ERE standard allows for more expressive regular expressions, though not all programs support it. The egrep program traditionally supported the ERE dialect, but the GNU version of grep also supports ERE when run with the -E option. In ERE, the set of metacharacters is expanded to include: ( ) { } ? + | Alternation Alternation allows for a match with one of multiple expressions. Similar to square brackets that allow a character to match one of several characters, alternation allows for matching one of multiple strings or regular expressions. Alternation is represented by the pipe (|): echo "AAA" | grep -E 'AAA|BBB'# Output:AAA echo "BBB" | grep -E 'AAA|BBB'# Output:BBB echo "CCC" | grep -E 'AAA|BBB'# Output: (no match) Grouping You can group elements of regular expressions and treat them as a single unit using parentheses. The following expression matches filenames starting with bz, gz, or zip. Without the parentheses, the regular expression would change meaning to match filenames starting with bz or containing gz or zip. ls /bin | grep -E '^(bz|gz|zip)' Output: bzcat bzgrep bzip2 bzip2recover bzless bzmore gzexe gzip zip zipdetails zipgrep zipinfo zipsplit Quantifiers Quantifiers specify the number of times an element can occur. BRE supports several quantifiers: ? — Matches the preceding element zero or one time, meaning the previous element is optional: echo "tet" | grep -E 'tes?t'# Output:tet echo "test" | grep -E 'tes?t'# Output:test echo "tesst" | grep -E 'tes?t'# Output: (no match) * — Matches the preceding element zero or more times. Unlike ?, this element can appear any number of times: echo "tet" | grep -E 'tes*t'# Output:tet echo "test" | grep -E 'tes*t'# Output:test echo "tesst" | grep -E 'tes*t'# Output:tesst + — Similar to *, but requires at least one match of the preceding element: echo "tet" | grep -E 'tes+t'# Output: (no match) echo "test" | grep -E 'tes+t'# Output:test echo "tesst" | grep -E 'tes+t'# Output:tesst In BRE, special metacharacters { and } allow you to specify minimum and maximum match counts for the preceding element in four possible ways: {n} — Matches if the preceding element occurs exactly n times. {n,m} — Matches if the preceding element occurs at least n and at most m times. {n,} — Matches if the preceding element occurs n or more times. {,m} — Matches if the preceding element occurs no more than m times. Example: echo "tet" | grep -E "tes{1,3}t"# Output: (no match) echo "test" | grep -E "tes{1,3}t"# Output:test echo "tesst" | grep -E "tes{1,3}t"# Output:tesst echo "tessst" | grep -E "tes{1,3}t"# Output:tessst echo "tesssst" | grep -E "tes{1,3}t"# Output: (no match) Only the lines where s appears one, two, or three times match the pattern. Regular Expressions in Practice To conclude, let’s look at a couple of practical examples of how regular expressions can be applied. Validating Phone Numbers Suppose we have a list of phone numbers where the correct format is (nnn) nnn-nnnn. Out of a list of 10 numbers, three are incorrectly formatted. cat phonenumbers.txt Output: (185) 136-1035 (95) 213-1874 (37) 207-2639 (285) 227-1602 (275) 298-1043 (107) 204-2197 (799) 240-1839 (218) 750-7390 (114) 776-2276 (7012) 219-3089 The task is to identify the incorrect numbers. We can use the following command: grep -Ev '^\([0-9]{3}\) [0-9]{3}-[0-9]{4}$' phonenumbers.txt Output: (95) 213-1874(37) 207-2639(7012) 219-3089 Here, we used the -v option to invert the match and output only lines that do not match the specified format. Since parentheses are considered metacharacters in ERE, we escaped them with backslashes to treat them as literals. Finding Improper File Names The find command supports checking paths with regular expressions. It’s important to note that, unlike grep which matches parts of lines, find requires the whole path to match. Suppose we want to identify files and directories containing spaces or potentially problematic characters. find . -regex '.*[^-_./0-9a-zA-Z].*' The .* sequences at the beginning and end represent any number of any characters, which is necessary because find expects the entire path to match. Inside the square brackets, we use negation to exclude valid filename characters, meaning any file or directory name containing characters other than hyphens, underscores, digits, or Latin letters will appear in the output. Conclusion This article has covered a few practical examples of Bash regular expressions. Creating complex regular expressions might seem challenging at first. But over time, you’ll gain experience and skill in using regular expressions for searches across various applications that support them.

DNF (Dandified Yum) is the next-generation version of Yum, the default package manager for CentOS and Fedora distributions. It is designed to resolve dependencies more efficiently, handle larger package sets, and improve performance over its predecessor. DNF simplifies the management of software packages by allowing users to install, update, and remove packages from the command line with a user-friendly interface. Installing and Removing Packages with DNF One of the primary functions of DNF is installing and removing software packages. To install a package using DNF, you need root or sudo privileges. The syntax is straightforward: sudo dnf install package_name For example, to install the Apache HTTP server: sudo dnf install httpd To remove a package, the command is similar: sudo dnf remove package_name For example, to remove Apache HTTP server: sudo dnf remove httpd Updating and Upgrading Packages Keeping your system up to date is essential for security and performance. DNF makes this process simple. To update all the packages on your system, use: sudo dnf update This command will update installed packages to the latest versions available in the configured repositories. If you want to upgrade your entire system to the latest release (such as when moving between CentOS versions), you can use: sudo dnf upgrade The difference between update and upgrade is that the latter will also remove obsolete packages, whereas update does not. Searching for Packages in DNF DNF allows users to search for packages before installing them. This is helpful if you're unsure of the exact package name or want to explore available options. To search for a package: sudo dnf search <keyword> For example, to search for packages related to Apache: sudo dnf search apache DNF will list all packages that match the search term, along with a brief description. Managing Repositories with DNF Repositories are essential for managing where DNF pulls its packages from. DNF automatically handles repository configuration files, usually located in /etc/yum.repos.d/. You can add, enable, or disable repositories with DNF. To add a new repository, you need to create a .repo file in /etc/yum.repos.d/. For example, let's say you want to add the EPEL (Extra Packages for Enterprise Linux) repository, which provides additional packages not available in the default CentOS repositories. Install the EPEL repository using DNF EPEL is available as a package that can be installed directly: sudo dnf install epel-release This command automatically creates the necessary .repo file and enables the EPEL repository. Manually adding a repository If you want to manually add a repository, you would create a .repo file, for instance, myrepo.repo, in /etc/yum.repos.d/, and add the following content: [myrepo]name=My Custom Repobaseurl=http://example.com/repo/centos/$releasever/$basearch/enabled=1gpgcheck=1gpgkey=http://example.com/repo/RPM-GPG-KEY-myrepo Here: name specifies the name of the repository. baseurl defines the URL from where the packages will be downloaded. enabled=1 ensures the repository is active. gpgcheck=1 enables GPG key checking for security. gpgkey provides the URL to the GPG key used to verify the packages. To disable the epel repository: sudo dnf config-manager --set-enabled epel To enable it again: sudo dnf config-manager --set-enabled epel Cleaning Up Unused Packages Over time, your system may accumulate unnecessary packages and cache files, which take up valuable space. DNF includes a built-in command to clean up unused packages and free up disk space: sudo dnf autoremove This will remove any orphaned packages that are no longer required by the system. Additionally, you can clean up cached data using: sudo dnf clean all This command clears all cached package files stored in /var/cache/dnf/. Troubleshooting DNF Issues Occasionally, you may encounter issues when managing packages with DNF. Common problems include broken dependencies or repository errors. Here are some troubleshooting tips: Broken dependencies: If you're facing dependency issues, try running: sudo dnf install --best --allowerasing This command attempts to resolve conflicts by allowing DNF to erase conflicting packages. Corrupted cache: If the cache becomes corrupted, clean it up using: sudo dnf clean metadata Failed transactions: If a DNF transaction fails, try rebuilding the database: sudo rpm --rebuilddb By using these tips, you can quickly resolve most issues you might face with DNF. Conclusion DNF is a powerful and efficient package manager that makes software management on CentOS easy. From installing and updating packages to managing repositories and cleaning up unused files, DNF provides a wide range of features to ensure your system runs smoothly. With this guide, you should be well-equipped to handle package management tasks on your CentOS system. On Hostman, you can try Linux VPS hosting for your projects. 

Bash (Bourne-Again SHell) is a command interpreter in UNIX-like operating systems that allows for task automation at the command line level. Bash scripts are files containing a sequence of commands that can be executed by the Bash interpreter. Bash scripts can be used to automate repetitive tasks. For example, if you need to generate and send a report via email every day, you can write a bash script that performs these actions automatically. This saves a lot of time and reduces the likelihood of errors. In this article, we will cover the basic concepts and tools for writing Bash scripts in Linux. Bash Script Syntax Bash scripts can be written in any text editor and must have executable permissions. Let’s consider some of the most popular editors: Nano is a simple text editor that comes with most Linux distributions. It has an intuitive interface and useful features like syntax highlighting. Vim is one of the most popular text editors for Linux, though it may seem complicated for beginners. Vim offers many features to speed up coding, such as syntax highlighting, autocompletion, and macros. Emacs is another popular text editor for Linux. It also has many features that can simplify the coding process. One of its main features is the ability to run the Bash interpreter inside the editor, allowing you to test scripts without exiting the editor. At the beginning of each script, there must be a line called a shebang, which tells the operating system which interpreter to use to execute the script. The shebang should start with a hash symbol (#) followed by an exclamation mark (!), and then the path to the interpreter. To use the Bash interpreter, the shebang will look like this: #!/bin/bash While writing the script, you can also leave comments that start with a hash symbol and continue until the end of the line. Comments will not be executed by the interpreter and are used to describe the functionality of the script. For example: # This is a comment Below, we will write our first script. Suppose we want to create a script in Linux that greets the user and displays the current date and time on the screen. To do this, create a file named greeting.sh in any directory on your computer and add the following code: #!/bin/bash echo "Hello, $USER!" echo "Today is $(date)" The first line indicates that this is a Bash script. The next line, echo "Hello $USER!", outputs a greeting with the current user's name. $USER is a system variable that contains the name of the current user. The third line, echo "Today is $(date)", displays the current date and time. $(date) is used to call the date command, which returns the current date and time in the system's format. When creating a Bash script, it’s important to ensure the file is executable. To do this, you need to change the file permissions. We’ll cover this and how to run the script in the next chapter. Running Scripts To run a script in Linux, it must have executable permissions. To make a file executable, you can use the chmod command (short for "change mode"). This command allows you to change the access permissions of files and directories in Linux. The syntax for the chmod command is as follows: chmod [options] access_rights file where access_rights is a special code that sets the access permissions for a file or directory, and file is the path to the file or directory whose permissions you want to change. To make a file executable, you need to add the execute (x) permission to its access rights. For example, to make the greeting.sh file executable, use the following command: chmod +x greeting.sh This command will add execute permissions for the current user. Now, we can run the Bash script in Linux by invoking it from the terminal: ./greeting.sh The result of running the script is shown below. Command Line Parameters Command line parameters allow you to pass arguments to Linux scripts when they are run. Command line parameters can be accessed in the script as $1, $2, $3, etc., where $1 is the first parameter, $2 is the second parameter, and so on. Let's rewrite the script from the previous chapter to greet the user using a command-line argument: #!/bin/bashecho "Hello $1!" Then run the script, passing the $USER argument: ./greeting.sh $USER The result is shown below. Additionally, you can use special command line parameters: $0 — the name of the script (i.e., the name of the file that was run) $# — the number of passed parameters $* or $@ — a list of all passed parameters (as a single string or array, respectively) $? — the return code of the last executed command For example, to display the number of passed parameters, you can use the following code: #!/bin/bash echo "Hello $1!" echo "Number of passed parameters: $#" The result of running the script is shown below. Variables Variables in Bash are used to store data, such as strings and numbers. They can be explicitly defined by assigning a value or implicitly defined through automatic assignment during certain operations. To create a variable in Bash, you need to assign it a value using an equal sign (=). For example: company="Hostman" Note that there should be no spaces between the variable name, the equal sign, and the value. You can retrieve the value of a variable by specifying its name after the echo command and the $ sign. For example: echo $company It's also possible to assign a variable value through user input using the read command. For example, the following script prompts the user for their name and stores it in a variable: #!/bin/bash echo "What is your name?" read name echo "Hello, $name!" The result of this script is shown below. In Bash, there are several special variables that are automatically defined and filled by the system. For example, the $HOME variable contains the path to the user's home directory, while $PWD contains the path to the current working directory.  Additionally, there are environment variables that are defined by the system and can be used in scripts. For example, $PATH contains a list of directories where Bash looks for executable files. Variables can also be used to pass values between different commands and scripts. For example, to pass a variable’s value from one script to another, use the export command: export variable_name Conditional Operators Conditional operators allow you to execute a specific set of actions depending on whether a condition is true or false. In Bash scripts, conditions are written in brackets and passed to the if command. The syntax of the if operator looks like this: if [ condition ] then commands to execute if the condition is true fi Here, in the square brackets, you specify the condition that needs to be checked. If the condition is true, the commands between then and fi will be executed. For example, let’s write a Linux script, evenodd.sh, that checks whether the number entered by the user is even or odd: #!/bin/bash echo "Enter a number: " read n if (( $n % 2 == 0 )) then echo "The number $n is even" else echo "The number $n is odd" fi In this example, we use the % operator, which calculates the remainder of division by 2. If the remainder is 0, the number is even; otherwise, it’s odd. The result of running the script is shown below. Additionally, there are several comparison operators that can be used in conditional constructions: -eq – equal to; -ne – not equal to; -gt – greater than; -lt – less than; -ge – greater than or equal to; -le – less than or equal to. For example, to check if the variable $a is greater than the variable $b, you can write the following: if [ $a -gt $b ] then echo "$a is greater than $b" fi It is important to remember that you need to use spaces around the comparison operators in conditional constructions. If there are no spaces, Bash will treat this as one large string instead of a comparison operation. In addition to if, Bash scripts also use the case structure. This allows you to check a variable's value against several possible options. We will discuss this in the next chapter. The Case Construction The case construction in Bash scripts allows you to simplify writing conditional operators for comparing variables with multiple possible values. The syntax of the case construction is as follows: case variable in pattern1) command1 ;; pattern2) command2 ;; pattern3) command3 ;; *) default command ;; esac where variable is the variable to check, pattern1, pattern2, pattern3 are the possible values to check, and command1, command2, command3 are the commands to execute depending on the value of the variable. The * symbol at the end of the list of values is used as a default handler if none of the values match the variable. For example, let’s look at a script that checks the day of the week and performs the corresponding action: #!/bin/bash day=$(date +%u) case $day in 1) echo "Today is Monday" ;; 2) echo "Today is Tuesday" ;; 3) echo "Today is Wednesday" ;; 4) echo "Today is Thursday" ;; 5) echo "Today is Friday" ;; 6) echo "Today is Saturday" ;; 7) echo "Today is Sunday" ;; *) echo "Invalid day of the week" ;; esac In this example, we use the day variable, which we define using the date +%u command. In this case, %u is used to obtain the numeric value of the day of the week, from 1 (Monday) to 7 (Sunday). Then we compare this variable with the days of the week using the case construction. If its value matches a certain day of the week, we display the corresponding message. If the value does not match any of the listed days, we display an error message. The result of running the script is shown below.  Loops Loops in Bash are used to perform repetitive actions. There are two types of loops: for and while. The for loop is used to execute commands for each element in a list. The syntax of the for loop is as follows: for variable in list do commands done Here, the variable takes the value of an element from the list, and for each of them, the commands between do and done are executed. Example: #!/bin/bash for i in {1..10}; do echo "Number: $i" done In this example, i takes values from 1 to 10, and for each of them, the echo "Number: $i" command will be executed. The result of running this loop will look like this: The while loop is used to execute commands as long as the condition remains true. The syntax of the while loop is as follows: while [ condition ] do commands done Here, in square brackets, you specify the condition that is checked before each iteration of the loop. The commands between do and done will be executed as long as the condition is true. Example: #!/bin/bash count=1 while [ $count -le 10 ]; do echo "Count: $count" count=$((count+1)) done In this example, count increases by 1 after each iteration of the loop. When the value of count reaches 10, the loop terminates. The result of running this loop will look like this: Functions Functions in Bash are used to group commands into logically related blocks. Functions can be called from a script using their name.  The syntax of a function is as follows: function_name () { commands_and_expressions } The function name must start with a letter or an underscore and can contain only letters, numbers, and underscores. After the function name comes a list of arguments in parentheses. The commands and expressions to be executed when the function is called must be enclosed in curly braces. Here’s an example of a function that outputs the current time and date: #!/bin/bash print_date () { echo "Today's date: $(date)" } print_date # Function call The result of running the script is shown below. Functions can also accept arguments, which are passed as parameters inside the parentheses when calling the function. Here’s an example of a function that takes two arguments and outputs their sum: #!/bin/bash sum_numbers () { result=$(( $1 + $2 )) echo "The sum of $1 and $2 is $result" } sum_numbers 10 20 # Function call In this example, $1 and $2 are variables that contain the values of the first and second arguments, respectively. sum_numbers 10 20 will call the sum_numbers function with the arguments 10 and 20, and output the following result: Functions can also return values using the return keyword. Let’s rewrite the previous example using this new knowledge: #!/bin/bash sum_numbers () { result=$(( $1 + $2 )) return $result } sum_numbers 12 24 # Function call echo "The sum of the numbers is $?" # Output Here, the result is stored in the result variable and returned from the function using the return command. The $? variable contains the return code of the function, which in this case is the result of the sum calculation. The result of running the script is shown below. There is another way to handle the result of a function call without using return. Let’s slightly modify the previous script: #!/bin/bash sum_numbers () { result=$(( $1 + $2 )) echo $result } sum=$(sum_numbers 9 11) echo "The sum of the numbers is $sum" # Output Here, instead of using $? and return, we store the result of the function call in the sum variable and then output its value. The result is shown below. Working with Files and Directories Bash scripts can be used to perform various operations with files and directories in Linux. For example, to check if a file exists, you can use the following command:  test -e filename  If the file exists, the command will return a value of 0; otherwise, it will return a non-zero value. To work with directories in Bash scripts, you can use commands like cd, mkdir, rmdir, ls, and others. Script Debugging Debugging Bash scripts can be a challenging task because problems can be caused by various factors, such as syntax errors, incorrect use of variables or functions, etc. For debugging Bash scripts, you can use tools like set -x, set -v, and set -e. The set -x command allows you to display the commands before they are executed The set -v command displays the values of variables before they are used The set -e command stops the execution of the script in case of an error Conclusion Bash scripts are a powerful tool for automating tasks in UNIX-like operating systems. In this article, we covered the basic concepts and tools for writing Bash scripts, such as syntax, variables, conditional operators, loops, functions, and running scripts. We hope this guide helps you become a more productive and experienced Linux user. You can buy Linux VPS for your projects on Hostman.