Sign In
Sign In

Input/Output Redirection in Linux

Input/Output Redirection in Linux
Hostman Team
Technical writer
Linux
10.04.2025
Reading time: 8 min

One of the key principles of the UNIX philosophy is that all command-line interface (CLI) commands should accept text as input and produce text as output. Since this concept was applied in the development of UNIX (and later Linux), there are commands to receive text as input, perform an operation on it, and then produce text as output. Commands that read text input, modify it in some way, and then produce text output are sometimes called filters.

To use filter commands and work with text streams effectively, we should understand several types of redirection available with most commands: pipelines, standard output redirection, error output redirection, and input redirection.

Standard Output

When a command executes without errors, the resulting output is called standard output, also known as STDOUT. By default, this output is sent to the terminal where the command is run.

We can redirect standard output from the command so it goes to a file instead of the terminal using the greater-than symbol (>) followed by the target file. For example, the command ls ~ lists files in the home directory. To save this list to a text file:

ls ~ > /tmp/home.txt

The contents of home.txt would then look like this:

cat /tmp/home.txt

Desktop
Documents
Downloads
Music
Pictures
Public
Templates
Videos

Using a single > creates a new file or overwrites an existing one. Using two greater-than symbols >> also creates a file if it doesn't exist but appends output to the end if the file already exists. For example, to append the output of the date command:

date >> /tmp/home.txt   
cat /tmp/home.txt

Desktop
Documents
Downloads
Music
Pictures
Public
Templates
Videos
Sun Mar 30 07:36:02 UTC 2025

Standard Error

In Linux, when a command encounters an error, it produces output known as standard error, also called stderr or STDERR. Like standard output, it is usually sent to the same terminal. The file descriptor number for standard error is 2.

For example, trying to run:

ls /junk

Will produce:

ls: cannot access /junk: No such file or directory

Since this output goes to stderr, using just > won't redirect it:

ls /junk > output

ls: cannot access /junk: No such file or directory

To redirect error messages, use the correct file descriptor:

ls /junk 2> /tmp/ls.err

Like with standard output, > will create or overwrite the file. To append to the error log instead:

ls /junk 2>> /tmp/ls.err

Some commands produce both stdout and stderr. For example:

find /etc -name passwd

/etc/pam.d/passwd
/etc/passwd
find: '/etc/ssl/private': Permission denied

You can redirect these into two separate files:

find /etc -name passwd > /tmp/output.txt 2> /tmp/error.txt

To verify:

cat /tmp/output.txt

/etc/pam.d/passwd/etc/passwd
cat /tmp/error.txt

find: '/etc/ssl/private': Permission denied

If you don't want to display or save error messages, redirect them to /dev/null. This file acts like a trash bin where all input disappears. Any output type can be redirected there, most commonly stderr:

find /etc -name passw 2> /dev/null

/etc/pam.d/passwd
/etc/passwd

Combining Standard Output and Error

To redirect both stdout and stderr into a single file, use either of the following methods:

ls > /tmp/ls.all 2>&1
ls &> /tmp/ls.all

Both commands create the file /tmp/ls.all containing both standard output and error messages. In the first command, 2>&1 means "send stderr to the same place as stdout." The second uses &> as shorthand for "redirect all output."

Standard Input

Standard input, also known as stdin or STDIN, typically comes from the keyboard and is entered by the user running the command. While most commands can read input from files, some expect the user to type input manually using the keyboard.

One common way to use text files as standard input is by creating script files. Scripts are simple text files that are interpreted by the shell when given the appropriate permissions and start with #!/bin/sh in the first line, which tells the shell to interpret the script as standard input:

cat examplescriptfile.sh

#!/bin/sh
echo HelloWorld

When a script file is called in the command line using the ./ syntax, the shell executes all commands in the script file and returns the result to the terminal window or wherever the output is directed:

./examplescriptfile.sh

HelloWorld

In some cases, it’s useful to redirect standard input so that it comes from a file instead of the keyboard. A good example where input redirection is desirable is the tr command. The tr command translates characters by reading from standard input, converting one set of characters to another, and writing the transformed text to standard output.

For example, the following tr command takes input from the user (via keyboard) to convert all lowercase letters to uppercase:

tr 'a-z' 'A-Z'

hello
HELLO

The tr command will not stop reading from standard input unless it is terminated. You can do this by pressing Ctrl+D.

The tr command does not accept a filename as a command-line argument. To perform translation using a file as input, use input redirection. To do this, enter the command with its parameters and arguments, followed by the less-than symbol < and the file path to be used for input. For example:

cat Documents/animals.txt

1 retriever
2 badger
3 bat
4 wolf
5 eagle
tr 'a-z' 'A-Z' < Documents/animals.txt

1 RETRIEVER
2 BADGER
3 BAT
4 WOLF
5 EAGLE

Command Pipelines

Command pipelines are often used to efficiently apply filter commands. In a command pipeline, the output of one command is sent to another command as input. In Linux and most operating systems, the vertical bar | represents a pipeline between two commands.

For example, if the output of the history command is very long, you can send it to the less command to display one page at a time:

history | less

An even better example is taking the output of history and filtering it using the grep command. In the following example, the text output from history is redirected to grep as input. The grep command matches lines containing "ls" and sends the result to standard output:

history | grep "ls"

1  ls ~ > /tmp/home.txt
5  ls l> /tmp/ls.txt
6  ls 1> /tmp/ls.txt
7  date 1>> /tmp/ls.txt
8  ls /junk
9  ls /junk > output
10 ls /junk 2> /tmp/ls.err
11 ls /junk 2>> /tmp/ls.err
14 ls > /tmp/ls.all 2>&1
15 ls &> /tmp/ls.all
16 ls /etc/au* >> /tmp/ls.all 2>&1
17 ls /etc/au* &>> /tmp.ls.all
20 history | grep "ls"

Command pipelines become especially powerful when combining three or more commands. For example, view the contents of the os.csv file in the Documents directory:

cat Documents/os.csv

1970,Unix,Richie
1987,Minix,Tanenbaum
1970,Unix,Thompson
1991,Linux,Torvalds

The following command line extracts some fields from os.csv using the cut command, then sorts those lines using sort, and finally removes duplicates using uniq:

cut -f1 -d',' Documents/os.csv | sort -n | uniq

1970
1987
1991

The tee Command

The tee command splits the output of a command into two streams: one is directed to standard output (displayed in the terminal), and the other is written to a file.

The tee command is useful for logging the output of a command or script. For example, to record the execution time of a process, start with the date command and copy the output to a file timer.txt:

date | tee timer.txt

Tue Apr 2 02:21:24 UTC 2025

The timer.txt file now contains a copy of the date, the same output as shown above:

cat timer.txt

Tue Apr 2 02:21:24 UTC 2025

To append the time to the end of timer.txt, use the -a option:

date | tee -a timer.txt

Tue Apr 2 02:28:43 UTC 2025

To run multiple commands as one line, use the semicolon ; as a separator:

date | tee timer.txt; sleep 15; date | tee -a timer.txt

Tue Apr 2 02:35:47 UTC 2025
Tue Apr 2 02:36:02 UTC 2025

The command above displays and writes the first date output, pauses for 15 seconds, then displays and writes the second date output. The timer.txt file now contains a persistent execution log.

The xargs Command

Command options and parameters are typically specified directly in the command line as arguments. Alternatively, you can use the Linux xargs command to collect arguments from another input source (such as a file or standard input) and pass them to a command. xargs can be called directly and will accept any input:

xargs

Hello
There

To exit xargs, press Ctrl+C.

By default, xargs passes input to the echo command if no other command is explicitly given. After pressing Ctrl+D, xargs sends the input to echo:

Pressing Ctrl+D after exiting xargs with Ctrl+C will also exit the current shell. To send input to echo without exiting the shell, press Ctrl+D during xargs execution.

The Linux xargs command is most useful when used in a pipeline. The following example uses the touch command to create four files. The file names are 1a, 1b, 1c, and 1d, based on the output from the echo command:

echo '1a 1b 1c 1d' | xargs touch
ls

1a  1c  Desktop Downloads  Pictures  Templates  timer.txt
1b  1d  Documents  Music   Public    Videos

Conclusion

We’ve reviewed input/output stream redirection in Linux: standard output redirection, error output redirection, input redirection, and pipelines. Understanding these capabilities makes working with bash scripts easier and helps efficiently administer servers running Linux-based operating systems.

Linux
10.04.2025
Reading time: 8 min

Similar

Linux

Installing and Configuring Grafana

Working with any IT project becomes much easier when the administrator has a wide range of metrics and monitoring data at their fingertips. It's even better when the data is presented in a clear and visual format. This is where tools like Grafana come in — an open-source solution designed to gather information from various sources and consolidate it into visual reports. Grafana supports multiple platforms — Windows, macOS, Linux (including popular distributions like Debian, Ubuntu, CentOS, Fedora, OpenSuse, or RedHat). It can work with databases such as SQLite, MySQL, and PostgreSQL. With so many options, administrators rarely need to adapt the solution to their environment. In this tutorial, we'll go over how to install Grafana, configure it, and work with dashboards. Installing Grafana on CentOS Stream When ordering a Linux VPS, users can install any Linux operating system. Usually, this is one of the common distributions like CentOS or Ubuntu. For this example, we'll assume the OS is already installed and ready for Grafana and other utility programs. Let's import the GPG keys: wget -q -O gpg.key https://rpm.grafana.com/gpg.key sudo rpm --import gpg.key Create a new official repository configuration: sudo nano /etc/yum.repos.d/grafana.repo Add the following content to the file: [grafana] name=grafana baseurl=https://rpm.grafana.com repo_gpgcheck=1 enabled=1 gpgcheck=1 gpgkey=https://rpm.grafana.com/gpg.key sslverify=1 sslcacert=/etc/pki/tls/certs/ca-bundle.crt Install the application: sudo dnf install grafana Enable autostart and launch Grafana: sudo systemctl enable grafana-server sudo systemctl start grafana-server Check the status to ensure Grafana is running: sudo systemctl status grafana-server You should see a message confirming that the service is loaded and active. This step is especially useful if someone previously worked with the server or installed a custom Linux build with bundled utilities. Installing Grafana on Ubuntu The process is similar: we install Grafana from the official repository after preparing the system to trust the source. Run these commands: wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add - sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main" sudo apt update sudo apt install grafana sudo systemctl enable grafana-server sudo systemctl start grafana-server sudo systemctl status grafana-server Firewall Configuration for Grafana By default, Grafana uses port 3000. Here's how to open it in different firewalls. For iptables: Add the rule: sudo iptables -A INPUT -p tcp --dport 3000 -m state --state NEW -j ACCEPT Save the rules so they persist after reboot: sudo service iptables save Restart iptables to apply changes: sudo systemctl restart iptables For firewalld: firewall-cmd --zone=public --add-port=3000/tcp --permanent systemctl reload firewalld Default Login and Password Grafana uses the default login/password: Username: admin Password: admin If forgotten, reset it with: grafana-cli admin reset-admin-password --homepath "/usr/share/grafana" new_password Data Sources and Plugin Installation Grafana supports numerous data sources: Prometheus, Graphite, OpenTSDB, InfluxDB, and more. It also allows plugin installations to enhance functionality. For example, to install the Zabbix plugin, run: grafana-cli plugins install alexanderzobnin-zabbix-app systemctl restart grafana-server After restart, go to Configuration > Plugins and find Zabbix. After you enable it, you can configure it under Data Sources. This same process applies to other plugins like Grafana PostgreSQL or Grafana Elasticsearch. Working with Grafana Dashboards The core of Grafana is dashboards — sets of panels that visually display data. Users can create their own dashboards by clicking New Dashboard and selecting panel types. Dashboard Types: Graph – multiple metrics in one panel. Stat – single metric graph. Gauge – speedometer-style display. Bar Gauge – vertical bar graph. Table – table with multiple metrics. Text – freeform text. Heatmap – heatmap display. Alert List – list of Grafana alerts. Dashboard List – list of favorite dashboards. You can also display logs from external sources using Grafana Logs, and export/import dashboards for reuse. For advanced control, refer to the official documentation. You can directly edit the grafana.ini file to change: Default ports Log storage paths Proxy settings User access controls Feature toggles Conclusion Grafana is a powerful and flexible monitoring solution. To fully unlock its potential, experiment with dashboards, try manual config via grafana.ini, and explore third-party plugins. As an actively developed project, Grafana remains one of the top data visualization and monitoring tools.
17 April 2025 · 4 min to read
Linux

How to Copy Files over SSH

The SSH (Secure Shell) protocol is a network protocol for remote command-line management of operating systems, widely considered the standard for remote access to *nix machines. It allows secure login to a server, remote command execution, file management (creating, deleting, copying, etc.), and more. Most cloud and hosting providers require SSH to access their services. In this article, we’ll look at how to copy files over SSH on both Windows and Linux systems. How SSH Works SSH can securely transmit any data (audio, video, application protocol data) through an encrypted communication channel. Unlike outdated and insecure protocols like Telnet and rlogin, SSH ensures data confidentiality and authenticity — essential for internet communications. Here’s how a secure connection between a client and server is established: TCP Connection Setup: By default, the server listens on port 22. Both sides share a list of supported algorithms (compression, encryption, key exchange) and agree on which to use. Authentication: To prevent impersonation, both parties verify each other's identities using asymmetric encryption (public/private key pairs). First, the server is authenticated. On the first connection, the client sees a warning with server details. Trusted server keys are stored in /home/<username>/.ssh/known_hosts. Key Generation: Once the server is verified, both sides generate a symmetric key to encrypt all data exchanged. User Authentication: This is done using either a password or a client-sent public key stored in /home/<username>/.ssh/authorized_keys on the server. The most popular implementation on Linux is OpenSSH, which comes pre-installed on most distributions (Ubuntu, Debian, RHEL-based, etc.). Clients like PuTTY or MobaXterm are used on Windows. Since Windows 10 and Server 2019, OpenSSH tools are also available natively. You can learn more about working with SSH in our tutorial. File Copying via SSH Two main utilities for copying files over SSH in Linux are scp and sftp. Both come with OpenSSH. SSH supports two protocol versions: 1 and 2. OpenSSH supports both, but version 1 is rarely used. Autocompletion Setup To enable Tab-based autocompletion when using scp, set up public key authentication: Generate a key pair: ssh-keygen You’ll see output like: Generating public/private rsa key pair. Enter file in which to save the key (/home/user/.ssh/id_rsa): Enter passphrase (empty for no passphrase): By default, your keys (id_rsa for private and id_rsa.pub for public) are saved to ~/.ssh/. Now copy the public key to the remote machine: ssh-copy-id [username]@[ip-address] After entering the user's password, you’ll see a message confirming the key was added. Secure Copy (SCP) For small data transfers (e.g., service configs), scp is best. Copy from local to remote: scp test.txt user@192.168.1.29:/home/user/ Copy multiple files: scp test1.txt test2.txt user@192.168.1.29:/home/user/ Copy from remote to local: scp user@192.168.1.29:/home/user/test.txt ~/ Copy directories: scp -r testdir user@192.168.1.29:/home/user/ Remote-to-remote copy: scp gendo@192.168.1.25:/home/gendo/test.txt user@192.168.1.29:/home/user/ Secure FTP (SFTP) SFTP is another utility included in OpenSSH. As of OpenSSH 9.0, scp now uses SFTP by default instead of the old SCP/RCP protocol. Unlike classic FTP, sftp transmits encrypted data over a secure tunnel. It does not require a separate FTP server. Example usage: sftp misato@192.168.1.29 sftp> ls sftp> lcd testdir/ sftp> get test.txt sftp> bye Graphical file managers like Midnight Commander and Nautilus use sftp. In Nautilus, the remote server appears like a local folder, e.g., user@ip. Copying Files Over SSH on Windows Use the pscp command-line tool from PuTTY to copy files on Windows. Copy to server: pscp C:\server\test.txt misato@192.168.1.29:/home/misato/ Copy from server: pscp misato@192.168.1.29:/home/misato/test.txt C:\file.txt List files on remote server: pscp -ls user@192.168.1.29:/home/misato Use quotes for paths with spaces: pscp "C:\dir\bad file name.txt" misato@192.168.1.29:/home/misato/ To get help, run: pscp Conclusion We’ve covered how to copy files to and from a server using the secure SSH protocol. If you work with cloud servers, understanding SSH is essential — it’s the standard method for remote access to *nix machines and a vital part of everyday DevOps and system administration.
14 April 2025 · 4 min to read
Linux

What is a Daemon in Computing?

The term daemon comes from a word in ancient Greek mythology that referred to an immaterial being influencing the human world. In computing, especially in UNIX-like operating systems, a daemon is a background process that runs without direct interaction from the user. It doesn’t depend on a terminal or user interface and typically starts with the system boot or under specific conditions. What is a Daemon The main function of a daemon is to provide specific services to other processes or users. For example, a daemon might listen on network ports waiting for connections, monitor system events and respond when certain conditions are met, manage scheduled jobs (like cron), send emails (sendmail), and more. In Windows, the closest equivalent to a daemon is a service. The difference lies mainly in how they're started, registered, managed, and configured within operating systems. However, their purpose is the same: to ensure continuous background operation of certain functions or services. Key Characteristics of a Daemon Runs in the background: Users typically don’t see the daemon’s interface; it doesn’t write to standard output (or redirect it to logs), nor does it request keyboard input. Autonomous: A daemon starts either at system boot when triggered by an init system (like systemd), or manually by a user (via scripts, cron, etc.). Long-lived: Ideally, a daemon runs indefinitely unless a critical error occurs or it receives an explicit stop signal. Isolated: Usually runs under a separate user/group account to minimize privileges, making services more secure and easier to manage. Logging: Instead of using standard input/output, daemons log information to log files or the system logger (journald, syslog, etc.), which is helpful for debugging and diagnostics. Daemons in Linux Historically, nearly all system background tasks in Linux are implemented as daemons. The OS includes dozens of them, each responsible for a specific function. Here are some examples: sshd (Secure Shell Daemon): Listens on port 22 (by default) and allows remote users to connect via encrypted SSH. Without sshd, remote terminal access is almost impossible. cron: A job scheduler daemon. It checks crontab entries and runs scripts or commands on a schedule, such as log cleanup, sending reports, system checks, etc. syslogd / rsyslog / journald: System logging daemons that collect messages from the kernel, utilities, other daemons, and apps, and save them in log files or the journal. NetworkManager or Wicd: Daemons that manage network settings — automating connections to wired/wireless networks, switching, configuring VPNs, and more. These daemons start at system boot and are registered with the system service manager (e.g., systemd). They run until the system is shut down or rebooted. Users interact with them indirectly — through config files, terminal commands (service, systemctl), or network requests (if the daemon provides HTTP/S, SSH, or another network interface). How to Create and Manage Daemons To implement a daemon, follow these steps: Forking the process: The parent process calls fork() and continues running the daemon code in the child process. Detach from controlling terminal (setsid): To avoid user interference (e.g., closing the terminal), the daemon calls setsid() to start a new session and become its leader. Close standard input/output descriptors: Since the daemon shouldn't write to the screen or wait for input, stdin, stdout, and stderr are closed or redirected to log files. Handle signals and logging: To support graceful shutdown or config reloads, the daemon must handle signals (SIGTERM, SIGHUP, etc.). Logging is usually done via syslog or files. Main loop:  After initialization, the daemon enters its main loop: waiting for events, handling them, and repeating until stopped. Let’s see how to create a daemon on Ubuntu 22.04 using a Hostman cloud server 1. Write the Daemon in C Create a file called mydaemon.c and insert the following code: #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <syslog.h> int main() { // Open syslog openlog("mydaemon", LOG_PID, LOG_DAEMON); syslog(LOG_NOTICE, "Daemon started"); // Main infinite loop while (1) { // Your background tasks: monitoring, queue handling, etc. syslog(LOG_NOTICE, "Performing task..."); sleep(60); } // If we ever exit the loop syslog(LOG_NOTICE, "Daemon stopped"); closelog(); return 0; } 2. Compile the Program First, update your packages: sudo apt update && sudo apt upgrade Install the GCC compiler if not already installed: sudo apt install gcc Compile the daemon: gcc mydaemon.c -o mydaemon 3. Move the Executable Move the binary to /usr/local/bin/, a standard location for custom utilities: mv mydaemon /usr/local/bin/mydaemon 4. Create a systemd Service Create a unit file called mydaemon.service: sudo nano /etc/systemd/system/mydaemon.service Insert the following content: [Unit] Description=My Daemon After=network.target [Service] Type=simple ExecStart=/usr/local/bin/mydaemon Restart=on-failure [Install] WantedBy=multi-user.target Explanation of the fields: Description: Description shown in systemctl status. After=network.target: Ensures the daemon starts after the network is up. Type=simple: The daemon doesn’t fork, it runs as a single process. ExecStart: Path to the daemon executable. Restart=on-failure: Restart automatically if the daemon crashes. WantedBy=multi-user.target: Makes the service start in the standard multi-user environment. 5. Start and Monitor the Daemon sudo systemctl daemon-reload # Reload systemd configuration sudo systemctl start mydaemon # Start the daemon sudo systemctl status mydaemon # Check its status If everything works, the status will show active. To view logs: journalctl -u mydaemon.service -e Examples of Daemon Usage Web Servers Their job is to listen on a network port (usually 80 or 443), accept HTTP/HTTPS requests, generate a response (an HTML page, JSON data, etc.), and send the result back to the client. In most cases, a web server starts with the system and continues running until the server is shut down or a stop command is issued (e.g., systemctl stop nginx). Database Daemons MySQL/MariaDB, PostgreSQL, MongoDB — all of these are also daemons. They start with the system and continue running in the background, accepting requests from client applications or web services. These daemons typically log activity, support configuration via files, and are managed using special utilities (or systemd). Job Schedulers (cron, atd) The cron daemon checks the schedule table (crontab) and runs programs at the times or intervals specified by the user. This makes it possible to automate backups, system updates, health checks, and many other routine tasks. atd is a similar daemon but executes tasks only once at a specific time (unlike cron, which runs tasks on a regular schedule). Access and Control Services (sshd, xrdp) sshd (Secure Shell Daemon) provides remote access via the SSH protocol. xrdp enables remote desktop connections using the RDP protocol. It acts as a daemon that listens for network connections on a specified port. Init System Daemons (systemd, init, Upstart) In modern systems, the role of the "main daemon" is fulfilled by systemd (replacing the older SysV init system). systemd is the first process to start after the kernel and is responsible for launching and managing all other services and processes. It starts them in parallel and handles their dependencies. Simply put, systemd is itself a daemon that “orchestrates” all others in the system. Advantages and Disadvantages of Daemons Advantages: Automation: Daemons enable system behavior to be automated — from responding to network requests to scheduling tasks — without user intervention. Isolation: Running under separate user/group accounts and detaching from terminals enhances security by limiting potential damage in case of compromise. Continuous Operation: A daemon can keep servicing requests (like a web server) without interruption even if the user logs out or the console is closed. Manageability: Linux provides system tools (e.g., systemd, init scripts) to centrally manage all daemons: starting, stopping, restarting, and logging. Disadvantages: Debugging Complexity: Since daemons run in the background and don’t output to the console, debugging requires thorough logging and more complex setups (debug flags, tracing, etc.). Security Risks: If a daemon runs with elevated privileges (e.g., as root), any vulnerability can potentially compromise the entire system. It's best to run daemons under limited accounts. Dependency Management: Some daemons may fail if, for example, they need network access before the network is up. Modern init systems solve this, but with classic SysV init scripts, this used to be a common issue. Increased Resource Usage: Any constantly running background process consumes system resources (memory, CPU time). If there are too many daemons, this can impact performance, especially on systems with limited resources. Conclusion Daemons are central to Linux operating systems' architecture, offering vast automation and background services capabilities. They allow administrators to flexibly configure network operations, scheduled tasks, logging, security systems, and many other components. Writing your own daemon requires understanding processes, signals, system calls, and careful attention to logging and security. Modern init systems (especially systemd) have simplified daemon management and service logic, making the creation of custom services more structured and flexible. However, it remains a complex field that demands careful design, debugging, and ongoing maintenance.
11 April 2025 · 8 min to read

Do you have questions,
comments, or concerns?

Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
Email us
Hostman's Support