When writing Bash scripts, you will sometimes find yourself in situations where you need to read a file line by line. For example, you may have a text file containing data that should be processed by the script.
In this tutorial, we will discuss how to read a file line by line in Bash.
Reading a File Line By Line Syntax
The most general syntax for reading a file line-by-line is as follows:
while IFS= read -r line do echo "$line" done < input_file
or the equivalent single-line version:
while IFS= read -r line; do echo $line; done < input_file
How does it work?
The input file (
input_file) is the name of the file you want to be open for reading by the
read command. The
read command reads the file line by line, assigning each line to the
line variable. Once all lines are processed the while loop will terminate. The internal field separator (
IFS) is set to the null string to preserve leading and trailing whitespace which is the default behavior of the
Reading a File Line By Line Examples
Let’s take a look at the following example. Suppose, we have a file named
distros.txt containing a list of some of the most popular Linux distributions and their package managers separated with comma (
Ubuntu,apt Debian,apt CentOS,yum Arch Linux,pacman Fedora,dnf
To read the file line by line you would run the following code in your terminal:
while IFS= read -r line do echo "$line" done < distros.txt
while IFS= read -r line do if [[ "$line" == *"apt"* ]]; then echo "$line" fi done < distros.txt
When reading file line by line, you can also pass more than one variables to the read command which will split the line into fields based on the
IFS. The first field is assigned to the first variable, the second to the second variable, and so on. If there are more fields than variables, the leftover fields are assigned to the last variable.
In the following example, we are setting IFS to a comma (
,) and passing two variables
pm to the read command. Everything from the beginning of the line until the first comma will be assigned to the first variable (
distro) and the rest of the line will be assigned to the second variable (
while IFS=, read -r distro pm do echo "$pm" is the package manager for "$distro" done < distros.txt
apt is the package manager for Ubuntu apt is the package manager for Debian yum is the package manager for CentOS pacman is the package manager for Arch Linux dnf is the package manager for Fedora
Alternative File Reading Methods
Using a Process Substitution
Process substitution allows you to pass output from command as a filename:
while IFS= read -r line do echo "$line" done < <(cat input_file )
Using a Here String
Here String is a variant of Here document. The string
(cat input_file ) will keep the newlines:
while IFS= read -r line do echo "$line" done <<< $(cat input_file )
Using File descriptor
You can also provide the input to the loop using a file descriptor:
while IFS= read -r -u9 line do echo "$line" done 9< input_file
When working with file descriptors use a number between 4 and 9 to avoid conflict with shell internal file descriptors.
In Bash, we can read a file line-by-line by providing the filename as an input to a while read loop.
If you have any questions or feedback, feel free to leave a comment.