Iterating With Loops (Bash)

Imagine having to edit 10,000 lines of text within a CSV file, this would be a shit load of work if done manually, which is where looping comes in. When a task or a series of tasks needs to be repeated, it can be put inside a loop.

In this guide, we would look at for, while, and until loops in bash. The for loops through a list of words, while and until executes until a condition is either true or false.

Starting With...

For Loops

The for is an inbuilt shell keyword, you can verify by doing the following:

user@server:~$ type for
for is a shell keyword

Since, this is a shell keyword, you can either use it in a script and even directly at the command line. The for loop lets you iterate (perform again) over a series of words within a string or a list of files. Consider the following example:

for f in $( ls ); do
    echo "file name is" : $f
done

We started with the for keyword, and we then declare f to be the variable that will take different values contained in $( ls ), what we are basically doing is reading from the list on the right to populate the variable parameter on the left.

Imagine, you have files in your directory with the following name:

• cat
• dog
• hen
• egg

The ls command (The ‘ls’ is used to list information about files, e.g file names) would pick the first file name, in this case, cat, it would store it in the f variable, in the second line, we are echoing the output of the f variable, and lastly, the done keyword indicates that the code that used the value of $f has finished and $f can take a new value, so, the next value would be 'dog', and it would repeat the same process until it gets to 'egg', that is the idea of a for loop, as long as there are items to be processed in the list, the loop will execute until the list is exhausted.

This is the output of the above command:

user@server:~$ for f in $( ls ); do
> echo "file name is" : $f
> done
file name is : cat
file name is : dog
file name is : egg
file name is : hen

Take another example:

for i in 1 2 3 4; do
   echo "Number $i"
done

The above command looks pretty easy, the i variable would pick and store the first item, in this case, '1', in the second line, we are echoing the output of the i variable, and lastly, the done keyword indicates that the code that used the value of $i has finished and $i can take a new value, so, the next value would be '2', and it would repeat the same process until it gets to '4'. Here is the output:

user@server:~$ for i in 1 2 3 4; do
> echo "Number $i"
> done
Number 1
Number 2
Number 3
Number 4

You can also use brace expansion to speed things up, it can save you a ton of time when working with a range of terms.

For example, to specify an interval of the number we would be using, we can do the following:

user@server:~/bin$ echo {1..9}
1 2 3 4 5 6 7 8 9

In for loop, you do:

for i in {1..9}; do
   echo "Number $i"
done

To create a number between 1 and 10, counting by two we do:

user@server:~$ echo {1..10..2}
1 3 5 7 9

In for loop, you do:

for i in {1..10..2}; do
   echo "Number $i"
done

To create by 3, you do:

user@server:~/cat$ echo {1..10..3}
1 4 7 10

In for loop, you do:

for i in {1..10..3}; do
   echo "Number $i"
done

Here are more examples of for loops:

for u in james joy john william ; do
    useradd -d /home/$u -m $u
    echo "$u:defaultpass" | chpasswd
    passwd -e $u
done

In the above command, we read from the list on the right to populate the variable parameter on the left, in this case, james, joy and so on. Practically, this is what the above script is doing:

• Create the user james, and create the user home directory /home/james
• Set the password for james
• Expire the password so it will need to be reset on the first login for the user james

We then loop back and repeat the process for the user joy, john, and william.

Using Dynamic List With for loop

So far, we haven't really been discussing about dynamic list when using for loop.  To create dynamic lists, we use globbing techniques to populate the list. Consider the following example:

for f in * ; do
ls -la "$f"
done

The above command would list all the files in the current directory

When a list is generated, such as with file globbing, we should quote the expansion of the variable parameter. Without the quotes, it is
possible that a space will get included that will cause the command to fail.

You can also isolate a specific file names, e.g book:

for f in book* ; do
ls -la "$f"
done

This would list any files that starts with book, to isolate a specific extension, e.g jpg, you can do:

for f in *.jpg ; do
ls -la "$f"
done

Controlling the Loop with Break and Continue

What if at some point, we need to exit the loop prematurely or perhaps exclude certain items from processing, e.g you might only want to process only a file without touching the directory. For this case, we have loop control keywords, such as break and continue.

The break keyword is used to break out of a loop, be it a for or while loop, the continue keyword is used to end the current iteration in a loop, but continue with the next.

Let's take a couple of examples, a for loop to only process files:

for f in * ; do
[ -f "$f" ] || continue
chown root:root "$f"
done

The above script changes the user and group and all the files in the current directory, this is what we are doing:

• The * search will return all files
• [ -f "$f" ] - this ensures we are only processing files, and if it is not a file e.g if it is a directory, the continue keyword would skip the remaining commands inside the body of the enclosing loop for the current iteration and passes program control to the next iteration of the loop.
• If it is a file, we change the user and group to root

If we need to run the loop until we found a file and then exit the loop immediately,  we can adjust the code so that we can iterate through each file. If the file is a file then we exit the loop with the break keyword:

for f in * ; do
    [ -f "$f" ] && break
    done
echo "File name is: $f"

Output:

user@server: for f in * ; do
>     [ -f "$f" ] && break
>     done
root@blog: echo "File name is: $f"
File name is: book.jpg

To Print all file names, we again, use the continue keyword:

for f in * ; do
    [ -f "$f" ] || continue
    file_name="$file_name $f"
done
echo File Name are: "$file_name"

The above script print all the files founded in the current directory, this is what the code is doing:

• The * search will return all files
• [ -f "$f" ] - this ensures we are only processing files, and if it is not a file e.g if it is a directory, the continue keyword would skip the remaining commands inside the body of the enclosing loop for the current iteration and passes program control to the next iteration of the loop.
• file_name="$file_name $f" - This append the file name to file_name variable, and once we exit the list we print all the complete file names

Output

user@server: for f in * ; do
>     [ -f "$f" ] || continue
>     file_name="$file_name $f"
> done
user@server: echo File Name are: "$file_name"
File Name are:  book.jpg book.txt gg.txt out.txt

While loops and Until loops

We have previously looked at for loop, where we use it in looping through a list of words or items, and now we are unto the while and until loops. The while and until loops is based on whether the condition is true or false.

A while loops while the condition is true and until loops while the condition is false. Consider the following example using while loop:

count=10
    while (( count >= 0 )) ; do
    echo -e "$count \c"
    ((count--)) ; done
echo

The above example counts and print from 10 through 0, it does the following:

• We set an initial value of where the count should begin, in this case, we want the count to begin at 10; count =10
• We then say, "while the count is greater and equals to 0, do the next step, for the first run, it is true, we have 10, so...
• It prints 10
• count-- reduces the count by 1, you can also write it as count=count - 1

We then repeat it until it gets to zero, this is the output:

user@server:~$ whileloop.sh
10 9 8 7 6 5 4 3 2 1 0

Let's rewrite the code with an until loop:

count =10
    until (( count < 0 )) ; do
    echo -e "$count \c"
    (( count -- )) ; done
echo

The only thing we changed in the above script is the logic, we are saying, until the variable of count is lesser than zero, keep looping.