Another powerful filter in GNU/Linux is sed (stream editor), which is a program for performing basic editing tasks on the output of another program or on a file.
sed or stream editor can be used to perform basic text modification on an input stream (a file or input from a pipeline). sed' works by making only one pass over the input(s), and is consequently more efficient than other filter programs.
When used as is, the output is only produced on the screen, the actual file is not changed at all, so, if you want to change the file, then you can use a redirection into another file or by specifying the edit option, which would take a look at below
Its syntax is as follows:
sed action [files]
You specify an action that you want to perform, and on what file or files you want to specify that action on. If you don't specify a file, then it performs the action on the input from either the standard input or a piped information from another program.
sed can perform several actions at a time, e.g
- sed -e action1 -e action2 [files]
- sed -f scriptfile [files] - You take all the action you want to perform and put it in a script file, the f option would open the script file, examine all the actions in it, and then it performs all those actions on the given input.
Actions specified on the command line are almost always enclosed in single quotes to prevent shell interpretation of special characters, so, when using the sed program, you need to take note of this.
Let's take a look at some of the actions you can do with sed.
Substitution In sed
s/foo/bar/- This is a text substitution, and it changes the first occurrence of foo on each line to bar
This is how you can do it practically:
sed 's/Hello/World/' textfile.txt
The above command would replace all instances of "Hello" with "World" in "testfile.txt'
'sed' writes output to standard output (screen) by default, so, whatever you do it won't replace the final process. Use '-i' to edit files directly instead of printing to standard output (screen) only.
The following command modifies 'testfile.txt' and does not produce any output:
sed -i 's/Hello/World' textfile.txt
By default 'sed' prints all processed input ( with the exception of input that has been modified or deleted by commands such as 'd'). You can use '-n' to suppress output, and the 'p' command to print specific lines.
The following example prints only line 20 of the input file:
sed -n '20p' textfile.txt
The following example would print line 1 to 3:
user@blog:~$ sed -n '1,3 p ' /etc/passwd root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin bin:x:2:2:bin:/bin:/usr/sbin/nologin
To perform several actions, you can do the following:
sed -e 's/Hello/World' -e 's/foo/bar/' textfile.txt
To change the shell of a a pi user from /bin/bash to /usr/bin/sh, we do:
user@blog:~$ sed -n ' /^pi/ s/\/bin\/bash/\/usr\/bin\/sh/p ' /etc/passwd user@blog:~$ pi:x:0:0:pi:/pi:/usr/bin/sh
The issue was that the default delimiter is a forward slash so I escaped each
forward slash, so, you just need to add a backward slash for the escape to come through, I have highlighted it with red.
This is one option, another way to do this is to know that the first delimiter we use defines the delimiters. Meaning, you can use any character as a delimiter. We can use a dash (-) to specify the delimiter:
user@blog:~$ sed -n ' /^root/ s-/bin/bash-/usr/bin/sh-p ' /etc/passwd user@blog:~$ pi:x:0:0:pi:/pi:/usr/bin/sh
Alternatively, you can also use an @ symbol, so, its your call:
sed -n ' /^pi/ s@/bin/bash@/usr/bin/sh@p ' /etc/passwd
Don't forget, this would only print to the standard out, so, if you want to edit the file directly, then you use the -i option:
sed -i ' /^pi/ s@/bin/bash@/usr/bin/sh@p ' /etc/passwd
Working on a specific range
It is possible to specify the range of line number, this way, you can restrict the level of what the process would do:
For example, to print a line that only has the word root, you can do:
user@blog:~$ sed -n '/^root/ p' /etc/passwd root:x:0:0:root:/root:/bin/bash
The above can also be done in the grep command
If you want to perform the action on line 1 - 2, you do:
user@blog:~/bin$ sed 1,2's/foo/bar/' text bar bar foo foo
As you can see, it only restricts the process to line 1 and 2.
What if you want to perform an action on a specific line all the way to the last line, then you can do:
user@blog:~/bin$ sed '3,$''s/foo/bar/' text foo foo bar bar
The $ refers to the last line, so, I am telling sed to work from line 3 all the way to the last line. Notice that I am using a single quote, this is to prevent shell interpretation of special characters.
Deleting Lines In sed
You can also delete a specific line in sed, so, to delete the line 5 to 10, you do:
sed -i '5,20d' textfile.txt
or if you want to delete all lines with a specific word in it, you do:
sed -i '/foo/d' textfile.txt
It is also possible to delete all lines except for lines that match certain criteria:
So, for example, if you don't want to delete from line 10 to the end of the line in the file, you do:
sed -i '5,$!d' textfile.txt
This would delete all the lines that do not fall in that range, so, in our case, it would only delete lines 1 to 4.
Or you can also specify that you don't want a line that contains a certain word to be deleted, e.g
sed -i '/foo/!d' textfile.txt
We are saying delete every line, except the lines that have the foo word included.
To conclude this guide, let me show you an interesting thing you can do with sed, look at the following script:
#!/bin/sed -f s/myusername/newusername/ /bar/d
This is very similar to a shell script, in short, it is a shell script, just with a sed interpreter, when you run this program, it will be an executable script that you can run like any other executable script, but instead of being interpreted by the shell program, it will be interpreted by the sed program.
As you can see the lines of code in the script are sed actions rather than shell script command, so what are we actually doing?
We are basically substituting "myusername" with "newusername", and we are also deleting any lines that we find in the input that contains "bar"
I'll save the script as sedscript, so, let me show you an example of how we can utilize it:
lastlog | sedscript
I am piping the output of lastlog to the sedscript, and it would perform the action and processes anything that it matches.