Episode two of the Linux Tools series. Another must-have for your linux toolbox is sed, which stands for stream editor, and is pronounced like the english word "said". To really wield this tool like a pro, it's helpful to already have a good understanding of regular expressions and streams, but the learn-as-you-go method works too.

The linux man page for sed describes it thus: A stream editor is used to perform basic text transformations on an input stream (a file or input from a pipeline).  While in some ways similar to an editor which permits scripted edits (such as ed), sed works by making only one pass over the input(s), and is consequently more efficient. But it is sed's ability to filter text in a pipeline which particularly distinguishes it from other types of editors.

One of the simplest and most common uses of sed is to replace a word or sequence of words with something else. For example, let's say you have a config template (cust.conf) which contains placeholders that need to be replaced with something specific to a particular customer; maybe you have "[Customer_Name]" in several places in the config template and you want to replace that with the actual customer name. Here's the command to do that:

sed -i -e "s/\[Customer_Name\]/The Real Customer Name/g" /path/to/cust.conf

Let's break that down...

The Switches

The -i switch means "in-place", so it changes the original file. You could also leave that off and pipe the output of sed to a new file.

The -e stands for expression, and in the example is just a single regular expression (we could have used -f and then the name of a script file instead).

The Expression

The s in the expression means we are substituting matches of what's between the first two delimeters (in this case, the "/", but it could be pretty much anything you want), with what's between the second and third delimeter. The "[" and "]" have special meaning in regular expressions, so we must escape them with the "\". The g stands for "globally" which means we will change every instance of a match in the stream, if we left it off we would only change the first occurence of a match.

The Stream

Finally we provide the path to the file on which we want sed to operate; this doesn't have to be a single file, it could have been any valid stream.

Wrapping It Up

With all that in mind, what if we want to perform the action on all conf files in a directory? Here's another example that demonstrates using a different delimeter and operating on many files:

sed -i -e "s~\[Customer_Name\]~The Real Customer Name~g" /path/to/*.conf

or, using a script file and piping output to a new file:

sed -f /path/to/script_file /path/to/cust.conf > /path/to/new.conf

This really only scratches the surface of what you can do with sed. Some more exhaustive examples can be found by a simple web search. Here are some of the ones I like:

https://gist.github.com/r2k0/1152840

https://www.geeksforgeeks.org/sed-command-in-linux-unix-with-examples/