Sed tutorial unix pdf

Let me first cover the 'p' and other pattern flags. These flags can specify what happens when a match is found. Let me describe them.

Sed , by default, is the same way. If you tell it to change a word, it will only change the first occurrence of the word on a line. You may want to make the change on every word on the line instead of the first. For an example, let's place parentheses around words on a line. The current version of Solaris's sed as I wrote this can get unhappy with patterns like this, and generate errors like "Output line too long" or even run forever.

I consider this a bug, and have reported this to Sun. As a work-around, you must avoid matching the null string when using the "g" flag to sed. Sed only operates on patterns found in the in-coming data.

That is, the input line is read, and when a pattern is matched, the modified output is generated, and the rest of the input line is scanned.

The "s" command will not scan the newly created output. If a second "s" command is executed, it could modify the results of a previous command.

I will show you how to execute multiple commands later. Specifying which occurrence With no flags, the first matched substitution is changed. With the "g" option, all matches are changed.

There is an easier way to do this. You can add a number after the substitution command to indicate you only want to match that particular pattern. For instance, if you want to leave the first word alone, but change the second, third, etc. Without the space, sed will run a long, long time. Note: this bug is probably fixed by now. This is because the number flag and the "g" flag have the same bug.

It can be any number from 1 to If it makes a substitution, the new text is printed instead of the old one. If you use an optional argument to sed, "sed -n," it will not, by default, print any new lines. I'll cover this and other options later. When the "-n" option is used, the "p" flag will cause the modified line to be printed. With it, you can specify a file that will receive the modified data.

You must have exactly one space between the w and the filename. You can also have ten files open with one instance of sed. This allows you to split up a stream of data into separate files.

Using the previous example combined with multiple substitution commands described later, you could split a file into ten pieces depending on the last digit of the first number.

You could also use this method to log error or debugging information to a special file. Combining substitution flags You can combine flags when it makes sense. Please note that the "w" has to be the last flag. Arguments and invocation of sed previously, I have only used one substitute command.

A sed guru never uses two processes when one can do. If you give sed one argument, it must be a command, and sed will edit the data read from standard input. If there is more than one argument to sed that does not start with an option, it must be a filename. The sed substitute command changes every line that starts with a " " into a blank line. Grep was used to filter out delete empty lines. Wc counts the number of lines left.

Sed has more commands that make grep unnecessary. And grep -c can replace wc -l. I'll discuss how you can duplicate some of grep 's functionality later. Let me clarify this. When used, only lines that match the pattern are given the command after the address. Please note that if you do not include a command, such as the "p" for print, you will get an error. This will help you distinguish between flags that modify the pattern matching, and commands to execute after the pattern is matched.

This is done by placing a backslash at the end of each line:! The long argument of the command is sed --version sed -h The -h option will print a summary of the sed commands. The long argument of the command is sed --help A sed interpreter script Another way of executing sed is to use an interpreter script.

Create a file that contains:! On the Sun when I wrote this , you can have several comment lines anywhere in the script. Modern versions of Sed support this. If the first line contains exactly " n" then this does the same thing as the "-n" option: turning off printing by default.

This could not done with a sed interpreter script, because the first line must start with "! It worked when I first wrote this Note that "! However, "! Passing arguments into a sed script Passing a word into a shell script that calls sed is easy if you remembered my tutorial on the UNIX quoting mechanism.

To review, you use the single quotes to turn quoting on and off. A simple shell script that uses sed to emulate grep is:! Using sed in a shell here-is document You can use sed to prompt the user for some parameters and then create a file with those parameters filled in. You could create a file with dummy values placed inside it, and use sed to change those dummy values. A simpler way is to use the "here is" document, which uses part of the shell script as if it were standard input:!

If you type in "," the next line will be: The value is I admit this is a contrived example. This example does the same thing:! Better form would be to put double quotes around the evaluation of the value:! It's really quite simple. Each line is read in. Each command, in order specified by the user, has a chance to operate on the input line. After the substitutions are made, the next command has a chance to operate on the same line, which may have been modified by earlier commands.

If you ever have a question, the best way to learn what will happen is to create a small example. If a complex command doesn't work, make it simpler. If you are having problems getting a complex script working, break it up into two smaller scripts and pipe the two scripts together. Addresses and Ranges of Text You have only learned one command, and you can see how powerful sed is.

However, all it is doing is a grep and substitute. That is, the substitute command is treating each line by itself, without caring about nearby lines. What would be useful is the ability to restrict the operation to certain lines. Some useful restrictions might be: Specifying a line by its number. Specifying a range of lines by number. All lines containing a pattern. All lines from the beginning of a file to a regular expression All lines from a regular expression to the end of the file.

All lines between two regular expressions. Sed can do all that and more. Every command in sed can be proceeded by an address, range or restriction like the above examples. The restriction or address immediately precedes the command: restriction command Restricting to a line number The simplest restriction is a line number. Sed uses the same convention, provided you terminate the expression with a slash. It isn't necessary, but without it the command is harder to fathom.

Sed does provide a few extra options when specifying regular expressions. But I'll discuss those later. If the expression starts with a backslash, the next character is the delimiter. It's hopeless. There is a lesson here: Use comments liberally in a sed script.

You may have to remove the comments to run the script under a different older operating system, but you now know how to write a sed script to do that very easily! Comments are a Good Thing. You may have understood the script perfectly when you wrote it. But six months from now it could look like modem noise. And if you don't understand that reference, imagine an 8-month-old child typing on a computer. Ranges by line number You can specify a range on line numbers by inserting a comma between the numbers.

Assuming a " " starts a comment, you can search for a keyword, remove all comments until you see the second keyword. The second pattern turns off the flag. If the "start" and "stop" pattern occurs twice, the substitution is done both times. If the "stop" pattern is missing, the flag is never turned off, and the substitution will be performed on every line until the end of the file.

You should know that if the "start" pattern is found, the substitution occurs on the same line that contains "start. That is, the next line is read and the substitute command is checked. If it contains "stop" the switch is turned off. Switches are line oriented, and not word oriented. You can combine line numbers and regular expressions. I will show you later how to restrict a command up to, but not including the line containing the specified pattern.

It is in Operating in a pattern range except for the patterns But I have to cover some more basic principles. Before I start discussing the various commands, I should explain that some commands cannot operate on a range of lines. I will let you know when I mention the commands. In this next section I will describe three commands, one of which cannot operate on a range. Delete with d Using ranges can be confusing, so you should expect to do some experimentation when you are trying out a new script.

A useful command deletes every line that matches the restriction: "d. Wc can count the lines, and expr can subtract 10 from the number of lines.

A Bourne shell script to look at the last 10 lines of a file might look like this:! Or it can be a single regular expression. You would have to explicitly type in the tab. Note the order of operations above, which is in that order for a very good reason. Comments might start in the middle of a line, with white space characters before them. Therefore comments are first removed from a line, potentially leaving white space characters that were before the comment.

The second command removes all trailing blanks, so that lines that are now blank are converted to empty lines. The last command deletes empty lines. Together, the three commands remove all lines containing only comments, tabs or spaces. This demonstrates the pattern space sed uses to operate on a line. The actual operation sed uses is: Copy the input line into the pattern space. Apply the first sed command on the pattern space, if the address restriction is true.

Repeat with the next sed expression, again operating on the pattern space. When the last operation is performed, write out the pattern space and read in the next line from the input file. Printing with p Another useful command is the print command: "p.

The command sed 'p' will duplicate every line. Another way of duplicating head 's functionality is to print only the lines you want. Sometimes you need to perform an action on every line except those that match a regular expression, or those outside of a range of addresses. The "! The "-v" option to grep prints all lines that don't contain the pattern.

As you may have noticed, there are often several ways to solve the same problem with sed. This is because print and delete are opposite functions, and it appears that "! The following table, which shows the results of by test, demonstrates the difference: Relations between d, p, and!

Of course for files longer than 20 lines, you will get more than 10 lines for the last two cases. The q or quit command There is one more simple command that can restrict the changes to a set of lines. It is the "q" command: quit. This command is most useful when you wish to abort the editing after some condition is reached.

The "q" command is the one command that does not take a range of addresses. Obviously the command sed '1,10 q' cannot quit 10 times. Instead sed '1 q' or sed '10 q' is correct. Hardly worth the buildup. All that prose and the solution is just matching squiggles.

Well, there is one complication. Since each sed command must start on its own line, the curly braces and the nested sed commands must be on separate lines. Previously, I showed you how to remove comments starting with a ". You could perform the same action as before, but limit the change to the first lines:! This inverts the address, which removes comments from all lines except those between the two reserved words:!

If you did not want to make any changes where the word "begin" occurred, you could simple add a new condition to skip over that line:!

If you use the same method you used for "begin" then the sed engine will not see the "end" to stop the range - it skips over that as well. The solution is to do a substitute on all lines that don't have the "end" by using!

Anything else will be considered part of the file name. The "w" command also has the same limitation as the "w" flag: only 10 files can be opened in sed. Reading in a file with the 'r' command There is also a command for reading files.

Change the order and it will not work. There are two subtle actions that prevent this from working. The first is the "r" command writes the file to the output stream. The file is not inserted into the pattern space, and therefore cannot be modified by any command. Therefore the delete command does not affect the data read from the file. The other subtlety is the "d" command deletes the current data in the pattern space. Once all of the data is deleted, it does make sense that no other action will be attempted.

Therefore a "d" command executed in a curly brace also aborts all further actions. As an example, the substitute command below is never executed:! The file that is included has a predetermined name. It would be nice if sed allowed a variable e. Alas, sed doesn't have this ability. You could work around this limitation by creating sed commands on the fly, or by using shell quotes to pass variables into the sed script.

Suppose you wanted to create a command that would include a file like cpp , but the filename is an argument to the script. The older versions of sed only allow one line as a comment, and it must be the first line. The last example could be:! Because an entire line is added, the new line is on a line by itself to emphasize this. There is no option, an entire line is used, and it must be on its own line. The syntax to these commands is finicky, like the "r" and "w" commands.

Append a line with 'a' The "a" command appends a line after the range or pattern. This example will add a line after every line with "WORD:"! Insert a line with 'i' You can insert a new line before the pattern with the "i" command:!

The "d" command would terminate the current actions. You can combine all three actions using curly braces:! However these white space characters may or may not be ignored if they start the text following a "a," "c" or "i" command.

In SunOS, both "features" are available. Most commands operate on the pattern space, and subsequent commands may act on the results of the last modification. The three previous commands, like the read file command, add the new lines to the output stream, bypassing the pattern space. Address ranges and the above commands You may remember that earlier I warned you that some commands can take a range of lines, and others cannot.

The "c" or change command allows this, and it will let you change several lines into one:! Regular expressions are line oriented. Searching for patterns that covers more than one line is not an easy task. Hint: It will be very shortly. Sed reads in a line of text, performs commands which may modify the line, and outputs modification if desired.

The main loop of a sed script looks like this: The next line is read from the input file and places it in the pattern space. If the end of file is found, and if there are additional files to read, the current file is closed, the next file is opened, and the first line of the new file is placed into the pattern space.

The line count is incremented by one. Opening a new file does not reset this number. Each sed command is examined. If there is a restriction placed on the command, and the current line in the pattern space meets that restriction, the command is executed.

Some commands, like "n" or "d" cause sed to go to the top of the loop. The "q" command causes sed to stop. Otherwise the next command is examined. After all of the commands are examined, the pattern space is output unless sed has the optional "-n" argument.

The restriction before the command determines if the command is executed. If the variable is false and the first pattern is found, the variable is made true.

If the variable is true, the command is executed. That was a mouthful. If you have read carefully up to here, you should have breezed through this. You may want to refer back, because I covered several subtle points. My choice of words was deliberate. It covers some unusual cases, like: what happens if the second number is less than the first number?

Here is another review, this time in a table format. The next line is "CD. That review is a little easier to follow, isn't it? You need to edit multi-line patterns to do this. Transform with y If you wanted to change a word from lower case to upper case, you could write 26 character substitutions, converting "a" to "A," etc.

Sed has a command that operates like the tr program. It is called the "y" command. If you wanted to change the second word in a line to upper case, and you are using classic sed, you are out of luck - unless you use multi-line editing. Hey - I think there is some sort of theme here! However, GNU sed has a uppercase and lowercase extension. Displaying control characters with a l The "l" command prints the current pattern space.

It is therefore useful in debugging sed scripts. I found it useful to print out the current pattern space, while probing the subtleties of sed. The "n" command will print out the current pattern space unless the "-n" flag is used , empty the current pattern space, and read in the next line of input.

The "N" command does not print out the current pattern space and does not empty the pattern space. It reads in the next line, but appends a new line character along with the input line itself to the pattern space. The "d" command deletes the current pattern space, reads in the next line, puts the new line into the pattern space, and aborts the current command, and starts execution at the first sed command.

This is called starting a new "cycle. Like "d," it stops the current command and starts the command cycle over again. However, it will not print the current pattern space. You must print it yourself, a step earlier. If the "D" command is executed with a group of other commands in a curly brace, commands after the "D" command are ignored.

The next group of sed commands is executed, unless the pattern space is emptied. If this happens, the cycle is started from the top and a new line is read. The "p" command prints the entire pattern space. Neither the "p" nor the "P" command changes the patterns space. Some examples might demonstrate "N" by itself isn't very useful. Instead, it combines the first and second line, then prints them, combines the third and fourth line, and prints them, etc.

If you wanted to search for a line that ended with the character " ," and append the next line to it, you could use! Here is a way to look for the string "skip3", and if found, delete that line and the next two lines. If you wanted to match 3 particular lines, it's a little more work. The next example will look for two words which are either on the same line or one is on the end of a line and the second is on the beginning of the next line.

If found, the first word is deleted:! The typical order is "N," "P" and lastly "D. You can use two invocations of sed to do this although it is possible to do it with one, but that must wait until next section. The first sed command will output a line number on one line, and then print the line on the next line. The second invocation of sed will merge the two lines together:! As an example, if you had a file that had a hexadecimal number followed by a word, and you wanted to convert the first word to all upper case, you can use the "y" command, but you must first split the line into two lines, change one of the two, and merge them together.

That is, a line containing 0x1fff table2 will be changed into two lines: 0x1fff table2 and the first line will be converted into upper case. I will use tr to convert the space into a new line, and then use sed to do the rest.

The command would be. You can embed a new line in a substitute command, but you must escape it with a backslash. Heavy sigh. Here is the example:! When found, it indicates the place a blank used to be.

A backslash is a good character, except it must be escaped with a backslash, and makes the sed script obscure. Save it for that guy who keeps asking dumb questions.

That's the ticket. Or use the C shell and really confuse him! I think I'm getting carried away. Well, this has some subtle issues here.

There is one more "location" to be covered: the hold buffer or hold space. Think of it as a spare pattern buffer. Can do insertion, deletion, search and replace substitution. SED command in unix supports regular expression which allows it perform complex pattern matching. Learn unix. Sample Commands Replacing or substituting string : Sed command is mostly used to replace the text in a file. Skip to content. Change Language.

Related Articles. Table of Contents. Improve Article. Save Article. Like Article. Recommended Articles. Article Contributed By :.