|
|
|
|
|
       |
 |
|
|
Sed, the Stream Editor by Ted Holt You're probably familiar with text editors such as SEU, which runs under OS/400, or Notepad, which comes with Microsoft Windows. These are interactive editors. They let you load a file into memory and make changes to it. But there is another type of editor, called a non-interactive editor or a stream editor, that applies a list of commands to a file in order to update it. Though stream editors are not used nearly as much as interactive ones, they can come in quite handy.
In this short article, I want to introduce you to sed, a stream editor that runs in OS/400's Qshell environment. I'd like to give you an idea of what this powerful utility does and give you some direction on how to use it. You'll find that sed can take care of a lot of miscellaneous tasks for you. Sed reads one or more input files, or the standard input file (stdin), applying all applicable editing commands to each record. That is, after all applicable editing commands have been applied to the first input record, sed reads the second input record and applies editing commands to it. Sed is a non-destructive editor. It does not modify the input files, but writes the modified data to the standard output file (stdout). Sed commands have three parts: addresses, a function, and arguments. The addresses tell sed which lines to edit. The function tells sed what to do to the selected lines. The arguments are additional data that a function needs to do its work. Sed Addresses A sed address is either a line number or a regular expression. Lines are numbered consecutively, beginning with the first record of the first input file. The special value $ stands for the last line of input. The regular expression is much like the regular expressions used with other Unix utilities, such as grep and awk. It contains literals and wildcards that can be used to match the data in records. If you do not specify an address, sed applies the function to all lines of the input. If you specify one address, sed modifies the lines that match the address. If you specify two addresses, sed matches one or more ranges of lines. Each range begins with the first record that matches the first address and ends with the next record that matches the second address. The two addresses must be separated by a comma. Sed Functions Sed includes the functions needed to change records, add new records, and delete records, as you would expect, plus other functions. A function is always a single character. As with all things Unix, case matters. Function h and H are two different functions. These are some of the most common functions:
Examples The best way I know to give you a taste of what sed can do is to show you some simple examples. The input data I'm going to use is a favorite poem of mine, which I have stored in a file named poem.txt. A wise old owl lived in an oak; The more he saw the less he spoke; The less he spoke the more he heard: Why can't we all be like that bird? EDWARD HERSEY RICHARDS In each example, the command I typed at the Qshell command line is shown in boldfaced, non-italicized characters. The output I got from running the sed command is in normal, italicized characters. Example 1 sed 's/oak/osage orange tree/' poem.txt A wise old owl lived in an osage orange tree; The more he saw the less he spoke; The less he spoke the more he heard: Why can't we all be like that bird? EDWARD HERSEY RICHARDS There is no address, so sed processes every line. The function is s, which stands for substitute. Sed replaces the first instance of oak in each line with osage orange tree. Example 2 sed 's/he/she/' poem.txt A wise old owl lived in an oak; Tshe more he saw the less he spoke; Tshe less he spoke the more he heard: Why can't we all be like that bird? EDWARD HERSEY RICHARDS Sed replaces the first instance of he in each line with she. Oops! That's not quite what I expected. Did you notice that sed didn't touch the author's middle name? The pattern and replacement strings are case-sensitive. Example 3 sed 's/he/she/g' poem.txt A wise old owl lived in an oak; Tshe more she saw tshe less she spoke; Tshe less she spoke tshe more she sheard: Why can't we all be like that bird? EDWARD HERSEY RICHARDS The "g" on the end of the substitute command stands for global. That means that sed is to substitute all instances of he on the selected lines with she. Since there is no address on the command, sed edits all lines. It appears that I need to further refine this sed command. Example 4 sed 's/ he / she /g' poem.txt A wise old owl lived in an oak; The more she saw the less she spoke; The less she spoke the more she heard: Why can't we all be like that bird? EDWARD HERSEY RICHARDS Well, that's better. Notice that the pattern and replacement strings can contain blanks. Example 5 sed 's/[;:]/./g' poem.txt A wise old owl lived in an oak. The more he saw the less he spoke. The less he spoke the more he heard. Why can't we all be like that bird? EDWARD HERSEY RICHARDS The brackets ([]) in the pattern string.indicate a set of characters. Sed replaces any character in the set with a period (.). The semicolons (;) and colons (:) that end the lines have been changed to periods. Example 6 sed 5d poem.txt A wise old owl lived in an oak; The more he saw the less he spoke; The less he spoke the more he heard: Why can't we all be like that bird? EDWARD HERSEY RICHARDS The address is 5, which means that the function applies only to line 5. The function is d, which means to delete the line from the output stream. Sed writes all lines except for line 5 to stdout. Example 7 sed '5,$d' poem.txt A wise old owl lived in an oak; The more he saw the less he spoke; The less he spoke the more he heard: Why can't we all be like that bird? The address is line 5 through the end. Sed deletes line 5 and all following lines. Example 8 sed '5,$!d' poem.txt EDWARD HERSEY RICHARDS The address is line 5 through the end. The function is negated with the exclamation point (!); therefore, sed deletes all lines except for line 5 and following lines. Example 9 sed '/oak/a\ (Oaks are nice!)\ ' poem.txt A wise old owl lived in an oak; (Oaks are nice!) The more he saw the less he spoke; The less he spoke the more he heard: Why can't we all be like that bird? EDWARD HERSEY RICHARDS The address is /oak/. When sed finds a line containing oak, it appends the line (Oaks are nice!). Notice that the command is entered over three lines. That is, I had to press Enter three times before sed began to run. The first time I pressed Enter was after the append command, a. The second time I pressed Enter was after the text to be appended. The last time I pressed Enter was when I finished the command. Notice that all lines but the last one have to be terminated with a backslash (\). Example 10 sed '/EDW/i\ written by \ a wise man, \ ' poem.txt A wise old owl lived in an oak; The more he saw the less he spoke; The less he spoke the more he heard: Why can't we all be like that bird? written by a wise man, EDWARD HERSEY RICHARDS The pattern is EDW. When sed finds a record that contains EDW, it inserts two lines of text. Like appended text, inserted text must go on separate input lines. In this example, I pressed Enter four times before the command ran. Example 11 sed \ 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/' \ poem.txt A WISE OLD OWL LIVED IN AN OAK; THE MORE HE SAW THE LESS HE SPOKE; THE LESS HE SPOKE THE MORE HE HEARD: WHY CAN'T WE ALL BE LIKE THAT BIRD? EDWARD HERSEY RICHARDS The transform function, y, translates the characters in one set to their corresponding characters in another set. In this example, lowercase characters are replaced with their uppercase counterparts. It was not necessary for me to enter the command on three lines. I did so to keep the lines of the command shorter. Notice I continued the command by ending all but the last line with a backslash. Example 12 sed -e '/all/q' -e '1s/lived/boogied/' \ -e 's/more/MORE/g' -e 's/less/LESS/g' poem.txt A wise old owl boogied in an oak; The MORE he saw the LESS he spoke; The LESS he spoke the MORE he heard: Why can't we all be like that bird? The -e option allows you to enter more than one command on the command line. Each command is preceded by -e, so Qshell lets you repeat -e as necessary. The first command tells sed to quit the edit when it processes a record with the value all in it. The second command substitutes boogied for lived in the first record. The third and fourth commands capitalize the words more and less for all lines of the input data. Example 13 cat owl.txt 1i\ A Wise Old Owl\ = ==== === ===\ by Edward Hersey Richards\ ========================= 5q sed -f owl.txt poem.txt A Wise Old Owl = ==== === === by Edward Hersey Richards ========================= A wise old owl lived in an oak; The more he saw the less he spoke; The less he spoke the more he heard: Why can't we all be like that bird? File owl.txt has a list of two editing commands: one insert command and one quit command. The insert command inserts four lines of text before line 1. The quit command terminates the sed editing session when the fifth input record is read. The -f option tells sed to apply the commands in owl.txt to poem.txt. Files that contain sed commands are often called sed scripts. Example 14 sed '/wise/,/heard/y/abcdefg/ABCDEFG/' poem.txt A wisE olD owl livED in An oAk; ThE morE hE sAw thE lEss hE spokE; ThE lEss hE spokE thE morE hE hEArD: Why can't we all be like that bird? EDWARD HERSEY RICHARDS This command has two addresses: /wise/ and heard. The range of lines is the first line containing wise and the first following line that contains heard, which resolves to lines 1 and 3 respectively. Sed converts the lowercase letters a through g to their uppercase equivalents. What Do You Think? Well, do you think there might be some way to use sed in your shop? Sed works best with free-format text, so it might be clumsy to try to use sed with RPG source code, but it works fine with SQL queries, FTP scripts, and CL. Give it a whirl and have some fun!
|
Editors
Contact the Editors |
|
Last Updated: 9/26/02 Copyright © 1996-2008 Guild Companies, Inc. All Rights Reserved. |
ADVANCED SYSTEMS CONCEPTS