EDTF and End-of-Line Delimiters, Take Two

Published: August 30, 2006

Hey, Ted:

In EDTF and End-of-Line Delimiters, you demonstrated a method for removing carriage-return characters from end-of-line delimiters in stream files. Unfortunately, your method only works in interactive jobs. I need to do the same operation on a file that I need to FTP. Do you have a way to remove these characters in batch?

--Chuck

You have pointed out yet another violation of Holt's Law of Interactive Programs, which states: "The problem with interactive programs is that you can't run them in batch." I have a few methods for you, Chuck. First, here's one from Shalom Carmel, a really sharp guy who reads this newsletter and who has contributed several good ideas.

Shalom suggested using the sed utility, which is a batch editor, under Qshell or PASE. The sed tool is good for making changes to text files when you can't be present to win. Here is Shalom's solution.

sed 's/.$//' file1 > file2

Sed reads file1 and writes the converted data to file2. The s subcommand tells sed to substitute one expression with another one. The expressions are coded between the slashes. The period before the dollar sign is a wildcard that can match any character. The dollar sign represents the end of the line. Therefore, the first expression means that the character immediately preceding the end of line, which is always a carriage-return, is to be substituted with an empty expression.

Shalom goes on to say, "The thing to watch is the codepage of the result file. Qshell will create the new file with the codepage of the job, while PASE will create it with the default ASCII codepage. More simple yet powerful sed commands can be found at www.student.northpark.edu/pemente/sed/sed1line.txt."

Thanks, Shalom. Here is an easier Qshell method.

cat file1 >file2

I'm not sure why this one works. The cat utility seems to know that lines can be ended with either CR/LF or LF alone and accepts a file with either delimiter. However, when Qshell writes the file, it uses the LF character only to terminate the records, according to the Qshell standard.

I've shown these commands as they would be entered into a shell script. To run them from CL, embed them in the QSH command.

QSH CMD('sed ''s/.$//'' file1 > file2')
QSH CMD('cat file1 >file2')

Last, here is a method that uses CL commands.

CRTPF FILE(QTEMP/SAVE) RCDLEN(256)

CPYFRMSTMF FROMSTMF(file1)
           TOMBR('/QSYS.LIB/QTEMP.LIB/SAVE.FILE/SAVE.MBR')
           MBROPT(*REPLACE)

CPYTOSTMF FROMMBR('/QSYS.LIB/QTEMP.LIB/SAVE.FILE/SAVE.MBR')
          TOSTMF(file2)
          ENDLINFMT(*LF)

The default behavior of Copy From Stream File (CPYFRMSTMF) is to accept any combination of carriage-return and line-feed characters as end-of-file markers. You can use the ENDLINFMT parameter to check for a certain delimiter. In this example, there is no need to use ENDLINFMT. So, CPYFRMSTMF reads the stream file and writes to a scratch database file. Then Copy To Stream File writes to a stream file, terminating each line with a line-feed character.

--Ted