Stream Files and End-of-Line Delimiters, Take Three

Published: November 15, 2006

Hey, Ted:

I do a bit of development in Cygwin under Windows and then move it to the iSeries using Qshell. It turns out that Qshell doesn't like the carriage-return/linefeed combination (CRLF) in shell scripts, so I often need to change CRLF to LF--as you wrote about in two previous articles--in bulk. Here are some methods I use to convert end-of-line delimiters in stream files.

The first method is a simple Perl command.

perl -i.bak -pe 's/\r\n/\n/g' * 

This command adds the extension .bak to the end of the files and strips the CRLF, putting in LF instead. By using a wildcard, I can convert a whole directory at a time rather than one file at a time. One could just as easily handle a single file by changing the wildcard at the end from * to the name of the file that one wants to convert.

Another way is to use a Qshell script like this one, which I call dos2unix, after the Linux command of the same name:

#!/bin/bash 

# Replace CRLF with LF 

E_WRONGARGS=65 

if [ -z "$1" ] 
then 
echo "Usage: `basename $0` filename-to-convert" 
exit $E_WRONGARGS 
fi 

file="$1" 
sed 's/\r\n/\n/g' $file > $file.fixed 
cp -f $file.fixed $file 
rm -f $file.fixed 

An alternative to sed is tr:

tr -d '\r' < $file > $file.fixed 

--Buck Calabro

Thanks, Buck. I appreciate the solutions. Little junk like this tends to eat up my day.

--Ted

RELATED STORIES

EDTF and End-of-Line Delimiters

EDTF and End-of-Line Delimiters, Take Two