Admin Alert: A Simple Batch FTP Tutorial

by Joe Hertvik

In previous Admin Alerts, I've written about using FTP as an attended process during which someone sits at an OS/400 or DOS command line, opens an FTP session, and types in the commands to copy a file. In many real-world FTP situations, this isn't the case, however. Many times, you need to set up an automated FTP session that runs in batch mode on your iSeries or AS/400 and kicks off and copies files at a specified time each day.

This week, I'll present a plan for setting up an unattended FTP transfer that does exactly that: It automatically copies data to a companion server according to a predefined schedule. It doesn't take much work, but is slightly more complicated than a manual FTP transfer and may leave one FTP security hole in your system. Here's the drill for creating an unattended FTP copy between two iSeries or AS/400 boxes.

1. Create an FTP User Profile on your FTP Server Box

Assume the name of the FTP user for our example is FTPUSER, and that his password is GOFTP. I'll call the iSeries box that is initiating the FTP session ftpcln400 (the FTP client), and I'll refer to the AS/400 box that my client session is connecting to ftpsrv400 (the FTP server).

The key is to create FTPUSER as a special FTP user profile on the OS/400 box that is serving as your FTP server (ftpsrv400, in this case). This profile should have all the necessary authorities to whatever files you want to copy, but it should not have authority to do much else on your OS/400 server. This profile needs to be locked down tight because you need to hard-code or automatically input the password for this user profile in your batch FTP transfer function. This means that outside people may be able to find and view the user profile name and password on your partner iSeries or AS/400. I recommend that, after you've created the FTP user profile, you use the Change User Profile (CHGUSRPRF) command to lock down the following parameters for your FTP user:

CHGUSRPRF USRPRF(FTPUSER) USRCLS(*USER) 
   INLMNU(*SIGNOFF) LMTCPB(*YES)

These parameters close the following holes for this user profile:

• The User Class parameter, USRCLS(*USER), designates this profile as a standard user profile with no special authorities.
• The Initial Program to Call parameter, INLPGM(*NONE), specifies that no OS/400 program will be called if this user attempts to sign on from a command line.
• The Initial Menu parameter, (INLMNU(*SIGNOFF), tells OS/400 to automatically sign this user off if he signs on to your system.
• The Limit Capabilities parameter, LMTCPB(*YES), stops users from running commands on the OS/400 command line or selecting a command option from a command grouping menu. In addition, the program, menu, current library values, and the Attention key (ATTN) program cannot be changed for this user.

In addition to these parameters, be extremely stingy with granting DB2 UDB and IFS authorities to this user. As far as authorities go, this user should only be able to access the particular FTP files to transfer or receive and nothing else.

2. Create an FTP Input Script File on Your FTP Client Box

To test our batch transfer, take a copy of the sample Client Access customer master file that IBM provides on most AS/400s and copy that file between machines. The name of this file is QCUSTCDT, and it resides in the QIWS library. For our testing, make a copy of that file in the QGPL library of the ftpsrv400 machine by executing the following Copy File (CPYF) command:

CPYF FROMFILE(QIWS/QCUSTCDT) TOFILE(QGPL/QCUSTCDT)
   MBROPT(*REPLACE) CRTFILE(*YES)

Also, because we're copying a physical file between two iSeries boxes, it's important to create an empty copy of the file (complete with field definitions) on your target system. For UDB DB2 transfers between iSeries, if you're relying on the FTP GET subcommand to create your target file for you, it will copy over the data but may also copy the new file as a flat file, and you'll lose your field definitions in the process. That's why it's usually better to save physical files as OS/400 save files, then transfer the save file over, via FTP, and restore the file to the target system from your system. (We'll cover another time.)

The next step is to create an FTP input script file on your ftpcln400 client box. Your FTP session will use this script file to automatically enter the subcommands that enable your copy. Because we're using an automatic copy, this means we have to embed our FTP user profile name and password inside the script in order to sign on to our FTP server box, which creates a security hole on your system. This script can also be modified to input the user ID and password as CL variables or to retrieve those values from another secured location--such as a data area or a physical file--that is tightly locked down. However, you need to be aware that this technique contains a vulnerability, and adjust for it accordingly.

When creating our FTP input file, it must be stored in an OS/400 file member, where each FTP subcommand is entered as its own record. If you want to make it easy to edit your FTP input script, you can store it as a Text (TXT) member in an OS/400 source file, but this limits your FTP subcommands to 80 characters or less. It doesn't really matter where your store the FTP input script, however. The only requirement is that a CL program must be able to use the FTP input script member in an Override with Database File (OVRDBF) command.

For our purposes, store this information in the FTPINTPUT member of the QCLSRC source file in library QGPL. To set up our transfer, create the FTPINPUT member as a Text (TXT) member that contains the following subcommands:

REIN
USER FTPUSER GOFTP
NAMEFMT 1           
CD /QSYS.LIB/QGPL.LIB
LCD /QSYS.LIB/QGPL.LIB
EBCDIC       
GET QCUSTCDT.FILE (REPLACE
QUIT                       

For this transfer, the input script file will provide the following functions:

• The REIN (Reinitialize Session) subcommand reinitializes the FTP server session to the state the server was in when it was started. This allows you to automatically feed the ftpsrv400 a user profile and password. Without this subcommand, your automatic session may not be able to sign on to the target server.
• The USER FTPUSER GOFTP (Send Your User ID to the Remote System) subcommand tells the remote FTP server that your client wants to sign on anew by using user ID FTPUSER with password GOFTP. Change these values to the proper user ID and password for your own system.
• The NAMEFMT 1 (Select File Naming format) subcommand tells the remote iSeries FTP server that any upcoming file transfer locations and names must be coded using IFS notation. For an explanation of what NAMEFMT 1 does, see "Admin Alert: The OS/400 FTP Subcommand Glossary."
• CD /QSYS.LIB/QGPL.LIB (Change Working Directory or Library) changes the default working directory on the ftpsrv400 FTP server attached to QGPL.
• LCD /QSYS.LIB/QGPL.LIB (Change Working Library or Folder) changes the local working directory (library) on our client ftpcln400 iSeries to QGPL.
• EBCDIC (Change File Type to EBCDIC) designates that this will be an EBCDIC file transfer between two iSeries boxes. We could run this copy as binary, but EBCDIC ensures that our file record lengths will stay consistent between the two machines.
• GET QCUSTCDT (REPLACE (Copy a File from a Remote System) tells FTP to copy the first member of file QGPL/QCUSTCDT from the QGPL library on the remote system to the QGPL library on the local system (as defined in the CD and LCD subcommands). The (REPLACE parameter tells FTP to clear the target file before copying, if the file exists on the local system and the user has the proper authority.
• QUIT ends the FTP session.

If I needed to use a PUT subcommand to send the data to the FTP host, I could modify my script as follows:

 
REIN
USER FTPUSER GOFTP
NAMEFMT 1
CD /QSYS.LIB/QGPL.LIB
LCD /QSYS.LIB/QGPL.LIB
EBCDIC
PUT  QCUSTCDT.FILE
QUIT     

The only difference between this script and the previous script is the PUT statement. Everything else is identical. But the point is that you can place any FTP subcommand inside your script file, and your automatic FTP session will run those commands.

3. Create an FTP Output Results File on Your FTP Server Box

We also need to create a member that holds the results of your FTP copy. This is basically just a scratch member that we examine after the transfer has occurred to determine if everything went well. For our example, create another source TXT file in QGPL/QCLSRC called FTPOUTPUT.

4. Create a CL Program to Control Your Batch FTP File Copy

Now that we have an FTP input file that contains the FTP commands to run, as well as an FTP output file where we'll store the results of our copy, we now need to put it all together by creating a CL program to run the transfer. Here's a simple CL program called QCUSTCOPY that will run our automated unattended FTP transfer.

PGM

OVRDBF     FILE(INPUT) TOFILE(QGPL/QCLSRC) +
              MBR(FTPINPUT) /* Override your FTP input +
              input commands to the proper location */ 

CLRPFM  FILE(QGPL/QCLSRC) MBR(FTPOUTPUT) /* +
              Clear out the FTP output file to +
               eliminate records from past transfers */

OVRDBF     FILE(OUTPUT) TOFILE(QGPL/QCLSRC) +
              MBR(FTPOUTPUT) /* Override your FTP output +
              file to the proper location */        
              
FTP        RMTSYS(FTPSRV400) /* FTP over to the server +
              and get your data */

DLTOVR  FILE(*ALL)

ENDPGM

This CL program is very simple to code, and it performs the following functions.

The first Override Database File (OVRDBF) command assigns our FTPINPUT script member to an FTP variable called INPUT, which OS/400 FTP will use to execute the subcommands in the script.

The Clear Physical File (CLRPFM) statement clears the FTPOUTPUT member of all records from previous FTP transfers. This eliminates any confusion caused by old messages.

The second OVRDBF command overrides our FTPOUTPUT member to another variable called OUTPUT, which will be used to hold the resulting messages that are generated from our automatic FTP session.

The FTP command performs three functions: It starts an FTP session with the remote server specified in the Remote System (RMTSYS) parameter of the FTP command. It runs the FTP commands contained in the FTPINPUT member in QGPL/QCLSRC, performing whatever functions are listed into the FTPINPUT source member. It stores the results of each FTP command in the FTPOUTPUT member.

The last step in the CL is to execute the Delete Override (DLTOVR) command, which performs end-of-program maintenance by deleting all the overrides created in this program.

Before you run the transfer, make sure the FTP server is started on your remote iSeries box (ftpsrv400, in this case). Do this by executing the Start TCP/IP Server command, designating FTP as the server application to start:

STRTCPSVR SERVER(*FTP)

Then, when you run the QCUSTCOPY CL program from the FTP client (ftpcln400), it starts a session, automatically copies the designated files, and ends the program. You can view the results of the transfer in the FTPOUTPUT member and correct any errors that occur.

5. Schedule Automated FTP Copying Using the WRKJOBSCDE command

Now that you've distilled your designated FTP transfer into an easy-to-call CL program, you now need to put an entry in the OS/400 job scheduler, so that this program can copy files at the proper time (which may be done on a daily, weekly, or monthly basis). To do this, call the Work with Job Schedule Entries (WRKJOBSCDE) command to set up a standard day and time that this job will be executed. Once on the WRKJOBSCDE screen, press the F6 key and enter a new entry for running the FTP CL program at the appropriate times. Be sure to check the results stored in your FTPOUTPUT member on a regular basis to determine if the file transfer ran successfully; if it didn't, find out what errors occurred and correct them.

Sponsored By ITERA

Echo2 High Availability - Affordable, Reliable, Easy-to-Use Reliable, state-of-the-art high availability isn't as expensive and complex as you might think. Not only does Echo2 cost about half of other solutions, it so automated and self-managing that operation of the system can be done in less than an hour a day and production-to-backup switchovers can be accomplished in minutes. For your Free whitepaper, "An Introduction to iSeries High Availability," call 1-800-957-4511, email us at info@iterainc.com, or visit www.iterainc.com.