Volume 9, Number 32 -- September 1, 2009

Managed File Transfer: A New Product Category That's Here to Stay

Updated: September 2, 2009

by Alex Woodie

One of the great things about the Internet is it makes it exceptionally easy to distribute computer files. Within minutes, users can be up and running with free FTP and e-mail utilities, and begin sending files around the globe. However, as is the case with many things about the Internet, decentralized file transfer opens the door to security problems and inefficiencies that businesses should not tolerate. The solution advocated by many is a relatively new class of software product called managed file transfer, or MFT.

If you're wondering what this new term managed file transfer, or MFT, refers to and whether you should care, you're not alone. After all, the second great thing about the Internet--following its incredible capacity for technological democratization--is the speed at which new terms and whiz-bang technologies are foisted upon the masses. With that in mind, some caution over this new thing called MFT is warranted.

However, while you may not yet be familiar with MFT, there is a strong possibility that you could benefit from it.

What Is MFT?

MFT refers to a class of product that manages, secures, centralizes, and automates the transfer of files inside and outside of an organization. There are numerous MFT vendors, including some that support i OS and run on the System i server.

At the core of an MFT solution is often an FTP server---or, more likely, an SFTP (uses SSH) or FTPS server (uses SSL) that provides encrypted file transfers. Many MFT solutions also include provisions for sending files via SMTP, HTTP, or HTTPS. Still others rely on proprietary file transfer protocols, and include separate compression and encryption capabilities.

But MFT is not merely a collection of FTP, SMTP, or HTTP servers. On top of the transport layer, MFT solutions produce and provide full audit trails showing who transferred what files to where, and how and when they did it. This adds security to basic file transfer activities, and gives MFT a hand in regulatory compliance.

MFT also includes elements of automation, such as the capability to execute jobs when specific files arrive in specific folders, and to alert IT managers of unexpected situations. This allows MFT products to eliminate complicated scripting and reduce the need for expensive programming expertise.

MFT also identifies and records successful and failed file transfers to a customer or a partner. This provides elements of non-repudiation, and can help prevent those embarrassing instances where an organization isn't sure if a critical transfer actually worked.

A good way to think of MFT is as a "framework" for modern and secure FTP. This was the term used by Linoma Software in its new white paper published last week, titled "Beyond FTP: Securing and Managing File Transfers." Linoma also announced a new MFT product last week, which you can read more about at "Linoma Introduces MFT Software for External Exchanges."

Automation Benefits of MFT

Historically, programmers write scripts to automate batch-style FTP functions. This works fine on a limited scale, and if configurations rarely change. But relying on scripts can quickly become cumbersome when an organization is exchanging data with a lot of customers and partners, and when things like passwords, libraries, and IP addresses are constantly changing.

"That has been the de-facto method. 'Let's write a Perl script around FTP or SFTP," says Sam Morris, product marketing manager at Attachmate, which is currently rolling out a new MFT solution OEMed from Proginet called FileXPress Server (and yes it does run under i OS).

"But what happens is you start to bump into the limitations of those protocols and the utilities that leverage those protocols," Morris says. "For example how do you know for sure a file has transferred completely without any corruption? That's something that's challenging to do in context of FTP or SFTP script.

"Another example is when you encounter a network glitch. Knowing when that happens, when the failure occurs, and having file transfer agents automatically retrying that transfer, is something that's challenging to do [with scripting] and something that a good MFT solution is going to offer."

FileExpress Server, which Attachmate expects to formally announce this fall, utilizes a proprietary protocol called CFI (short for CyberFusion Integration) developed by Proginet that drives more intelligence into file transfers with features like check point restarts and cyclic redundancy checks (CRC). The product also includes a gateway for connecting over standard protocols.

Many MFT products also resemble scaled-down job schedulers. For example, an MFT product could be configured to perform several steps in response to the completion of a file transfer, such as convert a file into an Excel document, encrypt the document, and then distribute it via e-mail. Others can hook into schedulers via APIs or SOAP calls.

Keeping up with all the different protocols, including FTP, SFTP, FTPS, HTTP, HTTPS, and SMTP, is also a challenge for the do-it-yourself scripter, says Linoma's president Bob Luebbe. "That's a lot of effort to build all those different connectors and to be able to handle all the different formats to truly be able to connect to just about any system," he says. Linoma's product, GoAnywhere, supports all the open protocols, including the capability to directly connect to databases, and was recently certified on IBM's z/OS. It also runs on i OS, Linux, Windows, and Unix, giving it a wide-range of operating system support.

Security Benefits of MFT

MFT provides better security over basic FTP in three main ways: authentication, encryption, and logging.

Plain vanilla FTP relies on user names and passwords for authentication. Security is improved somewhat with FTPS, which delivers files securely over the Internet through an encrypted SSL tunnel, and implements certificate-based authentication. The competing standard SFTP, also creates an encrypted link, and uses passwords or keys for authentication.

But neither SFTP nor FTPS completely alleviates all security concerns if an organization has automated its FTP routines with scripts. "Most companies don't know how to properly protect the user names and passwords," Luebbe says. "If you open up the FTP scripts, you can see user names and passwords right in the clear. It's something that companies are getting dinged on by auditors. It not only exposes you, but it exposes your trading partner."

MFT solutions address this security concern by encrypting user names and passwords and storing them in a database. Regulatory compliance is a big driver for MFT, not only in terms of encrypting data transmissions and providing a framework for authentication, but also in terms of logging, Attachmate's Morris says.

"MFT lets organizations know from an auditing point of view who's transferring sensitive information between systems and people, and whether I'm successful with those transmissions," he says. "It also allows me to roll that up into a compliance report for PCI or HIPAA."

Above all, MFT allows users to centralize control over FTP, thereby avoiding the wrath of auditors for another day. "Auditors are really cracking down on companies that are just doing this casual use with FTP, where they're sending files all over the place from their desktops, or even from the iSeries," Luebbe says. "It's just really easy to crank up an FTP session and fire off files without having any security or auditing around what's getting sent."

Future of MFT

The recent buzz over MFT can be partly attributed to Gartner, which started tracking MFT about two years ago. Gartner says the MFT market currently accounts for $450 million to $600 million in yearly revenues, and is growing at 26 percent per year. IDC has also started tracking MFT.

A recent Gartner report predicted that MFT would grow in concert with another security-related IT discipline--encryption key management--and that both would become "mainstream" technologies in two to five years. That view was soundly endorsed by Gary Palgon, vice president of product management at nuBridges, which sells Exchange and Exchange for i MFT solutions.

"It's no longer enough to protect data in motion between business partners, or expect a firewall to protect it at rest," says Palgon, an expert in data security. "Today it takes a comprehensive data security program that secures confidential and sensitive information from the moment it's created until it's destroyed to adequately protect organizations."

The regulations are lagging in this regard. PCI DSS, for instance, mandates that credit card data be encrypted when its sent across the network, but it doesn't require encryption when that credit card data is moved internally, Palgon says. This is a key area where MFT can boost a company's security, keep it in front of the info-security curve, and (hopefully) out of the headlines.

With compelling ease-of-use and security benefits, you'll be hearing a lot more about MFT over the next few years. "A lot of customers we talk to have no idea what we're talking about when we talk about MFT. Because it is a really new term, a lot of people aren't aware of what it means," Luebbe says.

"We've have had a couple of customers say they want MFT," he continues. "But most of them say they have a specific need. They say we need to connect up to this partner with SFTP, can you help us out? We show them how to do that. They're like, 'Whoa, this product can do all kinds of different things.' We'll call them up three months later and now they're using the product for all sorts of different connections, and they've got a MFT solution. They just didn't know they needed one."

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot

Sponsored By

RPG/COBOL Design Recovery with
X-Analysis 8

The first step in any project starts with understanding:

                                                      · Structure Charts & Data Flows
                                                      · User Screen Flow Diagrams
                                                      · Extract Relational Data Model
                                                      · Extract Business Rule Logic
                                                      · UML Class/Activity/Use Case
                                                      · Analyze Java/VB with RPG/COBOL
                                                      · Variable-Level Impact Analysis
                                                      · Document Entire System

Run this exciting new release over your own system
and see all this in just two days.

Download a copy at www.databorough.com

Editor: Alex Woodie
Contributing Editors: Dan Burger, Joe Hertvik,
Shannon O'Donnell, Timothy Prickett Morgan
Publisher and Advertising Director: Jenny Thomas
Advertising Sales Representative: Kim Reed
Contact the Editors: To contact anyone on the IT Jungle Team
Go to our contacts page and send us a message.

Sponsored Links

Maximum Availability:  Upgrade to *noMAX - save 20% on current fees
ARCAD Software:  Start 5250 emulation sessions from your RDi workspace - download freeware!
COMMON:  Celebrate our 50th anniversary at annual conference, May 2 - 6, 2010, in Orlando


IT Jungle Store Top Book Picks

Easy Steps to Internet Programming for AS/400, iSeries, and System i: List Price, $49.95
The iSeries Express Web Implementer's Guide: List Price, $49.95
The System i RPG & RPG IV Tutorial and Lab Exercises: List Price, $59.95
The System i Pocket RPG & RPG IV Guide: List Price, $69.95
The iSeries Pocket Database Guide: List Price, $59.00
The iSeries Pocket SQL Guide: List Price, $59.00
The iSeries Pocket Query Guide: List Price, $49.00
The iSeries Pocket WebFacing Primer: List Price, $39.00
Migrating to WebSphere Express for iSeries: List Price, $49.00
Getting Started With WebSphere Development Studio Client for iSeries: List Price, $89.00
Getting Started with WebSphere Express for iSeries: List Price, $49.00
Can the AS/400 Survive IBM?: List Price, $49.00
Chip Wars: List Price, $29.95

The Four Hundred
CIOs Say Power Systems Are the Most Reliable

A Closer Look at IBM's Q2 Server Sales

Has IBM Given Up on the i?

Mad Dog 21/21: Terms and Conditions

Jack Henry Lays Out $17 Million for Goldleaf After Good 4Q

Four Hundred Guru
Validate DBCS-Open Data

Formatting Dates with SQL

Admin Alert: Correcting and Expanding the Program to Change User Passwords on the Fly

Four Hundred Monitor
Four Hundred Monitor's
Full iSeries Events Calendar

System i PTF Guide
August 29, 2009: Volume 11, Number 35

August 22, 2009: Volume 11, Number 34

August 15, 2009: Volume 11, Number 33

August 8, 2009: Volume 11, Number 32

August 1, 2009: Volume 11, Number 31

July 25, 2009: Volume 11, Number 30

July 18, 2009: Volume 11, Number 29

TPM at The Register
VMware vSphere gets more gadgets

Semiconductor sales rise 5.3% in July

Sun sales plummet 30.6% in Q4

AMD plays it cool with low-volt Istanbuls

Xen packages build-your-own-cloud kit

OpSource floats VMware cloud

Intel boosts Q3 guidance

Novell profits even as sales slide

VMware goes into hyper-drive with vSphere 4.0

Cray nabs PathScale compilers from SiCortex

Tibco snaps up DataSynapse for $28m

Sun goes over Rainbow Falls

Amazon does virtual private clouds

Big chip for big boxes: IBM cracks open lid on Power7


Maximum Availability
East Coast Computer

Printer Friendly Version

Managed File Transfer: A New Product Category That's Here to Stay

IBM to Formally Announce EGL Community Edition Today

Linoma Introduces MFT Software for External Exchanges

SEQUEL Updates i OS Time and Date Override Software

Cosyn Augments BPCS Accounting with AP Minder

News Briefs and Product Shorts:

TMW Says Windows-Based Dispatching System Can Run with Big System i Boys . . . Vision's Made-for-the-Blade HA Products Now Certified . . . IBS Hooks Up with rfXcel for E-Pedigree Tracking . . . Desktop Cost Cutting Measures Worth a Closer Look . . . IBM Delivers Optim Archiving and Test Software for JDE, But Goofs Up i OS Support . . .

Four Hundred Stuff


Subscription Information:
You can unsubscribe, change your email address, or sign up for any of IT Jungle's free e-newsletters through our Web site at http://www.itjungle.com/sub/subscribe.html.

Copyright © 1996-2009 Guild Companies, Inc. All Rights Reserved.
Guild Companies, Inc., 50 Park Terrace East, Suite 8F, New York, NY 10034

Privacy Statement