Secure FTP with AS3

Remember the File Transfer Protocol (FTP), that venerable utility from the early 1980s? It's been around a long time and now is a widely accepted standard. FTP is supported on almost all computer platforms from mainframe to microcomputer. Chances are it's being used within your organization for business purposes and often is used by individuals for personal file transactions. It is an older and more mature protocol compared to today's popular HTTP and SOAP.. An emerging standard, AS3, is about to make FTP viable for secure and reliable Internet transfers.

Documents are the lifeblood of business. Almost every business process involves one or more documents (paychecks, orders, invoices, etc.), which are increasingly assuming electronic formats. Much of their processing is performed using computer applications that often involve multiple applications and require moving documents between disparate computer platforms. Moving documents between physical locations is expanding as part of business-to-business (B2B) transactions. Although various forms of direct messaging between applications hold mindshare, file transfer remains a dominant, viable model for application integration. From manual, ad hoc transfers to scheduled and highly automated processes, FTP is one of the preferred utilities for moving documents.

Challenges conducting B2B with FTP

FTP traditionally has been used within the four walls of organizations and has not been exploited for document movement between businesses (unless expensive, private data lines were used). Until now, the protocol has had security and reliability deficiencies that prohibited business quality messaging across the Internet. For instance, the FTP standard does not inherently support the capability to retry sending documents if the target host is not available. Some simple scripting can address this situation, but as part of standard FTP, there is neither an acknowledgment to confirm receipt nor a way to ensure a resend does not result in a duplicate document on the receiving end. This functionality, often referred to as "guaranteed delivery," is necessary to ensure the reliability of business-critical file movement.

In addition to reliability problems, the standard does not address the security of documents transmitted. When the FTP standard was developing, the public Internet did not exist and closed networks ensured the privacy of file transfers. The inexpensive and ubiquitous nature of the Internet today has prompted many businesses to investigate using FTP for exchanging files with business partners. Businesses naturally have turned to FTP because of its familiarity for internal file movement. However, the lack of security measures (privacy, integrity, authentication) has curbed adoption of FTP for external file transfers.

Early attempts for securing FTP

The first significant step toward a standardized approach for securing FTP transmissions occurred in the late 1990s. A specification that identified how to apply the Secure Sockets Layer (SSL) to protect FTP transfers was introduced. FTP secured with SSL, often referred to as "Secure FTP" (or FTP-S), was offered as part of several commercial electronic commerce solutions. Secure FTP was a good first step but had some drawbacks. The main challenge with Secure FTP is that it works by encrypting the network-level packet communications. Many firewalls require access to these packets to control outside access to internal networks and ensure incoming information is not malicious. Packet encryption prevents firewalls from inspecting the flow of network traffic, and thus, many businesses' security groups were resistant or unwilling to deploy Secure FTP.

The fact that network level traffic is being encrypted rather than performed on the payload itself, means a document remains protected only during the associated transmission period. Many business scenarios require a document to remain encrypted until the receiving application is ready to process it. Organizations that move sensitive material (such as financial, healthcare or government) often require security artifacts be preserved to demonstrate that a transaction was performed at a certain time and by legitimate parties. The secure FTP standard does not support this capability commonly known as nonrepudiation.

Some organizations have overcome the lack of nonrepudiation in Secure FTP by leveraging separate tools such as PGP to encrypt the document independently from transport. The secured document then can be transmitted safely by FTP just like any other payload. A disadvantage to this approach is both parties must implement a proprietary solution.

AS3 functionality

These deficiencies, coupled with increased demand for Internet-based file transfer, were the motivation for a new specification from the Internet Engineering Task Force (IETF): AS3. This new specification defines how to perform secure and reliable file transfers with FTP in a standardized way to ensure interoperability between solutions (another important concern related to B2B transfers). As the name suggests, the AS3 specification has predecessors. AS1 and AS2 define how to provide security and reliability to the SMTP and HTTP protocols, respectively. AS3 completes the suite of security standards for TCP/IP protocols by applying the same to FTP.

From a user perspective, AS3 is a fairly simple protocol. The basic operational scenario is depicted in the figure below. One or more documents are passed from a business application to an AS3 solution. AS3 performs packaging and security processing and then attempts to deliver the payload via FTP to a receiving AS3 implementation. The receiving side reverses the process and delivers the payload, free from security and transport details to the necessary business system. Except for differences in transport details, AS3 is very similar to the AS1 and AS2 protocols. Many solutions likely will implement all three EDIINT protocols and provide a uniform interface with the capability to select the best one based upon payload or trading partner characteristics or preferences.

AS3 provides security and reliability features FTP does not. The formal specification and foundation upon open standards ensure AS3 can be implemented by many organizations and commercial software vendors in an interoperable manner. AS3 also provides for nonrepudiation and because encryption is document-based and doesn't interfere with network packets, it is much more compatible with firewalls than Secure FTP using SSL.

The AS3 specification addresses the following functionality necessary to support business quality messaging for Internet document exchange:

When should AS3 be considered?

If your business has a significant investment in FTP (i.e., scripting, interfaces with applications, or security functionality such as PGP), AS3 may be a good evolutionary path from proprietary solutions to standardized security and reliability. Many companies have clients demanding FTP because it is familiar, simple and cost-effective.

There are a number of alternative B2B transports available offering various (sometimes competing) technical capabilities. To identify the best transport for a given business scenario, it is important to understand the differences. The following table provides a comparison of the three EDIINT protocols as well as alternatives:

Specification status

The AS3 specification is in its infancy. Following the process by which the IETF develops standards, AS3 has been submitted as a technical proposal (referred to as an "Internet Draft") by interested parties. If and when interest grows in the standard, the EDIINT working group will begin formalizing the standard (reviews, modification, improvements, etc.). The specification will be subjected to community review as its status changes from draft to a Request For Comments (RFC) document, the final step before being embraced as an accepted standard.

This process progresses at different paces. For instance, while already in use by many businesses, the AS2 specification has remained an Internet Draft for more than five years. It likely will take AS3 several more years to become widely accepted and implemented broadly in commercial solutions. FTP is not viewed with as much enthusiasm as HTTP-based protocols that are oriented more for synchronous messaging and can provide service interactions. Right now, AS3 should be approached with some caution because it is a draft proposal, subject to revision and does not yet have significant backing by a large number of vendors or the user community.

Brian Gibb is Director, Standards and Strategic Technology of Sterling Commerce.