BLOB Your Data for the Web

by Paul Holm

My company has been working with a large manufacturer for the past few months, building a Java- and Web-based warranty application. The primary purpose of this application is to replace the slow and cumbersome fax and snail mail system currently in use. This application will allow authorized service centers to submit warranty claims, gain warranty approval, check warranty-claim status, and present a number of warranty-related reports via the Internet.

One of the most intriguing requirements was the ability to allow service centers to upload images, videos, illustrations, documents, and many other types of files. These files are often required in order to provide proof and validation of a warranty claim.

Based on this requirement, we had several technical features that we needed to build. These included the capability to upload any type of file from the user's system, to store the uploaded files, and to present the files to users.

This article walks through the process and related decisions that went into making this all work.

Uploading Files

This task involved prompting the user to select a file for upload, which can be easily achieved using standard HTML within a JavaServer Page (JSP). A common method for submitting data to the server within an HTML document is accomplished with the use of a form. A form allows for interaction with the user via several form controls, such as the select, input, and button tags. One of the most common tags that retrieve information from the user is the input tag. A standard input can take on several behaviors, which are specified by the type attribute. The default type is text, and results in a simple text input. Another very useful type is the file input, which can be specified via the following HTML:

<input type=file> </input>

The file input type creates a field through which users can upload files from their local computer or network. The value attribute specifies the name of the initial file, but it is typically ignored by browsers as a security precaution. The accept attribute gives a comma-separated list of media types accepted, allowing the browser to filter out inappropriate files. Current browsers generally ignore the accept attribute.

The file input presents a standard method to retrieve the data from the user; however, to actually read the data from the user's machine is more involved. To place a file input on the form, you must specify two attributes on the form. You must set the action attribute to POST, and the enctype attribute to multipart/form-data on the form tag. Further detail about the format of mulitpart/form-data is available in RFC 1867, on the Internet Engineering Task Force Web site.

The enctype attribute specifies the content type used in submitting the form, and defaults to application/x-www-form-urlencoded. This content type results in name/value pairs sent to the server as name1=value1&name2=value2. . . , with space characters replaced by a plus sign (+) and reserved characters, like the pound sign (#), replaced by %HH; where HH is the ASCII code of the character in hexadecimal. Line breaks are encoded as %0D%0A, a carriage return followed by a line feed.

The following example would allow a user to upload a file to a servlet:

<FORM METHOD=POST ENCTYPE="multipart/form-data"
ACTION="com.companyname.servlets.FileUploadServlet" >
Please select a file to upload: <INPUT TYPE=file
NAME="file_upload"/><P>
   <INPUT TYPE=submit/>

Storing Files

We considered two methods for storing the uploaded files: the iSeries IFS (Integrated File System) and the iSeries DB2 database.

Both of these approaches would have satisfied the requirements, but we chose the DB2 database and its BLOB (binary large objects) support for the following reasons.

The first reason was simplicity. Using the IFS requires the production of a naming scheme to associate files with a particular claim. This approach can become somewhat complex and problematic, in that it requires you to manage the link, mostly likely contained in each database record, between your code and the directory structure where those files reside. If at some point the directory structure changes or is reorganized, or the data is moved completely, the links to those files must be changed for all the old records in order to reflect the new location and organization.

With DB2 BLOB support, we simply store the files in a traditional record, along with its associated "metadata." Referencing and maintaining the link between your attachments and your database record becomes a non-issue.

We needed to record "metadata," such as upload time, upload user, and claim number, with each file. Using the DB2 approach, we were simply able to attach extra fields to our file and easily record this additional metadata. Using the IFS would have required an associated DB2 record to hold this metadata.

The second reason for choosing the DB2 database was performance. We were concerned that the DB2 BLOB support would not perform well, because of the need to read the BLOB out of the database for each viewing request. To minimize the risk, we put together a prototype application and verified that performance was adequate. We did not do any performance comparisons, but our thought was that the IFS may have an advantage because the Web server or browser could cache the files at their respective levels.

Creating a Database with BLOBs

A BLOB field is simply another DB2 field type. The following example shows the syntax for creating a database with a BLOB.

CREATE TABLE PLANETJ.FILES (
    ID INTEGER NOT NULL ,
    OWNER_ID INTEGER DEFAULT NULL ,
    NOTES CHAR(30) CCSID 37 DEFAULT NULL ,
    FILE_BLOB BLOB(10485760) DEFAULT NULL )

For this application, the field named FILE_BLOB was set to allow a file size of up to 10 megabytes. The size of this field would obviously be an area of consideration at design time, depending on the application being created and what was going to be stored in the BLOB. The default size for a BLOB field in DB2, if not explicitly specified as above, is 1 MB. The maximum size for a BLOB field in DB2 is 2 GB. We felt that, in most cases, pictures, documents, spreadsheets, and the like would typically be accommodated by 10 MB. The larger the allowable upload, the more space your file will potentially occupy.

Storing Files to the BLOB

Once the file has been uploaded by our Java servlet, it can be stored in the database, using standard JDBC. A very important that comes with changing the encoding type of a form is that the standard method of retrieving values that are submitted to the servlet, out of the REQUEST object, changes with the multipart/form-data. Normally, the information entered on the form is stored in the REQUEST, in the form of key-value parameters. The name of the interactive control for which the user entered data is the key that allows access to the value of the parameter.

Say, for instance, that we have a text input defined in the following manner:

       <INPUT TYPE="text" NAME="txtFirstName"></INPUT>
       

The associated Java code that would get this value from the request would be:

public void doPost(HttpServletRequest request, 
   HttpServletRequest response) throws ServletException, 
      IOException {
       
    String firstName = request.getParameter("txtFirstName");
       
       }

This scenario is changed with the multipart/form-data encoding. The parameters are now contained in the request as attributes, as opposed to parameters. You must use the getAttribute method of the REQUEST object to retrieve your information.

The following example illustrates how this is done.

public void doPost(HttpServletRequest request, HttpServletResponse 
   response) throws ServletException, IOException {
     try
     {   // Prepare a Statement:
         PreparedStatement stmt = conn.prepareStatement("insert 
		    into planetj.files (id, owner_id, notes, file_blob) 
			   values(8675309, 37, 'uploaded file from 
			      user', ?)");

         // Get the BLOB from the request
         byte[] bytes = request.getAttribute("file_upload");
         BlobObject aBlob = new BlobObject(bytes, bytes.length);
         // Set the parameter on the statement
         stmt.setBlob(1, aBlob);

         // Execute
         stmt.execute();

       // Close resources
       stmt.close();

     }
     catch(Exception ex)
     {
       System.out.println("Error occurred writing a BLOB: " + ex);
     }
}

Displaying BLOB Fields to the User

To display the files to the user, we again used traditional JDBC to read the appropriate records, as in the following example:

Select Notes, FILE_BLOB  from PlanetJ.Files where id = ? 

A BLOB field is read a little differently from traditional database types, because of its possible large size. BLOB fields are read using the simple JDBC result set apis. Once you have read the BLOB field, it is sent to the browser by obtaining the HTTPServletResponse and getting the outputStream.

public void doPost(HttpServletRequest request, HttpServletResponse 
   response) throws ServletException, IOException {
     try
     {   // Prepare a Statement:
         PreparedStatement stmt = conn.prepareStatement
		    ("select file_blob from planetj.files");

         // Execute
         ResultSet rs = stmt.executeQuery();

         while(rs.next())
         {
            try
            {
               // Get as a BLOB
               Blob aBlob = rs.getBlob(1);
               byte[] allBytesInBlob = aBlob.getBytes(1, (int) 
			      aBlob.length());
            }
            catch(Exception ex)
            {
               // The driver could not handle this as a BLOB...
               // Fallback to default (and slower) byte[] handling
               byte[] bytes = rs.getBytes(1);
            }
         }

       // Close resources
       rs.close();
       stmnt.close();

     }
     catch(Exception ex)
     {
       this.log("Error occurred reading a BLOB: " + ex);
     }
}

You've Been BLOB'd

BLOB support provides a viable and powerful facility to store binary data of any type in a secured, well-managed fashion. Combining BLOB support with Web applications can allow your applications to store and manage binary files when your application requirements dictate. BLOB away!

Paul Holm can be reached at pholm@planetjavainc.com.

Related Materials

Web Design Group's interactive forms Web page

Sponsored By ASNA

How Independent Computer Consultants is saving $50,000 with ASNA Independent Computer Consultants, Inc. (ICC) is a software development and consulting firm specializing in writing and maintaining business solutions that interface with the AS/400. With over 40 years of cumulative RPG programming experience, they have recently switched to AVR as their development environment of choice. ICC credits AVR for their expansion into the more lucrative client/server market. Originally, ICC specialized in AS/400 green screen-based applications, but with AVR and ASNA's data access tools they have experienced a great deal of success in writing and selling Windows applications. Their applications include front-end packages for portrait studios and jewelry manufacturers; a billing and standards package for the railroad industry; a lease track package for the real estate industry and an investment tracking package for financial firms. By entering into this new Windows market, ICC has experienced an annual increase in revenues of almost $50,000. "Using AVR has not only been fun-it has also increased our capabilities. We are now a more versatile company capable of handling a variety of computer needs." -Debbie Houser Download your FREE copy of ASNA Visual RPG today! www.asna.com/AVR_literature.asp