CF: Parsing Complex Fixed (Flat) Files

Despite many years of XML, the use of simple and complex flat files as a means of exchanging information between systems is still fairly common.
Parsing simple, tabular, flat files like the one below does not really pose a large challenge as either the OS (Text Drivers) or CF (CFHTTP) have tools to read them in without much fanfare.

Example 1: Tabular Fixed File



However, many times over, the examples are not as simple. The data on each row may be different; relationships may exist between lines of data etc.

Example 2: Complex Fixed File


For the complex files we normally write specialized parsers or transformations. Certainly a whole industry exists that deals with Extract Transform Load processes. Either choose requires considerable programming efforts.
After writing a parser for the umpteens fixed file that someone wanted to have loaded, I thought there needs to be a better way. The short of all this is that I created a more generic component to deal with this in native ColdFusion. The solution was to split up the code and the file definition.

Thus, in order to parse complex files, you provide an XML definition file which describes the flat file. Then call the component to do the heavy lifting.

Example 3: Flat File Definition XML



The outcome of such processing are standard ColdFusion objects that are easier to deal with and require less programmatic effort to implement.

I posted this project on RIAForge (FixedFileReader). It contains examples for different scenarios of varying complexity. I added skeletons for EDI X.12 and VCF4 formats.

May not work for all, but if it does, it is likely that it will save a lot of time.

Cheers,
Bilal