Creating Mendz.ETL-based Mappers

Mendz.ETL suggests three basic ingredients to ETL solutions: source adapter, mapper and target adapter. Here are some ideas for your mappers.

Mappers transform input data to an output format. In ETL, mappers perform the "T" or transform part. This can mean transforming an XML file to another XML format. Or perhaps a CSV file to positional file. Or maybe XML to flat file and vise versa. When derived from MapperBase, the TransformInputToOutput() method must be implemented. You can achieve the transformation using whatever strategy, technique or technology you want. The only requirement is that the implementation can accept the input as IEnumerable and return the output as IEnumerable.

For example, a FullSourceAdapter can be used to read a small XML file, which can then be passed to an XsltMapper implementation, the result of which can be passed to a FullTargetAdapter. Since .Net does not support streaming XSLT (yet) anyway, and for as long as the XML file is small, this easily makes sense. In this context, the call to Router.Route(source, mapper, target); is direct to the point: extract the full source, transform via XSLT, and load the full result to a target.

Mendz.Library provides IGenericMapper, IStreamingMapper, StreamingGenericMapperBase and GenericStreamingMapper. These can be used to help express mappings and transformations in a more OOP fashion. For example, you can map a CSV string to its POCO representation.

Mendz.ETL itself also provides IData* interfaces, which can be used to create data level conversions, filters, insertions, joins and splits. Implement IData* to create re-useable data manipulation utilities in your ETL solution. IData* implementations can be used in adapters, mappers, validators and/or joiners.

Callers call the mapper's Transform() method. Router.Route(), for example, consumes the mapper passed to it by calling its Transform() method. For mappers derived from MapperBase, note that it implements Transform() as follows:

1. If set, raises OnMapperStart.
2. For each input:
   1. If set, raises OnTransforming.
   2. Calls TransformInputToOutput().
   3. If set, raises OnTransformed.
   4. Yields the output.
3. If set, raises OnMapperEnd.

An event handler is invoked only if it is set (not null). If none of the events are set, the call is basically a loop to call the TransformInpuToOutput() implementation.

Get Mendz.ETL and start building your mappers like a pro!