Creating Mendz.ETL-based Validators

ETL involves agreements between partners with regards to how they will "speak" with each other. Validating what a partner sends and what you send to your partner is an important ETL requirement. Mendz.ETL supports the creation of validators, which can be called at the start and at the end of an ETL flow.

Validators derived from ValidatorBase implement the ValidateDocument() method. Developers can apply their preferred strategy, technique and technology to validate the document. The document can be an input or an output. Thus, it does not matter if it's an incoming or an outgoing document. What matters is that the document would pass the validation.

XML has XSD, which can be used to define an XML document format. Mendz.ETL intentionally does not define its own XsdValidator. Developers are encouraged to create their own XsdValidator. If created generic enough, it can be used where ever an XML validation is required.

Other formats like delimited, positional and even JSON don't really have any standard language for defining schemas. Brute force programming may be needed to validate them. Inventing your own "format definition language" is an option. Perhaps you can even use regular expressions. Regardless, whatever approach you use, validators have their special place in Mendz.ETL as ValidatorBase implementations.

The source and target adapters are the best place to validate documents. Mendz.ETL adapters support validator dependency injection via *Validator property assignment. In the source adapter, validation can be done before extracting. In the target adapter, validation can be done after loading. This is how the SourceAdapterBase and TargetAdapterBase are implemented. Thus, it is recommended that developers derive from these base classes to create adapters.

ValidatorBase implements the Validate() method as follows:

1. If document specification's IsValidate is false, return true.
2. If document specification's IsValidate is true:
   1. If set, raises OnValidating.
   2. Calls ValidateDocument().
   3. If set, raises OnValidated.
3. If exception is thrown:
   1. If IsThrowValidationException is true, rethrows the exception.
   2. If IsThrowValidationException is false, sets an ETLValidatorEventArgs instance's Exception property to the exception thrown, and IsValid property to false. Implementers are encouraged to set an OnValidated event handler, which can evaluate and act on its ETLValidatorEventArgs instance's Exception and IsValid properties.

By default, IsThrowValidationException is false, which is designed to let the validators and adapters complete their respective event sequences. ETLSourceAdapterEventArgs and ETLTargetAdapterEventArgs have the IsValid property which is set after calling Validate(). The adapter's event handlers can evaluate the IsValid property to perform status relevant operations (ex. cleanup, rollback, etc.).

An event handler is invoked only if it is set (not null). If none of the events are set, the call is basically a call to the ValidateDocument() implementation.

Get Mendz.ETL and start building your validators like a pro!