Filter Rows

    Once this transform is connected to a previous transform (one or more and receiving input), you can click on the “<field>”, “=” and “<value>” areas to construct a condition.

    To enter an IN LIST operator, use a string value separated by semicolons.

    OptionDescription

    transform name

    Optionally, you can change the name of this transform to fit your needs.

    Send ‘true’ data to transform

    The rows for which the condition specified is true are sent to this transform

    The rows for which the condition specified are false are sent to this transform

    The Condition

    Add Condition

    Click to add conditions. Add condition converts the original condition into a sub-level condition. Click a sub-condition to edit it by going down one level in the condition tree.

    The filter rows transform detects only fields in the input stream. If you want to filter rows based on a variable value, you must modify the previous transform (a table input for example) and include the variable as another field, for example:

    A query:

    FYI - instead of the Table Input trick above you can use the simple “Get Variables” transform to set parameters in fields.

    Filtering special characters

    To filter special characters like explicit EOF (e.g. from old cobol files) Use a REGEX expression in the “filter row” transform with the syntax: “\x{1A}” where \x mean HEX representation and 1A into parenthesis is the EOF char to match in HEX.

    All fields of this transform support metadata injection. You can use this transform with ETL Metadata Injection to pass metadata to your pipeline at runtime.

    The Filter Rows transform is a special MDI scenario, since it has a nested structure of filter conditions. The condition is given in XML format. The condition XML has the same format as we store the pipeline metadata in a .HPL file in XML format. We do not have a DTD (Document Type Definition) for the .HPL XML format, nor the condition.

    It is easy to get to an XML condition:

    1. Create a sample Filter transform with the different conditions you need. This sample transform gives you all the information, such as the values for the functions you use.

    2. Find the <condition> element and its nested elements and modify it accordingly to use it in your MDI scenario.