Stream Lookup

    The data coming from the Source transform is first read into memory and is then used to look up data from the main stream.

    OptionDescription

    Transform name

    Name of the transform this name has to be unique in a single pipeline

    Lookup Transform

    The Transform name where the lookup data is coming from

    The keys to lookup…​

    Fields to retrieve

    You can specify the names of the fields to retrieve here, as well as the default value in case the value was not found or a new field name in case you didn’t like the old one.

    Preserve memory

    Encodes rows of data to preserve memory while sorting. (Technical background: Hop will store the lookup data as raw bytes in a custom storage object that uses a hashcode of the bytes as the key. More CPU cost related to calculating the hashcode, less memory needed.)

    Key and value are exactly one integer field

    Preserves memory while executing a sort by . Note: Works only when “Preserve memory” is checked. Cannot be combined with the “Use sorted list” option.

    Enable to store values using a sorted list; this provides better memory usage when working with data sets containing wide row. Note: Works only when “Preserve memory” is checked. Cannot be combined with the “Key and value are exactly one integer field” option.

    Get fields

    Automatically fills in the names of all the available fields on the source side (A); you can then delete all the fields you don’t want to use for lookup.

    Get lookup fields

    Automatically inserts the names of all the available fields on the lookup side (B). You can then delete the fields you don’t want to retrieve

    All fields of this transform support metadata injection. You can use this transform with ETL Metadata Injection to pass metadata to your pipeline at runtime.