Mutate string processors

    The substitute_string processor matches a key’s value against a regular expression (regex) and replaces all returned matches with a replacement string.

    You can configure the substitute_string processor with the following options.

    Usage

    To get started, create the following pipeline.yaml file:

    copy

    Next, create a log file named logs_json.log. After that, replace the path of the file source in your pipeline.yaml file with your file path. For more detailed information, see Configuring Data Prepper.

    Before you run Data Prepper, the source appears in the following format:

    1. {"message": "ab:cd:ab:cd"}

    After you run Data Prepper, the source is converted to the following format:

    1. {"message": "ab-cd-ab-cd"}

    from defines which string is replaced, and to defines the string that replaces the from string. In the preceding example, string ab:cd:ab:cd becomes ab-cd-ab-cd. If the from regex string does not return a match, the key is returned without any changes.

    The split_string processor splits a field into an array using a delimiter character.

    Configuration

    You can configure the split_string processor with the following options.

    To get started, create the following pipeline.yaml file:

    1. pipeline:
    3.     file:
    4.       record_type: "event"
    5.       format: "json"
    6.   processor:
    7.     - split_string:
    8.         entries:
    9.           - source: "message"
    10.             delimiter: ","
    11.   sink:
    12.     - stdout:

    Next, create a log file named logs_json.log. After that, replace the path in the file source of your pipeline.yaml file with your file path. For more detailed information, see Configuring Data Prepper.

    Before you run Data Prepper, the source appears in the following format:

    1. {"message": "hello,world"}

    After you run Data Prepper, the source is converted to the following format:

    The uppercase_string processor converts the value (a string) of a key from its current case to uppercase.

    Configuration

    You can configure the uppercase_string processor with the following options.

    Usage

    To get started, create the following pipeline.yaml file:

    1. pipeline:
    2.   source:
    3.     file:
    4.       path: "/full/path/to/logs_json.log"
    5.       record_type: "event"
    6.       format: "json"
    7.   processor:
    8.     - uppercase_string:
    9.         with_keys:
    10.           - "uppercaseField"
    11.   sink:
    12.     - stdout:

    copy

    Next, create a log file named logs_json.log. After that, replace the path in the file source of your pipeline.yaml file with the correct file path. For more detailed information, see Configuring Data Prepper.

    Before you run Data Prepper, the source appears in the following format:

    1. {"uppercaseField": "hello"}

    After you run Data Prepper, the source is converted to the following format:

    1. {"uppercaseField": "HELLO"}

    The lowercase string processor converts a string to lowercase.

    Usage

    To get started, create the following pipeline.yaml file:

    1. pipeline:
    2.     file:
    3.       path: "/full/path/to/logs_json.log"
    4.       record_type: "event"
    5.       format: "json"
    6.   processor:
    7.     - lowercase_string:
    8.         with_keys:
    9.           - "lowercaseField"
    10.   sink:
    11.     - stdout:

    copy

    Next, create a log file named logs_json.log. After that, replace the path in the file source of your pipeline.yaml file with the correct file path. For more detailed information, see Configuring Data Prepper.

    Before you run Data Prepper, the source appears in the following format:

    After you run Data Prepper, the source is converted to the following format:

    1. {"lowercaseField": "testmessage"}

    The trim_string processor removes whitespace from the beginning and end of a key.

    Configuration

    You can configure the trim_string processor with the following options.

    To get started, create the following pipeline.yaml file:

    1. pipeline:
    2.   source:
    3.     file:
    4.       path: "/full/path/to/logs_json.log"
    5.       record_type: "event"
    6.       format: "json"
    7.   processor:
    8.     - trim_string:
    9.         with_keys:
    10.           - "trimField"
    11.   sink:
    12.     - stdout:

    copy

    Next, create a log file named logs_json.log. After that, replace the path in the file source of your pipeline.yaml file with the correct file path. For more detailed information, see Configuring Data Prepper.

    Before you run Data Prepper, the source appears in the following format:

    1. {"trimField": " Space Ship "}
    1. {"trimField": "Space Ship"}