The Tokenize transformation iteratively applies a regular expression against the input string.
However, unlike RETransform, Tokenize does not have to match the entire string: the regular expression used for the Tokenize transformation can match a subset of the input. After a match is found, Tokenize applies the regular expression again, starting at the character after the end of the last match.
Descriptive name of the transformation. You can enter any name here.
True or false. Specifies whether the match is case-sensitive.
Optional. Notes about the transformation.
The conditions under which this transformation is applied.
The default value to use if the condition is met and the input value is either not available or the regular expression does not match the input value.
The regular expression used for matching.
The names of the output strings. You can have multiple outputs for a given input string. The number of outputs must correspond to the number of capturing sub-patterns in the regular expression.
In the following example, the Tokenize transformation uses a regular expression to capture the names of the query strings (in cs-uri-query) and output the captured sub-pattern (the query name) to x-pull-query-name.
For the query string "a=b&c=d," the output would be a vector containing "a" and "c."
For information about regular expressions, see Regular Expressions .