Populating the Lookup Table
If you use the Categorize or FlatFileLookup transformations, the lookup table is loaded in memory and populated from a flat file whose location you specify when you define the transformation.
The flat file you specify must meet the following requirements:
- Each line in the file must represent a row in the lookup table.
- Columns in the file must be separated by an ASCII delimiter. You may use any character that is not a line-ending character and does not appear anywhere within the event data itself. When you define the transformation, you specify which character has been used to delimit the columns in the flat file.
If you use an ODBCLookup transformation, the lookup table is loaded into memory and populated from a table or view in an ODBC database that you specify. When you define the transformation, you must also specify the data source, user name, and password that the data workbench server must use to establish a connection to the database.
Lookup tables are loaded when the data workbench server initially begins constructing the dataset. Once established, lookup files are not meant to be changed. If you change the flat file or ODBC table that is used for the transformation phase, you are required to retransform the entire dataset. If you change a flat file that is used during the log processing phase, the new lookup data is applied to all new records that enter the dataset, but the changes are not applied retroactively.