Configuring Lookups
The Lookups tab on a Migrate Records step lets you define lookup configurations that retrieve values from related tables. Lookups are essential when the data you need is not directly available on the source record — for example, when the source record contains a user ID reference but you need the user's email address, which lives in a separate user table.
What Lookups Do
During extraction, Precision Bridge fetches the lookup table data and indexes it by the key field you specify. Then, for each source record, PB can match the record's reference value against the lookup index to retrieve one or more fields from the related table.
The lookup results are made available as a mapping type on the Fields tab, where you create a Lookup field mapping to write the retrieved value into a target field.
The Lookups Tab
The Lookups tab displays a list of all lookup definitions configured for the step. Each lookup definition specifies:
- Lookup name — A descriptive label for the lookup (e.g., "User Lookup", "Category Lookup")
- Connection — The connection used to access the lookup table (often the same as the source connection, but can be different)
- Lookup table — The table containing the related data
- Key field — The field in the lookup table that will be matched against the source record's reference value
- Value field(s) — One or more fields to retrieve from the lookup table when a match is found
Creating a Lookup Definition
To create a new lookup:
- Navigate to the Lookups tab on the Migrate Records step
- Click Add Lookup to create a new lookup definition
- Name the lookup descriptively (e.g., "Assigned To User")
- Select the connection that provides access to the lookup table
- Select the lookup table from the available tables on that connection (e.g.,
sys_user) - Select the key field — this is the field in the lookup table that will be used for matching. Choose the field that corresponds to the reference value on the source record (e.g.,
sys_idonsys_userif the source record stores usersys_idreferences) - Select the value field(s) — choose one or more fields to retrieve when a match is found (e.g.,
email,name)
How the Key Field Works
The key field establishes the link between the source record and the lookup table:
- The source record contains a reference value (e.g., an
assigned_tofield with a usersys_id) - The lookup's key field is the corresponding field in the lookup table (e.g.,
sys_idonsys_user) - During processing, PB matches the source record's reference value against the lookup table's key field
- When a match is found, the configured value fields are returned
The key field should be a unique identifier in the lookup table — typically a primary key or system ID. If the key field is not unique, PB uses the first matching record.
Multi-Field Lookups
A single lookup definition can retrieve multiple fields from the same lookup table. This is more efficient than creating separate lookups for each field you need.
For example, a "User Lookup" on sys_user might retrieve:
email— The user's email addressname— The user's display namedepartment— The user's department
Each retrieved field can then be used in a separate Lookup field mapping on the Fields tab, all referencing the same lookup definition.
Using Lookup Results in Field Mappings
After creating a lookup definition on the Lookups tab, you use it on the Fields tab:
- Navigate to the Fields tab
- Add or edit a field mapping for the target field you want to populate
- Select Lookup as the mapping type
- Choose the lookup definition you created
- Select which value field from the lookup to use for this mapping
- Configure the source expression — this is the field on the source record that contains the reference value to match against the lookup's key field
For example:
| Target Field | Mapping Type | Lookup | Value Field | Source Expression |
|---|---|---|---|---|
u_assigned_email |
Lookup | Assigned To User | email |
[assigned_to] |
u_assigned_name |
Lookup | Assigned To User | name |
[assigned_to] |
Both mappings use the same "Assigned To User" lookup but retrieve different value fields.
Performance Considerations
Lookups add processing overhead because PB must fetch and index the lookup table data before processing records. Keep these points in mind:
- Large lookup tables can slow down the extraction phase. If the lookup table has millions of records, consider adding a filter to the lookup to limit the data fetched (e.g., only active users).
- Multiple lookups each add their own data fetch. Minimize the number of distinct lookup tables when possible by using multi-field lookups.
- Lookup data is cached for the duration of the step execution. If the lookup table data changes during a long-running migration, the step will use the snapshot taken at the start.
- Connection differences — If the lookup table is on a different connection than the source, PB makes separate API calls to fetch the lookup data. Ensure the lookup connection is performant and accessible.
Cache Key Variables
Each lookup maintains an in-memory cache keyed by the values of the variables used in its query. When two source records produce the same key, PB serves the second one from the cache instead of re-running the query.
By default the cache key is composed of every variable used in the query (in the order they appear). This is the safe choice — two records only share a cache entry when they would have produced the same query.
The Cache Key Variables setting, on the lookup configuration, narrows the key to a subset of those variables. Records that share the selected values then share a cache entry even if their other query parameters differ. This is useful when many records resolve to the same logical lookup but the underlying query still needs additional parameters to fetch the result.
When to Narrow the Cache Key
Consider narrowing the cache key when:
- The lookup result depends on only a subset of the query variables.
- The remaining variables are required by the query (e.g. parent-scoped URL path parameters on an API endpoint), but do not influence the value being looked up.
- Many source records share the values of the narrowed key.
If the selected variables do not uniquely determine the lookup result, cached values will be returned for records that should have produced different results. The narrowed key should always correspond to the field(s) that determine the lookup outcome.
Example: JIRA Issue Transitions
JIRA's /issue/{issue_id}/transitions endpoint returns the available status transitions for a given issue. Two facts about this endpoint matter:
- The
issue_idpath parameter is required on every request — the endpoint cannot be called without it. - The set of transitions an issue can take is determined by its current status (and its workflow), not by the issue itself. Every "In Progress" issue in the same workflow returns the same list.
A lookup that resolves the target transition for each source record will, by default, cache by both issue_id and the source status. With thousands of issues this means thousands of API calls, even though the transition list only varies by status.
Setting Cache Key Variables to the source status (e.g. source_state) narrows the cache key so that all issues sharing a status share a cache entry — reducing the lookup to one API call per distinct status, while the query itself still receives the correct issue_id to fetch the first uncached entry of each group.
List- and Object-Valued Cache Keys
Cache keys support list-valued and object-valued variables (e.g. JIRA's labels field, or a structured payload returned by a formula). List entries and dictionary entries are compared by value, so two records with the same labels — in any insertion order, for dictionaries — share a cache entry.
Example: Looking Up User Email
Scenario: You are migrating incidents from ServiceNow. The source incident table has an assigned_to field that contains a sys_id reference to the sys_user table. The target system needs the user's email address instead of the sys_id.
Step 1: Create the lookup
- Go to the Lookups tab
- Add a new lookup named "User Lookup"
- Connection: Source ServiceNow connection
- Lookup table:
sys_user - Key field:
sys_id - Value fields:
email
Step 2: Create the field mapping
- Go to the Fields tab
- Add a mapping for the target field
u_assigned_email - Mapping type: Lookup
- Lookup: "User Lookup"
- Value field:
email - Source expression:
[assigned_to]
Result: During execution, for each incident, PB looks up the assigned_to sys_id in the sys_user table and writes the matching user's email to the u_assigned_email target field.
Tips
- Name lookups clearly — Use names that describe the relationship (e.g., "Assigned To User" rather than "Lookup 1") so they are easy to identify on the Fields tab.
- Reuse lookups — If multiple field mappings need data from the same related table, create one lookup with multiple value fields rather than multiple separate lookups.
- Filter large lookups — If the lookup table is very large, consider whether you can filter it down to only the records that are relevant to your migration.
- Verify key field uniqueness — The key field should uniquely identify records in the lookup table. If it is not unique, you may get unexpected results.
Comments
0 comments
Please sign in to leave a comment.