Migrations and Execution

This article covers questions about how migrations work end-to-end, how Precision Bridge handles re-runs, parent-child references, attachments, and incremental syncs.

How a Migration Runs

A Migrate Records step performs four phases for each chunk of records it processes:

1. Extract — connect to the source, apply the filter, retrieve records (and any configured lookup data).
2. Transform — for each record, evaluate all field mappings to produce the target record. This covers direct mappings, formulas, value translations (Match), cross-table retrievals (Lookup), and ID translations against table correlations.
3. Load — for each transformed record, check the target against the configured key fields. Depending on the match outcome and your settings, the record is inserted, updated, or skipped.
4. Output — record counts and any configured output variables are populated; table correlations are updated.

For the full reference, see Migrate Records Overview.

One Procedure vs. Many

A typical migration is one procedure with many ordered Migrate Records steps, not one procedure per entity. Order steps inside the procedure by dependency: identity data (users, contacts), then reference data (groups, categories), then transactional records (incidents, requests), then child records (journal entries, comments).

Use separate procedures only when there is a clear boundary — a different team owns the work, the schedule is different, or you need to invoke the workflow from multiple parents via the Call Procedure step. See Project Organization for the full pattern.

Re-Running a Migration Without Duplicates

Re-runs are safe when you configure key mappings on each Migrate Records step. A key mapping nominates one or more source/target field pairs that uniquely identify a record. On each run:

• If the target already contains a matching record, the configured record-exists action runs (Update, Skip, or Error).
• If no match is found, the record is inserted.

Combined with table correlations (which persist source→target ID maps across runs), a procedure can be run repeatedly — for example, on a schedule for incremental sync — without producing duplicates. See Key Mappings and Duplicate Handling.

Parent-Child References

When a child record references a parent (e.g. an incident's assigned_to field referencing a user), Precision Bridge translates the source parent ID into the corresponding target parent ID at insert time using a Table Correlation or a formula-based lookup.

This requires the parent step to have run first. If the user step has not produced any correlation entries when the incident step runs, assigned_to will resolve to empty or raise an error depending on your validation settings. Always order steps inside the procedure parent-first.

See Table Correlations Overview and ID Mappings for the full pattern.

Migrating Attachments

Attachment migration is a toggle on the Migrate Records step itself — not a separate step or a separate procedure. When the toggle is on, every migrated record is checked for attachments on the source; matching attachments are downloaded from the source and re-uploaded to the corresponding target record.

Considerations:

• Attachment migration is significantly slower than record migration because of the file download/upload round trip. A record with multiple large attachments may take seconds rather than milliseconds.
• Ensure the target system has the storage and per-attachment size budget to accept the migrated files.

See Attachment Migration.

Full Migrations vs. Incremental Migrations (Delta Migrations)

A full migration extracts all records matching the filter on every run, and lets key mappings handle insert-vs-update. Use it for one-time migrations and small datasets.

An incremental migration (also called a delta migration) uses a date-based filter to extract only what has changed since the last run.

The recommended approach uses the built-in system variable LAST_EXECUTION_TIMESTAMP, which holds the start time of the previous successful run and is maintained automatically by the platform:

[sys_updated_on] > LAST_EXECUTION_TIMESTAMP

No procedure input variable, no Assign Variables step, no external persistence — the next run reads the previous run's start time directly. The companion variable CURRENT_EXECUTION_TIMESTAMP holds the start time of the current run, which is useful when a later step in the same procedure needs the same instant the run began at.

On the first run, LAST_EXECUTION_TIMESTAMP is empty. Either toggle the Override switch on the variable for the first run (and pick a starting cutoff from the past-executions combobox or a custom value), or use a formula filter with a fallback such as LAST_EXECUTION_TIMESTAMP or datetime(2000, 1, 1, tzinfo=timezone.utc).

The Override switch is also how you re-run from a specific point in time — for example, to replay the last 48 hours after a target-side issue.

A custom procedure input variable (e.g. last_run_date) is still appropriate when the cutoff is sourced from outside the procedure (a target record, a CSV file, an external orchestrator). In that case, persist the cutoff explicitly between runs because procedure variables reset to their defaults at the start of every run.

See Migration Strategy — Incremental Migration for the full walkthrough.

Scheduling Migrations

Scheduled procedures are available in the Docker build only. The Schedules page is disabled in the Windows Native build. In the Docker build, a schedule can be set to run a procedure once at a specific time or repeatedly on an interval.

If you are on the Native build and need scheduling, switch to the Docker build — the same project files are compatible across builds.

Pre-Flight Validation and Dry Runs

Before running a full migration, validate with a small subset:

• Set a Limit on the Filter tab (e.g. 10–50 records) or tighten the filter to a single record number. Iterate on field mappings until the result is correct.
• Once mappings are stable, enable Sampling Validation (Validation tab) for production runs. After the migration completes, Precision Bridge re-queries a random sample of migrated records from the target and reports any mismatches between what it sent and what the target stored.

These two mechanisms are complementary, not interchangeable — Limit is a development shortcut, Sampling Validation is a post-migration accuracy check. See Testing and Sampling.

Per-Chunk Scripts for Bespoke Transformations

If a field-level transformation can't be expressed as a formula — for example, it needs a Python library, cross-record context, or per-chunk artifact output — attach a per-chunk script to the Migrate Records step. Scripts run in an isolated Python subprocess once per chunk, either before insert (mutations write back to the target) or after insert (observation and artifact generation).

See Per-Chunk Scripts.

Reviewing Execution Results

Every run produces an execution report accessible from the execution view. Headline metrics: extracted, inserted, updated, skipped, failed. Failed records carry the source data and the underlying error message so you can diagnose without re-running.

The report is exportable to CSV — attach it to any support ticket you raise. See Executing a Procedure.