Testing and Sampling

Thorough testing is essential for successful data migrations. This article covers strategies for validating your migration configuration before running it against production systems.

Two Ways to Run a Small Subset

Precision Bridge offers two distinct mechanisms for working with a subset of records, and they serve different purposes:

• Filter Limit (Filter tab) — extracts only the first N matching records. Use this during development to iterate quickly on field mappings and run a partial migration end-to-end.
• Sampling Validation (Validation tab, under Advanced Options) — runs the full migration but then verifies a random sample of migrated records against the source to detect mapping or data-fidelity issues. This is a post-migration accuracy check, not a development shortcut. See Advanced Options.

The remainder of this article uses "sampling" in the development sense — small subsets you migrate while iterating on mappings.

How to Use Sampling

1. Open the Migrate Records step.
2. On the Filter tab, set a Limit (e.g., 10–50 records) or tighten the filter to a single record number.
3. Execute the step — only the limited subset is extracted and processed.
4. Review the results to verify field mappings, lookups, and transformations.

What to Check

• Field values appear in the expected format
• Null values are handled correctly (no unexpected empty fields)
• Formula mappings produce the expected output
• Choice field translations are correct (match mappings)
• Lookup values resolve to the correct target records

Iterative Development Workflow

The most reliable approach to migration development follows an iterative pattern, gradually increasing scope:

Step 1: Single Record Test

Start with the narrowest possible filter — a single record by its unique ID:

[sys_id] = "abc123def456"

This verifies basic connectivity, table access, and that the step executes without errors.

Step 2: Small Sample (5-10 Records)

Expand to a small set of records to test field mappings across different data patterns:

[sys_id] in ["abc123", "def456", "ghi789", ...]

Check that your mappings handle different data values correctly — empty fields, special characters, different choice values.

Step 3: Broader Subset (25-50 Records)

Raise the Filter Limit (or broaden the filter) to a wider subset that exercises more of the data variation. This catches mapping issues that didn't surface in the narrow tests above.

Step 4: Full Run in Dev/Staging

Run the complete migration against a non-production target system. Review the execution report for: - Total record counts (extracted, inserted, updated, skipped, failed) - Error details for any failed records - Warnings that might indicate data quality issues

Step 5: Production Run

Once dev/staging results are validated, run against the production target.

Using Dev/Staging Environments

Always test in a non-production environment before migrating to production.

Setup

• Create separate connections for development, staging, and production targets.
• Name them clearly: Target - ServiceNow Development, Target - ServiceNow Production.
• Use the same source connection (reading from source is non-destructive).

Safety Practices

• Keep production target connections disabled or in a separate project until ready
• Review all field mappings and filters one final time before switching to production
• Consider having a colleague review the configuration before the production run

Comparing Source and Target

After a migration run, spot-check records in the target system to verify data quality.

What to Verify

• Field values: Key fields match the expected values from the source.
• Reference fields: Reference fields (e.g. assigned_to, parent) resolve to the correct target records, validating that ID mappings and table correlations worked.
• Choice fields: Translated values are correct (e.g. priority 1 mapped to Critical).
• Attachments: If attachment migration was enabled, files are present on the target records.
• Record counts: The number of migrated records matches expectations.

Logging and Monitoring

During Execution

• Watch the execution log for warnings and errors in real-time
• Monitor record progress (e.g., "Processing record 500 of 10,000")
• Check memory usage for very large migrations

After Execution

• Review the execution report summary (record counts by status)
• Examine error logs for failed records — each entry includes the source record data and error message
• Check for warnings even on successful runs — they may indicate data quality issues worth investigating

Regression Testing for Re-runs

Many migrations are designed to be run multiple times (e.g., incremental syncs or repeated test runs).

Key Mapping Behaviour

Key mappings ensure re-runs don't create duplicate records: - Update: Existing records are overwritten with new values - Skip: Existing records are left unchanged - Error: Execution stops if a duplicate is found

Testing Re-run Behaviour

1. Run the migration once — verify records are inserted correctly
2. Run the same migration again — verify key mappings trigger the expected behaviour (update/skip)
3. Modify a source record, run again — verify the target record is updated correctly

Related Documentation

• Advanced Options — record sampling and validation rules
• Key Mappings and Duplicate Handling — configuring duplicate detection
• Error Handling and Validation — validation rules and error patterns
• Executing a Procedure — running procedures and reviewing results