Step Type: Extract Records

The Extract Records step extracts a set of records from a source table and holds them in memory for processing by later steps. Unlike Migrate Records, extracted records are not sent to a target connection.

When to Use

• To extract data for intermediate processing before migration
• To gather records for analysis, validation, or conditional logic
• To build lists of values used by subsequent steps (e.g., for iteration)
• To extract records from one table for use in formulas or lookups in a later step

Configuration

Setting	Description
Connection	The source connection to extract from
Table	The source table to query
Fields	Specific fields to extract (optional — defaults to all fields)
Query Filter	Conditions to limit which records are extracted
Record Limit	Maximum number of records to extract (optional)
Extract Attachments	Optionally extract associated file attachments
Keyset Pagination	Enable keyset-based paging for reliable extraction of large datasets

Outputs

The Outputs tab captures information from the extraction into one or more procedure variables. Each row defines a single variable, with a Type that controls the shape of the value the variable holds. Multiple outputs from the same extraction are supported — for example, capturing a list of IDs and the record count in two separate variables.

Output Types

Type	Shape	Example	Typical Use
List of full records	List of records	[Record(id=1, …), Record(id=2, …)]	Iterate over each record in a later step
List of record IDs	List of primary-key values	[1, 2, 3]	Pass IDs into a subsequent filter or API call
List of values from a field	List of one field's values	["high", "medium", "high"] for priority	Aggregate or distinct-filter on a single field
Count of records	Integer	3	Step condition such as @count > 0, logging
Whether any records exist	Boolean	true	Condition gate on downstream steps
First record	Record (or None)	Record(id=1, …)	"Fetch one" patterns expecting a single record
First value from a field	Scalar	1 for id	Single-value lookup, e.g. email = @input → user_id
Dictionary of key field → value field	Dict	{1: "high", 2: "medium", 3: "high"}	Reference lookup later in the procedure
Dictionary of records keyed by a field	Dict of records	{1: Record(…), 2: Record(…)}	Indexed access during iteration or in a formula

Keys derived from a non-unique field collide — only the last record for each key is kept in the dictionary outputs.

Configuring an Output Variable

1. Open the Extract Records step and navigate to the Outputs tab.
2. Click the + button to add a new output variable.
3. Give the variable a name — subsequent steps reference it by name.
4. Choose a Type from the list above. For field-parameterised types, pick the source field(s) using the field selector that appears.
5. Repeat for any additional outputs needed from the same extraction.

Output variables are populated after the step finishes. They persist for the remainder of the procedure run and are available to all subsequent steps. A variable name that matches an existing variable overwrites it.

How It Works

1. The step connects to the specified source connection
2. Records are queried from the table using the configured filter
3. Only the specified fields are extracted (or all fields if none specified)
4. If attachments are enabled, associated files are also extracted
5. The configured output variables are populated from the extracted records (see Outputs)
6. Progress is tracked and reported during extraction

Example

Extract all active incidents from ServiceNow for processing:

1. Set the connection to your ServiceNow source
2. Set the table to incident
3. Set the filter to active=true
4. On the Outputs tab, add an output variable named active_incidents with type List of full records
5. Later steps reference active_incidents for iteration or analysis

To capture multiple shapes from the same extraction, add additional output rows — for example, an active_incident_ids variable of type List of record IDs alongside active_incidents, or an active_count variable of type Count of records for use in a conditional later in the procedure.