Save records
Write a workflow's output into a data object, and restate a subset of it without touching the rest.
Save Record is the step that writes a workflow's output back into your workspace. Whatever the chain produced, a parsed file, a reconciled table, an agent's structured answer, this is where it lands as records in a data object.
Most of the time you want it to append. Sometimes you want it to restate part of an object: re-stream one valuation date for one portfolio, and leave everything else alone. That is what replacement is for, and it is the option worth understanding before you use it.
Add the step
- In the editor, select the step whose output you want to store.
- In the Configure tab, find Save Record in the Data group (or type it into Search actions...). It is added after the selected step, already connected.
- In the side panel, choose the Data Object to write into.
- Click Save.
Save Record takes a Table. Most steps that produce records output one; an Ask Agent step does when its Structured output is on, so an agent's answers can be saved as records directly.
If the object does not exist yet, turn on Auto Create Data Object: a Data Object Name field appears for its name, and SPARK creates it on the first run rather than failing. Turning it on also turns on Auto Create Attributes, since an object the run creates has no attributes yet and a save that matches none would fail; either can still be turned off. Leave the toggle off when you would rather a missing object be an error.
Options for the write
| Option | What it does |
|---|---|
| Auto Create Attributes | Creates any attribute the incoming records carry but the object does not have yet. A column holding JSON or values longer than 255 characters gets a long text attribute |
| Sanitize Column Names | Strips characters attribute names cannot hold, so an awkward source header still maps. Needs Auto Create Attributes on |
| Overwrite Records | Overwrites existing records whose identifier matches an incoming one, instead of leaving them untouched |
| Ignore Mismatching Attributes | Skips a record whose value does not match the stored attribute type, rather than failing the run |
| Drop Duplicates | On by default. Drops duplicate records within the incoming batch |
Restate a subset of an object
Turn on Replace Existing Records and the step stops appending. Instead it deletes the existing records that the incoming data is restating, then writes the new rows.
What counts as "restating" is the Replace Key Columns you pick. Every existing record whose combination of those columns appears in the incoming data is removed first. If your payload carries two dates for one portfolio, those two date-and-portfolio combinations are cleared and nothing else is.
- Turn on Replace Existing Records. Two more fields appear.
- Open Replace Key Columns (Select key attributes) and pick the attributes that identify the subset, for example Date then Portfolio ID. Search the list, and drop a column with its ×.
- Optionally turn on Archive Replaced Records.
The picker lists the target object's real attributes rather than accepting free text, so a key column cannot be a typo. Order is yours to choose, and it is shown back the way you set it.
What gets deleted, exactly
Three rules decide the scope, and each one is worth knowing before a production run:
- Only the combinations present in your data. A record whose key combination is absent from the payload is never touched.
- An empty value is part of a combination. A record with no Portfolio ID is replaced by an incoming row with no Portfolio ID. It is a value, not a wildcard.
- Key columns are required. Replacement with no key columns would mean replacing the whole object, so the run is rejected instead, with Select at least one Replace Key Column when Replace Existing Records is enabled.
By default the replaced records are deleted permanently. Archive Replaced Records keeps them instead: they stay in the object, out of every default view, and can be restored. Not every object type supports archiving, so check that yours does before relying on it.
The system columns SPARK maintains, the id and the created and updated stamps, cannot be key columns and are not offered in the picker. An incoming record never carries a value for one, so a key built on them would match only what the platform happened to stamp.
When a save is refused
When the data object refuses the records, the step fails with the reason, naming the attribute and, for a long value, the row. For example:
- Code: Must be at most 2 characters. for an attribute's own rule
- Code must be unique: a record with Code = "IT" already exists. for a unique attribute
- Notes: Must be at most 255 characters; row 3 has 1,204. for a short text attribute given a long value
A save of more than 5,000 records goes in parts of 5,000, and the parts before a refused one are already written, so the reason says so: Records 1 to 5,000 were saved; the save stopped at records 5,001 to 10,000: followed by the reason. Fix the data or the attribute, then rerun.
Check the result
Open the run from the Executions tab and click the Save Record step to see what it wrote on its Output tab. On a first replacement run, compare the record count in the target object before and after, and confirm the rows you expected to survive are still there.