The order automations run in is decided by the engine, and a conversion swaps the engine. Conditions and formulas usually come across intact, since they’re what the converter translates and a reviewer reads line by line. The converter doesn’t translate the decisions the old engine made on its own, and those are the ones that change.
Many teams have recently moved legacy declarative automation onto a newer flow engine. I’d treat each conversion as a rewrite of code nobody has read end to end, and test it like a schema migration. Map every write first, then replay real saves through both versions and compare the database afterward.
Decisions the old engine made without asking
Run order is the obvious one. The old engine ran its automation types in a fixed sequence that somebody learned once and never wrote down. The new engine has its own default, so two steps touching one field can swap places without either one changing.
Then the save phase: before the record is written, after it’s written in the same transaction, or later in the background. Conversions tend to pull work earlier: field updates move before the save, where they cost no extra write, and background steps land inside it. That’s defensible, but a step that used to run once everything had settled now runs while other steps are still writing.
The third shift is run-as context. Plenty of old automations ran in system context and could read any field. A converted step may run as the user who pressed save, and if that profile hides a field, the step can read null and carry on without error.
A write map from the automation metadata
Automation definitions usually export as metadata, and that’s enough to fill one table with a row per step: automation, object, event phase, run position, run-as context, and the fields it reads and writes. Fields carry their object, like account.support_tier, because steps often touch records other than the one being saved.
Two kinds of row are hazards. A field with more than one writer keeps whatever the last writer set. A field one step reads before another writes it on the same save hands the reader a stale value, or nothing. Build the table for the old and the converted automations, and query both. A hazard found only in the converted list is a change nobody meant to make.
-- One row per automation step. Fields are named object.field.
CREATE TABLE automation_step (
step_id text PRIMARY KEY,
automation text NOT NULL,
object text NOT NULL, -- the object whose save runs this step
event_phase text NOT NULL
CHECK (event_phase IN ('before_save', 'after_save', 'async')),
run_position integer NOT NULL, -- order within object and phase; ties mean the engine picks
runs_as text NOT NULL CHECK (runs_as IN ('system', 'user')),
reads text[] NOT NULL DEFAULT '{}',
writes text[] NOT NULL DEFAULT '{}'
);
-- Both hazard types: fields with more than one writer, and fields a step
-- reads before another step on the same save writes them (same phase at the
-- same or a later position, or any later phase).
WITH phase AS (
SELECT * FROM (VALUES ('before_save', 1), ('after_save', 2), ('async', 3))
AS p(event_phase, phase_order)
),
step AS (
SELECT s.*, p.phase_order
FROM automation_step AS s
JOIN phase AS p ON p.event_phase = s.event_phase
),
w AS (
SELECT s.*, f.field
FROM step AS s CROSS JOIN LATERAL unnest(s.writes) AS f(field)
),
r AS (
SELECT s.*, f.field
FROM step AS s CROSS JOIN LATERAL unnest(s.reads) AS f(field)
)
SELECT 'multi_writer' AS hazard,
field,
string_agg(DISTINCT step_id || ' (' || event_phase || ' #' || run_position
|| ', runs as ' || runs_as || ')', '; ') AS detail
FROM w
GROUP BY field
HAVING count(DISTINCT step_id) > 1
UNION ALL
SELECT 'read_before_write' AS hazard,
r.field,
r.step_id || ' reads at ' || r.event_phase || ' #' || r.run_position
|| ' (runs as ' || r.runs_as || '), then ' || w.step_id
|| ' writes at ' || w.event_phase || ' #' || w.run_position AS detail
FROM r
JOIN w
ON w.field = r.field
AND w.object = r.object
AND w.step_id <> r.step_id
AND (w.phase_order > r.phase_order
OR (w.phase_order = r.phase_order AND w.run_position >= r.run_position))
ORDER BY hazard, field;
Ties count, because the engine picks which goes first. The map can’t show what a profile is allowed to see. The replay can.
Replaying recorded saves in two sandboxes
Record a few hundred real saves from production, each with its payload and the acting user’s profile. Replay each one, as that profile, into two sandboxes, one running the old automations and one the converted. Wait for background work to settle, then diff every touched record field by field, ignoring ids and timestamps.
The settings live in a manifest, so a rerun tests the same thing.
# replay.yml: paired replay for one conversion
conversion: support-automations
sample:
source: prod_save_log # each save's payload plus the acting user's profile
window_days: 30
size: 500
stratify_by: [profile, event]
scrub: [contact.email, contact.phone, case.description]
sandboxes:
baseline: sbx-old-automations
candidate: sbx-converted-automations
replay_as_profiles:
- support_agent
- portal_customer
- support_ops
- integration_user
settle:
min_wait_seconds: 60
until: background_queue_empty
max_wait_seconds: 900
compare:
match_records_on: external_key # ids differ between sandboxes
ignore_fields:
- "*.id"
- "*.created_at"
- "*.updated_at"
- "*.last_modified_by"
cutover_order:
- object: case
max_diffs: 0
- object: account
max_diffs: 0
- object: contact
max_diffs: 0
Stratify by profile. Internal users make most saves, so a plain random 500 can hold almost no portal customers or integration users, whose access looks least like an admin’s.
Scrubbed data and a queue that has to drain
Both sandboxes need production-shaped data, scrubbed: the accounts, contacts and entitlements each payload points at. Getting there can take longer than the conversion itself.
Background jobs make early diffs noisy: a record read before its background step finishes shows a difference that isn’t real. I’d set a minimum wait, poll until the queue is empty, and run the sample twice. If the runs disagree, the settle time is too short or a step depends on the clock, and no count means anything until that’s fixed.
Pinning the order instead of inheriting it
Where two steps write the same field, merge them, or have the others compute their value and hand it to one step instead of writing it. Pin run order wherever the engine allows it, because a default order belongs to the engine’s authors and a later release can change it.
Run-as deserves a decision per step. I’d allow system context only for steps with a short, fixed write list, since a step that can read everything can also copy a hidden value somewhere visible.
Portal cases that lost their priority
Consider a support team I’ve sketched for this post, at a software company that runs cases in its CRM. In the old setup, step A fills a blank case priority from the account’s support tier, in system context, and a background step B later sets the response target in hours from that priority. Agents pick a priority on the form. Portal cases arrive without one, so A sets it.
After conversion, B runs inside the same save, ahead of A, and both run as the user.
Replaying 500 recorded saves gives 37 diffs, every one a portal case with a blank priority and response target. Agent cases come through clean, because B reads the priority the agent picked.
The write map flags B for reading case.priority before A writes it, but pinning A first still leaves 37 diffs. The runs_as column explains why: converted A runs as the portal customer, who can’t see account.support_tier, so A reads null and leaves the priority blank. Two causes sit on the same 37 cases, and fixing either alone would leave the count where it is.
The fix merges them into one step that sets both fields, pinned first before save and running in system context. It reads the tier and writes nothing but those two fields. The next replay shows zero diffs, and case cuts over. Account is next.
Before an object cuts over
- Every hazard found only in the converted write map has an owner.
- Every multi-writer field is down to one writing step, or has a written reason why not.
- Run order is pinned for every object and phase the converted steps touch.
- Each step running as the user is checked, per profile, for fields it can’t read.
- The sample covers every profile that saved this object in the window, portal and integration users included.
- Two replays of the same sample give identical diffs.
- The diff count is zero, or each remaining diff is signed off by that field’s owner.
- Old automations switch off in the release that switches the converted ones on, and rollback reverses both. Leaving the old ones running for a week to be safe gives every shared field two writers, the exact hazard the write map exists to catch.