The experiment filesystem workflow is for complete experiment edits. It pulls supported experiment files to disk, tracks file hashes in .apex/experiment-filesystem.json, plans changes through Apex, and applies only supported file paths.

Choosing a workflow

The default loop for agents building or editing a test is apex experiments pull, edit supported files, apex experiments plan, apex experiments apply, then apex experiments qa status.

Set up a commerce test

Price, shipping, offer and post-purchase tests are written as files and saved with the Apex CLI. You then check and launch them on the experiment page in the dashboard. Tests that are already running are not affected.

1. Install the CLI once and sign in: npm install -g @drip-apex/apex-cli, then follow Install and sign in.

2. Create a draft experiment for the page the test runs on: apex experiments create --name "Free shipping from 60".

3. Pull it into a folder: apex experiments pull <slug> --out ./free-shipping.

4. Create commerce/shipping.json in the pulled folder (a new draft has no commerce file yet). Use one row in arms per variation, with the slugs from variations/. This file shows free shipping from 60 in variant-a and leaves the control unchanged:

   jsonCopy

   {
     "schemaVersion": 1,
     "kind": "shipping",
     "lever": { "catalogSource": "live", "currencyCode": "EUR", "enforcement": "none", "liveFreeShippingThreshold": null },
     "arms": [
       { "variation": "control", "control": true, "key": "control", "label": "Control", "threshold": null, "treatment": "none", "progressCopy": "", "reachedCopy": "", "bannerCopy": "", "flatRate": null, "percentOff": null, "target": { "productGids": [], "variantGids": [] } },
       { "variation": "variant-a", "key": "free-60", "label": "Free shipping from 60", "threshold": 60, "treatment": "free_shipping", "progressCopy": "", "reachedCopy": "", "bannerCopy": "", "flatRate": null, "percentOff": null, "target": { "productGids": [], "variantGids": [] } }
     ]
   }

   Price, offer and post-purchase tests use commerce/price.json, commerce/offer.json and commerce/post-purchase.json. Use apex commerce catalog search to find product and variant IDs for price tests.

5. Check it: apex experiments plan ./free-shipping --json. The plan lists everything launch would refuse. A threshold below your store's own free-shipping minimum needs checkout enforcement; the plan says so and names the missing settings.

6. Save the draft: apex experiments apply ./free-shipping --json.

7. Open the experiment in the dashboard, run QA and launch it there.

Canonical loop

apex experiments pull homepage-hero-trust-copy --out ./homepage-hero
apex experiments plan ./homepage-hero --json
apex experiments apply ./homepage-hero --json
apex experiments qa status ./homepage-hero --json
apex experiments qa probe ./homepage-hero --kind price --json

Edit only supported files in the pulled folder:

experiment.json
targeting.json
metrics.json
qa.json
launch.json
variations/control/variation.json
variations/variant-a/variation.json
variations/variant-a/mutations.json
commerce/price.json
commerce/shipping.json
commerce/offer.json
commerce/post-purchase.json
variations/variant-a/index.css
variations/variant-a/index.js
variations/variant-a/reset.js
variations/variant-a/anti-flicker.json

Commerce drafts are pulled into one typed file per lever. Every variation has an arm row (use sameAsControl: true for no treatment), while revisions, checksums, references, launch evidence, and price baselines remain under read-only serverOwned. mutations.json contains DOM mutations and checkout blocks, not typed commerce carriers.

A price lever that covers several products sets lever.productId to "selection"; launch QA and charged verification then check every listed variant on its own product.

For price, shipping, and offer levers, a treatment row with sameAsControl: true keeps that lever at control while sibling treatments change it. Plan lists these rows as same_as_control:<variation> in commerceLaunchPreview.checked. The saved treatment has no carrier for that lever. Post-purchase offers are drawn per checkout, so every treatment must carry the offer or none does.

One treatment can combine price, shipping, and offer changes. For example, set the treatment row in commerce/price.json to a lower price, set the same treatment in commerce/shipping.json to a free-shipping threshold, and use sameAsControl: true for that treatment in commerce/offer.json when its offer should remain unchanged. Plan includes a combined preview whose launch-only checks name the shared assignment and, when the arm changes both price and shipping, the price-shipping-combined Commerce Lab proof. Applying saves the draft; launching remains a dashboard click. Post-purchase offers cannot join a combined arm in this step because they are drawn per checkout.

.apex/ is CLI-owned metadata. Do not edit it directly.

Pull, plan, and apply

Successful apply JSON includes data.receipt (also in data.result.receipt). Older servers without receipts return data.receipt: null. Text output prints a Receipt block. The server reads the persisted experiment after the write and publication complete:

Note: when running a filesystem apply, the CLI returns a post-write receipt that includes the server's persisted experiment state and per-variation content digests. Verify the saved experiment state and the variation digests directly from the apply response (or by running apex experiments pull after the apply) to confirm what was persisted.

{
  "files": [{"path": "experiment/index.js", "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "submitted_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "normalizations": []}],
  "server_owned_files": ["qa.json", "launch.json"],
  "server_owned_fields": ["experiment.json#id", "experiment.json#testKey", "targeting.json#authorUrl", "variations/control/variation.json#artifactReview"],
  "experiment_id": "experiment-id",
  "status": "draft",
  "delivery_type": "mutation",
  "counting_type": "non_program",
  "traffic_allocation": 0.5,
  "schedule": { "scheduled_start_at": null, "scheduled_end_at": null },
  "goal_count": 2,
  "qa": null,
  "written_at": "2020-01-01T00:00:00.000Z",
  "variations": [{
    "id": "variation-id", "slug": "control", "name": "Control",
    "is_control": true, "redirect_url": null,
    "digests": {
      "mutations": "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945",
      "custom_js": null, "custom_css": null, "reset_js": null
    }
  }]
}

Variations are sorted control first, then by name. Digests are SHA-256 hex over stored content: recursively key-sorted compact JSON for mutations (array order preserved; a variation without stored mutations digests compact []), and raw text for code fields. A null code column yields null; an empty string yields its SHA-256 digest. Recompute these from a subsequent pull to compare content, parsing and canonicalizing mutations instead of hashing the formatted JSON file. qa is the latest run's run_id, status, and ISO approved_at or null; no run yields null. Schedules use ISO timestamps or null. written_at records receipt construction time. Failed applies have no server receipt. If the readback fails after the write committed, the apply still succeeds with receipt: null and a warning in result.plan.warnings (the server's final plan; data.plan is the pre-apply plan); pull the experiment to verify. The receipt describes persisted state and does not replace the publication outcome or establish QA approval. apex push is unchanged.

apex experiments apply returns confirmation-required exit code 4 in non-interactive mode only when a material change would revoke an existing QA approval and --accept-qa-invalidation is not present. A material change to an unapproved draft needs no acknowledgement. Plan JSON distinguishes these states with qaApproval.present and requiredAcknowledgment. The JSON refusal uses reason qa_approval_present, includes qa_approved_at, and names the required acknowledgement. Apply also returns confirmation-required exit code 4 in non-interactive mode when a targeting.json change attaches a Page and drops URL rules authored through the API, unless --confirm-targeting-replace is present.

Apply waits up to 90 seconds for the server, unless APEX_TIMEOUT_MS explicitly overrides the wait. After a timeout or connection reset, the CLI plans the same files once more: it reports a confirmed apply when the server already holds the changes, or returns apply_unconfirmed with the exact apex experiments apply <folder> command to run again.

When that verification returns a valid plan with zero changes, apply exits 0. JSON data includes applied: true, verifiedAfterTimeout: true, the effective timeoutMs, result: null, and receipt: null. The server holds the changes, but the timed-out response provides no receipt or publication confirmation.

Commerce QA probes can be run for draft or paused experiments; running, completed, and archived experiments are refused.

For a draft experiment, human output reports plain success:

Apply confirmed: the server already holds these changes (the response timed out after 90s).

For every non-draft experiment, including running and paused experiments, JSON also includes persisted: true, publication: "unconfirmed", warnings, and next_steps. applied: true confirms that the changes were saved; it does not confirm storefront publication. A non-running experiment can change a goal shared with a running experiment, so publication remains unconfirmed for all non-drafts. For example, these fields appear in data:

{
  "applied": true,
  "persisted": true,
  "verifiedAfterTimeout": true,
  "timeoutMs": 90000,
  "result": null,
  "receipt": null,
  "publication": "unconfirmed",
  "warnings": [
    "Storefront publication is unconfirmed; visitors may still receive the previous treatment."
  ],
  "next_steps": [{
    "command": "apex experiments status ./homepage-hero --json",
    "why": "Check experiment status before handover; persisted files do not confirm storefront publication."
  }]
}

Human output reports the saved apply, the publication warning, and the next step:

Apply saved: the server already holds these changes (the response timed out after 90s).
Warning: Storefront publication is unconfirmed; visitors may still receive the previous treatment.
Next: apex experiments status ./homepage-hero --json (Check experiment status before handover; persisted files do not confirm storefront publication.)

Asset upload is enabled by default. The CLI replaces large inline image data URIs and relative image references in variation code files and experiment-level custom code files with hosted Apex delivery URLs. It uploads equal content once per command. Use --no-asset-upload to send the code unchanged and let the server validate it.

targeting.json

Supported fields are url, attributes, pages, pageRules, segments, pageMatchMode, segmentMatchMode, and deviceType; authorUrl is server-owned provenance. deviceType accepts all, desktop, mobile, or tablet (omitted means all): assignment is limited to visitors whose detected device type matches; a multi-device audience uses a segment with a device.type condition.

Normalisation contract

JSON bodies and checkout JSON files may start with a UTF-8 BOM. Every receipt file carries submitted_sha256, hashing the exact submitted UTF-8 bytes including a BOM (or null when that path was not submitted), and normalizations. An unchanged file has equal submitted and stored digests and an empty list. Text output lists changed files on one Normalized on store: line. Older servers may omit these fields. apex schema --json publishes the receipt schema at files.receipt.

Without --sync, apply leaves authored files byte-identical, heals only the server-owned qa.json and launch.json, and records receipt file digests in the session manifest. Use apex experiments apply <folder> --sync or apex experiments pull to refresh authored files. JSON reports synced; a failed refresh reports false. An older server without receipt file digests leaves the authored-file manifest entries unchanged; the healed server-owned entries are still recorded. A failed --sync refresh falls back to the default behaviour. Plan and apply compare variation code after the same normalisation used for storage, so a local trailing newline alone is not a change and does not invalidate QA. Experiment-level code is still compared verbatim. Plan reuses asset URLs cached in the session manifest, leaves uncached references unchanged, and performs no uploads or manifest writes. The files example above is abbreviated: actual receipts contain every pull path, and server_owned_fields carries one variation.json#artifactReview entry per variation. These SHA-256 digests are proof of the stored pull bytes, including JSON whitespace; compare them directly against reviewed files without another pull.

Asset upload may replace references in submitted code with hosted URLs; without --sync, these replacements do not rewrite local authored files. The session manifest .apex/experiment-filesystem.json caches uploaded assets by the SHA-256 of the uploaded bytes, recording their hosted URL and delivery URL when supplied. Later applies reuse these URLs without uploading unchanged images again, avoiding spurious code changes and QA invalidation. Older manifests without an assets map start with an empty cache; cache writes preserve per-file digests, including when apply plans no changes. Apply JSON also includes dashboardUrl, or null without shop context, linking to the experiment through the dashboard shop switch route.

QA status

apex experiments qa status ./homepage-hero
apex experiments qa events ./homepage-hero --type custom

The target can be a pulled folder, experiment ID, or slug. When you pass a pulled folder, the CLI refreshes local files after reading the QA run.

QA checklist updates

apex experiments qa check ./homepage-hero \
  --item responsive_matrix \
  --status pass \
  --notes "Desktop and mobile smoke checks passed." \
  --evidence @qa-result.json

QA evidence must be a JSON object. Keep screenshots, logs, and artifacts outside qa.json; attach structured references through the QA command.

QA notes and approval

apex experiments qa notes ./homepage-hero --notes "Ready for stakeholder review."
apex experiments qa notes ./homepage-hero --acceptance "Responsive, tracking, and content checks passed."
apex experiments qa approve ./homepage-hero
apex experiments qa reopen ./homepage-hero

approve marks QA done after the server-side checklist gates are ready and moves the experiment to ready_to_launch. Its JSON includes experiment_status_before, experiment_status_after, and transition with from, to, and a reopen command. reopen returns the experiment to QA and clears scheduled launch state.

Filesystem rules

• Never hand-edit qa.json or launch.json as the source of truth.
• Use apex experiments qa to update QA runs and approval state.
• Use apex experiments qa events to read goal and custom events tagged with the latest QA run; pass --session, --minutes, or --type to narrow the readback.
• Use apex experiments plan before every apply.
• If you hit a conflict or stale filesystem state, pull fresh files, reapply your edits, and plan again.

Related pages

• Price basis for value-bearing treatments
• Experiment lifecycle commands
• Workspace and Operator commands
• CLI commands, JSON, and exit codes

Per-URL variation blocks

variations/<variation>/url-blocks.json is optional. When the file is absent, Apex treats the variation as having no URL blocks. A treatment variation can include the file with this shape:

[
  {
    "id": "product-a",
    "name": "Product A treatment",
    "urlRules": [{ "type": "simple", "pattern": "https://shop.example/products/a" }],
    "mutations": [{ "action": "text", "selector": "h1", "text": "A-specific headline" }],
    "css": ".offer { color: red; }",
    "js": "window.productTreatment = 'a';"
  }
]

Blocks are ordered and first match wins. The matching block substitutes its mutations, css, and js for the variation's flat files. When no block matches, Apex falls back to mutations.json, index.css, and index.js as the implicit default block. If those flat fields are empty, rendering deliberately fails open as a no-op: the visitor remains assigned and contributes to pooled experiment results. Control variations remain control on every URL.

Blocks do not have a per-block resetJs. Write block js to be idempotent. The runtime removes block CSS and runs its normal cleanup when the URL or assignment signature changes, but JavaScript side effects such as globals or DOM changes outside declared mutations remain the author's responsibility, just as they do for flat custom JavaScript.

Classification uses --counting-type standard|setup_test|aa_test|non_program on apex experiments create (default standard) and apex experiments update. Classification writes are draft-only. In pulled experiment.json, experimentType and countingType equal delivery_type and counting_type from apex experiments get --json. Omitting or setting countingType to null preserves the stored value. Classification edits invalidate QA approval; paused and running experiments reject these metadata changes.

File contract

Every JSON file written by apex experiments pull is described by the draft 2020-12 document at apex schema --json → data.files. The document and the local .apex/experiment-filesystem.json receipt carry schemaVersion: 1. Use GET /api/v1/experiments/filesystem/schema to compare a server's current document with the schema bundled in the installed CLI before editing or applying files.

This contract covers apex experiments pull only. Files created by the older apex workspace checkout operator workflow have no JSON Schema contract.

Compact output and evidence vocabularies

qa check, qa notes, and qa evidence accept --summary. JSON data contains only run (id, status, summary, approvedAt, updatedAt), changed, and validationErrors (empty on success). Changed values are the updated checklist item, stored notes and acceptanceCriteriaSummary, or evidence records newly added by ID. Without the flag, the full run is returned.

Help and schema --json list accepted choices. Invalid values fail locally with exit 3.

Plan and apply JSON include requiredAcknowledgment: accept-qa-invalidation when the plan is material to QA and the experiment holds an approval, otherwise null. The qaApproval object reports whether an approval is present, its approval time when available, and the experiment status. Apply still accepts the hidden deprecated --force alias, printing a notice to stderr.

QA status creates a run only when none exists. Reading an existing run never writes to it: updatedAt is its last stored mutation time, while computedAt is the ISO timestamp when the current checklist and summary were computed.

Read-only handover status

Run apex experiments status [folder-or-id] --json before handover. With no argument, it reads the current pulled folder. The JSON envelope's data contains context, experiment, versions, qa, next_steps (command and why), ready_for_handover, and notes. Human output shows the same sections.

Folder mode compares every supported file with its pulled hash (unchanged, modified, added, or deleted) and uses the read-only plan route for saved changes, QA invalidation, blockers, and warnings. ID, slug, and test-key mode reports versions: null and recommends pulling a folder first.

Status never writes local files, uploads assets, creates a QA run, or changes QA updatedAt. It reads the server filesystem's QA snapshot. Missing counts are null; missing identity names are null, particularly with shop credentials.

qa.approval_current is true when QA is approved, the approval is bound to a saved version (ready_version_id, which the server clears whenever saved test content changes), and the pulled folder has no local or saved changes. An older approval without a ready version is not current; status then suggests approving again. ready_for_handover is true when the experiment status is ready_to_launch, approval is current, every required check is done, and there are no blockers; the only next step is then to launch the test in the dashboard, which stays a human decision. Without a pulled folder, readiness reflects the saved version only.

Look up products for commerce drafts

Use apex commerce catalog search to find active products, then apex commerce catalog get <handle> to copy exact variant GIDs, base-currency prices, live costs, and subscription or bundle exclusions into a draft.

Commerce launch preview

For draft and other pre-launch tests, apex experiments plan returns a commerceLaunchPreview entry for every authored commerce file. Its warning-level issues show settings that launch composition would refuse without changing whether plan or apply succeeds. checked names the pure checks performed from the files, while launchOnly explains deferred checks and its when field says whether each runs at apply or launch. Relationship eligibility, the complete running-offer size budget, and overlap with running tests are read-only plan checks; a failed read appears as an unchecked warning.

Discount-code rules live in commerce/discount-codes.json. Each arm names a variation and either keeps control behavior with sameAsControl: true or sets block: true; an optional message object can override the default en and de customer messages. Checkout refuses entered codes for visitors in a blocking arm and shows the message; Apex makes no storefront DOM changes. For example:

{"schemaVersion":1,"kind":"discount-codes","arms":[{"variation":"control","control":true},{"variation":"sale","block":true,"message":{"en":"Codes cannot be combined with this sale."}}]}

Checkout messages

targeting.json cannot author surface or checkoutBlock. Retired test-level targeting is read-only history: pulls retain its raw keys, an unchanged pull/apply can remain a no-op, and targeting replacements return 409 checkout_mode_retired.

Keep a checkout message in the arm's variations/<slug>/mutations.json file (variations/variation-b/mutations.json for a fresh CLI draft; leave control as []). In a workspace folder, use src/mutations.json:

[
  {
    "action": "checkoutBlock",
    "checkoutBlock": {
      "placement": "checkout",
      "tone": "info",
      "eyebrow": "Shop with confidence",
      "heading": "Free returns for 30 days",
      "body": "Try your order at home.",
      "disclosure": "Return items in their original condition."
    }
  }
]

Each arm accepts one checkout message. The heading is required (1–120 characters after trimming, with visible text); eyebrow, body and disclosure are optional (60, 500 and 300 characters). Placement accepts only checkout and defaults to it; tone defaults to info and also accepts success, warning and critical. Unknown fields are refused. The legacy enabled boolean is deprecated; omit it for new messages (false hides the message).

Checkout messages work only on A/B tests, outside URL blocks, and cannot share a test with commerce/post-purchase.json. Duplicate messages return duplicate_checkout_block. Copy cannot state a money amount (a number next to a currency symbol, ISO code or currency word, or a percent next to off, save, sale or Rabatt); apply refuses it with 422 checkout_copy_states_money (CLI exit 3). A plain percent only warns at launch.

Pause a running test before changing its message, repeat its QA checkout, then resume. On paused or running tests, adding the first message to a test with no storefront change or removing the last message is refused with checkout_exposure_owner_frozen.

Messages can share an arm with DOM mutations, CSS, JS and commerce levers other than post-purchase. Combined tests count visitors on the storefront and show their storefront change only while Apex confirms the message is live. Message-only tests count visitors at checkout; buyers and QA testers must allow analytics, marketing and sale-of-data processing to see the message. Apex recommends a revenue primary goal. Control may carry a message, but it does not count as treatment work.

Run apex experiments plan <folder> --json, then apex experiments apply <folder> --json (add --accept-qa-invalidation if the test already holds QA approval; otherwise exit 4 requests confirmation), then apex experiments apply <folder> --sync so the folder picks up the stored entry (apply adds "placement": "checkout"). apex experiments status <folder> --json shows the QA checklist and file state, not launch blockers; the dashboard shows those. The CLI validates file names; the server validates contents. apex schema --json prints the contract offline. Before launch, follow Before launch, including adding the Apex Checkout Experiment block in the Shopify checkout editor. For each message arm, open a signed QA link in a real checkout: apex preview <id> <variation-id> --json --no-open prints one (treat it as a secret), and the Check panel's Preview links work too. Add a product, go to checkout, allow analytics, marketing and sale-of-data processing if the store asks, and wait for the message; a customer approval link does not count. Launch happens in the dashboard. See Combined storefront and checkout tests.

Commerce definition files

Experiments with a canonical definition pull one top-level commerce.json with revision, definition, and read-only serverOwned. plan shows definition and derived carrier changes plus launch preview; apply saves with revision checking. A stale file refuses with 409 commerce_definition_revision_conflict. Server output edits are ignored with commerce_server_owned_ignored. Deleting the file is refused; omit a lever from definition.levers to remove it.

For one release, experiments without a definition still pull commerce/price.json, shipping.json, offer.json and post-purchase.json. Changed supported files convert with commerce_files_v1_converted. Unchanged and opaque read-only files stay as they are. Mixing step 1 lever files with commerce.json refuses with commerce_definition_declared_twice. Discount codes remain in the separate commerce/discount-codes.json file in both formats.

Use apex commerce definition get/put for direct file export and editing. Legacy envelopes remain read-only until replaced by a full definition at their stored revision. A single all-scope percent rule from -90 to -1 without endings or overrides uses the launch-time baseline; other price rules compile to fixed prices.