Experiments
A/B tests.
8 endpoints.
GET /v1/experimentsPOST /v1/experimentsGET /v1/experiments/{experiment_id}PATCH /v1/experiments/{experiment_id}DELETE /v1/experiments/{experiment_id}POST /v1/experiments/{experiment_id}/startPOST /v1/experiments/{experiment_id}/stopGET /v1/experiments/{experiment_id}/results
GET
/v1/experimentsList A/B tests
Needs a key or a dashboard session
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
cursor | query | string | null | no | max length 200 |
limit | query | integer | no | default 50 · min 1 · max 200 |
Responses
- 200Successful Response
Page_ExperimentOut_
POST
/v1/experimentsExperiments Create
Needs a key or a dashboard session
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
Idempotency-Key | header | string | null | no | Unique per logical write (8-255 chars). Retries with the same key are safe. · pattern ^[A-Za-z0-9_\-:.]{8,255}$ |
Request body application/json ExperimentIn
| Name | Type | Required | Description |
|---|---|---|---|
name | string | yes | max length 200 |
kind | "ranking" | "synonyms" | "rules" | "ai_rewrite" | yes | |
variants | Variant[] | yes | |
primary_metric | "ctr" | "conversion" | "zero_result_rate" | "add_to_cart_rate" | no | default "ctr" |
traffic_percent | integer | no | default 100 · min 1 · max 100 |
Responses
- 201Successful Response
ExperimentOut
GET
/v1/experiments/{experiment_id}Experiments Get
Needs a key or a dashboard session
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
experiment_id | path | string | yes |
Responses
- 200Successful Response
ExperimentOut
PATCH
/v1/experiments/{experiment_id}Experiments Update
Needs a key or a dashboard session
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
experiment_id | path | string | yes |
Request body application/json ExperimentPatch
| Name | Type | Required | Description |
|---|---|---|---|
name | string | null | no | max length 200 |
variants | Variant[] | null | no | |
traffic_percent | integer | null | no | min 1 · max 100 |
Responses
- 200Successful Response
ExperimentOut
DELETE
/v1/experiments/{experiment_id}Experiments Delete
Needs a key or a dashboard session
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
experiment_id | path | string | yes |
Responses
- 200Successful Response
Deleted
POST
/v1/experiments/{experiment_id}/startExperiments Start
Needs a key or a dashboard session
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
experiment_id | path | string | yes |
Responses
- 200Successful Response
ExperimentOut
POST
/v1/experiments/{experiment_id}/stopExperiments Stop
Needs a key or a dashboard session
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
experiment_id | path | string | yes |
Responses
- 200Successful Response
ExperimentOut
GET
/v1/experiments/{experiment_id}/resultsResults with a significance test
Needs a key or a dashboard session
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
experiment_id | path | string | yes |
Responses
- 200Successful Response
ExperimentResults
Data shapes on this page
Page_ExperimentOut_
| Name | Type | Required | Description |
|---|---|---|---|
data | ExperimentOut[] | yes | |
next_cursor | string | null | no | |
total | integer | null | no | Present when cheap to count. |
ExperimentOut
| Name | Type | Required | Description |
|---|---|---|---|
name | string | yes | max length 200 |
kind | "ranking" | "synonyms" | "rules" | "ai_rewrite" | yes | |
variants | Variant[] | yes | |
primary_metric | "ctr" | "conversion" | "zero_result_rate" | "add_to_cart_rate" | no | default "ctr" |
traffic_percent | integer | no | default 100 · min 1 · max 100 |
id | string | yes | |
status | "draft" | "running" | "stopped" | "completed" | yes | |
started_at | string (date-time) | null | no | |
ended_at | string (date-time) | null | no | |
created_at | string (date-time) | yes |
Variant
| Name | Type | Required | Description |
|---|---|---|---|
key | string | yes | max length 16 |
weight | integer | yes | min 0 · max 100 |
config | RankingConfig | no | Only the fields that differ from the tenant's saved settings. |
RankingConfig
One ranking setup, used by the dashboard playground (`SearchRequest.config_override`) and by A/B variants (`Variant.config`). Every field is optional: a missing field means "use the tenant's saved value" (`TenantSettings.search`, same field names). Nothing here is saved.
| Name | Type | Required | Description |
|---|---|---|---|
ai_rewrite | boolean | null | no | |
rule_parser | boolean | null | no | |
in_stock_first | boolean | null | no | |
business_signals | map<string, number> | null | no | Weights 0..1, e.g. {popularity: 0.2, margin: 0.1}. {} turns them all off. |
typo_tolerance | boolean | null | no | |
hybrid_search | boolean | null | no | |
cheap_rerank_top_n | integer | null | no | min 10 · max 200 |
merch_rules | boolean | null | no | false = ignore all merchandising rules. |
disabled_rule_ids | string[] | null | no | |
synonym_sets | string[] | null | no | Tenant synonym sets that are on. "default" is the normal set. Example A/B: A = ["default"], B = ["default", "winter-2026"]. |
global_synonyms | boolean | null | no | false = ignore the platform's global synonyms. |
ExperimentIn
| Name | Type | Required | Description |
|---|---|---|---|
name | string | yes | max length 200 |
kind | "ranking" | "synonyms" | "rules" | "ai_rewrite" | yes | |
variants | Variant[] | yes | |
primary_metric | "ctr" | "conversion" | "zero_result_rate" | "add_to_cart_rate" | no | default "ctr" |
traffic_percent | integer | no | default 100 · min 1 · max 100 |
ExperimentPatch
| Name | Type | Required | Description |
|---|---|---|---|
name | string | null | no | max length 200 |
variants | Variant[] | null | no | |
traffic_percent | integer | null | no | min 1 · max 100 |
Deleted
| Name | Type | Required | Description |
|---|---|---|---|
deleted | boolean | no | default true |
id | string | yes |
ExperimentResults
| Name | Type | Required | Description |
|---|---|---|---|
experiment_id | string | yes | |
primary_metric | string | yes | |
variants | VariantResult[] | yes | |
p_value | number | null | no | The p-value used for `significant` (see `method`). |
significant | boolean | no | default false |
winner | string | null | no | |
method | string | no | How significance was computed.default "two-proportion z-test, alpha 0.05" |
updated_at | string (date-time) | yes | |
guidance | ExperimentGuidance | null | no |
VariantResult
| Name | Type | Required | Description |
|---|---|---|---|
key | string | yes | |
searches | integer | yes | |
metric_value | number | yes | |
ci_low | number | yes | |
ci_high | number | yes | |
sessions | integer | null | no | Shopper sessions in this variant (the test's unit). |
ExperimentGuidance
What to do now, and how big the test must be. Safe to read at any time (no peeking problem).
| Name | Type | Required | Description |
|---|---|---|---|
state | "not_started" | "collecting" | "keep_running" | "wait_full_week" | "winner" | "no_clear_difference" | "stopped_early" | "check_traffic_split" | yes | Stable key for the dashboard text. |
advice | string | yes | The same in plain English. |
min_sessions_per_variant | integer | yes | |
min_days | integer | yes | Days a test must run before a winner is shown (one full week). |
days_running | number | yes | |
planned_sessions_per_variant | integer | null | no | Sessions per variant for `power` to find a `mde_relative` change (always-valid test). |
fixed_horizon_sessions_per_variant | integer | null | no | The classic fixed-sample size, for reference (smaller; valid only if read once). |
progress | number | null | no | Smallest variant's sessions / planned sessions. |
sessions_per_day | number | null | no | Sessions in the test per day so far (or expected). |
days_to_plan | number | null | no | Expected days left until the planned size. |
baseline | number | null | no | Rate of the primary metric the plan is based on. |
baseline_source | "control" | "history" | "none" | no | default "none" |
mde_relative | number | yes | Smallest relative change the plan is sized for, e.g. 0.1 = 10%. |
power | number | yes | |
p_value_fixed_horizon | number | null | no | Classic z-test p-value. Not for decisions while the test runs (peeking). |
srm_p_value | number | null | no | Traffic split check (chi-square). |
srm_alarm | boolean | no | default false |