Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/glossary.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -746,6 +746,20 @@ Lower variance improves power and reduces how long experiments need to run.

---

## W

### Warehouse-native
A deployment mode where experiment analysis runs directly inside your own data warehouse instead of in the experimentation platform's cloud.
ABsmartly handles experiment assignment, management, statistical analysis and metric governance, while the underlying data — exposures, goals and attributes — stays in your warehouse.

Warehouse-native experimentation supports a single source of truth between experimentation and BI/finance reporting, helps meet data residency and compliance requirements (GDPR, SOC2, HIPAA) by keeping user-level data within your own infrastructure, and provides auditability through your existing data governance tools.

ABsmartly supports warehouse-native analysis on BigQuery, Snowflake, ClickHouse, Redshift and Databricks.

**Example:** Connecting ABsmartly to your Snowflake account, mapping your exposure and event tables to ABsmartly's expected schema, and letting ABsmartly compute experiment results on a refresh schedule — without copying user-level data out of Snowflake.

---

## Z

### Z-score
Expand Down
90 changes: 90 additions & 0 deletions docs/platform-release-notes/2026/05.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
import Image from "../../../src/components/Image";

# May 2026

## Overview

This release brings a long-awaited **Metric Changelog**, introduces the new **activity_per_period** metric type,
and ships several quality-of-life improvements across the Explore Metrics tab, experiment list, and Events dashboard.

---

## Metrics

### Metric Changelog

Every metric now has a full **change log** that captures every modification made over its lifetime, so you always know **what changed, who changed it, and when**.

Open the change log from any metric to see a chronological timeline of changes, with the most recent at the top. Each entry shows:

- **Who** made the change, with avatar and name
- **When** it happened (relative time, with the exact timestamp on hover)
- **Which metric version** the change produced
- **A short summary** of the change (e.g. _Changed owners_, _Added metadata_)
- **Field-level before/after diffs**, grouped into collapsible sections such as **Identity**, **Details**, **Definition** **Goal**, with clear add/remove indicators
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Clarify the section list formatting.

The list of collapsible sections appears to be missing proper punctuation or formatting between "Definition" and "Goal". This should likely be either:

  • "Definition, Goal" (if these are separate sections), or
  • Verify if "Definition Goal" is a single section name
📝 Proposed fix (if separate sections)
-- **Field-level before/after diffs**, grouped into collapsible sections such as **Identity**, **Details**, **Definition** **Goal**, with clear add/remove indicators
+- **Field-level before/after diffs**, grouped into collapsible sections such as **Identity**, **Details**, **Definition**, **Goal**, with clear add/remove indicators
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- **Field-level before/after diffs**, grouped into collapsible sections such as **Identity**, **Details**, **Definition** **Goal**, with clear add/remove indicators
- **Field-level before/after diffs**, grouped into collapsible sections such as **Identity**, **Details**, **Definition**, **Goal**, with clear add/remove indicators
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/platform-release-notes/2026/05.mdx` at line 23, The list item containing
"**Identity**, **Details**, **Definition** **Goal**" is missing proper
punctuation between "Definition" and "Goal"; update the text in that list entry
so the section names are unambiguous—either insert a comma to make
"**Definition**, **Goal**" if they are separate sections, or collapse them into
a single section name like "**Definition Goal**" (or adjust bolding to reflect
the intended single-section string); edit the same list line to ensure
consistent comma separation for all items such as "**Identity**, **Details**,
**Definition**, **Goal**" if there are four sections.


This gives teams a complete audit trail for metric governance which is useful for reviews, debugging unexpected results, and understanding the history behind a metric.
The same change log will soon be available for other assets like **experiments** and **features**.

<Image maxWidth="40rem" centered img="release-notes/metric-changelog.png" alt="Metric changelog showing chronological changes with who, when, and field-level diffs" />

---

### New Metric Type: `activity_per_period`

We've added a new metric type, **[activity_per_period](/docs/web-console-docs/goals-and-metrics/metrics/metric-types/activity-per-period)**, that measures user activity over a configurable time window (for example, _sessions per week_ or _orders per month_).
It's designed for cases where you care about the **frequency** of an action per user across a rolling period, rather than a single count or ratio.

See the [Activity per Period documentation](/docs/web-console-docs/goals-and-metrics/metrics/metric-types/activity-per-period) for more information.

### Improved Metrics Discoverability

Finding the right metric on the **Explore Metrics** tab is now much easier.
We've improved how metrics surface in the list, making it faster to locate the metrics you care about when investigating an experiment.

---

## Events

### Keyboard Navigation in the Event Details Dialog

The **Event Details** dialog now supports keyboard arrow navigation, so you can step through events without reaching for the mouse. A small change that makes browsing through long event lists noticeably faster.

---

## Experiments

### Custom Assignments (Early Beta)

We're opening up an **early beta** of **Custom Assignments** — a new way to define who should be **excluded from an experiment** and always served a specific variant instead.

Custom Assignments (also known as **exclusion rules**) are useful for:

- **Testing & QA** — force yourself or your team into a specific variant to verify the experience, without being counted as a participant.
- **Internal or beta users** — ensure groups like internal employees or beta testers always see a chosen variant, without polluting experiment data.

Excluded visitors see the configured variant but **are not tracked as participants**, so the experiment statistics stay clean.

See the [Custom Assignments documentation](/docs/web-console-docs/experiments/creating-an-experiment#custom-assignments-exclusion-rules) for details on how to configure them.

:::caution Early Beta — JavaScript SDK Only
Custom Assignments are currently in **early beta** and are **only available for the JavaScript SDK**. The feature is behind a flag and requires an **SDK update** before it can be used.
If you'd like to enable it for your account, please **[reach out to our team](mailto:support@absmartly.com)** and we'll help you get set up.
:::

### Group Experiment List by Experiment

You can now **group the experiment list by experiment**, making it easier to navigate large lists
especially when working with experiment families or multiple iterations of the same test.

---

## Bug Fixes

This release also includes a number of security, stability and reliability improvements based on feedback from users.

---

## Questions or Feedback?

We're always happy to help, so reach out if you have any questions or want to explore how to make the most of these new capabilities.
13 changes: 7 additions & 6 deletions docs/web-console-docs/experiments/creating-an-experiment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -116,20 +116,20 @@ their data will not be tracked.
When audience enforced is **off**, it is the responsibility of the experimenter to ensure that only eligible visitors are shown the experiments.
ABsmartly will only warn you when visitors not matching the audience are exposed to your experiment. In this case you would see an **audience mismatch warning** in your experiment. This option makes it easier to full-on when the experiment is completed as the audience filtering is not part of the experiment set up.

## Exclusion Rules
### Custom Assignments (Exclusion Rules)

Exclusion rules let you define which visitors or groups of visitors should be **excluded from the experiment** and instead always see a specific variant.

<Image maxWidth="40rem" centered img="experiment-create/exclusion-rules.png" alt="Configuring exclusion rules for an experiment" />

### When to Use Exclusion Rules
#### When to Use Custom Assignments

Exclusion rules are useful in two main scenarios:

- **Testing & QA** — Force yourself or your team into a specific variant to verify the experience before or during the experiment, without being counted as a participant.
- **Internal or beta users** — Ensure that a specific group of users (e.g. internal employees, beta testers) always sees the new variant without polluting the experiment data.

### How It Works
#### How It Works

An exclusion rule defines:

Expand All @@ -142,7 +142,7 @@ Excluded visitors are **not part of the experiment**. They will see the assigned
You can add multiple exclusion rules to a single experiment. Rules are evaluated in order — a visitor matching any rule will be excluded.
:::

### Example
#### Example

To exclude internal users and always show them Variant B:

Expand All @@ -152,8 +152,9 @@ To exclude internal users and always show them Variant B:

This ensures your team can experience the new variant in production while keeping the experiment data clean.

:::caution
Exclusion rules require an **SDK update**. Make sure your SDKs are up to date before using this feature.
:::caution Early Beta — JavaScript SDK Only
Exclusion rules (also known as **Custom Assignments**) are currently in **early beta** and are **only available for the JavaScript SDK**. The feature is behind a flag and requires an **SDK update** before it can be used.
If you'd like to enable it for your account, please **[reach out to our team](mailto:support@absmartly.com)** and we'll help you get set up.
See the [SDK documentation](/docs/APIs-and-SDKs/SDK-Documentation/running-your-first-experiment) for details.
:::

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
sidebar_position: 8
---

# Activity per Period

## Overview

An `Activity per Period` metric measures **how frequently** a user performs a goal action across a configurable time window
— for example, _sessions per week_ or _orders per month_.

Unlike a simple `Goal Count`, which sums all events per user, `Activity per Period` normalises activity into a **rate per period**, making it easier to compare user engagement over time and across cohorts of different exposure lengths.

You can configure:

- the **goal event** to track
- the **period length** (e.g. day, week, month)
- whether the period starts:
- from the user's first exposure to the experiment, or
- from the user's first occurrence of the goal event

## Examples

```javascript
context.track("session_start", {
source: "homepage",
device: "mobile"
});
```

Imagine you want to measure **average sessions per week per user** during your experiment.

You can create a `Sessions per Week` metric by:
- Selecting the `session_start` goal
- Setting the `Period` to 7 days
- Choosing whether periods start from the user's first exposure or first session

The metric then computes, for each user, the number of goal events per period, and aggregates the result across the experiment population.

**More examples**

- `Orders per Month`:
Average number of purchases per user per 30-day period.

- `Articles Read per Week`:
Average number of articles read per user, per 7-day period.

- `Active Days per Week`:
Average number of distinct days a user was active in a 7-day window.

## Good to know

- Useful when you care about **frequency** of an action rather than total volume or simple conversion.
- More robust than `Goal Count` when users have different exposure lengths — the per-period normalisation makes long-tenured and short-tenured users directly comparable.
- Filters on the goal event apply before the per-period rollup (for example: _purchases per week_ restricted to a specific category).
- Changing the period length alters the meaning of the metric and will require a new version.
- Ideal for experiments aimed at **engagement, habit formation, or retention frequency** — onboarding flows, notifications, recommendation systems, or loyalty mechanics.
Binary file added static/img/release-notes/metric-changelog.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading