Amplitude Wave Goes Paid: An Agent That Proposes Tests

This week’s four experimentation stories, from 5 to 12 November 2026, cover Amplitude’s new paid product agent, a RealPage test where showing fewer options lifted conversion by 12%, a GrowthBook score that grades your hypothesis as you type it, and a LaunchDarkly escape hatch for flags during an outage. Two of the four come with numbers you can learn from, including one result the vendor admits was inconclusive.

Amplitude Wave: an agent that finds the test, then measures it

On 12 November Amplitude made Wave available as a paid add-on for customers on its Growth and Enterprise plans. Wave is what Amplitude calls a product agent: you point it at one business goal for one area of your product, such as sign-up completion or new-user retention, and it reads behavioural data, session replays and customer feedback to find where people get stuck. Each finding becomes an “Opportunity” with a recommended change and a hypothesis to test, and Wave measures the result after the change ships.

The customer numbers are vendor-reported, so read them as claims. Amplitude says the AI image app PixAI saw 76% more opens of its Missions tab, nearly 10% more publishing and a 3% rise in day-one retention after acting on a Wave finding, and that DigitalOcean got 55 Opportunities across five product areas in six weeks. The more useful read is Amplitude’s own write-up from 9 November of using Wave on its product. In 100 replays of its agent homepage, users typed their own question in 80 sessions and picked a suggestion card in 12. Personalised cards then drew 2.43 sends per 100 exposed users against 0.94 for the generic ones, which Amplitude calls an initial readout that still needs validating. A second test, recommending the HTTP API during onboarding, showed 24.61% activation against 17.75% in control, and Amplitude says plainly that it was statistically inconclusive.

That honesty is the point for buyers. A tool that produces dozens of test ideas a month also produces dozens of chances for a false positive, so the bar for calling a winner has to rise with the volume. Our A/B test significance calculator and guide to CRO research methods cover the checks to run before you ship. Read the Wave launch post and how Amplitude uses Wave on itself.

RealPage: cutting 8.2 choices to 3 lifted conversion 12%

On 5 November GrowthBook published an interview with Andrew Miller, who runs growth and e-commerce product at RealPage, the US property software company. His team tests the move-in checklist that renters see two to three weeks before they move, where RealPage earns a commission when a renter sets up electricity through a partner. In Texas, renters opening the electricity task saw an average of 8.2 plans. Cutting that to three lifted conversion by 12%.

The loss is the better lesson. Prices had to be shown in cents per kilowatt hour, which few people understand, so the team tested an estimated monthly cost instead. More people picked a plan and moved on, but final checkout conversion fell, and cents per kilowatt hour won. The team’s best theory is that an estimate shown to the cent, such as $102.14 a month, looked too precise to trust, so the rerun will round to the nearest $5. It is a reminder to check the final conversion, not just the next click: a variant can win the step it changes and lose the sale. RealPage builds its tests in Optimizely, and Miller says a prototyping harness cut the time to launch a test from about two weeks to about a day. Our guide to conversion funnel optimisation covers measuring the whole path. Read the RealPage interview.

GrowthBook will score your hypothesis before the test runs

On 6 November GrowthBook’s chief executive, Graham McNicoll, described an experiment quality score the company is building. Rather than ask a chat model for one grade, it splits the hypothesis check into three yes or no questions: does it name a change, does it predict a measurable effect with a direction, and does it give a reason. A second check asks whether the goal metrics measure the predicted effect. “Redesigning the homepage will increase signups” passes the first two and fails the third, so it loses about a third of its points and the Quality tab tells you why.

GrowthBook runs the checks on Jev, a classification model from TypeSafe, which took about 0.1 seconds per experiment in its test against about 1.3 seconds for gpt-5.4-mini. At $0.042 per million input tokens and about 600 tokens a pass, 10,000 checks would cost roughly 25 cents, cheap enough to re-run about a minute after every edit, including on drafts. The feature is coming soon for Enterprise customers. If you want the same discipline without the software, our CRO hypothesis framework uses the same three parts. Read the post on the GrowthBook blog.

LaunchDarkly: change a flag when LaunchDarkly cannot be reached

LaunchDarkly’s documentation changelog records that on 6 November it published a topic on flag overrides, an experimental feature for server-side SDKs. You put a flag or segment definition in a local JSON or YAML file, and a running application picks it up without a restart or redeploy, whether or not the SDK can reach LaunchDarkly. LaunchDarkly says it is for incidents, such as an SDK serving stale flag data or a flag you cannot edit, and that overrides should be short-lived.

Two details matter if a flag also runs an experiment. An override stays in force after LaunchDarkly comes back, and while it does, changes made to the flag in LaunchDarkly have no effect on it. A simple override in the flagValues section serves the same value to every user. Note the time you add and remove one, because results from that window will not reflect your planned split. Access is limited for now: overrides need data saving mode, which only members of LaunchDarkly’s Early Access Program can use. Our A/B testing tools comparison covers where LaunchDarkly fits. See the flag overrides topic and LaunchDarkly’s changelog.