Experimentation News: PostHog Adds 44 Agent Tools

A quieter fortnight than the last one, and all of the substance came from a single PostHog release on 6 August. Three parts of it are worth a testing team’s attention: another large expansion of the agent tooling, proper access control on warehouse tables, and a batch of new data sources that changes what you can put a metric on.

44 more MCP tools, and experiments are in scope again

PostHog’s 6 August release added 44 new tools to its Model Context Protocol server, covering workflows, AI observability, canvas, tasks and experiments. That follows the 32 tools shipped on 3 August, which included experiment recalculation. Two large batches in four days is a clear statement of direction: the company is building for a world where an agent, not a person, is the thing calling the analytics API.

The useful question for a testing team is not whether to adopt this but where the boundary sits. An agent that can pull experiment results, recalculate them and summarise them is a genuine time saver for the reporting half of the job. An agent that can launch or stop an experiment is a different proposition, because the failure modes in experimentation are quiet ones: peeking at results and calling a winner early, running a variant on a broken segment, or shipping a change that never reached significance. Those errors do not throw exceptions, they produce confident-looking numbers. If you are wiring agents into this, wire them into reading, and keep a human on the decision to stop a test. Our CRO hypothesis framework and the sample ratio mismatch calculator are two of the checks that do not get easier just because a model is doing the reading. The release is in the PostHog changelog.

Per-table permissions arrive in the data warehouse

The same release lets administrators set permissions on individual tables within warehouse schemas, so a teammate can be given access to some tables and not others. It sounds like plumbing, and it is, but it is the plumbing that decides whether an experimentation programme can use company data at all.

The usual pattern in a growing company is that the warehouse holds everything, including payroll, customer PII and revenue detail, so access is granted at the schema level to a small group and everyone else works from exports that go stale. Table-level control is what lets you give an analyst or a contractor the events and revenue tables they need to define a metric without handing over the rest. If your experiment metrics are currently defined on whatever happens to be in your analytics tool rather than on the numbers finance actually uses, this is the kind of change that removes the excuse. Our guide to building a CRO programme covers why a shared metric definition matters more than the tool that computes it.

23 new warehouse sources and 161 new tables

The 6 August release also added 23 new data sources across commerce, marketing, cloud and public data, and 161 new tables to existing connectors including GitHub, Stripe and HubSpot. ClickHouse and Google Analytics sources moved from beta to general availability, and a Vercel source entered beta for syncing deployment data.

Deployment data is the interesting one. If you can join deployments to your event stream, you can answer the question that ruins more experiments than any statistical error: did something else ship in the middle of my test? A release that lands mid-experiment and changes the page, the load time or the funnel is a confound, and most teams discover it after the fact when a result refuses to replicate. The wider point about Stripe and HubSpot tables is the same one as above: the closer your experiment metric sits to the revenue system, the harder it is to argue with the result. If you are still measuring on proxy metrics, how to calculate conversion rate and CRO revenue uplift calculator are the two places that gap usually shows up. The full list is in the PostHog changelog.