Feature Experiments with Next.js + PostHog + Vercel
How to implement a bulletproof feature flagging system using Next.js, PostHog, and Vercel.
If you ever worked at a Startup, you know that Feature Experimentation is at the heart of startup growth. We typically use A/B tests and feature flags, to discover opportunities to boost conversion rates, increase user adoption, and unlock product value. These experiments provide crucial data on user behavior, funnel performance, and feature engagement - helping startups make informed decisions on their path to product-market fit and profitability.
Having played around with many different experiment tools over the years, I figured it'd be cool to break down my go-to stack for A/B testing. Let me walk you through how I run experiments using Next.js, PostHog, and Vercel.
The Concept
In A/B experiments you typically split your users into two different groups:
You then use an Analytics or Experiments SDK tool like PostHog to randomly assign users to these groups in a specific distribution. For example, if you do 50/50:
50% of the users will get the Control version
50% of the users will get the Treatment version
Behind the scenes, this is often done by the SDK using what we call a feature flag. A flag that enables of disables experience in the website based on your group.
Minimal Experiment Setup
To make the minimal effort to set up your experiment with PostHog and Next.js, you just need to follow the initial setup and then the experiments docs, and you will end up with something like this:
This will show a different version of the card based on the flag value that the user is getting. For a lot of use cases, this minimal setup is enough to validate the hypothesis that you want for the experiment to be successful. But with Server Side Rendered pages this can create a problem.
SSR Problem
You might have noticed that we are using the "use client" directive at the top of the file, and that's because the useFeatureFlagVariantKey() hook only works on the client side.
And for SSR pages this will create a flickering experience like this:
Reloading the page makes the card change from the default variant to the expected one.
Before the page loads, in the server, there is no flag value so the control variant is shown, after the client side loads, the hook requests the flag value to PostHog and returns the treatment variant. In my opinion, this causes a weird experience for the users, fortunately, we can fix it by loading the feature flag values on the server side.
The Fix: Load the flags on the server side
You can use the middleware.ts to load the flag values before the page loads, grab the values, and save them to a cookie using next/headers or any cookie JS package. Then you grab this cookie on the client side when you are initializing your PostHog provider and pass it there as bootstrap data. The PostHog team has a nice post about it.
But If you host your website in Vercel there is an even easier way to do it.
Easy fix with Vercel Flags and PostHog
Vercel has a nice feature called Flags, that enables you to launch experiments and toggle things ON/OFF on your website without any external Analytics & Experimentation tool. But also enables you to integrate with the current tool that you currently use, in my case PostHog.
This is what the proposal looks like:
And this is what the actual implementation looks like:
And then you can use this flag in any server component:
With this implementation, your UI will not have that flickering effect since the flag value was already there from the SSR. I like this approach because it creates a seamless user experience.
This time I'm reloading the page and the card variant doesn't change.
There are more things we can improve in this experiment setup like:
Providing the server-side fetched flags to the PostHog provider as bootstrap data
Overriding the flag value using Vercel Toolbar is very handy
Overriding the flag value specifically for e2e tests
But I can cover those topics in the next post.
That's a Wrap! Thanks for Reading
This is it for my current setup for A/B testing! Hope this helps you set up your own experiments with Next.js, PostHog, and Vercel. Happy testing!
Github Repo with Sample Project