How Experiments Work at Tavrn

If you and a friend are both using Tavrn, you might notice something: sometimes one of you has a feature the other doesn’t. Or a button looks different. Or a notification behaves slightly differently.

This isn’t a bug. It’s by design.

We run experiments. A lot of them. And we want to be transparent about how that works.

What Are Experiments? 🔬

Experiments (sometimes called A/B tests) are how we test new features, design changes, and improvements before rolling them out to everyone.

Here’s how it works:

1. You’re assigned a test group when you sign up - Every Tavrn account is randomly placed into one of several test groups
2. New features go to specific groups first - If we’re testing something new or potentially unstable, we release it to one or more test groups
3. We analyze how it performs - We watch how people interact with the feature, whether it causes issues, and if it actually improves the experience
4. Then we decide what to do next - Keep testing, expand to more groups, push to everyone, or scrap it entirely

Why We Do This 🎯

Testing prevents disasters.

If we pushed every new feature to all users immediately, we’d break things. A lot. Features that seem great in theory can be terrible in practice. Changes that work for one type of user might frustrate another.

By testing with smaller groups first:

• We catch bugs before they affect everyone
• We learn what works and what doesn’t
• We can iterate quickly without disrupting the entire platform
• We make better decisions based on real data, not guesses

What Gets Tested? 🧪

Anything in Tavrn could be an experiment.

• New features before general release
• UI changes and redesigns
• Performance optimizations
• Notification behaviors
• Message formatting options
• Call quality improvements
• Settings layouts
• Literally anything

If it’s new, unstable, or might significantly change how Tavrn works, it probably goes through testing first.

How Do I Know If I’m in an Experiment? 🤔

Short answer: You don’t, and that’s intentional.

Experiments aren’t distinctly marked because we need you to treat them like normal features. If you knew something was experimental, you’d interact with it differently - more cautiously, or more critically, or you’d give it extra attention it wouldn’t get in real usage.

We need real behavior, not “testing” behavior.

That said, if you’re seeing something your friend isn’t, or vice versa, there’s a good chance one of you is in a test group for that specific feature.

What Happens After Testing? 📊

After we gather enough data from an experiment, we make a decision:

Option 1: Push to general availability
The feature works well, users like it (or at least don’t hate it), and it’s stable. Everyone gets it.

Option 2: Keep testing
The feature shows promise but needs refinement. We iterate and run another round of testing with the same or different groups.

Option 3: Expand to more groups
It’s working, but we want more data before going all-in. We expand the test to additional groups.

Option 4: Kill it
The feature doesn’t work, causes problems, or just isn’t good enough. We scrap it and move on.

Not every experiment succeeds. And that’s fine. Better to kill a bad feature in testing than after everyone’s already using it.

Your Test Group Doesn’t Change ✅

Once you’re assigned a test group at signup, you stay in that group.

This ensures consistency. You won’t randomly bounce between different experiences every time you log in. Your test group is tied to your account.

That also means:

• You might get experimental features earlier than others
• Or you might get them later
• Some features you see might never make it to general availability
• Some features others have might eventually come to you

It’s random, but it’s consistent.

Privacy & Data 🔐

We analyze behavior, not people.

When we run experiments, we’re looking at:

• Click rates
• Feature usage
• Error rates
• Performance metrics
• General interaction patterns

We’re not looking at:

• Your private messages
• Who you’re talking to
• Personal information beyond what’s needed for the test

And as always: we don’t sell your data. Not to advertisers, not to third parties, not to anyone. Experiments are for improving Tavrn, not monetizing you.

Why We’re Telling You This 💬

Most platforms run A/B tests without ever mentioning it. You’re in experiments all the time on other apps - you just don’t know it.

We’d rather be transparent.

You deserve to know how Tavrn works, including the behind-the-scenes stuff. Experiments are a normal part of building software, but they shouldn’t be a secret.

If you notice something different, now you know why. If a friend has a feature you don’t, now you understand. If something suddenly changes or disappears, it was probably an experiment that didn’t work out.

What If I Don’t Want To Be In Experiments? 🚫

Honestly? There’s no opt-out.

Experiments are how we improve Tavrn. Without them, we’d either ship broken features to everyone or never ship anything new at all.

But here’s what we can promise:

• Experiments won’t compromise your privacy
• We won’t test anything that puts your account or data at risk
• If an experiment breaks something, we’ll fix it fast
• Stable, core features stay stable - we’re not randomly breaking chat or calls just to test things

Think of it this way: you’re helping build a better Tavrn just by using it. And in exchange, you sometimes get early access to features that might become permanent.

Transparency Matters 🧡

We’re building Tavrn in public. That means being honest about how we work, including the messy, experimental parts.

Not every test will be perfect. Not every feature will succeed. But by testing responsibly and being transparent about it, we can build something better without treating you like lab rats.

You’re not subjects in an experiment. You’re part of the process.

Questions about experiments?

Drop them in The Commons or use the feedback form in Settings > App Info. We’re happy to explain.

Building better, one experiment at a time. 🧡

– Team Tavrn