A/B Testing with ASP.NET MVC

UPDATE: this blog post is about the motivation behind writing ABTestMaster and under the hood technical details. If you just want to use it, install the Nuget package and visit the GitHub page for documentation.


A few weeks ago, I was looking for a neat and simple split testing library to help with some of my personal projects. Given the importance of A/B testing for anyone developing a web application (startups and established companies alike), I expected an avalanche of choices and at least a handful of ASP.NET MVC specific ones. Well, that’s not the case,. I didn’t find many and I did not like any of the ones I found. So, I wrote one and called it ABTestMaster (because I am not good with names)!

The Idea

Simplicity and ease of use were probably the most important factors in my mind when developing ABTestMaster. As a developer, I don’t want split testing code to contaminate my code (whether it’d be C#, HTML or god forbid, Javascript). Also, it should be trivial to add it to a project. So, the end result is a library that allows you to specify split views via simple action method filters. You need to inherit from AbTestMasterController and annotate a couple of action methods and… and well, that’s it.

How it works

When ASP.NET MVC application runs, ABTestMaster takes the name of assembly and looks for all its action filters (SplitView and SplitGoal). During the process, it adds all relevant data including action name, controller, area, and the parameters passed to it.

When a call is made to one of these action methods, the custom ControllerActionInvoker kicks in. It randomly picks one of the action methods in the same test group and calls it. If the call is successful (no exceptions, and 200’s family status code returned), it records this to a CSV file. It also saves the information in the browser’s cookie so next time the same view is displayed to the recurring user.

Another aspect of ABTestMaster is setting goals. You define what’s known as a “Sequence” and annotate an action method as a SplitGoal and when called, it records this and any SplitView called in the same sequence to a separate CSV file. This way, you will know which views were displayed to the user before the conversion.

I think developing a framework like this in a matter of days is a testament to the extendibility of ASP.NET MVC framework. So hat’s off to people at Microsoft for making this possible and doing community (and themselves) a favour.

How to Install

To help with installation and remove any need for initial configurations, I have created a Nuget Package that includes the necessary dll’s, adds the bootstrapping method call when application starts and creates the empty CSV files to the project.

What’s Next?

In this initial version, all views in the same test group have an equal chance of being displayed to the user. A natural feature would be to set the chance of a view being displayed or even set the maximum number of times it can be displayed.

Another feature which might come in handy is to configure it to write the view display data to a target of your choice, whether it’d be a file or a database record.

However, before I look at do of these, I am keen to get some feedback from some developers and see what they think. So if you happen to use it at some point or if you’d like to get involved, I’ll be happy to here from you. Leave a comment here or contact me via the GitHub page.

Nuget Package: https://www.nuget.org/packages/AbTestMaster/
GitHub: https://github.com/htayebi/AbTestMaster