28 Split Testing Tools for Mobile, WordPress and More
A short while ago, we published a roundup of 24 split testing tools in all price ranges. But those tools focused mainly on the Web experience, and there are many more tools out there.
This roundup focuses on tools for split testing mobile apps and WordPress sites, as well as a few bits of code for the really hardy code wranglers among you. Surely, among these 28 tools, you’ll find what you need.
Mobile App Split Testing Tools
These days, you’re just as likely to deliver content via a mobile app as via a website. If you do, or even if you’re doing some kind of hybrid thing, you need one of the tools listed below.
1. Amazon A/B Testing
Amazon has a free tool for creating and running in-app experiments. It has cross-platform support, working with the iOS, Android and (of course) Amazon platforms. The tool includes custom segments and is scalable. One case study video shows how Air Patriots used the Amazon tool to improve retention.
Arise works with Android and iOS, allowing you to test several variations from its dashboard. It includes goal tracking. The app is free for unlimited applications with a cap of 5,000 tested users.
Apptimize says “experimentation is key to discovering the truth” so it’s no wonder the company is gung ho about mobile testing. It aims to provide a way to do risk free iOS and Android app testing, allowing users to run visual tests quickly and roll out app updates immediately without needing to go through the app store. Apptimize allows staged rollouts of new features and includes analytics, too.
The basic package is free for 25,000 monthly active users and there are paid packages available for larger user numbers and additional features (such as using advanced targeting and analytics and custom APIs). Check out their Safetrek case study which resulted in a huge increase in the number of app store reviews and app shares.
Artisan’s split testing function allows for in-code testing, visual optimization, immediate publishing of variations and multiple configuration options through “Power Hooks.” It includes robust analytics too. The tool is free for up to 10,000 monthly active users. One case study shows how A View From My Seat used the tool to increase usage by 22%.
Bestly includes mobile analytics and split testing for iOS, with Android promised soon. It tracks user behavior in real time and allows app creators to roll out changes slowly to specified subsets of users. The price starts at $19 a month for 2,000 monthly visitors and 20,000 monthly events.
Fliptest is an iOS split testing tool which works with a single line of code. Tests can be set up and monitored online via a Web interface. They run instantly, providing real-time results.
Want to apply the power of Crazy Egg heatmaps on your app? Heatmaps highlights the hottest areas of your iOS app, allowing you to track gestures, device orientation, navigation flows and user engagement. There’s a 30-day free trial, then the price starts at $19 per month for a single app and 10,000 data samples.
The same developers have also created another tool, Hercules, which is still in beta. Hercules allows mobile A/B testing and in-browser experiments which are propagated live.
Kii is an application development platform that incorporates A/B and multivariate testing. Developers can make the desired variations live with a single click to help improve engagement, retention and usage. There’s a free level for one experiment a month with unlimited users. The premium package costs $199 per month but was also free at the time of writing.
Leanplum offers three main features. The first is split testing of an app without coding and without the need for app store approval. This includes testing in-app and push messaging. It works with advanced segments and also offers analytics with a wide range of filtering options. The analytics dashboard can be customized and offers automatic insights.
Leanplum also has built in collaboration, allowing project managers and marketers to do their own content management, testing and analytics reporting. It works with iOS apps, with a free level for 500 daily active users of a single app. See how Wanelo used testing to increase product saves by 27%.
Optimimo includes a range of mobile app testing methods. In addition to A/B testing, it offers heatmaps so you can check click patterns on different parts of the user interface. You can also analyze navigation flow and conduct usability surveys. Optimimo allows you to make variations live immediately, with real time updates on performance. It is available for both iOS and Android. You can test one app with up to 2500 visitors a month for free.
Splitforce provides testing for iOS, Android and Unity apps. It allows you to target user segments and locations, provides analytics and allows instant rollout of app changes. The tool includes adaptive learning algorithms to show what works best most often. Pricing starts at $20 a month for a single user account and app with 50,000 data points.
Swrve allows for testing all parts of the user experience but focuses on engagement rather than a single conversion event. Tests are completely customizable and, according to Swrve, very accurate. This tool allows for testing of messaging, notifications and user experience, and also provides robust analytics. One case study shows a 120% improvement in retention rates.
Taplytics says it’s “the world’s most fully featured mobile A/B testing platform.” Featuring an easy-to-use graphical interface that will suit most people, it also includes Code Based Experiments for those who want to take that approach. While it’s only available for iOS, the app allows users to segment results from those using different versions of the mobile operating system.
There’s a free pricing tier for a single project,with one team member, 1GB of data storage and 5,000 monthly active users. One case study on their blog shows how Launchpad Toys used Taplytics to help double in-app revenue.
Vessel is an A/B testing tool for all aspects of your mobile app. It works with the major mobile operating systems and HTML5. Built into the tool are funnel analysis, user segmentation, multivariate testing and the ability to immediately release winning variations. There’s a 30-day free trial, with the basic package starting at $150 per month for 100,000 monthly active users.
WordPress Split Testing Tools
If you’re using WordPress, then it’s worth considering a tool designed specifically for the platform. While some of the tools in our last roundup (such as Google Content Experiments and Optimizely) also work with WordPress, here are a few more.
15. AB Press Optimizer
AB Press Optimizer allows you to test variations of page elements such as images and buttons and content, such as headlines and text on a self-hosted WordPress site. To use it, you need the plugin plus the ability to use shortcodes (though you can use PHP if you wish).
You can run and get real time reporting on unlimited experiments with unlimited visitors for the $39 personal license, though you will have to upgrade if you want additional support.
16. AB Theme Testing
If you’re running a WordPress site, sometimes you need to get back to design basics and test a couple of themes. That’s where AB Theme Testing is useful. It integrates with Google Analytics so you can see your data in your Analytics account. This plugin costs $19. The developers have used it to tweak their own site, as this case study shows.
17. Convert Experiments
Convert Experiments provides A/B and multivariate testing for WordPress. Developed by Convert.com and Yoast (developer of WordPress SEO), it works for those who have a Convert Experiments account. When you use this plugin you get a special account allowing you to test 5,000 visitors for month for a year. You will have to setup tests on the Convert Experiments site.
Converthis is an A/B testing and conversion tracking plugin for WordPress. You can use it for click tracking and to track multiple page variations. Users can also adjust the weight given to particular options in the premium version.
The plugin allows you to test unlimited campaigns on unlimited websites with unlimited visitors even in the Lite version at $29.95. However, you will have to upgrade if you want to split test 6 variations, customize colors, and edit titles and descriptions.
19. Nelio A/B Testing
Nelio A/B Testing bills itself as a conversion optimization service for WordPress. You can use it to test almost any aspect of a WordPress site (posts, pages, titles, themes, heatmaps, clickmaps and CSS) and make automatic changes when you have a result. You don’t need any technical knowledge to use the plugin and can use all features from within the WordPress dashboard.
There’s a free trial, and the starting price is $19 per month for a single site with 5,000 tested page views.
20. Simple Page Tester
21. Marketing Optimizer for WordPress
Marketing Optimizer for WordPress is a full-featured split testing and conversion rate optimization plugin. It also integrates with the Web app of the same name. The plugin plays nicely with caching plugins and Gravity Forms, allows for testing of multiple pages (including rotating variations) and allows you to create new variations with a couple of clicks. The plugin is free; the whole suite starts at $149 per month.
Other Split Testing Tools
This last batch of split testing tools is for people who feel comfortable with code—really comfortable. As a marketer, you may not use these yourself, but the IT guys who develop your app might want to know about them.
22. Abba is a self-hosted split testing framework that has support for multivariate testing and has simple filtering functions. It works with Ruby.
23. Developed by Patrick McKenzie, A/Bingo attempts to bring a marketing tool to programming, allowing them to split test single lines of Ruby on Rails code. Case studies for landing pages, buttons and signup forms are on this page.
24. FluidFeatures Rails is another split testing tool for Rails. It includes multivariate testing and cohort definition.
26. Split is a Rack-based split testing framework which works with Rails and similar apps. It can be used to test page variations, conversions and more, with the option to show weighted alternatives.
27. Switchboard is a mobile split testing tool for Android and iOS which includes segmentation, user flow testing and much more. However, you will have to plug in an external solution to get analytics and attractive charts.
28. Vanity is another tool designed to work with Rails to split test design, content and feature changes. It can test virtually anything and integrates with Google Analytics.
Did you see your favorite split testing tool on this list? Let me know if it should be added to the list—and why.
Read other Crazy Egg articles by Sharon Hurley Hall.