20

A/B testing tool for designers

almost 4 years ago from , UX Developer & Tech Lead

We know we can easily do A/B testing once designs are coded and they are live as a website or an app.

I'm wondering if there is a tool that allows designers to do A/B testing before a design is live. Something that potentially allows also to test non web designs (i.e: a/b test letter heads or pamphlets etc.)

19 comments

  • Ole Harland, almost 4 years ago

    Create two pictures and put them into a google form with two radio buttons. Send this link to your test candidates and done.

    4 points
    • Thomas PalumboThomas Palumbo, almost 4 years ago

      But what are you actually testing? Opinions on design or usability?

      1 point
      • Bappy Golder, almost 4 years ago

        That depends on what I am testing. Sometime it is a matter of testing opinion if you are comparing say 20 different design of the same logo.

        Other times you might want comments on usibility of 2 different version of a same app.

        0 points
    • barry saundersbarry saunders, almost 4 years ago

      That'll test which ones people like, not which ones are effective.

      2 points
    • Bappy Golder, almost 4 years ago

      Hey thanks Ole, I tried doing this with Typeform. But doing it with form presents some problems like image zooming.

      Comments on images etc.

      0 points
  • Sam Bible, almost 4 years ago

    Depending on what feedback you're looking for, A/B testing might not be the best tool.

    A vital concept of A/B testing that often gets overlooked is that you're going for statistical significance. Without statistical significance results are often little better than guessing. The catch is that it often requires a larger sample size depending on the variables involved, which is why they are commonly conducted with a live audience. There's a great overview here.

    But sometimes it's incredibly helpful to just get a quick read on preferences, potential trouble spots, what stands out to visitors, etc. A couple of tools I've found really useful for that in the past are:

    3 points
    • Bappy Golder, over 3 years ago

      Hey Sam thanks for sharing the links of Zurb & Usibility Hub. They both look like great resources. I think what I'm after is more like the Preference Test option provided by Usibility Hub.

      0 points
  • Andric ThamAndric Tham, over 3 years ago

    Yes, there is. I’ve been using a tool called Chalkmark that lets you upload a bunch of static images to test “first clicks”. It’s been such a dream using it.

    It’s kind of like a usability test for mockups. You can give users a series of screens accompanied by a task, and then they have to click on wherever they feel helps them achieve that task.

    For example, if you want to test if users know where to click to checkout on an online shop, can have a task like: “You just added a pair of headphones to your cart and want to checkout. Where would you go to do so?”. You can set your cart icon to a “correct area”, and Chalkmark will record if users clicked on it or not, and give you a % score.

    To achieve an A/B test on a static design, simply upload the A version and B version(s) to Chalkmark as separate tests. Chalkmark will give you an option to recruit users from their panel (very cheap, compared to usertesting.com), or you can simply use the provided link and share it with your own contacts.

    To automate the process further, you can use Ethnio to intercept users on your site and send them randomly to different tests. You should also be able to build some kind of workflow or pipeline around the data that Chalkmark gives you since it’s a simple .xlsx file.

    3 points
    • Bappy Golder, over 3 years ago

      Hey Andric, thanks for the talking about Chalkmark. I'm keen to try Chalkmark out. It seems kind of pricy for one designer to get started though.

      But it definitely looks like something I'm looking for.

      0 points
  • Billy CarlsonBilly Carlson, over 3 years ago

    I have long wanted to build a much simpler tool for this. I feel like the examples already posted are either too feature crazy, or with the usability hub, too simple. Another feature I would like for this, is allowing the user to view multiple versions of the same design, so they can see how it evolved over design iterations.

    1 point
    • Bappy Golder, over 3 years ago

      Hey Billy, I totally agree with you. In fact this is one of the reasons I posted this question in the first place.

      I would like to build something for my team's internal use. Something that is not too feature heavy and possibly free for designers.

      Would you be interested to collaborate with me? Even if you do some early beta testing that might be helpful.

      0 points
      • Billy CarlsonBilly Carlson, over 3 years ago

        I would absolutely love to! I have been talking with my design team about creating wires and designs for this tool. Hit me up at billy [at] threadless dot com! It would be so fun if we could work something out.

        0 points
  • Paul ArmstrongPaul Armstrong, over 3 years ago

    I run a team of 5, and for the amount of traffic we get and the type of product we have, we've gone with VWO.com.

    As was stated previously, the most important part of any A/B testing of variations is the data. Make sure you are eliminating as many variables against the most robust data available. It's just the scientific method:

    1. Hypothesis : What do you want to learn, and what do you believe will be affected by this change? (e.g. changing the color of this button will increase clicks).
    2. Data : Gather about 6 months of data from (a source like) Google Analytics around your hypothesis (what's the CTA of this button?).
    3. Test : create the variation (b) that will run against control (a).
    4. Conclusion : Review the results of the CTA on A and B
    5. Next steps : What things should you implement based on what you learned?
    1 point
  • Brennan Smith, almost 4 years ago

    I use UsabilityHub and get good feedback from each user tested.

    1 point
  • chris caseychris casey, almost 4 years ago

    usabilityhub.com

    Upload two designs and ask community which they prefer

    1 point
  • Vipul. MishraVipul. Mishra, over 3 years ago

    If you want to try A/B testing on the flow - you can try http://CanvasFlip.com . Records user videos, heatmap & even shows how conversions will look like..

    Hope it helps.

    [Disclaimer: I am associated with CanvasFlip.]

    0 points
  • Johannes IppenJohannes Ippen, almost 4 years ago

    Hey Bappy, what you're looking for is not really an A/B Test but a survey – check out Google Consumer Surveys, they are great for that: https://www.google.com/analytics/surveys/

    0 points