ASK DN: Do you perform user testing?

over 8 years ago from , Art Director at 14four

If you do, how often do you do it? What user testing methods do you prefer? What is your process? Do you use specific software?


  • Adrian HowardAdrian Howard, over 8 years ago

    No. I perform usability testing. Because we're testing the product, not the customer ;-)

    I tend to work with agile/lean-ish teams. So the "how often" is usually once every sprint, or every couple of weeks.

    As for software — whatever seems the best option for the project/test at hand. Often none. Often just some screen sharing app (GotoMeeting, Skype, etc.). Occasionally things like Morae, Silverback, or remote tools like usertesting.com.

    Really the software isn't the key. Understanding how to get useful actionable info out of the test is. The software is mostly incidental to that. For info on technique I'd recommending reading these three books, in this order:

    1. Rocket Surgery made Easy, Steve Krug — Read this first. Fun to read and very short. Gives you enough basics to get going and be useful. Jargon free. More relevant to the rapid-iteration style usability testing many folk are now doing. Really helpful to people who don’t have a UX background.

    2. Handbook of Usability Testing, Jeffrey Rubin & Dana Chisnell — If RSME is useful, then this will fill in gaps. Solid practitioner’s guide aimed much more at folk whose "job" is doing usability testing. More suitable for those who need to do more "formal" usability testing in larger organisations.

    3. The Moderator’s Survival Guide: Handling Common, Tricky, and Sticky Situations in User Research, Donna Tedesco, Fiona Tranquada — What it says on the tin. A bunch of advice and war stories on dealing with common problems. Read once you’re past the basics to save yourself some on-the-job learning (Bias warning: one of my war stories is in this ;-)

    9 points
    • Tyreil P, over 8 years ago

      No. I perform usability testing. Because we're testing the product, not the customer ;-)

      Thanks for pointing out that distinction.

      I tend to work with agile/lean-ish teams. So the "how often" is usually once every sprint, or every couple of weeks.

      Wow, that seems like a lot of leg work to gather users. I'm assuming you're testing new users every time, right?

      I've picked up "Rocket Surgery Made Easy". I'm really looking forward to making it through your recommendations.

      0 points
      • Adrian HowardAdrian Howard, over 8 years ago

        Wow, that seems like a lot of leg work to gather users. I'm assuming you're testing new users every time, right?

        Depends on the team (I've turned into a sad consultant in my old age and work with various folk). For example, sometimes the user group is small enough that there's no option.

        That said — it can be a lot less leg-work than you think. A lot of the work in recruitment is setup: Finding your channels to end-users, negotiating with recruiters, sorting screening-questionnaires, etc.

        If doing usability testing is only occasional then yes – that's a lot of work to sort out for each test.

        However, once you switch to a continual model (we'll see X users every Y weeks) then most of that stuff is static — you just have to keep the process running.

        It's even more useful when you decouple the recruitment process from what you do with those customers. If you know that every other Wednesday you'll have N customers to talk to you can decide on the Monday or Tuesday whether the most useful thing would be usability testing, or interviews, or paper prototyping, or a co-design session, or… whatever.

        0 points
  • Mike A.Mike A., over 8 years ago

    Usability testing (not user testing, you are not testing people - as Adrian pointed out) is fundamental for every successful design project.

    There's no other effective way how can you validate your design / product / concept / ideas before launch.

    I recommend to start with usability testing as soon as possible - quick and actionable feedback is something that everyone on your team will appreciate.

    The chosen method always depends on what you want to learn and what are you testing. Don't go for specific tools, methods or software prior answering these questions.

    2 points
  • Alina WeinsteinAlina Weinstein, over 8 years ago

    Hi! At MyFab5 we do user testing as we develop new features or redesign old. I test my first versions, modify them based on feedback, and then test again. Right now we are overhauling the whole app and I have been user testing every week for the past 3 weeks. I prefer in person testing, especially, for mobile apps. It lets me see better what users are trying to tap and how they are trying to swipe. I usually use InVision or MarvelApp, install it on a user's phone (important!), and ask them to tell me what they see and what they want to do before they actually do it. User testing has given me invaluable feedback.

    1 point
  • Rasmus LandgreenRasmus Landgreen, over 8 years ago

    Take a read here, it's good. And how I think it should be done: http://blog.getbeagle.co/post/111948665206/beagling-our-way-to-a-minimum-viable-and-lovable

    I think the biggest challenge is balancing the finances/time-aspect. It's expensive to keep figuring things out and poking around, but the more time you spend early on, the better your chances of getting it right in the end are.

    So yes, definitely, I do test. But reality sucks ass.

    0 points
  • Thomas PetersenThomas Petersen, over 8 years ago (edited over 8 years ago )

    Not unless I am dealing with something truly new. I much rather prefer to launch the product and then test the actual feedback there.

    Usability testing is often a waste of time and money for most people involved and does not provide any real insights. What you end up testing is the prototype not the actual product which are two very very different things.

    This is also the reason why this field is luckily taking a backseat to actually launching products, especially the shallow products we see these days.

    There is enough knowledge about most design patterns to know whether they work or not.

    0 points
  • Andrea MontiniAndrea Montini, over 8 years ago

    Hi all, I'm planning the first usability testing for a new product we're working on. We're still in the MVP design phase, and we'd like to gain insightful feedback on the features we've thought and included in the product.

    I've wrote a document in which I pointed out some basic questions/tasks about the product, and we'll perform an on-site test to actually see what people do as they use the App.

    The best way is to test live, in my opinion. You can then set Direct Tasks and Open ones, which can bring people to talk wisely and freely about the App. You'll gain both qualitative and quantitative data, which are great insights you can use to improve the product.

    0 points
  • Samantha S, over 8 years ago

    Yeah, we measure dopaminergic activation

    0 points
  • Daniel WhyteDaniel Whyte, over 8 years ago

    Always test, but it's always different, you should checkout https://uxchecklist.github.io

    0 points