12

Best interviewing experience?

10 months ago from , UX @ Neighbor + uxtools.co

I’m a hiring manager and I’m always looking for ways to improve our interview process. I’d love to hear about your best (or worst) experiences. How are things we’ve tried:

  • Portfolio reviews
  • Whiteboard challenges
  • Take home challenges
  • Design game (we made a board game!)
  • “About Me” presentation to the team
  • Personal interview
  • “Teach Me Something” activity

I know there are very mixed feelings about design challenges, but I have to find ways to understand if designers can think on their feet and if I can work together with them on a project. Take home challenges hide a lot of that process and obscure what I need from an interview.

So, what are your experiences?

25 comments

  • Lee Williams, 10 months ago

    the more experience I have doing interviews and interviewing others the more I'm convinced it's a process with extremely little merit. I can gain 95% of what I need to know in order to hire someone based on their portfolio and an hour in person design review where they present to me and some coworkers and we ask a lot of questions. Whiteboard challenges have limited use and often the challenge is either loaded in a way where the interviewers presuppose way too much domain knowledge or assume the the interviewee has some baseline context that they may or may not have. I was once asked to whiteboard a series of harry potter apps (I've never read the books or watched any of the movies, so that was great...). Take home challenges are ridiculous and no one should do them. It shows an extreme lack of respect for peoples personal time and the expectation that someone should only spend a couple hours on a project is absurd. I was given a take home project on a Friday for an interview on Monday morning once where I was supposed to design a new dashboard for their enterprise product. This is absurd on many levels, but most of all because a dashboard at it's heart a summary of the entire product, a hierarchy of needs to for every single thing your product does and in many ways functions as navigation. Sure let me spend 3 hours on that for you...

    I've seen people who've interviewed great end up being terrible employees and I've seen the opposite where I had low expectations and they crushed it. You can't know. So does their work seem intelligent? Do they seem sane and fairly socially adjusted? Yes? Ok, hire them.

    16 points
    • Samantha S, 9 months ago

      nailed it.

      2 points
    • Taylor Palmer, 9 months ago

      I know everyone has bad design challenge stories (I have my own), but let me play devil's advocate to this comment for a bit:

      Take home challenges are ridiculous and no one should do them. It shows an extreme lack of respect for peoples personal time

      Is this really true? I don't think we should take advantage of people, but by that logic isn't the interview itself an invasion of someone's time? If time is the concern, why bring them on site at all?

      the challenge is either loaded in a way where the interviewers presuppose way too much domain knowledge or assume the the interviewee has some baseline context that they may or may not have

      This sounds like a poorly designed exercise, not a problem with all exercises everywhere.

      where I was supposed to design a new dashboard for their enterprise product

      We tried doing things like this once and quickly found it has many problems, but it's also inappropriate to ask a candidate to do work for you. Again, it's a specific problem with that challenge, not with all challenges everywhere.

      I've seen people who've interviewed great end up being terrible employees

      Doesn't that make you feel like there should have been something in the interview process to root that out?

      Appreciate the thoughts, keep them coming.

      0 points
      • Gabriel Sturk, 9 months ago

        Is this really true? I don't think we should take advantage of people, but by that logic isn't the interview itself an invasion of someone's time? If time is the concern, why bring them on site at all?

        Come on... Showing up for an interview is a choice the candidate actively made.

        0 points
        • Taylor PalmerTaylor Palmer, 9 months ago

          Yeah yeah, I know this is hyperbole. But completing the exercise is also an active choice.

          We’ve also used a take home challenge as a second chance as well. If people don’t do well in the interview but our gut says we missed something, we’ll give the candidate the option of a take home challenge to show us a different skill set.

          A candidate can drop out at that point. Some do. But we can’t move forward until we’re confident in that candidate.

          0 points
      • Lee Williams, 9 months ago

        It 100% shows a lack of respect of peoples time and it's extremely biased against people with families, people who teach, volunteer and illustrates general lack of understanding of a good work/life balance. I have a toddler and a disabled mother in law that we care for. I do not have free time after work...

        As for white boarding, step back for a second and critically look at what you learn from a 30 minute whiteboard session? What have your learned that you may not have from asking them really good questions about their portfolio?

        The problem isn't the interview process, the problem is people. Humans are bias and association machines. We can render a kinda sorta ok surface level judgement based on very limited information, but beyond that we become extremely inconsistent. More information often makes our decision making worse... I'd encourage you to read the latest Malcom Gladwell book talking to strangers.

        lastly a big part of UX is try and eliminate our extreme inability to make good personal decisions/judgements. You aggregate data, you look for patterns, more often than not ignore what people say and focus on what they do, and it's good practice that the person who actions on the research shouldn't be the one performing it...

        4 points
    • John PJohn P, 9 months ago

      I was once asked to whiteboard a series of harry potter apps

      Christ... the only response to that is "Read another book"

      0 points
    • Noah MittmanNoah Mittman, 9 months ago

      100%

      0 points
    • Dave C, 10 months ago

      Totally agree with Lee, I’ll add - the trouble with all interview techniques seems to be that they aren’t a realistic litmus for how well somebody can do the job.

      No matter what the format, it’s always a bit of a punt - some people will interview well and perform well in the role, yet others will interview just as well and perform poorly. It could be them, it could be you, it could be the company and all round dynamic. (It doesn’t help that every ‘product designer’ role is interpreted differently at every company!)

      Even if somebody is extremely good at their job they might not perform for you - think about a major sports star who has changed teams and not been able to replicate their previous form at their new club. Did they suddenly get bad at their sport?

      Whiteboard ‘challenges’

      I just don’t understand these at all. What is it supposed to measure? How good somebody is at getting up in front of a group of strangers and doing some fictional work on a whiteboard? Is this how you design in your actual job? I hope not!

      Take home ‘challenges’

      Little bit insulting if you have any experience and/or a portfolio. It’s supposed to show your ‘process’ I guess. But what does that mean? Why doesn’t your portfolio show your process? And as everybody is just going say they do Lean UX/design thinking/design sprints/UCD etc are you really learning anything?

      Taking it one step further, your process will largely be determined by the organisation you work for, the time and resources you have at your disposal and the deadlines you have to meet. And er, despite what the conference gurus tell you, results are obviously more important than process.

      As Lee says, look through a portfolio, have a chat for an hour or so and you should be able to make a decent judgment. But you will never achieve certainty and it will always be a bit of a gamble.

      0 points
  • Nelson TarucNelson Taruc, 10 months ago

    Hi Taylor! Here's what we do. Two design challenges, less than an hour total. Done in the presence of one or two other co-workers (not just designers, but the people most likely to work the hire on a daily basis) who evaluate with a pre-created grading rubric (super important to avoid bias).

    We have a few tests to choose from depending on the role, but here's my go-to default. Find a case study of an actual app design (e.g. any long-winded post on Medium lol) that spells out some random designer's process. The more obscure the better (this challenge is most effective if the candidates have never seen it before).

    Give designers time to read the case study (e.g. 10-15 minutes) and take notes if they wish.

    Then have the candidate present what they liked or disliked about the case study, and what they would've done differently. Finally, ask the candidate whether they would hire that designer at Lucid or not, and why.

    I have to note we don't do any follow up questions except to clarify anything confusing the candidate said. We intentionally leave the like or dislike question open ended to let the candidate fill the space (and avoid poisoning the result with leading follow-ups).

    This design challenge doesn't require a whiteboard, take home or any of that junk, but definitely will reveal how the candidate thinks on their feet, presents their case, and the type of designers they prefer to work with (or not). And this specific test takes 30 minutes.

    Here's the important part, before you use the case study in interviews, have each of your designers take the test first! This will inform your grading rubric of things you should expect or not expect to hear, and help you see whether the case study is too hard or easy to evaluate.

    Final note, the grading is done independently and alone. Only after grading do the co-workers share their opinions (again, to avoid influence and bias).

    4 points
  • Campbell Kenya, 10 months ago

    Whiteboard challenges

    Oh! Look at this http://they.whiteboarded.me/

    4 points
    • Taylor Palmer, 10 months ago

      I readily recognize the shortcomings of whiteboard challenges, but that's pretty jaded! Whatever works for them.

      0 points
    • John PJohn P, 10 months ago

      Don't see why the design world is suddenly desperate to repeat the mistakes of the engineering world.

      2 points
  • Jonathan ShariatJonathan Shariat, 9 months ago

    The closer you can test/replicate real working conditions, the better data you'll get if someone is right for the position. Something we started doing that gave us the most relevant data for our efforts was the white boarding challenge where its meant to be highly collaborative. Its not just the interviewee monologuing and asking questions. The interviewers are meant to engage equally as partners to complete the challenge. In a real work situation you're meant to collaborate. So it helps to see how they collaborate, think, take feedback, give or take direction, "yes and" ideas, etc.

    In addition we make sure that they are fully prepared for the meeting. We tell them exactly who they will meet with, what will be discussed and what we are hoping to learn about them in that meeting. We include some tips as well. The first onsite meeting is "de-pressurize" and prep them. They meet with other designers, they talk about life, get setup, and give advice. Its meant to put them at ease which of course they are trying to impress us but I've found that it takes the edge off the presentation

    Also super important to ensure everyone talking to the interviewee knows everything about them, their work, and what they are suppose to ask and prod for.

    Good interview questions should be preset to remove bias and increase quality.

    Have a debrief that same day if possible.

    2 points
    • Taylor PalmerTaylor Palmer, 9 months ago

      All great points, and I like the preset questions idea.

      Your approach to exercises sounds very similar to my own. I explain to them, "This isn't a challenge, puzzle, or brain teaser. There isn't a right answer and you won't be expected to find one. Instead, let's work together to solve a problem. Show me how to you like to think and work, and I'll do my best to help design with you along the way."

      I then introduce the prompt to them, and give them a few minutes alone to cool off, gather their thoughts, and wrap their head about the prompt.

      It's the best I've been able to do so far.

      0 points
      • Jonathan ShariatJonathan Shariat, 8 months ago

        Thats a good point about giving them a few minutes. Its funny how often we ask interviewers to do this crazy marathon day, talking almost nonstop for 7 hours. Then have them also talk to us over lunch. I havn't done this yet, but I wonder if having a break where we leave them alone with snack and drinks and bathroom break in the middle somewhere would help as well.

        0 points
    • Stefano TirloniStefano Tirloni, 9 months ago

      This is the best approach to me, a collaborative whiteboarding/design challenge.

      I found the take home exercises less effective: - it's quite hard to understand the task and provide the exactly outcome that they want (and please don't spend more that 4 hours.... )

      • they are async, if I have a lot of question it takes time to get the answers

      • unbalanced about time. I spent the famous 4 hours to make it but you review it in 5 minutes

      • you get few and low quality feedbacks at the end since you've worked alone ( we expected more craft on this than that.. )

      0 points
  • XXx dddoozza, 9 months ago

    First and foremost, a lot of your "examples" are utter nonsense and very bias.

    Hiring for UX/UI/Product all depends on the level of seniority that you're hiring for. I hope you don't think it's the same when hiring a junior vs a senior vs a director.

    Portfolio reviews are to showcase if the person knows how to present themselves (branding) per se. Also, they don't tell you shit about the person. Ultimately, I want to know how they created their portfolio, if they developed it themselves, how did they go about selecting the projects on there? After that, I go ahead and ask them about their favorite project to see if the passion is there.

    Whiteboard challenges? what? You want them to work for free with no sense of what the project and the politics it consists of?

    Take-home challenges? Again, you're trying to get free work.

    Design game? Why are you wasting people's time? Don't' you need someone to collaborate and enjoy working with the team?

    "About me" presentation? Do you want another dog and pony show? Quickly answer this, is the person an asshole? Are they rude? Do they not know what they're talking about? Now they have to prove that their attitude aligns with your bias?

    Personal interview...teach me something? You're getting creative for no reason. You can easily find a good working peer if you just stick to the basics. Is the portfolio good? Do they have a passion for the job? Are they assholes?

    I hope your leadership team has a good amount of PoC, cause all of these screams red flags to me.

    1 point
    • Taylor PalmerTaylor Palmer, 9 months ago

      Hi XXx dddoozzaa, thanks for taking the time to respond. Your response, however, isn't reflective of my initial post at all and is pretty disrespectful.

      I hope you don't think it's the same when hiring a junior vs a senior vs a director

      I never indicated that we don't change our hiring process for junior vs a director.

      Ultimately, I want to know how they created their portfolio, if they developed it themselves, how did they go about selecting the projects on there?

      That's an interesting perspective to a portfolio, I'll think more about that.

      Whiteboard challenges? what? You want them to work for free with no sense of what the project and the politics it consists of?

      A whiteboard challenge is a simple, fictional problem that we wireframe together on the whiteboard. This is not work for the company.

      Take-home challenges? Again, you're trying to get free work.

      Not trying to get free work, but I think we've already established in this thread the questionable ethic of doing this.

      "About me" presentation? Do you want another dog and pony show? Quickly answer this, is the person an asshole? Are they rude? Do they not know what they're talking about? Now they have to prove that their attitude aligns with your bias?

      You know absolutely nothing about what we do in this activity, so I'm surprised you feel so strongly about it. If you were hiring a team member, wouldn't you want to know where they went to school? Where they've worked? What they learned there? What they're passionate about in their work? If you're not interested in those parts of my life, then I'm not sure I would want to be on your team.

      You can easily find a good working peer if you just stick to the basics. Is the portfolio good? Do they have a passion for the job? Are they assholes?

      Are you saying that from experience? Have you hired and built a team? Portfolios can be heavily doctored and borrowed from other team members or students. Being passionate and not an asshole are basic characteristics of being a good human, not the only baseline for being a great designer. We have a team of 15 designers and only 2 have ever left the team in five years. I'm not just trying to find someone who can fog a mirror, I'm looking for someone who's as invested in this as I am.

      If I find someone who can walk and talk, I'll be sure to send them your way.

      Thanks!

      0 points
  • Adrian FurtunaAdrian Furtuna, 10 months ago

    Here's my best interview ever and something I borrowed for whenever I have to interview someone.

    I was a approached by a design manager for a bank. They wanted to build a whole new design studio, with designops and everything and they were looking for a UX lead. Rethink all of their interaction online and off. Sounded interesting so I said yes.

    We met for coffee. Which I found nice, rather than sitting in a conference room. And we simply talked. He very carefully guided the discussion around the design sphere locally and internationally, asking for my opinion on various subjects, podcasts, instagram design influencers, politics, previous workplaces, ways of working etc. Everything that he needed to know was taken out of the conversation and my tonality and passion for some of the subjects.

    For some of the subjects I even got my laptop out of the bag and showed how I approached things for some of the issues we were talking about and he showed me some of his.

    More than this I really appreciated the transparency. I was told the budget range from the very start, how their organization was set right now and that it was a start of the road kind of engagement.

    1 point
  • Alf SalibAlf Salib, 10 months ago

    I know there are very mixed feelings about design challenges, but I have to find ways to understand if designers can think on their feet and if I can work together with them on a project.

    Why not do that by just... y'know... hiring them? Let them show you how they work as a designer by letting them actually work as a designer.

    Too often there are only two possible outcomes of a job application — You're either hired full time or you're not. Why not give them a "paid design challenge" by bringing them in for a small project on a freelance/contract basis, paying them for their time, and reevaluating after the project is finished?

    Both parties actually benefit from this approach, no matter what the outcome is. If it's positive, then you've hired a designer, and the applicant gets hired. Win-win. If it's not meant to be, then at least you got some design work done during the hiring process, and the applicant got a bit of money in their pocket and some more experience under their belt. Win-win.

    1 point
    • Taylor PalmerTaylor Palmer, 9 months ago

      I like this idea, and we've been considering this from an "apprenticeship" perspective for jr/associate level designers, but this doesn't really work out for sr-level designers. They're looking for high pay and benefits, not a few months of uncertainty.

      The market I'm in (Salt Lake City) is a really competitive design market right now, so designers aren't going to jump ship from their nice design job to possible have a career-ending three month experiment.

      0 points
  • Klark Dollson, 9 months ago

    Hi! I started to find design jobs but don't find anything and decide to change job. When I shared thoughts with my friends they advise me to start trading and search for something on the Internet, because it's good to work and I can work at home. And I started to look for information on the Internet: read the article on the topic of trading, watch the video and then came across a site https://fx-list.com/blog/articles/profiting-from-currency-trading with information for brokers company. Now I have a lot of free time and have a positive, social impact.

    0 points
  • Taylor Palmer, 9 months ago

    I know this has sort of just turned into rants about whiteboard challenges, but hiring is hard. Interviewing and finding the right people is hard. Hiring the wrong people can destroy teams from the inside out. Hiring the wrong people, after salary and benefits and healthcare and equity and performance improvement plans, can be a million dollar mistake.

    Thanks for the thoughts so far.

    0 points
  • Jane BrewerJane Brewer, 9 months ago

    Very clear and informative.

    0 points