Using retrospectives for product feedback

How I used a tried-and-true retrospective format to get feedback about my internal product.

If you’re familiar with retrospectives you probably think of them as a way for your team to pause, reflect on how the last couple of weeks went and make adjustments to your team’s ways of working.

That’s usually how I think of them also. Then this week I had the chance to experience a new use for retrospectives - as a way to collect feedback for a product.

This approach works well if your product enables a process inside your organization.

Instead of examining how your team works (or doesn’t work) together, you’re examining how your users experience your product on a day to day basis.

The approach is not terribly complicated, and it can identify feedback and context you may not have uncovered otherwise.

Here’s how it happened.

A little bit of context

The product I spend most of my time on these days supports the sale of service contracts. Customers select the terms of the service contract and provide payment, then a team of customer success staff take that information and set up the service contracts in a different system.

The tech lead and I were going to do a training session for the team who process the orders to introduce some changes we’re planning for their interface for handling the orders.

In effect, it’s a kanban board to track orders instead of software development tickets.

That similarity may have been want sparked the tech lead to suggest we do a retrospective first to make sure we were aware of all the key issues the customer success staff experienced.

It helped that me, the tech lead, and the customer success staff were all in the same room.

Collect Feedback

We put four flip chart pages up on the wall of the conference room and handed out sticky notes.

I asked the customer success staff to think about the last 90 days of doing the order process and write their thoughts down on sticky notes around these four topics:

• What’s going well?

• What could be better?

• What have you learned?

• What still puzzles you?

The only restriction I placed was one item per sticky note.

Asking people to write their ideas on sticky notes (a friend of mine calls this “brainwriting”) instead of shouting ideas out offers a couple of benefits.

• One, you’re more likely to get more thoughts from those reluctant to speak up.

• Two, you see peoples ideas in their own words. This is particularly helpful when you’re using this approach to collect feedback because you can identify how people who use your product refer to it.

I gave the customer success staff a few minutes to write silently and at the end of that time I asked them to put the notes up on the appropriate sheets.

I’ve found that when people put their notes up without reading them out loud, you avoid self censoring that can happen when people are nervous to voice their ideas out loud.

Group the Feedback

Next, I had the customer success staff collectively get up to look at the sticky notes on the wall and group any similar items together.

This activity had the dual purpose of consolidating similar ideas and giving everyone a chance to see what everyone else wrote.

At the end of that exercise we had some grouped items and a few orphan sticky notes across the four pages.

In the isn’t that ironic category - there were two sticky notes both with the word “duplicates”. Thankfully they were grouped together.

Rank the Feedback

Next I counted up the number of groups on the sheets, counting sticky notes that were by themselves as a “group”. There were 14 groups so I told the customer success staff that they each had 5 votes that they could use to indicate which items they wanted to discuss. They could put all their votes on one thing, or spread them out.

As with all the other steps, there were a couple of reasons to do this. One, I wanted to determine the order in which we were going to discuss the items. This is especially helpful when you have a limited time and a lot of items.

Second, this informal ranking in this setting gave an indication of which items felt most pressing at the moment.

An important caveat - these rankings do not directly determine priority, but they provide an additional piece of information worth considering.

We ended up with one group with five votes, three groups with four votes each, three groups with two votes, and two groups with one vote each. The items that didn’t get any votes were the ones in the What’s going well? category.

Figures no one wanted to talk about the good stuff going on.

And in case you’re wondering, the two duplicate sticky notes got two votes.

Discuss and determine action

Once we had all of the items ranked, we started with the group that got five votes. I read through the sticky notes and then asked for additional context and comments.

I made it a point not to make assumptions about what the sticky notes referred to and ask for clarification. This is where the benefit of this approach came to light. We were we able to identify several pieces of feedback and get the all important context and explanation behind the items.

That’s not something you always get when you receive feedback asynchronously.

Not only that, but the tech lead and I were able to hear about about multiple issues and identify potential patterns that existed between them.

A key part of retrospectives is to identify actions the team can take to address the items that came up during the discussion. This case was no different. As we discussed a particular item, the tech lead and I steered the conversation to understand what changes would make the situation better.

Those changes weren’t always specifically product changes. In some cases the issue was something that we could address with some training, and in other cases we already had a fix underway.

As an example, when we came to the duplicate duplicates, we found out that referred to duplicate orders that made their way through the system. It turns out we had recently released a change to avoid creating duplicate orders, and were in the process of removing those duplicates.

Results from the discussion

The end result of the discussion was the tech lead and I had a better handle on what the customer success staff, and the staff had better insight into some upcoming changes to the product.

I also came out of the discussion with a set of feedback and some key actions that I need to add to the backlog. I should note that not all of the feedback directly generated backlog items, but we did identify some things that we could fix relatively quickly that will have a big impact on the staff’s experience.

We also uncovered some information that drove the tech lead and I to bring forward a more significant change that we had been going back and forth on whether it was worth doing. We heard enough during the discussion to tell us it was something we should do sooner rather than later.

Good technique for this situation

I’m going to things there. For one thing, I have some updating to do in the backlog.

And I wanted to point out that using a retrospective format to guide a conversation to gather feedback in this situation because our product directly supports a process. It was a natural fit because of the nature of retrospectives.

There will probably be other situations where this approach doesn’t make as much sense, and that’s ok.

Thanks for reading

Thanks again for reading InsideProduct.

If you have any comments or questions about the newsletter, or there’s anything you’d like me to cover, let me know.

Talk to you next time,

Kent J. McDonald
Founder | KBP.Media