It is critical for you to understand how your product teams are performing. It’s all too easy for teams to make encouraging noises whilst spinning their wheels, delaying the point where problems are revealed causing delays to be compounded. What is needed is a regular check-in, or ‘Delivery Review’ on each team’s progress against their plans:
- Are they delivering work as expected?
- Is this work resulting in the impact expected?
A Delivery Review is a quick weekly check-in can answer both questions, as well as offer an opportunity to unblock teams that are getting stuck. This format gives an overview of operational metrics, delivery progress and team status, and sets up a joint problem solving session with leadership for teams that need it.
There are three pieces to the Delivery Review system:
- Delivery Update: teams fill in a simple template each week
- Delivery Review: leadership reviews the Delivery Updates and decides which teams (if any) need closer attention
- Office Hours: time for leadership and teams to come together and solve problems together
Delivery Update
This is a simple template for each to complete, each week. It should take the product manager or delivery manager 5–10 minutes per week, and consists of three sections:
- Operational metrics: the impact you want the team to drive (lagging indicators)
- Delivery progress: the work the team has done (leading indicators)
- Team report: team context that provides colour around what’s been going on in the past week
These sections are described in more detail below, and there is a link to the template here.
Operational metrics
In this section are a handful of metrics (typically 1–3) that the team is trying to influence. This should include metrics that define the objectives for the team, as well as any underlying drivers of these metrics. As the team delivers features, we’d expect these metrics to change, with a certain amount of lag, that will depend on the go-to-market strategy and metrics selected.
Delivery progress
This is where delivery is tracked. Each initiative or epic gets its own row, and each week the progress towards delivering this is tracked. Ideally this is done by measuring some quantifiable metric, such as tickets or story points complete. Clearly, a certain degree of pragmatism needs to be used when interpreting such measures, as obviously tickets can be of varying size, or developers could spend most of their week working on tickets that weren’t quite completed.
However, despite the limitations of tracking progress in this way, you can get a very quick read on the progress of teams, and spot work that appears to be slowing down. This allows for early intervention before too much time is lost or rework is required, so don’t let lack of perfect metrics prevent you from tracking something to get regular feedback from the team.
Team report
In this section the teams calculate how many engineering days they had, allowing for sickness, holiday and so on. They also record the split of work between different objectives. This is measured in the same way that delivery progress is tracker, e.g. number of tickets or story points completed.
It’s important to also call out technical work — planned work to increase the delivery capacity of the team — and operational load — any unplanned work, whether bugs, ad hoc requests from other teams, or anything else that prevents the team from planned work. One of the immediate benefits of this Delivery Update is that it quickly highlights high amounts of operational load that were previously invisible, and causing teams to be inexplicably slow. Finally there is a section for blockers, risks and mitigations, which together with comments on across the sheet allow teams to provide more colour to what is going on.
Delivery review
Product leadership (including product, engineering and design) meet each week to review the Delivery Update and see if any teams need support. The format of the Delivery Update means that you can easily see both progress week-to-week, but also the relationship between what the team is working on (team health), what they are shipping (delivery progress) and the impact this has on their target metrics (operational metrics).
It’s important to recognise that whilst the goal is to move the operational metrics, the only way to do this is through improving delivery — working on the best features, and shipping them promptly at quality. There’s no point dragging teams into Office Hours to berate them about their operational metrics if their strategy and delivery are sound.
“I have seen far too many people who upon recognising today’s gap try very hard to determine what decision has to be made to close it. But today’s gap represents a failure of planning sometime in the past”
Andy Grove, former CEO of Intel
Usually 3–5 minutes is sufficient to discuss a team’s progress, and decide whether that team should be invited to Office Hours the next day.
Office Hours
Office Hours is a one hour session each week that every PM and Tech Lead keeps free. Any team that wants to talk through problems with leadership can attend, and any team that the leadership wants to ask further questions or challenge can be asked to come along to. Teams generally need a 15 minute slot within Office Hours, with an agenda sent out the day before, following the Delivery Review. As teams only show up for their slot, it makes for a quick, efficient meeting.
Getting the tone right in Office Hours is critical. The product teams and leadership need to be working together to unblock the teams and get things moving. The more collaborative you set the tone, the more trust will exist between teams and leadership, and the more productive the discussion will be. Give the teams a hard time without helping them find solutions, and you’ll make them dread Office Hours and try to hide poor performance.
Summary
This process doesn’t replace good product and design reviews, where you can really understand and challenge the teams’ thinking. Rather, it complements them by providing an excellent way of tracking progress across multiple product teams in a short amount of time, highlighting teams that are struggling, and allowing you to focus your efforts on unblocking them. As always, process should be an accelerator to teams, not an obstacle. If this process is working well, the teams will not only move faster, but be happy to participate in it.
Further reading
- Yes, Product Managers Should Project Manage – Mind The Product
- Modern Project Management for Product Managers – Sachin Rekhi
- 5 tips & skills for insanely successful project management – Blink
