Measuring success of a product operations team

  • 12 July 2021
  • 6 replies
  • 67 views

Userlevel 1
Badge +1

Hi there! I’ve recently joined as the first prod ops hire on a team with around 12 PMs. I’m finding that with the sheer volume of projects / processes / tasks to tackle, measuring success holistically can be a challenge.

To combat this I’ve been thinking through surveying the internal stakeholders of the various product teams on a quarterly cadence to understand how well informed / connected they are with product. I figure, this would correlate to processes within the product organization around communication and transparency within the org.

I’d love to hear folks thoughts on this approach. cheers!


6 replies

@Anthony Amenta - The way I’ve seen it done is to utilize a rating scale (0.0-1.0), similar to OKR success measurement. For example a quarter’s KR might be: “Roll out ProductBoard across all PMs”
Success definitions:
0.0 - 0.3 = Rolled out to only leadership, did not get training done for any PMs. Other priorities distracted us.

0.4 - 0.7 = Rolled out to all PMs, but training was limited. Not all PMs are fully up to speed in using it. Some PMs are using it fully, others are still getting familiar with how to incorporate it.

0.8 - 1.0 = Nearly all PMs are using it frequently and they’re embedding the usage of roadmaps with dependency discussions with stakeholders. Stakeholders frequently refer to PB and can identify what PMs plan to build in upcoming quarters with some PM help.

Badge

@Anthony Amenta I just completed my inaugural survey for a similar approach. I wrote about my process here: https://medium.com/fullscript/product-operations-from-zero-to-one-4a1739535d7a. The survey can be repeated to measure the impact Product Ops is making. Hope this helps!

Userlevel 1
Badge +1

@Joshmcla this is EXACTLY what I was looking to do. Thank you for building in the open and sharing your experience. I hope to do something very similar!

Userlevel 3
Badge +3

@Anthony Amenta I just completed my inaugural survey for a similar approach. I wrote about my process here: https://medium.com/fullscript/product-operations-from-zero-to-one-4a1739535d7a. The survey can be repeated to measure the impact Product Ops is making. Hope this helps!

Kudos to you, @Joshmcla -- this is an excellent write-up on starting a Product Ops position and team.
Admittedly, I haven’t thought about utilizing Productboard in a way to consolidate feedback internally about the product team - but it makes perfect sense. As you said “Treat your Product Organization like a Product” -- it only makes sense to outline deliveries and measure insights.
 

Badge

The only other thing I would add here is to define what success looks like for product ops. This is your “product vision” so to speak. I’d suggest co-visioning this with the other 12 PMs + key leadership stakeholders. Only then can you effectively decide on what to measure for success. This is important because like many have pointed out already, product ops means different things for different organizations. Are you a consultant group? Embedded into product teams/areas? How hard/soft are expectations for adherence to product practices? What’s product ops relationship like with other disciplines? These questions should ideally have hypotheses ready for validation and iteration.

@Anthony Amenta this is an awesome discussion and I’m glad that you raised it. Actually I touch a little bit on the important to measure your interactions (in this case was specific related the communications sessions that your PM organization has) in a talk that Manuel Pereira and I did included in the Product Ops Unwrapped Talk Series.

Feel free to check it and give your thoughts about it :point_down:

 

Reply