WordPress Triage Team: A 3 Month Reflection

A workbench in front of a tool rack on the wall with saws and chisels

Last week (June 11, 2019) marked the 3 month mark since the initial WordPress Core Triage Team kickoff meeting. As the team lead, a significant portion of my time every week has been spent in the trenches triaging tickets in the WordPress Core Trac instance.

The amount of time spent each week triaging tickets has been difficult to track and varies greatly week to week depending on higher priority tasks that come up, and pitching in to help ensure releases are pushed out smoothly. These fluctuations are very roughly visible in the ticket graph.

With WordCamp Europe Contributor Day later this week, I wanted to take some time to reflect on what I found. Below you’ll find some observations I have had in the last three months, as well as some workflows and focus areas that I have found helpful.

During Contributor Day, I will be leading Core Triage Team table. If you find this sort of work interesting (or just want to meet), please stop by and say hi! 👋

What is Successful Triaging?

There are ~6,600 open tickets in Trac at the time of this post. This number goes up and down week to week by as many as 100-200 tickets. Looking at the 30,000 ft. view is not helpful and very intimidating, even to more senior contributors! So, how do you know if your efforts are meaningful and successful? How do you know that you are making progress?

In the last three months, this is what I have identified as the best definition to keep in mind as a day to day guide while triaging.

Effective triaging is moving a ticket one step closer to a resolution every time the ticket is touched.

Because every ticket is a unique little snowflake that has a different journey and requirements, this can mean many things. For one ticket, this could mean actually closing. But for others, this will mean adding has-patch, needs-unit-tests, reporter-feedback, or a suggestion to close. Any action that moves the ticket closer to a resolution (no matter what that resolution is) is a successful effort, even if the update requires revisiting the ticket shortly after (for example, having to revisit a ticket in 4 weeks to see if any reporter-feedback was left).

A person holding up a black camera lens. The city of Los Angeles is in focus through the lens.
Photo by Devin Avery on Unsplash

Limit Your Focus

In order to avoid being overwhelmed, it’s very important to limit your focus to smaller, more manageable ticket reports. For example, if you focus on the Active Tickets report (all tickets with activity in the last 2 weeks), which currently has 2,145 tickets, you will rarely see this list get smaller by any meaningful amount. Any action to a ticket on that list will just cause that ticket to persist in that report for an additional two weeks.

Instead, focus on reports that you can “empty”, or at least make progress on. For example, every component in Trac has one report that meets this description: the “Tickets with 0 Replies” reports. These can be found by clicking on the number in the “0 Replies” column on the WordPress Core Components page. Any ticket without a written response (keyword, milestone, etc. updates don’t count) will appear in this list.

Note: these numbers link to a Trac query with a hard coded list of ticket IDs. To see an updated list of tickets after working on these reports, you will need to revisit the Components page for a new link to the updated list. These links are also cached, so sometimes it takes a little time to update.

A screenshot of the WordPress Core Components page at https://make.wordpress.org/core/Components/, which shows the number of open tickets, number of unanswered tickets, and 7 day velocity for each component.
A screenshot of the WordPress Core Components page on the Make WordPress Core blog.

How To Prioritize

Below is a list of reports and ticket lists that I regularly focus on with the frequency that I have been triaging them.

Closed Tickets Needing a Secondary Review (Report 47)

This report requires bug gardening permissions on Trac to triage, but contains tickets that have recently been closed and may need a secondary review.

A ticket can be closed by anyone, but milestones can only be removed by bug gardeners. This allows for secondary review of ticket closures by another contributor, who then needs to remove the milestone. 
If there are any tickets on this report, they need this secondary review from a bug gardener.

Report 47 description on Trac.

It’s not uncommon for newer contributors to mistakenly close tickets when a working patch has been uploaded, or it may not be clear that there is work left to be done. This report usually has no more than a dozen issues (if any at all) at any given time, making it very easy to clear. Clearing this report means that nothing has been mistakenly closed, and everything that still needs addressing remains open.

My Tickets (Report 24)

This report contains all tickets that you’ve accepted, that were assigned to you, or that you’ve reported. I try to visit this report at least once a week. If you spend any time triaging, I recommend this as the first report you should go through. These are tickets you have expressed an interest in working on, have found first hand, or represent ideas or suggestions that you had.

If you are an owner of a ticket and you are unable to continue contributing to (or have lost interest in), you should remove yourself as the ticket’s owner and provide a brief summary of what has accomplished so far, what is remaining, and what the blockers may be. Many contributors will not grab a ticket that has an assigned owner to prevent duplicating work or stepping on someone else’s toes. This clears the path for a different contributor to step up as the new owner.

It’s also really easy to forget tickets you have created or assigned yourself. Triaging this list is what I like to refer to as “Good Open Source Contributor Citizenship” (™?).

Commit Candidates (Report 9)

Report #9 is another one that I try to triage once a week. The commit keyword can be added to a ticket by anyone with bug gardener permissions in Trac, but only contributors granted commit access are allowed to actually make the changes to the official WordPress SVN repository after confirming the appropriateness and readiness of the suggested changes.

When I visit this list, I try to review any ticket that is not owned by another committer, committing if I am comfortable making the changes being suggested. I think of this as calling on the contributors who are raising their hand to say “I have something I think is ready!”

Ancient Tickets (Report 43)

Right now, I visit this report once a week. This report has tickets that have not been modified in the last three years. Tickets on this list have probably fallen through the cracks due to loss of interest or lack of feedback from a committer or component maintainer. Tickets in this list may also be close candidates if the request is outdated or no longer relevant.

Suggesting Close / Needs 2nd Opinion (Report 17)

This report contains all of the tickets with the close or 2nd-opinion keywords. I visit this list once every 3-4 weeks. As long as a reasonable amount of time has passed since the recommendation to close was made, tickets in this report can often be closed with little to no effort if there has been no objection to closing from another contributor (see the Keywords Are Your Friends section below for an explanation).

Reporter Feedback Requested

This is a ticket query that I realized would be great to triage while writing this post. Many times, tickets marked reporter-feedback do not contain enough information for another person to reproduce the reported issue or understand the ticket’s purpose. Without feedback from the person who created the ticket, it can’t be validated or move forward.

In these instances, the ticket should be closed out with a friendly message that it can be reopened if someone (preferably the original reporter) can clarify the ticket’s purpose or intention.

Moving forward, I plan to visit this ticket query every 3-4 weeks.

Unanswered Tickets (Component by Component Basis)

I mentioned this earlier, but one of the best ways to dive in is picking a component you are comfortable with and clearing the “Tickets With No Reply” list. I am a maintainer for the General Component and I decided to start there for a few reasons.

  • It was the component with the highest total number of open tickets (approaching 600).
  • It was the component with the highest number of unanswered tickets (approaching 260).
  • It is the default component selected when a new ticket is created.

Many times newer contributors or bug reporters open a ticket and leave the component as the default (General) because they don’t really know where the ticket belongs or they don’t even know to change that field. This floods the component with tickets that actually belong in other components, or that belong in the WordPress.org Support Forums instead.

Triaging this list has resulted in 74 tickets being closed or moved to the correct component and the unanswered ticket list now sits at 184. This is a big step towards ensuring these tickets receive the attention needed from the contributors best suited for the job.

Because the General component is selected by default, keeping it groomed is essential to preventing tickets from being overlooked.

6 sillhouettes simultaneously jumping in front of a sunrise
Photo by Timon Studler on Unsplash

Keywords Are Your Friends

Effectively using keywords and ensuring their accuracy is one of the most important aspects of triaging. Many contributors monitor reports for specific keywords or keyword combinations. Many components also hold bug scrubs for specific keyword reports.

Sometimes tickets are ignored for months, or even years! Ensuring that tickets have correct keywords can help surface them into different reports which could help them avoid falling through the cracks for long periods of time.

Keywords can also be used to set expectations. For example, adding the close keyword, explaining why you feel the ticket should be closed, and outlining what would need to happen for the ticket to move forward. This helps ticket reporters and other contributors interested in those tickets understand what is required for them to be reconsidered. If those requirements are not met and no component maintainers or committers object in a reasonable amount of time (2-6 weeks, depending on the nature of the ticket), the ticket can be closed with minimal effort when triaging the close keyword report.

Reviving Old Tickets

One thing that I have been pleasantly surprised with is the percentage of contributors that return to tickets that have been dormant for 1, 2, or even more years. Even contributors that have been inactive since posting on that specific ticket have returned to add more details or context to issues on many occasions.

It’s also common that problems have been solved by the reporter, or solved by other unrelated changes to WordPress Core since the ticket was last updated.

A collection of old, worn books on book shelves
Photo by Thomas Kelley on Unsplash

Story Telling Metrics

I’ll close with some metrics that I think illustrate the work that the team has performed over the last 3 months.

All statistics below are for the time frame of March, 11, 2019 to June 11, 2019. Also, anywhere “Triage Team” is used refers to Sergey Biryukov and myself.

To start, I wanted to illustrate how many tickets were created in the 3 month time frame.

1,074 tickets were created in Trac. Of these tickets:

  • 295 were closed as fixed
  • 180 were closed as invalid, worksforme, or wontfix
  • 9 were closed as maybelater
  • 21 were reported-upstream
  • 91 were closed as duplicates.

Note: These tickets may or may not have been closed by the Triage Team. More metrics specific to the team are below.

That leaves 478 opened tickets in the last 3 months. Some other metrics for this time frame (March 11, 2019-June 11, 2019):

Closed Tickets

  • 1,014: Total tickets closed the last three months.
  • 525 (52%): Tickets closed by Triage Team in the last 3 months.

Modified Tickets

  • 2,330: Total tickets modified in last 3 months.
  • 1,736 (75%): Total tickets touched by any committer.
  • 1,296 (56%): Total tickets touched by Triage Team members in the last three months.
  • 503 (22%): Tickets touched by committers in the last 3 months (excluding the Triage Team).
  • 309 (13%): Tickets touched by the Triage Team and another committer.

Conclusion

Joining The Effort

If you’ve made it this far, I would say that this initiative interests you in some way. I recommend reading the summary for the initial team meeting. It’s full of some more basic information about how to get started contributing to this priority for 2019.

Thanks!

Special thanks goes out to my teammate William Earnhardt (@earnjam) for working with me to create some ElasticSearch queries that were run against his TracSearch database to determine the metrics.

If you have been regularly contributing to triaging efforts and I have missed it, please do reach out so your work can be included in these statistics! Also, if you have any interesting ideas for metrics to track, please share!


2 responses to “WordPress Triage Team: A 3 Month Reflection”

  1. Thank you for that, Jonathan.

    I have recently found myself frequently skimming the Core Trac by systematically trying to “fix” tickets, and your article allowed me to consider things from other angles.

    My first concern is that I recently saw on Core Trac several tickets that are not relevant to be there because they address more support issues (I myself created one when I was just starting).
    Couldn’t we find out what path leads a user/contributor to create such a ticket on the Trac? This would allow actions to be taken (producing documentation where there is none, improving existing documents for example) to limit tickets of this kind.

    Also, I recently committed 2 hours a day to contribute to Core. And in general, I spend this time on Core Trac. My methodology is to go through the new tickets in the #core-newtickets channel on Slack, and try to solve them.
    After a few weeks, I noticed that this method is not very effective because I find myself after that time without any concrete results.

    I would like to know if you have any advice on how to optimize this period of time, in order to make my contribution more effective.

    Thank you.

Leave a Reply

Your email address will not be published. Required fields are marked *