A retrospective of the old release management process:

The good, the bad, and the ugly

The release management process at Zoho isn't centralized. Each product team has its own schedule, process, roles, and tools to roll out updates effectively. This makes sense for us because we have over 90 ManageEngine products alone. It wouldn't be possible for one team to oversee all releases.

We have the right tools now but before we established a streamlined release management process, we faced multiple challenges.

Problem 1:

Lack of dedicated tools

When we started out, we weren’t using the dedicated release management workflow tool that we have now. Team members didn’t have a top-level view of each feature’s status. Some members felt the process lacked transparency—and they were right.

When the schedule is made public and updated religiously, it allows team members to plan out merging, testing, and other phases of their current and upcoming releases. Knowing when their input will be required also allows them to plan their day accordingly.

Problem 2:

Lack of knowledge sharing between collaborating teams

Initially, the entire release management process, including testing, was handled by a few members, but tasks weren’t divided across the entire product team. This made it challenging to mobilize releases, and features started to pile up on the stack. Overall, the whole process lacked flexibility.

Problem 3:

Unnecessary downtime for critical resources

Let’s say we’re trying to deploy patches, like security patches or OS patches, onto a mail server. If there are multiple patches, they need to be applied simultaneously to avoid repeated downtime and service disruption in the mail server. We needed a clear strategy from day zero so we could map out the required functionalities for each release and get them done all at once.

Problem 4:

Lack of communication

Feature owners must get final approval from a designated approval board. You do your job, send it to someone for approval, and send it out. Sounds pretty straightforward, right? However, this didn’t work out to be the well-oiled machine we thought it would be. Approval, despite automatic notifications, took an unusually long time, creating a setback in the pipeline of releases.

Upon further assessment, we realized this may be because reviewers aren’t always involved in the entire process; instead, they were called in at the end for their approval. This meant they needed more time to study the stage and understand the release and how it affects users. We decided at this point that, going forward, the best way to speed things up was to involve the approval board at all stages of the release and engage in discussions as needed.

Problem 5:

Non-parallel approvals

Approvals in each stage also took too much time because they weren’t conducted simultaneously. We had to modify that by enabling individual approval processes concurrently. For instance, client review, API review, and migration review can be done simultaneously since they’re done by different people.