Mastering Monitored Rollouts

Learn how to monitor rollouts on Azure for faster, safer releases

Development teams are under pressure to release faster while keeping bugs under control, and often make tradeoffs between agility and risk management. But what if you could increase your release velocity and reduce risk at the same time?

One approach that strikes a balance between rapid deployment and risk mitigation is the concept of monitored rollouts. Monitored rollouts involve gradually releasing new features while continuously tracking their performance and impact at the feature-level. This strategy helps ensure that potential issues are caught early, minimizing disruption to users. Developers can also use the insights to better understand which features need further iterations and which are good to push forward.

Microsoft Azure recently released a robust suite of tools to help facilitate monitored rollouts, allowing you to deploy new features with greater confidence and control. This blog post will focus on new capabilities built into Azure App Configuration that will help you implement and manage monitored rollouts.

What is a monitored rollout?

A monitored rollout is a phased approach to releasing new software features or updates. Instead of deploying changes to all users simultaneously, you release them gradually to a small subset of users first. During this phase, you closely monitor the performance and impact of changes at the feature level. This allows you to detect and address issues before they affect the entire user base.

Would you rather react to customer support tickets and stay up late to release a hotfix, or would you rather detect an issue early and roll back the release to fix it instantly? Releasing features without observing their impact can lead to lower customer satisfaction, reduced product utilization and increased costs as developers triage issues. But in the scenario where you implemented a monitored rollout, you probably fixed the issue before your customers even noticed and are now working on the next great feature.

In short, monitored rollouts enable you to:

• Catch potential problems before your customers notice them.
• Limit exposure by gradually increasing the number of users who see new features.
• Gain valuable insights on how the new feature performs and how users interact with it.

Let’s get started with monitored rollouts on Azure

The best way to get started with monitored rollouts on Azure is to set up an App Configuration to store settings and feature flags. That way, whether you deploy to App Service, Azure Kubernetes Services or Azure Functions, your application settings will be centralized and easy to manage.

App Configuration includes feature management capabilities. Feature flags let you manage and test features at runtime without re-deploying code. Turning features on and off within App Configuration is key to being able to test individual features independently from your deployment process. It also means you can roll features back just as quickly if there are problems or bugs.

You’ll also want to set up Application Insights, which provides deep insights into application performance. Experimentation brings these capabilities together and provides a way to systematically test the impact of feature flags against the data gathered by Application Insights. By leveraging feature flags, enabling Application Insights telemetry, and attaching an experimentation workspace to your App Configuration store, you’ll be able to analyze user interactions and other application data to measure the impact of specific features.

Setting up feature flags:

1. In the “feature manager” blade of Azure App Configuration, create variant feature flags for your application. Your variants can be simple on/off for this exercise.
2. For each feature flag, decide how you want to allocate traffic. Your choice can be based on conditions like user segments, geographic locations, or other criteria.
3. Now you can control the rollout of feature variants to specific users or groups, which allows you to test new features with a subset of users before a full rollout.

Setting up experimentation:

1. Connect an Application Insights resource to your configuration store.
2. Connect an experimentation workspace to your store.
3. Run a percentage rollout with variant flags.
4. Enable telemetry and create metrics for your experiment.
5. Monitor the impact of your features on your tracked metrics.

When Application Insights, feature manager, and experimentation come together, you can: 

• Configure feature flags to target specific user segments or groups for more granular control and feedback.
• Monitor key metrics, such as response times, error rates and user interactions. Set up alerts for anomalies that could indicate issues.
• Collect and analyze user feedback to understand how the new feature is performing from the user’s perspective.
• Based on the data collected, make adjustments and iterate before fully rolling out the feature.
• Gradually increase exposure once you see that the new feature is performing well.
• Continuously monitor the performance of features enabled via flags and make adjustments as needed.
  

Best practices for monitored rollouts

As you integrate experimentation into your DevOps practice, here are some best practices to remember:

1. Define clear metrics
   Determine which metrics are critical for assessing the success of the rollout. This could include performance indicators, error rates, and user engagement metrics. Establishing clear criteria for success helps guide decision-making during the rollout.

2. Start small and scale gradually
   Begin with a small user base or traffic percentage when rolling out a new feature. This approach minimizes the risk of widespread issues and allows you to gather initial feedback.
   
3. Continuously monitor and adapt
   Regularly review performance data and user feedback. Be prepared to make adjustments to the rollout strategy based on real-time insights.
   
4. Communicate with stakeholders
   Keep stakeholders informed about the rollout plan, progress, and any issues encountered. Transparent communication ensures that everyone is aligned and can respond to potential issues effectively.
   
5. Take action
   Be prepared to revert the new feature in case you encounter significant issues. Being able to quickly roll back minimizes disruption and ensures business continuity.
   

Conclusion

Monitored rollouts are a powerful strategy for managing product releases with confidence and control. By leveraging Azure native tools–including Application Insights, feature manager, and experimentation–you can effectively implement gradual deployments, minimize risk, and ensure a smooth transition for your users. Monitored rollouts allow you to gather valuable feedback, make data-driven decisions, and enhance the overall quality of your product releases.