When managing a complex UC environment, a fundamental requirement is an overview of the current state of the complete ecosystem and an effective alerting system that identifies when issues are occurring, or ideally before they occur. This requires a dashboard that provides an end-to-end visual representation and drill-downs to aid fast troubleshooting.
Measuring QoS: MOS or the R-Factor?
The Quality of Service (QoS) of voice and video is usually the most reliable indicator of the health of the entire UC environment and generally appears as a metric on CIO dashboards. It can be measured by Mean Opinion Score (MOS) – a subjective measure using a 1 to 5 scale, where 5 is excellent, or measured by the R-Factor, which uses a scale of 1 to 100, with 100 being excellent. MOS takes into account the CODEC being used as well as network measures such as packet loss, latency and jitter.
From MTTR to MTTI
Mean Time to Repair (MTTR), the speed at which problems can be resolved, is another common metric. However, a commonly held view is that 80% of MTTR is Mean Time to Identify (MTTI). Once the issue has been identified, a resolution is often quick to follow.
Averages Ignore Outliers
Subjective measures like these are a generalization and only encapsulate certain elements of a call. So, despite a high MOS, users may still be experiencing issues such as insufficient sound levels. Scores are also calculated on the mean data, so can ignore outlying issues: they can miss an important issue such as a high percentage of low scores coming from a concentrated group of affected users.
Real-time Monitoring and Alerts
UC performance management tools should monitor QoS in real-time and alert IT Operations to issues as they are occurring.
Testing Inside and Out
Proactive testing of UC environments using agents that generate synthetic calls internally, combined with an external testing service automatically placing real calls into the organization, will identify whether there are issues with incoming and outbound calls.
Tying UC metrics back to the business objectives
The demands of the business on IT Operations has significantly increased in recent years. It's easy to forget that at a minimum, all end users expect voice calls to just work. This was much easier to achieve using dedicated phone lines, but when calls are routed over the same network as all other data, issues are bound to occur. Add to this the network bandwidth demands of video traffic and application sharing and you have significant complexities. Those who take the time to plan and have the foresight to include ongoing monitoring and troubleshooting as a prerequisite to a UC deployment are most likely to have satisfied users within the business. IT Operations leaders need to work closely with the business to negotiate a service-level agreement that balances the criticality of the business with the cost to deliver the service. For example, it might be acceptable to experience the occasional downtime in some areas of the business but not in others, such as the customer contact center. When determining what matters most to the business, IT Operations leaders should look to the strategic objectives set by the CEO and how these relate to the individual business units that IT must serve. Aligning IT metrics to business metrics should not be a ‘set and forget' process, as requirements invariably change over time. And finally, when deciding what metrics to measure, thought should be given to the old saying...
"Not everything that can be counted counts, and not everything that counts can be counted."