Back in 2006, organizations were still predominantly using network uptime measurements to evaluate how well they were doing when delivering business critical data to end-users. The basic fact that the computer network is up or down doesn't really tell organizations if their IT teams are able to ensure seamless delivery of applications to end-users. Organizations are now seeking not only to reduce time needed to resolve performance issues as they happen, but also to take actions to prevent these problems before they disrupt key business processes.

For these reasons, organizations are increasingly moving away from a network-centric approach and are striving to measure the level of application performance as measured by the end-users.

1. The need for end user monitoring Solutions

1. Monitoring the business impact of managing application performance

Traditionally, CIOs, IT Directors, IT Managers, and network architects had been types of job roles that were being tasked with ensuring seamless delivery of IT and business services. However, line of business owners and other business executives are becoming increasingly interested in learning about how delivery of business critical data impacts their organizations' bottom lines. It should be noted that business executives are not interested in performance metrics that show how IT infrastructure is performing, but rather they are focused on translating performance indicators such as application availability and time between failures into business metrics such as customer satisfaction, employee productivity, cost savings, and brand recognition.

In order for organizations to enable their business executives to achieve their goals, they need to monitor performance of their infrastructure not only from the perspective of their IT departments, but from the business perspective as well. This goal can only be achieved if quality of delivering IT services is being measured from the perspective of the users of these services.

2. Changes in end-user expectations

The market for application performance management technologies is changing and becoming more complex. One of the key drivers of changes in this market is that end-user expectations for the quality of these services are increasing.

As residential data communication services improve in terms of speed and availability, end-users are expecting the same levels of application performance for data that they access at work as they experience for data that they use for their personal purposes. At the same time, organizations are deploying new applications that use more complex infrastructure and are becoming more critical for business. This makes it more difficult for IT departments to support the delivery of applications to end-users and to ensure that problems with application performance do not cause disruption of key business processes.

3. Monitoring application uptime is no longer enough to ensure seamless delivery of business services

Organizations are increasingly realizing the importance of monitoring and improving performance of business-critical applications. However, their effectiveness in executing on these strategies heavily depends on the approach that they are taking when evaluating application performance.

Many organizations still rely on application uptime as a key metric for evaluating application performance. Even though this metric allows IT departments to measure availability of business services that are being delivered to end-users, it does not allow them to measure actual quality of IT services that business users are experiencing. For that reason, organizations are increasingly using performance metrics such as application and transaction response times and end-user experience indexes that allow them to monitor the speed of applications and to evaluate the quality of the end-user experience.

2. End-user Monitoring Technologies

As organizations become increasingly interested in measuring application performance from the perspective of end-users, they are faced with an additional challenge: what end-user monitoring techniques are the best fit for their needs? The end-user monitoring market is still not well defined, and it includes several classes of technology that were designed to address the same problem.

1. Monitoring end-user experience from inside and outside of the corporate firewall

From a technology perspective, there are two approaches to monitoring the quality of delivery of business services to end-users: monitoring the internal application delivery infrastructure (servers, networks, databases, application infrastructure) and monitoring application performance from outside of the corporate firewall. Typically, organizations choose to focus resources on monitoring an internal infrastructure to accomplish their goals for application performance management, but organizations in certain industry sectors depend heavily on the performance of Web applications that are being accessed by customers and prospects. These organizations need to understand how their revenue generating applications accessed over the public Internet are performing from the perspective of their customers and prospects regardless of their geographical location. In order to achieve that goal, they are deploying application performance management solutions that are not monitoring internal infrastructure, but performance of different parts of application delivery chain that are being used by customers when accessing these applications.

2. Passive vs. Active monitoring

End-user technologies typically fall into two distinct categories:

• Solutions for active monitoring, also known as "synthetic" monitoring
• Solutions for passive monitoring, also known as "real-user experience" monitoring

Technologies for active end-user experience monitoring are based on generating simulated transactions for measuring application and transaction response times. These solutions periodically send requests across the network, so they can measure the response time of application servers and network infrastructure; they collect this data to understand levels of performance that end-users are experiencing. These agents are very light, and they do not cause any additional burden on network traffic. Solutions for passive end-user experience measurement monitor actual transactions and reporting application response times from a server, network, or application perspective.

Both of these methodologies have their advantages and help organizations with measuring application performance. However, solutions for synthetic monitoring are more effective in addressing the top goal for application performance management: prevent performance issues before end-users are impacted. These solutions allow organizations to pro-actively test parts of their infrastructure and identify potential performance bottlenecks before they cause disruption of business processes.

3. Other end-user monitoring techniques

In addition to these end-user monitoring techniques, organizations can choose from technologies such as:

Client-based solutions for monitoring performance of internally used applications:

These technologies are based on software clients that are installed on end-user devices and collect information about application speed and availability and store this data into centralized repositories. These types of solutions allow organizations to gain visibility into the quality of end-user experience per user, user group, and network location. For example, these clients can be deployed on multiple geographically dispersed devices to monitor the quality of the end-user experience and the impact that carrier networks, operating systems, and browsers are having on application performance. Additionally, organizations can use these solutions for monitoring applications internally, where these solutions allow them to get better insight into end-user device performance by collecting metrics such as CPU usage, response times, and business process metrics.

Solutions for monitoring end-user usage patterns: These solutions do not necessarily monitor application performance, but they allow organizations to capture information about how end-users are using business critical applications and, consequently, enable IT and business managers to understand a correlation between application performance and the level of usage of these applications.

3. Capabilities Needed

1. Monitoring application response times for each transaction

Top-performing organizations are increasingly moving from an application-specific approach for managing the performance of business critical applications to the approach that is based on monitoring performance for each transaction. This approach allows them to identify each transaction that is causing issues with customer satisfaction, employee productivity, revenue growth, or brand recognition. More importantly, by using this approach organizations are more likely to identify true levels of application performance from the users' perspective as opposed to monitoring performance in aggregate.

2. Monitoring business metrics related to application performance

Organizations are adopting metrics that go beyond traditional IT measurements when evaluating application performance. Additionally, they are expanding the traditional set of stakeholders to include business executives and line-of-business owners. In order to serve the needs of these individuals, organizations need to develop dashboards that will allow business executives to translate the quality of end-user experience into business metrics such as employee productivity, cost of managing enterprise infrastructure, and customer satisfaction.

3. Monitor the impact on end-user experience across infrastructure tiers

Organizations are looking not only to improve their ability to identify issues with the quality of the end-user experience, but also to resolve these issues in timely manner. In order to be able to identify a root cause for application performance issues, end-user organizations need to develop capabilities that allow them to understand what infrastructure tiers are causing these performance problems. More importantly, these organizations need to be able to evaluate the impact of each of the infrastructure tiers based on the quality of the end-user experience.

4. Ability to prevent application performance issues before business users are impacted

Being proactive regarding managing the quality of the end-user experience for enterprise applications is no longer optional for many organizations. These organizations are concerned with reducing time to troubleshoot and mean time to repair performance problems, but more importantly, they all want to prevent issues with the quality of the end-user experience before business users are impacted. Inability to prevent problems with the quality of the end-user experience could lead to revenue loss and declines in responsiveness to customer needs and employee productivity. Additionally, inability to achieve this goal could create additional workloads for IT staff, which could lead to these employees constantly dealing with performance problems instead of spending their time on creating new and innovative IT services. In order to achieve this goal, organizations need to develop capabilities that will allow them to improve visibility into the entire transaction flow, from the data center to the end-user. This would also allow them to make educated decisions about parts of their infrastructure that need to be optimized in order to ensure that they can deliver IT services in a seamless fashion.

5. Capabilities for generating synthetic transactions for measuring end-user experience

In order to be able to prevent performance problems before they cause disruption of business processes organizations need to ensure that they can improve visibility into their IT infrastructure and be able to identify potential problems before end-users start calling the help-desk. The ability to generate synthetic transactions allows organizations to improve visibility into their infrastructure and to be more proactive when executing their strategies for ensuring the optimal level of quality of the end-user experience. Having this capability in place also enables organizations to make better decisions when planning changes to the enterprise infrastructure, and it allows them to eliminate performance bottlenecks before new IT services are rolled out.

4. ManageEngine's Capabilities

ManageEngine Applications Manager is one of the few technology vendors in the application performance market that offers capabilities for measuring the end-user experience from both inside and outside of the firewall. It's approach is based on synthetic (agent-based) monitoring, which enables organizations to generate simulated transactions and monitor response times for these transactions. In addition to capabilities for simply monitoring the end-user experience and alerting IT teams when a problem occurs, Applications Manager provides robust capabilities for troubleshooting and resolving performance issues. The combination of proactive monitoring of the end-user experience and the ability to drill down to different infrastructure tiers to identify the root cause of the problem allows organizations to significantly improve their success rates in preventing end-user experience problems and reduce the time needed to repair performance problems when they do happen.