Monitoring of Servers with ManageEngine^TM Applications Manager

Introduction to On Demand TV - Japan

On Demand TV Inc, is a broadband service provider in Japan, providing Video on Demand services and distributing image contents over B-FLET'S, high-speed Internet connections via optical fiber of NTT EAST CORPORATION and NTT WEST CORPORATION. It enables each subscriber to enjoy high-quality multi-channel broadcasting at home. With the catch phrase "WHAT YOU LIKE, WHEN YOU LIKE, and HOW MUCH YOU LIKE", the Company is steadily getting more subscribers and has started the first "On Demand TV High-Vision" service in Japan since July, 2006.

Construction of Systems and Monitoring

With a view to provide 24-hour 365-day services throughout Japan, the Company has two computer centers in Tokyo and in Osaka respectively. The Company has decided to adopt widely-used standard Operating Systems and hardware for accumulating operational know-how of systems as quickly as possible.

(1) Monitoring of systems

• Monitoring the system resources or traffic by using commercial or free monitoring tools
• Manual inspection by regular patrol throughout a computer center

(2) Monitoring of broadcasting

• Monitoring the stability of broadcast by receiving IP-based streams

(3) Monitoring of services

• Monitoring the operation of each service on the side of service provider by using a free monitoring tool
• Monitoring the operation of services by vendor tools

Yasuyuki Taniguchi, person in charge of operations management of systems in Engineering Division of the Company reminisces “We thought it would be sufficient to consider to that extent at the time of starting operation”.

Difficulty in Understanding the Operational State of all Systems

After the start of the service, the number of subscribers were increasing considerably. They, however, encountered lot of problems such as an occurrence of a failure at the specified hour under certain circumstances as follows;

1. The system receives no request from any subscriber and response delays the DB listener process
2. Although no notification of failure reaches to the monitoring system, an internal error is given to a subscriber.
3. Fault occurs at a specific time as requests from subscribers increase without knowing the root cause.

There he had to tackle work relating to monitoring application servers, database servers and collecting information reported to vendors. He found it difficult to understand whether or not all the functions of commercial applications are in operation

Taniguchi retrospectively says, "With the monitoring tools of those days we could collect only trend information of memory usage or CPU loads for each node, but it was very difficult to detect the fall in system performance, say, CPU usage. So, we had to check logs, capture the communication between STB (Set Top Box) and application servers very frequently. It led to lot of heavy manual work".

Applications Manager Monitors Systems and Collects Data in an Integrated Manner

During the pursuit of an application monitoring tool suitable for the Company, Taniguchi came across ManageEngine Applications Manager which features the monitoring functions of various DB servers, application servers, or HTTP servers in an integrated manner including Oracle DB, Oracle AS, Apache. It had graphical displays of the operational state of all applications. He immediately installed Applications Manager and commenced new monitoring.

The results were the following;

1. It became possible to properly monitor the availability of Oracle DB and Oracle AS.
2. It became easy to collect information on performance of Oracle DB and Oracle AS. Also, the root cause analysis report on faults went to vendors quickly.
3. It became easy to understand the trend of changes relating to the number of requests processed or the average processing time because it was possible to refer to the history of changes.
4. It became easy to collect information on performance of Apache server and it was possible to detect any symptom of failure much faster than ever.

As stated above, the need for manual monitoring was reduced. It enabled them to proactively detect and investigate failure with the help of Applications Manager.

Failure in Streaming Process

[Stage1] The Company need was to check whether or not each subscriber really has a valid license while receiving a request for streaming contents. During the streaming process it would sometime fail to issue a key necessary for decrypting encrypted contents.

Investigation is necessary to address this failure because this process was done by the interaction among servers including the license server. So Taniguchi with the help of Applications Manager monitored the number of requests processed by Oracle As 10g (which had the application that retrieves license information on the license server), average processing time, and the number of busy servers of Apache.

Monitoring those data helped to decide whether the problem was with the performance of the license server or with the applications on the license server or with the key issuance server.

[Stage2] As a result, it was possible to limit the scope of the problem within the application on the license server. It was, however, necessary to analyze the log information for further investigation. By capturing the timing of the occurrence of failure, it would easier to pinpoint the problem. alarms could be generated based on the threshold values of the response time.

Role of Applications Manager from the User Point of View

The contents distribution service in which the Company engages in must be sensitive to the response from subscribers or clients. As an engineer in charge of constructing systems, Taniguchi harbors great expectations on ManageEngine Applications Manager. “Our Company is steadily increasing the number of subscribers by engaging in distribution of high-quality video of rich contents and also by providing various service menus. Hence there would be more systems to monitor”. He continues “It is preferable to support more various needs from users, such as, a trend analysis function of memory usage for each process”

We at ManageEngine will endeavor to respond to various demands of customers and continue to develop such products to realize stable management of systems.