HAProxy Monitoring Tool

HAProxy - An Overview

HAProxy (stands for High Availability Proxy) is a popular open source TCP/HTTP Load Balancing software and proxying solution for TCP and HTTP-based applications. It is fast, reliable and particularly suited for very high traffic web sites and powers quite a number of the world's most visited ones.

Monitoring HAProxy - What we do

As HAProxy is most commonly used to improve the performance and reliability of a server environment by distributing the workload across multiple servers (e.g. web, application, database), the best way to ensure proper HAProxy performance and operation is by monitoring its key metrics of the Listener, Frontend, Backend and Server. Applications Manager's HAProxy Monitoring tool provides you with:

  • Frontend metrics such as client connections and requests.
  • Backend metrics such as availability and health of backend servers.
  • Health metrics that reflect the state of your HAProxy setup.

You can correlate frontend metrics with backend metrics for a more comprehensive view of your infrastructure and quickly identify potential hotspots. Applications Manager helps you become aware of performance bottlenecks and take quick remedial actions before your end users experience issues.

Creating a new HAProxy monitor

Prerequisites for monitoring HAProxy metrics : Click here

Rest API to add a new HAProxy monitor : Click here

To add a HAProxy Monitor, follow the steps given below:

  1. Go to New Monitor and click on Add New Monitor link.
  2. Select HAProxy.
  3. Specify the Display Name of the HAProxy monitor.
  4. Enter the HostName or IP Address of the host where stats of the HAProxy runs.
  5. Enter the deatils for authEnabled, Port, Username, password and haproxystatusurl. By default, the value of Port will be 8080.
  6. Enable the SSL option, if SSL is required in HA Proxy.
  7. Specify the Polling Interval in minutes.
  8. Choose the Monitor Group with which you want to associate the HAProxy Monitor (optional).
  9. Click Add Monitor(s). This discovers HAProxy servers from the network and starts monitoring them.
  • Security/Firewall Requirements:The HAProxy host and port should be accessible from the Applications manager installed machine.
  • User Privilege:The required user credentials are to be provided.

Monitored Parameters

Go to the Monitors Category View by clicking the Monitors tab. Click on HAProxy under the Web Server/Services Table. Displayed is the HAProxy bulk configuration view distributed into three tabs:

  • Availability tab displays the Availability history for the past 24 hours or 30 days.
  • Performance tab displays the Health Status and events for the past 24 hours or 30 days.
  • List view enables you to perform bulk admin configurations.

Click on the monitor name to see all the server details listed under the following tabs:

Overview

PARAMETER DESCRIPTION
HAPROXY UTILIZATION DETAILS
HAProxy Utilization Percentage HAProxy's CPU utilization in percentage.
Running Tasks Total number of tasks currenlty running in HAProxy
CONNECTION RATE
Connections per Second HAProxy's client connections per second.
CONNECTION DETAILS
Current Connections The number of current connections.
PIPE DETAILS
Current Pipes The number of current pipes open.

Listener

PARAMETER DESCRIPTION
SESSION UTILIZATION
Utilized Sessions The number of sessions used.
Free Sessions The number of inactive sessions.
TRANSACTION DETAILS
Service Name The name of the listener.
Denied Requests The number of requests denied due to security concerns (ACL-restricted).
Denied Responses The number of responses denied due to security concerns (ACL-restricted).
Request Errors The number of requests whose responses yielded an error.
OVERVIEW
Current Sessions The current number of sessions being used to issue requests.
Utilized Sessions  (%) The percentage of sessions used.
Bytes In  (KB) The number of bytes received by the listener.
Bytes Out  (KB) The number of bytes sent by the listener.
Address The http address of the listener (host:port or "unix". IPV6 has brackets around the address)
Mode Mode of the proxy (tcp, http, etc.)
Status The service status (UP/DOWN/NOLB/MAINT/MAINT(via)/MAINT(resolution)...)
TRANSACTION DETAILS
Denied Requests The number of requests denied due to security concerns (ACL-restricted).
Denied Responses The number of responses denied due to security concerns (ACL-restricted).
Request Errors The number of requests whose responses yielded an error.
Request Denied by Connection The number of requests denied by a connection.
Request Denied by Session The number of requests denied by a session.

Frontend

Frontend metrics provide information about the client’s interaction with the load balancer itself.

PARAMETER DESCRIPTION
SESSION UTILIZATION
Service Name The name of the frontend service.
Utilized Sessions The number of sessions used.
Free Sessions The number of inactive sessions.
TRANSACTION DETAILS
Service Name The name of the frontend service.
Denied Requests The number of requests denied due to security concerns (ACL-restricted).
Denied Responses The number of responses denied due to security concerns (ACL-restricted).
Request Errors The number of requests whose responses yielded an error.
OVERVIEW
Service Name The name of the frontend service.
Current Sessions The current number of sessions being used to issue requests.
Utilized Sessions  (%) The percentage of sessions used.
Session Rate The number of sessions created per second.
Max Session Rate   The maximum number of new sessions per second.
Bytes In  (KB) The number of bytes received by the frontend.
Bytes Out  (KB) The number of bytes sent by the frontend.
Mode Mode of the proxy (tcp, http, etc.)
Status The frontend status (UP/DOWN/NOLB/MAINT/MAINT(via)/MAINT(resolution)...)
TRANSACTION DETAILS
Service Name The name of the frontend service.
Denied Requests The number of requests denied due to security concerns (ACL-restricted).
Denied Responses The number of responses denied due to security concerns (ACL-restricted).
Request Errors The number of requests whose responses yielded an error.
Total Requests The total number of HTTP requests.
Request Denied by Connection The number of requests denied by "tcp-request connection" rules.
Request Denied by Session The number of requests denied by "tcp-request session" rules
Request Rate The rate of HTTP requests per second.
Max Request Rate The maximum number of HTTP requests per second
Connections per Second The number of connections over the last elapsed second.
HTTP RESPONSE COUNT
Service Name The name of the frontend service.
Total HTTP Responses HAProxy exposes the number of responses by HTTP status code - 1xx, 2xx, 3xx, 4xx, 5xx and other codes. Ideally, all responses forwarded by HAProxy would be class 2xx codes.
COMPRESSION DETAILS
Service Name The name of the frontend service.
Bytes into Compressor The number of HTTP response bytes fed to the compressor.
Bytes from Compressor The number of HTTP response bytes emitted by the compressor.
Bytes bypassed Compressor The number of bytes that bypassed the HTTP compressor.
Responses Compressed   The number of HTTP responses that were compressed.

Backend

PARAMETER DESCRIPTION
QUEUE UTILIZATION
Utilized Queue The currently queued requests.
Free Queue The number of free queued requests.
SESSION DETAILS
Utilized Sessions The number of sessions used.
Free Sessions The number of inactive sessions.
OVERVIEW
Service Name The name of the backend service.
Total Weight Servers can be assigned a weight parameter to manipulate how frequently the server is selected.
Total Downtime  (sec) The total downtime in seconds. The value for the backend is the downtime for the whole backend, not the sum of the server downtime.
Downtime Percentage The total downtime in percentage. This is calculated with the uptime of HAProxy.
Active Servers The number of active servers.
Backup Servers The number of backup servers
No. times Server selected The total number of times a server was selected, either for new sessions, or when re-dispatching. The server counter is the number of times that server was selected.
Mode Mode of the proxy (tcp, http, etc.)
Status The frontend status (UP/DOWN/NOLB/MAINT/MAINT(via)/MAINT(resolution)...)
SESSION DETAILS
Service Name The name of the backend service.
Current Queue Size The sum of total queue size of all the backend servers.
Utilized Queue  (%) The percentage of queues used.
Current Sessions The current number of sessions being used to issue requests.
Utilized Sessions  (%) The percentage of sessions used.
Session Rate Session rate is the number of new sessions per second.
Max No.of new Sessions The maximum number of new sessions that can be created per second.
Time Since Last Session Assigned  (sec) The seconds since last session assigned to server/backend.
TRANSACTION DETAILS
Service Name The name of the backend service.
Bytes In  (KB) The Rate of bytes in on backend hosts.
Bytes Out  (KB) The Rate of bytes in on backend hosts.
Denied Requests The requests denied by "tcp-request connection" rules.
Denied Responses The requests denied by "tcp-request session" rules.
Connection Errors The number of requests that encountered an error trying to connect to a backend server.
Response Errors The number of response errors generated by your backends.
RESPONSE SUMMARY
Service Name The name of the backend service.
Total HTTP Responses HAProxy exposes the number of responses by HTTP status code - 1xx, 2xx, 3xx, 4xx, 5xx and other codes. Ideally, all responses forwarded by HAProxy would be class 2xx codes.
Average Session Time (ms) The average time taken for the sessions.
Average Response Time (ms) The sum of average queue, connect and response time.
Connection Retries Multi-process configurations involving source port ranges may cause random failures under high loads (because more than one process may try to use the same source port to connect to the same server). The system will report an error and a retry will happen, picking another port. A high value in the "retries" parameter comes with increased CPU usage and processing time.
Requests Redispatched The number of times a request was redispatched to another server. The server value counts the number of times that server was switched away from.
Client Aborts The number of data transfers aborted by the client.
Server Aborts The number of data transfers aborted by the server.
COMPRESSOR DETAILS
Service Name The name of the backend service.
Bytes into Compressor The number of HTTP response bytes fed to the compressor.
Bytes from Compressor The number of HTTP response bytes emitted by the compressor.
Bytes bypassed Compressor The number of bytes that bypassed the HTTP compressor.
Responses Compressed  The number of HTTP responses that were compressed.

Server

PARAMETER DESCRIPTION
QUEUE UTILIZATION
Utilized Queue The currently queued requests.
Free Queue The number of free queued requests.
SESSION DETAILS
Utilized Session The number of sessions used.
Free Session The number of inactive sessions.
OVERVIEW
Service Name The name of the server.
Server Weight The total server weight. The "weight" parameter is used to adjust the server's weight relative to other servers. All servers will receive a load proportional to their weight relative to the sum of all weights, so the higher the weight, the higher the load.
Total Downtime (sec) The total server downtime (in seconds).
Downtime Percentage The total downtime in percentage. This is calculated with the uptime of HAProxy.
Active Server The number of active servers.
Backup Server The number of backup servers.
No. times Server selected The total number of times a server was selected, either for new sessions, or when re-dispatching.
Address The Server IP address.
Mode Mode of the proxy (tcp, http, etc.)
Status The frontend status (UP/DOWN/NOLB/MAINT/MAINT(via)/MAINT(resolution)...)
SESSION DETAILS
Service Name The name of the server.
Current Queue Size The sum of total queue size of all the servers.
Utilized Queue  (%) The percentage of queues used.
Current Sessions The current number of sessions being used to issue requests.
Utilized Sessions  (%) The percentage of sessions used.
Session Rate Session rate is the number of new sessions per second.
Max No.of new Sessions The maximum number of new sessions that can be created per second.
Time Since Last Session Assigned  (sec) The seconds since last session assigned to server/backend.
TRANSACTION DETAILS
Service Name The name of the server.
Bytes In (KB) The total number of bytes recieved by the server in the last second from the client.
Bytes Out (KB) The total number of bytes sent by the server in the last second.
Denied Responses The number of requests denied due to security restrictions.
Connection Errors The number of requests that encountered an error trying to connect to a server.
Response Errors The number of response errors generated by your server.
Throttle Percentage The throttle percentage for the server, when slowstart is active, or no value if not in slowstart.
RESPONSE SUMMARY
Service Name The name of the server.
Total HTTP Responses HAProxy exposes the number of responses by HTTP status code - 1xx, 2xx, 3xx, 4xx, 5xx and other codes. Ideally, all responses forwarded by HAProxy would be class 2xx codes.
Average Session Time (ms) The average time taken for the sessions.
Average Response Time (ms) The sum of average queue, connect and response time.
Connection Retries Multi-process configurations involving source port ranges may cause random failures under high loads (because more than one process may try to use the same source port to connect to the same server). The system will report an error and a retry will happen, picking another port. A high value in the "retries" parameter comes with increased CPU usage and processing time.
Requests Redispatched The number of times a request was redispatched to another server. The server value counts the number of times that server was switched away from.
Client Aborts The number of data transfers aborted by the client.
Server Aborts The number of data transfers aborted by the server.
HEALTH CHECKS
Service Name The name of the server.
Failed Checks The number of failed checks. (Considers counts checks as failed only when the server is up)
Failed Health Checks The number of failed health checks.
Health Check Response Code The Response Code of the failed health check.
Health Check Content, Health Check Description, Health Check Description, Health Check Duration in ms The failed health checks details.
Agent Check Response Code, Agent Check Content, Agent Check Description, Agent Check Duration in ms The last agent check details.

Configuration

PARAMETER DESCRIPTION
CONFIGURATION DETAILS
Process ID The process id of HAProxy assigned by the system.
Server Start Time The time when the server is last restarted.
Version The version of HAProxy.
Maximum Memory Maximum allowed memory for HAProxy.
Maximum Sockets Maximum number sockets allowed to create.
Maximum Connections The maximum number of connections HAProxy can handle is limited by your configuration and platform
Maximum Pipes Maximum number of pipes allowed to create for tranferring information
Maximum File descriptors Maximum number of file descriptors allowed to open.