Amazon ELB
Background
Built in Amazon Elastic load balancer dashboards and consoles to gain deep-level insights on your global content delivery network all across your cloud AMIs at once.
The application is aimed to run on ELB access logs stored in one of your S3 buckets. It is required to enable the logging of the Load Balancer to generate the logs and configure the path where log data will be written to).
Steps
- Add Log Data In XpoLog, When adding a log to XpoLog you can now set a Log Type (logtype). For Amazon ELB set the following logtypes:
- AWS
- ELB
- access
- The ELB access logs are placed in a S3 bucket that was configured, usually with the name structure: {string}.log
It is required to configure a S3 account for XpoLog to connect and read the required data from the S3 bucket. - Once all required information is set click next and edit the log pattern, this step is crucial to the accuracy and deployment of the Amazon ELB App. Use the following conversion table to build the XpoLog pattern out of the access log format.
Example
The Amazon ELB access log format is:
date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken x-forwarded-for ssl-protocol ssl-cipher x-edge-response-result-type cs-protocol-version
In XpoLog this pattern will be translated into:
{date:Date,yyyy-MM-dd'T'HH:mm:ss.SSSSSS}Z {text:elb,ftype=elb} {text:client,ftype=remoteip}:{text:port} {text:backend,ftype=target}{block,start,emptiness=true}:{text:port}{block,end,emptiness=true} {text:request_processing_time,ftype=time-taken} {text:backend_processing_time,ftype=processtarget} {text:response_processing_time,ftype=processrequestmilli} {text:elb_status_code,ftype=respstatus} {text:backend_status_code,ftype=targetstatus} {text:received_bytes} {text:sent_bytes,ftype=bytesent} "{choice:Request-Method,ftype=reqmethod;,GET;POST} {text:Request-URI-FULL}{regexp:Request-URI-Subject,ftype=requrl;refName=Request-URI-FULL,[XPLG_PARAM((.*\?[^=]+)|(/[^/]+/[^/]+))].*$} {text:Request-Protocol,ftype=reqprotocol}" "{text:User Agent,ftype=useragent}" {text:ssl-cipher,ftype=ssl-cipher} {text:ssl-protocol,ftype=ssl-protocol}
For more information see below:
Amazon ELB Access Log Format Conversion Table
Format String | Description | XpoLog Pattern | XpoLog ftype |
|---|---|---|---|
timestamp | The time when the load balancer received the request from the client, in ISO 8601 format. For WebSockets, this is the time when the connection is closed. | {date:Date,yyyy-MM-dd HH:mm:ss} | |
elb | The resource ID of the load balancer. If you are parsing access log entries, note that resources IDs can contain forward slashes (/). | {text:elb,ftype=elb} | elb |
client:port | The IP address and port of the requesting client. | {text:client,ftype=remoteip} | remoteip |
backend:port | The IP address and port of the registered instance that processed this request. If the load balancer can't send the request to a registered instance, or if the instance closes the connection before a response can be sent, this value is set to This value can also be set to | {text:backend,ftype=target} | target |
request_processing_time | The total time elapsed, in seconds, from the time the load balancer received the request until the time it sent it to a target. This value is set to | {text:request_processing_time,ftype=time-taken} | time-taken |
backend_processing_time | The total time elapsed, in seconds, from the time the load balancer sent the request to a target until the target started to send the response headers. This value is set to | {text:backend_processing_time,ftype=processtarget} | processtarget |
response_processing_time | The total time elapsed (in seconds) from the time the load balancer received the response header from the target until it started to send the response to the client. This includes both the queuing time at the load balancer and the connection acquisition time from the load balancer to the target. This value is set to | {text:response_processing_time,ftype=processrequestmilli} | processrequestmilli |
elb_status_code | The status code of the response from the load balancer. | {text:elb_status_code,ftype=respstatus} | respstatus |
backend_status_code | The status code of the response from the registered instance. | {text:backend_status_code,ftype=targetstatus} | targetstatus |
received_bytes | The size of the request, in bytes, received from the client (requester). For HTTP requests, this includes the headers. For WebSockets, this is the total number of bytes received from the client on the connection. | {text:received_bytes} | |
sent_bytes | The size of the response, in bytes, sent to the client (requester). For HTTP requests, this includes the headers. For WebSockets, this is the total number of bytes sent to the client on the connection. | {text:sent_bytes,ftype=bytesent} | bytesent |
Request-Method | The request method - ether GET or POST. | {choice:Request-Method,ftype=reqmethod;,GET;POST} | reqmethod |
Request-URI-FULL | The request line from the client enclosed in double quotes and logged using the following formats:Ether HTTP method + protocol://host:port/uri + HTTP version, or the URL is three dashes, each separated by a space, and ending with a space ("- - - "). | {text:Request-URI-FULL} | |
| Request-URI-Subject | Identifies the resource upon which to apply the request. | {regexp:Request-URI-Subject,ftype=requrl;refName=Request-URI-FULL,[XPLG_PARAM((.*\?[^=]+)|(/[^/]+/[^/]+))].*$} | requrl |
| Request-Protocol | The protocol which was used in the request. | {text:Request-Protocol,ftype=reqprotocol} | reqprotocol |
user_agent | A User-Agent string that identifies the client that originated the request. The string consists of one or more product identifiers, product[/version]. If the string is longer than 8 KB, it is truncated. | {text:User Agent,ftype=useragent} | useragent |
ssl_cipher | [HTTPS listener] The SSL cipher. This value is recorded only if the incoming connection was established after a successful negotiation. Otherwise, the value is set to | {text:ssl-cipher,ftype=ssl-cipher} | ssl-cipher |
ssl_protocol | [HTTPS listener] The SSL protocol. This value is recorded only if the incoming connection was established after a successful negotiation. Otherwise, the value is set to | {text:ssl-protocol,ftype=ssl-protocol} | ssl-protocol |