com.atlassian.confluence.content.render.xhtml.migration.exceptions.UnknownMacroMigrationException: The macro 'html' is unknown.

Amazon ELB

Owned by Omry Koschitzky
Dec 20, 2017
3 min read

Background

Built in Amazon Elastic load balancer dashboards and consoles to gain deep-level insights on your global content delivery network all across your cloud AMIs at once.

The application is aimed to run on ELB access logs stored in one of your S3 buckets. It is required to enable the logging of the Load Balancer to generate the logs and configure the path where log data will be written to). 

Steps

  1. Add Log Data In XpoLog, When adding a log to XpoLog you can now set a Log Type (logtype). For Amazon ELB set the following logtypes:
    1. AWS
    2. ELB
    3. access
  2. The ELB access logs are placed in a S3 bucket that was configured, usually with the name structure: {string}.log
    It is required to configure a S3 account for XpoLog to connect and read the required data from the S3 bucket.
  3. Once all required information is set click next and edit the log pattern, this step is crucial to the accuracy and deployment of the Amazon ELB App. Use the following conversion table to build the XpoLog pattern out of the access log format.

 

Example

The Amazon ELB access log format is: 

date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken x-forwarded-for ssl-protocol ssl-cipher x-edge-response-result-type cs-protocol-version

 

In XpoLog this pattern will be translated into:

{date:Date,yyyy-MM-dd'T'HH:mm:ss.SSSSSS}Z {text:elb,ftype=elb} {text:client,ftype=remoteip}:{text:port} {text:backend,ftype=target}{block,start,emptiness=true}:{text:port}{block,end,emptiness=true} {text:request_processing_time,ftype=time-taken} {text:backend_processing_time,ftype=processtarget} {text:response_processing_time,ftype=processrequestmilli} {text:elb_status_code,ftype=respstatus} {text:backend_status_code,ftype=targetstatus} {text:received_bytes} {text:sent_bytes,ftype=bytesent} "{choice:Request-Method,ftype=reqmethod;,GET;POST} {text:Request-URI-FULL}{regexp:Request-URI-Subject,ftype=requrl;refName=Request-URI-FULL,[XPLG_PARAM((.*\?[^=]+)|(/[^/]+/[^/]+))].*$} {text:Request-Protocol,ftype=reqprotocol}" "{text:User Agent,ftype=useragent}" {text:ssl-cipher,ftype=ssl-cipher} {text:ssl-protocol,ftype=ssl-protocol}

 

For more information see below:

Amazon ELB Access Log Format Conversion Table

 

Format String

DescriptionXpoLog PatternXpoLog ftype

timestamp

The time when the load balancer received the request from the client, in ISO 8601 format. For WebSockets, this is the time when the connection is closed.

{date:Date,yyyy-MM-dd HH:mm:ss}

 

elb

The resource ID of the load balancer. If you are parsing access log entries, note that resources IDs can contain forward slashes (/).

{text:elb,ftype=elb}

elb

client:port

The IP address and port of the requesting client.

{text:client,ftype=remoteip}

remoteip

backend:port

The IP address and port of the registered instance that processed this request.

If the load balancer can't send the request to a registered instance, or if the instance closes the connection before a response can be sent, this value is set to -.

This value can also be set to - if the registered instance does not respond before the idle timeout.

{text:backend,ftype=target}

target

request_processing_time

The total time elapsed, in seconds, from the time the load balancer received the request until the time it sent it to a target.

This value is set to -1 if the load balancer can't dispatch the request to a target. This can happen if the target closes the connection before the idle timeout or if the client sends a malformed request.

{text:request_processing_time,ftype=time-taken} 

time-taken

backend_processing_time

The total time elapsed, in seconds, from the time the load balancer sent the request to a target until the target started to send the response headers.

This value is set to -1 if the load balancer can't dispatch the request to a target. This can happen if the target closes the connection before the idle timeout or if the client sends a malformed request.

{text:backend_processing_time,ftype=processtarget}

processtarget

response_processing_time

The total time elapsed (in seconds) from the time the load balancer received the response header from the target until it started to send the response to the client. This includes both the queuing time at the load balancer and the connection acquisition time from the load balancer to the target.

This value is set to -1 if the load balancer can't send the request to a target. This can happen if the target closes the connection before the idle timeout or if the client sends a malformed request.

{text:response_processing_time,ftype=processrequestmilli}

processrequestmilli

elb_status_code

The status code of the response from the load balancer.

 {text:elb_status_code,ftype=respstatus}

respstatus

backend_status_code

The status code of the response from the registered instance.

{text:backend_status_code,ftype=targetstatus}

targetstatus

received_bytes

The size of the request, in bytes, received from the client (requester). For HTTP requests, this includes the headers. For WebSockets, this is the total number of bytes received from the client on the connection.

{text:received_bytes}

 

sent_bytes

The size of the response, in bytes, sent to the client (requester). For HTTP requests, this includes the headers. For WebSockets, this is the total number of bytes sent to the client on the connection.

 {text:sent_bytes,ftype=bytesent}

bytesent

Request-Method

The request method - ether GET or POST.

{choice:Request-Method,ftype=reqmethod;,GET;POST}reqmethod

Request-URI-FULL

The request line from the client enclosed in double quotes and logged using the following formats:Ether HTTP method + protocol://host:port/uri + HTTP version, or the URL is three dashes, each separated by a space, and ending with a space ("- - - ").

{text:Request-URI-FULL} 
Request-URI-SubjectIdentifies the resource upon which to apply the request.{regexp:Request-URI-Subject,ftype=requrl;refName=Request-URI-FULL,[XPLG_PARAM((.*\?[^=]+)|(/[^/]+/[^/]+))].*$} requrl
Request-ProtocolThe protocol which was used in the request.{text:Request-Protocol,ftype=reqprotocol}reqprotocol

user_agent

A User-Agent string that identifies the client that originated the request. The string consists of one or more product identifiers, product[/version]. If the string is longer than 8 KB, it is truncated.

{text:User Agent,ftype=useragent}

useragent

ssl_cipher

[HTTPS listener] The SSL cipher. This value is recorded only if the incoming connection was established after a successful negotiation. Otherwise, the value is set to -.

{text:ssl-cipher,ftype=ssl-cipher}

ssl-cipher

ssl_protocol

[HTTPS listener] The SSL protocol. This value is recorded only if the incoming connection was established after a successful negotiation. Otherwise, the value is set to -.

{text:ssl-protocol,ftype=ssl-protocol}

ssl-protocol