Versions Compared

Old Version 10

changes.mady.by.user Omry Koschitzky

Saved on

compared with

New Version 11

changes.mady.by.user Omry Koschitzky

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Tomcat server can be configured to use different type of logging systems, the server has default logging configuration and can be configured to use log4j. Tomcat can also create access logs based on the Access Log Valve.

...

If the Server is using the log4j library for logging please follow the steps documented in adding logs from log4j 1.2 or log4j 2.* 

 

Tomcat Access Logs Configuration

  1. Add Log Data In XpoLog, When adding a log to XpoLog you can now select the Log Type (logtype) for Apache Tomcat Access with the following logtypes:
    1. tomcat
      1. in addition select the log type - access

 

Tomcat Access Logs Configuration

  1. Add Log Data In XpoLog, When adding a log to XpoLog you can now select the Log Type (logtype) for Apache Tomcat Access with the following logtypes:
    1. tomcat
      1. in addition select the log type - access

 

Tomcat access logs are created with the AccessLogValve or with ExtendedAccessLogValve implementation.

For the configuration look into the server server.xml under conf or other webapp configuration files and search for the following:

 

The shorthand pattern pattern="common" corresponds to the Common Log Format defined by '%h %l %u %t "%r" %s %b'.

The shorthand pattern pattern="combined" appends the values of the Referer and User-Agent headers, each in double quotes, to the common pattern.

 

 

 

 

In the Apache Tomcat configuration file, usually server.xml or by default, located under the conf/ directory (Linux "/etc/tomcat/conf") search for the Access Log Valve element in the XML:

"%a %l %u %t \"%r\" %>s %b %D \"%{Referer}i\" \"%{User-Agent}i\"" combined

The following sequence is the log structure definition for the Httpd server %a %l %u %t \"%r\" %>s %b %D \"%{Referer}i\" \"%{User-Agent}i\"

In XpoLog such pattern access logs are created with the AccessLogValve or with ExtendedAccessLogValve implementation.

For the configuration look into the server <TOMCAT-HOME/conf/server.xml> /  (Linux "/etc/tomcat/conf/server.xml")  or other webapp configuration files and search for the following:

<Engine name="Catalina" defaultHost="localhost">

   <Host name="localhost" ...

       <!-- Access log processes all example.

              Documentation at: /docs/config/valve.html
              Note: The pattern used is equivalent to using pattern="common"
       <Valve className="org.apache.catalina.valves.AccessLogValve"
            directory="logs"
            prefix="localhost_access_log."
            suffix=".txt"
            pattern="%h %l %u %t "%r" %s %b"
        />
    </Host>
</Engine>

Thepattern field may defined also as below:

The shorthand pattern pattern="common" corresponds to the Common Log Format defined by '%h %l %u %t "%r" %s %b'.

The shorthand pattern pattern="combined" appends the values of the Referer and User-Agent headers, each in double quotes, to the common pattern.

In XpoLog such pattern (combined) will be translated into:

{ip:Client IP,ftype=remoteip;type=;,} {string:Remote Log Name,ftype=remotelog;,} {string:Remote User,ftype=remoteuser;,} [{date:Date,locale=en,dd/MMM/yyyy:HH:mm:ss z}] "{choice:Method,ftype=reqmethod;,GET;POST} {string:URL,ftype=requrl;,}{block,start,emptiness=true}?{string:Query,ftype=querystring;,}{block,end,emptiness=true} {string:reqprotocol,ftype=reqprotocol;,}" {number:Status,ftype=respstatus;,} {number:Bytes Sent,ftype=bytesent;,} "{string:Referer,ftype=referer;,}" "{string:User Agent,ftype=useragent;,}"{eoe},end,emptiness=true} {string:reqprotocol,ftype=reqprotocol;,}" {number:Status,ftype=respstatus;,} {number:Bytes Sent,ftype=bytesent;,} "{string:Referer,ftype=referer;,}" "{string:User Agent,ftype=useragent;,}"{eoe}


XpoLog Pattern Wizard
When configuring access logs for Tomcat in the XpoLog pattern wizard, paste the pattern directive value into the wizard in order to generate the correct XpoLog pattern for our example you will need to paste: %h %l %u %t "%r" %s %b
Note: If the pattern value is common or combined simply past them into the wizard and XpoLog will build the right pattern as well.

 

 

Apache Tomcat Access Log Format Conversion Table both for AccessLogValve and for ExtendedAccessLogValve

logtyep should be set to: tomcat, access

...

, close to the end of the request

processing. In addition to the formats supported by strftime(3), the following format tokens are supported:

Format StringDescriptionXpoLog PatternXpoLog ftype

%a

Remote IP-address

{ip:RemoteIP,ftype=remoteip}

remoteip

%A

Local IP-address

{ip:LocalIP,ftype=localip}

localip

%B

Size of response in bytes, excluding HTTP headers.

{number:BytesSent,ftype=bytesent}

bytesent

%b

Bytes sent, excluding HTTP headers, or '-' if zero

{text:BytesSent,ftype=bytesent}

bytesent

%{Foobar}C

The contents of cookie Foobar in the request sent to the server. Only version 0 cookies are fully supported.

{string:Cookie_< FOOBAR >}

Replace < FOOBAR > with cookie name

 

%D

The time taken to serve the request, in microseconds.

{number:ResponseTimeMicroSecs,ftype=responsetimemicro}

responsetimemicro

%F

Time taken to commit the response, in millis

{number:ResponseTimeMilliSecs,ftype=responsetimemilli}

responsetimemilli

%h

Remote host name (or IP address if enableLookups for the connector is false)

{text:Remotehost,ftype=remotehost}

 

remotehost

%H

The request protocol

{ text:RequestProtocol,ftype=reqprotocol}

 

reqprotocol

%{Foobar}i

The contents of Foobar: header line(s) in the request sent to the server. Changes made by other modules (e.g. mod_headers)

affect this. If you're interested in what the request header was prior to when most modules would have modified it, use mod_setenvif to copy the header into an

internal environment variable and log that value with the %{VARNAME}e described above.

{text:<FOOBAR>}

https://en.wikipedia.org/wiki/List_of_HTTP_header_fields and so on it goes for the different headers.

 

 

%{Referer}i

Referer

{ text:Referer,ftype=referer}

 

referer

%{User-agent}i

User-agent

{ text:User-agent,ftype=useragent}

 

useragent
%{X-Forwarded-For}iX-Forwarded-For

{text: X-Forwarded-For,ftype=forwardforip} OR

{ip: X-Forwarded-For,ftype=forwardforip}

 

forwardforip

%I

Current request thread name (can compare later with stacktraces)

{text:RequestThread,ftype=thread}

thread

%l

Remote logical username from identd (always returns '-')

{text:logicalname, ftype=logicalname}

logicalname

%m

The request method

{text:RequestMethod,ftype=reqmethod}

reqmethod

%{Foobar}o

write value of outgoing header with name xxx

{string:<FOOBAR>}

 

 

%p

The canonical local port of the server serving the request

{number:ServerPort,ftype=serverport}

serverport

%{format}p

The canonical local port of the server serving the request or the server's actual port or the client's actual port. Valid formats are canonical, local, or remote.

%{canonical}p

%{local}p

%{remote}p

{number:ServerPort,ftype=serverport}

{number:LocalServerPort,ftype=localserverport}

{number:RemotePort,ftype=remoteport}

serverport

localserverportt

remoteport

%q

The query string (prepended with a ? if a query string exists, otherwise an empty string)

{text:QueryString,ftype=querystring}

OR

Suggest a regexp that will build a list of parameters as cloumns.

The query string (prepended with a ? if a query string exists, otherwise an empty string)

querystring

%r

First line of the request (method and request URI)

{text:FirstLine,ftype=reqfirstline}

TBD - might be parsed to multiple value and types}

 

reqfirstline

%s

Status. For requests that got internally redirected, this is the status of the *original* request --- %>s for the last.

{number:ResponseStatus,ftype=respstatus}

. For requests that got internally redirected, this is the status of the *original* request --- %>s for the last.

respstatus
%SUser session ID{text:UserSessionId,ftype=sessionid}sessionid

%t

Time the request was received (standard english format)

{date:Date,locale=en,dd/MMM/yyyy:HH:mm:ss z}

 

 
%{format}t

The time, in the form given by format, which should be in an extended strftime(3) format (potentially localized). If the format starts with begin: (default)

the time is taken at the beginning of the request processing. If it starts with end: it is the time when the log entry gets written, close to the end of the request

processing. In addition to the formats supported by strftime(3), the following format tokens are supported:

sec

sec

number of seconds since the Epoch

msec

number of milliseconds since the Epoch

usec

number of microseconds since the Epoch

msec_frac

millisecond fraction

usec_frac

microsecond fraction

These tokens can not be combined with each other or strftime(3) formatting in the same format string. You can use multiple %{format}t tokens instead.

The extended strftime(3) tokens are available in 2.2.30 and later.

{date:Date,locale=en,dd/MMM/yyyy:HH:mm:ss z}

sec number of seconds since the Epoch
msec

number of milliseconds since the Epoch
usec

number of microseconds since the Epoch
msec_frac

millisecond fraction
usec_frac

microsecond fraction

These tokens can not be combined with each other or strftime(3) formatting in the same format string. You can use multiple %{format}t tokens instead.

The extended strftime(3) tokens are available in 2.2.30 and later.

{date:Date,locale=en,dd/MMM/yyyy:HH:mm:ss z}

sec number of seconds since the Epoch
msec number of milliseconds since the Epoch
usec number of microseconds since the Epoch
msec_frac millisecond fraction
usec_frac microsecond fraction

 

%T

The time taken to serve the request, in seconds.

{number:ResponseTimeSecs,,ftype=processrequestsec}

 

processrequestseci

%u

Remote user that was authenticated (if any), else '-'

{text:User,ftype=remoteuser}

Remote user (from auth; may be bogus if return status (%s) is 401)

remoteuser

%U

The URL path requested, not including any query string.

{text:RequestURL,ftype=requrl}

The URL path requested, not including any query string.

requrl

%v

 Local server name

{text:ServerName,ftype=servername}

servername  

 

 

 

 

 

<Engine name="Catalina" defaultHost="localhost">

 

 

02    ...

 

 

03    <Host name="localhost" ...

 

 

04        <!-- Access log processes all example.

 

 

05               Documentation at: /docs/config/valve.html

 

 

06               Note: The pattern used is equivalent to using pattern="common"

 

 

07        -->

 

 

08        <Valve className="org.apache.catalina.valves.AccessLogValve"

 

 

09            directory="logs"

 

 

10            prefix="localhost_access_log."

 

 

11            suffix=".txt"

 

 

12            pattern="%h %l %u %t "%r" %s %b"

 

 

13        />

 

 

14    </Host>

 

 

15 

 

 

...

 

%T

The time taken to serve the request, in seconds.

{number:ResponseTimeSecs,,ftype=processrequestsec}

 

processrequestseci

%u

Remote user that was authenticated (if any), else '-'

{text:User,ftype=remoteuser}

Remote user (from auth; may be bogus if return status (%s) is 401)

remoteuser

%U

The URL path requested, not including any query string.

{text:RequestURL,ftype=requrl}

The URL path requested, not including any query string.

requrl

%v

 Local server name

{text:ServerName,ftype=servername}

servername
  

 

 

The ExtendedAccessLogValve conversion table below:

Format StringDescriptionXpoLog PatternXpoLog ftype
bytes Bytes sent, excluding HTTP headers, or '-' if zero

{text:BytesSent,ftype=bytesent}

bytesent
c-dnsRemote host name (or IP address if enableLookups for the connector is false)

{ip:RemoteIP,ftype=remoteip}

remoteip
c-ip Remote IP address

{ip:RemoteIP,ftype=remoteip}

remoteip
cs-methodRequest method (GET, POST, etc.)

{text:RequestMethod,ftype=reqmethod}

reqmethod
cs-uriRequest URI

{text:FirstLine,ftype=reqfirstline}

TBD - might be parsed to multiple value and types}

reqfirstline
cs-uri-queryQuery string (prepended with a '?' if it exists)

{text:QueryString,ftype=querystring}

OR

Suggest a regexp that will build a list of parameters as cloumns.

The query string (prepended with a ? if a query string exists, otherwise an empty string)

querystring
cs-uri-stemRequested URL path

{text:RequestURL,ftype=requrl}

The URL path requested, not including any query string.

requrl
date The date in yyyy-mm-dd format for GMT{date:Date,locale=en,yyyy-MM-dd} TBD - time and date in sperate fileds. 
s-dnsLocal host name

{text:ServerName,ftype=servername}

servername
s-ipLocal IP address

{ip:LocalIP,ftype=localip}

localip
sc-statusHTTP status code of the response

{number:ResponseStatus,ftype=respstatus}

. For requests that got internally redirected, this is the status of the *original* request --- %>s for the last.

respstatus
time Time the request was served in HH:mm:ss format for GMT{date:Date,locale=en,HH:mm:ss} TBD - time and date in sperate fileds. 
time-takenTime (in seconds as floating point) taken to serve the request

{number:ResponseTimeSecs,,ftype=processrequestsec}

processrequestseci
x-threadname Current request thread name (can compare later with stacktraces)

{text:RequestThread,ftype=thread}

thread