Tomcat server can be configured to use different type of logging systems, the server has default logging configuration and can be configured to use log4j. Tomcat can also create access logs based on the Access Log Valve.
Tagging
All Tomcat/Catalina logs are tagged by logtype - tomcat
In addition there are the following log types that must be assigned for the Tomcat App to be deployed on:
- Catalina out log or Console out log will be tagged by logtype - out
- Catalina Servlet or webapp log will be tagged by logtype - servlet
- Access logs will be tagged by logtype - access
Defualt logging configuration can be found under the conf direcotry (tomcat/conf) or *nix /etc/tomcat../conf/ logging.properties file. Usually the access log will be defined in server.xml file under the conf directory.
Default Log structure and configuration
For log type out and servlet, those logs can be name by default catalina.out, and will be located under the logs directory. Use the following XpoLog pattern for those logs
Example 1:
{date:Date,dd-MMM-yyyy HH:mm:ss.SSS} {priority:Priority,ftype=severity,ALL;FINEST;FINER;FINE;INFO;CONFIG;WARNING;SEVERE;ERROR} [{text:thread,ftype=thread}] {text:Source,ftype=source} {string:Message,ftype=message}
Custom Logging
If the Tomcat server is configured to use external logging with log4j or other java.util framework than use XpoLog pattern wizard and defenition to configure the log pattern correctly for the app to work.
Make sure that if you are using log4j wizard you will need to setup the log sources and manually apply the Tomcat/Catalina tags on them for the App to work correctly.
References
Tomcat 7
https://tomcat.apache.org/tomcat-7.0-doc/logging.html
Access Logs: https://tomcat.apache.org/tomcat-7.0-doc/config/valve.html#Access_Logging
Tomcat 8
Logging: https://tomcat.apache.org/tomcat-8.0-doc/logging.html
Access Logs: https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html#Access_Logging
Tomcat 9
https://tomcat.apache.org/tomcat-9.0-doc/logging.html
Access Logs: https://tomcat.apache.org/tomcat-9.0-doc/config/valve.html#Access_Logging
Log4J
If the Server is using the log4j library for logging please follow the steps documented in adding logs from log4j 1.2 or log4j 2.*
Tomcat Access Logs Configuration
- Add Log Data In XpoLog, When adding a log to XpoLog you can now select the Log Type (logtype) for Apache Tomcat Access with the following logtypes:
- tomcat
- in addition select the log type - access
- tomcat
Tomcat access logs are created with the AccessLogValve or with ExtendedAccessLogValve implementation.
For the configuration look into the server <TOMCAT-HOME/conf/server.xml> / (Linux "/etc/tomcat/conf/server.xml") or other webapp configuration files and search for the following:
<Engine name="Catalina" defaultHost="localhost">
<Host name="localhost" ...
<!-- Access log processes all example.
Documentation at: /docs/config/valve.html
Note: The pattern used is equivalent to using pattern="common"
<Valve className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
prefix="localhost_access_log."
suffix=".txt"
pattern="%h %l %u %t "%r" %s %b"
/>
</Host>
</Engine>
The pattern field may defined also as below:
The shorthand pattern pattern="common"
corresponds to the Common Log Format defined by %h %l %u %t "%r" %s %b
The shorthand pattern pattern="combined"
appends the values of the Referer
and User-Agent
headers, each in double quotes, to the common
pattern.
In XpoLog such pattern (combined) will be translated into:
{ip:Client IP,ftype=remoteip;type=;,} {string:Remote Log Name,ftype=remotelog;,} {string:Remote User,ftype=remoteuser;,} [{date:Date,dd/MMM/yyyy:HH:mm:ss z}] "{choice:Method,ftype=reqmethod;,GET;POST} {string:URL,ftype=requrl;,}{block,start,emptiness=true}?{string:Query,ftype=querystring;,}{block,end,emptiness=true} {string:reqprotocol,ftype=reqprotocol;,}" {number:Status,ftype=respstatus;,} {number:Bytes Sent,ftype=bytesent;,} "{string:RefererQuery,ftype=refererquery;,}{regexp:Referer,ftype=referer;refName=RefererQuery,^([\w-]+://[^?]+|/[^?]+)}" "{string:User Agent,ftype=useragent;,}"{eoe}
In XpoLog such pattern (common) will be translated into:
{ip:Client IP,ftype=remoteip;type=;,} {string:Remote Log Name,ftype=remotelog;,} {string:Remote User,ftype=remoteuser;,} [{date:Date,dd/MMM/yyyy:HH:mm:ss z}] "{choice:Method,ftype=reqmethod;,GET;POST} {string:URL,ftype=requrl;,}{block,start,emptiness=true}?{string:Query,ftype=querystring;,}{block,end,emptiness=true} {string:reqprotocol,ftype=reqprotocol;,}" {number:Status,ftype=respstatus;,} {number:Bytes Sent,ftype=bytesent;,}{eoe}
Apache Tomcat Access Log Format Conversion Table both for AccessLogValve and for ExtendedAccessLogValve
logtyep should be set to: tomcat, access
Format String | Description | XpoLog Pattern | XpoLog ftype | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Remote IP-address | {geoip:Client IP,ftype=remoteip;type=;,} | remoteip | ||||||||||
| Local IP-address | {ip:LocalIP,ftype=localip} | localip | ||||||||||
| Size of response in bytes, excluding HTTP headers. | {number:BytesSent,ftype=bytesent} | bytesent | ||||||||||
| Bytes sent, excluding HTTP headers, or '-' if zero | {text:BytesSent,ftype=bytesent} | bytesent | ||||||||||
| The contents of cookie Foobar in the request sent to the server. Only version 0 cookies are fully supported. | {string:Cookie_< FOOBAR >} Replace < FOOBAR > with cookie name | |||||||||||
| The time taken to serve the request, in microseconds. | {number:ResponseTimeMicroSecs,ftype=responsetimemicro} | responsetimemicro | ||||||||||
%F | Time taken to commit the response, in millis | {number:ResponseTimeMilliSecs,ftype=responsetimemilli} | responsetimemilli | ||||||||||
| Remote host name (or IP address if enableLookups for the connector is false) | {text:Remotehost,ftype=remotehost}
| remotehost | ||||||||||
| The request protocol | {string:reqprotocol,ftype=reqprotocol;,}
| reqprotocol | ||||||||||
| The contents of Foobar affect this. If you're interested in what the request header was prior to when most modules would have modified it, use mod_setenvif to copy the header into an internal environment variable and log that value with the | {text:<FOOBAR>} https://en.wikipedia.org/wiki/List_of_HTTP_header_fields and so on it goes for the different headers.
| |||||||||||
| Referer | {string:RefererQuery,ftype=refererquery;,}{regexp:Referer,ftype=referer;refName=RefererQuery,^([\w-]+://[^?]+|/[^?]+)}
| referer | ||||||||||
| User-agent | {string:User Agent,ftype=useragent;,}
| useragent | ||||||||||
%{ X-Forwarded-For}i | X-Forwarded-For | {text: X-Forwarded-For,ftype=forwardforip} OR {ip: X-Forwarded-For,ftype=forwardforip}
| forwardforip | ||||||||||
| Current request thread name (can compare later with stacktraces) | {text:RequestThread,ftype=thread} | thread | ||||||||||
| Remote logical username from identd (always returns '-') | {text:logicalname, ftype=logicalname} | logicalname | ||||||||||
| The request method | {text:RequestMethod,ftype=reqmethod} | reqmethod | ||||||||||
| write value of outgoing header with name | {string:<FOOBAR>}
| |||||||||||
| The canonical local port of the server serving the request | {number:ServerPort,ftype=serverport} | serverport | ||||||||||
| The canonical local port of the server serving the request or the server's actual port or the client's actual port. Valid formats are
| {number:ServerPort,ftype=serverport} {number:LocalServerPort,ftype=localserverport} {number:RemotePort,ftype=remoteport} | serverport localserverportt remoteport | ||||||||||
| The query string (prepended with a | {text:QueryString,ftype=querystring} OR Suggest a regexp that will build a list of parameters as cloumns. The query string (prepended with a | querystring | ||||||||||
| First line of the request (method and request URI) | {text:FirstLine,ftype=reqfirstline} TBD - might be parsed to multiple value and types}
| reqfirstline | ||||||||||
| Status. For requests that got internally redirected, this is the status of the *original* request --- | {number:ResponseStatus,ftype=respstatus} . For requests that got internally redirected, this is the status of the *original* request --- %>s for the last. | respstatus | ||||||||||
%S | User session ID | {text:UserSessionId,ftype=sessionid} | sessionid | ||||||||||
| Time the request was received (standard english format) | {date:Date,locale=en,dd/MMM/yyyy:HH:mm:ss z}
| |||||||||||
%{ format}t | The time, in the form given by format, which should be in an extended the time is taken at the beginning of the request processing. If it starts with processing. In addition to the formats supported by
These tokens can not be combined with each other or The extended | {date:Date,locale=en,dd/MMM/yyyy:HH:mm:ss z} sec number of seconds since the Epoch | |||||||||||
| The time taken to serve the request, in seconds. | {number:ResponseTimeSecs,ftype=processrequestsec}
| processrequestseci | ||||||||||
| Remote user that was authenticated (if any), else '-' | {text:User,ftype=remoteuser} Remote user (from auth; may be bogus if return status ( | remoteuser | ||||||||||
| The URL path requested, not including any query string. | {text:RequestURL,ftype=requrl} The URL path requested, not including any query string. | requrl | ||||||||||
| Local server name | {text:ServerName,ftype=servername} | servername | ||||||||||
|
The ExtendedAccessLogValve conversion table below:
Format String | Description | XpoLog Pattern | XpoLog ftype |
---|---|---|---|
bytes | Bytes sent, excluding HTTP headers, or '-' if zero | {text:BytesSent,ftype=bytesent} | bytesent |
c-dns | Remote host name (or IP address if enableLookups for the connector is false) | {ip:RemoteIP,ftype=remoteip} | remoteip |
c-ip | Remote IP address | {ip:RemoteIP,ftype=remoteip} | remoteip |
cs-method | Request method (GET, POST, etc.) | {text:RequestMethod,ftype=reqmethod} | reqmethod |
cs-uri | Request URI | {text:FirstLine,ftype=reqfirstline} TBD - might be parsed to multiple value and types} | reqfirstline |
cs-uri-query | Query string (prepended with a '?' if it exists) | {text:QueryString,ftype=querystring} OR Suggest a regexp that will build a list of parameters as cloumns. The query string (prepended with a | querystring |
cs-uri-stem | Requested URL path | {text:RequestURL,ftype=requrl} The URL path requested, not including any query string. | requrl |
date | The date in yyyy-mm-dd format for GMT | {date:Date,locale=en,yyyy-MM-dd} TBD - time and date in sperate fileds. | |
s-dns | Local host name | {text:ServerName,ftype=servername} | servername |
s-ip | Local IP address | {ip:LocalIP,ftype=localip} | localip |
sc-status | HTTP status code of the response | {number:ResponseStatus,ftype=respstatus} . For requests that got internally redirected, this is the status of the *original* request --- %>s for the last. | respstatus |
time | Time the request was served in HH:mm:ss format for GMT | {date:Date,locale=en,HH:mm:ss} TBD - time and date in sperate fileds. | |
time-taken | Time (in seconds as floating point) taken to serve the request | {number:ResponseTimeSecs,,ftype=processrequestsec} | processrequestseci |
x-threadname | Current request thread name (can compare later with stacktraces) | {text:RequestThread,ftype=thread} | thread |