note

Overview

Overview

Resources consumption really changes between different clusters and XpoLog. Nowadays, it is possible to define very large machines or multiple small machines for applications deployment – which is preferred?

From our experience, the “rule” is medium-large machines that will balance between performance results and not too many cluster nodes to be managed.
There is no fixed formula as XpoLog is used for a wide variety of tasks. The details below will some important key notes based on our experience.

Generally, for low daily volume of data (<5 GB/Day) a laptop or standard machine is sufficient, but when it comes production deployments, there are a several recommendations to consider.

Supported Operating Systems (standalone installation)

The following operating systems are supported for standalone installation:

Supported Application Servers

The following are supported Application Servers for deployment of XpoLog as an application (requires JAVA 1.8):

Note:  XpoLog is known to run on additional application servers. However, it was tested only on the above servers.

Memory and JAVA

Memory is an important resource - aggregations are memory consumers and many data structures are stored in it. Depends on your daily volume, consider a 32GB or 16 GB machines. Less than 8 GB is less effective and impacts performance (it is not recommended to use machines with more than 64 GB). Java 1.8 is mandatory , and while allocating memory to XpoLog (XpoLog lax file) ensure that the Min (Xms) and Max (Xmx) allocated memory are identical – this will prevent heap size from resizing in runtime.
Always leave memory to the OS that XpoLog runs on:

In any case, do not allocated more than 32GB to XpoLog as crossing it mainly reduces memory management efficiency, CPUs usage and GC handling of large heap size.

Finally, pay attention to Swapping – swapping memory to disk significantly reduces performance. Try to lower swappiness as much as possible - this value controls the OS swapping policy.

Supported Browsers

The following browsers are supported:

Required Ports

The following are the default ports which should be opened:

Hardware Recommendations

CPUs

You should choose a modern processor with multiple cores. Standard machines running XpoLog utilize 4-8 cores.
If you need to choose between faster CPUs or more cores, choose more cores.

Disks

Disks are important for XpoLog - XpoLog processing engines mostly perform write operations but not only, while XpoLog UI engines mostly perform read operations but not only.
XpoLog does many bulk reads/writes and many disk seeks and as disks are usually the slowest subsystem in a server, they easily become the bottleneck of the XpoLog cluster.
If you can afford SSDs, they are by far superior to any spinning media. If SSD is not an option, try to obtain the fastest disks possible (high-performance server disks, 15K RPM).
Network Attached Storage (NAS) is optional but note that NAS is often slower and displays larger latency. RAID 0, 10, 01, 0+1 will give the best performance, while RAID 5 will offer the worst performance.

It is mandatory to perform tests on your set of data and configuration deployment to determine optimized performance since it may vary between different types of data and searches XpoLog performs.

Note: In Linux deployments, it is mandatory to allocate a high number of open files and processes to the user that runs XpoLog - for more information please see Post Installation Recommendations or contact support@xplg.com for more details.

Network

A fast and reliable network is obviously important to performance in a distributed system. Modern data-center networking (1 GbE, 10 GbE) is sufficient for most clusters.
From our experience it is recommended to avoid clusters that are distributed over multiple data centers / geographical locations

General Recommendations Standard

The following is hardware recommendation for up to 3 concurrent users, and < 10 GB of daily logs volume:

Recommended

The following is hardware recommendation for up to 5 concurrent users, and < 10 GB of daily logs volume:

Cluster

The following is hardware recommendation for up to 25 concurrent users, and higher daily logs volume:
(contact support@xplg.com to determine if a cluster is needed. More information about clustering can be found here. See installations details at HA Cluster Installation (multi-machines))