System Architecture - HA cluster

High availability (HA) deployment of XPLG

The most highly available deployment of XPLG is achieved through a multi-node cluster architecture. XPLG's clustering mechanism autonomously and dynamically manages all tasks based on the processes available within the cluster nodes.

A high-availability (HA) clustered deployment offers several key benefits:

1. Scalability: By adding additional processes to the cluster, XPLG can support any required volume of data, ensuring optimal performance even as demand grows.

2. Separation of User Activity and Backend Processing: The system provides a clear separation between user activity and backend data processing. This separation helps maintain high service quality by ensuring that backend processes do not impact the user experience.

3. Multiple Points of Failure: Each node in the cluster is capable of functioning as both an alerting and processing unit in the event of a failure, thus preventing data loss and service disruptions.

4. Fast Disaster Recovery (DR): XPLG incorporates automated procedures to maintain regular configuration backups. These backups can be swiftly used to restore the system in case of failure.

5. High Availability (HA): In the event of a node failure, the cluster manager immediately detects the failure and generates an alert. Until the failed node is restored, its tasks are automatically reassigned to other operational nodes, ensuring continuous service delivery.

6. Fault Tolerance: XPLG ensures seamless recovery in the event of a cluster node failure or even a complete cluster failure. The system can recover promptly and accurately, completing any pending data processing, report generation, or monitoring tasks.

Architecture diagram

The diagram below illustrates a clustered environment with multiple user interface nodes and data processing nodes, showcasing the architecture's design for high availability and fault tolerance. Additionally, the diagram includes representations of on-premise data sources, data digestion processes, and cloud-based data sources, demonstrating how XPLG handles both local and cloud environments in a unified, scalable architecture.

Open

Open

Diagram Annotations

[0] Integration with Active Directory / SSO for authentication and authorization, allowing secure access to the system by users.

[1] Users Traffic Load Balancer - (optional) - In environments with multiple user interface (UI) nodes, the load balancer (LB) distributes users' activities across the nodes to ensure efficient resource utilization and maintain consistent performance.

[2] UI Nodes Array - A set of processes dedicated to handling user activity, ensuring smooth interaction and responsive performance for users.

[3] Storage - A shared file system accessible by all cluster nodes. It stores both hot (frequently accessed) and cold (infrequently accessed) data, enabling efficient data management and retrieval..

[4] Management Nodes Array - These nodes are responsible for cluster management, data processing, and managing listeners.

• M = MASTER: The orchestrating process that manages the cluster.

• P = Processors: Processes responsible for handling tasks such as data collection, indexing, monitoring, reporting, etc.

• L = Listeners: Processes that receive data from various data shippers and external sources.

[5] Listeners Load Balancer - (optional) - In cases where multiple listeners are deployed, the load balancer distributes traffic between nodes, ensuring that data reception is evenly managed and processed.

Clouds Connectivity Architecture
[6] PortX - an XPLG PortX instance/cluster that can run on a virtual machine (VM), as a standalone Docker container, or on managed Kubernetes/OpenShift. PortX centralizes cloud data, providing secure and efficient connectivity to the main cluster, whether it is hosted on another cloud or on-premise.

[7] Cloud Services API - PortX plugins interface with native cloud services to collect data, ensuring seamless integration with cloud-based systems and applications.

[8] Other cloud sources that generate data - Servers (applications/infrastructure), Kubernetes/OpenShift clusters and other cloud components that ships data to PortX in the cloud.

[9] Cloud storage - PortX manages data pipelines, allowing the organization to determine which data should be sent to the main cluster, and which data should remain within the cloud for short- or long-term storage, optimizing management and data efficiency.

Please see installation instructions, our support team is happy to consult/assist when needed, contact us at support@xplg.com