ICALEPCS2023 - List of Authors (Gatsi, T.)

Paper	Title	Page
THPDP047	ELK Stack Deployment with Ansible	1411
	T. Gatsi, X.P. Baloyi, J.L. Lekganyane, R.L. Schwartz SARAO, Cape Town, South Africa
	The 64-dish MeerKAT radio telescope, constructed in South Africa, became the largest and most sensitive radio telescope in the Southern Hemisphere until integrated with the Square Kilometer Array (SKA). Our Control and Monitoring system for Radio Astronomy Project such as MeerKAT produces a lot of data and logs that require proper handling. Viewing and analysis to trace and track system issues and as well as investigate technical software issues require one to go back in time to look for event occurrence. We therefore deployed an ELK software stack ( Elasticsearch, Kibana, Logstash) using Ansible in order to have the capability to aggregate system process logs. We deploy the stack as a cluster comprising lxc containers running inside a Proxmox Virtual Environment using Ansible as a software deployment tool. Each container in the cluster performs cluster duties such as deciding where to place index shards and when to move them. Each container is a data node that makes up the heart of the cluster. We deploy the stack as a cluster for load balancing purposes. Logstash ingests ,transforms and sends the data to the Kibana Graphical User Interface for visualization. Elasticsearch indexes, analyzes, and searches the ingested data into Kibana and our Operations Team and other system users can visualize and analyze these logs on the Kibana GUI frontend.
	Poster THPDP047 [0.503 MB]
DOI •	reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THPDP047
About •	Received ※ 03 October 2023 — Revised ※ 12 October 2023 — Accepted ※ 13 December 2023 — Issued ※ 19 December 2023
Cite •	reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)

Paper

Title

Page

ELK Stack Deployment with Ansible

1411

T. Gatsi, X.P. Baloyi, J.L. Lekganyane, R.L. Schwartz
SARAO, Cape Town, South Africa

The 64-dish MeerKAT radio telescope, constructed in South Africa, became the largest and most sensitive radio telescope in the Southern Hemisphere until integrated with the Square Kilometer Array (SKA). Our Control and Monitoring system for Radio Astronomy Project such as MeerKAT produces a lot of data and logs that require proper handling. Viewing and analysis to trace and track system issues and as well as investigate technical software issues require one to go back in time to look for event occurrence. We therefore deployed an ELK software stack ( Elasticsearch, Kibana, Logstash) using Ansible in order to have the capability to aggregate system process logs. We deploy the stack as a cluster comprising lxc containers running inside a Proxmox Virtual Environment using Ansible as a software deployment tool. Each container in the cluster performs cluster duties such as deciding where to place index shards and when to move them. Each container is a data node that makes up the heart of the cluster. We deploy the stack as a cluster for load balancing purposes. Logstash ingests ,transforms and sends the data to the Kibana Graphical User Interface for visualization. Elasticsearch indexes, analyzes, and searches the ingested data into Kibana and our Operations Team and other system users can visualize and analyze these logs on the Kibana GUI frontend.

Poster THPDP047 [0.503 MB]

DOI •

reference for this paper ※ doi:10.18429/JACoW-ICALEPCS2023-THPDP047

About •

Received ※ 03 October 2023 — Revised ※ 12 October 2023 — Accepted ※ 13 December 2023 — Issued ※ 19 December 2023

Cite •

reference for this paper using ※ BibTeX, ※ LaTeX, ※ Text/Word, ※ RIS, ※ EndNote (xml)