Installing ELK Stack (Elasticsearch, Logstash, Kibana) on CentOS 8

ELK Stack is a popular open-source application stack used for log management and data analytics. It consists of three main components:

Elasticsearch

Elasticsearch is a powerful distributed search and analytics engine. It is built on top of Apache Lucene and allows you to store, search, and analyze massive volumes of data in real-time.

Logstash

Logstash is a server-side data processing pipeline that ingests, transforms, and sends data to Elasticsearch or other outputs. It helps you collect, parse, and normalize log data from various sources and enrich it before storing it in Elasticsearch.

Kibana

Kibana is an open-source data visualization and exploration tool. It provides a user-friendly interface for analyzing and visualizing data stored in Elasticsearch. You can create customizable dashboards, charts, and maps to gain insights from your log and metric data.

To install ELK Stack on CentOS 8, follow these steps:

sudo dnf update -y

sudo dnf install -y java-11-openjdk-devel

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

sudo vi /etc/yum.repos.d/elasticsearch.repo

Add the following content to the file:

[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/oss-7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

sudo dnf install -y elasticsearch-oss

sudo vi /etc/elasticsearch/elasticsearch.yml

Uncomment and modify the following settings:

network.host: localhost
discovery.type: single-node

sudo systemctl start elasticsearch
sudo systemctl enable elasticsearch

sudo dnf install -y logstash

sudo vi /etc/logstash/conf.d/logstash.conf

Add the following content to the file:

input {
  file {
    path => "/var/log/application.log"
    start_position => "beginning"
  }
}

filter {
  grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
  }
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
  }
}

sudo systemctl start logstash
sudo systemctl enable logstash

sudo dnf install -y kibana-oss

sudo vi /etc/kibana/kibana.yml

Uncomment and modify the following settings:

server.host: "localhost"
elasticsearch.hosts: ["http://localhost:9200"]

sudo systemctl start kibana
sudo systemctl enable kibana

1. Update system packages:
2. Install Java Development Kit (JDK):
3. Import Elasticsearch GPG key:
4. Create a repository file for Elasticsearch:
5. Install Elasticsearch:
6. Configure Elasticsearch:
7. Start and enable Elasticsearch service:
8. Install Logstash:
9. Configure Logstash:
10. Start and enable Logstash service:
11. Install Kibana:
12. Configure Kibana:
13. Start and enable Kibana service:

After completing these steps, you should have ELK Stack installed and ready to use on your CentOS 8 server. You can access Kibana by opening your web browser and navigating to http://localhost:5601.

From the Kibana interface, you can configure index patterns, create visualizations, and build dashboards to analyze and visualize your log data stored in Elasticsearch.