SIEM Solution Part2: Advanced TLS Security Implementation for Elasticsearch and Kibana

Table of Contents

1. Introduction
2. Prerequisites
3. Setting Up Basic Security for Elasticsearch Cluster

• Generating the Certificate Authority
• Generating Node Certificates
• Encrypting Internode Communications with TLS
• Starting Elasticsearch

4. Setting Up Basic Security and HTTPS for Elastic Stack

• Enable TLS for HTTP Layer
• Generate HTTP Certificates
• Configure Elasticsearch for HTTPS
• Configure Kibana for HTTPS

5. Resolving Filebeat Log Parsing and Certificate Issues

6. Conclusion

7. References

Introduction

This document provides detailed instructions for implementing TLS security on Elasticsearch and Kibana within an Elastic Stack environment. It covers the configuration of basic security measures, including TLS encryption for internode communication and HTTP traffic, and resolving common issues related to Filebeat log parsing and certificates.

Prerequisites

• Access to Elasticsearch and Kibana installation directories.
• Basic understanding of TLS and certificate management.
• Administrative privileges on the Elasticsearch and Kibana servers.
• Installed versions of Elasticsearch, Kibana, and Filebeat.

Setting Up Basic Security for Elasticsearch Cluster

When configuring Elasticsearch for the first time, passwords for the ‘elastic’ user are generated automatically, and Transport Layer Security (TLS) is configured by default. For manual security configurations before starting Elasticsearch nodes, TLS settings can be adjusted at any time, including updating node certificates.

Generating the Certificate Authority

To secure your Elasticsearch cluster, ensure encrypted and verified internode communication using mutual TLS. Start by generating a dedicated Certificate Authority (CA) for your cluster with the elasticsearch-certutil tool:

# Go to the
 /usr/share/elasticsearch/bin  

sudo ./elasticsearch-certutil ca

Follow prompts to accept default settings or configure specific options, such as setting a password for production environments. This generates the elastic-stack-ca.p12 file containing your CA’s public certificate and private key.

Generating Node Certificates

On a single node, use the CA generated above to create certificates for each node in your cluster.

sudo ./elasticsearch-certutil cert --ca /usr/share/elasticsearch/elastic-stack-ca.p12

This generates elastic-certificates. p12, containing node certificates, private keys, and the CA certificate. Copy this file to the $ES_PATH_CONF directory on every node if u used a multiple node.

Copy the certificate file to the configuration directory:

sudo cp /usr/share/elasticsearch/elastic-certificates.p12 /etc/elasticsearch/certs

Set appropriate permissions:

sudo chown root:elastic-certificates.p12
sudo chmod 660 /etc/elasticsearch/certs/elastic-certificates.p12

Encrypting Internode Communications with TLS

To secure internal communication between nodes, configure TLS settings in each node’s elasticsearch.yml configuration file.

Adjust verification mode according to your security needs. If passwords were set during certificate creation, store them securely in the Elasticsearch keystore.

Add keystore passwords:

# Keystore

sudo ./elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password

# Truststore

sudo ./elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

Add the following settings:

Starting Elasticsearch

Restart Elasticsearch on each node:

sudo systemctl restart elasticsearch
sudo systemctl status elasticsearch

Ensure all nodes use TLS for transport to maintain secure communication within the cluster.

Check the elastic search that it can run on https.

Setting Up Basic Security and HTTPS for Elastic Stack

Enable TLS for HTTP Layer:

To secure communication between clients and your Elasticsearch cluster, enable TLS on the HTTP layer. Follow these steps to generate and configure the necessary certificates.

Generate HTTP Certificates:

Stop Elasticsearch and Kibana on every node. and run the Elasticsearch HTTP Certificate Tool.

Sudo ./bin/elasticsearch-certutil http

· When asked to generate a CSR, enter n.

· When asked to use an existing CA, enter y and provide the path to elastic-stack-ca.p12.

· Specify the certificate expiration period (e.g., 90D for 90 days).

· When asked to generate one certificate per node, enter y.

· Provide node names, hostnames, and IP addresses for each node.

Unzip the generated elasticsearch-ssl-http.zip file. This file contains directories for both Elasticsearch and Kibana with the necessary certificates and configurations.

sudo unzip /usr/share/elasticsearch/elasticsearch-ssl-http.zip

Configure Elasticsearch for HTTPS

Copy the HTTP certificate:

sudo cp /usr/share/elasticsearch/elasticsearch/http.p12 /etc/elasticsearch/certs/

Set permissions:

sudo chown elasticsearch:elasticsearch /etc/elasticsearch/certs/http.p12
sudo chmod 644 /etc/elasticsearch/certs/http.p12

Edit elasticsearch.yml to enable HTTPS:

Add keystore password:

sudo ./elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password

Start Elasticsearch on each node.

Configure Kibana for HTTPS

Encrypt Traffic Between Kibana and Elasticsearch:

Copy elasticsearch-ca.pem from the unzipped elasticsearch-ssl-http.zip to the $KBN_PATH_CONF directory and also change the permissions.

sudo cp /usr/share/elasticsearch/kibana/elasticsearch-ca.pem /etc/kibana/

Edit kibana.yml:

Encrypt Traffic Between Browser and Kibana

Generate a server certificate for Kibana:

sudo ./elasticsearch-certutil csr -name kibana-server

# unzip it

sudo unzip /usr/share/elasticsearch/csr-bundle.zip

Send kibana-server.csr to a CA for signing and obtain the signed certificate (e.g., kibana-server.crt).

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil cert --pem -ca /usr/share/elasticsearch/elastic-stack-ca.p12 -name kibana-server

sudo unzip /usr/share/elasticsearch/certificate-bundle.zip

Obtain a signed certificate and configure Kibana:

sudo cp /usr/share/elasticsearch/kibana-server/kibana-server.crt /etc/kibana/
sudo cp /usr/share/elasticsearch/kibana-server/kibana-server.key /etc/kibana/

Edit kibana.yml:

Restart Kibana and verify access via HTTPS:

https://<your_kibana_host>.com.

Resolving Filebeat Log Parsing and Certificate Issues

When configuring Filebeat to work with secured HTTPS traffic, you may encounter issues such as the inability to parse logs over HTTPS or errors related to certificates signed by an unknown authority. Follow the steps below to resolve these issues.

Copy and convert the certificate file:

Copy the elastic-certificates.p12 file from your Elasticsearch server to the Filebeat server. Convert the .p12 File to PEM Format: Extract the CA certificate in PEM format.

sudo openssl pkcs12 -in /etc/filebeat/certs/elastic-certificates.p12 -out /etc/filebeat/certs/ca-cert.pem -cacerts -nokeys

Verify the PEM File: Ensure the resulting ca-cert.pem file is correctly formatted.

Update Filebeat configuration:

sudo nano /etc/filebeat/filebeat.yml

Set permissions and restart Filebeat:

sudo chmod 644 /etc/filebeat/certs/ca-cert.pem
sudo chown root:filebeat /etc/filebeat/certs/ca-cert.pem
sudo systemctl restart filebeat
sudo filebeat setup

Check Filebeat logs:

sudo tail -f /var/log/filebeat/filebeat.log

Conclusion

Implementing TLS for both transport and HTTP layers in your Elastic Stack ensures secure communication between nodes and clients. This enhances data protection by encrypting sensitive information, preventing unauthorized access, and meeting compliance standards. For businesses, this setup strengthens internal security, safeguards client data, and builds trust with stakeholders. Adopting TLS is a strategic measure to fortify your organization’s cybersecurity and ensure robust data integrity and confidentiality.

PART 1 Of This Article: https://medium.com/@sobanmalikk/siem-solution-installing-and-configuring-suricata-ids-with-elastic-stack-on-ubuntu-f998c699d042

References

https://www.elastic.co/guide/en/elasticsearch/reference/current/manually-configure-security.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/security-basic-setup.html