Recently, I installed ELK(7.6.2) along with File beat on Amazon Web Service. I will share my experience with details of steps and issues faced during installation.
1) Create a AWS EC2 instance of Amazon Linux AMI (and not Amazon Linux 2 AMI), instance type smaller than this will create space issues and other issues.
4) Set inbound ec2 rules on port 9200(default port of elasticsearch) and 5601 (default port of kibana)
5) Associate elastic IP to this instance.
While restarting instance after elasticsearch and Kibana installation, 1 check out of 2 (Instance Status check: These checks monitor your software and network configuration for this instance.) failed and an instance got terminated automatically. So, my suggestion is to create an instance with minimum configuration as recommended above.
Below series of ec2 instances shows the structure of how the setup was configured.
1) EC2 instance of small size - Java 8, and tomcat8 along with tomcat manager was installed. This was considered a dev server where developers used to create a war file of microservice/Rest end point locally and then deploy a war file on it tomcat manually.
Filebeat was installed so that logs can be sent to elasticsearch server on other EC2 instance.
2) EC2 instance of medium size - Java 8, Apache Maven, Jenkins and tomcat along with tomcat manager was installed. Here deployment used to happen using jenkins job on tomcat server.
Jenkins Job was triggered automatically when developers used to commit code in AWS Code commit and changes were deployed on QA tomcat server.
Filebeat was installed so that logs can be sent to elasticsearch server on other EC2 instance.
3) AWS EC2 instance with size of medium and minimum 16 GB HDD: ElasticSearch and Kibana was installed on this instance.
Steps to install ElasticSearch on EC2 instance:
1) Import elastic search GPG key
#sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
2) Create a file named elasticsearch.repo under /etc/yum.repos.d/ and paste below content and save the file.
[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
Note: Adding 7.x will install the latest version of 7.x. You can mention 6.x to install the older version.
3) Install elasticsearch yum install elasticsearch
4) Goto cd /etc/elasticsearch/ and edit elasticsearch.yml
#vi elasticsearch.yml
cluster.name:
node.data : true
network.host : 0.0.0.0
discovery.seed_hosts : ["localhost", "127.0.0.1", "<elastic-ip of your EC2 instance on which ElasticSearch is installed>"]
5) Run this to start elasticsearch automatically while starting ec2 instance.
# sudo chkconfig --add elasticsearch
6) Start elasticsearch service
# sudo -i service elasticsearch start
7) Run http://<elastic-ip of your EC2 instance on which ElasticSearch is installed>:9200/ and you will get a response.
Please follow my upcoming post for Kibana and Filebeat installation.
.jpg)