Outline an implementation strategy broken down into (high level) tasks with solution to provide app and log monitoring for apps running in containers in AWS -- 2
₹600-10000 INR
Closed
Posted about 2 years ago
₹600-10000 INR
Paid on delivery
Context:
For the monitoring and troubleshooting of our services and applications, we currently use a
combination of CloudWatch, New Relic, ELK, and CloudTrail. At the moment we have our
applications running on AWS EC2 instances and ECS clusters, both FARGATE and EC2. The
infrastructure is defined using Terraform and the configuration is managed by Puppet. The
applications are written in different programming languages, e.g. PHP, Python, Node.js, and Go.
Some of our applications communicate with each other via event streaming using Apache Kafka.
Problem:
The demands on our monitoring and logging process are increasing in multiple dimensions. Our
applications are being migrated to containers, and new applications are also being containerized,
which creates possibilities to use new stacks to run our applications. Having our application in a
container means every new container and their host need to be monitored and have their logs
properly collected. On busy hours the container count can increase to accommodate the
workload,requiring the new containers to be well configured for monitoring.
An application can be served by multiple containers from different hosts. Having multiple
containers running increases the difficulty to properly monitor or find the application or container
that generated a specific log. Additionally, the containers can be stopped for any reason, making it
more complex to troubleshoot. While cost is not the biggest concern, we need a solution that can
scale and properly accommodate multiple applications running on multiple containers.
Notes:
Please outline an implementation strategy broken down into (high level) tasks. The solution
concept should specifically address the problems stated above, and follow related best practices.
Do not be afraid to make assumptions when needed. Things change in life and in business and we
are more interested in how you approach a problem than if you know every factor going in.
Before jumping to a solution concept, please shortly elaborate on your reasoning for choosing a
particular approach, specifically naming alternate options, your reasons for not choosing those,
along with any other attributes that factored into your final decision.
tech stack that freelancer should already be experienced with are as follows:
1. AWS (Cloudwatch, CloudTrail, EC2, Fargate, ECS, EKS)
2. Terraform
3. Ansible
4. ELK
5. New Relic
6. Jenkins
7. Puppet
8. Kafka
9. programming languages, e.g. PHP, Python, Node.js, and Go.
9th point is not mandatory if freelancer can provide solution after looking at project
Deadline is 7 days