Photo by Dominik Lückmann on Unsplash

Deploy with Docker and AWS Elastic Container Service (ECS)

I’ve worked on this project that needed some deployable back-end infrastructure and I was smart enough to put all the micro-services into Docker containers. Now I thought that having them into containers will make deployment easy, regardless of what Cloud provider I would use. I was right in that assumption but the journey to get my containers up in the cloud was not a smooth ride. This post is about a couple of hiccups that I have encountered, and how I went about solving them. Hopefully, you’ll learn something if you are in the same situation.

So first, I’ve heard about Kubernetes and wanted to be cool as everyone else and use Kubernetes, and namely AWS EKS. After researching the matter a little more I’ve realized that for my project it’s better to use ECS rather than EKS. Below are some of the reasons:

• EKS is more expensive, as aside from the provisioned hardware it's charging for the control plane which is I believe about $150 a month in addition to the hardware
• EKS requires a lot more configuration, the installation of additional 2 CLI tools, and the cluster creation takes a long time. Deletion of the cluster if you need that, even longer.
• ECS is cheaper, faster to configure, and just takes the Docker containers and runs them with an Agent on the machine, regardless if you run on EC2 or Fargate.
• The amount of time spent on setting up clusters and configurations for EKS is superior, and ECS can be up and running in very little time
• If the project grows in size and complexity, the containers could potentially be moved to EKS, given that Kubernetes in AWS still support Docker containers by then :) More here

So while for a larger fleet of services and infrastructure I would have chosen Kubernetes, for my small-mid size project AWS ECS works just fine, and it can be hooked with Code Deploy that ships your container every time you have a change in the Docker container image.

I won’t go into the exact details on how to configure the cluster as I believe AWS has good documentation and also there are a myriad of blog posts out there that cover this in detail. I’ll just describe on a high level what I had to do and some of the issues:

1. Always build your Image using the Dockerfile and as you add more dependencies to the container add them to the Docker file first, not to your service first and then later to your Dockerfile (latter is an antipattern)! This is important for people who are not Docker experts, such as I was before this, not that I am an expert now but I know a few things. If you build your container with docker run, form an image such as Ubuntu or a low-level OS one, and then use the CLI to add OS packages and forget to leverage Dockerfile, it will take you a lot of time to build back the image. So do yourself a favor and always build your images/containers with Dockerfiles. Perhaps that is super obvious for people who’ve done this for a long time but not obvious for people who are not Docker experts yet.
2. Tag and push the Docker Image to the Elastic Container Registry (ECR)
3. Create an ECS Cluster and specify what kind of VMs should run Fargate vs EC2, I’ve chosen EC2 as I felt I can have a little more control, over the VMs, however, Fargate might be as good. Not sure of the deltas.
4. Create a Task specifying the Image and the Cluster that we need to run it
5. Create a Service and associate the Task.

After these 4 steps, ECS will run the service on the machine and will execute the task, which basically behind the scenes will use an agent to run docker run commands.

One issue that I had to face was that my application layer was under HTTPS, and when calling the public instance of the EC2 where I had my API containers running that operation was failing because I did not have certificates running on the EC2 machine. So I had 2 options at those points:

1. Buy a domain and get a certificate
2. Use AWS API Gateway to proxy all the calls to my ECS/EC2 service.

I’ve chosen AWS API Gateway for the following reasons:

1. It’s endpoints already come with valid HTTPS certificates
2. If I want a custom domain I can still do that later
3. API Gateway has already implemented best practices from an API standpoint, like security for example (throttling, DDoS, etc..)
4. If I choose to use different services, such as Lambda functions aside from my Docker microservices, API Gateway can proxy access to all, rather than having to build an orchestration layer. That is debatable from an architecture standpoint, but I see this flexibility as an advantage.
5. The time to configure it was few minutes, as supposed to option 1 which I believe could have taken a lot more.

All in all, if you have Docker containers, using AWS ECS is a reliable way to do it, and it integrates nicely with all the other AWS services. Kubernetes and EKS have all the hype, but if you are not building large scale services and want something that is not overly complex and running in no time, you might want to look into ECS.

The only downside that I see with using ECS vs EKS is that EKS being Kubernetes behind the scenes, has better portability, in that if you want to take your YAML Kubernetes configuration file and go to a different cloud provider, it’s possible without changes in configs. ECS is just specific to AWS, and one would need to write configuration from the ground up if it changes cloud provider. However, if that is not something that you plan to do, ECS would be a feasible solution for your container deployments.

Thank you, for going thus far and I hope this was educational, if not at the very least entertaining and an easy read. As a software engineer, I’ve always benefited from shared online knowledge, and this is my attempt to give back.