port mapping didn't happen for a container deployed on AWS ECS(uses EC2)

Question

Context:

I am using Circle CI's aws-ecs/deploy-service-update orb to deploy my docker container by pulling the latest image in AWS ECR and deploy it in AWS ECS with AWS EC2 instance. This container is a Machine Learning model that accepts API requests at TCP port 3000(I am using fastAPI for this) and returns the predictions. After I deployed it I couldn't send requests to the public IP of the container instance of the task that deploys the container at port 3000 (This IP is not my EC2 instance's public IP; it only has private IP and public IP is disable).

Debugging

1. I checked my security group and made sure that the port 3000 is open to receive requests from all IPs(0.0.0.0), as part of the inbound rule.
2. I stopped the task(which automatically will stop the container running in the EC2 instance) with the thought that something may have gone wrong from Circle CI. Then, according to the service configuration(1 desired task) and task definition of AWS ECS, a new task has started(hence the container) automatically. But, I couldn't send requests to this either.
3. I SSHed into my EC2 instance to know if the port 3000 is open. This is when is when I learned that ports weren't mapped at all:
   As you can see, PORTS column is empty for the container and the container has to accept requests at port 3000 from the command.

And here are the open ports of the EC2 instance: As you can see, port 3000 is not listed here.

---

Here is the task with port mappings which deployed the container (to AWS ECS) that you see docker ps screenshot above:
In the task definition, you can see the port mappings I have defined for the container.

---

Here is the task running on my EC2 instance with the task-definition shown above and the network mode I am using is 'awsvpc':

---

Here's the "Networking" tab of ENI associated with the task, and also the inbound rule of the security group associated with the EC2 instance that the task is running inside, which accepts requests on port 3000 from all IPs.

EDIT 1:

After I did

docker run -p 3000:3000 <my-image:my-tag>

inside the EC2 machine(by SSHing from my laptop), I could send API requests and receive proper response to the container to it's public IP, of the cluster of AWS ECS. This means that ports are being mapped only when I run the container manually.

I had no problems with ports when I used FARGATE, when I updated the service from Circle CI or even when I manually started tasks.

So, how to automatically map ports when a task is run from AWS ECS service dashboard or from Circle CI? If I run docker container manually, I will not be able to get logs automatically from AWS Cloudwatch and will not be able to stop it from AWS ECS dashboard. Another container by AWS that is running in EC2 instance will take care of those things. It will route the logs to Cloudwatch and accepts stop the existing one and start commands to start a new container with new image stored in AWS ECR, without having to SSH everytime I would want to look at logs or start/stop containers.

What has gone wrong here, which led to ports not being mapped and How do I fix it and map ports properly, so i will be able to send API requests to my container.

Answer 1

Since you are using Network Mode: awsvpc the container will have its own IP address and will not use the IP of the EC2 instance. You can find the container's IP in the running task details.

Alternatively you can use Network Mode: host where the task will share the IP of the EC2 instance and will be accessible on the instance IP port 3000.

Hope that helps :)

Answer 2

After reading a lot of blog pages now I've come to a point that if you want to use privilege ports then you need to change your network mode in AWS to bridge.

Or you can use a Load balancer to map a port.

Hope this answer helps :)