ECS IAM roles trip up almost every startup engineer the first time. You set up your ECS service, deploy your container, and then it can't read from S3 or publish to SQS. You go hunting for the error and find Access Denied with no obvious fix.
The problem is almost always the same: you either didn't set a task role, or you put permissions on the wrong role. There are two IAM roles in every ECS task definition, and they do completely different things.
Here's how they actually work โ no Terraform required.
The Two Roles, In Plain English
Every ECS task has two roles:
Execution Role โ used by ECS before your container starts. Pulling the Docker image from ECR, fetching secrets from Secrets Manager, writing startup logs to CloudWatch. Your application code never touches this role. Think of it as ECS's backstage pass.
Task Role โ used by your application while it's running. If your Node.js app calls S3, your Python service reads from SQS, or your Go binary queries DynamoDB โ that's the task role doing the work. This is the one most people forget to set.
A simple mental model: execution role is "ECS launching your container," task role is "your container doing its job."
You need both. You cannot combine them into one role (well, you can, but don't โ it's a security anti-pattern that'll come back to bite you).
How Your App Gets AWS Credentials (Without Any Config)
Here's the part most guides skip: when you set a task role, your application picks up credentials automatically. You don't set environment variables, you don't hardcode keys, you don't do anything special.
The ECS agent exposes a credentials endpoint inside each container at:
http://169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
The AWS SDK's default credential chain checks this endpoint automatically. Every AWS SDK โ Python, Node.js, Go, Java, Ruby โ supports this out of the box.
# Python โ SDK finds the task role automatically
import boto3
s3 = boto3.client('s3', region_name='us-east-1')
response = s3.get_object(Bucket='my-app-bucket', Key='config.json')
// Node.js โ same deal
const { S3Client, GetObjectCommand } = require('@aws-sdk/client-s3');
const s3 = new S3Client({ region: 'us-east-1' });
const result = await s3.send(new GetObjectCommand({
Bucket: 'my-app-bucket',
Key: 'config.json'
}));
No credentials to rotate. No secrets to manage. The credentials are temporary (rotated automatically by AWS) and scoped to exactly the permissions on the task role.
This is one of the real advantages of running on AWS vs Railway โ on Railway, you'd have to inject AWS credentials as environment variables and manage rotation yourself.
Setting Up the Execution Role
If you're using NoahOps, the execution role is created and managed for you. If you're wiring this up manually, here's what it needs.
The execution role requires a trust policy that lets ECS assume it:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ecs-tasks.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
And it needs the AWS managed policy AmazonECSTaskExecutionRolePolicy, which covers ECR pulls and CloudWatch log writes. If you're using Secrets Manager to inject secrets into your containers (you should be), add this too:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"kms:Decrypt"
],
"Resource": [
"arn:aws:secretsmanager:us-east-1:YOUR_ACCOUNT_ID:secret:my-app/*"
]
}
]
}
Note the scoped ARN โ don't use "Resource": "*" here. Scope to the specific secrets your tasks need.
Setting Up the Task Role
The task role uses the same trust policy (ecs-tasks.amazonaws.com as the principal). What changes is the permissions โ these should be specific to what your application actually does.
A typical API service that reads from S3 and queues jobs:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ReadAppBucket",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::my-app-bucket",
"arn:aws:s3:::my-app-bucket/*"
]
},
{
"Sid": "QueueJobs",
"Effect": "Allow",
"Action": [
"sqs:SendMessage",
"sqs:GetQueueUrl"
],
"Resource": "arn:aws:sqs:us-east-1:YOUR_ACCOUNT_ID:job-queue"
}
]
}
A background worker that processes those jobs:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ConsumeJobs",
"Effect": "Allow",
"Action": [
"sqs:ReceiveMessage",
"sqs:DeleteMessage",
"sqs:GetQueueAttributes"
],
"Resource": "arn:aws:sqs:us-east-1:YOUR_ACCOUNT_ID:job-queue"
},
{
"Sid": "WriteProcessedFiles",
"Effect": "Allow",
"Action": [
"s3:PutObject"
],
"Resource": "arn:aws:s3:::my-app-bucket/processed/*"
}
]
}
Notice the worker role and the API role are different. The API can read from S3 and enqueue, but it can't dequeue or write to the processed folder. The worker can dequeue and write processed files, but it can't send new messages or read unprocessed objects. This is least privilege โ each service only has what it needs.
Wiring It Into Your Task Definition
In your ECS task definition, you reference both roles by ARN:
{
"family": "api-service",
"taskRoleArn": "arn:aws:iam::YOUR_ACCOUNT_ID:role/api-task-role",
"executionRoleArn": "arn:aws:iam::YOUR_ACCOUNT_ID:role/ecsTaskExecutionRole",
"networkMode": "awsvpc",
"requiresCompatibilities": ["FARGATE"],
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{
"name": "api",
"image": "YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/api:latest",
"essential": true,
"portMappings": [
{ "containerPort": 3000, "protocol": "tcp" }
],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "/ecs/api-service",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix": "api"
}
}
}
]
}
taskRoleArn โ your app's runtime permissions.
executionRoleArn โ ECS's startup permissions.
If you omit taskRoleArn, your application runs with no AWS credentials. Any SDK call that hits AWS will return Access Denied.
Debugging "Access Denied" Errors
When your container logs show Access Denied, this is the sequence to follow:
Step 1: Find out which role the running task is actually using.
aws ecs describe-tasks \
--cluster your-cluster-name \
--tasks YOUR_TASK_ID \
--query 'tasks[0].taskArn'
Then look up the task definition it's using and check what taskRoleArn is set to.
Step 2: List what permissions that role has.
aws iam list-role-policies --role-name your-task-role-name
aws iam list-attached-role-policies --role-name your-task-role-name
Step 3: Simulate the permission before you change anything.
The IAM policy simulator is underused. Use it:
aws iam simulate-principal-policy \
--policy-source-arn arn:aws:iam::YOUR_ACCOUNT_ID:role/your-task-role \
--action-names s3:GetObject \
--resource-arns arn:aws:s3:::my-bucket/some-key.json
It'll tell you allowed or implicitDeny/explicitDeny โ and if denied, which statement caused it. Much faster than guessing and redeploying.
Step 4: Check that the credentials endpoint is reachable.
If you have ECS Exec enabled (more on that below), you can shell into a running container and confirm:
curl $AWS_CONTAINER_CREDENTIALS_FULL_URI
If this returns a JSON blob with AccessKeyId, SecretAccessKey, and Token, the task role is being picked up. If it returns an error, the task role isn't attached or the endpoint isn't reachable.
The ECS Exec Permission Gotcha
ECS Exec lets you shell into a running container for debugging โ it's the equivalent of kubectl exec. To enable it, your task role needs an extra permission that most guides don't mention:
{
"Sid": "EnableECSExec",
"Effect": "Allow",
"Action": [
"ssmmessages:CreateControlChannel",
"ssmmessages:CreateDataChannel",
"ssmmessages:OpenControlChannel",
"ssmmessages:OpenDataChannel"
],
"Resource": "*"
}
This goes on the task role (not the execution role). Without it, aws ecs execute-command hangs or returns an error even if the service has enableExecuteCommand: true set.
Once it's in place:
aws ecs execute-command \
--cluster your-cluster \
--task YOUR_TASK_ID \
--container api \
--interactive \
--command "/bin/sh"
This is genuinely useful when you're chasing down Access Denied errors โ you can run aws s3 ls directly from inside the container to confirm which permissions the task role actually has.
Giving Each Service Its Own Role
If you have three ECS services โ API, worker, scheduler โ give each one a distinct task role. This isn't just security hygiene; it's operationally useful. When you add a new S3 bucket or SQS queue, you update the role for the one service that needs it, not a shared role that silently grants access to services that shouldn't have it.
The pattern:
api-task-roleโ read S3, send SQSworker-task-roleโ receive/delete SQS, write S3scheduler-task-roleโ put events to EventBridge, read parameter store
Each one is scoped tightly. If the worker service gets compromised, it can only do what a worker is supposed to do.
How NoahOps Handles This
When you deploy a service with NoahOps, we create and manage both the execution role and a service-specific task role automatically. As you add integrations โ RDS credentials via Secrets Manager, S3 buckets, SQS queues โ Noah AI updates the task role with the right permissions and reregisters the task definition.
You describe what your service needs in plain English. Noah handles the trust policies, the scoped ARNs, and the task definition update. No IAM console hunting, no "why is it still Access Denied" debugging after you thought you fixed it.
Request a free demo at noahops.com
FAQ
Do I need a task role if I'm not calling any AWS services?
Technically no. But you should still create one scoped to ssmmessages:* permissions so you can use ECS Exec for debugging when you need it.
Can I use the same execution role for all my ECS services?
Yes, and this is common. The execution role is about ECS plumbing, not application-level access. Most teams share a single ecsTaskExecutionRole across all services and customize task roles per service.
Why do I get Access Denied even after attaching the right policy?
Three common reasons: (1) You attached the policy to the execution role instead of the task role. (2) The ARN in the policy doesn't match the actual resource ARN โ even one character difference causes a silent deny. (3) There's an SCP (Service Control Policy) at the AWS Organization level blocking the action. Check all three before chasing a code issue.
Does this work the same on EC2 launch type vs Fargate?
Yes โ the task role works the same either way. The difference is that on EC2 launch type, if you don't set a task role, the container may fall back to the EC2 instance role (which is shared by all containers on that host). On Fargate, there's no instance role fallback โ no task role means no credentials.
How do I rotate credentials if they're provided by the task role?
You don't. AWS rotates them automatically. Task role credentials are temporary and expire roughly every 6 hours. The ECS credentials endpoint handles renewal transparently. This is the whole point โ no long-lived keys to manage.