Articles

SSH Access to AWS EC2: Keys, Security Groups, SSM (2026)

April 6, 2026 9 min read

EC2 instances refuse SSH connections because the instance isn’t reachable, the SSH daemon isn’t running, or the client’s credentials are bad.

Common Causes and Fixes for SSH Connection Refusal

  1. Security Group Blocking SSH Traffic: The most frequent culprit is a misconfigured security group. AWS security groups act as virtual firewalls for your EC2 instances. If the inbound rules don’t explicitly allow SSH traffic (TCP port 22) from your IP address or a trusted network, the connection will be dropped before it even reaches the instance.

    • Diagnosis: Navigate to the EC2 console, select your instance, and check its associated security groups. Then, go to the "Security Groups" section in the EC2 console and examine the inbound rules for the relevant group. Look for a rule allowing TCP traffic on port 22.
    • Fix: Add an inbound rule to the security group:
      • Type: SSH
      • Protocol: TCP
      • Port range: 22
      • Source: My IP (to allow only your current public IP) or a specific CIDR block (e.g., 192.168.1.0/24 for a private network).
    • Why it works: This explicitly permits SSH packets to reach the instance.

  2. Network Access Control List (NACL) Blocking SSH Traffic: While security groups operate at the instance level, NACLs are stateless firewalls that operate at the subnet level. If a NACL associated with the instance’s subnet denies inbound or outbound traffic on port 22, connections will fail.

    • Diagnosis: In the VPC console, find the NACL associated with your instance’s subnet. Check both the inbound and outbound rules. NACLs use numbered rules, and lower numbers are evaluated first. Ensure there isn’t a DENY rule for port 22 that precedes an ALLOW rule.
    • Fix: Add an inbound NACL rule allowing TCP traffic on port 22 from your IP address or network, and an outbound rule allowing ephemeral ports (1024-65535) back to your IP for the return traffic.
      • Inbound rule example: Rule number 100, Type SSH, Protocol TCP, Port range 22, Source Your_IP/32, Allow.
      • Outbound rule example: Rule number 100, Type Custom TCP, Protocol TCP, Port range 1024-65535, Destination Your_IP/32, Allow.
    • Why it works: NACLs control traffic at the subnet border. Allowing traffic through the NACL ensures it can reach the security group.

  3. Incorrect SSH Key Pair: You might be using the wrong private key to authenticate, or the public key on the EC2 instance does not match the private key you possess. When you launch an EC2 instance, you specify an SSH key pair. The public key is placed on the instance, and you use the corresponding private key to connect.

    • Diagnosis: Verify the .pem file (or equivalent) you are using with your ssh command. Check the instance’s launch details in the EC2 console to see which key pair was originally associated with it.
    • Fix: Ensure your ssh command uses the correct private key file using the -i flag: ssh -i /path/to/your-private-key.pem ec2-user@your-instance-public-ip. If you’ve lost the key or used the wrong one, you might need to stop the instance, detach its root volume, attach it to another instance, modify the authorized_keys file for the ec2-user (or relevant user) in /home/ec2-user/.ssh/authorized_keys, and then reattach the volume and restart the original instance.
    • Why it works: SSH authentication relies on a cryptographic challenge-response mechanism between your private key and the public key stored on the server. Using the correct key pair is fundamental.

  4. SSH Daemon Not Running or Misconfigured: The SSH service (sshd) might not be running on the EC2 instance, or it’s configured to listen on a different port or interface.

    • Diagnosis: If you can connect via SSM Session Manager (see below), you can check the status of the SSH daemon: sudo systemctl status sshd or sudo service ssh status. Look for output indicating it’s active and running. Also, check its configuration file: sudo cat /etc/ssh/sshd_config. Ensure Port 22 is uncommented and that ListenAddress is correctly set (or commented out to listen on all interfaces).
    • Fix: If the daemon isn’t running, start it: sudo systemctl start sshd or sudo service ssh start. If it’s not enabled to start on boot: sudo systemctl enable sshd. If the configuration is incorrect, edit /etc/ssh/sshd_config (e.g., change Port 22 back to 22), then reload the service: sudo systemctl reload sshd or sudo service ssh reload.
    • Why it works: The SSH daemon is the server process that listens for and handles incoming SSH connections. It must be running and properly configured to accept connections.

  5. Instance State or Networking Issues: The EC2 instance itself might be in a state where it cannot accept network connections, or there are underlying network problems within AWS.

    • Diagnosis: Check the EC2 instance’s status checks in the AWS console. If either "System Status Checks" or "Instance Status Checks" are failing, there’s a problem with the instance or the underlying AWS infrastructure. Also, verify that the instance has a public IP address assigned if you’re trying to connect from the internet.
    • Fix: For "Instance Status Checks" failures, try rebooting the instance from the EC2 console. For "System Status Checks" failures, you might need to stop and then start the instance (this will assign a new public IP if it was dynamic). If it’s a persistent issue, consider creating a new instance from a snapshot of the problematic one. Ensure the instance is in a public subnet if you expect it to have a public IP and be reachable from the internet.
    • Why it works: Status checks indicate fundamental health. Rebooting or stopping/starting can resolve transient issues or re-establish network connectivity.

  6. IAM Role for SSM Not Attached: If you’re relying on AWS Systems Manager (SSM) Session Manager as a fallback for troubleshooting SSH, the EC2 instance must have an IAM role attached that grants it permissions to communicate with SSM.

    • Diagnosis: In the EC2 console, select your instance. In the "Details" tab, look for the "IAM role" field. If it’s empty or doesn’t have an SSM-related role, this is the issue.
    • Fix: Create an IAM role with the AmazonSSMManagedInstanceCore policy attached. Then, attach this role to your EC2 instance. You can do this via the "Actions" -> "Security" -> "Modify IAM role" menu for the instance.
    • Why it works: The SSM Agent running on the instance needs this role to authenticate with the SSM service and receive commands, allowing you to access the instance even if SSH is broken.

If you’ve fixed all these, the next error you’ll likely encounter is a "Connection timed out" if the instance is still unreachable due to a persistent network issue or a "Permission denied (publickey)" if your SSH key is still incorrect.

Understanding EC2 SSH Access: Keys, Security Groups, and SSM

The most surprising thing about EC2 SSH access is that your SSH key pair is only half the authentication story; the other half is ensuring the network path is clear and the server is listening.

Let’s see how it works with a typical connection flow. Imagine you’re on your local machine, trying to connect to an EC2 instance:

  1. Local Machine: You type ssh -i my-key.pem ec2-user@ec2-public-ip. Your SSH client initiates a TCP connection request to ec2-public-ip on port 22.
  2. AWS Network: The request hits AWS’s network infrastructure.
  3. NACLs: The request first encounters the NACLs associated with the subnet your EC2 instance resides in. If an inbound rule allows TCP port 22 from your IP, it proceeds.
  4. Security Groups: The request then reaches the security group(s) attached to your EC2 instance. If an inbound rule allows TCP port 22 from your IP, it’s permitted.
  5. EC2 Instance: The request arrives at the instance.
  6. SSH Daemon (sshd): The sshd process, listening on port 22, receives the connection request.
  7. Authentication: sshd prompts your client for authentication. Your client presents the my-key.pem (private key). sshd uses the corresponding public key (stored in /home/ec2-user/.ssh/authorized_keys on the instance) to verify your identity.
  8. Connection Established: If authentication succeeds, you get a shell.

The Core Problem Solved: This entire mechanism is designed to provide secure, remote access to your compute instances. The key pair ensures only authorized users can authenticate, while security groups and NACLs control who can even attempt to connect, preventing unauthorized access at the network level.

Internal Workings and Levers:

  • SSH Key Pairs: You generate these locally (ssh-keygen). The public key (.pub) goes onto the EC2 instance, and the private key (.pem) stays with you. AWS handles injecting the public key during instance launch. You can manually add public keys to /home/ec2-user/.ssh/authorized_keys if needed (e.g., after losing access).
  • Security Groups: These are stateful firewalls attached directly to the instance. You define inbound and outbound rules. For SSH, you must have an inbound rule allowing TCP on port 22 from your IP or a trusted range.
  • NACLs: These are stateless firewalls for subnets. They are evaluated before security groups. They require explicit inbound and outbound rules for both the request and the response.
  • EC2 Instance State: The instance must be running and have passed its status checks.
  • SSH Daemon (sshd): This is the server process. It must be installed, running, and configured to listen on the correct port (default 22). Its configuration is in /etc/ssh/sshd_config.

AWS Systems Manager (SSM) Session Manager: This is a crucial tool when SSH breaks. It provides a command-line access to your instance via the SSM Agent, which runs on the instance. It bypasses traditional SSH port 22 and relies on the SSM service endpoint.

  • Setup: Requires an IAM role with AmazonSSMManagedInstanceCore policy attached to the EC2 instance, and the SSM Agent installed and running.
  • Usage: You can start a session from the AWS console or via the AWS CLI (aws ssm start-session --target i-xxxxxxxxxxxxxxxxx).

This allows you to diagnose network issues, check sshd status, or even manually fix SSH configurations without direct SSH access.

The one thing most people don’t realize is how the stateless nature of NACLs forces you to explicitly allow return traffic on ephemeral ports. If your outbound NACL rule for ephemeral ports is too restrictive, your SSH connection might establish but then fail to transmit data reliably, leading to hangs or disconnections. It’s not just about letting the connection in; it’s also about letting the response out on the correct return path.

Once you have reliable SSH access, the next challenge is often managing multiple keys and users securely across many instances.

More Deep Dives in Ssh

Want structured learning?

Take the full Ssh course →