As cloud infrastructure becomes the backbone of modern enterprises, ensuring the security of these environments is paramount. With AWS (Amazon Web Services) still being the dominant cloud it is important for any security professional to know where to look for signs of compromise. AWS CloudTrail stands out as an essential tool for tracking and logging API activity, providing a comprehensive record of actions taken within an AWS account. Think of AWS CloudTrail like an audit or event log for all of the API calls made in your AWS account. For security professionals, monitoring these logs is critical, particularly when it comes to detecting potential unauthorized access, such as through stolen API keys. These techniques and many others I've learned through the incidents I've worked in AWS and that we built into SANS FOR509, Enterprise Cloud Forensics.
1. Unusual API Calls and Access Patterns
A. Sudden Spike in API Requests
One of the first signs of a potential security breach is an unexpected increase in API requests. CloudTrail logs every API call made within your AWS account, including who made the call, when it was made, and from where. An attacker with stolen API keys might initiate a large number of requests in a short time frame, either probing the account for information or attempting to exploit certain services.
What to Look For:
- A sudden, uncharacteristic surge in API activity.
- API calls from unusual IP addresses, particularly from regions where legitimate users do not operate.
- Access attempts to a wide variety of services, especially if they are not typically used by your organization.
Note that Guard Duty (if enabled) will automatically flag these kinds of events, but you have to be watching to find them.
B. Unauthorized Use of Root Account
AWS strongly recommends avoiding the use of the root account for day-to-day operations due to its high level of privileges. Any access to the root account, especially if API keys associated with it are being used, is a significant red flag.
What to Look For:
- API calls made with root account credentials, especially if the root account is not typically used.
- Changes to account-level settings, such as modifying billing information or account configurations.
2. Anomalous IAM Activity
A. Suspicious Creation of Access Keys
Attackers may create new access keys to establish persistent access to the compromised account. Monitoring CloudTrail logs for the creation of new access keys is crucial, especially if these keys are created for accounts that typically do not require them.
What to Look For:
- Creation of new access keys for IAM users, particularly those who have not needed them before.
- Immediate use of newly created access keys, which could indicate an attacker is testing or utilizing these keys.
- API calls related to `CreateAccessKey`, `ListAccessKeys`, and `UpdateAccessKey`.
C. Role Assumption Patterns
AWS allows users to assume roles, granting them temporary credentials for specific tasks. Monitoring for unusual role assumption patterns is vital, as an attacker might assume roles to pivot within the environment.
What to Look For:
- Unusual or frequent `AssumeRole` API calls, especially to roles with elevated privileges.
- Role assumptions from IP addresses or regions not typically associated with your legitimate users.
- Role assumptions that are followed by actions inconsistent with normal business operations.
3. Anomalous Data Access and Movement
A. Unusual S3 Bucket Access
Amazon S3 is often a target for attackers, given that it can store vast amounts of potentially sensitive data. Monitoring CloudTrail for unusual access to S3 buckets is essential in detecting compromised API keys.
What to Look For:
- API calls related to `ListBuckets`, `GetObject`, or `PutObject` for buckets that do not typically see such activity.
- Large-scale data downloads or uploads to and from S3 buckets, especially if occurring outside of normal business hours.
- Access attempts to buckets that store sensitive data, such as backups or confidential files.
B. Data Exfiltration Attempts
An attacker may attempt to move data out of your AWS environment. CloudTrail logs can help detect such exfiltration attempts, especially if the data transfer patterns are unusual.
What to Look For:
- Large data transfers from services like S3, RDS (Relational Database Service), or DynamoDB, especially to external or unknown IP addresses.
- API calls related to services like AWS DataSync or S3 Transfer Acceleration that are not typically used in your environment.
- Attempts to create or modify data replication configurations, such as those involving S3 cross-region replication.
4. Unexpected Security Group Modifications
Security groups control inbound and outbound traffic to AWS resources. An attacker might modify these settings to open up additional attack vectors, such as enabling SSH access from external IP addresses.
What to Look For:
- Changes to security group rules that allow inbound traffic from IP addresses outside your trusted network.
- API calls related to `AuthorizeSecurityGroupIngress` or `RevokeSecurityGroupEgress` that do not align with normal operations.
- Creation of new security groups with overly permissive rules, such as allowing all inbound traffic on common ports.
5. Steps for Mitigating the Risk of Stolen API Keys
A. Enforce the Principle of Least Privilege
To minimize the damage an attacker can do with stolen API keys, enforce the principle of least privilege across your AWS account. Ensure that IAM users and roles only have the permissions necessary to perform their tasks.
B. Implement Multi-Factor Authentication (MFA)
Require MFA for all IAM users, particularly those with administrative privileges. This adds an additional layer of security, making it more difficult for attackers to gain access, even if they have stolen API keys.
C. Regularly Rotate and Audit Access Keys
Regularly rotate access keys and ensure that they are tied to IAM users who actually need them. Additionally, audit the use of access keys to ensure they are not being abused or used from unexpected locations.
D. Enable and Monitor CloudTrail and GuardDuty
Ensure that CloudTrail is enabled in all regions and that logs are centralized for analysis. Additionally, AWS GuardDuty can provide real-time monitoring for malicious activity, offering another layer of protection against compromised credentials. Consider AWS Detective to have some intelligence built on top of the findings.
E. Use AWS Config for Compliance Monitoring
AWS Config can be used to monitor compliance with security best practices, including the proper use of IAM policies and security groups. This tool can help identify misconfigurations that might leave your account vulnerable to attack.
Conclusion
The security of your AWS environment hinges on vigilant monitoring and quick detection of anomalies within CloudTrail logs. By understanding the typical patterns of legitimate usage and being alert to deviations from these patterns, security professionals can detect and respond to potential compromises, such as those involving stolen API keys, before they cause significant damage. As cloud environments continue to evolve, maintaining a proactive stance on security is essential to protecting sensitive data and ensuring the integrity of your AWS infrastructure. If you want to learn more about what to look for in AWS for signs of intrusion, along with Microsoft and Google clouds you might consider my class FOR509 running at SANS Cyber Defense Initiative 2024. Visit for509.com to learn more.
