Start preparing for your Next Exam | Use coupon – TOGETHER | Avail 30% discount

CloudTrail Logs

  1. Home
  3. AWS Certified Security Specialty
  5. CloudTrail Logs
  • Monitor existing system, application and custom logs in real time.
  • Send existing logs to CloudWatch; Create patterns to look for in logs; Alert based on finding of these patterns.
  • Free agents for Ubuntu, Amazon Linux, Windows.
  • Purpose
    • Monitor logs from EC2 instances in realtime. (track number of errors in application logs and send notification if exceed thresold)
    • Monitor AWS CloudTrail logged events (API Activity such as manual EC2 instance termination)
    • Archive log data (change log retention setting to automatically delete)
  • Log events – record stored to CloudWatch Logs with the Timestamp and Message to store.
  • Log Streams – Refers to the log events sequence sharing same resource (like for Apache access logs, they are automatically deleted after every 2 months).
  • Log Groups – Refer to log stream group sharing same settings for
    • Retention
    • monitoring
    • access control
  • CMetric Filters – define how a service would extract metric observations from events and turn them into data points for a CloudWatch metric.
  • Retention Settings – Settings for duration to keep events. Automatic deletion of expired logs.
  • The duration offered for Log Group Retention ranges from 1 day to 10 years.
  • CloudWatch Log Filters: filter log data pushed to CloudWatch; won’t work on existing log data, only work after log filter created, only returns
  • first 50 results. Metric contains 1. Filter Pattern 2. Metric Name 3. Metric NameSpace 4. Metric value
  • Modify rsyslog (/etc/rsyslog.d/50-default.conf) and remove auth on line number 9, sudo service rsyslog restart
  • Real-Time Log processing: It needs subscription Filters and applicable for AWS Kinesis Streams, AWS Lambda and AWS Kinesis Firehouse
  • aws kinesis command is used for creation/ describing stream. Command can also list the stream ARN. Them update the permissions.json file with ARN’s of the stream and role.

Advanced tasks with CloudTrail log files

  • Create multiple trails per region.
  • CloudWatch Logs are used to monitor CloudTrail log files
  • Share log files between accounts.
  • Log processing applications can be developed in Java by using CloudTrail Processing Library.
  • Validate log files to verify that they have not changed after delivery by CloudTrail.

To receive CloudTrail log files from multiple regions

  • Sign in to the AWS Management Console and open the CloudTrail console at https://console.aws.amazon.com/cloudtrail/.
  • Choose the option – “Trails”, and then select a trail name.
  • Next, click on pencil icon adjacent to “Apply trail to all regions”, and then select “Yes”.
  • Choose Save. The original trail will be replicated across all AWS regions. CloudTrail will deliver log files present in all regions to S3 bucket.

Validating CloudTrail Log File Integrity

  • Use CloudTrail log file integrity validation.
  • The feature using SHA-256 for hashing and SHA-256 with RSA for digital signing.
  • Thus making CloudTrail log files without detection, computationally infeasible to
    • Modify
    • delete
    • forge
  • Use CLI to validate files
  • With log file integrity validation, CloudTrail creates hash for every log file
  • Every hour, CloudTrail also creates a file (called a digest file) that references log files for last hour and has hash of each.
  • Each digest file is signed using private key of a public and private key pair.
  • After delivery, use public key to validate the digest file.
  • Every AWS region  has different key pairs in CloudTrail.
  • The digest files are delivered to S3 bucket associated with trail as CloudTrail log files.
  • The digest files are put into a folder separate from the log files.
  • Every digest file has digital signature of previous digest file if present.
  • The signature for current digest file is in the metadata properties of digest file S3 object.

Sharing CloudTrail Log Files Between AWS Accounts

The steps are

  • To share log file with an account, create IAM role for it.
  • For each of these IAM roles, create an access policy that grants read-only access to the account you want to share the log files with.
  • IAM user can take the required  role to retrieve log files, programmatically.

CloudTrail Processing Library

  • A Java library to process AWS CloudTrail logs
  • Details about CloudTrail SQS queue and code to process events, is to be provided
  • CloudTrail Processing Library will
    • polls SQS queue
    • reads and parses queue messages
    • downloads CloudTrail log files
    • parses events in the log files
    • passes events to code as Java objects.
  • It is scalable and fault-tolerant.
  • Handles parallel processing of log files
  • Manages network failures like network timeouts or inaccessible resources.
Menu