The ingestion of AWS WAF logs for alarms and API calls via AWS S3 using AWS CloudWatch and AWS CloudTrail.
Chronicle Data Types
Creating an AWS S3 Bucket
After you sign up for AWS, you're ready to create a bucket in Amazon S3 using the AWS Management Console. Every object in Amazon S3 is stored in a bucket. Before you can store data in Amazon S3, you must create a bucket.
- Sign into the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.
- Choose Create bucket.
The Create bucket wizard opens.
- In Bucket name, enter a DNS-compliant name for your bucket.
The bucket name must:
- Be unique across all of Amazon S3.
- Be between 3 and 63 characters long.
- Not contain uppercase characters.
- Start with a lowercase letter or number.
After you create the bucket, you cannot change its name.
Important: Avoid including sensitive information, such as account number, in the bucket name. The bucket name is visible in the URLs that point to the objects in the bucket.
4. In Region, choose the AWS Region where you want the bucket to reside.
Choose a Region close to you to minimize latency and costs and address regulatory requirements. Objects stored in a Region never leave that Region unless you explicitly transfer them to another Region. For a list of Amazon S3 AWS Regions, see AWS service endpoints in the Amazon Web Services General Reference.
5. Under Object Ownership, to disable or enable ACLs and control ownership of objects uploaded in your bucket, choose one of the following settings:
- Bucket owner enforced – ACLs are disabled, and the bucket owner automatically owns and has full control over every object in the bucket. ACLs no longer affect permissions to data in the S3 bucket. The bucket uses policies to define access control.
To require that all new buckets are created with ACLs disabled by using IAM or AWS Organizations policies.
Bucket owner preferred – The bucket owner owns and has full control over new objects that other accounts write to the bucket with the bucket-owner-full-control canned ACL.
If you apply the bucket owner preferred setting, to require all Amazon S3 uploads to include the
bucket-owner-full-controlcanned ACL, you can add a bucket policy that only allows object uploads that use this ACL.
- Object writer – The AWS account that uploads an object owns the object, has full control over it, and can grant other users access to it through ACLs.
- In Bucket settings for Block Public Access, choose the Block Public Access settings that you want to apply to the bucket.
We recommend that you keep all settings enabled unless you know you need to turn off one or more of them for your use case, such as hosting a public website. Block Public Access settings for the bucket are also enabled for all access points you create on the bucket. For more information about blocking public access.
6. (Optional) Under Bucket Versioning, you can choose if you wish to keep variants of objects in your bucket. For more information about versioning.
To disable or enable versioning on your bucket, choose either Disable or Enable.
7. (Optional) Under Tags, you can choose to add tags to your bucket. Tags are key-value pairs used to categorize storage.
To add a bucket tag, enter a Key and optionally a Value and choose Add Tag.
8. (Optional) Under Default encryption, you can choose to configure your bucket to use server-side encryption with either Amazon S3-managed keys (SSE-S3) or AWS KMS keys stored in AWS Key Management Service (AWS KMS) (SSE-KMS).
To disable or enable encryption, choose either Disable or Enable.
9. (Optional) If you want to enable S3 Object Lock, do the following:
- Choose Advanced settings.
Important: You can only enable Object Lock for a bucket when you create it, and you cannot disable it later. Enabling Object Lock also enables versioning for the bucket. After enabling you must configure the Object Lock default retention and legal hold settings to protect new objects from being deleted or overwritten.
- If you want to enable Object Lock, choose Enable, read the warning that appears, and acknowledge it.
- Choose Create bucket.
- Choose Advanced settings.
- Create a new S3 bucket for AWS CloudWatch and AWS CloudTrail logs. A pre-existing S3 bucket may also be used.
- Follow the AWS WAF logging instructions to send AWS WAF logs for API calls to the S3 bucket created in step one using AWS CloudTrail. These instructions also explain how to send logs Amazon WAF alarms logs via CloudWatch to an S3 bucket.
- Confirm AWS WAF Logs are flowing into the S3 bucket
- Follow the AWS S3 Bucket guide to create an IAM user that can access the S3 bucket.
- Provide the authentication information as per the AWS S3 Bucket Guide.
Configure Feed in Chronicle to ingest AWS logs
- Go to Chronicle settings, and click Feeds.
2. Click Add New.
3. Select Amazon S3 for Source Type.
4. Select AWS WAF for Log Type.
5. Click Next.
6. Select region and provide S3 URI of the Amazon S3 bucket you copied earlier. Further, you could append the S3 URI with:
As in the following example, so that Chronicle would scan logs each time only for a particular day:
7. Under URI IS A, select Directories including subdirectories. Select an appropriate option under Source Deletion Option, this should match with the permissions of the IAM User account you created earlier.
8. Provide Access Key ID and Secret Access Key of the IAM User account you created earlier.
9. Click Next and Finish.
10. Once the configuration is completed, need to validate the logs in chronicle using a regular expression as (".*") or with specific hostname, will provide the log source types which are ingesting to chronicle, below is the screen shot for reference.