Appian Logs via Amazon SQS

This article covers how to set up Amazon SQS queue and configure a feed in Chronicle to ingest the Salesforce logs.

Requirements

• Region
• Queue Name
• Account Number
• Queue Access Key ID
• Queue Secret Key

Configuration

Step 1: Set up an Amazon SQS queue

To set up an Amazon Simple Queue Service (SQS) queue, follow these steps:

1. Log in to the AWS Management Console at https://console.aws.amazon.com.
2. Navigate to the SQS service by searching for "SQS" in the service search bar or by selecting it from the list of available services.
3. Click the Create queue button to create a new queue.

• • Standard Queue: Offers unlimited transactions per second and at-least-once delivery.
  • FIFO Queue: Ensures first-in-first-out (FIFO) message processing and exactly-once delivery. If you require message ordering or deduplication, choose this option. FIFO queues have a ".fifo" suffix in their names.

4. Configure the queue settings:

• • Name: Provide a unique name for your queue. Queue names must be unique within an AWS account and have a maximum length of 80 characters.
  • Visibility timeout: This is the amount of time during which a message is invisible to other consumers after it has been read by one consumer.
  • Retention period: Specify the length of time, in seconds, that Amazon SQS retains messages in the queue.
  • Maximum message size: Set the maximum size for each message in the queue, in bytes.
  • Delivery delay: Specify the delay, in seconds, before a newly sent message becomes available for processing.
  • Message deduplication: Enable this option for FIFO queues to eliminate duplicate messages.
  • Content-based deduplication: If enabled, Amazon SQS uses the content of the message to generate a unique message deduplication ID.

5. Click the Create queue button to create the queue.

6. Once the queue is created, you will be redirected to the queue's details page. From here, you can obtain important information about the queue, such as its ARN (Amazon Resource Name), which is required for integration with other services.

Step 2: Configure the webhook in Appian

To configure the webhook in Appian, follow these steps:

1. Log in to the Appian Cloud Platform with your administrator credentials.
2. Go to the System Administration Console. You can access it by clicking on the gear icon in the top right corner of the Appian interface and selecting Admin Console.
3. In the System Administration Console, navigate to the Monitoring Configuration page. This page contains settings related to log forwarding.
4. Scroll down to the Log Forwarding section and click the Configure Webhook button.
5. In the Webhook URL field, enter the endpoint URL for your Amazon SQS queue. This URL should follow the SQS endpoint format and include the region, AWS account number, and queue name. For example: https://sqs.{REGION}.amazonaws.com/{ACCOUNT_NUMBER}/{QUEUE_NAME}
6. Configure the other log forwarding options based on your requirements. You can specify the log types and severity levels to be forwarded to the SQS queue. Adjust these settings as needed.
7. Scroll down and provide AWS access credentials for authentication. Click the Edit button next to the AWS Access Credentials section.
8. In the Connected System field, select the AWS connected system that you previously created or create a new one. This connected system should contain the AWS access key and secret access key with appropriate permissions to access the SQS queue.
9. Click Save to save the webhook configuration.
   Once the webhook is configured, Appian will start forwarding logs to the specified Amazon SQS queue based on the log types and severity levels you have configured. Provide this webhook URL in the appropriate field when configuring the webhook in Appian's System Administration Console for log forwarding. 

Step 3: Access key ID and Secret Key

To obtain the Amazon SQS access key and secret key, you need to create an IAM user in your AWS account and generate access keys for that user. Here's how you can do it:

1. Log in to the AWS Management Console at https://console.aws.amazon.com.
2. Navigate to the IAM (Identity and Access Management) service by searching for "IAM" in the service search bar or by selecting it from the list of available services.
3. In the IAM console, click Users in the sidebar and then click the Add user button.
4. Enter a username for the IAM user and select the access type:

• • If you want to provide programmatic access (e.g., for API or CLI access), select Programmatic access.
  • If you also want to allow the user to access the AWS Management Console, select AWS Management Console access.

5. Click the Next: Permissions button.

6. In the Set permissions page, you can either add the user to an existing group with the appropriate permissions or directly attach policies to the user. Select the necessary policies that grant access to SQS resources.

7. Click the Next: Tags button if you want to add tags. Otherwise, proceed to the next step.

8. Click the Next: Review button to review the user details.

9. Verify that the user details are correct, and then click the Create user button.

10. In the success page, you'll see a confirmation message along with the user's username. To view the access key and secret key, click the Download .csv button. This file contains the access key ID and secret access key for the user.

Store the access key ID and secret access key in a secure location. These credentials are important for accessing AWS resources using the AWS SDKs, APIs, or CLI.

Configure a feed in Chronicle

Complete the following steps to configure a feed in Chronicle to ingest the Salesforce logs:

1. Go to Chronicle settings and click Feeds.    
2. Click Add New. 

3. Select Amazon SQS for Source Type.    

4. Select Appian Cloud for Log Type.    

5. Click Next.    

6. Under the required field, paste the data collected by above process.

For example:

• • Region: us-east
  • Queue Name: Appian
  • Account Number: 123456789012
  • Queue Access Key ID: AKIA*****************
  • Queue Secret Key: qwertyuiopasdfghjklzxcvbnmqwertyuiopasd

7. Click Next and Finish.