Interpreting and Managing Behavior Profiles

This article describes the steps on interpreting and tuning multiple behavior models, filtering the behavior models on various metrics, disabling, enabling, and deleting the behavior models from the behavior analytics home page.

The Behavior Analytics page provides insights into user behavior through two key tabs: Agent Insights and Models. These tools help you understand, investigate, and respond to potential security risks based on user activity patterns.

Agent Insights

The Agent Insights tab displays behavioral insights generated by an AI agent trained on data from the past 7 days at the tenant level. The analysis specifically focuses on the principal.user.userid dimension, identifying behaviors detected by various models using this dimension. These behaviors are correlated and contextualized to generate actionable insights, each tagged with a risk level: High, Medium, or Low.

Each insight is enriched with entity metadata, offering a deeper understanding of the user's activity within their organizational context.

Contextual Risk Adjustment

Agent Insights also evaluate user behavior within the scope of their entity group, dynamically adjusting the assigned risk level based on whether the observed actions align with the user's defined responsibilities.

Example:
If a senior DevOps engineer performs maintenance on a Kubernetes pod, the action may initially be flagged as high risk. However, if the user is part of an entity group with legitimate access to Kubernetes pods, the risk level may be automatically downgraded to medium or low, depending on the context.

Insight Details

Each agent insight includes the following sections:

• Summary: A concise overview of the identified behavior.
• Timeline & Key Observations: Displays the model name, detection date, and details of the abnormal behavior. Clicking the model name redirects you to the Behavior Trends page, where you can view a graphical representation of all security signals generated over the past 7 days. The specific dimensions associated with this behavior are also shown.
• Hypothesis: Establishes the relationship between behaviors observed from the same user.
• Attack Flow: Visualizes the flow of events aligned with MITRE ATT&CK® tactics and techniques.
• Recommendations: Lists mitigation strategies to reduce or prevent the impact of specific tactics and techniques used by adversaries.

To view agent insights, see Viewing and Analyzing Agent Insights article. 

Viewing and Analyzing Agent Insights

To view and analyze insights on the Behavior Analytics page:

1. Click the hamburger menu in the top-left corner.

2. Under Security, click Behavior Analytics. You will land on the Agent Insights tab by default.

3. Use the available filters to refine insights:

• • Days: Filter insights from the last 7, 15, or 30 days.
  • Risk Level: Filter by High, Medium, Low, or All risk levels.
  • Sources: Currently, only "user" is supported as a source.
  • Bookmarked: Shows all the insights bookmarked by the logged in user. 

The page updates to show insights matching the selected filters.

4. Click View details or on the insight title or click the expand option if you want to view details of any specific insight. Each insight provides critical context to help you understand user behavior, identify potential threats, and take preventive or corrective action using the recommended mitigation strategies.

Models

The Models tab lists all behavior models. These may include:

• User-created models: Manually developed by analysts or security teams. To learn how to create a behavior model, see Creating a Behavior Model article. 
• Auto-generated models: Created automatically based on system recommendations. However, user intervention is required to create this model. To generate a model, see Generating a Model article. 

These models form the basis of the behavior detection and insights generated in the Agent Insights tab.

Bookmarking Insights Generated by an Agent

The bookmarking feature allows you to flag important insights for future reference and analysis. You can bookmark any insight generated by an agent and easily access it later from the Bookmarked filter.

Key Highlights

• Bookmarked insights are visible only to the user who marked them.
• You can add or remove bookmarks at any time.
• A count next to each insight indicates how many users have bookmarked it.

Bookmarking an Insight

To bookmark an insight generated by an agent, follow these steps:

1. Navigate to Behavior Analytics:
   • Click the hamburger menu in the top-left corner.
   • Under Security, click Behavior Analytics.
   • You will land on the Agent Insights tab by default.
2. Bookmark the Insight:
   • Locate the insight you want to bookmark.
   • Click the bookmark icon next to the insight.
   • Once bookmarked, the icon will update, and the count next to it will increase to reflect the total number of users who have bookmarked it.

Viewing Bookmarked Insights

To view insights you've bookmarked:

1. In the Agent Insights tab, locate the quick access filters.
2. Select the Bookmarked checkbox. The view will update to show only the insights you have bookmarked.

Viewing Bookmark Counts

Each insight displays a bookmark count, showing how many users have bookmarked it. This helps you identify which insights are widely considered noteworthy.

Model Recommendations

Resolution Intelligence Cloud recommends multiple models to start detecting user behavior. You can create behavior models based on the recommendations with ease on the behavior analytics home page. The AI suggests new models based on the combination of log sources and existing models. This functionality is available at the tenant level only.

There is also a refresh button next to the model recommendations, allowing you to generate five new recommendations each time it is refreshed. This ensures you always have the latest and most relevant suggestions for model creation.

Model generation using AI

AI integrated with behavioral analytics will create models based on tailored recommendations. When you select a recommendation to create a model, the AI gathers the necessary specifications and provides a draft of the initial model. However, you must manually review and complete any missing data before publishing the model for use.

Generating a Model

Use this procedure to create the draft version of a model using AI and publish the model manually. 

1. At the top left, hover over breadcrumb menu.

2. Under Security, click Behavior Analytics. The Behavior analytics page appears. 

3. Click Generate Model corresponding to the Model recommendation for which you want to generate a model, on the Agent Insights tab. 

The opens the New ML model form where the AI generates the specifications for the model and are these defaulted in the respective fields on the New Behavior Model form. 

5. Check the form and fill in the missing mandatory fields to generate the model.

6. Click Send for Review to submit the model for review. To save the model as a draft without sending it for review, use the Save as Draft option. This allows you to make changes and submit it for review later.

7. Click Approve or send the model to request changes. Clicking Approve will display a dialog box. Only the user with the publisher role can approve models.

8. Click Approve. This changes the status from Under Review to Approved.

You can now view the content packs. Models can only be published when they are associated with content packs.

9. Click Associate Packs. This opens the side panel where you can view the list of content packs available for the tenant.

10. Select the pack with which you want to associate the model.

11. Click Associate & Publish to publish the pack associated with the models. You can also click Associate to link the model and publish it later. This only changes the status to Ready to Publish. Until the model is published, you can keep adding the packs to which you want to associate the model.

Once published, the model's status changes to Published after being sent to Bigquery. It is now available and can be found in the generated list of models

Managing the Models listing page

The Behavior Analytics Listing Page displays a collection of models. The visibility of models depends on your operating environment:

Multi-Tenant Environment

• Domain Level:
  You can view models created by you and other users within your domain.
• Organization Level:
  You can view models created by you, as well as models inherited (percolated) from the domain account.
• Tenant Level:
  You can view models created by you, as well as models inherited from the domain or organization accounts.

Single-Tenant Environment

If you are operating in a single-tenant setup, you can only view models created by you and other users within your tenant.

You can reach any behavior model to drill down further from this listing page.

The columns given in the below image are described as follows:

Manage Columns (Table Settings) filter

The Manage Columns filter allows you to select your desired columns to display on the listing page. This simplifies the tuning of a model by picking the best parameters from these columns.

Procedural filters

On the right side of the listing page, an aggregated overview of models is displayed, categorized by Dimensions, Model Type, Model Status, Tactics, and Signal Generation. You can use the filter option to search for models with specific attributes or values across these categories.

Viewing a behavior profile offers you information beyond a traditional chart. You can create what-if Situations and tune the model, interact with the data (slice and dice) using the histogram/interactive filter on the right, or simply ask your question about the behaviors to Resolution Intelligence Cloud's conversation AI.

The page consists of the following components, categorized based on the outcomes you can achieve:

Outliers are visible only when both signal generation and the model are enabled. If the model is enabled but signal generation is disabled, outliers will still be detected, but the corresponding events or outliers will not be sent to Chronicle. When both the signal and model are enabled, outliers are sent as raw data to Chronicle, where parsers normalize the data into UDM format. The processed signals are then forwarded to Resolution Intelligence Cloud and can be viewed in the Resolution Signals page.

On the model details page, you can examine the behavior of any model by including or excluding certain attributes shown on the left of the page.

To analyze the trend of attributes,

1. Select the duration for which you want to view the behaviors and outliers generated. You can only select a date range of up to 30 days. If an outlier occurs for the first time within the selected 30-day period, it will be marked as a first occurrence. You can filter the signals by first occurrences, outliers, and inline behaviors.
2. Under Behavior Distribution over time, click on any bar in the graph. The selected dimensions and signals appear for the selected date range.
3. In the right panel, hover over each attribute (signal or IP address) and click on any of the following:
   • --> includes the attribute value in the analysis.
   • --> excludes the attribute value in the analysis.
   • --> include only this value and exclude others in the analysis.
4. Selected or excluded attribute values appear on the top of the bar chart. Under Aggregate Insights, you can view the total unique values and the total values for each dimension.

Reading charts and tables

The default landing page for selecting a behavior profile is the behavior analysis tab. In the behavior analysis tab, you can

1. Learn more about the behaviors the model is tracking

2. Investigate the behavior data using the interactive filtering on the right

3. Create what-if analysis on the model and tune the model for better performance

4. Ask questions to the behavior model using Resolution Intelligence’s conversational AI

The first histogram – highlighted in yellow in the image below – visualizes the total behaviors tracked for each day for the last 30 days. By default, the last day is selected in the histogram. You can modify the anomaly score given under the histogram anytime, and it ranges from 0 to 1. The total behaviors are updated according to the given anomaly score.

Note: The data displayed on this page reflects the selected day in the histogram. Changing the day here will update the data in the secondary histogram, the table, and the interactive filtering options on the right-hand side of the page.

The second histogram visualizes the distribution of anomaly scores for the selected day. This graph also includes an interactive slider, using which you can do a what-if analysis on the behavior model.

What if? By moving the slider across the distribution, you can visualize when certain behaviors shift from being non-anomalous to anomalous. This feature allows for a more granular and dynamic understanding of how the system tracks and scores behaviors, using which you can tune the model for better performance and false positives.

Note: The top left of the histogram highlights the total behaviors tracked, which is a Cartesian product of the unique value of each attribute selected in the model.

The table below the second histogram lists complete records data for the selected day. 

The columns in the below table are described as follows:

• Model Dimensions - The model dimensions refer to UDM fields, and the data from these fields is retrieved from the UDM table to train the model. For example, one of the models includes the following dimensions. These dimensions vary from model to model:

  • metadata_vendor_name – Identifies the company that developed the product generating the event data.

  • metadata_product_name – Identifies the vendor's product from which the event data is generated.

• Observed Behavior

  • Aggregated Activity Count (specific to rare occurrence model): This indicates the total number of times a specific activity has occurred for an entity over a defined period, highlighting how frequently the behavior has been observed. It compares the current activity count to historical data to determine whether the current behavior is rare. This column is displayed only in rare occurrence model types. 

• Behavior Insight - These fields show how each model identifies anomalies based on the baseline values for the source dimensions involved in the model.
  • Baselined Targets for Source (specific to Enumeration model): Informed by the training data, this column highlights the targets a source normally communicates with.
  • Baselined Unique Target Count for Source (specific to Enumeration model): The distinct count of targets a source normally communicates with. Learned during the training period, it evolves as the behavior of entities changes over time.
  • Unique Target Count for Day (specific to Enumeration model): Exhibited behavior on a given day. Highlights the distinct number of targets the source communicated with.
  • Variance Score Reason (specific to Enumeration model): Reasons for the Variance Score (anomaly score). Reasons include rarity (communicating with a new target) or deviation in activity (communication with a known target exceeding identified levels).
  • First Occurrence: The first observation of a behavior in the environment.
  • Last Seen Date: Date of the most recent observed behavior.
  • Baseline: Default behavior learned from historical data.
  • Deviation from Baseline behavior: Ratio of observed activity in a behavior on a given day against the learned normal behavior.
  • Variance Score: Also known as anomaly score, it is a score normalized to -1 to +1. -1 indicates reduced activity, and +1 signifies elevated activity compared to normal. The Variance score operates on a tailored algorithm that considers multiple dimensions of behavior.
  • Variance Certainty: The confidence level associated with the variance score. Influenced by the number of data points (data strength) observed for a behavior in the past and the distribution of data at the dimension level. 
    Note: Variance certainty help fine-tune false positives. 
    For instance, a behavior model could be configured to trigger a signal only when a behavior's Variance (anomaly) score surpasses 0.8 and its Variance certainty exceeding 0.9.
  • Alert Status: The status '1' indicates that the anomaly score exceeds the set abnormal condition 

The Interactive filtering on the right can help you control/filter the data you want to see in the table. But the interactive filtering also serves another purpose. It also highlights patterns in the selected data, sorted in descending order.

Note: Pre-model anomalies are now displayed on the bar graph in an “Unavailable” state. These anomalies are not sent to Chronicle but are retained for model training purposes. Once the model is trained—using up to 60 days of historical data—post-model anomalies are detected and sent to Chronicle as valid signals. If the model is temporarily disabled, anomalies generated during that period appear grayed out on the graph and are also excluded from forwarding to Chronicle. 

 

Identifying the model to tune from the listing page

A challenging task when creating behavior models is understating which models are generating false-positives and how to go about tuning them.

When architecting behavior analytics, we realized its importance and incorporated a lot of metrics when modelling data.

Metrics for identifying a model to tune:

1. Data completeness/average Data completeness: The completeness of the data used for model training or alerting is critical. If the model operates on limited or partial data, it may result in an increased number of false-positives. Continuous monitoring of the events under analysis helps identify gaps in the ingested data, allowing for adjustments to improve data completeness
2. Total Behaviors over Variance Limit (since Model creation): This metric indicates whether a model has a high number of outlier behaviors compared to the variance limit set during its creation. An excessive number of outlier behaviors suggests that the variance limit may need adjustment to better reflect the expected behavior patterns.
3. Behaviors Over Variance Limit (Last Run): While Total Behaviors Over Variance Limit provides an overall view, this metric focuses on identifying the number of outliers observed during a specific time frame, typically the last run of the model. It offers insights into the immediate performance of the model and helps pinpoint any sudden spikes in outlier behaviors.

Using simulation to tune a model

Simulation plays a vital role in fine-tuning behavior models. By simulating various scenarios and input data, you can assess the model's performance under different conditions and identify potential areas for improvement. This iterative process enables you to adjust parameters, thresholds to optimize the model's accuracy and reduce false-positive alerts.

In summary, leveraging a combination of metrics for model evaluation and employing simulation techniques for tuning allows for continuous improvement in behavior analytics, ultimately enhancing the reliability and effectiveness of the Resolution Intelligence Cloud.