Correlation Policies

  • Updated
In this article:

    Correlation policies are a set of conditions include typically Signal source, Signal category, Asset type and etc. that match with given fields, and hence generate a situation from multiple similar signals triggered from integrated systems.

    Correlation policy allows you to correlate the similar threats over a network into a single signal to reduce the network traffic and improve the efficiency of a support team to mitigate the issues faster. Use the correlation feature to respond in real time to threats over a large network.

    Correlation policy uses the different time windows (Open, Fixed, and Sliding) to provide a gap between one signal to another in order to correlate similar signals.

    • Open: No time is given for correlation of signals
    • Fixed: After a fixed time, correlation stops
    • Sliding: Correlation moves with addition of a new signal. Suppose if correlation happens at 7 PM initially, then a new signal adds to the correlation, the time window moves forward.

    Use case

    A Router or a Switch is connected to multiple devices over a network. If there is an issue in router or in switch, network fails to send signal to the connected devices. In such case, there will be multiple signals (that are similar) generated and assign them to multiple personnel. 

    If you define what type of signals to be correlated in a policy, the policy correlates the similar signals and assign them as single signal to the responder. Refer this documentation on how you configure a correlation policy.

    Features of Correlation Policy

    • Improves the efficiency of support team to respond to the signals quickly.
    • Takes less time in identifying, segregating, analyzing, and resolving a signal.
    • Enable Domain and Organization users to import the existing rules defined in a policy to their existing or new customers.
    • Saves cost to an organization.

    How Correlation Policy Works

    Resolution Intelligence Cloud consumes raw event data from the integrated monitoring tools where it merges the events into signals so that you can visualize the life cycle of an issue over time. Resolution Intelligence correlates related signals into Situations for visibility into high-level, actionable problems.

    Knowing how Resolution Intelligence Cloud process raw event data into signals and which signals are grouped together to create a Situation can help you to use and configure Resolution Intelligence Cloud more effectively.

    Resolution Intelligence Cloud uses certain properties such as host, service, application, or device and check or sensor to process the raw event data. These properties are used for various purposes.

    • During Correlation, Resolution Intelligence Cloud uses these properties to determine which events are part of same issue.
    • In the default correlation pattern, Resolution Intelligence Cloud uses the properties (i.e host, service, application, or device) to check if the signals are relevant to each other.
    • In the Resolution Intelligence Cloud interface, main title and sub-title of a situation are constructed using above mentioned properties.
    {
           "class": "Device Health",
           "subClass": "CPU",
           "category": "Linux",
           "subCategory": "RedHat",
           "scope": "global",
           "customerId": null,
           "partnerId": null,
           "owner": "NetEnrich"
    }

    Merging signals into a Situation

    Signals are merged into the same Situation when they have same application key and matching properties (i.e host, service, application, or device and check or sensor). A Situation can have one or more signals which are shown in the Situation timeline. The current status and properties of a Situation in Resolution Intelligence Cloud denotes most recent signal which is determined by the time stamp property.

    Clustering Signals into a Situation

    After merging signals into a Situation, the configurator rules in Resolution Intelligence Cloud provides an additional suppression of noise and enhanced visibility by grouping related signals into a single, high level Situation. For example, internet connectivity of a router may cause several checks before it enters into a critical state. All of these signals are grouped together to form a Situation, so that you can see the latest information on a Situation timeline. Resolution Intelligence Cloud uses correlation rules to establish a relation between signals and applies pattern recognition to cluster the similar signals into a Situation.

    Resolution Intelligence Cloud can correlate any number of related signals into a single Situation.

    {
           "_id" : 1,
           "data" : {
                  "class": "Device Health",
                  "subclass": "CPU",
                  "category": "Linux",
                  "subcategory": "RedHat",
                  "scope": "global",
                  "customerId": null,
                  "partnerId": null,
                  "owner": "NetEnrich"
           },
        "query" : {
                  "bool": {
                         "should": [
                               {
                                      "term": {"metric": "cpu.load"}
                               },
                               {
                                      "match": {"subject": "CPU load very high"}
                               }
                         ]
                  }
           }     
    }

    Defining Correlation Patterns

    Use the following parameters to define the relationship between signals which are established as a result of correlation patterns.

    • Source Systems: Based on the type of integrated monitoring systems, the pattern applies. For example, show signals coming from Jira, and OpsRamp.
    • Tags: Dynamic patterns correlates the signals based on certain type of tags. For example, correlate all the signals that come from same cluster and have the same check.
    • Time Window: The gap between the signals when they are initiated. For example, network related signals can happen within a short period, while load issues arise over a long period of time. The correlation patterns follow the below time windows:
      • Fixed time window - A fixed block of time in which correlation of signals happens. For example, If the correlation period is for 15 minutes, and starts correlation at 6.00 PM, then the time blocks are 6.15 PM, 6.30 PM and so on.
    "time_window" : {
        "enble" : "true",
        "type" : "fixed",
        "starttime" : "00:00:00",
        "window_seconds" : "900"
    },
      • Signal based fixed time window - Time window starts when first signal correlated. For example, if the first signal starts at 10 PM and time window is 20 minutes, then the time window is till 10.20 PM.
    "time_window" : {
        "enble" : "true",
        "type" : "first_signal_fixed",
        "window_seconds" : "900"
    },
      • Signal based sliding time window - Time window starts from the first signal in correlation and ends after a defined time period since the timestamp of last signal arrived in correlation. For example, if the first signal starts at 9.10 AM and the time window is 15 minutes, then timer starts for correlation and ends at 9.25 AM. If another signal is added to the correlation at 9.20 AM, then the time window will be pushed to 9.35 AM and continuous the process till no signal is added to the correlation.
    "time_window" : {
        "enble" : "true",
        "type" : "sliding",
        "window_seconds" : "900"
    },
      • Default - If no time window is specified, default time window is for 15 minutes.
      • Minimum - Depending on the system feasibility, the minimum time window is 1 to 5 minutes.
      • Maximum - The maximum time window for any correlation is 72 hours (Assuming there no impact on correlation for 24 hours or greater).
    • Filters (Optional): The conditions that are customized to correlate the highly related signals. For example, correlate only network-related signals by data center.

    Applying Pattern Recognition

    To form dynamic clusters, Resolution Intelligence Cloud stores the patterns that match the incoming signals, and merges into a single Situation. Whenever a new signal is received, Resolution Intelligence Cloud evaluates it against any matching patterns and attaches that signal to an active Situation, if it matches with that Situation pattern. If the received signal does not match with the existing patterns of an situation, then a new Situation is created with respective to that signal. The matching patterns determine the Situation title with the widest time window.

    As Resolution Intelligence Cloud receives the signals from different sources, the process continuous until all pattern evaluation is completed, and the detailed information is available for every Situation. No new signals are added to a Situation when all pattern windows have elapsed. All matching signals determine the life cycle of a Situation, and the Situation is in open state, till all signals are resolved.

    For example, you can create patterns that correlate:

    • Network-related connectivity issues within the same data center.
    • Application-specific checks on the same host.
    • Load-related signals from multiple servers in the same database cluster.
    • Low memory signals on a distributed cache.

    Related articles

    Comments

    0 comments

    Please sign in to leave a comment.