Defender alerts

[bpluta-splunk]

Details

2 new detections to leverage Microsoft defender alerts

[pyth0n1c]

FYI, I know that both of these detections are marked “manual test” but if I remove that, they fail unit testing for the same reason.
The justification that “they will fail integration testing since Risk Score is dynamically calculated” mean they SHOULD pass Unit testing.
I will look into these more closely tomorrow and hopefully we can get them approved+merged.

The failure currently is as follows during unit testing: exception: The observable field(s) {'user'} are missing in the detection results

[bpluta-splunk]

FYI, I know that both of these detections are marked “manual test” but if I remove that, they fail unit testing for the same reason. The justification that “they will fail integration testing since Risk Score is dynamically calculated” mean they SHOULD pass Unit testing. I will look into these more closely tomorrow and hopefully we can get them approved+merged.

The failure currently is as follows during unit testing: exception: The observable field(s) {'user'} are missing in the detection results

Based on the sample in data_sources and the way the search is constructed, when flowing through a normal ES search, the risk_score field should be populated. If that is what the unit testing is looking for, this should pass. While there is a risk_score defined as part of the detection, it will be overridden based on the eval in the search.

The user field may be null, but the src field should always have a value as that is the risk_object we are going after.

[ljstella]

The user field may be null, but the src field should always have a value as that is the risk_object we are going after.

The user field is currently configured as a risk object- below, I've translated the tags.observable for one of the detections to our new rba config where this will be more obvious:

  message: $severity$ alert for $dest$ - $signature$
  observable:
  - name: dest
    type: Endpoint
    role:
    - Victim
  - name: user
    type: User
    role:
    - Victim
  - name: file_name
    type: File Name
    role:
    - Attacker
  - name: process
    type: Process Name
    role:
    - Attacker
  - name: ip_address
    type: IP Address
    role:
    - Attacker
  - name: registry_key
    type: Registry Key
    role:
    - Attacker
  - name: url
    type: URL String
    role:
    - Attacker

becomes

rba:
  message: $severity$ alert for $dest$ - $signature$
  risk_objects:
  - field: dest
    type: system
    score: 50
  - field: user
    type: user
    score: 50
  threat_objects:
  - field: file_name
    type: file_name
  - field: process
    type: process_name
  - field: ip_address
    type: ip_address
  - field: registry_key
    type: registry_key
  - field: url
    type: url

Our current validation mechanisms can be summarized as follows:

a. If an observable is a Risk Object, EVERY result must contain a non-null value for each Risk Object.
b. If an observable is a THREAT object, AT LEAST 1 of the results must contain a non-null value for each Threat Object

[bpluta-splunk]

In this case the user field, as a risk_object can be null, but in some cases will be populated. Since we know the src field will always be populated, do you just have that as victim observable? Essentially if user is populated, we want to map it to the user risk object field. But the user field may not always be populated. So how do we make it optional, but still map to a risk_object if populated? Is this a testing vs config issue?

[ljstella]

In this case the user field, as a risk_object can be null, but in some cases will be populated. Since we know the src field will always be populated, do you just have that as victim observable? Essentially if user is populated, we want to map it to the user risk object field. But the user field may not always be populated. So how do we make it optional, but still map to a risk_object if populated? Is this a testing vs config issue?

We don't. There's no mechanism currently to do this on our side, and its for a very specific reason: null values in risk objects create unpredictable failure states in ES itself where the risk action fails silently- while we can't constrain customer environments to only run a magically working config based on their data that we can't see, we do enforce that the sample datasets avoid null values for risk objects to prevent the possible failures in our testing pipelines.

[patel-bhavin]

@bpluta-splunk : the way our testing is setup, we would like all risk related fields to be populated. While the dataset contains user field for several of the events, it doesnt for some of them. I will updating the two test datasets by trimming them down to a single event that has all the fields present for the detection to fire! Will keep you posted!

[bpluta-splunk]

since we know some of the defender alerts will not have the user field populated, seems like the best action is to remove the user from the observable.

[patel-bhavin]

So after updating the dataset, we are still seeing failures but now for threat_object since the data is such that when user is present, other fields are empty causing new failures.

I agree with Bryan that the detection should be set up the way it is, and when the field is empty nothing gets attributed to risk. I recommend continue to make it as a manual test as when I test it locally, it works. We would anyway need manual test since the risk_score is calculated dynamically!

I also think we want to continue having this as a requirement : If we start seeing more issues where we have to tag more manual tests, lets considering updating our unit testing contsraints

Previously Addressed

[pyth0n1c]

After discussion with the team and manual testing review, this looks good. Approved!