Adding Data to Repeating Fields
When a schema field is defined as repeated, such as repeated string or repeated AttachmentRecord, there are multiple methods for creating an array that can be populated with key-value pairs.
The following examples show how to create and populate an array of ThreatIntelligenceIndicators objects.
Using the ADDFIELD Function ⫘
ADDFIELD(collection, fieldName, fieldValues)
Adds a field to an array of objects. The values of the field for each object are specified by fieldValues (also an array). The name of the new field is specified by fieldName. If collection is NULL, a new array of objects is created, each with a single field (fieldName) with the provided values.
Example ⫘
{"id":"abc123_yessss","azureTenantId":"11111111-cccc-4444-9999-bbbbbbbbbbbb","azureSubscriptionId":null,"riskScore":null,"tags":[],"activityGroupName":null,"assignedTo":null,"category":"DefenseEvasion","closedDateTime":null,"comments":[],"confidence":null,"createdDateTime":"2022-11-03T14:58:39.8189203Z","description":"A Windows executable file that has been renamed was launched. Attackers might be attempting to use the executable to perform malicious activities, such as running malicious code or connecting to an attacker-controlled network address. By changing the path or file name of a system executable, attackers can avoid certain behavior-based detection technologies that rely on default paths and file names.","detectionIds":[],"eventDateTime":"2022-11-03T14:51:19.3637988Z","feedback":null,"incidentIds":[],"lastEventDateTime":null,"lastModifiedDateTime":"2022-11-03T14:59:15.9766667Z","recommendedActions":[],"severity":"medium","sourceMaterials":["https://security.microsoft.com/alerts/abc123_yessss","https://security.microsoft.com/incidents/12345"],"status":"newAlert","title":"System executable renamed and launched","vendorInformation":{"provider":"Microsoft Defender ATP","providerVersion":null,"subProvider":"MicrosoftDefenderATP","vendor":"Microsoft"},"alertDetections":[],"cloudAppStates":[],"fileStates":[{"name":"weirdFile.exe","path":"C:\\Users\\jdoe\\AppData\\Local\\Temp","riskScore":null,"fileHash":{"hashType":"sha1","hashValue":"8d7f701jk1234567jiff78q9e482hm41ef9a0h12k6k"}},{"name":"mammals.bat","path":"D:\\Destination","riskScore":null,"fileHash":{"hashType":"sha1","hashValue":"aaaaaabbbbbbbcccccdddddd1111122223333444"}}],"hostStates":[{"fqdn":"le_host.acme.com","isAzureAdJoined":true,"isAzureAdRegistered":null,"isHybridAzureDomainJoined":null,"netBiosName":null,"os":"Windows10","privateIpAddress":"10.220.100.100","publicIpAddress":"8.8.8.8","riskScore":"high"}],"historyStates":[],"investigationSecurityStates":[],"messageSecurityStates":[],"networkConnections":[],"processes":[],"registryKeyStates":[],"securityResources":[],"triggers":[],"userStates":[{"aadUserId":"777aaaaa-5555-4444-bbbb-777555555777","accountName":"jdoe","domainName":"HQ","emailRole":"unknown","isVpn":null,"logonDateTime":null,"logonId":null,"logonIp":null,"logonLocation":null,"logonType":null,"onPremisesSecurityIdentifier":null,"riskScore":null,"userAccountType":null,"userPrincipalName":"jane.doe@acme.com"}],"uriClickSecurityStates":[],"vulnerabilityStates":[]}
threatIntelligenceIndicators$ = ADDFIELD(NULL, "type$", json["$.fileStates[*].fileHash.hashType"])
threatIntelligenceIndicators$ = ADDFIELD(threatIntelligenceIndicators$, "value$", json["$.fileStates[*].fileHash.hashValue"])
Creating Key-Value Pairs and Adding to an Object ⫘
Example ⫘
{"id":"abc123_yessss","azureTenantId":"11111111-cccc-4444-9999-bbbbbbbbbbbb","azureSubscriptionId":null,"riskScore":null,"tags":[],"activityGroupName":null,"assignedTo":null,"category":"DefenseEvasion","closedDateTime":null,"comments":[],"confidence":null,"createdDateTime":"2022-11-03T14:58:39.8189203Z","description":"A Windows executable file that has been renamed was launched. Attackers might be attempting to use the executable to perform malicious activities, such as running malicious code or connecting to an attacker-controlled network address. By changing the path or file name of a system executable, attackers can avoid certain behavior-based detection technologies that rely on default paths and file names.","detectionIds":[],"eventDateTime":"2022-11-03T14:51:19.3637988Z","feedback":null,"incidentIds":[],"lastEventDateTime":null,"lastModifiedDateTime":"2022-11-03T14:59:15.9766667Z","recommendedActions":[],"severity":"medium","sourceMaterials":["https://security.microsoft.com/alerts/abc123_yessss","https://security.microsoft.com/incidents/54321"],"status":"newAlert","title":"System executable renamed and launched","vendorInformation":{"provider":"Microsoft Defender ATP","providerVersion":null,"subProvider":"MicrosoftDefenderATP","vendor":"Microsoft"},"alertDetections":[],"cloudAppStates":[],"fileStates":[{"name":"weirdFile.exe","path":"C:\\Users\\jdoe\\AppData\\Local\\Temp","riskScore":null,"fileHash":{"hashType":"sha1","hashValue":"8f7f1jk1234567jiff78q9e482hm41ef9a0h12k6k"}},{"name":"mammals.bat","path":"D:\\Destination","riskScore":null,"fileHash":{"hashType":"sha1","hashValue":"aaaaaabbbbbbbcccccdddddd1111122223333444"}}],"hostStates":[{"fqdn":"le_host.acme.com","isAzureAdJoined":true,"isAzureAdRegistered":null,"isHybridAzureDomainJoined":null,"netBiosName":null,"os":"Windows10","privateIpAddress":"10.220.100.100","publicIpAddress":"8.8.8.8","riskScore":"high"}],"historyStates":[],"investigationSecurityStates":[],"messageSecurityStates":[],"networkConnections":[],"processes":[],"registryKeyStates":[],"securityResources":[],"triggers":[],"userStates":[{"aadUserId":"777aaaaa-5555-4444-bbbb-777555555777","accountName":"jdoe","domainName":"HQ","emailRole":"unknown","isVpn":null,"logonDateTime":null,"logonId":null,"logonIp":null,"logonLocation":null,"logonType":null,"onPremisesSecurityIdentifier":null,"riskScore":null,"userAccountType":null,"userPrincipalName":"jane.doe@acme.com"}],"uriClickSecurityStates":[],"vulnerabilityStates":[]}
fileInfo0.type$ = json["$.fileStates[0].fileHash.hashType"]
fileInfo0.value$ = json["$.fileStates[0].fileHash.hashValue"]
fileInfo1.type$ = json["$.fileStates[1].fileHash.hashType"]
fileInfo1.value$ = json["$.fileStates[1].fileHash.hashValue"]
threatIntelligenceIndicators$ = IF malwareInfo.value$ THEN [malwareInfo] ELSE []
threatIntelligenceIndicators$ = IF fileInfo0.value$ THEN threatIntelligenceIndicators$ + [fileInfo0] ELSE threatIntelligenceIndicators$
threatIntelligenceIndicators$ = IF fileInfo1.value$ THEN threatIntelligenceIndicators$ + [fileInfo1] ELSE threatIntelligenceIndicators$
Adding Key-Value Pairs to event_metadata and Other KeyValuePairsIndexed Fields ⫘
The following example shows how to populate the event_metadata schema field to capture valuable data that does not fit into a primary schema field.
Example ⫘
Jul 29 19:03:55 10.12.133.192 1 2021-07-29T19:03:55.527482+00:00 epp.deeparmor.io Taegis NGAV - - - DANOTIFY : Event:"Action" EventTime:"2021-07-29T19:00:33.368744+00:00" AlertGuid:"ffg5bfc9-61b6-4cf6-dd34-33c613bb6edf" Account:"df45d555-5abd-4b33-9bb3-c6a43ee2208d" Partner:"SECUREWORKS" FileName:"macho_abnormal_file" FilePath:"C:\Users\taegisnextgenav\Downloads\macho_abnormal_file" FileSize:"335872" SHA1:"5A1F8BC4873B036AF13A908C5140FCBD0E550D81" FileType:"macho" AlertType:"SUSPICIOUS" Score:"0.93905586" ActionTaken:"QUARANTINED" DetectionMethod:"RTFM" First:"2021-07-23T17:19:16.929711+00:00" DACloud:"CONNECTED" Network:"CONNECTED" ThreatCategory:"Trojan" ThreatName:"macOS.Trojan" ThreatLevel:"5" RunningApps:"ApplicationFrameHost, csrss, ctfmon, dllhost, dwm, explorer, fontdrvhost, LogonUI, Microsoft.Photos, mmc, msedge, OneDrive, rdpclip, RuntimeBroker, SearchApp, SecurityHealthSystray, ShellExperienceHost, sihost, StartMenuExperienceHost, svchost, taskhostw, TextInputHost, winlogon, YourPhone" DeviceName:"NextGenAVTest" DeviceUsername:"(Not Logged In)" DeviceGuid:"36425b3b-c7b0-450d-9373-96a261607ebf" DeviceOS:"Microsoft Windows 10 Pro 10.0.19042" DeviceAgent:"2021.6.0" DeviceIP:"10.0.0.4" DeviceIPV4:"10.12.128.8" DeviceMac:"00113718B4F2" DeviceFqdn:"NextGenAVTest" DeviceGroup:"Test-Lab-DA" DevicePolicy:"Detection and Protection" DeviceScore:"95" DeviceCreated:"2021-07-23T13:48:08.340082+00:00"
deviceFqdn.key$ = "DeviceFqdn"
deviceFqdn.value$ = vals["DeviceFqdn"]
deviceGroup.key$ = "DeviceGroup"
deviceGroup.value$ = vals["DeviceGroup"]
deviceScore.key$ = "DeviceScore"
deviceScore.value$ = vals["DeviceScore"]
eventMetadata$.record$ = eventMetadata$.record$ + [deviceFqdn, deviceGroup, deviceScore]