Log Retention
Overview ⫘
The Secureworks® Taegis™ XDR log retention policy applies to all customers. By default, all customers enjoy 12 months of log retention, with additional retention periods possible at an additional cost (see Data Storage Upgrade). XDR stores all ingested client data inside our cloud hosted data lake in compressed form. This includes data from supported endpoint agents (if applicable), API data (such as Office 365, AWS GuardDuty, etc.), and syslog data forwarded to Taegis™ XDR Collectors.
Generic Syslog Data ⫘
XDR stores syslog data received through one or more XDR Collectors. This includes events from currently supported sources (see Available Integrations) that are received but not normalized into one of our security schemas, as well as any custom parsers built by the Secureworks Professional Services team. XDR also accepts data from non-supported syslog sources and stores these events in a generic way so that they are searchable within XDR, and can also be placed within reports as desired (see below). If the data is normalized into generic events, it will no longer be stored; however, all normalized events still contain the original unaltered message in the original_data field.
The format of logs ingested and stored in the generic table conforms to RFC-3164. The following is a visual example of this format:
Generic Table Format1
How are generic events generated? ⫘
Unsupported syslog sources are parsed into the generic schema. Because an unsupported source doesn’t have a dedicated parser associated with it, all events from the unsupported source are normalized into the generic schema. Normalization of generic events means that XDR is parsing data into the format specified by RFC-3164. All generic events from unsupported sources are still available for searching, reporting, and custom rule creation.
Sending unsupported syslogs allows users to search and incorporate these generic events into reporting, if desired. Custom rule creation using the generic schema is also possible. For example:
FROM generic WHERE ((sensor_type MATCHES 'MICROSOFT_WINDOWS_SNARE' AND original_data CONTAINS 'The audit log was cleared') AND original_data CONTAINS '1102')
Note
XDR only stores data that is not normalized as generic events. All normalized events still contain the original unaltered message in the original_data field. For more information, see FAQ: Generic Events and Normalized Data.
Note
Currently, generic events do not automatically generate alerts, nor do detections work against them. If a currently unsupported security device generating syslog data needs to have alerts generated and detections executed, please reach out to your Customer Success Manager (CSM) for options on custom parser creation through the Secureworks® Professional Services organization.
How do I normalize syslogs? ⫘
RFC-3164 specifies four essential elements be present for parsing:
- A PRI (priority) value
- A timestamp
- A hostname
- A message, containing a tag or PID (process ID) and the message content
XDR can successfully parse many variations of these required elements. If, for example, a source isn’t correctly displaying timestamp, hostname, or message content, please refer to the diagram above and compare the format of the events being generated. It is possible that minor configuration changes may be needed to the syslog format generated to allow for generic events to be ingested and normalized into XDR.
Retention Periods ⫘
By default, all customers have 12 months of retention within the XDR data lake. Customers can purchase up to 48 months of extended retention, for a total retention of 60 months. If you are interested in purchasing longer retention, please contact your account representative.
Data Storage Volume ⫘
XDR customers are allotted a data storage volume based on their organization’s selected data volume subscription (Standard or Upgraded). By default, the Standard data volume applies. The Standard data allowance is calculated by multiplying the contracted endpoint count by 4 GB per month. For example, if the contracted endpoint value is 500, the total data cap would be 500 * 4 GB = 2000 GB of capacity per month. Customers requiring additional data storage capacity can purchase an Upgraded subscription which allows for 20 GB per calendar month per endpoint. Please contact your sales representative for additional details concerning this option.
How is the Volume of Stored Data Calculated? ⫘
XDR measures data volume at rest to calculate monthly log retention volume against monthly log retention allowance. All data stored within the XDR data lake is counted towards the aggregate capacity allowed by the monthly plan. This includes data from supported endpoint agents, API data, and syslog data stored. All data at rest is stored in compressed form, and the data utilization is calculated using the compressed data form.
Data Storage Upgrade ⫘
If a customer’s data storage volume exceeds the Standard allowance, then additional fees may apply to upgrade the data volume subscription. Exceeding the data volume allowance will not result in any loss of data stored or disruption of data ingestion. Upgrade fees will not be applied automatically. Specific guidelines and processes for working with customers to upgrade their data volume subscription are pending the release of features in the product that will enable customers to track their data usage against their allotted volume and will notify customers when upgrade discussion may need to take place.
Related Topics ⫘