Azure Data Collector
data collectors integrations azure microsoft
Secureworks® Taegis™ XDR has a collector that can be set up in Microsoft Azure. The Taegis™ XDR Collector for Azure is provided as a Virtual Hard Disk (VHD). Each collector is uniquely identified by its configuration. Configuration data is injected into an Azure VM via Custom Data. For convenience, a sample Azure Resource Manager template is generated for your XDR Collector upon completion of the provisioning process. This sample template creates the following resources:
- Storage Account — A public-facing Microsoft Storage Account with a private Container is created to store the XDR Collector VHD. The Storage Account should not be used for other purposes.
- Azure Managed Image — The XDR VHD is copied to the Storage Account and converted to a Managed Image.
- Bootstrap Virtual Machine — To ease the deployment process an ephemeral bootstrap instance is created to copy the XDR VHD to your Azure Resource Group and Storage Account and is deleted once the XDR Collector is online.
- XDR Collector Virtual Machine — A single Azure Virtual Machine instance where syslog devices can send traffic.
- Network Security Group — The Network Security Group is attached to the XDR Collector instance in order to allow udp/514 and tcp/601 traffic.
- Container Instance — An Azure Container Instance is used to clean up temporary bootstrap resources.
XDR provides options to download the sample template or launch it directly. The sample template should work in most Azure Virtual Network environments—but you must review it carefully to ensure it conforms to all network and security policies. Advanced users can extract the Custom Data and VHD Shared Access Signature (SAS) URL (found as the sourceVHD
variable in the ARM template) from the template to incorporate into their own IaC or automation tools. The sample template uses the SAS URL to copy the XDR VHD to your Azure account to create a Managed Image. If you elect not to use the sample ARM template you need to use the SAS URL to either download the VHD or use the Azure CLI tools to copy the VHD to your own Storage Account.
Note
The SAS URL is only valid for three hours from the time ARM template is generated.
Note
The sample template provided requires the Owner
role OR the Contributor
plus User Access Administrator
roles.
Configuration Notes ⫘
-
The sample template creates a Network Security Group attached to the XDR VM that accepts
udp/514
andtcp/601
from the Azure Virtual Network it is deployed to. Outbound traffic is restricted to tcp/443 to XDR API Endpoint,udp/123
for NTP and bothtcp/53
andudp/53
for DNS. All other inbound and outbound traffic will be blocked. If your network policies, security policies or network requirements are different it may require modifications to the sample template. -
The sample template also creates a Network Security Group attached to the bootstrap virtual machine that allows outbound
tcp/443
. The bootstrap VM requires access tohttps://aka.ms
and Azure Storage Accounts to successfully copy the VHD to your account. If you have other egress network controls you must allow these connections until the collector has been fully provisioned. After the XDR Collector is online the bootstrap instance is removed—including its security group. You may remove any exceptions. If the bootstrap VM is not removed during this process, you can remove it manually. -
The XDR Collector for Azure does not currently support a highly available (HA) setup. If you chose to customize the template or deploy with a different automation toolset and elect to deploy the instance as a Virtual Machine Scale Set, ensure that the size is always set to
1
. Running more than one Collector with the same configuration (Custom Data) is not supported. -
The sample template allocates a 200G Data Disk volume attached as
/dev/sdc
to the collector. If you customize the template or implement your own automation, ensure that the device is always attached as/dev/sdc
with a minimum size of200G
.
Note
The XDR Collector can support up to 200K EPS (events per second) for properly configured cloud and on-premises collectors.
Note
Third-party tools or applications cannot be installed on any XDR Collector.
Connectivity Requirements for Data Collectors ⫘
Regions
XDR Regional Configuration ⫘
Some configuration specifics of XDR depend on the region you are deployed in (US1, US2, US3, EU).
Any device that uses its own SSL certificate, including Cloud-based and On-Premises Data Collectors, must safelist the following destination IP addresses or domains in order to avoid conflict. If using an AWS data collector, please refer to the AWS table.
For Most Data Collectors ⫘
Source | Destination | Port/Protocol | Notes |
---|---|---|---|
Data Collector IP or hostname | US1collector.ctpx.secureworks.com18.217.45.178/32 3.16.4.173/32 18.224.219.97/32 13.59.146.90/32 3.16.16.254/32 18.223.74.238/32 US2collector.delta.taegis.secureworks.com52.14.113.127/32 3.141.73.137/32 3.136.78.106/32 US3collector.foxtrot.taegis.secureworks.com44.229.101.49 35.166.77.47 34.214.135.78 EUcollector.echo.taegis.secureworks.com18.158.143.139/32 35.159.14.37/32 52.59.37.234/32 |
TCP/443 | Safelisting device access to XDR |
Data Collector IP or hostname | NTP severs IP/Hostnames provided during provisioning | UDP/123 | Safelisting device access to NTP servers This rule is only necessary when custom NTP servers are provided during provisioning. |
Data Collector IP or hostname | 0.pool.ntp.org 1.pool.ntp.org 2.pool.ntp.org 3.pool.ntp.org |
UDP/123 | Safelisting device access to default NTP server. This rule is only necessary when custom NTP servers are not provided during provisioning. |
Data Collector IP or hostname | DNS server IPs provided during provisioning | UDP/53 TCP/53 |
Safelisting device access to DNS servers |
Note
If using local NTP, the access must be safelisted both to and from the data collector on those networks.
For AWS Data Collectors ⫘
Source | Destination | Port/Protocol | Notes |
---|---|---|---|
AWS Data Collector IP or hostname | US1collector.ctpx.secureworks.com18.217.45.178/32 3.16.4.173/32 18.224.219.97/32 13.59.146.90/32 3.16.16.254/32 18.223.74.238/32 US2collector.delta.taegis.secureworks.com52.14.113.127/32 3.141.73.137/32 3.136.78.106/32 US3collector.foxtrot.taegis.secureworks.com44.229.101.49 35.166.77.47 34.214.135.78 EUcollector.echo.taegis.secureworks.com18.158.143.139/32 35.159.14.37/32 52.59.37.234/32 |
TCP/443 | Safelisting device access to Taegis XDR via hostname |
AWS Data Collector IP or hostname | NTP severs IP/Hostnames provided during provisioning | UDP/123 | Safelisting device access to NTP servers This rule is only necessary when custom NTP servers are provided during provisioning. |
AWS Data Collector IP or hostname | 169.254.169.123 | UDP/123 | Safelisting device access to default NTP server. This rule is only necessary when custom NTP servers are not provided during provisioning. |
AWS Data Collector IP or hostname | DNS server IPs provided during provisioning | UDP/53 TCP/53 |
Safelisting device access to DNS servers |
Proxy Support ⫘
Cloud-based and On-Premises Data Collectors attempt to discover local proxy settings on the host if they are unable to connect directly to the internet.
Cloud-based and On-Premises Data Collectors also support a hard-coded proxy. If you need to create a data collector that contains a hard-coded proxy, please submit a support request with the following required information:
- Proxy IP
- Proxy Port
If the proxy is configured but is unavailable or not reachable, the data collector will fall back to a direct connection.
Note
Cloud-based and On-Premises Data Collectors do not support hard-coded authenticated proxies at this time. A proxy with man in the middle (MITM) capability needs to safelist the above network connections.
Install and Configure an XDR Azure Collector ⫘
Start the process to configure your XDR Azure Collector in XDR from Integrations > Data Collectors. It will create a template for you, then forward you to your Azure account Console to complete the configuration.
- Select Integrations from the Taegis XDR menu and then choose Data Collectors.
- Select Actions > Add Collector from the top right.
Add New Collector
- Select Cloud-Hosted as the collector type and then select Next.
- Fill in the required name and hostname fields, and the optional description, host proxy, and NTP servers fields, and then select Create Collector.
Note
You have the option to specify your own NTP servers if desired, and to add an HTTP proxy address, which must follow the following format: [http\[s]://\[user:pass@]hostname\[:port]|http://<hostname>[:port]]
.
Note
Default and custom NTP settings are only used during initial Data Collector setup. Once connectivity is established, the Data Collector synchronizes time via the XDR backend connection.
Create Cloud Collector
Tip
To add the eStreamer app to the collector to retrieve all security event logs from your Cisco Firepower Threat Defense (FTD) device, see eStreamer App. For more information, see the Cisco FTD Firewall guide.
-
The Install Collector section displays the following options:
- Amazon Web Services (AWS) — This option is for deploying the XDR Collector to AWS.
- Google Cloud Platform (GCP) — This option is for deploying the XDR Collector to GCP.
- Microsoft Azure — This option is for deploying the XDR Collector to Azure.
-
Select Microsoft Azure and then Deploy in Azure.
Azure Collector
Note
You can also download the Azure Resource Manager template in json format by selecting Download ARM Template.
The Azure console displays the Template Deployment page in a new browser window with a preloaded Azure Resource Manager template (JSON) that includes parameters for the collector.
Azure Configuration
- Choose the Azure Subscription that this deployment should be associated with.
- Choose an existing or new Resource Group. A new Resource Group is recommended.
- Choose the Azure region for the deployment. This should be the same region as the Azure Virtual Network you want to deploy the collector in.
- Add a Vm Name for the collector.
- Choose a Vm Size for the collector. We currently recommend Standard_D4s_v3 as it is able to handle a majority of deployments we see and allows for additional log sources added later. You may select a lesser SKU for an environment with few reporting sources, but we recommend choosing the Standard_D4s_v3 SKU.
- Enter the name of the Resource Group which contains your Azure Virtual Network.
Important
The Azure Virtual Network resource mentioned above needs to exist prior to deploying the collector (or if the resource does not exist it needs to be created) and it needs to comply with the requirements mentioned in the Connectivity Requirements. Deployment will fail if this requirement is not met.
- Enter the name of the Azure Virtual Network to deploy the collector.
- Enter the name of the Azure Virtual Network Subnet to deploy the collector.
- Optionally, enter the address of the Static Private IP to assign to the collector or leave this field blank to have one auto-assigned.
- Leave the Utc Now field as the default value. This is only used to generate unique names for the deployment.
- Leave the Random Seed field as the default value. This is needed to generate random data for a temporary password used during the deployment. The default is a random GUID created at template runtime.
- Review the Azure Marketplace Terms and check the box to agree to the terms and conditions.
- Click Purchase to start the deployment.
Important
You must turn on Office 365 audit logging for XDR to receive data from it. Audit logging for Office 365 is off by default. For more information, see Turn Office 365 audit log search on or off.
Access Troubleshooting Console ⫘
The Admiral console allows you to access information about a deployed XDR Collector locally. The tools provided within Admiral assist in device setup and troubleshooting of common problems such as network connectivity. For more information, see Admiral Console.
Edit Your XDR Collector Configuration ⫘
Important
Making changes to the XDR Collector configuration of a live system carries the risk of rendering the device inoperable. The XDR Collector will make every attempt possible to rollback to the previous configuration when a configuration change is unsuccessful. XDR Collector configuration changes should be treated with the same level of caution used for any other kind of change in your environment according to your risk and change management guidelines. You should always be prepared to redeploy the device.
Certain configuration parameters of a running and healthy XDR Collector can be changed on a live collector. To edit these parameters, select Actions and choose Edit Collector Configuration from a collector details page. Editable fields include the hostname, proxy settings, and NTP server. If you require a change to the network interface configuration, it is required to provision a new XDR Collector.
Edit Collector Details
Edit Collector Configuration
After submitting a XDR Collector configuration change, a banner will appear to indicate that the change is pending and the edit action will no longer be available until the change has completed. The pending changes will be pushed to your XDR Collector where they will be applied and connectivity testing conducted.
Edit Collector Configuration Pending
If the pending changes result in the XDR Collector no longer able to successfully connect, the change will be rolled back to the previous configuration and a failure message will appear in the banner.
Edit Collector Configuration Rolled Back
If the change is successful, you will receive a successful message in the banner once the change has completed.
Edit Collector Configuration Success
In rare circumstances, it's possible that the configuration change and rollback are both unsuccessful. Example scenarios include, but are not limited to, changes to the underlying network during the change or network connectivity failures to the backend during an inflight change. In these circumstances, you will see a failure banner and the XDR Collector will need to be redeployed.
Edit Collector Configuration Failed
Once the change is complete, you should download the new Azure ARM Template and apply the updates to your current deployment. This will ensure that your new configuration will persist should a new instance be created.
Manage Integrations Collector Downloads