Taegis Endpoint Agent for Linux Troubleshooting

integrations endpoints edr taegis agent secureworks

This document provides guidance on initial agent troubleshooting steps you can take and information you can gather prior to reaching out to Secureworks support for assistance with agent issues.

Support Scripts ⫘

There are two support scripts for the Taegis™ XDR Endpoint Agent for Linux: linux_sysinfo.sh and supportScript.sh. See the following sections for an overview of each.

linux_sysinfo.sh ⫘

Download script here: linux_sysinfo.sh.

linux_sysinfo.sh is a script to assist with gathering important information on a system potentially in need of support. In order for all the information to be gathered, it needs to be run as root. There is currently only one option available for this script to manipulate the time that strace runs on the system.

Command	Description
./linux_sysinfo.sh	Default strace = 3 seconds
./linux_sysinfo.sh --strace-secs <secs>	strace time =

Output for running this script can be redirected to a file using the following command: ./linux_sysinfo.sh > <file-name>.

Agent process information collection depends on if the agent is running. If the agent is not running, strace information will not be recorded and stack information related to the agent pid will not be dumped. If the agent is running, strace and stack information will be recorded. Strace information will be located in the same location that the script is run in a file with the name straceTaegisLog.txt. Stack information will be printed to the console or redirected to a file depending on if the file redirection command was used.

Regardless if the agent is running, verify that statistical information exists under each of the following headers after the script is run. This is currently the only snapshot information this script gathers.

#"=========== Hardware Configuration ============"
#"=========== CPU Info =========================="
#"=========== Mounts ============================"
#"=========== OS Identification ================="
#"=========== lsb_release ======================="
#"=========== Uname ============================="
#"=========== Kernel Version ===================="
#"=========== Logged In Users ==================="
#"=========== Active Processes =================="
#"=========== Cpu Usage % ======================="
#"=========== Memory Usage % ===================="
#"=========== Network Connections ==============="
#"=========== Crashes ==========================="
#"=========== Agent Configuration ==============="

supportScript.sh ⫘

Download script here: supportScript.sh.

supportScript.sh is a script to assist the Linux agent with installation. It checks that all the required files and directories exist on the current system while also enforcing that permissions are properly set for each file and directory. If the permissions are improperly set or a file is missing, the script will fail and inform you of the installation error. This script needs to be run as root. There are currently four commands for this script.

Command	Description
./supportScript.sh pre-registerCheck	Check agent file locations and permissions before any taegistctl commands are run
./supportScript.sh post-registerCheck	Check agent file locations and permissions after taegisctl register is run
./supportScript.sh post-startCheck	Check agent file locations and permissions after tageisctl start is run
./supportScript.sh service-status	Echo the status of agent services

Connectivity Issues ⫘

• Verify the agent's Connection Status from the Endpoint Agents Summary table of Endpoint Agents in XDR.
• Ensure connectivity requirements are met by allowing communication to the domains through any firewalls.
• Incorrect registration details may have been presented. Check the registration key and server for any unintended white spaces.
• Is this a cloned device from a prior registered endpoint? If so, it may be considered duplicate and is being rejected. We recommend you uninstall and reinstall the agent with the correct registration details.

Installation ⫘

• Ensure rpm or deb package is not corrupt. Verify the checksum matches what is available in XDR.
• Ensure the package has correct file permissions.
• Ensure the user is able to perform installations.
• Examples of failure messages you may receive during registration include:

Connection error:

2022-04-07 17:36:23.167 E [T:3562] 15 17d46:320 Connection unsuccessful
2022-04-07 17:36:23.167 E [T:3562] 15 17d46:178 Registration failed

Invalid registration key:

2022-05-31 16:58:25.389 E [T:29653] 15 17d46:345 https://reg.d.taegiscloud.com/agent-register/v1/register 400 {"message":"invalid registration_key"}
2022-05-31 16:58:25.408 E [T:29653] 15 17d46:178 Registration failed

SELinux configuration:

[user@localhost ~]$ sudo /opt/secureworks/taegis-agent/bin/taegisctl register
SELinux is in Enforcing mode; exiting.

If this happens, remember to include the --allow_enforcing switch to taegisctl register. For more information, see SELinux/AppArmor and the Agent.

Auto Upgrade Failures ⫘

• Provide updater log: <install_path>/taegis-agent/log/updater.log.
• Check if taegis-update service is running: <install_path>/taegis-agent/bin/taegisctl status.
• Allow taegis-agent-prod-builds.s3.us-east-2.amazonaws.com through firewalls.

Performance Issues ⫘

In order to troubleshoot performance issues like CPU, memory spike, and application crashing, provide Secureworks Support with the following information and logs.

Provide the following Information ⫘

• The hostname of affected system
• The role and function of the endpoint
• Whether it is a virtual machine or running on physical hardware
• The version the agent is running
• Applications running on the endpoint
• A description of the performance issues encountered on the endpoint
• OS and kernel information of the endpoint - output of the command uname -a
• Output of the command top with Irix mode off (run top command and press Shift + i)
• Output of the command cat /proc/cpuinfo
• Output of the command free -m
• Output of the command service --status-all | more
• Share the output of the following script: linux_sysinfo.sh; see Support Scripts for more information

• The agent.log file located at <install_path>/secureworks/taegis-agent/log/

  Note

  Default installation path: /opt/secureworks/taegis-agent/

• Output of a diagnostics report on the affected system to verify Taegis Endpoint Agent service status and network connectivity from within the following location:

  <install_path>/secureworks/taegis-agent/etc/agent_diagnostic_report

  

  Taegis Agent Diagnostics Report

• Output of the command collect_perf to collect performance metrics from the affected system while reproducing the performance issue. The output will be found in the following directory:

  <install_path>/secureworks/taegis-agent/log/

Debug Logging ⫘

Turning on debug logging can help you collect further information from the affected system.

1. On the affected system edit the scwx_agent.json file in /etc/scwx_agent.json and set logging level D for debug mode. Save the file.

Set Debug Logging Level

2. Run the command, cat /etc/scwx_agent.json to verify logging level is set to debug mode (D)

3. Restart the Taegis Endpoint Agent service using the below commands:

sudo /<install_path>/secureworks/taegis-agent/log/bin/taegisctl stop

sudo /<install_path>/secureworks/taegis-agent/log/bin/taegisctl start

4. Reproduce the performance issues for a 10 minute window. During this time, collect the output of the command:

top -H -p $(pidof taegis)

5. After the 10 minute window, provide all log files from the following directory: /<install_path>/secureworks/taegis-agent/log/

Once all information from the affected endpoint has been collected, remove the scwx_agent.json file from /etc/ directory and restart the Taegis Endpoint Agent service to remove it from debug mode.

Provide all the captured information from the affected system(s) to via a support ticket to Product Support.

Service Not Starting ⫘

• Run <install_path>/taegis-agent/bin/taegisctl status to check if driver was loaded; for example:

sudo /opt/secureworks/taegis-agent/bin/taegisctl status

Agent Service Status    :  running   
Updater Service Status  :  running
Driver Loaded           :  true  
Agent is Registered     :  true     
Sink URL                :  wss://sink.c.taegiscloud.com:8443/ws

• If driver was loaded, run <install_path>/taegis-agent/bin/taegisctl start again. If service is still not running, get output of journalctl -xe and log: <install_path>/taegis-agent/log/agent.log.

• If driver is not available, run <install_path>/taegis-agent/bin/taegisctl register [--key <regkey>] [--server <servername>] [--allow_missing_driver] with the same registration details as before. The flag --allow_missing_driver at the end allows service to start despite not having a driver available to load. Agent is designed to start by default if and only if a driver is available and loaded.

Uninstall ⫘

Typical issues are due to the user not having the right privilege to perform uninstall operations. Ensure user has an elevated role to perform uninstall.