Domain 5: Troubleshooting (22%)¶

Overview¶

The second-largest domain asks you to diagnose. Questions give symptoms and logs, and you pick the cause or the next command. The skill is a method: define the symptom, find the layer that is failing, check the obvious and cheap things first, change one thing at a time, and confirm the fix. Almost every symptom here maps to something you configured in Domains 1 to 3, so this domain rewards understanding over memorizing.

• 5.1 Monitoring concepts and configurations.
• 5.2 Hardware, storage and OS issues.
• 5.3 Networking issues.
• 5.4 Security issues.
• 5.5 Performance issues.

📖 CompTIA Linux+ XK0-006 Exam Objectives (PDF, version 5.0) - The official list of every topic and tool named in this domain

5.1 Monitoring¶

Monitoring is about knowing a service is in trouble before users tell you.

• SLI (service-level indicator) is a measurement, such as the fraction of successful requests. SLO (objective) is the target for it, such as 99.9% over 30 days. SLA (agreement) is the contractual promise to a customer, with consequences if missed. The order to remember: the SLI is measured, the SLO is the internal goal, the SLA is the contract.
• Data acquisition: SNMP polls devices or receives traps (unsolicited alerts); a MIB defines the objects a device exposes. Monitoring can be agent-based (software on the host sends data) or agentless (the monitor polls over SNMP, SSH or an API).
• Health checks test that a service answers correctly, for example an HTTP endpoint returning 200. Webhooks push event notifications to another system. Log aggregation ships logs from many hosts to one searchable place.
• Thresholds define when a metric is abnormal; crossing one raises an alert or event that sends a notification (email, chat, pager). Good thresholds are based on a measured baseline, not a guess. Alerts that fire constantly get ignored.

5.2 Hardware, storage and OS issues¶

Work from the symptom to the layer, using the logs and tools from earlier domains.

Rescue approach for a system that will not boot: boot a rescue target (systemd.unit=rescue.target or emergency.target on the kernel line) or a live image, mount the root filesystem, check fstab and logs, and fix the configuration.

5.3 Networking issues¶

A layered method keeps you out of trouble: link, address, route, name, service, firewall.

ip link show                 # is the interface up, and is there a carrier?
ip -br address               # does it have the expected address?
ip route                     # is there a default route?
ping -c3 192.0.2.1           # can I reach the gateway?
ping -c3 198.51.100.10       # can I reach beyond it by address?
dig example.com              # does name resolution work?
ss -tlnp                     # is the service listening, and on what address?
tracepath example.com        # where does the path stop?
tcpdump -i eth0 port 53      # are packets actually arriving and leaving?

5.4 Security issues¶

5.5 Performance issues¶

Performance work starts from a baseline of normal behavior; "slow" only means something compared with normal. Identify the constrained resource first: CPU, memory, disk I/O or network.

Reading load average. The three numbers from uptime are the average number of runnable and uninterruptible tasks over 1, 5 and 15 minutes. Compare to the CPU count (nproc): a load of 8 on a 4-core machine is oversubscribed. High load with low CPU use and high I/O wait means processes are blocked on disk or network, not computing. Swapping (nonzero si/so in vmstat) means the system ran out of RAM and is paging, which makes everything slow.

A method: note the symptom and when it started, check uptime and top for the constrained resource, check dmesg and the journal for errors, change a single variable, and compare against the baseline. Hardware errors (dmesg, SMART) should be ruled out before tuning software.

Common exam traps¶

• A failed ping does not prove a host is down; ICMP may be blocked.
• "No space left on device" with free blocks means inodes (df -i) or a quota.
• Deleted-but-open files hold space until the process is restarted.
• Ping by IP working while names fail is a DNS problem.
• Group changes need a new login session.
• High load average with idle CPU points at I/O wait.
• Fix SELinux with context or boolean changes, not by disabling it.

Key terms¶

• Kernel panic - a fatal kernel error from which the system cannot recover, often caused by drivers, a corrupt initrd or failing hardware.
• Inode exhaustion - running out of inodes so no new files can be created even though free space remains.
• OOM killer - the kernel mechanism that terminates processes to free memory when the system runs out of RAM and swap.
• SMART - disk self-monitoring data, read with smartctl, that can reveal a failing drive.
• SLI - a service-level indicator, a measured value such as the share of successful requests.
• SLO - a service-level objective, the internal target set for an SLI.
• SLA - a service-level agreement, the contractual commitment to a customer about service levels.
• SNMP - the Simple Network Management Protocol, used to poll devices or receive alerts called traps.
• MIB - a management information base, which defines the objects an SNMP device exposes.
• Baseline - a record of normal performance used to judge whether current behavior is abnormal.
• Health check - a test that a service responds correctly, such as an HTTP endpoint returning 200.
• Log aggregation - collecting logs from many hosts into one searchable location.
• MTU mismatch - a difference in maximum packet size along a path that makes small packets work while large transfers hang.
• Link negotiation - the speed and duplex agreement between an interface and its switch port; mismatches cause errors.
• Dual stack - running IPv4 and IPv6 at the same time, where a broken IPv6 path can cause delays.
• Load average - the average number of runnable and uninterruptible tasks over 1, 5 and 15 minutes.
• I/O wait - the share of time the CPU sits idle because processes are waiting on disk or network I/O.
• Swapping - moving memory pages to disk because RAM is exhausted, which greatly slows the system.
• Context switching - the kernel changing the CPU from one task to another; very high rates add overhead.
• namei -l - lists the owner and permissions of every component in a path, revealing where access is blocked.
• faillock - shows and resets PAM failed-login counters that lock an account.
• cipher negotiation - the TLS or SSH step where client and server choose a mutually supported cipher suite.