Network Analysis

Network Analysis Skill

This skill provides detailed guidance for analyzing network configuration and connectivity from sosreport archives, including interfaces, routing, firewall rules, and DNS configuration.

When to Use This Skill

Use this skill when:

• Analyzing the /sosreport:analyze command's network analysis phase
• Investigating network connectivity issues
• Diagnosing firewall or routing problems
• Verifying network configuration

Prerequisites

• Sosreport archive must be extracted to a working directory
• Path to the sosreport root directory must be known
• Understanding of Linux networking concepts

Key Network Data Locations in Sosreport

1. Network Interfaces:

   • sos_commands/networking/ip_-o_addr - IP addresses
   • sos_commands/networking/ip_link - Link status
   • sos_commands/networking/ip_-s_link - Link statistics with errors
   • etc/sysconfig/network-scripts/ - Network configuration files (RHEL)

2. Routing:

   • sos_commands/networking/ip_route - Routing table
   • sos_commands/networking/ip_-6_route - IPv6 routing table
   • proc/net/route - Kernel routing table

3. Network Connections:

   • sos_commands/networking/netstat_-neopa - Active connections
   • sos_commands/networking/ss_-tupna - Socket statistics
   • proc/net/tcp - TCP connections
   • proc/net/udp - UDP connections

4. Firewall:

   • sos_commands/firewalld/ - Firewalld configuration
   • sos_commands/iptables/iptables_-vnxL - iptables rules
   • sos_commands/nftables/ - nftables configuration

5. DNS and Resolution:

   • etc/resolv.conf - DNS servers
   • etc/hosts - Static hostname mappings
   • etc/nsswitch.conf - Name resolution order

6. Network Services:

   • sos_commands/networking/networkmanager_info - NetworkManager status
   • systemctl status NetworkManager output

Implementation Steps

Step 1: Analyze Network Interfaces

1. List all network interfaces:

   if [ -f sos_commands/networking/ip_-o_addr ]; then
     cat sos_commands/networking/ip_-o_addr
   fi

2. Check interface states:

   if [ -f sos_commands/networking/ip_link ]; then
     # Look for interface states (UP/DOWN)
     grep -E "^[0-9]+:" sos_commands/networking/ip_link
   fi

3. Parse interface information:

   • Interface name (eth0, ens192, etc.)
   • State (UP/DOWN)
   • IP addresses (IPv4 and IPv6)
   • MAC address
   • MTU size

4. Check for interface errors:

   if [ -f sos_commands/networking/ip_-s_link ]; then
     # Look for RX/TX errors, drops, overruns
     cat sos_commands/networking/ip_-s_link
   fi

5. Identify interface issues:

   • Interfaces with no IP address (when expected)
   • Interfaces in DOWN state (when should be UP)
   • High error counts (RX/TX errors, drops)
   • Duplicate IP addresses
   • MTU mismatches

Step 2: Analyze Routing Configuration

1. Check default route:

   if [ -f sos_commands/networking/ip_route ]; then
     grep "^default" sos_commands/networking/ip_route || echo "No default route found"
   fi

2. Review routing table:

   if [ -f sos_commands/networking/ip_route ]; then
     cat sos_commands/networking/ip_route
   fi

3. Check IPv6 routing:

   if [ -f sos_commands/networking/ip_-6_route ]; then
     cat sos_commands/networking/ip_-6_route
   fi

4. Identify routing issues:

   • Missing default route
   • Multiple default routes (conflicting)
   • Incorrect gateway addresses
   • Route to nowhere (unreachable gateway)

Step 3: Analyze Network Connectivity

1. Check active connections:

   if [ -f sos_commands/networking/netstat_-neopa ]; then
     cat sos_commands/networking/netstat_-neopa
   elif [ -f sos_commands/networking/ss_-tupna ]; then
     cat sos_commands/networking/ss_-tupna
   fi

2. Count connections by state:

   # Count TCP connection states
   if [ -f sos_commands/networking/netstat_-neopa ]; then
     grep "^tcp" sos_commands/networking/netstat_-neopa | awk '{print $6}' | sort | uniq -c
   fi

3. Find listening services:

   # Show what's listening on which ports
   if [ -f sos_commands/networking/netstat_-neopa ]; then
     grep "LISTEN" sos_commands/networking/netstat_-neopa
   fi

4. Check for connection issues:

   • Excessive TIME_WAIT connections
   • Many connections in SYN_SENT (connection attempts failing)
   • High number of CLOSE_WAIT (application not closing)
   • Port conflicts (multiple services on same port)

Step 4: Analyze Firewall Configuration

1. Check if firewalld is active:

   if [ -d sos_commands/firewalld ]; then
     # Firewalld is present
     if [ -f sos_commands/firewalld/firewall-cmd_--list-all-zones ]; then
       cat sos_commands/firewalld/firewall-cmd_--list-all-zones
     fi
   fi

2. Review iptables rules:

   if [ -f sos_commands/iptables/iptables_-vnxL ]; then
     cat sos_commands/iptables/iptables_-vnxL
   fi

3. Check firewall zones and rules:

   • Active zones
   • Allowed services
   • Allowed ports
   • Rich rules
   • Drop/reject policies

4. Identify firewall issues:

   • Required ports blocked
   • Overly permissive rules (any any accept)
   • Conflicting rules
   • Missing rules for services

Step 5: Analyze DNS Configuration

1. Check DNS servers:

   if [ -f etc/resolv.conf ]; then
     cat etc/resolv.conf
   fi

2. Review /etc/hosts:

   if [ -f etc/hosts ]; then
     # Show non-comment, non-empty lines
     grep -v "^#\|^$" etc/hosts
   fi

3. Check hostname resolution:

   # Check hostname
   if [ -f hostname ]; then
     cat hostname
   fi
   
   # Check FQDN
   if [ -f etc/hostname ]; then
     cat etc/hostname
   fi

4. Verify nsswitch configuration:

   if [ -f etc/nsswitch.conf ]; then
     grep "^hosts:" etc/nsswitch.conf
   fi

5. Identify DNS issues:

   • No DNS servers configured
   • Unreachable DNS servers (check connectivity in logs)
   • Incorrect search domains
   • Hostname resolution failures in logs

Step 6: Check for Network Errors in Logs

1. Look for network-related errors:

   # Connection refused errors
   grep -i "connection refused" sos_commands/logs/journalctl_--no-pager 2>/dev/null | head -20
   
   # Timeout errors
   grep -i "timeout\|timed out" sos_commands/logs/journalctl_--no-pager 2>/dev/null | head -20
   
   # Network unreachable
   grep -i "network.*unreachable\|no route to host" sos_commands/logs/journalctl_--no-pager 2>/dev/null | head -20
   
   # DNS resolution failures
   grep -i "could not resolve\|dns.*fail\|name resolution" sos_commands/logs/journalctl_--no-pager 2>/dev/null | head -20

2. Check for link state changes:

   grep -i "link.*up\|link.*down\|carrier.*lost" sos_commands/logs/journalctl_--no-pager 2>/dev/null | head -20

3. Look for network device errors:

   grep -i "network.*error\|eth[0-9].*error\|transmit.*error" var/log/dmesg 2>/dev/null