F5 Troubleshoot

F5 BIG-IP Troubleshooting

Troubleshooting Principles

1. Define the problem -- What exactly is broken? Who reported it? What is the expected vs actual behavior?
2. Gather facts -- List objects, check stats, read logs. Never assume.
3. Consider possibilities -- Based on facts, list likely root causes
4. Create action plan -- Test one variable at a time
5. Implement and verify -- Make one change, verify, document
6. Document -- Record what was found and what fixed it

How to Call the Tools

The F5 MCP server provides 6 tools. Call them via mcp-call with the required environment variables:

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" <tool_name> '{"param":"value"}'

Available Tools for Troubleshooting

Symptom: "Virtual Server Not Responding to Clients"

Clients report they cannot connect to the application VIP.

Step 1: Verify Virtual Server Exists and Is Enabled

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" list_tool '{"object_name":"vs_webapp_https","object_type":"virtual"}'

Check:

• Does the virtual server exist? If not, it was deleted or never created.
• Is it enabled: true? If disabled, someone took it out of service.
• Is the destination (VIP:port) correct?
• Is a pool assigned?
• Is sourceAddressTranslation configured? (Without SNAT/automap, return traffic may bypass the BIG-IP.)

Decision tree:

• VS does not exist -> Recreate it (use f5-config-mgmt skill)
• VS is disabled -> Re-enable: update_tool with {"enabled":true}
• VS has no pool -> Assign pool: update_tool with {"pool":"pool_name"}
• VS has no SNAT -> Check if servers have BIG-IP as default gateway; if not, add automap

Step 2: Check Virtual Server Statistics

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" show_stats_tool '{"object_name":"vs_webapp_https","object_type":"virtual"}'

Analyze:

If status is offline: The virtual server is marked down because the associated pool has no available members. Proceed to Step 3.

If current connections = 0 but status is available: The VIP is healthy but no clients are connecting. The issue is upstream of the BIG-IP:

• DNS not resolving to the VIP address
• Firewall blocking traffic to the VIP
• Client network routing issue
• VIP is on wrong VLAN/subnet

Step 3: Check the Associated Pool

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" list_tool '{"object_name":"pool_webapp","object_type":"pool"}'

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" show_stats_tool '{"object_name":"pool_webapp","object_type":"pool"}'

Check:

• Are any members available? If all members are offline, the pool is down.
• What monitor is assigned? Is it appropriate for the service?
• Are members enabled or disabled? Disabled members were intentionally drained.
• What is the member-to-connection distribution? Is one member handling all traffic?

If all members are offline -> Go to "Pool Member Marked Down" section below.

Step 4: Check Logs for Errors

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" show_logs_tool '{"lines_number":"200"}'

Scan for:

• 01010028 -- No members available in pool (confirms pool down)
• 01010025 -- Connection limit reached on virtual server
• 0107142f -- SSL handshake failure
• 01070417 -- HTTP parse error
• 01010240 -- Connection queue full
• Timestamps correlating with the reported outage

Step 5: Check Profiles and iRules

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" list_tool '{"object_name":"","object_type":"profile"}'

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" list_tool '{"object_name":"","object_type":"irule"}'

Check:

• Is the correct SSL profile assigned for HTTPS virtual servers?
• Is the HTTP profile assigned when HTTP inspection is needed?
• Are any iRules rejecting or redirecting traffic incorrectly?
• Is a persistence profile causing traffic to stick to a down member?

Symptom: "Pool Member Marked Down"

Health monitor is marking one or more pool members as offline.

Step 1: Identify Which Members Are Down

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" list_tool '{"object_name":"pool_webapp","object_type":"pool"}'

Record: Which members are offline, which are available, which are disabled.

Step 2: Check Pool Statistics for the Down Member

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" show_stats_tool '{"object_name":"pool_webapp","object_type":"pool"}'

Analyze:

• When did the member go down? (Check stats timestamps)
• Was there a gradual decline or sudden failure?
• Are connections draining from the down member?

Step 3: Check Logs for Monitor Failure Details

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" show_logs_tool '{"lines_number":"500"}'

Scan for these patterns:

Common root causes for pool member down:

1. Server is actually down -- The application crashed, the OS is down, or the server was rebooted
2. Network path issue -- Firewall between BIG-IP and server blocking health check traffic, or routing issue on server VLAN
3. Monitor mismatch -- HTTP monitor expecting 200 but application returns 301/302 redirect
4. Monitor URI wrong -- Health check URI returns 404 because the page does not exist
5. Port mismatch -- Monitor checking wrong port (e.g., monitor on 80 but server on 8080)
6. SSL mismatch -- HTTP monitor used but server requires HTTPS (or vice versa)
7. Response timeout -- Server responds but too slowly for the monitor interval/timeout
8. Receive string mismatch -- Monitor expects specific string in response that changed after app deployment
9. Source IP issue -- Server firewall blocking the BIG-IP self-IP used for health checks

Step 4: Verify Monitor Configuration

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" list_tool '{"object_name":"pool_webapp","object_type":"pool"}'

From the pool config, identify the monitor name and verify:

• Type: HTTP, HTTPS, TCP, ICMP, or custom
• Interval/timeout: Is the timeout shorter than the interval? (Must be: timeout < interval * 3+1 for 3 failures)
• Send string: What request is sent? (e.g., GET /health HTTP/1.1\r\nHost: app.example.com\r\n\r\n)
• Receive string: What response is expected? (e.g., 200 OK or healthy)
• Destination: Is it *:* (use member address:port) or a specific IP:port?

Step 5: Remediation

If the server is healthy but the monitor is wrong, fix the monitor:

Update the pool with a correct monitor:

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" update_tool '{"url_body":{"monitor":"tcp"},"object_type":"pool","object_name":"pool_webapp"}'

If a member needs to be temporarily removed (graceful drain):

Update the pool without the problematic member:

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" update_tool '{"url_body":{"members":["10.1.1.10:80","10.1.1.11:80"]},"object_type":"pool","object_name":"pool_webapp"}'

WARNING: This removes the member entirely. Existing connections will be terminated. For graceful drain, disable the member instead if the API supports it.

If a replacement member needs to be added:

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" update_tool '{"url_body":{"members":["10.1.1.10:80","10.1.1.11:80","10.1.1.14:80"]},"object_type":"pool","object_name":"pool_webapp"}'

Symptom: "Connection Limits / Persistence Issues"

Users report intermittent connectivity, session drops, or being load-balanced to a different server mid-session.

Step 1: Check Virtual Server Connection Statistics

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" show_stats_tool '{"object_name":"vs_webapp_https","object_type":"virtual"}'

Check for connection limit issues:

• Is connectionLimit set and being reached?
• Are clientsideCurConns near the limit?
• Is the connection queue filling up? (Check logs for 01010240)

If connection limit is being hit:

Either increase the limit or scale out with additional pool members:

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" update_tool '{"url_body":{"connectionLimit":0},"object_type":"virtual","object_name":"vs_webapp_https"}'

Setting connectionLimit to 0 removes the limit entirely.

Step 2: Check Persistence Configuration

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" list_tool '{"object_name":"vs_webapp_https","object_type":"virtual"}'

Persistence troubleshooting:

Step 3: Check Pool Member Connection Distribution

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" show_stats_tool '{"object_name":"pool_webapp","object_type":"pool"}'

If one member has vastly more connections than others:

• Persistence is sticking too many sessions to one member
• Consider changing from source-address to cookie persistence
• Consider changing load balancing method from round-robin to least-connections

Step 4: Check Logs for Connection Errors

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" show_logs_tool '{"lines_number":"300"}'

Scan for:

• 01010025 -- Connection limit reached
• 01010240 -- Connection queue full
• 01060102 -- Rate limit reached
• TCL error -- iRule causing connection drops
• reset cause -- Connection resets (RST) from server or BIG-IP

Symptom: "SSL/TLS Certificate Problems"

Users see certificate warnings, SSL handshake failures, or HTTPS connections fail entirely.

Step 1: Check SSL Profile Configuration

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" list_tool '{"object_name":"","object_type":"profile"}'

Check the SSL client profile assigned to the virtual server:

• Is a client SSL profile assigned? (Required for HTTPS VIPs)
• Which certificate and key are referenced?
• What TLS versions are enabled? (TLS 1.2 and 1.3 should be enabled; TLS 1.0 and 1.1 should be disabled)
• What cipher suites are configured?

Common SSL issues:

Step 2: Check Virtual Server for SSL Profile

IP_ADDRESS=$F5_IP_ADDRESS Authorization_string=$F5_AUTH_STRING python3 $MCP_CALL "python3 -u $F5_MCP_SCRIPT" list_tool '{"object_name":"vs_webapp_https","object_type":"virtual"}'

Verify the correct SSL profile is assigned in the profiles list with context: clientside.