Most “the server is down” reports are really connectivity problems. Work from the bottom up (interface, address, route, DNS, port, application) and each step tells you where to stop. These tools replaced ifconfig, netstat and route, which are deprecated in the net-tools package.
Versions in use on this date. Ubuntu 18.04.3 LTS uses Netplan with systemd-networkd or NetworkManager as renderers and iproute2 4.15. CentOS 7.6 still uses the network scripts in /etc/sysconfig/network-scripts/. (CentOS 7.7 is due within days; nothing here changes.)
Troubleshooting path
flowchart TB
S[Cannot reach service] --> L1{Link up?<br/>ip link}
L1 -->|no| F1[Cable or virtual NIC problem]
L1 -->|yes| L2{Has address?<br/>ip addr}
L2 -->|no| F2[DHCP or static config]
L2 -->|yes| L3{Default route and gateway ping?<br/>ip route}
L3 -->|no| F3[Routing or gateway problem]
L3 -->|yes| L4{Name resolves?<br/>dig}
L4 -->|no| F4[DNS settings]
L4 -->|yes| L5{Port open?<br/>nc or ss}
L5 -->|no| F5[Service down or firewall]
L5 -->|yes| L6[Application layer<br/>curl logs]
Interfaces, addresses and routes
ip -br link # interface states in brief
ip -br addr # addresses
ip route # routing table; "default via ..." is the gateway
ip route get 8.8.8.8 # which route and source address would be used
ping -c 3 192.168.1.1 # gateway
ping -c 3 8.8.8.8 # internet by IP (separates routing from DNS)
traceroute -n example.com # path (apt install traceroute)
Pinging an IP that works while a name fails points to DNS. If neither works, look at the address and gateway first.
DNS
dig example.com +short
dig example.com A @8.8.8.8 # ask a specific resolver
dig -x 8.8.8.8 +short # reverse lookup
dig example.com MX +short
cat /etc/resolv.conf # on Ubuntu 18.04 points to 127.0.0.53 (systemd-resolved)
systemd-resolve --status | grep -A3 'DNS Servers'
getent hosts example.com # what applications really see (uses nsswitch and /etc/hosts)
A name that works in dig but not in an application usually means /etc/hosts, nsswitch.conf or a search-domain problem; getent shows the application’s view.
Ports and sockets
sudo ss -tulpn # listening TCP/UDP sockets with owning process
ss -s # summary of connections
ss -tn state established '( dport = :443 )' | head
nc -zv db1 5432 # can I reach that port? (netcat)
curl -I https://example.com # HTTP response headers
curl -v telnet://host:25 # raw connection test
Local Address of 127.0.0.1:5432 means the service accepts local connections only; 0.0.0.0:5432 or *:5432 accepts from all interfaces. This is the first thing to check when a service “works on the server but not from outside”.
Static IP on Ubuntu 18.04 with Netplan
# /etc/netplan/01-netcfg.yaml
network:
version: 2
renderer: networkd
ethernets:
ens3:
dhcp4: no
addresses: [192.168.1.50/24]
gateway4: 192.168.1.1
nameservers:
addresses: [8.8.8.8, 1.1.1.1]
sudo netplan try # applies, then reverts in 120 s unless you confirm
sudo netplan apply
netplan try protects you from locking yourself out over SSH. YAML needs spaces, not tabs.
Static IP on CentOS 7
# /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE=Ethernet
BOOTPROTO=none
NAME=eth0
DEVICE=eth0
ONBOOT=yes
IPADDR=192.168.1.51
PREFIX=24
GATEWAY=192.168.1.1
DNS1=8.8.8.8
sudo systemctl restart network
Temporary changes made with ip addr add are lost at reboot; make permanent changes in these files. Set the hostname with sudo hostnamectl set-hostname web01 and keep /etc/hosts consistent.
Capturing traffic
sudo tcpdump -i any -nn port 53 -c 20 # DNS queries
sudo tcpdump -i eth0 host 10.0.0.5 and port 443 -w capture.pcap
Capture on the server when you need proof of whether packets arrive; the strace and tcpdump post later in this series goes further.