PXE Boot Failure Troubleshooting Guide: An Insider's Perspective

Introduction: The PXE Boot Chain

Preboot Execution Environment (PXE) boot is a cornerstone of modern, automated IT infrastructure, enabling the network-based deployment of operating systems. From a sysadmin's perspective, a successful PXE boot is a delicate chain of events involving firmware, network hardware, DHCP/TFTP/HTTP protocols, and server configuration. A failure at any point breaks the entire process. This guide cuts through the noise, providing a problem-oriented, rapid-diagnosis path based on common failure patterns in enterprise environments.

Problem 1: Client Fails to Obtain IP Address (DHCP Discovery/Offer Failure)

Symptoms: The PXE client hangs at "DHCP..." or similar messages, eventually timing out with an error like "PXE-E51: No DHCP or proxyDHCP offers were received." The client may show no link light on the NIC.

Diagnosis & Resolution Path:

1. Physical Layer Check: Verify network cable and switch port. Ensure the switch port is configured for the correct VLAN and is not administratively shut down. For wireless, PXE is generally not supported.
2. DHCP Server Reachability: Confirm the DHCP server is running and its service is bound to the correct network interface. Check firewall rules (often iptables or firewalld on Linux) on the DHCP server to ensure UDP ports 67 (bootps) and 68 (bootpc) are open.
3. DHCP Scope Configuration: Inspect the DHCP scope options. PXE requires specific options:
   • Option 66 (Boot Server Host Name): Must point to your TFTP server's IP address.
   • Option 67 (Bootfile Name): Must correctly point to the initial bootloader file (e.g., undionly.kpxe, ipxe.efi, grubx64.efi). The path is relative to the TFTP root.
   For UEFI clients, ensure you are serving the correct UEFI bootloader. A common insider mistake is configuring Option 67 for a legacy BIOS PXE client (e.g., undionly.kpxe) while the hardware is in UEFI mode, which requires a different file (e.g., ipxe.efi).
4. ProxyDHCP (if used): If using a separate PXE/proxyDHCP server (like WDS or a specialized PXE server), verify it is running and can communicate with the client's subnet. Use a network sniffer (e.g., tcpdump -i eth0 port 67 or port 68) on the server to see if DHCP packets are arriving.

When to Seek Professional Help: If the DHCP server is part of a complex, distributed Active Directory or heavily segmented network where you lack administrative control over routers or DHCP relay agents.

Problem 2: Client Receives IP but Fails to Download Boot Files (TFTP/HTTP Failure)

Symptoms: Client gets an IP address but fails with errors like "PXE-E32: TFTP open timeout," "PXE-T01: File not found," or "TFTP error: Access violation." It may also hang after displaying the bootloader filename.

Diagnosis & Resolution Path:

1. TFTP Server Daemon: Ensure the TFTP daemon (e.g., tftpd-hpa, atftpd) is running. Check its configuration file (e.g., /etc/default/tftpd-hpa) to confirm the TFTP_DIRECTORY (root) is set correctly and has proper permissions. The TFTP root directory and all files within (especially bootloader files and kernel images) must be world-readable (chmod 644).
2. Firewall (Again): The TFTP server uses UDP port 69. Ensure it's not blocked by a host-based or network firewall. A frequent oversight is forgetting that TFTP uses random high-numbered ports for data transfer after the initial connection; stateful firewalls usually handle this, but restrictive rules can block it.
3. File Path and Case Sensitivity: The path in DHCP Option 67 is case-sensitive and relative to the TFTP root. If Option 67 is set to pxelinux.0, the file must exist at $TFTP_ROOT/pxelinux.0. Verify the file's presence and spelling.
4. Switching to HTTP: Modern bootloaders like iPXE can chainload to HTTP, which is more reliable and faster than TFTP for large files. If TFTP issues persist, configure your bootloader (e.g., ipxe.efi) to fetch the kernel and initrd from an HTTP server (like Nginx or Apache). Ensure the web server is running and the file paths in your boot configuration (e.g., pxelinux.cfg/default or iPXE script) are correct.
5. File Size & Block Size: Some legacy TFTP clients have issues with large files or default block sizes. You can adjust the block size in your TFTP server configuration (e.g., --blocksize 1468 for tftpd-hpa).

When to Seek Professional Help: When dealing with proprietary hardware that has buggy or non-standard PXE ROM implementations requiring specific TFTP workarounds.

Problem 3: Bootloader Loads but Kernel/Image Fetch Fails

Symptoms: The initial bootloader (e.g., pxelinux.0, iPXE) loads and displays a menu, but selecting an entry fails to load the kernel (vmlinuz) or initrd, resulting in a kernel panic or network error.

Diagnosis & Resolution Path:

1. Boot Configuration File: Examine the bootloader's configuration file. For SYSLINUX/pxelinux, this is typically in $TFTP_ROOT/pxelinux.cfg/default or a client-specific file (like its IP in hex). Check the kernel and append initrd=... lines. Paths are usually relative to the TFTP root unless an absolute URI is used.
2. File Integrity and Permissions: Verify the kernel and initrd images exist at the specified paths. Download and verify their checksums. Ensure they are not corrupted and have read permissions.
3. Network Interface Naming: A critical, behind-the-scenes pitfall in Linux is inconsistent network interface naming. The kernel boot parameters (append line) might pass ip=dhcp or a static IP configuration expecting an interface like eth0, but the actual system may boot with an interface named ens192 or enp3s0. This causes the booted system to have no network. Use the ifname= kernel parameter or transition to predictable interface names in your deployment templates.
4. Root Filesystem Specification: If booting a live image or installer, ensure the root= parameter or the initrd configuration correctly points to the network location (e.g., root=live:http://server/path/to/squashfs) or the correct device.

When to Seek Professional Help: When troubleshooting custom-built initrd images that may lack necessary drivers or modules for your specific hardware (e.g., RAID or special NIC drivers).

Prevention and Best Practices

1. Environment Segmentation: Use a dedicated VLAN and subnet for PXE booting during deployment phases. This isolates broadcast traffic and simplifies firewall rules. Implement DHCP snooping on switches to prevent rogue DHCP server interference.

2. Configuration as Code: Manage your TFTP root directory, boot configurations, and DHCP scope options using version control (like Git) and configuration management tools (Ansible, Puppet, Chef). This ensures consistency, provides rollback capability, and documents changes.

3. Dual-Stack Boot Servers: Maintain separate bootloader files and configurations for legacy BIOS (PXE) and UEFI clients. Use DHCP options 60 (Client-Architecture) and 93 (Client System Architecture) or a single, smart bootloader like iPXE that can handle both.

4. Robust Infrastructure: Prefer HTTP over TFTP for serving large kernel and image files. Implement monitoring for your TFTP/HTTP/DHCP services. Use centralized logging (e.g., syslog) to capture server-side daemon messages for audit trails.

5. Comprehensive Testing: Maintain a physical or virtual "golden client" for each major hardware model/type in your fleet. Test any change to the PXE infrastructure against these clients before rolling out to production. Document known hardware quirks (e.g., certain NICs requiring specific driver options in the kernel command line).

6. Clear Documentation: Beyond this guide, maintain an internal runbook that details your specific network topology, server IPs, configuration file locations, and recovery procedures for complete PXE service failure.