Installing FreeBSD 8.x via serial console and PXE

— by Jeremy Chadwick <>

Table of Contents

1 Requirements
2 Common Paths/Terms Used
3 Installation
3.1 Configuring DHCP
3.2 Configuring TFTP
3.3 Configuring NFS
3.4 Extracting the FreeBSD CD contents for use via NFS
3.5 Rebuilding pxeboot for exclusive serial access
3.6 Configuring loader.conf
3.7 Fixing file permissions to work with TFTP and NFS
3.8 Step 8: Attempting your first PXE boot
3.9 Step 9: Choosing an installation medium
4 Creating /boot.config for high-speed serial
5 Important Networking options
6 Post-installation Configuration options

Chapter 1 Requirements

These are the things you are absolutely going to need, no questions asked. If you don't have any of these, you should stop here.

Note: This procedure works for i386 and amd64. There's no need to change any of the below pathnames for amd64. It also doesn't matter if you build the pxeboot(8) binary on an amd64 box while the machine that plans on using it is i386; the bootstraps are purely i386.

Chapter 2 Common Paths/Terms Used

Below are a list of terms, paths, directories, and filenames which are commonly used throughout this document.

Term Description The IP of the DHCP, TFTP, and NFS server The IP of the box which we're doing the PXE boot on/from
nfsserver.home.lan FQDN of
newbox.home.lan FQDN of
/usr/local/freebsd8 Directory containing contents of the FreeBSD 8.x "disc1" CD; you will be modifying some files in this directory, so it cannot be a mounted CD (e.g. read-only)
All of these are the different stages of the FreeBSD bootstraps. All are used when physical media is used for installation (e.g. floppies, CDROM, hard disk, USB thumb drive, etc.), but aren't used when PXE booting. loader(8) is technically used, but it's actually called pxeboot(8). See below for details on that
This is the sole binary that gets fetched via TFTP after the Intel PXE boot ROM gets an IP via DHCP. The binary itself is actually a modified version of loader(8), and includes some extra features (which we'll use)

Chapter 3 Installation

3.1 Configuring DHCP

I'm familiar with ISC's DHCP server, available as net/isc-dhcp41-server. The important pieces you need to add to your /usr/local/etc/dhcpd.conf are here:

host newbox.home.lan {
	  hardware ethernet 00:11:22:33:44:55;
	  fixed-address newbox.home.lan;
	  filename "freebsd8/boot/pxeboot";
	  option root-path "/usr/local/freebsd8";

Since not a single other document on the Web has described what the important options do, I'll describe their purpose:


This line isn't necessary if the NFS server and the DHCP server are on the same machine. Including it doesn't hurt.

filename "freebsd8/boot/pxeboot";

When a DHCP request is made from the Intel PXE boot ROM on the NIC, there are additional parameters you can tell it besides just an IP number. This argument tells the PXE boot ROM what filename to do a TFTP request for now that it has an IP address. It's the first piece of code which gets run after the initial DHCP IP negotiation is completed.

option root-path "/usr/local/freebsd8";

This is for FreeBSD. It tells pxeboot(8) where the root filesystem should be mounted from, NFS-wise. Thus, the kernel will do the equivalent of the following, in an attempt to get a working root filesystem, and then start init(8):

# mount -t nfs /

At this point, you should enable the ISC DHCP server in /etc/rc.conf, and start the daemon using the appropriate rc.d script in /usr/local/etc/rc.d.

3.2 Configuring TFTP

Uncomment the following line in /etc/inetd.conf, and modify it so that the -s flag points to /usr/local (that's where tftpd(8) will chroot(2) before serving content):

tftp    dgram   udp     wait    root    /usr/libexec/tftpd      tftpd -l -s /usr/local

Add the following to /etc/rc.conf (assuming it's not already there):


Now start the inetd service, thus starting tftpd(8):

# /etc/rc.d/inetd start

If you already had inetd running prior to modifying /etc/inetd.conf, all you need to do is send a SIGHUP to it:

# /etc/rc.d/inetd reload

3.3 Configuring NFS

Add the following to /etc/rc.conf, again assuming they're not already there:


Now we need to add an NFS export. Edit /etc/exports and add the following lines to it:

/usr/local/freebsd8    -network 192.168.1 -mask

This example assumes your network is; if it's something else, you'll need to make appropriate changes.

Next, don't forget to make the exported directory. Don't worry about the ownership or permissions, we'll fix those later:

# mkdir /usr/local/freebsd8

Now start all of the above services:

# /etc/rc.d/rpcbind start
# /etc/rc.d/mountd start
# /etc/rc.d/nfsd start

If these were running beforehand and you simply edited /etc/exports, all you need to do is send a SIGHUP to mountd(8): to get it to re-read the exports:

# /etc/rc.d/mountd reload

You should verify the NFS mount which was added is being exported:

# showmount -e
Exports list on localhost:

3.4 Extracting the FreeBSD CD contents for use via NFS

We're going to need to copy all the contents off the FreeBSD 8.x "disc1" CD to /usr/local/freebsd8 before continuing. In this example, the NFS server has the ISO file for 8.0-STABLE "disc1" on it, and we're going to extract the contents.

FreeBSD tar is linked to a library called libarchive, which understands the ISO9660 format and Joliet extensions. Simply put, this means you can extract the contents of an .iso file using tar:

# tar -C /usr/local/freebsd8 -pxvf 8.0-RELEASE-amd64-disc1.iso

3.5 Rebuilding pxeboot for exclusive serial access

FreeBSD's bootloader/bootstraps, by default, are intended to be used and viewed via a VGA console. When it comes to utilising a serial port, things get a little hairy:

First, serial access is limited to 9600bps tops, and requires that the bootloader be rebuilt to support higher speeds. Sure, you can override this using loader.conf, but that's for loader(8) and not for boot(2) or earlier stages.

Most people work around this by creating a file in their root filesystem called /boot.config that contains the argument -S115200 as well as -Dh (see the FreeBSD Handbook for details). This allows for serial output during the boot2 stage, as well as subsequent stages such as loader.

However, when PXE booting, there's only one piece of the bootstrap used: pxeboot(8). This means the only solution is to rebuild the boot blocks with a serial port speed that has the speed you want (in this case, 115200bps). But first, let's clear something up: pxeboot(8) is actually nothing more than loader(8) with some extra code (see src/sys/boot/i386/pxeldr.S) in it. loader(8) uses a library called libi386, which is a library used by the bootstraps only (it's it's not something you'll find in /usr/lib or /usr/libexec). libi386 is what actually controls what the serial port speed is set to, thus, anything using libi386 should have support for higher serial console speeds. The BOOT_COMCONSOLE_SPEED and BOOT_COMCONSOLE_PORT variables are used in libi386.

Secondly, pxeboot(8) reads loader.conf "too late" during the boot procedure, so if you have issues that require commands to be typed into the loader ("OK") prompt, you're out of luck. Rebuilding pxeboot(8) to use the serial port exclusively solves this.

On the NFS server, do the following:

# rm -fr /usr/obj/*
# cd /sys/boot

Note: Do not type make install after this step! We don't want to modify the bootstraps on the NFS server itself, we just want a 115200bps-capable pxeboot(8) binary that uses the serial port exclusively!

The removal of all contents within /usr/obj may seem crazy, but it's actually required for some of the below steps to work properly. If /usr/obj is populated from an old buildworld or buildkernel, the make will end up storing files there rather than in the current working directory.

The first parameter, BOOT_PXELDR_ALWAYS_SERIAL=1, explicitly causes all output to go to the serial port instead of the VGA console. This means that you'll get serial output early on (before loader.conf is read), which can really come in handy if things go awry. This option doesn't appear to be documented anywhere besides in the source code.

The second parameter, BOOT_COMCONSOLE_SPEED=115200, causes the serial port to be initialised at 115200bps. If you honestly prefer to use 9600bps, you don't have to specify this at all. This option is documented. For those somewhat familiar with serial consoles on FreeBSD: by specifying this parameter you won't need comconsole_speed in loader.conf.

Finally, there's another undocumented option specific to pxeboot(8) (and isn't part of libi386): BOOT_PXELDR_PROBE_KEYBOARD=1. This causes pxeboot(8) to to look for an attached PS/2 or AT keyboard, and if there is one, use the VGA console and local keyboard as the console device. Otherwise, it defaults to serial console.

Now that we have a 115200bps-capable pxeboot(8) binary, let's use it instead of the one originally on the CD image:

# cd /sys/boot/i386/pxeldr
# cp pxeboot /usr/local/freebsd8/boot

At this point, a PXE booting client should now have the capability to set the serial port speed to 115200bps, and should provide output early on during the booting process.

Finally, don't forget to clean up your /sys/boot directory; we don't want cruft laying around in there:

# cd /sys/boot
# make clean

3.6 Configuring loader.conf

Now we have to update /usr/local/freebsd8/boot/loader.conf to tell it to utilise serial console, and more importantly, point to where the root filesystem (for booting) will be. The lines in bold are the ones you want to add; the others shown should already be there (don't mess with them!):


vfs.root.mountfrom tells the kernel that after it's done booting, to try to load the root filesystem from ufs:/dev/md0 before starting init(8). It's important to note that the device parameter is /dev/md0 and not /dev/md0c like on FreeBSD 7.x. This is normal; there have been GEOM and libdisk improvements which no longer require the "c" slice to exist (you'll find this to be true when creating slices during sysinstall(8) too).

If you specify the wrong device, the root filesystem won't mount. If the root filesystem mounting procedure fails, FreeBSD will continue on and attempt to mount the root filesystem via NFS, which is not what we want. If you want to know more about this part of the booting procedure, you should look at the related Forth files, as well as loader.rc in the same directory.

The mfs_root loading mechanism is programmed to support gzip decompression automatically, so it will also look for the /boot/mfsroot.gz file. In FreeBSD 8.x this actually works, while in FreeBSD 7.x and earlier the gzip decompression would randomly fail.

3.7 Fixing file permissions to work with TFTP and NFS

We now need to fix the permissions on all the files we've extracted, as well as the permissions on the pxeboot(8) binary we copied. This is important for two reasons:

  1. tftpd(8), for some reason, requires that all files it reads (or writes) have their global read (or write) bit set. This is despite the fact that use of the -s or -u flags to tftpd(8) results in credentials being changed. I've verified this by looking at the tftpd(8) source code. In this situation, tftpd(8) will need to be able to read /usr/local/freebsd8/boot/pxeboot
  2. nfsd(8), for some reason, isn't mapping requests to user/group nobody:nobody. The only workaround I found was to fix the permissions on all files/directories to be globally readable. I've posted to freebsd-stable asking what the deal is

So here's what you have to do:

# find /usr/local/freebsd8 -type d -print0 | xargs -0 chmod 0755
# find /usr/local/freebsd8 -type f -print0 | xargs -0 chmod 0644

3.8 Attempting your first PXE boot

Alright, it's time to give it a try and see if it breaks! Be patient, since most of the TFTP loading is slow (it's UDP-based, and uses small UDP packets). If all goes well, you should see something like this appear:

Consoles: serial port
BIOS drive A: is disk0
BIOS drive C: is disk1

PXE version 2.1, real mode entry point @9cb6:0106
BIOS 638kB/1047488kB available memory

FreeBSD/i386 bootstrap loader, Revision 1.1
(root@nfsserver.home.lan, Thu Feb 18 21:03:14 PST 2010)
pxe_open: server addr:
pxe_open: server path: /usr/local/freebsd8
pxe_open: gateway ip:
Loading /boot/defaults/loader.conf
/boot/kernel/kernel text=0x88d680 data=0xdb0f4+0xa3104 syms=[0x4+0x95110+0x4+0xcc64f]

 |                                         |
 |                                         |      ______
 |                                         |     |  ____| __ ___  ___
 |          Welcome to FreeBSD!            |     | |__ | '__/ _ \/ _ \
 |                                         |     |  __|| | |  __/  __/
 |                                         |     | |   | | |    |    |
 |  1. Boot FreeBSD [default]              |     |_|   |_|  \___|\___|
 |  2. Boot FreeBSD with ACPI disabled     |      ____   _____ _____
 |  3. Boot FreeBSD in Safe Mode           |     |  _ \ / ____|  __ \
 |  4. Boot FreeBSD in single user mode    |     | |_) | (___ | |  | |
 |  5. Boot FreeBSD with verbose logging   |     |  _ < \___ \| |  | |
 |  6. Escape to loader prompt             |     | |_) |____) | |__| |
 |  7. Reboot                              |     |     |      |      |
 |                                         |     |____/|_____/|_____/
 |                                         |
 |                                         |
 |                                         |
 |  Select option, [Enter] for default     |
 |  or [Space] to pause timer  10          |

Hit Enter here, and let things boot normally. If all goes well, you should begin to see standard kernel output. Notible lines are shown here:

uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
uart0: console (115200,n,8,1)
md0: Preloaded image </boot/mfsroot> 4423680 bytes at 0xc0f6dfe0
Trying to mount root from ufs:/dev/md0
/stand/sysinstall running as init on serial console

These are the predefined terminal types available to
sysinstall when running stand-alone.  Please choose the
closest match for your particular terminal.

1 ...................... Standard ANSI terminal.
2 ...................... VT100 or compatible terminal.
3 ...................... FreeBSD system console (color).
4 ...................... FreeBSD system console (monochrome).

5 ...................... xterm terminal emulator.

Your choice: (1-5)

Pick a terminal type, and the rest should be obvious: the standard sysinstall(8) selection menu, and all that jazz.

3.9 Choosing an installation medium

When prompted what installation medium you want to install from, you can pick Install over NFS and enter the name of the share path manually, e.g.:

Or, if you want to install from a public FreeBSD server via HTTP or FTP or some other method, you can choose that. It's up to you.

From there, the installation should go smoothly. But there are some pending things you should remember to configure after the installation but before rebooting...

Chapter 4 Creating /boot.config for high-speed serial

Note: This section is under development.

This item is incredibly important. Yes, I realise I haven't written this section of the document yet, but it's still important! Failure to set this up will result in FreeBSD's serial ports operating at a maximum speed of 9600bps. Remember the rigmarole we went through in Chapter 3.5? You get to deal with that again, but this time for the standard non-PXE bootstraps.

I often forget this step when deploying a new server. The workaround is to set your terminal/serial connection's speed to 9600bps after rebooting the new system post-installation, then create /boot.config after-the-fact. This is hoky, and may result in some garbage on your screen briefly, but it works and is a one-time deal. The command echo "-S115200 -Dh" > /boot.config is all that's needed, followed by a reboot.

This nonsense could be avoided if sysinstall(8) offered the ability to edit an arbitrary file on the filesystem post-install but pre-reboot; you know, like what's offered when it comes to editing /etc/ttys in Chapter 6?

Chapter 5 Important Networking options

Note: This section is under development.

When configuring your network interface, remember to set a DNS server! Otherwise, things like sshd(8) (see below) won't start.

Chapter 6 Post-installation Configuration options

Note: This section is under development.

Things I strongly recommend setting when given the chance to "set any last configuration options" prior to the machine rebooting:

The last item is incredibly important. Without it, your machine will boot as normal, but you'll never see a login: prompt on the serial port. You'll very likely have to go to the physical console and reboot the machine manually to fix this mistake.

当前网页内容, 由 大妈 ZoomQuiet 使用工具: ScrapBook :: Firefox Extension 人工从互联网中收集并分享;
若有不妥, 欢迎评注提醒:


订阅 substack 体验古早写作:

点击注册~> 获得 100$ 体验券: DigitalOcean Referral Badge

关注公众号, 持续获得相关各种嗯哼:


关于 ~ DebugUself with DAMA ;-)
公安备案号: 44049002000656 ...::