User:Mjb/FreeBSD on BeagleBone Black

From Offset
Jump to navigationJump to search

Since late 2015 I've been experimenting with running FreeBSD on a BeagleBone Black (revision C). Any questions/comments, email me directly at root (at) skew.org.

See also:


Contents

Initial installation

  1. Get a compressed snapshot (.img.xz file) from FreeBSD's ARMv7 snapshots on ftp.freebsd.org.
  2. Uncompress it (e.g., with 7-Zip).
  3. Use a disk image writing program (e.g. Rufus on Windows, or 'dd' on BSD/Linux) to write the image to a micro SD card. I use a 64 GB card via a USB adapter in my PC.
  4. With power off, insert the micro SD card in the BBB.
  5. Use an ethernet cable to connect the LAN port to your router, if you want to be able to SSH in.
  6. Hold the boot button (it's the button near the card) and plug in the BBB. Keep holding the boot button for a few seconds to be sure it boots from the micro SD card. It will remember to boot from the card until the power is cut. Don't worry, there's a way to make it always boot from the SD card (rename the MLO file in the built-in drive's boot partition), but you don't want to do that until you're sure the OS on the SD card is working.

This is what I got on the console the first time I booted 11.0-STABLE (which was an armv6 build, back when "armv6" was for both ARMv6 and ARMv7): 

U-Boot SPL 2016.05 (Nov 17 2016 - 04:05:25)
Trying to boot from MMC1
Card doesn't support part_switch
MMC partition switch failed
*** Warning - MMC partition switch failed, using default environment

Here it is initially trying to boot from a nonexistent special partition on the eMMC (the built-in flash drive which ships with Debian Linux installed). A developer explains this is normal: MMC cards/devices are a bit different than SD, and one of the differences is that mmc supports a special "boot partition" that's separate from the main data in the device. So uboot tries to use the mmc boot feature, but the eMMC on the BBB isn't set up that way, so it just reports the error and moves on to booting the normal way. 

reading u-boot.img
reading u-boot.img


U-Boot 2016.05 (Nov 17 2016 - 04:05:25 +0000)

       Watchdog enabled
I2C:   ready
DRAM:  512 MiB
MMC:   OMAP SD/MMC: 0, OMAP SD/MMC: 1
reading u-boot.env

** Unable to read "u-boot.env" from mmc0:1 **
Using default environment

Net:   <ethaddr> not set. Validating first E-fuse MAC
Could not get PHY for cpsw: addr 0

This PHY message is a problem; see below. 

cpsw, usb_ether
reading uEnv.txt
** Unable to read file uEnv.txt **
Press SPACE to abort autoboot in 2 seconds
Booting from: mmc 0 ubldr
reading ubldr
271961 bytes read in 19 ms (13.6 MiB/s)
## Starting application at 0x88000098 ...
Consoles: U-Boot console
Compatible U-Boot API signature found @0x9ef36c70

FreeBSD/armv6 U-Boot loader, Revision 1.2
(root@releng2.nyi.freebsd.org, Thu Nov 17 04:16:49 UTC 2016)

DRAM: 512MB
Number of U-Boot devices: 3
U-Boot env: loaderdev='mmc 0'
Found U-Boot device: disk
  Checking unit=0 slice=<auto> partition=<auto>... good.
Booting from disk0s2a:
/boot/kernel/kernel data=0x6d5424+0x146bdc syms=[0x4+0x7e9d0+0x4+0x920d4]

Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...
/boot/dtb/beaglebone-black.dtb size=0x84f2
Loaded DTB from file 'beaglebone-black.dtb'.
Kernel entry at 0x88200100...
Kernel args: (null)
Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-STABLE #0 r308738: Thu Nov 17 04:21:53 UTC 2016
    root@releng2.nyi.freebsd.org:/usr/obj/arm.armv6/usr/src/sys/BEAGLEBONE arm
FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0)
VT: init without driver.
CPU: Cortex A8-r3 rev 2 (Cortex-A core)
 Supported features: ARM_ISA THUMB2 JAZELLE THUMBEE ARMv4 Security_Ext
 WB enabled LABT branch prediction disabled

It is confusing, but the BBB's Cortex A8 is an ARMv7 CPU. FreeBSD has two classes of ARM support: FreeBSD/arm supports ARMv4 and ARMv5 CPUs, and (prior to FreeBSD 12), FreeBSD/armv6 supports ARMv6 and ARMv7 CPUs. That's why compiled code on the BBB prior to FreeBSD 12 always refers to "armv6". 

LoUU:2 LoC:3 LoUIS:1
Cache level 1:
 32KB/64B 4-way data cache WT WB Read-Alloc
 32KB/64B 4-way instruction cache Read-Alloc
Cache level 2:
 256KB/64B 8-way unified cache WT WB Read-Alloc Write-Alloc
real memory  = 536870912 (512 MB)
avail memory = 513359872 (489 MB)
Texas Instruments AM335x Processor, Revision ES1.2
random: entropy device external interface
kbd0 at kbdmux0
ofwbus0: <Open Firmware Device Tree>
simplebus0: <Flattened device tree simple bus> on ofwbus0
simplebus1: <Flattened device tree simple bus> on simplebus0
simplebus2: <Flattened device tree simple bus> mem 0x210000-0x211fff on simplebus1
ti_scm0: <TI Control Module> mem 0-0x7ff on simplebus2
aintc0: <TI AINTC Interrupt Controller> mem 0x48200000-0x48200fff on simplebus0
aintc0: Revision 5.0
cpulist0: <Open Firmware CPU Group> on ofwbus0
cpu0: <Open Firmware CPU> on cpulist0
pmu0: <Performance Monitoring Unit> irq 0 on ofwbus0
am335x_prcm0: <AM335x Power and Clock Management> mem 0x200000-0x203fff on simplebus1
am335x_prcm0: Clocks: System 24.0 MHz, CPU 1000 MHz
ti_pinmux0: <TI Pinmux Module> mem 0x800-0xa37 on simplebus2
gpio0: <TI AM335x General Purpose I/O (GPIO)> mem 0x44e07000-0x44e07fff irq 7 on simplebus0
gpiobus0: <OFW GPIO bus> on gpio0
gpioc0: <GPIO controller> on gpio0
gpio1: <TI AM335x General Purpose I/O (GPIO)> mem 0x4804c000-0x4804cfff irq 8 on simplebus0
gpiobus1: <OFW GPIO bus> on gpio1
gpioc1: <GPIO controller> on gpio1
gpio2: <TI AM335x General Purpose I/O (GPIO)> mem 0x481ac000-0x481acfff irq 9 on simplebus0
gpiobus2: <OFW GPIO bus> on gpio2
gpioc2: <GPIO controller> on gpio2
gpio3: <TI AM335x General Purpose I/O (GPIO)> mem 0x481ae000-0x481aefff irq 10 on simplebus0
gpiobus3: <OFW GPIO bus> on gpio3
gpioc3: <GPIO controller> on gpio3
uart0: <TI UART (16550 compatible)> mem 0x44e09000-0x44e0afff irq 11 on simplebus0
uart0: console (115384,n,8,1)
iichb0: <TI I2C Controller> mem 0x44e0b000-0x44e0bfff irq 17 on simplebus0
iichb0: I2C revision 4.0 FIFO size: 32 bytes
iicbus0: <OFW I2C bus> on iichb0
iic0: <I2C generic I/O> on iicbus0
am335x_pmic0: <TI TPS65217 Power Management IC> at addr 0x48 irq 62 on iicbus0
iicbus0: <unknown card> at addr 0xa0
tda0 at addr 0xe0 on iicbus0
tda1 at addr 0xe0 on iicbus0
iichb1: <TI I2C Controller> mem 0x4802a000-0x4802afff irq 18 on simplebus0
iichb1: I2C revision 4.0 FIFO size: 32 bytes
iicbus1: <OFW I2C bus> on iichb1
iic1: <I2C generic I/O> on iicbus1
iichb2: <TI I2C Controller> mem 0x4819c000-0x4819cfff irq 19 on simplebus0
iichb2: I2C revision 4.0 FIFO size: 32 bytes
iicbus2: <OFW I2C bus> on iichb2
iic2: <I2C generic I/O> on iicbus2
iicbus2: <unknown card> at addr 0xa8
iicbus2: <unknown card> at addr 0xaa
iicbus2: <unknown card> at addr 0xac
iicbus2: <unknown card> at addr 0xae
sdhci_ti0: <TI MMCHS (SDHCI 2.0)> mem 0x48060000-0x48060fff irq 20 on simplebus0
mmc0: <MMC/SD bus> on sdhci_ti0
sdhci_ti1: <TI MMCHS (SDHCI 2.0)> mem 0x481d8000-0x481d8fff irq 21 on simplebus0
mmc1: <MMC/SD bus> on sdhci_ti1
ti_wdt0: <TI Watchdog Timer> mem 0x44e35000-0x44e35fff irq 23 on simplebus0
ti_mbox0: <TI System Mailbox> mem 0x480c8000-0x480c81ff irq 26 on simplebus0
ti_mbox0: revision 4.0
am335x_dmtimer0: <AM335x DMTimer2> mem 0x48040000-0x480403ff irq 28 on simplebus0
Event timer "DMTimer2" frequency 24000000 Hz quality 500
am335x_dmtimer1: <AM335x DMTimer3> mem 0x48042000-0x480423ff irq 29 on simplebus0
Timecounter "DMTimer3" frequency 24000000 Hz quality 500
am335x_rtc0: <AM335x RTC (power management mode)> mem 0x44e3e000-0x44e3efff irq 34,35 on simplebus0
am335x_rtc0: AM335X RTC v1.0.6
spi0: <TI McSPI controller> mem 0x481a0000-0x481a03ff irq 37 on simplebus0
spi0: scheme: 0x1 func: 0x30 rtl: 1 rev: 2.11 custom rev: 0
spibus0: <OFW SPI bus> on spi0
usbss0: <TI AM33xx integrated USB OTG controller> mem 0x47400000-0x47400fff on simplebus0
usbss0: TI AM335X USBSS v0.0.13
musbotg0: <TI AM33xx integrated USB OTG controller> mem 0x47401400-0x474017ff,0x47401000-0x474011ff irq 63 on usbss0
usbus0: Dynamic FIFO sizing detected, assuming 16Kbytes of FIFO RAM
usbus0 on musbotg0
musbotg1: <TI AM33xx integrated USB OTG controller> mem 0x47401c00-0x47401fff,0x47401800-0x474019ff irq 64 on usbss0
usbus1: Dynamic FIFO sizing detected, assuming 16Kbytes of FIFO RAM
usbus1 on musbotg1
cpswss0: <3-port Switch Ethernet Subsystem> mem 0x4a100000-0x4a1007ff,0x4a101200-0x4a1012ff irq 38,39,40,41 on simplebus0
cpswss0: CPSW SS Version 1.12 (0)
cpswss0: Initial queue size TX=128 RX=384
cpsw0: <Ethernet Switch Port> on cpswss0

Watch out for this: 

cpsw0: Failed to read from PHY.
cpsw0: attaching PHYs failed
device_attach: cpsw0 attach returned 6

If you see this, as well as that PHY message from U-Boot, it means the NIC did not initialize. I'm told this happens sometimes. I don't think there is any way to make it work without power cycling the board. So, after you get a login prompt, you must login as root (password is root), 'shutdown -p now', and then after the power light turns off, disconnect and reconnect the power cable to try again.

This is what you should get when it works: 

miibus0: <MII bus> on cpsw0
smscphy0: <SMC LAN8710A 10/100 interface> PHY 0 on miibus0
smscphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
cpsw0: Ethernet address: 84:eb:18:e2:8e:56

Continuing on... 

fb0: <AM335x LCD controller> mem 0x4830e000-0x4830efff irq 43 on simplebus0
ti_adc0: <TI ADC controller> mem 0x44e0d000-0x44e0dfff irq 44 disabled on simplebus0
ti_adc0: scheme: 0x1 func: 0x730 rtl: 0 rev: 0.1 custom rev: 0
ti_pruss0: <TI Programmable Realtime Unit Subsystem> mem 0x4a300000-0x4a37ffff irq 53,54,55,56,57,58,59,60 on simplebus0
ti_pruss0: AM33xx PRU-ICSS
gpioled0: <GPIO LEDs> on ofwbus0
cryptosoft0: <software crypto>
Timecounters tick every 10.000 msec
usbus0: 480Mbps High Speed USB v2.0
usbus1: 480Mbps High Speed USB v2.0
am335x_pmic0: TPS65217C ver 1.2 powered by AC
tda0: TDA19988
ugen1.1: <Mentor Graphics> at usbus1
uhub0: <Mentor Graphics OTG Root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ugen0.1: <Mentor Graphics> at usbus0
uhub1: <Mentor Graphics OTG Root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
uhub0: 1 port with 1 removable, self powered
uhub1: 1 port with 1 removable, self powered
tda0: failed to read EDID
tda1: TDA19988
tda1: failed to read EDID

tda0 and tda1 are HDMI devices; "failed to read EDID" probably just means nothing is plugged into the HDMI port. 

mmcsd0: 64GB <SDHC 00000 1.0 SN 0A1806A0 MFG 02/2015 by 27 SM> at mmc0 48.0MHz/4bit/65535-block
mmcsd1: 4GB <MMCHC S10004 0.8 SN 34D2DDBF MFG 02/1999 by 112 0x0000> at mmc1 48.0MHz/8bit/65535-block
Trying to mount root from ufs:/dev/ufs/rootfs [rw]...
warning: no time-of-day clock registered, system time will not be set accurately

The following only appears on first boot, and there is a long pause at the end of the list of super-block backups (the number of which depends on the size of your disk): 

Growing root partition to fill device
GEOM_PART: mmcsd0s2 was automatically resized.
  Use `gpart commit mmcsd0s2` to save changes or `gpart undo mmcsd0s2` to revert them.
mmcsd0s2 resized
mmcsd0s2a resized
super-block backups (for fsck_ffs -b #) at:
 2093248, 2616512, 3139776, 3663040, 4186304, 4709568, 5232832, 5756096,
 6279360, 6802624, 7325888, 7849152, 8372416, 8895680, 9418944, 9942208,
 10465472, 10988736, 11512000, 12035264, 12558528, 13081792, 13605056,
 14128320, 14651584, 15174848, 15698112, 16221376, 16744640, 17267904,
 17791168, 18314432, 18837696, 19360960, 19884224, 20407488, 20930752,
 21454016, 21977280, 22500544, 23023808, 23547072, 24070336, 24593600,
 25116864, 25640128, 26163392, 26686656, 27209920, 27733184, 28256448,
 28779712, 29302976, 29826240, 30349504, 30872768, 31396032, 31919296,
 32442560, 32965824, 33489088, 34012352, 34535616, 35058880, 35582144,
 36105408, 36628672, 37151936, 37675200, 38198464, 38721728, 39244992,
 39768256, 40291520, 40814784, 41338048, 41861312, 42384576, 42907840,
 43431104, 43954368, 44477632, 45000896, 45524160, 46047424, 46570688,
 47093952, 47617216, 48140480, 48663744, 49187008, 49710272, 50233536,
 50756800, 51280064, 51803328, 52326592, 52849856, 53373120, 53896384,
 54419648, 54942912, 55466176, 55989440, 56512704, 57035968, 57559232,
 58082496, 58605760, 59129024, 59652288, 60175552, 60698816, 61222080,
 61745344, 62268608, 62791872, 63315136, 63838400, 64361664, 64884928,
 65408192, 65931456, 66454720, 66977984, 67501248, 68024512, 68547776,
 69071040, 69594304, 70117568, 70640832, 71164096, 71687360, 72210624,
 72733888, 73257152, 73780416, 74303680, 74826944, 75350208, 75873472,
 76396736, 76920000, 77443264, 77966528, 78489792, 79013056, 79536320,
 80059584, 80582848, 81106112, 81629376, 82152640, 82675904, 83199168,
 83722432, 84245696, 84768960, 85292224, 85815488, 86338752, 86862016,
 87385280, 87908544, 88431808, 88955072, 89478336, 90001600, 90524864,
 91048128, 91571392, 92094656, 92617920, 93141184, 93664448, 94187712,
 94710976, 95234240, 95757504, 96280768, 96804032, 97327296, 97850560,
 98373824, 98897088, 99420352, 99943616, 100466880, 100990144, 101513408,
 102036672, 102559936, 103083200, 103606464, 104129728, 104652992, 105176256,
 105699520, 106222784, 106746048, 107269312, 107792576, 108315840, 108839104,
 109362368, 109885632, 110408896, 110932160, 111455424, 111978688, 112501952,
 113025216, 113548480, 114071744, 114595008, 115118272, 115641536, 116164800,
 116688064, 117211328, 117734592, 118257856, 118781120, 119304384, 119827648,
 120350912, 120874176, 121397440, 121920704, 122443968, 122967232, 123490496,
 124013760, 124537024
random: unblocking device.
/etc/rc: WARNING: hostid: unable to figure out a UUID from DMI data, generating a new one

Continuing on: 

Setting hostuuid: 5f5965bc-ac7e-11e6-b765-5dd0daef826d.
Setting hostid: 0x3f53a6c4.
Starting file system checks:
/dev/ufs/rootfs: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/ufs/rootfs: clean, 14935306 free (282 frags, 1866878 blocks, 0.0% fragmentation)
Mounting local filesystems:.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
random: unblocking device.
Soft Float compatibility ldconfig path:
Setting hostname: beaglebone.
Setting up harvesting: [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy: .

If the NIC did not start (see above), then you will see this: 

Starting Network: lo0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Starting devd.

If the NIC is working and connected, you should see this: 

cpsw0: link state changed to DOWN
cpsw0: link state changed to UP
Starting Network: lo0 cpsw0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
cpsw0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        ether 84:eb:18:e2:8e:56
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Starting devd.

The BBB is configured for DHCP by default. If there's a DHCP server on your LAN, you will see something like this: 

Starting dhclient.
DHCPDISCOVER on cpsw0 to 255.255.255.255 port 67 interval 6
DHCPOFFER from 10.0.1.1
DHCPREQUEST on cpsw0 to 255.255.255.255 port 67
DHCPACK from 10.0.1.1
bound to 10.0.1.14 -- renewal in 43200 seconds.

In any case, you should see this next: 

add host 127.0.0.1: gateway lo0 fib 0: route already in table
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Generating host.conf.
Creating and/or trimming log files.
Starting syslogd.
Clearing /tmp (X related).
Updating motd:.
Mounting late filesystems:.

Host key generation only happens once: 

Generating RSA host key.
2048 SHA256:MR5l2HEXwe95a1h/t+kTUZo3yt0/QFBRIfkIEGCRSzw root@beaglebone (RSA)
Generating ECDSA host key.
256 SHA256:XWZ6yhBhbfMNYlZTE6l6C1SdTj4QSHrPReRJ05fUCm4 root@beaglebone (ECDSA)
Generating ED25519 host key.
256 SHA256:HqUdXnOFQ7coxafJ29s6A2r3CHTcxMg86YdHLn45nCM root@beaglebone (ED25519)

Continuing on: 

Performing sanity check on sshd configuration.
Starting sshd.
Starting cron.
Starting background file system checks in 60 seconds.

You might also see this. I don't know why: 

mount: /dev/ufs/rootfs: Device busy

And finally, a login prompt: 

Thu Nov 17 04:29:38 UTC 2016

FreeBSD/arm (beaglebone) (ttyu0)

login:

Things that can go wrong

NIC not initialized

Boot messages indicating the NIC (network interface card, i.e. the ethernet port) did not initialize: 

Net:   <ethaddr> not set. Validating first E-fuse MAC
Could not get PHY for cpsw: addr 0

cpsw0: Failed to read from PHY.
cpsw0: attaching PHYs failed
device_attach: cpsw0 attach returned 6
  • Starting network line only mentions lo0, not cpsw0. Subsequent notifications also differ:
    • No mention of cpsw0: link state changed to DOWN and cpsw0: link state changed to UP.
    • ifconfig info for lo0 is not followed by info for cpsw0.
  • Starting dhclient line and subsequent DHCP notifications missing, even though DHCP is enabled.

Usually this problem goes away if you power cycle the unit. It is not enough to do a soft reboot.

fat register err

I once saw this at boot time: 

U-Boot SPL 2014.10 (Oct 01 2015 - 02:23:53)
MMC: block number 0x100 exceeds max(0x0)
MMC: block number 0x200 exceeds max(0x0)
*** Error - No Valid Environment Area found
Using default environment

** Can't read partition table on 0:0 **
** Partition 1 not valid on device 0 **
spl_register_fat_device: fat register err - -1
### ERROR ### Please RESET the board ###

It came up again after a power cycle. On the third boot, everything worked fine.

Controller timeout

On my BBB, this often happened under FreeBSD 10, seemingly at random: 

sdhci_ti0-slot0:  Controller timeout
g_vfs_done():ufs/rootfs[READ(offset=162066432, length=4096)]error = 5
sdhci_ti0-slot0: ============== REGISTER DUMP ==============
sdhci_ti0-slot0: Sys addr: 0x00000000 | Version:  0x00003101
sdhci_ti0-slot0: Blk size: 0x00000200 | Blk cnt:  0x00000010
sdhci_ti0-slot0: Argument: 0x011be180 | Trn mode: 0x0000193a
sdhci_ti0-slot0: Present:  0x01e70106 | Host ctl: 0x00000006
sdhci_ti0-slot0: Power:    0x0000000d | Blk gap:  0x00000000
sdhci_ti0-slot0: Wake-up:  0x00000000 | Clock:    0x00000107
sdhci_ti0-slot0: Timeout:  0x0000000d | Int stat: 0x00000000
sdhci_ti0-slot0: Int enab: 0x017f00fb | Sig enab: 0x017f00fb
sdhci_ti0-slot0: AC12 err: 0x00000000 | Slot int: 0x00000000
sdhci_ti0-slot0: Caps:     0x06e10080 | Max curr: 0x00000000
sdhci_ti0-slot0: ===========================================
mmcsd0: Error indicated: 1 Timeout
g_vfs_done():ufs/rootfs[WRITE(ommcsd0: Error indicated: 1 Timeout
ffset=9523298304, length=8192)]error = 5
g_vfs_done():ufs/rootfs[WRITE(offset=9673617408, length=4096)]error = 5
mmcsd0: Error indicated: 1 Timeout
g_vfs_done():ufs/rootfs[WRITE(offset=636809216, length=4096)]error = 5
mmcsd0: Error indicated: 1 Timeout
g_vfs_done():ufs/rootfs[WRITE(offset=638910464, length=4096)]error = 5
mmcsd0: Error indicated: 1 Timeout
g_vfs_done():ufs/rootfs[WRITE(offset=639856640, length=4096)]error = 5
g_vfs_done():ufs/rootfs[WRITE(offset=643227648, length=4096)]error = 5
g_vfs_done():ufs/rootfs[WRITE(offset=644841472, length=65536)]error = 5
g_vfs_done():ufs/rootfs[WRITE(offset=661487616, length=32768)]error = 5
g_vfs_done():ufs/rootfs[WRITE(offset=662437888, length=32768)]error = 5
g_vfs_done():ufs/rootfs[WRITE(offset=667222016, length=32768)]error = 5

... these can go on for a long time, and the system is unusable in this state. Eventually they might end with a crash: 

initiate_write_filepage: already started
panic: initiate_write_inodeblock_ufs2: already started
KDB: enter: panic
[ thread pid 9 tid 100068 ]
Stopped at      $d:     ldrb    r15, [r15, r15, ror r15]!
db>

Under FreeBSD 11, I have not seen this happening yet.

Essential first steps

Hopefully you already know how to do some basic things in a command shell, i.e. a text-only interface where you type in commands and have access to files. You should understand the basic idea of files & directories, shell scripts, symbolic links (symlinks), file ownership and permissions, daemons and services, IP addresses, ports, and domain names. You should be aware that as a FreeBSD user, you're also a system administrator responsible for configuring and securing many aspects of your operating system—mostly without the aid of graphical user interfaces (GUIs), which are complex add-ons. You should be aware of The FreeBSD Handbook and manual pages (manpages) as sources for more information.

Log in

Set up terminal

Make the terminal environment match your settings in your terminal emulator:

  • setenv LANG en_US.UTF-8 – assuming your emulator is set to use UTF-8, US English locale (if applicable).

If you are not connected via SSH, then you need to also set these:

  • setenv TERM xterm-256color – best match for Tera Term's VT100 with ANSI color enabled
  • stty rows 46 cols 132 – but use the values matching your terminal size

Required for the rest of this document: configure the tcsh shell's builtin 'echo' command to recognize C-style/SysV-style escape codes:

  • set echo_style = both

The escape codes are as described in the sh(1) manual page, for that shell's builtin 'echo -e' command: \a = bell, \b = backspace, \c = end string & don't output a newline, \e = escape, \f = form feed, \n = newline, \r = carriage return, \t = tab (horizontal), \v = vertical tab, \\ = backslash, \0### = ASCII character with octal code ###.

I mainly use tab and newline.

Support DNS-free resolution of certain hostnames

It helps to ensure the currently configured hostname ("beaglebone" in FreeBSD snapshots) resolves:

  • hostname -f && hostname -s
  • ee /etc/hosts – add the hostname(s) reported by the previous command as localhost aliases. Use Esc Enter Enter to save the file and exit the editor.

The hostname is set in /etc/rc.conf. "beaglebone" is just a temporary name. You can keep using it, but ideally you should change the "beaglebone" to be a fully qualified domain name (FQDN), even if it's just "beaglebone" with the DHCP-assigned search domain appended. The search domain can be found in /etc/resolv.conf, but make sure you do not include the trailing dot—that is, in /etc/rc.conf, you should enter something like hostname="beaglebone.example.org", not hostname="beaglebone.example.org.". Then run service hostname restart. Also add the same name to /etc/hosts, mapped to your actual IP address (not 127.0.0.1 or ::1).

I also want to make sure a nearby NTP server can be reached via a made-up hostname 'timenistgov', even when DNS isn't working. I use a couple of the NIST servers listed at http://tf.nist.gov/tf-cgi/servers.cgi (I chose one server by its IPv4 address and a different server by its IPv6 address).

The following assumes the NIC is working:

  • if ( { ( host time.nist.gov. > & /dev/null ) } ) echo `host time.nist.gov. | grep 'has address' | head -1 | awk '{print $NF}'`'\ttimenistgov' >> /etc/hosts || echo 'DNS is not working; assuming 128.138.141.172 and 2610:20:6f96:96::4 are OK.\nVerify at http://tf.nist.gov/tf-cgi/servers.cgi when you get a chance.' && echo '128.138.141.172\ttimenistgov\n2610:20:6f96:96::4\ttimenistgov' >> /etc/hosts
  • if ( { ( host time.nist.gov. | grep -q 'has IPv6 address' ) } ) echo `host time.nist.gov. | grep 'has IPv6 address' | head -1 | awk '{print $NF}'`'\ttimenistgov' >> /etc/hosts

If the NIC is not working, just do this:

  • echo 'Assuming time.nist.gov is at 128.138.141.172 and 2610:20:6f96:96::4.\nVerify at http://tf.nist.gov/tf-cgi/servers.cgi when you get a chance.' && echo '128.138.141.172\ttimenistgov\n2610:20:6f96:96::4\ttimenistgov' >> /etc/hosts

Set time zone and clock

You can set the time zone manually, or by running the interactive wizard.

To use the wizard:

  • tzsetup (and when asked about the CMOS clock, answer Yes because it uses UTC, not local time!)

To do it manually, make /etc/localtime be a symlink to the correct file in /usr/share/zoneinfo. For example, for Mountain time (USA):

  • ln -s /usr/share/zoneinfo/MST7MDT /etc/localtime
  • rm -f /etc/wall_cmos_clock – this empty file tells the OS the CMOS clock is local time; get rid of it!

Now set the clock with the help of NTP servers on the Internet (assuming you have Internet access). Assuming DNS is working:

  • ntpd -g -q timenistgov pool.ntp.org

(If DNS is not working for some reason, then remove pool.ntp.org from the command line.)

The output should look something like this: 

16 Nov 22:00:02 ntpd[686]: ntpd 4.2.8p8-a (1): Starting
16 Nov 22:00:02 ntpd[686]: Command line: ntpd -g -q timenistgov pool.ntp.org
16 Nov 22:00:02 ntpd[686]: proto: precision = 2.208 usec (-19)
Nov 16 22:00:02 beaglebone ntpd[686]: leapsecond file ('/var/db/ntpd.leap-seconds.list'): stat failed: No such file or directory
16 Nov 22:00:02 ntpd[686]: leapsecond file ('/var/db/ntpd.leap-seconds.list'): stat failed: No such file or directory
16 Nov 22:00:02 ntpd[686]: Listen and drop on 0 v6wildcard [::]:123
16 Nov 22:00:02 ntpd[686]: Listen and drop on 1 v4wildcard 0.0.0.0:123
16 Nov 22:00:02 ntpd[686]: Listen normally on 2 cpsw0 10.0.1.14:123
16 Nov 22:00:02 ntpd[686]: Listen normally on 3 lo0 [::1]:123
16 Nov 22:00:02 ntpd[686]: Listen normally on 4 lo0 [fe80::1%2]:123
16 Nov 22:00:02 ntpd[686]: Listen normally on 5 lo0 127.0.0.1:123
16 Nov 22:00:02 ntpd[686]: Listening on routing socket on fd #26 for interface updates
26 Nov 09:44:51 ntpd[686]: ntpd: time set +819888.298970 s
ntpd: time set +819888.298970s

If it hangs before the last line, the DNS or the NIC is probably not working.

(Instead of ntpd, you could use the deprecated 'ntpdate'—i.e. ntpdate timenistgov pool.ntp.org—but I recommend getting accustomed to using ntpd.)

Use date to see if the clock is set right. If not, set it manually. For example:

  • date 201510080847.49 – sets the clock to 2015-10-08 08:47:49, local time. Use date -u if you're instead giving it UTC time.

Now edit /etc/ntp.conf so you won't need to specify servers on the command ntpd or ntpdate command lines:

  • ee /etc/ntp.conf

In that file, add the following line above server 0.freebsd.pool.ntp.org iburst: 

server timenistgov iburst

(Yes, you could have done this before running ntpd, but the timestamp on ntp.conf would be wacky since your clock wasn't set yet.)

Enable automatic clock setting

On the BeagleBone Black, there's no real-time clock (RTC) battery, so the clock needs to be set every time you reboot. This is a good idea anyway, especially as the world moves toward secure services being dependent on accurate clocks.

FreeBSD apparently remembers the time of shutdown and uses that to reset the clock on reboot. So if the machine is off for a day, your clock is only a day behind when you power it on.

The preferred way to set the clock automatically is to run an NTP daemon (ntpd) to get the time from some trusty servers on the Internet as often as needed.

FreeBSD's stock ntpd is not set up to run automatically, and I prefer OpenNTPD because it's easier to configure and update. So one of the first things to set up is the ports collection, and then OpenNTPD; see my instructions for that. However, if you want to run the stock ntpd, this is all you do:

  • echo 'ntpd_enable="YES"' >> /etc/rc.conf
  • echo 'ntpd_sync_on_start="YES"' >> /etc/rc.conf
  • service ntpd start
Here are some other options, and the reasons I don't use them:

Putting ntpdate_enable="YES" in /etc/rc.conf is simple but is not ideal, because 1. ntpdate is deprecated, 2. they haven't set up an ntpdate equivalent that runs the preferred command ntpd -g -q yet, and 3. it only syncs the clock once at bootup—the clock will drift after that.

It is also possible to run ntpd or ntpdate as a cron job, e.g. with something like 22 2,6,10,14,18,22 * * * /usr/sbin/ntpd -g -q > /dev/null in root's crontab. But it won't run at startup when it's needed most (especially on the BeagleBone!), it will run too often or not often enough, and the adjustments it makes may be either too slow (with -x, when there's a big adjustment to make) or too coarse for some apps & services to tolerate (without -x).

Optional: configure ntpd logging

I prefer to keep a separate log for messages from the NTP server.

  • Put this in /etc/syslog.conf:
ntp.*                                   /var/log/ntp.log
  • Create an empty log file to start: touch /var/log/ntp.log
  • service syslogd reload
  • To enable rotation of the log file, put this in /etc/newsyslog.conf:
/var/log/ntp.log           644  3     *    @T00    JCN

Enable firewall

There are 3 different firewalls, the most popular on BSD being IPFW. By default, it will disallow all external network traffic, including your SSH connections. Enable it like this:

  • echo 'firewall_enable="YES"' >> /etc/rc.conf

The firewall won't actually run until you reboot or you manually start the ipfw service. Don't do that yet. Just keep reading.

You need this:

  • echo 'firewall_quiet="YES"' >> /etc/rc.conf

This setting tells the standard firewall scripts to use ipfw's -q option to suppress the announcement of each processed rule on stdout. This is important if you'll be starting up ipfw from an SSH session, because it will keep the SSH session from getting killed after processing the initial flush at the beginning of the standard rulesets. Unless you used nohup, the death of the SSH session would abort the script that was loading the rules, thus locking you out and requiring console access to fix.

How it works

In /etc/rc.conf, the optional variable firewall_script="..." tells /etc/rc.d/ipfw (the script that launches ipfw) where to find a script containing firewall rules and other configuration commands. It defaults to /etc/rc.firewall, which does initial loopback (localhost-to-localhost) and mandatory IPv6 configuration, then uses /etc/rc.conf's firewall_type to determine what to do next. If firewall_type is one of the standard values (open, client, simple, closed, workstation, or the default do-nothing type UNKNOWN), it will apply certain rules for those types of configurations; look in /etc/rc.firewall for details. Then it's up to you to run a separate script for further customization, e.g. via /etc/rc.local. If not one of the standard values, firewall_type must be the path to your own shell script, and your script must do all the initial configuration itself. (I don't see any need to do it that way, but if you really want to, read Building a Rule Script in the FreeBSD Handbook and take a look at /etc/rc.firewall's setup_loopback and setup_ipv6_mandatory routines for the initial config.)

Ideal configuration for a typical server

In the past I always just used the "open" type and put my supplemental rules in /etc/rc.local, but I feel an open firewall is too risky these days. The "workstation" type appears to provide sane defaults, permitting any loopback or outbound connections (preserving state), as well as inbound DHCP and some ICMP. It is easily configured:

  • echo 'firewall_type="workstation"' >> /etc/rc.conf

The rest of these lines are specific to the "workstation" type:

  • echo 'firewall_allowservices="any"' >> /etc/rc.conf
  • echo 'firewall_trusted=""' >> /etc/rc.conf
  • echo 'firewall_myservices="22/tcp 25/tcp 80/tcp 443/tcp 587/tcp 853/tcp"' >> /etc/rc.conf
  • echo 'firewall_logdeny="YES"' >> /etc/rc.conf
  • echo 'firewall_nologports="137 138 1900 3702 17500"' >> /etc/rc.conf

firewall_allowservices="..." is either any or a list of IPs and networks allowed to connect. This is used as the "from" value in the ipfw commands.

firewall_trusted="..." is a list of IP addresses and networks with unrestricted access (no ports blocked). If you will be connecting to this computer from a trusted host with a static IP address, feel free to add that address here.

firewall_myservices="..." is a list of ports (or service names from /etc/services) which you want to allow other hosts to access, aside from the defaults. TCP is assumed, but it's preferable to add /tcp to suppress a warning. Other options are /udp and /proto. The ports in this example are for SSH (22), SMTP (25 & 587), HTTP (80), HTTPS (443), and secure DNS (853). When you add public services, add them to the list and restart ipfw. The ports you list here will have an "allow tcp from any to me" rule with fairly low number (2500, 2600, 2700, etc.), so if you need to deny some traffic to these ports, you must make sure the deny rules all come first. Thus it is probably better to not use this feature and instead just manually add the 'allow' rules such that they come after your 'deny' rules for those ports.

firewall_logdeny="YES" establishes a final rule which denies all traffic and logs the first 500 times it is invoked. After that the rule still works, but to resume logging you have to run ipfw resetlog (which happens daily anyway). This logging can be reduced a bit by using firewall_nologports="..." to specify a list of blocked ports for which access attempts should not be logged. You can't specify the TCP or UDP for this variable; it applies to both. Ports I'm not going to log include 137 & 138 (NetBIOS), 1900 (UPnP), and 3702 (WS-Discovery); these are all used by periodic scans of my LAN by Windows services. 17500 is Dropbox LAN Sync.

It's recommended that you don't let your logs get overwhelmed with repeated messages:

  • sysctl net.inet.ip.fw.verbose_limit=5
  • echo net.inet.ip.fw.verbose_limit=5 >> /etc/sysctl.conf

If you are using an 11.0-STABLE snapshot, you might not have the net.inet.ip.fw.verbose_limit OID. You should have it if you rebuild the kernel from source.

/etc/syslog.conf is already configured to dump the messages into /var/log/security. The net.inet.ip.fw.verbose_limit applies there, so you see things like "last message repeated 37 times" instead of 42 (5+37) copies of the same message. The raw messages, along with messages from other sources, also scroll through the kernel message buffer, which you can view with dmesg -a. These raw messages won't have the limit applied; you'll see all 42. The kernel message buffer is only 96 KB, although I believe this can be increased to an arbitrary number of bytes via a kern.msgbufsize=###### entry in /boot/loader.conf (but keep in mind you don't have infinite RAM, so don't set it to many megabytes!).

Concerns

The "workstation" configuration uses keep-state rules which result in the creation of dynamic rules to allow inbound traffic in response to outbound. As mentioned in the FreeBSD Handbook:

The dynamic rules facility is vulnerable to resource depletion from a SYN-flood attack which would open a huge number of dynamic rules. To counter this type of attack with IPFW, use limit. This option limits the number of simultaneous sessions by checking the open dynamic rules, counting the number of times this rule and IP address combination occurred. If this count is greater than the value specified by limit, the packet is discarded.

Unfortunately, the "workstation" configuration does not have the limit option enabled on the dynamic rules.

I will create my own workstation-plus-limits configuration and will update these instructions accordingly. In the meantime, it seems to be working OK as-is.

Enable and further customize rules

All set? Give it a whirl:

  • service ipfw start

See what rules are currently in effect:

  • ipfw -dS list

Enable putting custom rules in /etc/ipfw.rules and loading it at startup:

  • ee /etc/rc.local
#!/bin/sh
#
# This file is a deprecated but convenient method of launching additional
# "local daemons" (or just running any other startup tasks) at the very
# end of the boot process. See the rc(8) manual page.
#

# load variables from rc.conf (comment out if not needed)
#
#if [ -z "${source_rc_confs_defined}" ]; then
#    if [ -r /etc/defaults/rc.conf ]; then
#        . /etc/defaults/rc.conf
#        source_rc_confs
#    elif [ -r /etc/rc.conf ]; then
#        . /etc/rc.conf
#    fi
#fi

# load additional firewall rules
rules="/etc/ipfw.rules"
[ -f $rules ] && echo -n " $rules" && . $rules

It's best to write the scripts such that they can be edited and run again without causing problems. Here's an example of /etc/ipfw.rules made safe by ensuring rules from previous runs of the script are deleted before being created anew: 

# These sets of rules are numbered so they can be toggled via (e.g.):
# ipfw set disable 1 enable 2
# ipfw delete set 1
#
# To see all the loaded rules and their set numbers:
# ipfw -S list

ipfw="ipfw -q add"
ipfw_delete="ipfw -q delete"
mysqld="3306"
shoutcast="7778"

#
# only allow local access to MySQL
#
# see also bind-address in [mysqld] section of /var/db/mysql/my.cnf
# (ideally it is set so as not to even listen on non-localhost IP addresses)
#
$ipfw_delete set 1
$ipfw 10000 set 1 allow tcp from me to me $mysqld
$ipfw 10001 set 1 deny tcp from any to me $mysqld

# Deny search engine spiders access to the SHOUTcast server
#
# IP address ranges obtained from http://chceme.info/ips/
# and converted to CIDR notation with http://www.mikero.com/misc/
#
$ipfw_delete set 2
# Google (GoogleBot)
$ipfw 11000 set 2 deny tcp from 64.233.160.0/19 to me $shoutcast
$ipfw 11001 set 2 deny tcp from 66.102.0.0/20 to me $shoutcast
$ipfw 11002 set 2 deny tcp from 66.249.64.0/19 to me $shoutcast
$ipfw 11003 set 2 deny tcp from 72.14.192.0/18 to me $shoutcast
$ipfw 11004 set 2 deny tcp from 74.125.0.0/16 to me $shoutcast
$ipfw 11005 set 2 deny tcp from 209.85.128.0/17 to me $shoutcast
$ipfw 11006 set 2 deny tcp from 216.239.32.0/19 to me $shoutcast
#
# MSN/Live (MSNBot)
$ipfw 11100 set 2 deny tcp from 64.4.0.0/18 to me $shoutcast
$ipfw 11101 set 2 deny tcp from 65.52.0.0/14 to me $shoutcast
$ipfw 11102 set 2 deny tcp from 131.253.21.0/24 to me $shoutcast
$ipfw 11103 set 2 deny tcp from 157.48.0.0/12 to me $shoutcast
$ipfw 11104 set 2 deny tcp from 207.46.0.0/16 to me $shoutcast
$ipfw 11105 set 2 deny tcp from 207.68.128.0/17 to me $shoutcast
#
# Yahoo! (Yahoo! Slurp)
$ipfw 11200 set 2 deny tcp from 8.12.144.0/24 to me $shoutcast
$ipfw 11201 set 2 deny tcp from 66.196.64.0/18 to me $shoutcast
$ipfw 11202 set 2 deny tcp from 66.228.160.0/19 to me $shoutcast
$ipfw 11203 set 2 deny tcp from 67.195.0.0/16 to me $shoutcast
$ipfw 11204 set 2 deny tcp from 68.142.192.0/18 to me $shoutcast
$ipfw 11205 set 2 deny tcp from 72.30.0.0/16 to me $shoutcast
$ipfw 11206 set 2 deny tcp from 74.6.0.0/16 to me $shoutcast
$ipfw 11207 set 2 deny tcp from 98.136.0.0/14 to me $shoutcast
$ipfw 11208 set 2 deny tcp from 202.160.176.0/20 to me $shoutcast
$ipfw 11209 set 2 deny tcp from 209.191.64.0/18 to me $shoutcast

#
# loathsome web crawlers
#
# SemrushBot keeps crawling despite repeated 403s
$ipfw 11500 set 2 deny tcp from 46.229.168.64/27 to me

# Deny access by musicdna.com (209.239.114.98, id-server-1 thru
# id-server-82.musicdna-interface.com, maybe others)
# See http://forums.winamp.com/showthread.php?t=375367
#
$ipfw_delete set 3
$ipfw 12000 set 3 deny tcp from 209.239.114.0/24 to me
$ipfw 12001 set 3 deny tcp from 5.9.107.103 to me
$ipfw 12002 set 3 deny tcp from 5.9.155.122 to me
$ipfw 12003 set 3 deny tcp from 5.9.51.229 to me
$ipfw 12004 set 3 deny tcp from 5.9.65.18 to me
$ipfw 12005 set 3 deny tcp from 5.9.72.172 to me
$ipfw 12006 set 3 deny tcp from 46.4.50.245 to me
$ipfw 12007 set 3 deny tcp from 46.4.50.76 to me
$ipfw 12008 set 3 deny tcp from 78.46.103.227 to me
$ipfw 12009 set 3 deny tcp from 78.46.107.51 to me
$ipfw 12010 set 3 deny tcp from 78.46.22.220 to me
$ipfw 12011 set 3 deny tcp from 78.46.22.223 to me
$ipfw 12012 set 3 deny tcp from 78.46.22.224 to me
$ipfw 12013 set 3 deny tcp from 78.46.22.226 to me
$ipfw 12014 set 3 deny tcp from 78.46.22.227 to me
$ipfw 12015 set 3 deny tcp from 78.46.22.228 to me
$ipfw 12016 set 3 deny tcp from 78.46.23.30 to me
$ipfw 12017 set 3 deny tcp from 78.46.23.31 to me
$ipfw 12018 set 3 deny tcp from 78.46.23.34 to me
$ipfw 12019 set 3 deny tcp from 88.198.36.231 to me
$ipfw 12020 set 3 deny tcp from 88.198.46.35 to me
$ipfw 12021 set 3 deny tcp from 95.216.3.160 to me
$ipfw 12022 set 3 deny tcp from 95.216.7.88 to me
$ipfw 12023 set 3 deny tcp from 136.243.1.81 to me
$ipfw 12024 set 3 deny tcp from 136.243.17.81 to me
$ipfw 12025 set 3 deny tcp from 144.76.104.237 to me
$ipfw 12026 set 3 deny tcp from 144.76.107.10 to me
$ipfw 12027 set 3 deny tcp from 144.76.117.107 to me
$ipfw 12028 set 3 deny tcp from 144.76.125.197 to me
$ipfw 12029 set 3 deny tcp from 144.76.137.183 to me
$ipfw 12030 set 3 deny tcp from 144.76.159.207 to me
$ipfw 12031 set 3 deny tcp from 144.76.173.86 to me
$ipfw 12032 set 3 deny tcp from 144.76.189.210 to me
$ipfw 12033 set 3 deny tcp from 144.76.217.74 to me
$ipfw 12034 set 3 deny tcp from 144.76.217.76 to me
$ipfw 12035 set 3 deny tcp from 144.76.217.80 to me
$ipfw 12036 set 3 deny tcp from 144.76.217.82 to me
$ipfw 12037 set 3 deny tcp from 144.76.217.83 to me
$ipfw 12038 set 3 deny tcp from 144.76.218.70 to me
$ipfw 12039 set 3 deny tcp from 144.76.218.71 to me
$ipfw 12040 set 3 deny tcp from 144.76.218.73 to me
$ipfw 12041 set 3 deny tcp from 144.76.218.74 to me
$ipfw 12042 set 3 deny tcp from 144.76.218.75 to me
$ipfw 12043 set 3 deny tcp from 144.76.218.79 to me
$ipfw 12044 set 3 deny tcp from 144.76.218.81 to me
$ipfw 12045 set 3 deny tcp from 144.76.218.82 to me
$ipfw 12046 set 3 deny tcp from 144.76.218.86 to me
$ipfw 12047 set 3 deny tcp from 144.76.218.87 to me
$ipfw 12048 set 3 deny tcp from 144.76.219.106 to me
$ipfw 12049 set 3 deny tcp from 144.76.221.131 to me
$ipfw 12050 set 3 deny tcp from 144.76.221.132 to me
$ipfw 12051 set 3 deny tcp from 144.76.221.134 to me
$ipfw 12052 set 3 deny tcp from 144.76.221.135 to me
$ipfw 12053 set 3 deny tcp from 144.76.221.136 to me
$ipfw 12054 set 3 deny tcp from 144.76.237.11 to me
$ipfw 12055 set 3 deny tcp from 144.76.237.12 to me
$ipfw 12056 set 3 deny tcp from 144.76.237.13 to me
$ipfw 12057 set 3 deny tcp from 144.76.237.14 to me
$ipfw 12058 set 3 deny tcp from 144.76.237.16 to me
$ipfw 12059 set 3 deny tcp from 144.76.237.178 to me
$ipfw 12060 set 3 deny tcp from 144.76.237.179 to me
$ipfw 12061 set 3 deny tcp from 144.76.237.67 to me
$ipfw 12062 set 3 deny tcp from 144.76.237.69 to me
$ipfw 12063 set 3 deny tcp from 144.76.237.9 to me
$ipfw 12064 set 3 deny tcp from 144.76.33.13 to me
$ipfw 12065 set 3 deny tcp from 144.76.75.113 to me
$ipfw 12066 set 3 deny tcp from 148.251.21.4 to me
$ipfw 12067 set 3 deny tcp from 148.251.235.145 to me
$ipfw 12068 set 3 deny tcp from 148.251.235.174 to me
$ipfw 12069 set 3 deny tcp from 148.251.236.174 to me
$ipfw 12070 set 3 deny tcp from 148.251.247.21 to me
$ipfw 12071 set 3 deny tcp from 148.251.87.41 to me
$ipfw 12072 set 3 deny tcp from 176.9.140.2 to me
$ipfw 12073 set 3 deny tcp from 176.9.18.184 to me
$ipfw 12074 set 3 deny tcp from 178.63.12.130 to me
$ipfw 12075 set 3 deny tcp from 178.63.23.84 to me
$ipfw 12076 set 3 deny tcp from 178.63.67.47 to me
$ipfw 12077 set 3 deny tcp from 178.63.83.80 to me
$ipfw 12078 set 3 deny tcp from 199.189.84.115 to me
$ipfw 12079 set 3 deny tcp from 199.189.85.100 to me
$ipfw 12080 set 3 deny tcp from 209.126.120.77 to me
$ipfw 12081 set 3 deny tcp from 209.126.122.20 to me
$ipfw 12082 set 3 deny tcp from 209.239.114.98 to me

#
# Deny access by MarkMonitor (easier said than done)
$ipfw 12200 set 3 deny all from 64.124.14.0/25 to me
#
# Deny access by Cyveillance
$ipfw 12300 set 3 deny all from 38.100.21.0/24 to me
$ipfw 12301 set 3 deny all from 38.100.41.64/26 to me
$ipfw 12302 set 3 deny all from 38.104.29.36/30 to me
$ipfw 12303 set 3 deny all from 38.104.57.28/30 to me
$ipfw 12304 set 3 deny all from 38.104.206.136/30 to me
$ipfw 12305 set 3 deny all from 38.105.71.0/25 to me
$ipfw 12306 set 3 deny all from 38.105.109.168/29 to me
$ipfw 12307 set 3 deny all from 38.105.109.192/29 to me
$ipfw 12308 set 3 deny all from 38.109.157.208/29 to me
$ipfw 12309 set 3 deny all from 38.127.194.192/27 to me
$ipfw 12310 set 3 deny all from 38.127.197.64/26 to me
#
# disallow IGMP traffic (used by multicast and nosey routers)
#
$ipfw_delete set 4
$ipfw 13000 set 4 deny igmp from any to any

# Instead of using firewall_myservices in /etc/rc.conf, allow anyone to
# these ports only after checking they don't meet any earlier rules.
#
$ipfw 64000 set 30 allow tcp from any to me 25
$ipfw 64010 set 30 allow tcp from any to me 80
$ipfw 64020 set 30 allow tcp from any to me 443
$ipfw 64030 set 30 allow tcp from any to me 465
$ipfw 64040 set 30 allow tcp from any to me 587
$ipfw 64050 set 30 allow tcp from any to me 853
$ipfw 64060 set 30 allow tcp from any to me 6277
$ipfw 64070 set 30 allow tcp from any to me 7778
$ipfw 64080 set 30 allow tcp from any to me 7779
$ipfw 64080 set 30 allow tcp from any to me 7780
$ipfw 65090 set 30 allow tcp from any to me [redacted... it's my SSH port]

(Set #1, enforcing only loopback connections to MySQL, is redundant unless you have 3306/tcp in firewall_myservices, but it makes me feel better to have the explicit rules for it.)

You might want to block all of the Amazon Web Services ranges as well:

  • echo '\n#\n# disallow traffic from Amazon Web Services\n#\n$ipfw_delete set 5' >> /etc/ipfw.rules
  • fetch https://ip-ranges.amazonaws.com/ip-ranges.json
  • grep ip_prefix ip-ranges.json | cat -n | awk '{gsub(/[^0-9\.\/]/,"",$3); printf "$ipfw %d set 5 deny ip from %s to me\n",$1+14000,$3}' >> /etc/ipfw.rules
  • rm ip-ranges.json

Anyway, after you edit firewall settings in /etc/rc.conf:

  • service ipfw restart

And after you edit firewall rules in /etc/ipfw.rules (assuming they're written safely, like I did above):

  • sh /etc/ipfw.rules

Enable unattended file system repair

If you have been doing a lot of unclean shutdowns, or your disk is just flaky, you want to continue to allow fsck to do its default behavior of running at bootup when needed.

Although it is safest to do it this way, it can take a long time, and you can't log in or do anything until it finishes. Sometimes it may even demand human interaction at the console.

Disabling the need for human interaction is easy:

  • ee /etc/rc.conf

Add this to /etc/rc.conf: 

# in case of unclean shutdown, allow fsck to run at boot without interaction
# (with risk of an overzealous "repair" occurring)
fsck_y_enable="YES"

If you also want to speed it up, you can add this to make the fsck not start until the daemons are started and the system is already in multi-user mode: 

# delay fsck until after boot (risky if daemons access corrupt files)
background_fsck="YES"
background_fsck_delay="120"

The downside of this is there is a risk that the daemons will try to use corrupt files, which could be catastrophic. I try not to use this option until I am confident the system is running smoothly.

Enable TRIM on UFS file systems on SSDs

TRIM prolongs the life of drives that use flash memory. UFS supports it, but you have to enable it. See http://www.wonkity.com/~wblock/docs/html/ssd.html for an explanation.

Swap partitions cannot use TRIM, unfortunately.

On my BeagleBone Black, the root file system is on a micro SD card. I couldn't find any info to confirm that my particular card supports TRIM, but apparently most do. It's probably safe to assume that snapshots do not have TRIM enabled. So, to enable TRIM on the root file system, do this:

  • shutdown now
  • mount -u -r -f /
  • tunefs -t enable /
  • mount -u -w /
  • reboot

I have also read that the need for TRIM is overstated, because only the most heavily loaded servers run the risk of wearing out flash drives. A typical server has nothing to worry about.

And, I have read that drives with "SLC" NAND are far more reliable than those with "TLC".[1]

Reboot

If you did not enable TRIM, I would test at this point and make sure the system can do a reboot without anything crazy happening.

  • shutdown -r now – using 'shutdown' instead of 'reboot' ensures daemons are properly stopped.

It helps to have the console visible so you can see the messages, but as long as the system comes back up, you can see the console contents with dmesg -a.

Change the SSH port

Public servers get hammered by attacks on port 22. The attacks will drop off sharply if you just move to a different port.

  • ee /etc/ssh/sshd_config
  • Uncomment Port 22 and change the 22 to a number between 1024 and 65535, and that preferably isn't used for anything else. Exit the editor.
  • While you are in there, set UseDNS to no. See https://unix.stackexchange.com/questions/56941/what-is-the-point-of-sshd-usedns-option for details. Basically, there are several reasons to leave it off, but the main one for me was that there's a chance that DNS won't work (e.g. because Unbound could not start) which can interfere with SSH access.
  • Exit the editor.
  • service sshd reload

You won't get kicked out of any existing SSH sessions, but you should now only be able to establish new ones via the new port.

Configure user accounts

You want to make it so that you're never logging in as the superuser (root), but rather log in as a user who is in the wheel group and thus has permission to use the su command to become the superuser. Aside from a modicum of security, it allows you the freedom to change your default shell.

Change root password

FreeBSD snapshots come with 'root' as the root user's password. This is a terrible password. Choose a better one (max. 128 characters) and set it:

  • passwd

Add a wheel-group user account

  • adduser

Use defaults for most choices, but set the shell to tcsh and enter wheel when asked about other groups. Here's a sample session: 

# adduser
Username: mike
Full name: Mike Brown
Uid (Leave empty for default):
Login group [mike]:
Login group is mike. Invite mike into other groups? []: wheel
Login class [default]:
Shell (sh csh tcsh nologin) [sh]: tcsh
Home directory [/home/mike]:
Home directory permissions (Leave empty for default):
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]:
Enter password:
Enter password again:
Lock out the account after creation? [no]:
Username   : mike
Password   : *****
Full Name  : Mike Brown
Uid        : 1002
Class      :
Groups     : mike wheel
Home       : /home/mike
Home Mode  :
Shell      : /bin/tcsh
Locked     : no
OK? (yes/no): yes
adduser: INFO: Successfully added (mike) to the user database.
Add another user? (yes/no): no
Goodbye!

Log in as a regular user

  • exit – Log out of the root account. If you're on the console, you'll get the login prompt again. Now's a good time to try connecting via SSH, but you can stay at the console if you want.
  • Log in with the regular user name and password you created above.
  • Repeat the steps to set up your terminal, then come back here.

From now on, log in as this user and always use su - (or, I prefer su -m) when you want to do things as root. The rest of this guide assumes you've done that.

Disable root login via SSH

  • su -m – Become root. Enter the new root password you created above.
  • ee /etc/ssh/sshd_config – Uncomment the PermitRootLogin no line, and exit the editor.
  • service sshd reload will make it take effect now, or you can wait till next reboot.

Remove 'freebsd' user account

The FreeBSD armv6 snapshots apparently come with a regular user account named 'freebsd', with the password 'freebsd'. Get rid of it!

  • rmuser -yv freebsd

If you have Internet access and time to spare, now would be a good time to update to the very latest source code and rebuild the system, just to make sure you will be able to do it when the time comes. See the following sections of this document: 1. #Set build options in /etc/make.conf, 2. #Create swap space, 3. #Updating the system from source code.

Configure SSH for user

The first convenience I like to set up is being able to log in via SSH or SCP without being prompted for my account password.

Inbound

Make sure you are not root when doing this.

  • mkdir ~/.ssh
  • ee ~/.ssh/authorized_keys
  • Paste in the public key(s) from the SSH client(s) you'll be using to log into the BeagleBone with, one per line, then exit the editor. The keys need to be in OpenSSH format (all the key data on one line). If it's a block of text beginning with "BEGIN SSH2 PUBLIC KEY" then it's the wrong format.
  • exit and log in again, and verify that you are not prompted for a password. Being prompted for a passphrase is OK, if your key requires it.

Outbound

Generate private & public keypairs for connecting to other servers via ssh or scp:

  • ssh-keygen -t ecdsa – generate ECDSA keypair for connecting to newer servers (OpenSSH 5.7 & up)
  • ssh-keygen -t rsa – generate RSA keypair for connecting to older servers

Copy the contents of the public key files (the ones whose names end in '.pub') into the ~/.ssh/authorized_keys file on the hosts you'll be connecting to.

I like to also be extra safe and speedy when I am connecting other hosts (make sure you set echo_type like I told ya):

  • echo '# see ssh_config(5) for options\nCheckHostIP yes\nCompression yes' > ~/.ssh/config

If needed, further edit ~/.ssh/config and add any other special things you need for connecting to specific hosts. For example, in root's config, if you want it to always use a particular non-root 'foo' identity: 

# see ssh_config(5) for options
CheckHostIP yes
Compression yes
Host *example.org
  Port 22
  User foo
  IdentityFile ~foo/.ssh/id_rsa

Customize user terminal and shell

At this point I just use scp to copy some files over from my other servers, to provide the terminal & shell behavior I'm used to.

  • scp -p example.org:'{.cshrc,.login,.alias,.colors,.nanorc}' ~
  • mkdir ~/.nano && scp -p example.org:.nano/'{README,tcsh-*}' ~/.nano
  • cd ~/.nano && ln -s /usr/local/share/nano NANOCONFIDIR && ln -s tcsh-bsd.nanorc tcsh.nanorc

See my fancy ~/.cshrc and ~/.login for info about these files.

I also like this to be in ~/.alias.local, so when at the console I can run bbb after logging in, and any other time I need a terminal reset. This is BeagleBone & Tera Term-specific, of course: 

# alias to reset terminal, for use with Tera Term, which only sets vt100 terminal type and does not set rows & columns
# ... relies on 'vtn' alias as well
alias bbb 'setenv TERM xterm-256color; setenv TERMCAP xterm-256color:ti@:te@:tc=xterm-256color; stty rows 46 cols 132; source ~/.cshrc; vtn; echo BBB terminal reset.'

Install the ports collection and portmaster

Don't install pkg

If you try to run pkg, it will install an old "bootstrap" version. The better option is to install the ports collection, then install portmaster (see next section). Portmaster depends on pkg and will install a current version from the ports collection.

Fetch current ports

This will take forever.

  • portsnap fetch extract

If you are using FreeBSD-CURRENT, several lock order reversal messages with stack traces come up. These are normal and are for debugging FreeBSD-CURRENT. See https://www.freebsd.org/doc/faq/troubleshoot.html#idp59162064 

root@beaglebone:~ # portsnap fetch extract
Looking up portsnap.FreeBSD.org mirrors... 7 mirrors found.
Fetching public key from isc.portsnap.freebsd.org... done.
Fetching snapshot tag from isc.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Fetching snapshot generated at Sat Jul 18 00:01:57 UTC 2015:
0254d062f604bc2ee66614ff8a9f9158847d0dea1cb903100% of   75 MB  863 kBps 01m30s
Extracting snapshot... done.
Verifying snapshot integrity...
[huge list of ports folders trimmed]
Building new INDEX files... done.

Install portmaster

  • cd /usr/ports/ports-mgmt/portmaster
  • make install

Set build options in /etc/make.conf

Settings in make.conf affect everything built with make, including software in the ports collection. The make.conf(5) manual page explains some of the options available for building kernel and world, but you can also include anything specific to ports, such as to provide default answers to things you would set in the 'make configure' stage. See the ports(7) man page and /usr/ports/Mk/bsd.options.desc.mk for common options (minus the WITH_ or WITHOUT_ prefix, in the latter). Unfortunately, many other options are buried in the individual port Makefiles, not documented anywhere.

  • ee /etc/make.conf
##
## options for 'make update' (of world, ports, docs):
##
# use svnlite(1) or svn(1) for source updates; CVS_UPDATE and SUP_UPDATE are no longer supported
SVN_UPDATE=yes
# use svnlite(1) which comes with FreeBSD 10 & up; otherwise it tries /usr/local/bin/svn (svn from ports)
SVN=svnlite

##
## options for 'make buildworld':
##
# when building top(1), only allocate enough space to handle 75 users, rather than 10000
TOP_TABLE_SIZE=151

##
## options for building ports:
##
# WITH_* and WITHOUT_* are deprecated in favor of OPTIONS_SET and OPTIONS_UNSET
# I have just not removed the old settings above because some ports still use them

# my non-Intel CPU (armv6) does not support SSE or MMX
OPTIONS_UNSET+= MMX SSE
WITHOUT_MMX=yes
WITHOUT_SSE=yes

# support IPv6 and HTTPS
OPTIONS_SET+= HTTPS IPV6
WITH_HTTPS=yes
WITH_IPV6=yes

# ports needing OpenSSL should use LibreSSL
# (options are base, openssl, openssl-devel, libressl, libressl-devel).
# some ports have issues, e.g. ftp/curl must be built with the TLS-SRP option disabled
OPTIONS_SET+= SSL_PORT OPENSSL_PORT
DEFAULT_VERSIONS+= ssl=libressl

# don't build or install GUIs, including X11 libraries
OPTIONS_UNSET+= GUI X11
WITHOUT_GUI=yes
WITHOUT_X11=yes

# don't waste time on tests when building ImageMagick
OPTIONS_UNSET+= IMAGEMAGICK_TESTS
WITHOUT_IMAGEMAGICK_TESTS=yes

# when building FreeType, enable subpixel rendering capability (disabled by default due to patent issues)
WITH_LCD_FILTERING=yes
OPTIONS_SET+= LCD_FILTERING

# Avoid dialogs asking to accept certain licenses
LICENSES_ACCEPTED=DCC

As of Nov. 2016, many ports still use the WITH and WITHOUT options, so that's why I keep them in there instead of just using OPTIONS_SET and OPTIONS_UNSET. This means that some ports will issue warnings about the deprecated options, but I don't see any alternative.

See also my ccache installation instructions.

Networking

You won't be able to do much else if you're not connected to the Internet. The network interface needs to be up and you need to be able to reach remote hosts on the Internet, with the assistance of a DNS server.

Useful commands

  • service netif restart – reset the network configuration, picking up any /etc/rc.conf config changes
  • service routing restart – reset the routing tables to sane defaults
  • resolvconf -u – run resolvconf to pick up updated settings; may regenerate /etc/resolv.conf

BeagleBone NIC failure possibility

The boot log of my first attempt to run FreeBSD 10-STABLE in July 2015 showed cpsw0 as the network interface, but there was no sign of it when running ifconfig. I asked about it on the freebsd-arm list. Someone said via private email that the BeagleBone's NIC fails at random on startup, for reasons unknown. I set it aside and waited for a new snapshot. All boots since then have worked fine.

Optional: Disable IPv6 if unsupported upstream

The BBB supports IPv6 and it should just work, but if your other LAN hardware or ISP only supports IPv4, you have the option of disabling IPv6 in the kernel:

  • echo ipv6_enable="NO" >> /etc/rc.conf

I assume a reboot is needed after doing this, although service netif restart && service routing restart might work. The effect should be a (perhaps imperceptible) speedup of network operations, because it won't be trying IPv6 before falling back to IPv4.

Similarly, you can prevent IPv6 support from being built in ports:

  • echo WITHOUT_IPV6=yes >> /etc/make.conf

Personally, I would leave IPv6 enabled unless you notice problems.

/etc/hosts

This list of hostname-to-IP-address mappings is a supplement to the DNS system, not part of it. The Unbound server does not access it. The hosts file is normally only consulted by software which uses standard C library (libc) functions to look up hostnames/IPs, and those functions look in /etc/hosts before trying to get the info via the DNS system (i.e. actual DNS servers) (/etc/nsswitch.conf settings can affect this). So when you run curl, for example, /etc/hosts is consulted, but when you run host, it won't be consulted, because that program is specifically just for looking up things via DNS servers.

It is a good idea to keep this file small, because it is parsed and its data is cached by each process that does those lookups.

As mentioned previously, I suggest doing the following:

  • Add your hostname (the output of hostname -s, and hostname -f if it's different) to the localhost aliases.
  • Add an entry for a known NTP time server.

Here are the non-comment lines from my /etc/hosts: 

::1        localhost localhost.my.domain beaglebone
127.0.0.1       localhost localhost.my.domain beaglebone
128.138.141.172         timenistgov
2610:20:6f15:15::27     timenistgov

I left the "localhost.my.domain" in there as a reminder to myself to update it when I move to a resolvable domain. (Then I'll be updating the hostname in /etc/rc.conf as well.)

resolvconf

/etc/resolv.conf is where your DNS servers are normally listed, but this file is no longer supposed to be edited manually by default. Instead, you let resolvconf(8) manage it for you. It has its own config file, /etc/resolvconf.conf (which does not exist by default).

Basically, software that wants to write to /etc/resolv.conf has to do it through resolvconf. Those apps are "subscribers" to resolvconf's service. Resolvconf takes their version of resolv.conf as a suggestion, applies its own rules to amend it, and then takes care of doing whatever needs to be done itself.

I think this /etc/resolvconf.conf will work for me as long as I am not running my own local resolver (e.g. Unbound): 

# Don't forget to run 'resolvconf -u' after changing this file

# Always-good resolver IP addresses to prepend to the list
# 75.75.75.75 & 75.75.76.76 = Comcast; 8.8.8.8 = Google
name_servers="75.75.75.75 75.75.76.76 8.8.8.8"

# The DHCP server in our Apple AirPort Time Capsule (6th Gen.) assigns itself
# as the resolver, but is apparently blocking DNSSEC, so let's never use it
name_server_blacklist="10.0.1.1"

DHCP

DHCP is enabled by default (/etc/rc.conf contains ifconfig_DEFAULT="DHCP") so it should just work if you boot while connected (e.g. by Ethernet cable) to a DHCP-enabled router.

A line in /etc/rc.conf assigns a hostname of "beaglebone". Remove this line if you expect your DHCP server to assign a different hostname.

Overriding DHCP-assigned DNS

I think the /etc/resolvconf.conf above will work for me, but it's possible that /etc/resolv.conf will be automatically rewritten by dhclient directly when leases are renewed. Perhaps this only happens when resolvconf is disabled or prevented from rewriting /etc/resolv.conf; I'm not sure.

If dhclient is going to be touching /etc/resolv.conf, then I think I'll need to put another list of nameserver overrides into /etc/dhclient.conf. For example, I could add prepend domain-name-servers 75.75.75.75 75.75.76.76; to the /etc/dhclient.conf section for my network interface.

However, when running my own nameserver (Unbound; see below), there is a different procedure.

Yes, this is all very confusing.

Static IP address

Get your address, subnet mask, and router address. Append to /etc/rc.conf, replacing "#" as necessary:

For IPv4: 

ifconfig_cpsw0="inet #.#.#.# netmask #.#.#.#"
defaultrouter="#.#.#.#"

For IPv6: 

ipv6_ifconfig_cpsw0="inet #:#:#:#"
ipv6_defaultrouter="#:#:#:#"

It's OK to leave ifconfig_DEFAULT="DHCP" in as a fallback.

More info is in the manual pages for rc.conf and ipconfig.

Be ready to access the console if something goes wrong when you restart the network (this may kill your connections, even at the console on the BeagleBone):

  • service netif restart

You should see that resolvconf has rewritten /etc/resolv.conf to not contain DHCP-assigned info.

Unbound

Unbound is the BIND replacement in FreeBSD 10 and up. It is a DNSSEC-enforcing, caching resolver.

It's good to have access to a caching resolver on your own network so you're not constantly looking up the same domain names on your ISP's DNS server. It's especially recommended if you're going to be running public services which do DNS lookups, like mail.

My current notes for Unbound are here: Unbound on FreeBSD 10.

Create swap space

Generally you should dedicate part of your disk space to be used as if it were extra memory. This is called swap space. These are your options for creating swap space (pick one):

There are performance, convenience, and security tradeoffs, of course. The same drive is usually slower than separate drive. Files are usually slower than partitions. Encrypted is slower than unencrypted. And on these solid-state drives, partitions (since they can't use TRIM) will wear out faster than files on TRIM-enabled file systems (but for something as lightly used as swap, this is not really a concern).

Also, I am not 100% sure files are the way to go on the BeagleBone. In my testing (mainly with an encrypted swap file), sometimes I would get random errors when swap was used, as if there was RAM data corruption. I have no idea what the actual cause is, though. It could be unrelated.

Use the eMMC Debian partition for swap

If you don't want to use the Debian partition anymore, you can designate all 3.5 GB of it for swap. This will destroy the Debian installation—so you probably first want to enable permanent booting from the SD card. And of course, after designating the partition as swap, you will need to remove any swap files you created earlier. Do it in a sensible order, so you're never caught without swap. For example:

  • swapon /dev/mmcsd1s2.eli — i.e., start using the eMMC for swap immediately. You will get a warning about having too much swap; ignore it.
  • service encrypted_swapfile stop or swapoff /dev/md1.eli — this will take a while if pages need to be transferred from the old swap to the new.
  • Add a line to /etc/fstab:
/dev/mmcsd1s2.eli	none	swap	sw	0	0
  • Remove from /etc/fstab any references to swap files.
  • Delete the actual swap files from the system—e.g., rm /usr/swap0 /usr/swap1
  • In /etc/rc.conf, change encrypted_swapfile_enable to "NO" (assuming you had it as "YES").

Unencrypted swap file

Here's how to make a 2 GB unencrypted swap file:

  • Create a 2 GB file by doing one of the following:
    • truncate -s 2G /usr/swap0 – creates the file /usr/swap0 without filling it; it will grow when used
    • dd if=/dev/random of=/usr/swap0 bs=1m count=2000 — creates & fills the file /usr/swap0 with zeroes
    • dd if=/dev/random of=/usr/swap0 bs=1m count=2000 — creates & fills the file /usr/swap0 with random bytes

The truncate method is the fastest and best option for use on a solid-state drive like in the BeagleBone. It is as if the file is filled with zeroes, but it doesn't actually take up space until data is written to it.

  • chmod 0600 /usr/swap0
  • Add a line to /etc/fstab:
md99	none	swap	sw,file=/usr/swap0	0	0
  • swapon -aL

Encrypted swap file

Encrypted swap space improves security but penalizes performance.

To set it up, in theory, in /etc/fstab you could say md99.bde or md99.eli to make the previous example be encrypted, but it doesn't seem to work. This may be a bug. Discussion: https://forums.freebsd.org/threads/encrypt-swap-file.44519/

As per that thread, instead of using an entry in /etc/fstab, you can use a script that runs at startup to successfully create & mount an encrypted swap file. It works for me if I just save the following to /usr/local/etc/rc.d/encrypted_swapfile: 

#!/bin/sh

# PROVIDE: encrypted_swapfile
# REQUIRE: swaplate
# BEFORE: LOGIN
# KEYWORD: shutdown

. /etc/rc.subr

name="encrypted_swapfile"
rcvar="encrypted_swapfile_enable"

start_cmd="${name}_start"
stop_cmd="${name}_stop"

load_rc_config $name
: ${encrypted_swapfile_enable:="NO"}
: ${encrypted_swapfile_file:="/usr/swap0"}
: ${encrypted_swapfile_size:="2G"}

SWFILE="$encrypted_swapfile_file";
SWFILEDIR=$(dirname "$SWFILE");
SWDEVLINK="/var/run/encrypted_swapfile_device";

encrypted_swapfile_start()
{
	# Create and mount a one-time encrypted swap file.
	# This is a workaround for the inability to do this via an /etc/fstab entry.
	# See https://forums.freebsd.org/threads/encrypt-swap-file.44519/#post-292933
	#
	if [ ! -e "$SWDEVLINK" ]; then
		if [ -w "$SWFILEDIR" ]; then
			truncate -s "$encrypted_swapfile_size" "$encrypted_swapfile_file" &&
			chmod 0600 "$encrypted_swapfile_file" &&
			SWMD=$(mdconfig -a -t vnode -f "$encrypted_swapfile_file") &&
			if [ $? -eq 0 ] && [ -n $SWMD ] && [ -e "/dev/$SWMD" ]; then
				chmod 0600 "/dev/$SWMD" &&
				geli onetime -e AES-XTS -l 256 -d "/dev/$SWMD" &&
				chmod 0600 "/dev/$SWMD.eli" &&
				swapon "/dev/$SWMD.eli" &&
				ln -f -s "/dev/$SWMD.eli" "$SWDEVLINK";
				unset SWMD;
			fi
		else
			echo "Could not create encrypted swap file in $SWFILEDIR; check permissions." &&
			return 1;
		fi
	else
		SWMD=$(readlink "$SWDEVLINK") &&
		swapinfo | grep -vq "^$SWMD " &&
		echo "Encrypted swap file already exists; enabling." &&
		swapon "$SWMD";
		unset SWMD;
	fi
	return 0;
}

encrypted_swapfile_stop()
{
	if [ -e "$SWDEVLINK" ]; then
		swapoff "$SWDEVLINK" &&
		rm "$SWDEVLINK" &&
		rm "$SWFILE";
	else
		echo "No encrypted swap file found; nothing to stop.";
		rm -f "$SWDEVLINK";
	fi
}

run_rc_command "$1"

Then you can just run service encrypted_swapfile start and get the swap space immediately. In /var/log/messages, or if you reboot, you should see: 

GEOM_ELI: Device md1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: software

Verify with swapinfo that it's working: 

Device          1K-blocks     Used    Avail Capacity
/dev/md1.eli      2097152        0  2097152     0%

The script above ensures this swap device is destroyed when shutdown(8) is run. Otherwise, if the file is not empty, the kernel will panic after unmounting the file systems.

Updating the system with freebsd-update

freebsd-update allows you to update the base system (or whatever you have configured in /etc/freebsd-update.conf) in order to e.g. easily keep up with security patches. Unfortunately, this is not yet an option on ARM devices like the BeagleBone Black because (as of 2020) it is still not a "Tier 1" architecture. Sorry!

Updating the system from source code

BeagleBone is not supported by FreeBSD 12.2 and 12.3 due to a bug affecting the clock and I/O. It should be fixed in FreeBSD 13. Don't try to upgrade to 12.2 or 12.3 in the meantime.

Since I installed FreeBSD from a snapshot, it doesn't include /usr/src, the usual home of the base system sources and docs. It's handy to at least have the base system source code, though, because you need it in order to apply security patches and to rebuild the OS or components thereof.

The source code is in a Subversion repository. To fetch it, you need to choose which client to use:

  • svnlite, a lightweight client which comes with FreeBSD and is functionally identical to svn.
  • svn, the standard, bloated client installed by the devel/subversion port.
  • svnup, a dedicated source-pulling client installed by the net/svnup port.

The problem with svnlite and svn is they keep a 2nd "pristine" copy in /usr/src/.svn. The source code is a little over 1 GB, so that means over 2 GB of disk space will be needed if you use either of those clients. svnup is better in this regard; it doesn't keep a pristine copy. So if all you care about is fetching the latest source code, not tracking and submitting your own changes to it, I recommend using svnup.

That said, I use svnlite because it doesn't require installing more software, and I have plenty of disk space.

Fetch system source code

Preparation:

  • Make sure /usr/src is empty or nonexistent: rm -fr /usr/src
  • Make sure you have enough disk space: df -h

To obtain the system source code for the first time, assuming /usr/src does not exist:

This URL is for FreeBSD 11-STABLE, of course.

The reason for setting TMPDIR to /var/tmp is because svnlite defaults to using /tmp, which on the snapshots for the BeagleBone is configured (in /etc/fstab) to use a memory file system of only 50 MB, too small to support a checkout of the base system.

Update system source code

Once fetched, you can thereafter just do an update instead of the full checkout.

Preparation:

  • Add to /etc/make.conf: SVN_UPDATE=yes (it's needed by 'make update')
  • If you have not installed svn, then also add to /etc/make.conf: SVN=svnlite (it's needed by 'make update').

To update the source code, either of these commands will do the same thing:

  • env TMPDIR=/var/tmp svnlite update /usr/src

or

  • cd /usr/src; env TMPDIR=/var/tmp make update

However, these will only update the unmodified files in /usr/src. Modified files will be left intact. So if you previously patched some files, they are not going to get replaced, even if you do a full checkout!

These commands will come in handy for dealing with this situation (change /usr/src as needed to only look at certain directories):

  • env TMPDIR=/var/tmp svnlite status /usr/src – tells you what's out-of-sync (ideally, outputs nothing)
  • env TMPDIR=/var/tmp svnlite revert -R /usr/src – shows diffs of all the changed files
  • env TMPDIR=/var/tmp svnlite revert -R /usr/src – reverts most changes

The revert command does not remove extra files you have added, like '*.orig' files left over from patching. For that it's best to just delete the affected files manually, or delete the folder and use svnlite revert -R on it.

If you are really desperate, it's safe to obliterate your entire /usr/src and start over.

An update will only get recent patches to the branch you checked out. If you instead want to upgrade—switch to a new version of FreeBSD—even just to do a minor version bump, you will need to first do this (but substitute the branch you want, of course):

Optional: apply custom patches

If you need patches that aren't yet committed to the official source code, you can apply them and then either rebuild the full system or just the affected portions.

Example, assuming certain patches for Unbound on FreeBSD 10 are in your home directory:

  • patch /usr/src/contrib/unbound/util/configlexer.lex ~/configlexer.patch
  • patch /usr/src/usr.sbin/unbound/local-setup/local-unbound-setup.sh ~/local-unbound-setup.patch

If you want to rebuild the whole system, proceed to the next section. If you instead want to just rebuild Unbound, for example, do this:

  • cd /usr/src/lib/libunbound && make obj && make depend all install
  • cd /usr/src/usr.sbin/unbound && make obj && make depend all install

The exact procedure can vary depending on what was affected, but generally it involves doing those makes from within the proper source directory. Don't try to build things directly in /usr/src/contrib.

If patching fails

If patching fails, you can just grab the whole folders you need via Subversion.

For example, I tried to apply patches to ntp as directed in a security advisory, but quite a few of the patches failed. The affected files were all in /usr/src/contrib/ntp and /usr/src/usr.sbin/ntp, so this is what I did:

It prompted me for a few conflicts. I entered tc ("their side of conflict") to keep updated files, or r ("mark resolved") to keep totally new files.

Since ntpd does not include libraries needed by the rest of the system, I saw no reason to heed the security advisory's instructions to do a full buildworld/installworld:

  • cd /usr/src/usr.sbin/ntp && make obj && make depend all install
  • service ntpd restart

Ensure adequate swap space

The FreeBSD snapshots for the BeagleBone don't include swap space, which is disk space used as extra RAM. Some parts of the system require a lot of RAM to build. If you don't allocate some swap space, then about 9 hours into it, make buildworld will fail when compiling lib/clang/libllvmx86disassembler.

So if you haven't done so already, follow the directions in the preceding section to create swap space.

Ensure adequate temp space

If you do not have enough room in /tmp, then partway through the build or install process, you get an error message like "objcopy: elf_update() failed: I/O error: No space left on device". I don't know how much room in /tmp is needed, but the BeagleBone snapshots configure /tmp to be a 30 or 50 MB RAM disk, which is too small. A simple workaround is umount tmpfs which will unmount the RAM disk and return to using the regular file system for /tmp.

Before you can unmount it, first you have to make sure nothing is using /tmp. On my system, that means stopping MySQL and any PHP processes. I normally do this anyway, but it is easy to forget, especially after the post-installkernel reboot.

  • service mysql-server stop (actually I have a script for this which does other stuff too)
  • service php-fpm stop
  • service ttrssd stop
  • umount tmpfs

Discussion: https://forums.freebsd.org/threads/58749/

Optional: exclude optional components of Clang

At some point in 2017, building world began taking roughly triple the amount of time it used to, due to updates to the C compiler (clang) and the inclusion of optional parts which used to be excluded.

The buildworld time can be reduced slightly by excluding the optional parts of the Clang C/C++ compiler. Just add this to /etc/src.conf: 

WITHOUT_CLANG_EXTRAS=yes
WITHOUT_CLANG_FULL=yes
WITHOUT_LLDB=yes

See the src.conf man page for details of what these flags do. (Supposedly WITHOUT_LLDB=yes is the default on armv6, but it doesn't hurt to specify it anyway.)

Optional: install and configure ccache

devel/ccache will help reduce build times by caching and reusing certain outputs of the C compiler.

I think you'll need about 1.6 GB of cache for a full buildworld/buildkernel. By default it sets up 5 GB, so you should be OK.

See my ccache-enabling instructions.

Optional: enable meta mode

In 2015, experimental optimizations called meta mode and DIRDEPS_BUILD were added to the build system. In 2017, these options were greatly enhanced and refactored.

I have a hard time making sense of the original documentation describing how it works, and it seems what is now called DIRDEPS_BUILD is what used to be called meta mode, just to make things extra confusing. The best documentation I can find now is at https://xmj.github.io/articles/sysadmin/builds_ccache_memcached.html and the WITH_META_MODE / WITH_DIRDEPS_BUILD / WITH_DIRDEPS_CACHE explanations in the src.conf(8) man page.

To enable meta mode,

  • make sure WITH_META_MODE=yes is in /etc/src-env.conf (creating this file if necessary)
  • run kldload filemon (if filemon.ko is not already loaded)

Then, if you do not obliterate /usr/obj before every build, the build system can utilize metafiles produced by the previous runs of make(1) in order to decide whether existing files are out of date and thus whether a new build is needed.

The first time you do this, I expect it will not be any faster than usual. But thereafter, you should be able to update the source and then your buildworld/buildkernel will take less time because only the newly updated parts will be built.

I have not yet tested running with the DIRDEPS options.

Optional: shut down non-essential services

I have had bad luck with my BeagleBone Black going haywire when it gets too busy.

Consider shutting down any non-essential services while the build is running, so as much CPU and RAM as possible is available for the build.

I also temporarily disable cron jobs which might do memory-intensive things, e.g. database maintenance.

Check for new issues and features

Look at /usr/src/UPDATING and see what's new. Sometimes there is something important to know about or do. (For example, when upgrading to FreeBSD 12, before doing installworld, you have to make sure there's an ntpd user and group!)

For the next steps, there is a recipe at the top of /usr/src/Makefile and there is a slightly different one at the 'Rebuilding World' section of The FreeBSD Handbook. What I am presenting here is my own sequence which is mostly the same, but with enhancements and explanations.

Optional: remove remnants of past builds

If you want to do a full rebuild from scratch, then first remove all traces of old builds:

  • if ( -d /usr/obj ) chflags -R noschg /usr/obj && rm -rf /usr/obj – this is partly my own construction.

If you are trying to use meta mode or DIRDEPS_BUILD, you probably don't want to do this, unless it is your first build of this major version of FreeBSD, or unless there was a problem.

Optional: protect against disconnection

If you are at the console, or if you want the build/install process to stop if you get disconnected, then you don't have to do anything special. But if you are connecting over a network via SSH, you should plan for the possibility of getting disconnected.

One option is to install and run GNU screen (the sysutils/screen port), and do everything in a virtual terminal. You can just reattach to that terminal if you get disconnected.

If you use tcsh, another option is to tell the shell not to terminate your processes if you get disconnected:

  • set nohup

You won't be able to reattach, but you can be pretty sure that your buildworld or whatever ran to completion (or fatal error).

When upgrading from FreeBSD 10 or 11 to FreeBSD 12 or newer, it's very easy to overlook this comment in /usr/src/UPDATING: 

20171005:
    The arm port has split armv6 into armv6 and armv7. armv7 is now
    a valid TARGET_ARCH/MACHINE_ARCH setting. If you have an armv7 system
    and are running a kernel from before r324363, you will need to add
    MACHINE_ARCH=armv7 to 'make buildworld' to do a native build.

You actually have to add it to all three commands:

  • script /var/tmp/buildkernel.out make buildworld MACHINE_ARCH=armv7
  • script /var/tmp/buildkernel.out make buildkernel KERNCONF=BEAGLEBONE MACHINE_ARCH=armv7
  • script /var/tmp/installkernel.out make installkernel KERNCONF=BEAGLEBONE MACHINE_ARCH=armv7

Failure to do this will result in armv6 binaries, which will still work, but can interfere with the use of pkg; you'll see messages like "wrong architecture: FreeBSD:12:armv7 instead of FreeBSD:12:armv6".

If you accidentally built armv6 code and want to redo it right, make sure to first remove or rename /usr/obj, and if using ccache, run ccache -C before trying again.

Build world and kernel

  • cd /usr/src
  • script /var/tmp/buildworld.out make buildworld – my own modification of advice to use 'script'. Building world for 12-RELEASE from scratch takes over 73 hours on the BBB (and 12.2 takes over 110 hours!). Adding '-j4' doesn't help a bit.
  • script /var/tmp/buildkernel.out make buildkernel KERNCONF=BEAGLEBONE – takes about 5 hours on the BBB.

With ccache and meta mode, however, a rebuild with minimal changes takes less than 3 hours (world) and 2 hours (kernel)!

Install kernel

  • script /var/tmp/installkernel.out make installkernel KERNCONF=BEAGLEBONE – only takes a minute or two.

Optional: reboot and maybe drop to single-user mode

In theory, it is safer to reboot (so you start using the new kernel), and to have console access so you can do the next steps in single-user mode. But in my experience, it doesn't matter.

Whether rebooting is even required is unclear. one guide suggests it is not required. The Makefile and the Handbook suggest it is required, but they disagree on how to do it.

I always go ahead and do it. This is the procedure, depending on whether you have console access:

If you do not have console access:

  • shutdown -r now – the Makefile says reboot, or boot -s in single-user mode, but these will not run the rc(8) shutdown scripts!
  • Wait for the system to come back up. It will put you back in multi-user mode like normal, but with kernel and world out of sync.
  • Log in as root, or as a regular user and then su to root.
  • set nohup - again, this is my own construction.

Alternatively, if you do have console access:

  • shutdown now (per the Handbook). This will drop the system to single-user mode, and you will probably be in sh instead of tcsh.
  • mount -u /
  • mount -a -t ufs
  • swapon -a
  • [ -f /etc/wall_cmos_clock ] && adjkerntz -i – the check for /etc/wall_cmos_clock is my own construction.

Run mergemaster

Warning: These instructions are only for mergemaster, but mergemaster is deprecated and will not work in FreeBSD 13.0 and up. The new tool is etcupdate and in works similarly (run with -p before installworld, then with -B after). But you also have to bootstrap it first, if you've already been using mergemaster.

/etc is not updated automatically in the install process. You have to do that by running mergemaster.

  • cp -Rp /etc /etc.old – you are just making a backup in case you screw something up.
  • mergemaster -p `stty -a | head -1 | tr ';' '\012' | grep columns | awk '{print "-w",$1}'` – Only the -p is required; the rest is my own addition to ensure sdiff uses the actual screen width rather than the default of 80 columns.

This is the first of two mergemaster runs. This time, it is only going to do the safest operations, and it usually requires no user input.

Install world

  • As mentioned previously, make sure /tmp has a lot of room.
  • cd /usr/src && script /var/tmp/installworld.out make installworld – my own construction, again. Takes about 17 minutes.

Run mergemaster again

  • mergemaster -iF `stty -a | head -1 | tr ';' '\012' | grep columns | awk '{print "-w",$1}'` – obviously my own construction. The -iF flags are only mentioned in the Handbook.

This is the second of the two mergemaster runs and usually takes me about 5 to 15 minutes, depending on what changed.

mergemaster does as much as it can on its own, but the replacement of some files requires your input. For each of those files, it gives you a diff (hopefully you are familiar with diff output) followed by a prompt for you to choose what to do. The diff compares the old version (with lines denoted by "-") and the new version (with the equivalent lines denoted by "+") which is not yet installed. Your choices are:

    • d = delete/ignore the temporary (new) file; if you want to manually merge it later.
    • i = install the temporary (new) file, overwriting the existing file. Do this for any files you've never edited. Also do it for /etc/mail/sendmail.cf and /etc/mail/submit.cf; just remember to rebuild those files later (cd /etc/mail && make all install restart).
    • m = merge, in which case it walks you through an edit of the new file by showing you equivalent chunks of both files, and you choose l (left/old) or r (right/new) as the one to keep; or you can concatenate and edit them in your $VISUAL editor; press ? for help.
    • v = view the diff again.

Delete obsolete files

This will delete files the system thinks is no longer needed. It should be safe.

  • make delete-old -DBATCH_DELETE_OLD_FILES – that other guide mentioned above says to do make check-old, which runs the same checks and then tells you to run make delete-old. Omit -DBATCH_DELETE_OLD_FILES if you want to be prompted to confirm the deletion of every file.

After it is done, it will mention that you can delete old libs, too. Don't do this yet!

Reboot again

  • shutdown -r now (if you're in multi-user mode) otherwise reboot...and pray that everything works.
  • Log in as root, or as a regular user then su to root.

If anything fails, of course you must stop and figure out what happened and how to recover; don't go on to the next step!

Make sure tmpfs wasn't re-created

It's possible /etc/fstab now has a new entry for a too-small tmpfs mounted at /tmp (30M). If so...:

  • edit /etc/fstab to fix it to your liking
  • umount tmpfs

Rebuild sendmail config

If you are using sendmail and have customized it in any way:

  • cd /etc/mail && make all install restart

Delete obsolete libraries

This may or may not be safe. It removes libraries which are not needed by the OS, but some of them may be still needed by ports and packages. But if you are going to be building all your ports/packages from scratch, it should be fine.

  • cd /usr/src && make delete-old-libs -DBATCH_DELETE_OLD_FILES

As before, omit -DBATCH_DELETE_OLD_FILES if you want to be prompted to confirm the deletion of every lib file it thinks is no longer needed. You have to enter "y" and press Enter for each one!

Optional: confirm new version

  • freebsd-version -k reports the version of the kernel that will be running after the next reboot.
  • freebsd-version -r reports the version of the currently running kernel.
  • freebsd-version reports the current userland version.
  • egrep "^(REVISION|BRANCH)" /usr/src/sys/conf/newvers.sh reports the userland version ready to be built.

You might think uname would tell you this stuff, but it is unreliable; see the freebsd-version man page.

Reinstall pkg

Fetch and install a new version of pkg:

  • pkg-static install -f pkg

And if you delete all your packages in the next step, you will need to install pkg yet again. You can do it the same way, or you can install it from a port, e.g. by installing portmaster.

Rebuild installed packages

Although it may not always be strictly necessary, after updating the OS, it's a good idea to rebuild all the installed third-party packages ("rebuild ports"). You could do portmaster -af, but that would actually be ~30% slower than removing everything first and rebuilding from scratch, as directed in the portmaster man page:

  1. portmaster --list-origins > /var/tmp/installed-port-list – saves a list of installed ports.
  2. portsnap fetch update – updates the ports collection so that you will install the latest versions of everything.
  3. portmaster -ty --clean-distfiles – removes outdated distfiles. Takes a long time
  4. portmaster -Faf – fetches latest distfiles for all installed ports; no asking for confirmation.
  5. pkg delete -afy – deletes all installed packages (including portmaster); no asking for confirmation.
  6. rm -rf /usr/local/lib/compat/pkg – deletes libraries used by old versions of ports.
  7. Back up any files in /usr/local you wish to save, such as configuration files in /usr/local/etc.
  8. Manually check /usr/local to make sure it only contains files not created by ports.
  9. cd /usr/ports/ports-mgmt/portmaster && make install – installs portmaster & pkg.

The next step is to rebuild all the ports you had before, except portmaster and pkg. The instructions say to do like this, but keep reading:

  1. sed -I '' -E '/^ports-mgmt\/(pkg|portmaster)$/d' /var/tmp/installed-port-list – removes portmaster & pkg from the list of previously installed ports.
  2. portmaster -D --no-confirm `cat /var/tmp/installed-port-list` – attempts to build and install the previously installed ports.
  3. rm /var/tmp/installed-port-list – deletes the list of previously installed ports.

The bad thing about this procedure is it assumes there won't be any problems building any of the new ports. Also it does not account for any changes in port names. And if you want to omit some ports this time around, you may have a hard time knowing which other ports in the list are dependencies which can also be omitted.

So my recommendation is to just use /var/tmp/installed-port-list as a guide, and to prioritize and only install ports one at a time, in this general order:

  1. ccache. You can expect maybe a 5%–6% hit rate, but that's still better than nothing.
  2. Ports needed ASAP, especially if they have few dependencies: ccache, procmail, rsync, nano, curl.
  3. Enormous ports which are dependencies for many others: perl5, openssl or libressl, python27, mysql56.
  4. Ports with gobs of dependencies and which aren't urgently needed: php56, nginx, mediawiki, spamassassin.
  5. Everything else: mutt, mtr-nox11, mrtg, sa-utils, tt-rss (+ xcache, php56-mcrypt)...

If you've upgraded to a RELEASE version of FreeBSD, you can use the standard packages collection. Here's what I did after upgrading to 12-RELEASE:

  • pkg install procmail – may not be available; build with portmaster instead
  • pkg install portmaster
  • pkg install nano
  • pkg install rsync
  • pkg install curl
  • pkg install -g perl5-5.28\* – check for the default version number in /usr/ports/UPDATING first
  • pkg install python27
  • pkg install libressl
  • pkg install mtr-nox11
  • pkg install php72 – check for the default version number in /usr/ports/UPDATING first. But this ended up being a waste of time because the packages for php72-* modules I needed were not up-to-date. I had to build everything myself:
    • pkg install gmake
    • pkg install autoconf
    • portmaster -i textproc/php72-xml www/php72-session textproc/php72-ctype textproc/php72-dom sysutils/php72-fileinfo security/php72-hash sysutils/php72-fileinfo sysutils/php72-posix net/php72-xmlrpc devel/php72-json databases/php72-pdo_mysql databases/php72-pdo databases/php72-mysqli converters/php72-mbstring devel/php72-pcntl ftp/php72-curl
  • pkg install mysql57-server – I was upgrading from MySQL 5.6 to 5.7, so there was more to do after this, of course; see next document.
  • portmaster mail/procmail – I'm using portmaster because procmail is currently not available as a package
  • pkg install mutt
  • pkg install sa-utils && /usr/local/etc/periodic/daily/sa-utils
  • portmaster -i www/nginx – must build from source in order to get HTTP_FANCYINDEX
  • portmaster --packages-if-newer -i www/tt-rss – must build from source in order to avoid installing GD, X11 libs

Accessing files on the eMMC

At boot time using U-Boot versions before 2016.07, when a valid MMC or SD card is detected in the external slot, FreeBSD assigns that drive to /dev/mmcsd0 (because the BBB always probes the external slot first). FreeBSD then assigns /dev/mmcsd1 to the built-in flash drive (the eMMC). If a valid card is not detected in the external slot, then the eMMC is /dev/mmcsd0 (apparently you already installed FreeBSD on the eMMC in this case). On newer versions of U-Boot, the eMMC is always /dev/mmcsd1 and the external card is always /dev/mmcsd0.

Assuming the eMMC is /dev/mmcsd1, its DOS/MBR boot sector is /dev/mmcsd1s1, and the Debian Linux partition is /dev/mmcsd1s2.

gpart list can be useful for figuring this out. It shows a provider named mmcsd1s1 with the type !14 (apparently that's DOS), and it shows a provider named mmcsd1s2 with type linux-data.

The command file -s /dev/mmcsd1s1 shows that the !14 provider is a DOS/MBR boot sector, and file -s /dev/mmcsd1s2 shows that the linux-data provider is an ext4 filesystem.

Boot partition

Accessing files on the boot partition is easy. Here's how to do it temporarily:

  • mkdir /emmc.root
  • mount -r -t msdosfs /dev/mmcsd1s1 /emmc.root
  • ls /emmc.root

The -r mandates read-only access. If you need read-write access, omit that flag, or run mount -u rw /emmc.root.

For permanent access:

  • mkdir /emmc.root
  • echo '/dev/mmcsd1s1\t/emmc.root\tmsdosfs\tro\t0\t0' >> /etc/fstab
  • mount /emmc.root

For write access, use rw instead ro.

Enable permanent booting from the SD card

Once FreeBSD is running smoothly, you'll want to make it so you don't have to physically hold the BeagleBone's boot switch button in order to boot directly into FreeBSD instead of Debian.

There are two ways to do it. One is to rename the eMMC 2nd-stage boot loader (MLO) so that the first stage (on-board) can't find it:

  • Follow the directions above, giving yourself read & write access to the eMMC's DOS/MBR sector.
  • Then rename the MLO file: cd /emmc.root && mv MLO MLO.old

Another option is to unset the active (bootable) flag:

  • gpart unset -a active -i 1 mmcsd1

However, there's a report that this method may not actually work.

Either way, it's a good time to test it:

  • shutdown -p now and then wait for BeagleBone's lights to go off.
  • Remove and reattach the power cable. It should boot right into FreeBSD!

Move FreeBSD to the eMMC

Yet another possibility is to put FreeBSD on the eMMC:

However, I have not yet attempted this myself, so I am hesitant to recommend it as an option.

Debian partition

The ext2 and ext3 file systems are natively supported by FreeBSD's ext2fs, but ext4 is not. You need to use the fuse driver and its ext4 add-on:

  • portmaster sysutils/fusefs-ext4fuse – also installs gmake and fusefs-libs, so it takes a while
  • rehash
  • mkdir /foo
  • kldload fuse — to make this permanent, you could add fuse_load=YES to /boot/loader.conf
  • ext4fuse /dev/mmcsd1s2 /foo
  • ls /foo

I found that the ext4fuse driver is a bit flaky. If I run du -sh /foo, it partially works, but some directories suddenly mirror the root directory and are not traversable. The effect can last sometimes for a little while, then correct itself.

Also, you will not be able to run Linux binaries unless you build a custom kernel with linux support, load the linux kernel module, and change a kernel config setting. There's a handbook section about this. Probably you also must install the emulators/linux_base-c6 port (CentOS userland, hopefully not too different from Debian).

More notes

This document continues in my notes for FreeBSD on BeagleBone Black – Additional software.

Retrieved from "https://offset.skew.org/index.php?title=User:Mjb/FreeBSD_on_BeagleBone_Black&oldid=4150"