Ubuntu 16.04 LTS servers and found pending security updates. Immediately my eyebrow was raised because I always configure apt to automatically install security updates.

Trouble finds me

Running apt-get to install the pending security updates manually soon revealed the root cause of my problems:

...
dpkg: error processing archive /var/cache/apt/archives/linux-image-4.4.0-141-generic_4.4.0-141.167_amd64.deb (--unpack):
 cannot copy extracted data for './boot/vmlinuz-4.4.0-141-generic' to '/boot/vmlinuz-4.4.0-141-generic.dpkg-new': failed to write (No space left on device)
...

No space left on device.

Sure enough the /boot partition was full and since one of the security updates was a new Linux kernel that needed to be unpacked to that location everything grinds to a halt.

~# df -h
Filesystem                         Size  Used Avail Use% Mounted on
udev                              1003M     0 1003M   0% /dev
tmpfs                              205M   22M  184M  11% /run
/dev/mapper/woodwidewiki--vg-root   47G  4.9G   39G  12% /
tmpfs                             1023M     0 1023M   0% /dev/shm
tmpfs                              5.0M     0  5.0M   0% /run/lock
tmpfs                             1023M     0 1023M   0% /sys/fs/cgroup
/dev/sda2                          473M  471M     0 100% /boot
/dev/sda1                          511M  4.5M  507M   1% /boot/efi
tmpfs                              205M     0  205M   0% /run/user/1000

Looking at /boot I saw it was full of old kernel images and the tiny 471M partition fills up quickly (Yes, one lesson here is I should probably make bigger /boot partitions! 😊). Thankfully it’s a simple matter of running apt-get autoremove --purge to have apt intelligently clean away old kernels that aren’t in use (while still leaving one previous version for fallback).

Being proactive

Running apt-get autoremove manually fixes the immediate problem but what will prevent this from happening again in a year when more old kernels pile up?

Thankfully apt provides a way to have the system automatically clean up old kernels proactively with the APT::Periodic::AutocleanInterval setting. You can read more about this and other apt settings in man apt.conf.

To have my system automatically clean away old kernels every 7 days was just a matter of editing /etc/apt/apt.conf.d/10periodic and changing:

APT::Periodic::AutocleanInterval "0";

to

APT::Periodic::AutocleanInterval "7";