Disk space utilization monitoring in Google StackDriver

Setting up disk space utilization alerts in Google Stackdriver is almost unbelievably tricky. It’s easy to think that you can just choose the metric “Disk Utilization” and add a condition to alert when disk utilization crosses some threshold for some length of time (such as over 55% for 1 hour).

File System Type Filter

Your first problem is that a Linux system has many types of file systems, even if it only has one disk. You probably want to alert when your root disk is running out of space; therefore, you need to filter out non-disk filesystems like procfs or tmpfs. Add a device filter to select exactly which devices you want to alert on; in my case, with only one disk per server, I match only on device rootfs.

Disk Metrics State Filter

Metric “Disk Utilization” is actually a collection of states, only one of which you want to alert on. You can choose from states: free, reserved, or used. If you don’t select, it defaults to “free,” so your condition needs to trigger if “free” falls below a threshold. Otherwise, you need to add another filter to set “state=used” and set a condition to trigger if “used” goes above the threshold.

Final Configuration

The screenshot below shows how to set up an alerting trigger when disk used goes above a threshold:

Stackdriver Disk Space Alert Configuration
Stackdriver Disk Space Alert Configuration

Leave a Comment

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.