Blag Ivanov & James Newman October 16, 2023

Month in Review - September 2023

Part of the Month in Review series

#hardware #media #monitoring #alerting #databases #networking #maintenance
Month in Review - September 2023

Welcome to a new monthly series covering what we released and have been working on for the last month. As well as providing and update while we continue to work on our content cadence, juggling full time jobs, and ongoing homelab projects.

Processing Physical Media

James put together an excellent piece on processing physical media all the way from extraction to transcoding and compression.

Unraid Homelab Monitoring

Blag has been on the journey of building out a monitoring system and figuring out where observability begins.

The original Prometheus and Grafana monitoring stack were installed two years ago. Since then, a steady stream of exporters has continuously been added alongside dashboards. There is no shortage of metrics at this point.

MariaDB Monitoring

Updating the Prometheus MySQL exporter version to 0.15.0 and getting the new connection configuration file working.

In a somewhat unplanned detour, Blag dove head first into MariaDB and WordPress configuration and performance tuning. This was to be expected given that he brought all of his sites in-house back in May

Alerting

Metrics only tell part of the story and periodically checking a dashboard, let alone twenty plus, is not scalable. Creating and fine-tuning alerts and notifications have taken a front seat over the last month. While by no means a done deal, the current alerts have already proven useful.

Learning the right way of building Grafana alerts through dashboard panels and instead of raw metric queries.

Improving Discord notifications through the use of templates.

Setting up the Grafana Image Renderer plugin and receiving mostly reliable image attachments.

New Internet Line and WAN Failover

The Homelabber received a new symmetrical 1 Gigabit Internet pipeline. This left us with two available Internet lines. The only responsible thing to do was lasso them into failover connection using our shiny new pfSense firewall.

A guide is on the way and is almost ready, Blag is currently testing the Dynamic DNS failover.

James’ First Failing Hard Drive

James experienced his first disk warning this month thanks to Scrutiny. While the hard drive continues to run, Scrutiny is reporting a failure for a Seagate disk that is a little over a year old. A new hard drive has been ordered and added to the array, adding some parity. Let’s see how long the failing disk now lasts in the array, and when Unraid will report the failing disk.

The Scrutiny Bandwagon

During our last call, James had mentioned configuring Pushover notifications for Scrutiny while on the topic of his failing disk.

As of writing this, Blag set up Discord notification on Scrutiny and lo and behold, another failing disk!


In this Series

Month in Review