Getting started with drive health monitoring for your fleet

Start with the drives that are actually at risk

Most teams reach for SMART monitoring after something breaks. That is the wrong trigger. If you are setting up drive health monitoring for the first time, the goal is not to confirm that healthy drives are healthy. The goal is to establish a telemetry baseline before failures start compressing your options.

This guide covers the practical decisions you make when deploying fleet-wide drive health monitoring: what to instrument first, which attributes to watch in the first week, and how to read a risk report without drowning in noise.

Step 1: Decide what to instrument first

Not every drive in your fleet deserves the same monitoring priority on day one. Start with drives that are under active production write workload and are older than 18 months. These are the drives most likely to show wear-based signals worth tracking.

Archive tiers, cold backup drives, and rarely-written volumes can be added later. Their failure signatures are different because their wear patterns are different, and mixing them into your initial dataset just adds noise before you have a baseline established.

A practical starting point for a 200-drive mixed fleet: focus the first week on the top-write-volume 60 to 80 drives. Get a clean baseline on those before expanding. You will have more confidence in what "normal" looks like when the scope is bounded.

Step 2: Install the agent and wait 48 hours

The Crest agent runs as a systemd service or Docker container on each storage host. For standard SMART-enabled SATA drives, no additional tooling is required. For NVMe drives, the agent uses the NVMe admin command interface, which requires nvme-cli to be available on the host (or the equivalent kernel interface on your distro).

A typical install on a single storage host looks like this:

curl -sSL https://agent.gocrest.org/install.sh | sudo bash
sudo systemctl enable --now crest-agent
crest status

After install, you will see an output similar to this within 60 seconds:

DRIVE FLEET STATUS  (agent v1.4.2 | host: storage-node-03)
---------------------------------------------------------
Drive ID     Model               Health   Days to Risk  Status
/dev/sda     Seagate ST4000NM    98       --            Healthy
/dev/sdb     Seagate ST4000NM    97       --            Healthy
/dev/sdc     WD Gold WD4003FZEX  94       --            Healthy
/dev/nvme0   Samsung PM983       99       --            Healthy
---------------------------------------------------------
Last polled: 2026-05-30 09:12:44 UTC

The important discipline here: do not act on the first 48 hours of data. The model needs a baseline window before it can distinguish normal variance from a degradation trend. Treat the first two days as observation only.

Step 3: Understand what the health score actually measures

The health score shown in the dashboard is not a simple threshold check against SMART attribute values. It is a composite score derived from the trajectory of several attributes over time, normalized against the expected wear curve for that specific drive model family.

This distinction matters in practice. A drive with reallocated sector count (SMART attribute 5) at a value of 12 does not automatically get a low health score. If that drive has been at 12 for eight weeks with no upward movement, that may be within its normal operating range. A different drive that moved from 0 to 4 over three days will score worse, because the rate of change is the signal.

The attributes Crest weights most heavily vary by drive type. For SATA HDDs, the primary contributors are attributes 5 (reallocated sectors), 187 (reported uncorrectable errors), and 197 (current pending sector count). For NVMe drives, the model uses available spare percentage, media error count, and the critical warning byte from the NVMe health log.

Step 4: Reading your first risk report

After 72 hours, sort the fleet view by risk score descending. Any drive with a risk score above 0.7 warrants a closer look. Any drive with a predicted failure window under 14 days should be scheduled for replacement in the next maintenance window.

In the first week, you will likely find a few drives that have been degrading slowly for months without anyone noticing. This is normal. Attribute 5 or 187 being non-zero is more common than you might expect in a mature fleet. The question is whether those values are stable or trending upward.

What you should not do: schedule replacements for every drive that is not at score 100. That is not what the scores mean. A drive at 88 with stable attributes is not a replacement candidate. A drive at 76 with a rising pending sector count over two weeks is.

Common mistakes in the first month

The most frequent problem we see with new deployments is treating individual SMART attribute thresholds as binary pass/fail gates. Attribute 5 reaching a value of 1 does not mean the drive is about to fail. Many drives operate for years with low but stable reallocated sector counts. The threshold-based mental model, inherited from years of SMART tools that report only "PASSED" or "FAILED", produces too many false positives and trains teams to ignore storage alerts.

The second common mistake is comparing health scores across drive models without accounting for the different wear curves. A score of 85 on an enterprise Seagate Exos drive does not mean the same thing as a score of 85 on a consumer-grade Western Digital Blue. The model normalizes per drive model family for exactly this reason, but teams that build their own spreadsheet summaries often lose this context.

Third: not correlating health scores with drive age and total data written. A drive at score 80 that has been running for 4 years and accumulated 200 TB of writes is a different risk profile than a 6-month-old drive at score 80. The prediction window accounts for this, but the raw score alone does not tell the full story.

What a healthy monitoring cadence looks like

Once you have a baseline established, the operational cadence becomes straightforward. Check the fleet dashboard once during your weekly infrastructure review. Anything in "At Risk" status with a window under 14 days goes into the replacement queue for the next maintenance window. Anything in "Watch" status gets noted and re-checked the following week.

Monthly, cross-reference the at-risk list with your drive inventory to see if any replacement drives need to be pre-ordered. For a fleet where you know the expected annual replacement rate, keeping a small stock of common drive models on the shelf removes the lead time variable from the reactive path entirely.

We are not saying SMART data is useless. The standard SMART attributes are a valuable first-pass signal, and drives that fail the SMART overall health check should be replaced immediately. The gap is the class of failures that degrade over weeks without triggering the binary SMART pass/fail threshold. That is what the trend-based model is built to catch.

Next steps: connecting the alert stack

Once you are comfortable reading the fleet dashboard, the next logical step is wiring alerts into your existing incident workflow. Crest connects to PagerDuty, Slack, and Prometheus out of the box. For most SRE teams, the PagerDuty integration is the highest-value connection: it means drive health risk surfaces through the same channel as everything else in your runbook, without requiring a new dashboard to monitor.

Start with a low-severity route for "Watch" status drives and a higher-severity route for drives with a predicted window under 7 days. That mapping covers the majority of actionable scenarios without generating alert noise on drives that are being monitored but do not need immediate attention.