Years of Data in Gigabytes, Not Terabytes

Traditional systems collect high-resolution data, then batch-process it hours later to create downsampled tiers - requiring maintenance windows and creating sync issues. Netdata writes all retention tiers (per-second, per-minute, per-hour) simultaneously during collection in a single operation, eliminating batch jobs entirely and ensuring perfect tier consistency.

Netdata achieves 0.6 bytes per sample through Gorilla compression combined with ZSTD, fixed-step page design that eliminates per-point timestamp storage, and extent-based compression that packs related metrics together. This delivers 2-20× better efficiency than traditional TSDBs, enabling years of per-second retention in gigabytes rather than terabytes.

No - higher tiers use aggressive aggregation ratios (60:1 for per-minute, 3600:1 for per-hour) and the same compression, making them minimal overhead. Total storage for 2 years across all three tiers is significantly less than storing 2 years in traditional hot storage, despite maintaining all tiers simultaneously.

Traditional time-series databases use read-modify-write cycles requiring periodic compaction to maintain performance. Netdata uses an append-only (WORM) architecture where data is never modified after writing - only appended. This eliminates compaction entirely. When retention limits are reached, the oldest datafile is simply deleted (atomic operation) and a new one created - no complex merge operations or downtime required.

Yes - Netdata supports flexible time-based AND/OR size-based retention per tier. Configure 7 years for SOX, 6 years for HIPAA, or 1 year for PCI-DSS. You can set time limits (e.g., retention time = 7y), size limits (e.g., retention size = 50GiB), or combine both strategies where data is deleted when either limit is reached first.

Query performance remains consistent across all tiers - sub-second response whether accessing yesterday’s per-second data or 2-year-old hourly aggregates. Netdata’s query engine automatically selects the optimal tier based on time range, and all tiers use the same DBENGINE interface with no rehydration delays or separate archive query tools.

Each Netdata Agent and Parent maintains its own three-tier storage independently. Agents can run in RAM-only mode (offloading storage to Parents), or maintain full retention locally. Parents aggregate data from multiple Agents while maintaining the same efficient three-tier structure - enabling linear scalability from 1 to 100,000+ nodes with identical architecture.

Netdata automatically detects gaps in higher tiers after restart and backfills them from lower tiers without manual intervention. The system queries Tier 0 for missing periods, rebuilds Tier 1 and Tier 2 aggregations with accurate sum/min/max/count values, and sets a backfilled flag - ensuring perfect tier consistency after any outage.

Yes - while Netdata’s Tier 2 provides cost-effective 2-7 year retention, you can export data to S3, GCS, or Azure Blob for regulatory archives beyond operational needs. Netdata maintains the operational data you query frequently, while external object storage handles compliance archives you rarely access - providing the best of both worlds.

Prometheus requires manual recording rules for every metric you want to retain long-term, plus complex Thanos/Cortex setup for multi-tier storage. Netdata provides automatic aggregation across all tiers with zero configuration. Netdata benchmark testing shows 15× longer retention in the same disk space, with significantly reduced operational overhead compared to Prometheus at scale - all in a single binary.