Huskhoard

HuskHoard: Open-Source Data Tiering & Cold Storage Engine for Linux HuskHoard is a high-performance, automated data-tiering engine (HSM) built in Rust. It allows you to use high-speed local storage (NVMe/SSD) for active data while silently "hibernating" cold data to low-cost destinations like LTO tape libraries, S3-compatible cloud buckets, or secondary HDDs. Unlike traditional backup software that requires a manual "restore," HuskHoard uses Transparent Stubbing. Files remain visible in your local file system (ext4, XFS, Btrfs) at their full size. When an application tries to read a hibernated file, HuskHoard intercepts the request via the Linux fanotify API, recalls the data in the background, and serves it to the application seamlessly. Key Features: Zero-FUSE Architecture: By utilizing the native Linux fanotify kernel API, HuskHoard avoids the performance overhead and "mount" stability issues associated with FUSE-based tools like Rclone or S3FS. StreamGate Technology: Allows you to "seek" through massive files stored on tape or S3 instantly. You can watch 4K video or run scripts against a file stored on LTO tape with zero local disk impact. Enterprise-Grade Hardware Support: Native support for LTO-5 through LTO-9 tape drives (/dev/nstX). It manages hardware positioning and block alignment to extend drive life and maximize throughput. N-Way Replication: Automatically mirrors cold data across multiple destinations—for example, store one copy on a local HDD, one on LTO tape, and one in an S3 bucket simultaneously. No Vendor Lock-in: Data is stored in standard Zstd-compressed streams verified by BLAKE3 checksums. Metadata can be exported to Apache Parquet, ensuring your data is recoverable even without HuskHoard. High Performance: Written in Rust for memory safety and maximum speed, making it suitable for data-heavy environments like Media & Entertainment (M&E), scientific research, and homelabs. Why HuskHoard is a Powerful Alternative: To LucidLink: Provides a similar "cloud-as-local-disk" experience but is open-source, supports LTO tape for long-term archiving, and removes monthly subscription costs. To Tiger Bridge / Quantum StorNext: Brings enterprise-level hierarchical storage management (HSM) to standard Linux servers without the five-figure price tag or proprietary data formats. To Rclone: While Rclone is great for syncing, HuskHoard is a "live" engine that manages your local disk space automatically, offloading files based on age or policy so your SSD never fills up. Technical Specs: Language: Rust License: AGPL v3 OS: Linux (Kernel 5.1+) Backends: AWS S3, Wasabi, Backblaze B2, LTO Tape, POSIX paths.