Facebook’s Tectonic Filesystem: Efficiency from Exascale

Facebook's Tectonic is a distributed filesystem designed to operate at the exabyte scale, consolidating storage needs that were previously handled by specialized systems like Haystack (for blobs), f4 (for warm storage), and HDFS (for data warehousing). By moving to a single, general-purpose multitenant filesystem, Facebook aimed to reduce operational complexity and eliminate resource fragmentation, where some clusters were bottlenecked on IO while others had spare capacity but lacked storage space.

The system achieves its massive scalability through a disaggregated metadata architecture. Unlike traditional approaches that might use range partitioning, Tectonic uses hash partitioning for its metadata layers to prevent hotspots. This metadata layer scales independently from the chunk storage layer, allowing the system to handle billions of files and exabytes of data within a single cluster that can span an entire datacenter.

To support diverse workloads on shared infrastructure, Tectonic employs tenant-specific optimizations driven by a client-side microservice architecture. This allows different applications to dictate how they interact with storage—for example, the data warehouse can use Reed-Solomon encoding for efficiency, while latency-sensitive blob storage use replicated quorum appends. Performance isolation is managed through "traffic groups," which group applications with similar patterns, simplifying resource management compared to per-application quotas.

Key Concepts

• Disaggregated Metadata: Metadata is separated from data storage and split into independently scalable layers, using hash partitioning to avoid hotspots.
• Tenant-Specific Optimizations: Clients control storage interactions, allowing different durability and performance strategies (e.g., replication vs. erasure coding) for different workloads.
• Traffic Groups: A mechanism for grouping applications with similar traffic patterns to simplify performance isolation and resource sharing in a multitenant environment.
• Exabyte Scale: The ability to scale a single filesystem instance to span an entire datacenter, hosting multiple large tenants.