Terra (Primary Compute)

• Hardware & OS
• ZFS Storage Architecture
• Container Architecture
• Performance & Stability Optimizations
• Luna (Backup Server)

Hardware & OS

Hardware & OS

Terra is the primary compute node for the Flora Family homelab, designed for high-concurrency tasks like AI processing, media transcoding, and ZFS storage management.

🖥️ Physical Specs

• Motherboard: Asus Prime X570-Pro
• CPU: AMD Ryzen 9 5950X (16 Cores, 32 Threads)
  • Scaling Driver: amd_pstate=active (EPP)
  • Frequency Range: 550MHz - 5000MHz
• RAM: 128GB (4x32GB) Corsair Vengeance LPX DDR4 3200MHz CL16 (Non-ECC)
  • Note: Currently running at 2133MHz; XMP/DOCP troubleshooting in progress.
• GPU: Nvidia Quadro P2200 (5GB VRAM)
• Chassis: iStar D-400 with Red Drive Doors (4U Rackmount)
• Power Supply: be quiet! Dark Power Pro 11 650W (80 Plus Platinum)
• **Expansion & Storage Connectivity:
  • LSI 9207-8i HBA: Driving the 4x 12TB HDD stack.
  • Motherboard SATA: Driving the 6x 2TB SSD stack.
  • 4-port Gigabit NIC: Realtek chipset; currently unused.

🌬️ Cooling (Noctua Standards)

• CPU Cooler: Noctua NH-C14S (with dual 140mm fans).
• NVMe Cooling: Both drives equipped with heatsinks including integrated active fans.
  • Note: Active cooling reduced temperatures significantly. Current stats (composite): drive 0400 @ 42C, 0100 @ 37C.
• Case Fans: All stock fans replaced with Noctua units.
• Additional Airflow: Added 40mm fan on the HBA and 2x 80mm fans at the chassis front.

Fan Connectivity Map

Sensor	Description	Connection
fan1	CHA_FAN1	back exhaust fans (2x80mm)
fan2	CPU_FAN	bottom CPU fan (1x120mm)
fan3	CHA_FAN2	SDD drive cage fan (1x40mm)
fan4	CHA_FAN3	front intake fans (2x80mm)
fan5	?	Shows 0 rpm
fan6	W_PUMP	HBA & NVMe fans (1x40mm + 2x20mm)
fan7	?	Shows 0 rpm
?	CPU_OPT	top CPU fan (1x120mm)
?	AIO_PUMP	empty
-	-	HDD drive cage fan (1x80mm)

💾 Storage Media

• NVMe: 2x 500GB WD Black SN7100 (Used for OS mirrors and ZFS Special VDEVs/L2ARC).
• SSD: 6x 2TB Samsung 870 EVO (Main pool).
• HDD: 4x 12TB Seagate Exos X16 (Bulk pool).

⚙️ OS & Software

• OS: Ubuntu 24.04 LTS
• Kernel: ZFS-on-Linux (OpenZFS)
• Management: Sanoid/Syncoid for snapshots and replication.

ZFS Storage Architecture

ZFS Storage Architecture

Terra utilizes ZFS for primary data storage, leveraging its advanced features for data integrity, snapshots, and caching.

🏊 Pools

🚀 main (SSD Pool)

• Hardware: Raid-Z2 (6x 2TB SSD)
• Total Capacity: ~12TB Raw
• Use Case: High-performance data, VM disks, Docker root, and active AppData.
• L2ARC: 48GB limit (leveraging NVMe partitions).

🐘 bulk (HDD Pool)

• Hardware: Raid-Z1 (4x 12TB HDD)
• Total Capacity: ~48TB Raw
• Use Case: Large media storage, backups, and long-term archival.
• Special VDEV: Mirrored NVMe partitions used for metadata offloading to speed up file listing on HDDs.

📂 Key Datasets & Mounts

• /srv: Docker-compose configuration files (Stored on main).
• /main/appdata: Primary persistent storage for containers (Stored on main).
• /bulk/pics: Family photo/video archive (Stored on bulk).

🛡️ Health & Maintenance

🔍 Automated Monitoring (smartd)

Drive health is monitored via smartmontools with automated short and long self-tests configured to email results. Test schedules are staggered to prevent resource contention.

• NVMe (2x 500GB): Short tests daily at 1am & 2am; Long tests quarterly on the 1st-2nd.
• HDD (4x 12TB): Short tests daily at 3am-6am; Long tests on the 4th, 6th, 8th, and 10th of each month.
• SSD (6x 2TB): Short tests daily from 1am-6am; Long tests quarterly on the 3rd, 5th, 7th, 9th, 11th, and 12th.

🧹 ZFS Scrubbing

A scrub is performed automatically on the 3rd Sunday of every month to ensure data integrity and prevent bitrot.

Scrub Schedule (Cron): 24 0 15-21 * * root if [ $(date +\%w) -eq 0 ] && [ -x /usr/lib/zfs-linux/scrub ]; then /usr/lib/zfs-linux/scrub; fi

Container Architecture

Container Architecture

The Flora Family homelab relies on a standardized Docker deployment pattern to ensure security, portability, and ease of management.

📂 Directory Layout

To maintain a clean separation between configuration and state, Terra uses two primary directories:

• /srv/[service-name]/: Contains docker-compose.yaml and .env files.
  • Ownership: root:root (Modified only via sudo).
  • Permissions: folders 755, configs 644, .env 600.
• /main/appdata/[service-name]/: Contains persistent application data.
  • Ownership: Mapped to the specific service account (UID/GID).
  • Permissions: 755 (folders) / 644 (files) generally.

🛠️ Deployment Standards

🔐 Non-Root Execution

Every service is configured to run as a non-privileged user, with the specific exceptions of Watchtower and Beszel Agent (which require root/host socket access to monitor system health and container status).

Non-root execution is achieved through one of three methods:

1. Standard user: flag: For images that support it (e.g., user: "51100:61100").
2. Environment Variables: Many linuxserver.io images use PUID/PGID variables.
3. Custom Dockerfile Builds: Used for "scratch" images or minimalist images (like OpenWebUI and SearXNG) to manually inject the desired UID/GID and install local CA certificates.

🔗 Shared Infrastructure

Common backend services are standardized to simplify inter-container networking and permissions:

• Postgres: Standardized on GID 70 across all instances (Immich, n8n, etc.).
• Redis/Valkey: Standardized on GID 970 for caching and session management.
• Shared Identities: The node-user (UID 1000) is shared between several JS-based apps (n8n, Uptime Kuma) where common file access is required.

🔄 Lifecycle Management

• Updates: Managed by Watchtower, configured to run every Sunday at 04:00 AM.
• Image Tagging:
  • Pinned: Databases and critical infrastructure (e.g., postgres:18) to prevent breaking updates.
  • Floating: Application layers (e.g., sonarr:latest) to receive automated security patches.

Performance & Stability Optimizations

Performance & Stability Optimizations

This page documents the specific kernel and hardware tweaks applied to Terra (Ryzen 5950X / Ubuntu 24.04) to optimize for container performance, storage throughput, and long-term stability.

🛡️ Stability & Power Management (The "Golden Config")

Typical Current Idle (UEFI)

• Setting: Advanced > AMD CBS > CPU Common > Power Supply Idle Control = "Typical Current Idle"
• Benefit: Prevents the Ryzen "Sleep of Death" where the PSU cuts power during deep C-state transitions. Essential for 24/7 server stability on X570 boards.

Processor C-State Limit (Kernel)

• Driver: processor.max_cstate=1
• Benefit: Software-side enforcement of the Idle fix. Limits the CPU to C1 (Halt/Idle) instead of deeper sleep states (C6), preventing interrupt processing delays and hardware hangs.
• Configuration: Added to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub.

65W Eco Mode (Manual PBO)

• Settings: PPT: 88, TDC: 60, EDC: 90
• Benefit: Drastically reduces thermals (peak temps dropped from ~90°C to 58°C) without significant impact on multi-core performance. Ideal for 4U rackmount chassis airflow.
• Configuration: UEFI > Precision Boost Overdrive > Manual.

⚡ CPU & Power Performance

AMD P-State (Active Mode)

• Driver: amd_pstate=active (EPP - Energy Performance Preference)
• Benefit: Allows the kernel to communicate directly with the Zen 3 power management controller. Enables a wider frequency range (550MHz - 5000MHz).

IRQ Balancing

• Service: irqbalance (Installed/Enabled)
• Benefit: Distributes hardware interrupts (HBA, NVMe, NIC) across all threads, preventing Core 0 bottlenecks.

🧠 Memory Tuning

Speed & Stability

• Configuration: Currently hard-coded to 2133MHz at AUTO voltage.
• Note: Troubleshooting in Feb 2026 confirmed that 3000MHz at 1.35V caused instantaneous reboots. Stability requires sticking to JEDEC defaults for now.

Static Hugepages

• Setting: hugepages=2048 (Reserving 4GB total as 2MB pages)
• Benefit: Reduces TLB misses for memory-heavy apps like Minecraft and Postgres.

💾 I/O & Network Tuning (sysctl)

ZFS Write Buffering

• Settings: vm.dirty_background_bytes = 268435456, vm.dirty_bytes = 1073741824
• Benefit: Prevents the kernel from buffering too much data in RAM before flushing to ZFS, eliminating system-wide UI/process stutter.

Fluent-bit Batching (ClickHouse)

• Setting: Flush 60 in fluent-bit.conf
• Benefit: Reduced ClickHouse CPU usage from 80% spikes to <5% by reducing transaction frequency and optimizing ZFS write contiguousness.

🎮 GPU Optimization (Quadro P2200)

Persistence Mode

• Command: sudo nvidia-smi -pm 1
• Benefit: Eliminates the 2-second initialization delay for AI inference and transcoding containers.

Luna (Backup Server)

Luna (Backup Server)

Luna is the secondary compute and backup node for the Flora Family homelab. It serves as the primary replication target for Terra.

🖥️ System Status

• Current Status: ONLINE (Recovered 2026-02-25)
• Role: ZFS Replication Target, Thermal Monitoring
• Observability: Fully integrated into the Vector/ClickHouse logging pipeline (Feb 2026).

💾 Storage Architecture (ZFS)

• Pool Configuration: Striped Raid-Z1 vdevs
• Capacity:
  • 4x 4TB Drives
  • 4x 3TB Drives

🌡️ Monitoring

• Integrated Script: /usr/local/bin/cpu_temp.sh (Runs via CRON every 5 minutes)
• Reporting: Logs to ClickHouse via Vector.
• Average Temps: ~46°C - 48°C (Idle/Light load)

⚙️ Software & Management

• Management: Syncoid target for Terra's datasets.
• Logging: Migrated from Fluent-Bit to Vector (Feb 25, 2026).
• Sanoid: Configured with --quiet to reduce log spam from snapshot cleanup.
• Timezone: Correctly set to America/New_York (EST). System clock and Vector timestamps are aligned.

🛠️ Recovery Notes (Feb 2026)

• Luna was brought back online after an outage.
• Vector configuration synchronized via Ansible with Terra.
• Verified successful syncoid backup completions from Terra at 07:44 AM EST (Feb 25).