Building My Own TrueNAS Core 11-bay+ ATX Based on ServerBuilds’ NAS Killer 4.1 Build
I recently built a server based on the NAS Killer 4.1 from (serverbuilds.net) by JDM_WAAAT. It has a Xeon 1270V2 CPU from 2012 which has 4 cores, 8 threads, and 32 GB of error-correcting code memory. It is a really affordable setup, as most parts are used and older enterprise-grade. I considered buying UnRAID, but decided on TrueNAS to ensure superior multi-disk performance with ZFS.
Why would you want to DIY your own TrueNAS? I looked at the offers of Synology, Qnap, Asustor, and found them lacking. I could either go with a 4-bay for $400 to $600 (without drives) or build my own for under $400 (also without drives) which can hold over eight 3.5" hard drives + three 2.5" + later convert the two 5.25" bays.
In addition, off-the-shelf NAS boxes usually feature low-power CPUs, whereas I picked up a used Xeon 1270V2 with lots of power for apps and advanced features of TrueNAS. Expansion? There are ways to add additional stacks of hard drives if you run out of space with DIY builds as well if you have the space in your workspace.
Another advantage for the specialized NAS system, at least TrueNAS, is that all your files are automatically compressed and take up a little less space on the disk array depending on the file type.
DIY vs Synology vs QNAP
How does these different options stack up against DIY?
TrueNAS DIY NAS Killer 4.1 | Synology DS920+ | QNAP TS-932PX-4G |
Xeon 4 Core / 8 Thread | Celeron Quad-Core | ARM 4 Quad-Core |
32GB DDR3 ECC (Mad Maxed) | 4GB DDR4 Non-ECC (Upgradeable to 8GB) | 4GB DDR4 Non-ECC (Upgradable to 16GB) |
8+ 3.5" HDDs + 3 2.5" SSD (Upgradeabe to 20+ HDDs) | 4x 3.5" HDDs (Upgradeabe to 9 HDDs) | 5x 3.5" HDDs + 4 2.5" SSD (Upgradeable to 17 HDDs) |
2x1Gbe | 2x1Gbe | 2x1Gbe |
+ 10Gbe for $30ish | + 10Gbe for N/A | Included |
Supports 2+ M.2 SSD | Supports two M.2 SSD | Not Supported |
Less than $400* | $549.99 | $599.99 |
These are literally the other two boxes that I thought I would get until I came across the updated NAS Killer 4.1 build, which is pretty killer! ^_^
The NK4.1 build price depends heavily on eBay market prices and / or whether you can offer the seller a lower price. I saved a little on the motherboard (-$5) and the 4x8GB sticks of the DDR3 ECC memory (-$5.99).
Another reason I was attracted to the 4.1 builds was the included LSI SAS2008, an onboard SAS controller. This not only saves $25ish for a SAS controller, but also space for additional cards such as a 10Gbe Ethernet card or an NVMe add-on card.
There are other less expansive instructions on the website and they have also released NAS Killer 4.2 with an all-in-one MB / CPU / RAM from eBay.
Parts List
- Case: Cooler Master N400 (used)
- CPU: Xeon E3-1270V2 (used)
- MB: S5512WGM2NR-B-CGN (used)
- SAS: LSI SAS2008 onboard (On-Board)
- (Flashed with IT-Mode)
- RAM: 4x8GB PC3L-12800E (used)
- CPU Cooler: Thermaltake Gravity i2 (NEW)
- Power Supply: Thermaltake TR2 500W (NEW)
- SATA Cables: 6 Pack SATA Cable (new)
- Fans: ARCTIC F12 PWM PST Value Pack (new)
- Optional: 10Gbe SPF+ Network Card (used)
I bought the Cooler Master N400 in like new condition from Amazon and it arrived as such.
Hard Drive Selection
I ended up picking up five 8TB easystore external hard drives from Best Buy because I learned that the warranty may be honored if you hang on to the external case and the little circuit board that integrates the drive with the plastic WD casing. They turned out to be 5400RPM drives which seems to totally fine and will hopefully help reduce the noise when file transfers are in progress.
UPDATE: WD easystore 8TB has dropped in price by $10 and now costs $129.99 / per drive. It is worth attempting to get the price matched through chat or return and buy again. I had a friendly CS rep that was able to refund $50 due to this price drop.
The OCZ 120GB SSD I had lying around and is doing just fine as a boot drive for TrueNAS.
- OCZ 120GB SSD
- 5X WD easystore 8TB Hard Drives
- Inside: WDC WD80EDAZ-11T (5400RPM)
Usable Storage Capacity
WDC WD80EDAZ-11T (5400RPM) | Parity |
WDC WD80EDAZ-11T (5400RPM) | Parity |
WDC WD80EDAZ-11T (5400RPM) | ZRAID2 |
WDC WD80EDAZ-11T (5400RPM) | ZRAID2 |
WDC WD80EDAZ-11T (5400RPM) | ZRAID2 |
Total raw storage capacity: | 40.00 TB |
Zpool storage capacity: | 36.25 TiB |
ZFS usable storage capacity: | 20.80 TiB |
I ended up with a total of 20.80 tebibytes of ZFS usable storage capacity.
CrystalDiskMark
I couldn't get disk # 5 to break 200MB/s after multiple CrystalDiskMark tests, but at these speeds, there is no worry about saturating Gigabit Ethernet.
Disk # | Read (MB/s) | Write (MB/s) |
Disk # 1 | 212.25 | 215.00 |
Disk # 2 | 210.82 | 211.39 |
Disk # 3 | 222.88 | 221.23 |
Disk # 4 | 202.83 | 203.96 |
Disk # 5 | 197.27 | 198.15 |
Hard Drive Temperature
Since I haven't changed out or added to the 120mm case fans, I was seeing disk temperatures up to 50C on one of the drives. While the other drives are running a bit cooler.
How to Check Hard Drive Temperature in TrueNAS
Go to Reporting Section:
- SELECT: DISK
- SELECT: DEVICES: Select the devices you want to check on.
- SELECT: METRICS: Disk Temperature
I placed a small circular fan in front of the server box, for now, to help keep all five 3.5" drives nice and cool until I order more silent case fans. I will probably replace the stock case fans that came with the Cooler Master N400 Computer Case as they are not the quietest or most efficient case fans.
[lasso box="B00NTUJTAK" title="ARCTIC F12 PWM PST Value Pack" description="INNOVATIVE DESIGN: Design of fan blades improves airflow and facilitates efficient ventilation, impeller was designed with a focus on minimizing the noise level yet delivering the desired airflow and pressure" link_id="3712" ref="amzn-arctic-f12-pwm-pst-5-pack-120-mm-pwm-pst-case-fan-with-pwm-sharing-technology-pst-value-pack-very-quiet-motor-computer-fan-speed-230-1350-rpm-black-white" id="25885"]
I found one fan that collided with the power connector of the LGA1155 motherboard, so I was forced to either screw it obliquely or install it somewhere else.
Later, as a replacement for the two supplied fans from the Cooler Master N400 case, I received a pack of 5 almost silent Arctic fans. At this point, when the system is not loaded, the system is about as noisy as our dishwasher with temperatures between 32-34C. Now I feel much more comfortable keeping this storage server online for a longer period of time. :)
Later I received a pack of 5 almost silent Arctic fans as a replacement for the two fans supplied. The system is about as loud as our dishwasher at this point when the system is doing nothing. Now the HD temperatures are all between 32-33C idle. Now I feel much more comfortable to keep this storage server online for longer periods of time. :)
File Transfer Performance
I saw between 45-65MB/s over the 5G WiFi which is acceptable and useable for backing up small batches of images. I checked the TrueNAS network reporting and confirmed that. Saw up to 594.71 mb/s (or 74.34MB/s) over the network interface.
Next, I tried a transfer from my desktop which was directly connected to the TrueNAS server. No switch was involved in this trial and this was to be expected with 5 drives in RAIDZ2 over 1Gbe ethernet. Transferring a lightroom catalog folder that contained just over 25GB of files (54,154 files) took 20 minutes with speed starting out at 100-110MB/s until the system ran out of ZFS Cache, which is contained in RAM/memory. It held pace around 22-25MB/s for the remainder of the file transfer. CPU pivoted between 5% to 10% for average usage and 40-48C for CPU temps but typically remained around 40-42C. I saw it spike to 48C but never stayed for more than a brief moment.
I tried again with a bunch of files in a 1.4GB zipped file and I saw 113MB/s consistently with GBe!
Mellanox 10G Gigabit SFP+
Down the road, I could add-on a 10Gbe Mellanox card (MNPA19-XTR) to both the server and my desktop for much faster ingesting of backups or incorporate some super-fast NVMe storage over the 10Gbe. I am curious to learn how fast file transfers could be with just the five 5400RPM drives! :)
Big shout out to ServerBuilds.net for all of the help and advice for getting this build up and running. If it wasn't for them I might have ended up with a proprietary setup.