Although I really wanted I haven’t been able to measure the impact of the various configurations on plotting performance. Answer is very simple, lack of spare HW. I only have a single i7-4770K desktop machine and I did not feel the need to buy additional HW, I just bought a bunch of disks. Whether it was a good idea or not it’s a different story but I don’t think it makes much sense now – so I stick to what I have.
- CPU: I7-4770K (4 cores / 8 threads)
- Memory: 4x8GB DDR3
- Temp drive: 2xGIGABYTE NVMe SSD 1TB m2 (connected via PCIe)
- Farmer drives: Bunch of 3,6 TB Toshiba disks (5400 RPM)
- Raspberry Pi 4 – 8 GB
- Disks connected via USB adapter (HDD docking stations)
The reason why I went with the Gigabyte NVMe is very simple that had the best TBW value (1600) where I live at a reasonably good price.
Below you can find a few things you need to consider checking/changing. I know it may sound contradictory that I haven’t been able to measure the impact of the config changes but here I am suggesting a few things, but in general my system was optimized to be able to run a few virtualized workloads so should be better than leaving things on the defaults. Some of these settings may be called differently or not exists at all in your case so you may need some digging.
Hardware related setting
Many times TBW or Percentage Used value is not reported correctly by the devices – I don’t know the reason but I am impacted too. One of my device has FW version 01 the other has FW version 02. 02 reports Percentage Used correctly but 01 does not (no FW upgrade so reached out to the vendor for support). If your check your drives in Hard Disk Sentinel or other S.M.A.R.T tools some might might be in Bad condition even though TBW is nowhere near to the factory settings. You should not be worried too much but worth getting a tool from the vendor to see what it reports.
Memory speed – Maximum or use XMP profile
Node interleaving – Disabled
Turbo Mode – Enabled
C-states – Disabled
P-states – Disabled
Hyper Threading – Enabled
CPU cores – All
You can leave Virtualization (Vt-d) turned on
Power settings (BIOS)
Set power to maximum performance
Firmware in general
Try to keep the FW up to date, especially check it if you don’t get adequate performance. My advice before upgrading the FW check the release notes.
Additionally if you don’t have enough SATA or M.2 ports on your motherboard and you need to buy a PCIe card to connect the disks please keep it in mind that not every PCIe slots are equal, make sure you select the fastest one (X16). Some cases if you have one X16 and another X8 you install two cards then both may be running at X8 speed.
People might have problems due to auto updates (unplanned reboots), or due to power policy settings. Make sure Windows won’t spin down your disk, your system wont go to sleep.
If you use external docking station for your disk it may actually turn the device to sleep by default and there is no way to turn it off. For this reason you may need to create some scripts to generate I/O load..
Put things into
# script will write to the X:\keepalive.io every 9 minutes
get-date | Out-File -FilePath X:\keepalive.io
Start-Sleep -Seconds 540
Then run the script from CMD
powershell.exe -windowstyle hidden -file $location_of_your_keepalive.ps1 script
Put things into
echo $DATE > /mnt/hdd/keepalive.io
Then add it to crontab:
*/9 * * * * /usr/bin/keepalive.sh
There are tons of different ways to achieve the above (keeping your disks alive) this is just two very simple solution.
This requires a lot of testing and sizing and there are other sites explaining this in great detail. In my case I got better performance (10-15%) faster plotting by increasing the number of Threads / plot to 4. I can run 2 plots in parallel per NVMe and stagger the second batch usually by 1,5-2 hours. So in total I get around 8-9 plots / day. Not much I know :).