Posts: 1,180
Threads: 1,181
Joined: Feb 2020
Reputation:
1
I record whale calls in the ocean using underwater recorders called hydrophones. Each one creates 1464 Gb of compressed audio files/year. These decompress using a host software from the hydrophone into 5.94 Tb of .wav files. I have three of these hydrophones, meaning I produce 5.94 Tb * 3 hydrophones = 17.82 Tb of .wav files/year. I need to store these but I also need to read them, run analysis on them, and have a proper backup. Right now I have 2 Ironwolf 3.5' drives, 20 Tb each, in the QNAP, and they are configured in RAID 1 using the windows storage manager on my laptop. I am able to decompress the compressed files at a rate of ~ 10 Mbs, and download the decompressed .wav files onto the QNAP at a rate of ~ 40 Mbs. The compressed files read off of Samsung EVO micro-SD cards which are plugged directly into a slot in my laptop, the laptop is connected to the QNAP with a 6-foot long USB-C 2.o cable rated for 10 Gbs. How can I speed up file transfer onto the QNAP? Thank You!
Posts: 4,313
Threads: 2
Joined: Jun 2022
Reputation:
28
It's impressive to hear about your work with hydrophones and the substantial amount of data you're dealing with.
Given the substantial amount of data you handle, transitioning to a Thunderbolt Direct-Attached Storage (DAS) solution like Pegasus or Areca with hardware RAID could significantly boost your transfer speeds. These Thunderbolt DAS systems offer faster data transfer rates compared to USB-based DAS solutions, making them ideal for handling large datasets efficiently.
However, it's essential to consider potential bottlenecks within your setup. While Thunderbolt DAS systems provide faster transfer speeds, the performance also depends on the speed of the drives themselves. Using SATA SSDs instead of HDDs can greatly enhance transfer speeds due to their faster read/write capabilities.
Moreover, the RAID configuration plays a crucial role in performance. In a 4-drive setup, RAID 10 typically offers faster performance compared to RAID 5, especially for write-intensive operations. It's worth noting that the TL D-800C is typically connected to a QNAP NAS and set up in RAID 5. If connected directly to a PC, it would require software RAID 5 or a similar setup to maintain data redundancy.
Additionally, if the TL D-800C is configured in JBOD mode, it won't benefit from the speed boost that comes from combining drive pools. Therefore, consider your RAID configuration carefully based on your specific performance requirements.
