So after reading LifeZero’s SUPER FAST 4TB RAID UNDER $1500 a few weeks ago, I wanted one.
My personal machine is a soon to be replaced PCI-X G5, but fortunately for my curiosity, the production company i work with was in a desperate storage crunch.
We have a 4TB SAN, (full.), and we have a growing documentary (over 2.5TB) taking up most of the room.
Adding more storage to the SAN right now means a new fibre switch, as well as upgrading to xSAN 2.0, and of course the new volume, a technicians time. So basically upgrading everything would cost us way more than we can spend right now.
So the band-aid for now is the Fast 7TB Raid 5 for under $2500 I built based off of LifeZero’s model.
The downside. It isn’t shard storage. It’s a bit of a nuisance, but we’re making work. We can access the drive over the network, but the machine with this Raid is still connected to the SAN, so some files have to make a couple stops to end up on the Raid, but no biggie.
Parts I Used:
ProAvio Editbox 8MS – $559.35 +ship/tax at Timeline Digital Inc.
High Point Technology RocketRaid 3522 – $499 (for my convenience, might get a better deal if i’d shopped around, but first looks maybe not) also at Timeline Digital Inc.
2x MiniSaS to MiniSaS cables (Included with the Editbox 8MS from Timeline)
8x 1TB Hard Drives – ~$100-130 each from Newegg
Total: ~$1860 +tax/shipping and depending on which hard drives you get.
The Case
John Flowers over at LifeZero used the Editbox 8ML, and i originally intended to duplicate his setup.. but apparently the Infiniband sporting 8ML is no longer made.. replaced by the 8MS. You can still find the 8ML, but I could only find it in black.. and I wanted Silver! (i’m kind of serious, mostly joking.)
The other things is that I found the new, supposedly faster 8MS for under $550 at the time, including cables from Timeline Digital Inc. So i’m all for that.
Raid Card
For convenience and speed of getting all the parts, I ordered the Raid Card from them as well. (Provantage had it listed today $7 cheaper.. but at the time, we needed things fast.. and the more vendors, the more chance you have to wait) After hearing the discussion on the system John Flower’s built, and looking into things a bit myself, I decided on the HighPoint Tech RocketRaid 3522, a full hardware Raid Card. 2 External Mini-SAS ports. At the time, it was cheaper than the Caldigit, who I believe have dropped the price on their card now, but I’ve also been told, but have not confirmed that the Caldigit reverses the pin order on their card so that it only works with their products.. I don’t know if that’s true… maybe it just means you have to buy their cables as well.. But like I said, I haven’t confirmed that.. so take it with a grain of salt.
Hard Drives
Walter over at Timeline Digital also convinced me to go with Hitachi. His reasoning being, that while they’re slightly slower than the 1TB Seagates, he’s had more than a few Seagates get returned.. where as no problems with the Hitachis.. So i took his advice and went with the Hitachi.
The Hitachi’s ended up being a little more expensive, but this is an investment, and I’ve had a few Seagates die on me in the past. I’ve also had WD, Maxtor, Lacie.. anyways.. Hard Drives are a crap shoot, but all good so far.
Installing the card
I put the RR3522 into the 2nd slot, (16x on the newer mac pros) there’s no slot utility thing as they lane speeds are locked in the newer towers… I believe the card is only a 4 or 8x card.. so it should live totally happy where it’s at.
I kind of want to swap it around and see if there are any performance boosts in different slots.. but it’s working well.. so no big incentive to do that.
Building the Raid
Screw the drive into the tray, slide it in, turn the key. repeat.
This thing came with screws with every tray, as well as the case came with enough on it’s own to install all the drives.. so i have about 40 left over screws (if not more). Nothing wrong with that.
Installing Raid Drivers
Put in the CD, navigate to the proper driver, install and reboot.
success!
I believe this is the point where I actually plugged in the MiniSAS cables from the Raid to the Mac Pro..
After booting, navigate to http://(your ip address):7402
The Raid is handled by a pretty simple web interface. Click on the url to access your raid.
Login is RAID password is hpt
You can change this once inside the web app.
The manual here is your friend. there are essentially step by step instructions.
You have to initialize all the drives before forming your Raid.
After this you create the Raid Array.. I named mine BEAST.
You’ll notice it says Initializing foreground. I ended up changing this to background so i could begin transferring files earlier.
It took about two days to initialize the background raid.. it was 8 1TB drives.. so i wasn’t too surprised.
Clicking on SHI up top shows you the health of the drives and raid.. pretty cool.
At this point the Raid should show up as a single volume in the OSX Disk Utility. This is where the benefit of true hardware raid cards come in… all that Raid management is done on the card through that interface. So it doesn’t drain any system resources. Is nice.
Initial Speed Test with newly formed empty 7TB Raid 5
pretty freaking sweet.
another test shortly after reveals the downside of large drives.. it seems since smaller drives can fit more spindles, they’ll have faster read speeds.. this doesn’t bother me too much.. as this thing is still faster than i need. it’s great.
i made a little icon if anyone wants it.
Updated Speedtest…
With 5TB of Data on the drives, that’s some pretty awesome speed…
Totally works for what we need. and works very well. Easily handles all the RED footage we could throw at it.
The only thing i can’t figure out is how to get the email notification to work.. no luck so far.. but if you have any insight let me know..
29 Comments
Excellent writeup. I read John Flowers’ post about his as well. Actually found this through his Twitter. I am so jealous!
BTW I use that same Extreme Drive icon on my backup drive.
Thanks for your advice..I plan to make one for me but i want to put a rackmount case , which case do you propose ? ?
Thanks in advance
@Nasos Karamalegkos
I know ProAvio makes some MiniSAS Rackmountable options, and I think they have a new one coming out soon.. but I don’t have any experience with them. They also aren’t sold without drives afaik.
Beyond that I really don’t know… The rackmount systems I usually see are usually fibre channel based..
@Bradon thomas
I have seen the the proavio s8ms which is rackmountable…Do you think that i ll have problems if i make the same set up with you with this case ? ?
Seems like that one should work just fine. Let me know how it goes!
Nice write up! I really wish I had gone with a proper RAID like this rather than doing a ghetto internal RAID.
NICE!!!
I bought one from http://www.timelinedigitalinc.com
They suggest to get the Areca 1221x in stead since the 3522 card is not compatible with 10.5.6 at this point.
I also bought their RAIDStream FireWire RAID 2TB as my back up drive
http://www.timelinedigitalinc.com/raidstream-firewire-raid-c-118_119.html
thanks so much for this review, I am going to build one of these myself, need some guidance, what do you think if I were to use
a similar box but with only 4 slots like this one:
http://www.timelinedigitalinc.com/editbox-eb4ms-4disk-minisas-disk-array-digital-media-p-307.html
nice and cheap, and then add 4 1TB hitachi drives for a raid 0
what do you think? do you think it would perform as good or better than yours?
do you have any regrets on the raid card you bought? recommend a new one?
thanks man
thanks Lucas! can’t wait to see PI!
I don’t have any regrets.. it’s been a few months and everything is going swell. I’ve heard there’s a new version of the HPT RR3522, so that might be worth looking into, but it’s probably more expensive. I’ve heard good things about the CalDigit card and the Areca 1221x as well.
If you call Timeline Digital, they’ll help you with any questions as well, they’re super knowledgeable.
With 4 drives in Raid 0, it’ll be fast, I don’t know exactly how fast, but if you can spend the extra money, I’d personally recommend doing the 8 drive box in Raid 5.. It’s kind of the best of both worlds with speed and a little bit of redundancy.
if you aren’t going to be doing uncompressed HD, then you probably could get away with Raid 5 on the 4 drive box..
hth
b
well thanks for the schooling but I have been searching and reading that none of those cards that you mentioned are compadible with MAC os 10.5.6
do you know of any raid cards of similar performance that are compatible?
One of the previous commenters mentioned that the Areca 1221x was compatible with 10.5.6. I haven’t had any issues with the RR3522, but I’m pretty certain that machine is still 10.5.5..
Give Timeline a call/email, they’ll have a solution.
Hay again
After the initializing in the disk utility i see only a volume (5.0TD HPT …..) do you use the disk utility to erase the volume and name it Beast to appear under the volume ? ? ?
Just built a similar system with new RE4-GP 2Tb Western Digital drives. At first I was getting really erratic speeds on my AJA system test. After putting a few hundred gigs it seems we’ve smoothed out at about 500 MB/s read and write. I also have the RR 4322, which is capable of 1000 read/write.
I have my RR in slot 3. Do you guys think that might be a problem?
Having the RR in slot 3 shouldn’t be a problem, it’d be interesting to test switching slots unless you’ve got something filling the spot already.. but if you’re not having problems and getting speeds like that, I think you’re in good shape.
I think slot 3 is the right slot – thanks for the feedback. Here’s a fun little tidbit though, these Raid edition RE4-GP drives are most likely 5400, and have this ‘intellipower’ feature that, while cooler and uses less power, is actually kind of sluggish. No single speed test is ever even close to the same, which is giving me much cause for concern. Obviously in an HD environment, consistant 7200 rpm drives are key. I think I’m going to have to send these drives back and wait for WD to release 2Tb “Caviar Black” drives (which have been leaked online as coming out soon). Thanks for the article though, very useful!
Ended up having to return the 2TB RE4-GP drives. I got ’slot disks’ warnings in FCP and had difficulty laying back prores to HDCam. Because the drives constantly spin down when not in use, you get about a second lag when editing as well. That’s about a million years in TV time, so, I RMA’d. I’m now running 1Tb WD Caviar Blacks and switched to slot 4. I’m coonsistantly getting 630 MB/s write and 380 MB/s read speeds. It seems that slot 4 buys you an additional 100 MB/s write. Just thought I’d share…
oops, that should say ’slow disks’.
very interesting, and very good to know.
i was a little weary of the energy saving drives for that reason.. so very good to know..
First, my compliments. This is just a great article, and really shows the best potential of the internet, as well as the indie spirit. What a great resource!
Now a question:
Did anyone out there build one of these with just 4 drives for an older g5 with PCI-x slot? And if so, what kind of speed are you getting out of a Raid 0? I would like to build the 8 drive version, but I haven’t been able to find an 8port pci-x card, so I may be forced to go with a four. Also, is there any benefit from using a 10,000 rpm drive over 7200 if using an older pci-x card? Any help would be appreciated.
here’s an LSI PCI-X card that can handle 8 drives:
http://www.lsi.com/storage_home/products_home/host_bus_adapters/sas_hbas/lsisas3801x/
i’m sure 10,000 rpm drives would add some performance.. but may not warrant the cost increase and the storage space decrease.
Brandon-
Thanks for the link. I am using a dual 2.5 G5 Mac though, and this card doesn’t appear to support Mac OS. Have you heard differently?
i don’t have any experience with that specific card. I know it’s getting harder and harder to find peripherals for PCI-X machines.
looks like the Areca ARC-1120ML can do what you need.
Thanks Brandon!Do you think if I went with a 4 enclosure for now, and used a port multiplier at a later date I would get the same performance and speed as an all in one 8 array? What kind of speed can one expect in a 4 raid 0 array? Enough for uncompressed HD?
i doubt it. a lot of the speed stems from writing to a higher number of drives. i don’t have benchmarks for 4 drives in raid 0. i’d expect it to be fast, but i don’t think it’ll be fast enough for uncompressed HD, but i don’t know for sure.
i’d recommend going with an 8 drive enclosure now, the process of adding the additional drives to the existing 4 would cause you to lose all data on the existing drives and reformat them into a single new array if that’s what you were planning on doing…
might as well do it from the start.
Brandon,
I took your advice and went with an 8 enclosure. Was wondering if you’ve had any trouble with your Hitachi drives? Since I’m using a slower PCI-X slot, I’d like to go with a RAID 0 for the speed boost, and just back up my project files until I get enough cash for a drobo backup.
Awesome! you won’t regret it.
i haven’t had any issues with the Hitachi drives.
Then I will go with the Hitachi’s. Which model did you go with?
i don’t remember the exact model, but here’s the a hitachi they’re selling one newegg.com that should work fine.
http://www.newegg.com/Product/Product.aspx?Item=N82E16822145233&Tpk=1TB%20SATA
Post a Comment