As the title suggests i’m looking for the best way to backup files, do you use the net or an external drive or something else? Thanks
100/1000 LAN with a RAID1 or RAID5 NAS or Server, or RAID5 in your machine.
If you don´t need fast and anytime access to your data, Icy Dock via eSATA or USB3.
Put HDD in, back up, take HDD out, store in climated room.
Thanks, are RAIDS on my machine safe to use, from what i understand everything i do on one drive is cloned to another, what happens if files become corrupt or you pick up a virus both disks would be hosed, or am i not understanding the process?
I use spideroak, which is an online backup service. Fairly handy as you can get to it from multiple machines and it’s all automatic. The first 2gbs are free. Also the work is offsite, which can be handy, encrypted and it doesnt’ actually delete any files until you tell it to.
Or you can take a wifi external hard drive and use a sync software to sync your files.
The whole topic also depends on what you got to backup
In the studio we record life footage directly to harddrives, uncompressed renderframes from CG, Avid projects…
It´s nothing where you can use “online backup”, talking of hundrets of gigs here
As for raid:
The last virus I had was the formboot virus on my 486dx-2/33. If you use your machine with a certain awareness you don´t get viruses. If you get a virus and copy it to your backup you´r beyond hope but you can run one AV on your system, one on the backup system
Depending on the sensitivity of your data it might be a good idea though to setup a small homeserver because:
- only LAN access to the harddrives, WAN only for server updates, linux server, strict iptable policy.
- you don´t need a RAID card in your machine or rely on “crappy” onboard RAID
- a seperate system only with backup data and maybe only FTP access for autobackup makes it almost impossible for a virus to spread to the backups.
- if you need access to your data remotely you still can setup FTP access to the WAN to be able to login from anywhere, you can also run remote sessions or login via SSH for instance. If the server has enough ooomph you can make a .blend on your netbook, upload it to the server and render it ther ein the commandline via SSH, or simply work directly on the server via a remotesession.
With a seperate server you also have the advantage that you could use for instance Ubuntu Server which is easy to use and use software raid which is quite good. Another advantage is EXT3/4 which is a bit more robust than NTFS. A disadvantage is that the CPU has to do the RAID stuff. With a RAID controller, the controller does most of the work.
The best option is RAID5. It´s striping with a parity disk. It needs a minimum of 3 identical disks. 2 disks are in striping, meaning if you have databits 01010101, the raid stores them “simultaniously” works good if you got SATA2+ with AHCI activated so it supports NCQ.
Then disk1 would store the 0 while disk2 would store the 1 at the “same time” it´s quite fast.
The 3rd disk is the parity disk. It works with logical operations… Simplified said it stores the logical result of the other two disks.
Lets say on disk1 we got a 0 and on disk2 we got a 0.
The parity disk stores the logic: 0 AND 0 = 1 (1&0 = 0, 0&1 = 0, 0&0 = 1, 1&1 = 1)
So if disk2 dies, you got left 0 AND ? = 1
So you put a new disk in the array and restore the data solving the logic.
With RAID1 you got simple mirroring. You need 2 disks, data is mirrored automatically. One dies, replace it, re-initialize mirror. The end.
You got all the options.
If you got time and not too much data, a simply external USB drive does it you can store in your shelf. Not running disks aren´t really prone to fail
If it needs to be faster, Icy dock with eSATA/USB3
If you want it all, a server with RAID.
Cheap, Reliable, Fast. Choose any two
I’ve just popped out and got two external drives, seemed like the best solution for me, they are usb 3 compatible so when i get to upgrade my computer i should be good to go.
One is formatted to NTFC and the other to EXT 4.
They come with some auto backup software for windows but i i’ll just do things manually as i never really trust anything that says auto.
Thanks again arexma, the data’s not that sensitive just personal junk, blend files, textures, alphas etc, but i really enjoyed your knowledge, thanks for sharing.
If you’re using Linux, rsnapshot is really quite nice. I use rsnapshot for a local backup of my office machines and I rsync that data to an offsite computer for a secondary backup.
Thanks Fweeb, will check it out.
arexma definitely has the more comprehensive solution for a studio backup solution, especially with HUGE files.
Personally I use Dropbox.com and have done so for 3 years now. I can’t even begin to express how useful it is for me.
- Free for 2GB storage, or $100USD/year for 50GB storage
- Files are backed up instantly via internet (smart network/lan usage)
- Automatic file versioning accessible on via website!
- Un-delete files (stored deleted files don’t count against your storage limit)
- Mac/PC/Linux compatible
- App for any smartphone (nice way to show renders to prospective clients)
- Public folder to share large files
- Share folders with other dropbox users(amazing for group projects)
I don’t work with lots of video footage, but I do have a fair share of single 250mb-800mb files that sync no problem. I have two computers, both with dropbox installed and they both always have the same files, along with multiple clients and contractors who I have a network of shared folders with as well. Everything blender related for me is in Dropbox. It’s saved my butt many many times over the past few years!
Thanks crazycourier, i thought about dropbox and Ubuntu One but went for some external hard-drives in the end, a bit old fashioned i know. One day i’ll catch up with the times, Lol.
Don’t mistake yourself as being behind the times. There are a variety of good reasons to avoid these (albeit very handy) online services. In particular, I’m referring to availability and control. By using an external hard drive or your own backup file server, you know exactly where your data is and who as access to it. Sure the online services are convenient, but you have to weigh that against control. For myself, I’m too paranoid to trust my data on a server on which I don’t have root access.
Same here, but imagine you keep all backups on a couple of drives, do you then store these in different places? I, myself, once a year, make one full system backup and all, and this drives goes to the “vault”, is that too paranoid?
Nah. I think it´s responsible - depending on the data.
I even encrypt some of them. They can contain client, any sensible, NDA potected or prototype data.
I tend to password protect my drives but was also thinking of encrypting (some of) the data, What program do u use for that?
You can use http://www.truecrypt.org/downloads ?
I have looked into this TrueCrypt software and it’s ok but there is one big issue for me. The file you encrypted (say you encypt a patition of your stick), remains visible. So if I loose my stick, somabody who finds it knows there’s some hidden content.
I am looking more in terms of the Kingstone SecureTravelle.exe, if you are not logged in, the drive seems to be empty
So if I remove the SecureTraveler.exe file, nobody might ever guess there is secure content on this stick, it’s just an empty device
If I log in, then the content magically apears
where could I find an app like that?
You can do the same with TrueCrypt, if you keep the encrypted data (crypt) on an unencrypted volume. Looks like random data. You can also save encrypted data (a crypt) inside an encrypted volume (a crypt inside a crypt), then you open the first level and looks like random data. Add a few files for plausible deniability then you only give the password for the outside container. The only problem, as with SecureTraveler, if you write over the top of the hidden data, the hidden data is gone. You need the password to find the hidden data.
Every way I tried, there is always a file that is the size of 1.2GB, which is huge for just one file, which is a massive clue there’s secure content inside. Even if you hide a partition inside this firstly encrypted file, the container file is huge, even if it content look like random data.
I see what you mean,
by having an encrypted file, it can’t be overwritten and by hiding the true content inside the encrypted file, amongst some decoy files, nobody will know there is more data to be found.
I never use it! But I did do many work to protect my PC, like dedusting and protect the screen of my computer.