After the release of my short film Ara’s Tale I am now in preparation on releasing all the production files under a cc-by license.
The problem I am facing with this is the sheer amount of data I have. All the files sum up to a total of 18GB.
Hosting this on my own webspace is out of the question. Now I would ask the community for any ideas on how to best distribute this amount of data.
Any input is highly appreciated.
My first main question would be whether that includes simulation data and renders. If it does, what is the size of the project without them? Those files would be useful for someone to use I’m sure, but if the other files are closer to say 4-5GB in total there might be some more feasible options. Community torrent for example.
Not that an 18gb torrent inherently wouldn’t work, but I certainly see your point. Australian internet plans are still generally metered by bandwidth usage even if the caps are fairly high these days. Certainly higher than they were anyway. That and you need to have people seeding that much data.
Secondly, .blend files can compress really well depending on the type of data that’s in them. Compressing each .blend file in Blender via the menu option would take ages if you weren’t doing it per file, but compressing a folder full with winzip, 7zip, etc still helps a lot in many cases.
Thirdly - uncompressed textures? I can’t remember what you were using in terms of file type, but if it was a compressible file type you might be able to batch process a fair bit without that big a loss in (re)render quality.
Most of those are suggestions on how to lower the data size overall, not where to host it. I’ve never hosted that much stuff before other than the Sintel files which I wasn’t really involved with on the storage side of things.
No renders, but simulation data (cloth and hair). The size for the simulation data is ~5GB. That would still leave me with 13GB.
Thats where I see the problem as well.
This could possible save me another 1 GB, but I have to test this.
Mostly PNG, so no further compression doable.
Thanks anyway for this checklist.
It at least sparked the idea to chop up the project into standalone parts, with maybe the textures being reduced in resolution and filetype ( -> jpg). This wont recreate the movie as it is, but I guess that is not the point anyway.
Lets see. Maybe there are some more ideas coming in.
Loramel check http://ourproject.org/ maybe you can host it as svn repo
I figured you probably had thought of that much already but it wasn’t specified. Will let you know if any other ideas come to mind.
After a lot of tinkering and thorough cleanups, I think I have now found a way to reduce and split the project into managable chunks:
base production files (1.1 GB): all blendfiles (models, sets, shot files, compositing) but without the simulation caches. All concept drawings, production notes, poster creation, script, storyboards, title/credit creation files.
texture lowres (1.1 GB): all the used textures but scaled down to half the resolution
single zipped files for each shot simulation cache directory. Total 6.2 GB, but typically ~140MB per shot. These should be available as single files on a per shot basis
full res textures (5 GB): all the originally used textures in full resolution.
The last two parts (simulation files and fullres textures) will have to wait a bit before they are available online.
With this chunking I should be able to use google drive to host these files.
What do you think about this approach ?
That sounds far more manageable. So to get this right, users will be able to grab the production files and low res textures at around 2.2GB and be able to experiment from there? IE, if they were going to experiment with hair and cloth sim using your settings as a starting point they would be rebaking the caches anyway with their new tests.
Thats basically my intention, yes.
If users want to explore the rendersetup/layer and the compositing, they will be able to download the individual baked cache files just for the shot they need to start the render.
For the moment I will wait with the fullres textures and see if demand is here and then find a way on how to transfer them.
I hope to have the basic files up in the weekend.