Canon 5d mark ii h.264 mov's crashing blender

Hi Everyone,

I’m using ubuntu 10.04 64 bit and I’m having difficulty importing the h.264 mov files produced by the Canon 5d mark II. This happens in all of the builds of 2.5 I have tested and 2.49. 2.49 crashes when scrubbing over an imported clip and 2.5 crashes while trying to import the file. In the past, I have been able to work with these mov’s in 2.49. If I am not mistaken, I think the crashing came after my update to ubuntu 10.04. I am also unable to import these files into pitivi although I had previously been able to work with these video files without any problems.

I searched here in blenderartists and saw a previous thread regarding problems related to colorspace and the canon 5d mark ii files. In this thread, crashing is also referred to but it seems it is unresolved. Rather than dig up this previous thread I am starting this one to hopefully collect any wisdom on fixing this issue.

Beyond crashing blender, I am also experiencing a problem in all programs with the playback of these h.264 files from the Canon 5d mark II. I have no difficulty with playback speed but the color is rendered incorrectly. As I understand it, these files are YUV while RGB is the standard for display on computers. This was a universal problem back when the 5d was first introduced but there was an update to quicktime (quicktime 7.6) that fixed this display problem. I am unsure if this colorspace problem is related to the crashing in blender.

Any insight on these issues would be greatly appreciated.

Update your FFMPEG to latest which should solve the crashes. Also you won’t get a decent import of your 5D mov’s without it. :slight_smile:

I searched here in blenderartists and saw a previous thread regarding problems related to colorspace and the canon 5d mark ii files. In this thread, crashing is also referred to but it seems it is unresolved. Rather than dig up this previous thread I am starting this one to hopefully collect any wisdom on fixing this issue.

I did a bug report on the crash and the conclusion was an FFMPEG problem not blenders.

Beyond crashing blender, I am also experiencing a problem in all programs with the playback of these h.264 files from the Canon 5d mark II. I have no difficulty with playback speed but the color is rendered incorrectly.

Does that mean you have realtime or close to it playback of these 5D mov’s or you’re happy with slower? Interested as I use a T2i and struggle with realtime or even close to it, never mind edit. :slight_smile:

I’m using 2.49b to edit, with AVISynth. Inputing movies via .avs scripts and blender generates proxies to work with. Does a darn good colour space conversion to RGB and any other video preprocessing.

Render incorrectly? Colour darker or lighter? Do you have a 2.5 build with colour management issues, since resolved? Do you get the colour problem with CM off?

As I understand it, these files are YUV while RGB is the standard for display on computers. This was a universal problem back when the 5d was first introduced but there was an update to quicktime (quicktime 7.6) that fixed this display problem. I am unsure if this colorspace problem is related to the crashing in blender.

Any insight on these issues would be greatly appreciated.

Your h264 mov’s are full range YCbCr BT709 video colour space, but importing them into blender means an immediate colour space conversion to RGB. Which has nothing to do with PC monitors or playback more blender’s internal processing, but then most NLE’s convert to RGB and back for effects / blending / compositing very few work in native YCbCr throughout. But what is at question is the way that colour space conversion is done and the ‘damage’ that can be done before even getting to play back with quicktime or other. On playback YcbCr etc are converted to RGB but that is a seperate process.

I wouldn’t say YUV -> RGB conversion was a ‘universal’ problem for the 5D, only if anyone used quicktime to decode the mov’s. Stu Maschwitz nailed it very early on and blogged about it on Prolost years ago :-). Many simply didn’t or won’t entertain quicktime and had alternative workflows, unwrapping the h264 from the mov container, which takes seconds and working either directly with h264 or transcode to Cineform or MXF etc.

Hopefully new FFMPEG and recent 2.5 build will solve some of the issues. AVISynth will solve all but one, no proxies in 2.5 so far. :slight_smile:

Hi Yellow,

Thank you for sharing your experience and suggestions. I have updated ffmpeg and I am still encountering the same crashes in blender 2.49 and 2.5. I am able to open the files as mp4 rewrapped via ffmpeg. These mp4 files playback at correct speed if you set the render to 25% and turn off anti-aliasing. And in terms of color management, I don’t notice any change in the renders with color management on or off in the latest beta 2.53.

To clarify, the color problem I mentioned and still have is the contrasty image that is a result of 16-235 YcbCr looked at in RGB 0-255. Or at least this is my understanding of why it is displayed with more contrast as the original videos appear in camera. I do thank you for pointing out that it is YcbCr and not the earlier analog standard YUV.

And you are right that Stu noted this problem on his blog. I have been working with 5d files since it’s launch and remember well the shift to working with the correct color range. Thinking back, I admittedly can’t remember how I resolved this colorspace issue after I migrated fully to working in ubuntu. Perhaps an earlier version of ffmpeg handled the color correctly and the latest versions have some problem.

Ultimately, I would prefer to find an open source work-flow that does not involve transcoding or rewrapping the 5d files. I have found the video editing program Pitivi to be the best at handling the original 5d files. It plays them back perfectly except for the fact that it now displays them with the same contrasty image.

Just a thought, I wonder if my cross program problems with correctly displaying the canon files are related to gstreamer. I am unclear as to it’s relationship to ffmpeg.

Again thank you for sharing and I appreciate any further perspectives on how to solve these colorspace issues.

That’s more like it, unwrapping to mp4 and 25%. :slight_smile: CM has affected and not affected video output over the development time of 2.5 but resolved now maybe it’s just switched off in the code for VSE, I’m unsure.

Crashing is a difficult one to pin down, I don’t seem to get them, I would say that Blender is pretty stable.

To clarify, the color problem I mentioned and still have is the contrasty image that is a result of 16-235 YcbCr looked at in RGB 0-255. Or at least this is my understanding of why it is displayed with more contrast as the original videos appear in camera. I do thank you for pointing out that it is YcbCr and not the earlier analog standard YUV.
With latest 2.5 and FFMPEG, preferably a svn build of FFMPEG, not Ubuntu’s ‘stable’ version, you should have no import problems, there should be no 16 - 235 expanded to 0 - 255. Not with progressive sources, like your mp4’s. Interlaced sources still expand contrast though as this has not been resolved in Blender’s code or FFMPEG’s afaik. I’m on Ubuntu Lucid, but Koala was ok too.

What FFMPEG command line are you using? It’s just a demux not re encode?

To check you could use the VSE’s luma waveform and RGB Parade, if you see horizontal gaps in the scopes then your sources have been stretched. Also if the waveform hits the bottom green line ‘0’ RGB and the top line ‘255’ RGB (110% YCbCr) then most likely that its getting stretched. Unless you shot a very contrasty scene. :slight_smile:

I import T2i source regularly and have no problem with contrast expansion.

And you are right that Stu noted this problem on his blog. I have been working with 5d files since it’s launch and remember well the shift to working with the correct color range. Thinking back, I admittedly can’t remember how I resolved this colorspace issue after I migrated fully to working in ubuntu. Perhaps an earlier version of ffmpeg handled the color correctly and the latest versions have some problem.
FFMPEG has never handled colour space conversion ‘correctly’, but then ‘correctly’ is a bit inaccurate, it does a typical contrast expansion to RGB because of the premise that Yxx ‘16’ is black and Yxx ‘235’ is white. But that is a bit outdated and unhelpful these days with reference to video editing anyway.

Even now FFMPEG does a poorer job than for example AVISynth, I get near double the unique colours in a frame using AVISynth to do the conversion to RGB than FFMPEG.

But the thing here is that generally the contrast problems you’re experiencing are from a Yxx to RGB conversion, other vid editors (and remember Blender’s VSE is not really an NLE) on Linux or other OS, don’t do a conversion straight off. FFMPEG just plays the video at the levels you recorded in camera, passes through, the NLE does a conversion to RGB and back to Yxx for compositing transitions and applying colour effects, probably done in 8bit precision and sRGB gamma encoded colour space.

Blender offers 32bit float precision and linear work flow for blending / composting / rendering. :slight_smile:

Ultimately, I would prefer to find an open source work-flow that does not involve transcoding or rewrapping the 5d files. I have found the video editing program Pitivi to be the best at handling the original 5d files. It plays them back perfectly except for the fact that it now displays them with the same contrasty image.
Well I think every 5D, 7D, T2i user is in the same position, transcoding to another format like DNxHD, Cineform Neoscene, MXF or adopt a proxy work flow. Or buy CS5 for the Mercury Engine. Production Suite looks fantastic value. :slight_smile:

Just a thought, I wonder if my cross program problems with correctly displaying the canon files are related to gstreamer. I am unclear as to it’s relationship to ffmpeg.
I like Pitivi, very basic, very slow development and niggly encoding presets/loss of presets and crashing but it’s gstreamer based with option to employ ffmpeg for formats gstreamer can’t handle, maybe it’s using FFMPEG in your case.

Again thank you for sharing and I appreciate any further perspectives on how to solve these colorspace issues.
The blog in my signature has a bit more info. Personally I think Blender’s VSE & Nodes is the best open source NLE.

Out of the important things personally is 32bit float processing, linear colour space compositing, usable scopes for analytical use, nodes, jack transport to sound apps and a 3D environment and all that jaz. It’s more stable than Kdenlive (well maybe not in your present situation) which I’ve seen and used since Jason first coded it years ago and Openshot is nice but…

There were recent commits for the VSE that improved memory handling, I wonder if that would improve decode performance with more capacity available? Probably not.

hi 3point, I’ve been watching blender-cia every day for the last two weeks now, seeing Campbell and Peter making loads of VSE related commits, which is great and building from svn daily. :slight_smile: Proxies soon hopefully. :slight_smile:

But not seeing any performance improvement yet regarding play back but maybe it’s asking too much of my hardware or Blender to playback h264AVC source. Prefetching doesn’t do anything in this case either. Using 2.49b with proxies for now. :slight_smile:

Making bug reports regularly as well, two this morning, getting weird video problems putting one VSE scene into another, sound works across scenes, but video has stopped working. :slight_smile:

The futures bright. :slight_smile:

Yeah I’m getting crashes on playback with audio disruption prior to crash. But just running SD video thru it. The future indeed looks bright tho’.

I’ve got the 7D and I can tell you H.264 is compressed and does not play well with Final Cut Pro, so I can’t imagine it playing well with any other editor. Basically it has to decompress on playback causing stuttering.
The Canon plug-in allows me to import the H.264 and transcode it to a better editing format like AppleProRes. If you don’t have that set up you can always convert with MPEGStreamclip from squared5.com (i think that’s right, too lazy to check).
But the main thing is convert the format to something smaller and uncompressed.
Hope that helps.

FloridaJo check out CS5 on PC, apparently the GPU accelerated decompression is awesome now. Search for Mercury Playback Engine

Thanks 3PE, but I do all my graphic stuff on a Mac.
AppleProRes is awesome. The Canon plugin does the job well for converting.
Cheers

The ProRes route is fine if your NLE supports it and the codecs bundled with it, I don’t think they are available separately?

Streamclip works well, used that on/off since getting the T2i but it relies on Quicktime to decompress to convert. QT has had a chequered past, but probably more sorted in recent.

Avid offer the LE codecs for free download which includes DNxHD, which becomes available to Streamclip to encode out to, but I’ve found that certain system codecs are not available in Streamclip even though installed such as Matrox I Frame, but SC just crashed constantly trying to go to DNxHD. :frowning:

DNxHD is readable by FFMPEG I think, so should go into Blender ok where as ProRes won’t directly. May do via AVISynth.

Good to see some others adding their voices and experiences to this thread. I have done some additional testing and searching online (including yellow’s informative blog) regarding my issues with colorspace and the 5d movs.

As per yellow’s suggestion, I used the command line and ffmpeg to demux the mov to mp4 using fmpeg -i input.mov -vcodec copy -acodec copy output.mp4 I assume this is the correct way to demux without transcoding. I’m also working with an updated svn version of ffmpeg.

My experience, as I suggested earlier is that blender can play these rewrapped mp4’s but is still crashing with the original mov’s. I also looked into the DNxHD format. I converted some mov’s to DNXHD using ffmpeg, “ffmpeg -i input.mov -vcodec dnxhd -b 75000k -vf “scale=1920:1080” -an output.mov” This worked only after I specified the scaling as the original canon 5d mov is seen as 1088. I read up on this and apparently it is 1088 vs. 1080, it’s just that some players and nle’s automatically ignore these additional 8 lines of black. The DNxHD playback in blender is certainly smoother than the original mov’s. I may try to incorporate this conversion to DNxHD into my workflow.

Whatever the format, however, the issue of mishandling of the colorspace of the files still remains. I guess it is possible, as yellow suggested, that I simply shot contrasty scenes; but I am still left feeling like I am missing some of the information in the darks and highlights. I’m not sure if this is true of the other cameras in the Canon dslr range, but the 5d mark ii produces .thm thumbnail files alongside the .movs. These thumbnail files appear to represent the full color range that I would ideally like to see in the mov files. Am I wrong in assuming this relationship and desire to have the .thm’s and .mov’s match in terms of color and dynamic range?

In searching for info on this subject, I have discovered a theory regarding how ffmpeg handles the 5d mov’s. In this blog post, the author notes that the Canon 5d mov’s are a relatively rare form of full range (0-255) YUV (I guess meaning YCbCr) and ffmpeg automatically interprets them as 16-235. He says that there are command lines options to intepret them as full range but they are not working. He believes the problem lies in the swscale library and suggests that he is working on his own tool to deal with the full range YUVs of the 5d. As this is currently unreleased, I’m curious if anyone has any suggestions or approaches to handling the full range of the 5d mov’s.

yellow, I assume your use of avisynth is a work around for just this problem. I am admittedly still in the middle of getting your suggested setup to work but hopefully this is a route that will yield full range files in blender.

And demuxing with FFMPEG should not do anything to the files with regard to loss of data. Blender accepts the mp4’s and thanks to a patch sometime ago from Troy Sobotka, will not contrast expand them. How Blender handles DNxHD with regard to contrast expansion, I’m not sure. Will test.

I get about 10-20% more data generally from using AVISynth for DSLR stuff. But for other formats Blender still contrast expands going to RGB, hence the need for something like AVISynth.

My experience, as I suggested earlier is that blender can play these rewrapped mp4’s but is still crashing with the original mov’s. I also looked into the DNxHD format. I converted some mov’s to DNXHD using ffmpeg, “ffmpeg -i input.mov -vcodec dnxhd -b 75000k -vf “scale=1920:1080” -an output.mov” This worked only after I specified the scaling as the original canon 5d mov is seen as 1088. I read up on this and apparently it is 1088 vs. 1080, it’s just that some players and nle’s automatically ignore these additional 8 lines of black. The DNxHD playback in blender is certainly smoother than the original mov’s. I may try to incorporate this conversion to DNxHD into my workflow.
Good news, an alternative maybe Cineform, a trial can be downloaded. However Blender can not import it directly, probably through AVISynth but that’s an extra process. I think there is an option to go to RGB with Cineform, taking the task from FFMPEG.

Whatever the format, however, the issue of mishandling of the colorspace of the files still remains. I guess it is possible, as yellow suggested, that I simply shot contrasty scenes; but I am still left feeling like I am missing some of the information in the darks and highlights. I’m not sure if this is true of the other cameras in the Canon dslr range, but the 5d mark ii produces .thm thumbnail files alongside the .movs. These thumbnail files appear to represent the full color range that I would ideally like to see in the mov files. Am I wrong in assuming this relationship and desire to have the .thm’s and .mov’s match in terms of color and dynamic range?
Is the thumbnail image darker or lighter than your video files? The thumbnails will have been generated by a cheap Yxxx -> RGB conversion in camera based probably on the shooting profile you’re using.

In searching for info on this subject, I have discovered a theory regarding how ffmpeg handles the 5d mov’s. In this blog post, the author notes that the Canon 5d mov’s are a relatively rare form of full range (0-255) YUV (I guess meaning YCbCr) and ffmpeg automatically interprets them as 16-235. He says that there are command lines options to intepret them as full range but they are not working. He believes the problem lies in the swscale library and suggests that he is working on his own tool to deal with the full range YUVs of the 5d. As this is currently unreleased, I’m curious if anyone has any suggestions or approaches to handling the full range of the 5d mov’s.
The blog post is going video to AE so going to RGB. This is the problem with FFMPEG, swscale I believe is hardcoded 16 - 235 and makes a pretty poor job of the Yxxx -> RGB conversion anyway but only conversion to RGB, leave it Yxxx and FFMPEG I think, from test’s I’ve done will pass it all through to the player. btw not all players and decoders are equal and all sorts of filters, scaling can occur and give odd results. I’ve tested FFDShow as the decoder and do the conversion to RGB in AVISynth and get full range. FFDShow passes it all through, so it would appear FFMPEG is not the problem just swscale.

I think his suggestion that the 5D is somehow different by shooting full range is not accurate and may mislead you. Many DSLR’s have the capacity and do shoot full range, depends on scene and exposure settings, just like my mpeg2 video camera and even old DV cams both shoot 0 - 255 range. (I’m not confusing scaled output there ie 16 - 235 contrast expanded by the decoder), true 0 - 255, might only be 10 to 250 (scene / cam settings dependant), then again some DSLR’s don’t like a Pentax k-x for example shooting mjpeg, throttled to 16 - 235 in cam. :slight_smile:

The noise addition is an interesting one for Quicktime, it’s considered useful to add very discrete noise when going from 8bit to 16bit munging the pixels about to create some interpreted extra data to work with, it’s not necessarily a bad thing, by the time you get to the encoder if it’s fine noise it won’t make it through to the output anyway unless encoded at a high bitrate. The noise could simply be the way QT does the Yxxx to RGB conversion. I’d not use QT to do that personally due to it’s history of scaling and such. He says going to DPX, I use Imagemagick to go to 16bit exr or tiff, there’s an AVISynth route to DPX too.

yellow, I assume your use of avisynth is a work around for just this problem. I am admittedly still in the middle of getting your suggested setup to work but hopefully this is a route that will yield full range files in blender.
Well, I’d certainly not call AVISynth a workaround :-), it offer some immense video processing power, pre & post processing that beats a few commercial offerings and it’s open source. :slight_smile: No real reason not to use it at what it is very good at. I use it more for mpeg2 and dv rather than DSLR source because Blender / FFMPEG still makes a hash of the Yxxx -> RGB conversion for mpeg2 and dv.

Blender acceptable for direct import of DSLR source but not ideal. :slight_smile:

Please keep going at AVISynth, it will be worth it if you really want to get the best out of your video with Blender in particular.

Can you post a sample to work with somewhere?

And this is just getting video into Blender. :slight_smile:

Depending on what is wanted for delivery, there are a number of hoops to go through on the way out of blender, the RGB to Yxxx conversion (which can be pretty poor going back again, giving excessive banding for example), squeezing 0 - 255 full range into 16 - 235 for playback so players don’t clip it off, :slight_smile: many encoders just encode what they are handed, going SD to HD for example (BT601 to BT 709 colour matrix conversion), or HD to SD. Accessibility to encoder settings in a meaningful / useful gui. All these can be done with ease with AVISynth too, rather than through blenders FFMPEG encoder menu flags. The benefit of rendering out to image sequences and using another tool for the encoding to video. :slight_smile:

EDIT

Tested DNxHD with Blender and it contrast expands on import like every other Yxxx codec/container I’ve tried except the DSLR mov’s which import ok, but are impossible to playback.

Image sequences or uncompressed / lossless RGB in an .avi container or using .avs scripts and proxies directly,(which is my preferred option) appear to remain the only routes. But proxies at the moment are only available in 2.49b.

Image sequence route generated by AVISynth + ImageMagick includes ability to write out 16bit tiff’s and import into blender.

So I am still having a bit of difficulty getting avisynth 2.5 to work with virtualdubmod in wine. I have these installed along with ffdshow and I am able to run simple avs scripts from virtualdubmod but I can’t get anything with DirectShowSource to play. Meaning I’ve yet to play or convert any of the 5d mov’s.

As per yellow’s suggestion, I have uploaded an original 5d mark ii mov to mediashare, here. Hopefully this will prove useful as a reference.

Also I don’t mean to clog up these forums with my difficulties getting ffdshow, avisynth, and virtualdubmod to work together in Ubuntu but I would appreciate any suggestions on how to get it working. It appears it may be the only free and ope nsource route for full color video from the Canon into blender.

Cheers.

hi on

DirectShowSource won’t work on Linux, it’s not implemented in Wine hence the use of FFDShow2 tryouts. It’s a bit tricky when trying to describe a process for both windows and linux. :slight_smile:

Sample .avs script:

LoadPlugin=“ffms2.dll”
FFmpegSource2(“C:\windows\profiles\machinename\Desktop\MVI_0560.MOV”)
ConvertToRGB(matrix=“PC.709”, interlaced=false)
flipvertical()

No space in Desktop it’s just this darn web form on BA stuffing it up. :slight_smile:

Make sure you have downloaded the avisynth plug in ffms2.dll, think the web location is on my blog and put it in your AVISynth plugins folder.

The flipvertical() is because the images come into blender upside down if you go the direct avs import into blender, only possible if you’re running a windows build of blender with wine, which is what I’m doing, crazy as it sounds. :slight_smile:

Otherwise load the .avs file above into VirtualDub, save as image sequence or uncompressed RGB. Careful with Virtualdub in general it contrast expands by default going to RGB. :wink: Need AVISynth to do all the work just VDub as the interface for opening and saving. :slight_smile:

Good Luck, PM me if you don’t want to clog up the forum. And if there’s stuff on my blog that makes no sense or misses something let me know, I’ll correct it. :slight_smile:

Hi again. :slight_smile:

Frame 0 via AVISynth: (Best to do save as, .png uncompressed straight out of AVISynth)

http://blendervse.files.wordpress.com/2010/07/onstest_0.png

Then some jpeg screen grabs:

Same frame in Image J, histogram reading 0 upwards:

http://blendervse.files.wordpress.com/2010/07/on_screenshot.jpg

Same frame in Gimp:

http://blendervse.files.wordpress.com/2010/07/onstest_screenshot_2.jpg

Same frame in Image J, histogram reading towards 255:

http://blendervse.files.wordpress.com/2010/07/onstest_screenshot_3.jpg

Your .MOV directly into Blender 2.53svn, no expansion:

http://blendervse.files.wordpress.com/2010/07/blender.jpg

Frame 0 rendered out from Blender’s direct Import:

http://blendervse.files.wordpress.com/2010/07/blender_235_out_imagej.jpg