Converting Blender's EXR to video with FFMPEG

Support Lighting and Rendering
,
1

Hi! So…

I’ve been playing with Blender’s EXR files and FFMPEG for a while now. Since EXRs are by far the fastest to write/read and carry all the data I can ever need at some point, and also can be as lightweight as needed.

Though ultimately, they are not simple 8bits sRGB jpegs, so I have to somehow deal with the color transforms to keep my color work as is. And when using multilayer EXR, ffmpeg simply can’t recognize the channels at all.
So… Two questions:

  • What’s the best transform method? Or better: is there a way to use Blender’s OCIO config in ffmpeg?
  • How to deal with multilayer files ? It’s half the reason why I want to use EXR in the first place, so it would be cool to make it work.
Detailed explanations:

In the past I came up to this solution for converting with ffmpeg:

ffmpeg ^
	-apply_trc iec61966_2_1
	-framerate 24 ^
	-start_number 1 ^
	-i "frames_%%03d.exr" ^
	-c:v libx265 -pix_fmt yuv420p -crf 23 ^
	-tune fastdecode ^
	"%~p1%~n1.mkv"

The important part here is -apply_trc iec61966_2_1 which is - so far - the color transform giving the closest result from what I do in Blender. Still not the same though.

Just today, I’ve found I could use -gamma 2.2 instead. After a few very quick tests, it seems to give closer results (and it’s an option I can actually easily memorize, so that’s cool). Though I’d need to do deeper tests to see how close it is.

Now, the issue is, … I can use ffmpeg only on simple EXR files, not on EXR multilayer files. The reason being, it seems ffmpeg can’t figure out which channels are what:

[exr @ 000001a6b639df80] Unsupported channel Composite.Combined.A.
[exr @ 000001a6b639df80] Unsupported channel Composite.Combined.B.
[exr @ 000001a6b639df80] Unsupported channel Composite.Combined.G.
[exr @ 000001a6b639df80] Unsupported channel Composite.Combined.R.
[exr @ 000001a6b639df80] Unsupported channel Main.AO.B.
[exr @ 000001a6b639df80] Unsupported channel Main.AO.G.
[exr @ 000001a6b639df80] Unsupported channel Main.AO.R.
[exr @ 000001a6b639df80] Unsupported channel Main.Combined.A.
[exr @ 000001a6b639df80] Unsupported channel Main.Combined.B.
[exr @ 000001a6b639df80] Unsupported channel Main.Combined.G.
[exr @ 000001a6b639df80] Unsupported channel Main.Combined.R.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject00.A.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject00.B.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject00.G.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject00.R.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject01.A.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject01.B.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject01.G.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject01.R.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject02.A.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject02.B.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject02.G.
[exr @ 000001a6b639df80] Unsupported channel Main.CryptoObject02.R.
[exr @ 000001a6b639df80] Unsupported channel Main.Debug Render Time.X.
[exr @ 000001a6b639df80] Unsupported channel Main.Debug Sample Count.X.
[exr @ 000001a6b639df80] Unsupported channel Main.Denoising Albedo.B.
[exr @ 000001a6b639df80] Unsupported channel Main.Denoising Albedo.G.
[exr @ 000001a6b639df80] Unsupported channel Main.Denoising Albedo.R.
[exr @ 000001a6b639df80] Unsupported channel Main.Denoising Depth.Z.
[exr @ 000001a6b639df80] Unsupported channel Main.Denoising Normal.X.
[exr @ 000001a6b639df80] Unsupported channel Main.Denoising Normal.Y.
[exr @ 000001a6b639df80] Unsupported channel Main.Denoising Normal.Z.
[exr @ 000001a6b639df80] Unsupported channel Main.Depth.Z.
[exr @ 000001a6b639df80] Unsupported channel Main.Noisy Image.A.
[exr @ 000001a6b639df80] Unsupported channel Main.Noisy Image.B.
[exr @ 000001a6b639df80] Unsupported channel Main.Noisy Image.G.
[exr @ 000001a6b639df80] Unsupported channel Main.Noisy Image.R.
[exr @ 000001a6b639df80] Missing red channel.
[exr @ 000001a6b639df80] Missing green channel.
[exr @ 000001a6b639df80] Missing blue channel.
Error while decoding stream #0:0: Invalid data found when processing input
Cannot determine format of input stream 0:0 after EOF
Error marking filters as finished
2

Multilayer exr’s are not meant to end up in a video format.

It’s meant for compositing, and from there you can export to a (non layer) exr or other file format frame sequence, or directly to a 8/10 bit video format like QT or something, if you want to edit the result in something like Resolve.

It would explain the unsupported channel errors in your tests. And video and high dynamic range is still a massive can of worms… :wink:

3

This doesn’t help.
I don’t want to do two separate file savings for editing and direct encoding, it’s a waste of time.
The combined RGB data channels are there in the file, there must be a way to use them in ffmpeg.

Besides, I think I remember being able to use multilayer EXR files from Nuke to ffmpeg without any issues, so the issue isn’t unbeatable.

4

When exporting from Nuke, were you exporting the endresult of the comp?
If so, you’re just exporting that, with Nuke’s grading on top to convert it from linear to a sRGB .mp4.

I did a quick look in the ffmpeg docs, but found no way to exclude the multilayer data from exr’s.
My suggestion would be to move this to the ffmpeg user forum, you might have better luck there for this one.

5

Recent versions of ffmpeg should support exr layers, assuming they were built with the appropriate flags.
Argument order should be something like ffmpeg -layer {layer_name} -i {sequence}.exr
You should be able to use the lut3d filter for applying transforms. Personally I’d still use oiiotool to apply color transforms and output a temporary tif file for encoding as a video file.

The reason Nuke exrs may have worked for you in the past is probably due to the beauty pass being in the RGBA channels where Blender prefixes always prefixes beauty channels with Combined.

6

Yeah that’s what I thought, when looking at the error it does say this:

[exr @ 000001a6b639df80] Unsupported channel Composite.Combined.A.
[exr @ 000001a6b639df80] Unsupported channel Composite.Combined.B.
[exr @ 000001a6b639df80] Unsupported channel Composite.Combined.G.
[exr @ 000001a6b639df80] Unsupported channel Composite.Combined.R.

So I guess I could do something like

ffmpeg -layer "Composite.Combined.R;Composite.Combined.G;Composite.Combined.B;Composite.Combined.A;" -i {sequence}.exr

I can’t find -layer in the documentation.

In the meantime I made some LUT’s… Can’t finde the perfect setting for having the exact same result though. But here we go:

7

It’s in the binaries’ help. You might only need the prefix Composite.Combined as the argument.
Also if you’re using the File Output composite node to export then I think the combined output will default to un-prefixed R,G,B,A if you leave the layer name empty. I haven’t tested that I’m sure I recall other posters doing it.

Here’s a copy of the exr specific ffmpeg options. You’re find them as part of output from ./ffmpeg -h full

EXR AVOptions:
  -layer             <string>     .D.V...... Set the decoding layer (default "")
  -gamma             <float>      .D.V...... Set the float gamma value when decoding (from 0.001 to FLT_MAX) (default 1)
  -apply_trc         <int>        .D.V...... color transfer characteristics to apply to EXR linear input (from 1 to 18) (default gamma)
     bt709           1            .D.V...... BT.709
     gamma           2            .D.V...... gamma
     gamma22         4            .D.V...... BT.470 M
     gamma28         5            .D.V...... BT.470 BG
     smpte170m       6            .D.V...... SMPTE 170 M
     smpte240m       7            .D.V...... SMPTE 240 M
     linear          8            .D.V...... Linear
     log             9            .D.V...... Log
     log_sqrt        10           .D.V...... Log square root
     iec61966_2_4    11           .D.V...... IEC 61966-2-4
     bt1361          12           .D.V...... BT.1361
     iec61966_2_1    13           .D.V...... IEC 61966-2-1
     bt2020_10bit    14           .D.V...... BT.2020 - 10 bit
     bt2020_12bit    15           .D.V...... BT.2020 - 12 bit
     smpte2084       16           .D.V...... SMPTE ST 2084
     smpte428_1      17           .D.V...... SMPTE ST 428-1