Error while rendering an animation- Cuda error Illegal Address

Support Animation and Rigging
#1

Hi everyone.

I’m rendering an animation consisting in 900 frames total at 720p 900AA samples usgin Cycles. I used the individual frames method, you know, let blender render each frame as an individual png image to insert them later as a time strip and render the actual animation. Yesterday it rendered about 140 frames but today when blender reached the frame number 200 it stopped giving me this error

“Cuda error: ILLEGAL_ADDRESS in cuCtxSynchronize()”

I restarted the rendering process and it seemed to continue normally but I stopped it and came here to get some advice. what is this? what does it mean? does this represent a hazard to my GPU?

Here’s a frame of the animation I’m making for you to see how simple it is, it’s not the big deal, I think it is optimized, reduced tile sizes 64x64, just 900AA samples, caustics disabled, light bounces are set to Min=0 Max=2. The animation consist in a fly-by thorugh a city, a small vehicle flying by the camera and that’s it. My meshes are really simple, no super detailed buildings, not even bump maps where used, the animation lasts 29 sec.

0136.jpg1279×719 94.3 KB

System specs:
8GB ram
AMD FX 6100 3.3GHz
NVIDIA GTX 750 Ti 2GB GDDR5 (http://www.asus.com/Graphics_Cards/GTX750TIOC2GD5/specifications/)
Windows 7 64 bit
Blender 2.73a

Thanks :smiley:

#2

I have this error too

still looking for an answer

#3

BUMP

Problem OP is having is not animation, but rendering based so thread should be moved to correct section on the board.

I also have this problem, which I can resolve by setting a smaller tile size for rendering. I initially rendered @512x512, as the scene started to grow I gradually had to decrease tile size, now I’m at a point where I can render @64x64 otherwise I usually get CUDA crash described by OP. (which is fine but as we know it, Cycles is much slower at smaller tile sizes)

Memory peaks at approx. 2GB, I have a GTX670 4GB so it’s not likely, that it’s a memory issue.

#4

Tried rendering with smaller tile sizes as well

Can’t get smaller than 64x64 or I get significant slowdown in render times

I still get the error

Could it be a bug with NVIDIA drivers or Blender itself?

#5

To better track the problem. All you mention the Blender version, operating system, graphic card, graphic card memory amount, nvidia driver version. Does the problem occurs only in animations?. Does the problem happens with any scene?, otherwise share the problematic scene.
Copy and share the error message.

Edit:
I ask this because the problem of CUDA out of memory can be related to many things. For example I’m having a similar problem to this old report:
https://developer.blender.org/T33465
the problem is related to a default custom escene that I have configured in blender and motion blur enabled. And I discovered that the problem not only occurs with animation, but when render multiple times any scene with motion blur enebled and without closing blender, the vRAM is not freed and memory accumulates progressively until give out of memory error. So just in case your problem is related, to test rename the User blender configuration folder:
https://www.blender.org/manual/getting_started/installing_blender/directorylayout.html

#6

Well, NVIDIA released a new driver yesterday, and now I’m getting a completely different CUDA error:

CUDA error: Launch exceeded timeout in cuMemcpyDtoH((uchar*)mem.data_pointer + offset, (CUdeviceptr)(mem.device_pointer + offset), size)

I’m not sure if it’s a different problem, or the same problem in more detail.

Anyways, for YAFU:

BLENDER VERSION: 2.75a (official release)

OS: Windows 10

GPU: NVIDIA GeForce GT 750M 2GB

NVIDIA DRIVER: GeForce Windows 10 Driver 355.82

ONLY IN ANIMATIONS?: No

ONLY ONE SCENE?: Only two for me so far. (one animation, one still image/render)

The problem isn’t consistent. Sometimes the scene renders perfectly fine and other times it just spits out a CUDA error.

However, I’m unable to render all 2141 frames of the animation without getting a CUDA error within 5 to 100 frames of rendering.

Here is one of the two problematic scenes (this is the still image/render): http://blend-exchange.giantcowfilms.com/b/433/

#7

Do you use this same graphic card for display on the system?
Download GPU-z. Run the render with this scene and while the render is being performed, you see in GPU-z what is the vRAM usage.

EDIT:
I’m sorry, I do not know why I had thought that this was about CUDA “out of memory” problem. Something in my mind made me confused. Perhaps you should seacrh in google if there is any related report with this error message, and if you do not find anything about it, you report the problem.

Apparently everything works well here with this scene, Desktop PC, Linux 64 bits, small/big tiles size, Blender 2.75a, GTX960 4GB, nvidia 355.11

#8

I submitted a bug report to the developers.

So far, it looks like Blender’s CUDA features need some work in order to properly function with Windows 10.

Hopefully, the developers will fix that.

#9

i have the solution to the ‘‘illegal address cuda error’’ i also started getting this message ,so after deleating all my appendid objects,
(which i thought was adding to many verts to my scene) still got slow 10 samples at a time rendering a 750 sample 200% resolution jpeg and then a freeze /crash, 1 and a half hour render time before the cuda error/crash of gpu (gtx 950) fixed solution…
i had 14 layers in use filled with separate objects total scene just over 500,000 verts , I PUT EVERYTHING ON ONE LAYER…
BOOM BOOM !! FIXED WHOO WHOO. now 750 samples is rendering nearly at 50 samples at a time/or pass on tile (256 tile size which i believe is max optimum size for my gpu at least . total render time 1 hour and a few mins , nearly half an hour quicker.
so watch your vertices count guys and layers , happy blendering :slight_smile: