Hi guys,
I have made an addon that uses RigNet to predict Armature and weights. It’s still experimental but it kinda works
you can find the code and instructions here
their predictive model works better with a T-pose, I haven’t tried to train a custom model yet
Have fun!
Impressive! Thanks for sharing. Would be great to see this integrated in Blender.
Interesting. I’ll give it a try. 
Very cool!
Had a quick glance at the code. Is it correct that the inference is run in PyTorch?
Hi guys!
@Metin_Seven, you mean as an official addon? At the moment it’s very unlikely: too many dependencies, not cross platform, and still not mature enough. But there is hope
@kabu, oh yes please, make yourself at home
@DeepBlender, I am totally new to neural network and AI, so far I have only converted their quickstart script in order to have RigNet talk with blender. You are correct: they use PyTorch. Is it a good framework for neural networks in blender? Do you have any suggestion towards a more direct implementation, perhaps having the model learn from .blend files instead? I haven’t touched the learning part yet
thank you all!
PyTorch is a great framework for the training of neural networks, alongside with TensorFlow. Both aren’t well suited to deploy trained networks for Blender or standalone applications in my opinion. Something like ONNX(Runtime) is better suited for this, as it allows you to easily load a neural network and execute it on CPU/GPU/… . There are converters from PyTorch/TensorFlow to ONNX, but not all operators are supported. I am currently extensively testing the ONNXRuntime and it looks very promising.
When it comes to train neural networks, the bottleneck is very often the loading/augmentation of the training data. That’s why I wouldn’t recommend to load training data from Blender directly, but rather to save the data in files that can directly be loaded for the training.
Holy crap, this is damn interesting !
I tried Installing this but after I create the rignet environment folder I have this error
Python: Traceback (most recent call last):
File “C:\Users\Animator-Ed\AppData\Roaming\Blender Foundation\Blender\2.91\scripts\addons\brignet-main\brignet.py”, line 28, in execute
from . import rignetconnect
File “C:\Users\Animator-Ed\AppData\Roaming\Blender Foundation\Blender\2.91\scripts\addons\brignet-main\rignetconnect.py”, line 10, in
import torch
File “C:\Program Files\Blender Foundation\blender-2.91.0-31\2.91\python\lib\site-packages\torch_init_.py”, line 127, in
raise err
OSError: [WinError 126] The specified module could not be found. Error loading “C:\Program Files\Blender Foundation\blender-2.91.0-31\2.91\python\lib\site-packages\torch\lib\caffe2_detectron_ops_gpu.dll” or one of its dependencies.
location: :-1
I even copy the lip site package files from the rignet env to blenders python but nothing it fails again I am thinking maybe a missmatch of python versions and the compiled windows RTree files but this installation is too hard
I have tried all I can following the exact steps as in the git and setting the environment for Rignet but I get this error
Python: Traceback (most recent call last):
File “C:\Users\Animator-Ed\AppData\Roaming\Blender Foundation\Blender\2.91\scripts\addons\brignet-main\brignet.py”, line 28, in execute
from . import rignetconnect
File “C:\Users\Animator-Ed\AppData\Roaming\Blender Foundation\Blender\2.91\scripts\addons\brignet-main\rignetconnect.py”, line 10, in
import torch
File “D:\rignet\Lib\site-packages\torch_init_.py”, line 127, in
raise err
OSError: [WinError 126] The specified module could not be found. Error loading “D:\rignet\Lib\site-packages\torch\lib\caffe2_detectron_ops_gpu.dll” or one of its dependencies.
location: :-1
I would appreciate if you can point me to what it might be happening
Thank you.
It requires PyTorch to be installed. The way to go is most likely to install pip for Blender’s Python and to install PyTorch for it (using pip). Be prepared to handle with PyTorch issues.
Don’t have time to give you appropriate links, but what I described should be sufficient to google for them.
Thanks but I when intalling the enviroment for rignet I intall pychart with this command
conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.1 -c pytorch
anyway I went and intall it for blender’s python too using
pip install torch==1.7.0+cu101 torchvision==0.8.1+cu101 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
but got more errors so I don’t know is just to difficult for me but defo pytorch is installed in both places now
Unless it needs CUDA since my GPU is AMD 480rx I don’t think it runs cuda but I also tried installing the cuda version but on this test it says is not runing 
Hi @ImpactFrames,
though the code seems to use non-cuda hardware for some operations, AFAIK RigNet is using the cuda framework.
The install can be painful, with lot of dependencies that can also vary according to the hardware and system: at the moment there’s no way around pip or venv/conda.
My opinion as a novice in this field: AI is just becoming to circulate among the masses (such as myself), and only a fraction of the hardware/software park is ready to host AI implementations
Hopefully this is going to change soon
Yes, I was thinking that since my computer might be too different from your configuration even if I manage to install it the results might take longer or be different from yours.
Right now I am installing all the dependencies without cuda using just cpu like in this guide https://github.com/rusty1s/pytorch_geometric
apparently I also have to set this path ‘C:\Program Files\Blender Foundation\blender-2.91.0-31\2.91\python\Scripts’ to the pythonpath if I don’t manage by today I will probably desist of the idea
BTW as a total newbie and having no idea what I am doing still it makes sense so far and having the computer do some of the rigging would be worth it for my workflow.
Thanks.
I went installed an old gforce 970gtx NVIDIA card with cuda I had on another can. Reinstalled the rignet environment again but since the python shipped with miniconda is either 3.8 or 2.7 it only let me install this wheel Rtree-0.9.4-cp38-cp38-win_amd64.whl since the pip install Rtree‑0.9.4‑cp37‑cp37m‑win_amd64.whl says it’s not available for my system despite of being also in the folder anyways I installed 3.8 then this line failed
pip install numpy scipy matplotlib tensorboard open3d==0.9.0 opencv-python
so I used
pip install numpy scipy matplotlib tensorboard open3d==0.10.0 opencv-python
instead and continued with all the other dependecies as the rignet environment guide
installed pytorch with cuda for the blender’s python and upgraded pillow as in your directions then pointed the rignet enviroment folder in blender and added blender python scripts to my sytem enviroment variable paths
result blender throws this beautiful error
Now I give up.
As mentioned previously, that’s very unfortunate, but not surprising… That’s literally the reason why I am looking into ONNX (+ Runtime), which is quite simple to install (at least the CPU version, but even the GPU one is relatively painless). I am planning to convert this model into ONNX in the future, but I don’t have the time at the moment.
Yes is like the minimal variation of package version and hardware configuration trows all off board but I recognize @pKrime efforts on bringing this to blender and I expect everyone else would also welcome your efforts streamlining this installation. via ONNx I am a noob but also doesn’t help that my hardware is always 3 or more gens behind, I guess other people may have an easier time than me installing it.
Hello everyone, sorry if this out of places, in meantime i try to understand whats wrong here, after finally make pytorch detected in blender, i’m running to new issues
Traceback (most recent call last):
File "D:\Aye\blend\2.90\scripts\addons\brignet-main\brignet.py", line 28, in execute
from . import rignetconnect
File "D:\Aye\blend\2.90\scripts\addons\brignet-main\rignetconnect.py", line 11, in <module>
from torch_geometric.data import Data
File "D:\Aye\Lib\site-packages\torch_geometric\__init__.py", line 2, in <module>
import torch_geometric.nn
File "D:\Aye\Lib\site-packages\torch_geometric\nn\__init__.py", line 2, in <module>
from .data_parallel import DataParallel
File "D:\Aye\Lib\site-packages\torch_geometric\nn\data_parallel.py", line 5, in <module>
from torch_geometric.data import Batch
File "D:\Aye\Lib\site-packages\torch_geometric\data\__init__.py", line 1, in <module>
from .data import Data
File "D:\Aye\Lib\site-packages\torch_geometric\data\data.py", line 8, in <module>
from torch_sparse import coalesce, SparseTensor
File "D:\Aye\Lib\site-packages\torch_sparse\__init__.py", line 13, in <module>
library, [osp.dirname(__file__)]).origin)
File "D:\Aye\blend\2.90\python\lib\site-packages\torch\_ops.py", line 105, in load_library
ctypes.CDLL(path)
File "D:\Aye\blend\2.90\python\lib\ctypes\__init__.py", line 364, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found
location: <unknown location>:-1
Is this means something that i not installed yet or could be another thing ?
Thank you for your attention
Hi guys,
sorry that it can be painful to make it work. If it wasn’t for all the insights that I have received on how to improve it, I would regret having released it so soon
to recap:
Unfortunately it’s a lot of work, so the friendly release might take some time
ah thank you i finally make it works !!!
https://twitter.com/TAGAPAW/status/1336820456486129666