Do those GPUs also brew a great coffee to survive tight deadlines?
4 Likes
Hi all! First post here.
I love the idea of AI assisted Re-topology. The more I talk about it with my friends and co-workers the more I realize that what we’re after isn’t so much a fully automated solution as an interactive and flexible one. For those of us working with scan data or retopologizing CAD… The difference between whats usable in production and what isn’t comes down to how we interact with the software and how much control can we exert over it.
The data driven quadrangulation solution would already save countless days on many projects if someone decided to give it a user interface that was simple yet powerful in an existing 3d package. 3D coat attempted this but it’s not quite there yet, it’s too clunky and doesn’t let you quickly layout the pactches at the start.
For blender it would be awesome to see a workflow that was a 3 step process:
-
Layout large patches that manually define the 5 point star locations. This can be done via quickly sketching on the surface with the only requirement being a completely closed shape before being recognized as a “patch.”
-
Generate initial topology via best guess topology flow.
-
Manually sketch to define individual patch edge flow.
For hard surface the first step could be generated via “hard edges” or an angle threshold.
What do you guys think?
P.S. If anyone is currently working on something like this I would love to lend my expertise in modeling from the last 10 + years to help make this tool production ready!
3 Likes
Hi Joel,
Just informing you of this, in case you haven’t read it yet:
An auto-retopologizer with very satisfactory results will be available for Blender soon (and is already available for a few 3D editors, such as 3ds Max, Maya and Modo).
I guess it’s not AI, but the results are pretty impressive.
1 Like
Do you know how much it will cost? 
1 Like
There is no reason in trying to enforce some sort of AI just for the sake of it. If a tool produces sufficiently good results, what should we want more?
2 Likes
Thanks Metin_Seven!, I’ve already tested Exoside’s retopology tool inside maya, and while it does produce decent results, it breaks down rather quickly on more complex models.
The benefit of the data driven quadrangulation is that it allows you to isolate individual patches of your model and edit them with a gesture in real time. Setting a bunch of numbers in a dialogue box and pressing “generate” again and again isn’t ideal for an artist who wants to work at a fast pace because it breaks the flow. Plus there’s often cases where the results are mostly fine but need a little work in isolated areas.
Fiddling with settings and blasting away the good parts along with the bad everytime you generate a new topology is a waste! Plus trying to merge to different auto retopo results gets messy very fast. (not that it isn’t possible but it IS counterproductive to the purpose of using an auto topology tool in the first place. It’s all about flexibility!
@anon98372585 True, but to date there has yet to be an auto retopo solution to the level that I think is needed. For example, Think of the texturing world before Substance… People didn’t know what they were missing until algorithmic created the substance tools which have completely redefined how we create materials. I’ve been modeling and texturing for VFX and games for the last 10 years and how we interact with verts really hasn’t changed much, and it’s time for a change, one that totally redefines how we model, and an integral part of that is a comprehensive auto re-topology tool-set, the likes of which does not yet exist…
1 Like
I am not disagreeing with you! My point is that it doesn’t matter whether a solution is AI driven or not. Contrary to other parts of the 3d production pipeline, there has been very little progress in dealing with meshes in machine learning. Because of this, it appears more likely to me that improvements in the coming years won’t be based on AI.
Exactly. AI is also (currently) much better at dealing with perceptual based problems such as denoising. Automatic retopology is almost purely computational with a whole bunch of fancy linear algebra, quadriflow is amazing and it will only get better over time. Making it AI wont make it magically better. You probably know this making your own denoiser anyways.
2 Likes
Most probably the same as for other 3D editors:
1 Like
Quad Remesher is now available!
1 Like
@Metin_Seven: So I wonder, where have I read about this already, it sounds strangely familiar…
Oh wait:
The Topogun-thread, this thread, the Retopology-Design-Task-thread, the Pablo-Dobarro-thread…
Guess you’ve been busy posting this like a re-tweet all over BA.
Honi soit qui mal y pense. 
greetings, Kologe
2 Likes
No hidden agenda, just a genuine fan of the developer’s work, and therefore eager to support him.
Also I’d love to see as many Blender users as possible to discover the power and convenience of ZRemesher, now accessible in the shape of Quad Remesher.
I didn’t mean to suggest a hidden agenda (that’d be just silly) and I understand your enthusiasm.
I’m just wondering about the kind of mess we might get if any major news from any of the aforementioned threads were cross-posted on all the others like this.
Though I understand it’s at least thematically closely related. Muddy waters, I guess.
greetings, Kologe
1 Like
I think the part that AI could assist with is perhaps more managing the data generated by the algorithm instead of building the points itself. Think about it. If you need to process hundreds of pieces for retopology then training the AI for a given type of mesh would prevent you from having to tune individual retopo settings per mesh.
What do you guys think?
1 Like
Could you explain how the workflow would look and which kind of settings could be tweaked by the AI?
@anon98372585, I’m thinking the most useful thing to start with would be training the AI to set the adaptive density and giving the object enough polygons to accurately represent the shape. You could do this by finding the bounding box of an object to first set the target total polygon count since larger objects typically will require more polygons. (to make the system smarter you could evaluate the complexity of the silhouette from the orthographic views as well.)
Once you have an overall target polygon count you need to set the adaptive density. This controls how close each polygon on a given model is to a “square” vs some being squares and some being rectangles.
Next it would have to be trained on several meshes labeled as having a “good distribution amount” which would be hand created by artists. From there it would have to be trained/told which automatic settings produced the closest results to the hand created meshes. Since neural networks seem to work best with images perhaps it can be trained to recognize screenshots of meshes with good topology rather than their actual positions in 3d space? Just a thought! Then the resulting meshes generated from the retopo tool would just get rendered to an image on disk and the images would be checked against the repository of approved mesh screenshots. standardizing the inputs and outputs would be easier too since orthographic views are pretty standard.
Thanks for the detailed description.
Am I understanding you correctly that from a user’s point of view, you would need to be able to define a polygon count and some value for the adaptive density? In your description you explain how an initial estimate for the polygon count might be computed, but I assume that this should still be configurable?
Is it true in general that a good topology consists of as many squares as possible? Are there exceptions for that? Let me know if you have some good references for that.
It would be difficult from my point of view to work with orthographic views, because you would need a lot of them and yet you could still not cover the whole topology.
My intuition is that working with UVs would likely be easier, as they can cover the whole mesh. All the data we want could be baked into textures. The mesh could automatically be unwrapped and a texture map could be computed which contains the position of each pixel in world space and possibly some additional information. Now, the neural network would need to figure out where to place vertices and how they need to be connected.
From my point of view, the most difficult question is how it could be trained. Using actual topology from artists would likely not work, because if you give the same mesh to several artists, each one will have a completely unique solution. We would need to work with something like a GAN (Generative Adversarial Network) to handle such a situation, but they are very difficult to deal with.
However, if you are telling me that a good mesh basically only exists of polygons that are as close as possible to a square, that’s something which can be computed and could be used for the training.
Using the UVs would be great! however the chances are that if a model needs to be retopologized, it wont have any UVs as most artists don’t do any UV work until after the model is finalized. However if you forced an automatic uv map to be generated that would be a good way, however I’m not sure if the cleanliness of the UVs or lack thereof would impact the results.
That being said, yes in most cases a uniform mesh leads to the absolute best results. The only reason we don’t do this all the time is due to time constraints and the difficulty of manually editing so many extra points, or even hardware limitations. The uneven density is something artists do for efficiency related to polygon count in regard to rendering. We always strive to give something to the lighting department that wont destroy their machine as they navigate the scene and attempt to light and relight it based on the director’s notes.
For human characters we add localized detail to areas while attempting to minimize the impact of the stretching. So you could say that in most cases we attempt to have polygons as close to squares as possible for any given “patch.” A patch would be defined as a boundary of 4 connected sides, made up of any curvature, length and number of edges for each side. (see below)
If you look at the screenshots in the link above you can almost see it as a “tension map” across the surface. We try to localize points to get more detail while reducing the overall tensions as much as possible, but a perfectly equalized tension/distribution isn’t achievable by human hands I think.
For hard surface we attempt to optimize to the lowest point count we can get while retaining the overall shape and localize details like this:
https://www.artstation.com/artwork/mYVE1
We use this low polygon “cage” mesh to keep things lite while subdividing at rendertime based on camera distance for efficiency.
This is purely because a dense mesh will become unmanageable to render when duplicated a thousand times around something like a star destroyer… oh and then say you have like 3 star destroyers in your scene… It can get out of hand rather quickily.
This is a great site about topology for anyone who would like to read further on the subject:
https://topologyguides.com/post/165094485335/artifacts
2 Likes
Thanks for the references. It will take me some time to get through those. I have a few new ideas how this might be approached.
Wrap does something in this context, although I guess it’s not AI / ML based: