How to smooth or even out a mesh for making shape keys?

Scenario: I have a character with an armature and a rigged dress, posed sitting on a stool which has caused some distortion for which I want to make a corrected shape key.

If I look into this part in the lap, the geometry is very ugly, I’d like to fill in and smooth this area mostly and bring it to a similar height to the top of the legs as you’d expect the fabric to sit…

Normally I’m just tweaking fabric for shape keys in sculpt mode and it’s fairly easy to do using grab/draw/inflate/smooth etc. This time it’s really surprised me that no matter what I try to do I can’t seem to get blender to work like I’m expecting it to - smooth for instance does the exact opposite, pushing the vertices into extreme angles.
Having read up, I think the problem I’m running into is that all the brushes operate based on the “average normals” under the brush and because these normals are already messed up a bit then I’m getting extreme results. My brain is looking at the object and thinking “I just want to fill that dent, should be easy” and blender is responding with “I don’t see a dent, I don’t even see a mesh, I’m just going to stretch these normals in the weird directions they’re already going”.

I’ve tried all the brushes, I’ve tried applying them in different cases, such as tweaking to sculpt on the Z plane (where I want to raise them up) etc and after many hours I’m just stumped as it looks like it’s easy to fix.
I’ve also tried in edit mode but I’m not really proficient enough a modeller to know how to smooth out this organic mesh in that mode.

I’m sure I’m just lacking some sort of trick or technique to solve this but I’m stumped right now so looking for suggestions please.

Edit: I should show what happens when I use something like smooth (or really any brushes do similar things)


Underneath! :grimacing:

Look into Facesets and mask from Faceset…you can then invert the mask so that only the faces you selected in edit mode are active and you can maintain the Faceset Boundaries…
Here is a quick tutorial by Blender Secrets…

also check out @BlenderBob, HERE
he uses it on hard-surface modeling to smooth the Facesets and keep the boundaries…

Thanks for the suggestion, I’ve already tried using face sets though.

It’s not really a problem I have masking the faces as I’ve hidden everything around so I can get at the right area, it’s more a problem smoothing or flattening them. I want to be able to re-even up the mesh and bring the dented bit up to a fairly even level.

It’s possible to get the right look by deleting the faces and doing a grid fill in edit mode, but being a shape key part way through an animation, the re-ordering of the vertex IDs with that destructive approach causes other problems with other shape keys and destroys the weight painting.

Have you tried proportional editing (letter O in the keyboard) it’s a bit tricky but you should be able to control it and achieve the look you desire.

Also you can add a curve object that resembles the curve you want to achieve and Snap onto its surface. Although with broken Normals meshes it’s usually a pain, but it’s worth a try.

Either way, if you’re normals are strange I wouldn’t try any of the computed ways, doing it manually, edge by edge, would be easier at this point.

I did try proportional editing and it helped but affected too much of the mesh around. Maybe I need to combine that with some hiding or masking so I’ll re explore that.

The curved surface idea and snapping I hasn’t thought of, very clever! I will also give that a go.

You’re right though, at this point I’ve spent so much time searching for what I thought must be an obvious thing I was missing that I could have edited each vertex individually at this point. :man_facepalming:t2:

This is weird though. Unless you accidently flipped Smooth Brush direction to “Enhance details” I would have though that maybe scale is messed up or something.

Tell me about it! Ahahaha

With proportional editing try switching from Vertices to Edges, it’s easier in my personal opinion but manual labor afterwards is no joke.

Also I would give a look to the Edit section (viewport panel by the Gizmo’s right side) , under Loop tools or mesh tools you can find something useful for manual modeling.

Best of luck :slight_smile:

Yep, weird is what I’ve been thinking over several days of trying to find what I’m suddenly doing wrong. I think it’s just normals pointing in different… damn, I can check that can’t I? I checked faces blue/red but there’s something I can use to show the normal direction. :man_facepalming:

I did try flipping the smooth brush to enhance too - both settings did exactly the same!
But use any of the brushes on the flatter part of the mesh - work as expected.
After reading the manual and some other resources, I’m sure it’s just about the normals being complicated and I guess not a load of geometry to go off. I’m really surprised the smooth, slide relax etc functions don’t work on it though as that’s what they’re for?

Hm, there should be plenty of geometry for this sort of thing… Can you maybe share the .blend file?

Even weirder, the normals look OK really.

I’ve uploaded the file, it’s 9MB even with everything cut out so still too big to attach.
https://1drv.ms/u/s!AsDYVQWipbyKg8Alm1FlEF4iRcztbQ

What puzzles me is I’ve made loads of shape keys for clothing and animation fixes before and I usually push stuff around in sculpt mode and don’t have any issues. This is the first time I’ve come across something that has just behaved unexpectedly and keeps doing so…

I will have to check it’s not a 3.0 issue tomorrow as I guess most of what I’ve done has been in earlier versions. :thinking:

Oh, quickly tried it in 2.93.4 and when opened in that version the dress no longer bends as all the vertex groups disappear. It certainly used to be fine in earlier versions as I started this character off in 2.9x

Very odd.

Oh hey, sorry for delayed reply. I think I got it.
That “simplySmooth” modifier? Turn it off and it’ll work (does it even do anything? It seems to be set to do nothing at the moment).
I’ve no idea why it behaves that way.

PS. Don’t forget to set Smooth Brush back to “smooth” direction.

Thank you, thank you, thank you!

That’s it, you worked it out. :smiley:

The SimplySmooth modifier is left behind after stitching the original dress together using Simply Cloth Pro, I’m not sure it’s even supposed to be there after converting the cloth to a mesh… I’ll log a bug report with the author of the addon.

After your fix, it took me around 4 minutes to make the corrective shape key and if I’d been unaware of the cause going forward with future projects no doubt I’d have wasted much time in the future. :clap: