How to cut rigged mesh with boolean difference modifier?

[Nihal]

What’s the best way to carve out part of a rigged mesh without altering the final geometry or making the texture look deformed? I’ve been trying a variety of ways, currently using a boolean modifier, and it almost works sometimes, but a couple of issues happen. I’m using 2.79 (stable and alpha).

Below are two example meshes. I should mention I don’t intend to cut only planes but also things with 3D volume, so texturing tricks won’t help.

Method 1: Boolean before armature

When I use boolean before the armature modifier then pose the mesh things get weird. It looks like the new edges generated by the intersection don’t inherit the main mesh weight, sticking to their rest pose coordinates:

Boolean before armature would be my preferred approach because it allows me to use a single cutter shape to carve all parts at once.

Method 2: Boolean after armature

This one can be done by also rigging the cutter meshes. It solves the unweighted edges issue and works fine… mostly.

I keep getting a ghost version of the cutter shape in some poses, as if it were joined to the main mesh and I don’t know why.

This issue notwithstanding, I don’t like this method very much. To use it I have to create a cutter shape for each part assigned to independent bones. Let’s say I have a stack of planes, all rounded in the same manner, each one assigned to an independent bone so they can move and bend in different ways. I’d have to create one cutter for each plane or it’d follow a single part of the mesh, failing to intersect with the other masked parts.

→ What I’m trying to achieve

I’m creating a mostly hard-surface customizable object with custom properties, shape keys and drivers, and one of its adjustable parameters are rounded corners. Because it’s dynamic—going from perfectly square corners to super rounded corners—I can’t apply the boolean modifier.

I can’t use the bevel modifier because I’m not working just with planes, and it also limits where I can place edge loops.

I can’t create a shape key rounding the main mesh geometry because it warps the UV map, what doesn’t work for me since I’m simulating the removal of these edges, not geometry deformation. As if you cut it out with scissors.

The question: How can fix the boolean vs rigging issues with any of these methods to achieve the expected result? Alternatively, is there a better and saner way to do what I’m trying to do?

[sozap]

Hello,
This kind of rig is always hard to solve,
What could work in your example case is taking the first approach “Boolean before armature” and use a lattice to deform the mesh instead of the armature. And the lattice is deformed by the bones.

This should work but it really depends on the shape of the mesh and your bone setup.

Another thing that could work, is to use bone envelopes instead of vertex groups, that should bypass the issues caused by boolean changing geometry.

Good luck, and I’m curious about the end result of all this !

[Nihal]

Oh my god. You’re right! I use Bone Envelopes so rarely I forgot this use case. It worked like a charm, requiring no extra work besides adjusting the envelop distance.

Thank you so much @sozap!

Here’s a test with a mesh with volume, a slightly more complex up cutter and bendy bones to really push it:

(looks like a mattress )

[sozap]

Cool !
Indeed bones enveloppes are the kind of things we never use but can come to rescue once in a while !
I’m glad it has helped you !

[bandages]

A couple of alternate ideas:

1. You can copy weights from a non-rendering copy that doesn’t have the boolean modifier with a data transfer. That will give your new vertices proper weights.

2. You can combine a shapekey with a driven UV warp to correct the UVs; might be a little tricky to create the inverse of your shapekey. Or, again, data transfer to copy UV from a non-rendering copy, sans shapekey.

[Nihal]

@bandages Thank you! I never used data transfer before but it was dead simple to implement.

I isolated a couple of pages from a rigged magazine I made some time ago to try it out. They have a slightly more complex armature (nested armature: deform separated from controls) and several close edge loops. It worked beautifully:

The page on the right is the source, on the left the one with boolean and data transfer. Where envelops start to get tricky to implement with complex armatures and lots of closely positioned mesh pieces this approach works well.

You guys rock!

[kabu]

One last thing: what you called “UV Warped” is what you’re expected to get. UV are a square collection of points. The topology is right.
You can push the warped to the expected by refining topology to look like it, but in the end it has to be approximated, because it’s discrete, not continuous.

[Nihal]

@kabu I’m aware of it and I never claimed the contrary.