UE5.0: Facial Control Rig

Rig Controller in Unreal Engine enable a seamless workflow from Mocap to Animation

Unreal Engine Control Rig Plugin opens up a new way of animation editing for animators in Unreal Engine. Animator can use facial mocap like ARKit and then record the data, edit the facial record data inside Unreal Engine. Whether to record it again or add a layer to control the current existing animation. The whole workflow is easy to setup and easy to edit. This implement save a lot time doing lip sync and facial expressions but also flexible enough to exaggerate the record data.

Facial Asset Requirements

In order to use both Facial Mocap system and rig controller in Unreal Engine. First, I recommend separate head and body to two different skeletons and skeletal meshes. The reason is simple, Facial animation can be record separately from body mocap. This setup simply gives you whole lot of room to either use the facial mocap data only or with body mocap data. Even you can use existing animation sequence and blend with your new facial animation data. There are other advantages separate head and body or even separate arms, tails, wings etc. These setups can give you further control different parts with each special animation blueprint for automatic work.

Second, the facial asset should keep both blendshapes and rigs i.e. Eyes rig, Mouth rig, rigs for lids, nose , etc. The reason is we're not only use 52 blendshapes to control facial expressions. That would be too limited. If the skeletal mesh contains rigs that would enable us to further edit the expressions with tiny little different change rotation or position of the rig. That create a lot more details.

So, overall, we need:

Control Rig

There are 3 solvers in control rig, Setup solve, Foward Solve and Backward Solve. These 3 solve in control rig makes the rig graph quite intimidating and messy. But it's quite simple Setup solve is for procedurally set up the same rig hierarchy. Foward Solve is for runtime control rig or manually control rig inside sequencer. Backward Solve is reverse the animation keys for sequencer use.

I recommend following the Metahuman control rig setup first before you start your own. But be aware of Metahuman facial control rig does not contain facial bones/rigs. They're using blendshapes for expressions only.

In my setup, I used Jaw bones, lips bones, cheek bones, Eyes bones, Nose bones, Brow bones, Neck bones and 52 blendshapes for combined controllers.

Procdural Animation

Here is an example using mocap data and add a layer to further control expressions. You can use it as a base animation and then edit the animation or add multiple layers to exaggerate the expressions.

Additional Feature: Animation Transfer

If you design the whole character bones hierarchy and setup. You can transfer the animation data just like charm. In this case, I have two very different facial skeletons. However, I designed the hierarchy almost exact the same, except for the character on the right has only right eye.

I can share the control rig with the right character and also share backward keys with this character. This brings you flexibly to maybe share some facial animations with other characters. For example, crowds and background characters. You don't need to create all the animations one by one, if their skeletons are not similar or share the same one.

This enables all kind of usage, even in live event. For instance, mocap plus control rig keyboard control for real-time facial combined expressions. This is such an exciting new implantation.

Last Update:  December 7, 2022 

Software: Unreal Engine 5.0

OS: Windows 10

Specs: RTX 3080, AMD 3900x, 64GB RAM


Reference::