Article: Phase-Functioned Neural Networks for Character Control

Game Engine Game Engine Support and Discussion
1

I just watched an excerpt from a very interesting video demonstrating a method of character control using data-driven neural networks. Specifically, this approach separates the phase of the motion (timing) to avoid some of the errors in alternative methods.

To be clear, this generates animations on the fly according to terrain conditions. Just watch the video to see how noticeably natural the motion is, and responsive.

Though I doubt many will attempt such a system (it’s tricky stuff), it’s interesting to read nonetheless.

Abstract:

We present a real-time character control mechanism using a novel neuralnetwork architecture called a Phase-Functioned Neural Network. In thisnetwork structure, the weights are computed via a cyclic function whichuses the phase as an input. Along with the phase, our system takes as inputuser controls, the previous state of the character, the geometry of the scene,and automatically produces high quality motions that achieve the desireduser control. The entire network is trained in an end-to-end fashion on alarge dataset composed of locomotion such as walking, running, jumping,and climbing movements fitted into virtual environments. Our system cantherefore automatically produce motions where the character adapts todifferent geometric environments such as walking and running over roughterrain, climbing over large rocks, jumping over obstacles, and crouchingunder low ceilings. Our network architecture produces higher quality resultsthan time-series autoregressive models such as LSTMs as it deals explicitlywith the latent variable of motion relating to the phase. Once trained, oursystem is also extremely fast and compact, requiring only milliseconds ofexecution time and a few megabytes of memory, even when trained ongigabytes of motion data. Our work is most appropriate for controllingcharacters in interactive scenes such as computer games and virtual realitysystems.

Link:

2

Do you want to try and apply it to the walking ragdoll actor?

3

When you create interactive characters which use high quality animation, it is extremely difficult to get all the transitions between the animations right. That is a very time consuming process, requiring to try different transition animations, adjusting the timing, making sure that interruptions are working properly too. Even in AAA games, it is not unusual to find special cases where those systems produce weird results.
If you have a look at the video in detail, you can easily see that especially the foot placement is often wrong on uneven terrain and that it is sliding too. However, I believe that with better training data, this should be resolvable. They have extracted the phase, most likely to help the network to learn faster. For practical purposes however, it would also be required to have e.g. the movement style as input and to allow differently proportioned and sized characters. Improving the performance once that is achieved would certainly be required too. In research, the performance is certainly important, but usually not the central factor.
Even though it is tricky stuff, I wouldn’t be surprised if it becomes the standard within a few years.

4

He slips on the stairs at ~4:00! Looks good, I’m interested to see where this goes…

One of these days I have to give NN’s a try.

5

google tensorflow :smiley: I’m learning it right now

6

Guramax,
I think the ai gym is included in pyBullet*

7

Thanks, atm I wanted to train a general AI that could recognize pattern from dataset I recorded from the game