Animation engine

[ASVIEST]

Animations is hard, to get it need to implement:

1. Bones:
   1.1. Skinning alghoritm (right better but harder LBS => DQS, for first version LBS is enough, BBW for weights generation)
   1.2. Inverse kinematics system
   1.3. Ragdoll = apply physics to bones
2. Mesh
   2.1. Delaunay triangulation as base mesh generator (can use https://github.com/Nycto/DelaunayNim or
   https://github.com/patternspandemic/delaunator-nim)
   2.2. Extend delaunay via Constrained Delaunay Triangulation (optional)
   2.3. Quad meshes (it simpler and predictable for some models/sprites than triangle meshes, algorithm for generation: QuadriFlow (or maybe QuadGPT))

I realy want BBW, but it hard to implement (for 3d):

1. need QP solver, ProxyQP with BSD-2-Clause license good option
2. Need Constrained Delaunay Tetrahedrization it veeery hard (see https://arxiv.org/pdf/2309.09805) but can use some library

[ASVIEST]

For first version easier implement BHW (bone head weighting) implementing paper: https://people.csail.mit.edu/ibaran/papers/2007-SIGGRAPH-Pinocchio.pdf instead of BBW. But of course it not so good as BBW.

Next level of hardcore (after BBW) is Robust Biharmonic Skinning Using Geometric Fields(https://arxiv.org/html/2406.00238v2), it can not rely on tetrahedrization and can find solution when other methods breaks, it also faster but it sooo hard, even harder than BBW because most hard problem for BBW is Constrained Delaunay Tetrahedrization.

[ASVIEST]

To understand how to implement BHW see:
https://github.com/haoz19/Automatic-Rigging
https://github.com/alexanderlarin/pynocchio

Also notice that for BHW we need only 4 Skin Attachment step from pinocchio algorithm because it was initially created for full auto rigging

[ASVIEST]

class of systems that implements auto rigging: AI based approaches. It also do skinning. Main disadvantage: non deterministic.

Unirig
RigAnything
SkinTokens

It generates both rigs and skinning so you can try to think that manually editing rigs impossible but it same problem as human made rig and painted weights, painted weights should somehow stay even when rig edited. If I understand correctly blender and other software uses Inverse Bind Matrix (IBM) as solution for this problem.

Logic like this:

HasPaintedWeights = User changed generated weights from SkinTokens or BBW/BHW

Move bone --+--HasPaintedWeights--> IBM
          --+---------------------> Re-run skinning (--use-skeleton for SkinTokens or BBW/BHW) -> IBM

[ASVIEST]

Need for BHW:

∆ is the discrete surface Laplacian, calculated with the cotangent formula [Meyer et al. 2003]:

I.e we limit j by vertices that connected with i (need edges (i, j))

[ASVIEST]

For solver I need CSC, so good option - do Coordinate Format (COO) -> CSC conversion.

[ASVIEST]

For non skeleton things and instancing many animation actors (but in cost in physics applicability) we need vertex animation textures:
https://www.youtube.com/watch?v=NQ5Dllbxbz4
https://medium.com/tech-at-wildlife-studios/texture-animation-techniques-1daecb316657
https://stoyan3d.wordpress.com/2021/07/23/vertex-animation-texture-vat/

[ASVIEST]

Interesting approach for animation blending:
https://theorangeduck.com/page/animation-blend-spaces-without-triangulation
https://docs.godotengine.org/en/stable/tutorials/animation/animation_tree.html#blend2-blend3

Next level is motion matching:
https://docs.o3de.org/blog/posts/blog-motionmatching/
https://mocaponline.com/blogs/mocap-news/motion-matching-games-guide?srsltid=AfmBOopcufzUInON1l3NODf-usNU7Zfz8w3i2E8EtjVpzzt8yFx-tbHi
https://github.com/GuilhermeGSousa/godot-motion-matching
https://dev.epicgames.com/documentation/unreal-engine/motion-matching-in-unreal-engine
o3de/o3de#7232
https://upc-virvig.github.io/MMVR/
Also there are Sparse Motion Matching

https://upc-virvig.github.io/MMVR/
https://dl.acm.org/doi/10.1145/3623264.3624442