Problem with animations

OpenGL
1

Hi,

I don’t usually ask for help online, but I’ve been stuck with this problem since at least 2 weeks and I am totally out of ideas.

I am learning OpenGL. Particularly, I am trying to develop a small game in Rust using Glium. In this case I’m importing materials with russimp which is just bindings for assimp.

I have successfully implemented meshes, camera, etc. But am completely stuck when it comes to animations.

I have followed OGLDEV’s skeletal animation tutorial, which I find to be great for beginners. I feel like I have a pretty good grasp of the maths that’s going on and the way to use the data that assimp provides to display the animations.

However, I really cannot get it to work in practice.

First, I tried to just run through the bones and applying their transformations. The idea is just to go from bind pose, to bind pose again (so I expect the code to return identity matrices for all the final bone transformations). I couldn’t get this to work at all with assimp provided offset matrices. I have a relatively complex mixamo model with animation, tried importing and re-exporting in blender so the file structure changes but eventually I gave up with that, and switched to the model OGLDEV uses in the tutorial (.md5mesh and .md5anim). I converted those to a single .fbx and also gdsl 2.0 binary. Nothing.

So I resorted to compute my own offset_matrices, from the global_transform at each node/bone, I can use that to render the bind pose, no problem. So I thought, ok, it should be fine from here.

But no. In fact, it’s not fine. I have tried to insert animations following the tutorial. It couldn’t be more wrong if I tried. If the current time_in_ticks is outside of animation range, it renders the bind pose. But anything within the animation time, renders complete gibberish. Mesh filling my full scene and stretching out to infinity and beyond.

I think that likely, the problem I couldn’t figure out regarding assimp’s offset matrices, and this one with the animations themselves, is related. But I can’t figure it out. I have a feeling it’s something to do with the structure in the file or something, but I don’t have enough experience to debug any of this…

Here’s code for the computation of the offset matrices (which lead to totally different matrices than that that is generated by assimp):

// Couldn't get assimp's own offset_matrices to work, therefore I calculate my own
fn compute_offset_matrices(scene: &Scene) -> Vec<Bone> {
    let mut all_bone_names = Vec::new();
    // TODO inefficient, bones are again read in extract_scene_bone_data
    // should be one time thing...
    scene.meshes.iter().for_each(|mesh| {
        for bone in mesh.bones.iter() {
            if all_bone_names.contains(&bone.name) {
                continue;
            }

            all_bone_names.push(bone.name.clone());
        }
    });

    let bones = recurse_offset_matrices(&all_bone_names, &scene.root.borrow().clone().unwrap(), &Matrix4::identity());

    return bones;
}

fn recurse_offset_matrices(
    all_bone_names: &Vec<String>,
    node: &Node,
    parent_transform: &Matrix4<f32>,
) -> Vec<Bone> {
    let global_transformation = parent_transform * utils::convert_russimp_mat(&node.transformation);

    let mut out = Vec::new();
    if all_bone_names.contains(&node.name) {
        out.push(Bone {
            name: node.name.clone(),
            offset_matrix: global_transformation.try_inverse().unwrap(),
        });
    }

    for child in node.children.clone().into_inner().iter() {
        out.append(&mut recurse_offset_matrices(all_bone_names, child, &global_transformation));
    }

    return out;
}

Here’s my animation code:

pub(crate) fn update_animation(
    skeleton: &Skeleton,
    animation: &mut AnimationState,
    frame_time: Instant,
    do_loop: bool,
) -> Vec<Matrix4<f32>> {
    let animation_start_time = animation.start_time;
    let animation = animation.animation_data.as_ref();

    let time_in_ticks = get_time_in_ticks(animation, animation_start_time, frame_time, do_loop);

    let mut final_matrices = vec![Matrix4::identity(); 128];

    if time_in_ticks.is_none() {
        return final_matrices;
    }

    let time_in_ticks = time_in_ticks.unwrap();

    recurse(
        animation,
        &skeleton.scene_root,
        &Matrix4::identity(),
        time_in_ticks,
        skeleton,
        &mut final_matrices,
    );

    final_matrices
}

fn get_time_in_ticks(
    animation: &Animation,
    animation_start_time: Instant,
    frame_time: Instant,
    do_loop: bool,
) -> Option<f32> {
    let elapsed = frame_time
        .checked_duration_since(animation_start_time)
        .expect("Animation start time cannot be later than current frame time");

    //let mut time_in_ticks = elapsed.as_secs_f32() * animation.ticks_per_second as f32;
    let do_loop = false;
    let mut time_in_ticks = 0.5 as f32;

    let animation_duration = animation.duration as f32;

    if do_loop {
        time_in_ticks %= animation_duration;
        return Some(time_in_ticks);
    } else if time_in_ticks <= animation_duration {
        return Some(time_in_ticks);
    }

    None
}

fn recurse(
    animation: &Animation,
    node: &Node,
    parent_transform: &Matrix4<f32>,
    time_in_ticks: f32,
    //channels: &HashMap<String, &NodeAnim>,
    skeleton: &Skeleton,
    out: &mut [Matrix4<f32>],
) {
    let mut local_transformation = utils::convert_russimp_mat(&node.transformation);

    let node_anim = find_node_anim(animation, node);
    if let Some(node) = node_anim {
        local_transformation = interpolate_node_anim(node, time_in_ticks);
    }

    let global_transformation = parent_transform * local_transformation;

    if skeleton.bone_index_map.contains_key(&node.name) {
        let bone_index = skeleton.bone_index_map[&node.name];
        let bone = &skeleton.bones[bone_index];

        out[bone_index] =
            skeleton.global_inverse_transform * global_transformation * &bone.offset_matrix;
        //out[bone_index] =
        //    global_transformation * &bone.offset_matrix;
    }

    for child in node.children.clone().into_inner().iter() {
        recurse(animation, child, &global_transformation, time_in_ticks, skeleton, out);
    }
}

// Helper: find the index of the keyframe just before 'time'
fn find_key_index<T>(keys: &[T], time: f32, get_time: impl Fn(&T) -> f32) -> usize {
    assert!(keys.len() > 1);

    for i in 0..keys.len() - 1 {
        if time < get_time(&keys[i + 1]) {
            return i;
        }
    }

    panic!();
}

// TODO replace this with a pre-computed hashmap ?
fn find_node_anim<'a>(animation: &'a Animation, node: &Node) -> Option<&'a NodeAnim> {
    for channel in animation.channels.iter() {
        if channel.name == node.name {
            return Some(channel);
        }
    }

    return None;
}

fn interpolate_node_anim(channel: &NodeAnim, time_in_ticks: f32) -> Matrix4<f32> {
    // Interpolate scaling
    let scaling = if channel.scaling_keys.len() == 1 {
        utils::convert_russimp_vec3(&channel.scaling_keys[0].value)
    } else {
        let i = find_key_index(&channel.scaling_keys, time_in_ticks, |k| k.time as f32);
        let next_i = (i + 1).min(channel.scaling_keys.len() - 1);
        let t1 = channel.scaling_keys[i].time as f32;
        let t2 = channel.scaling_keys[next_i].time as f32;
        let factor = (time_in_ticks - t1) / (t2 - t1);

        let value_start = utils::convert_russimp_vec3(&channel.scaling_keys[i].value);
        let value_end = utils::convert_russimp_vec3(&channel.scaling_keys[next_i].value);
        let value_delta = value_end - value_start;

        value_start + factor * value_delta
    };

    // Interpolate translation
    let position = if channel.position_keys.len() == 1 {
        utils::convert_russimp_vec3(&channel.position_keys[0].value)
    } else {
        let i = find_key_index(&channel.position_keys, time_in_ticks, |k| k.time as f32);
        let next_i = (i + 1).min(channel.position_keys.len() - 1);

        let t1 = channel.position_keys[i].time as f32;
        let t2 = channel.position_keys[next_i].time as f32;

        let delta_time = t2 - t1;
        let factor = (time_in_ticks - t1) / delta_time;

        let value_start = utils::convert_russimp_vec3(&channel.position_keys[i].value);
        let value_end = utils::convert_russimp_vec3(&channel.position_keys[next_i].value);
        let value_delta = value_end - value_start;

        value_start + factor * value_delta
    };

    // Interpolate rotation
    let rotation = if channel.rotation_keys.len() == 1 {
        UnitQuaternion::from_quaternion(utils::convert_russimp_quat(&channel.rotation_keys[0].value))
    } else {
        let i = find_key_index(&channel.rotation_keys, time_in_ticks, |k| k.time as f32);
        let next_i = (i + 1).min(channel.rotation_keys.len() - 1);

        let t1 = channel.rotation_keys[i].time as f32;
        let t2 = channel.rotation_keys[next_i].time as f32;

        let delta_time = t2 - t1;
        let factor = (time_in_ticks - t1) / delta_time;

        let value_start = utils::convert_russimp_quat(&channel.rotation_keys[i].value);
        let value_end = utils::convert_russimp_quat(&channel.rotation_keys[next_i].value);

        let q1 = UnitQuaternion::from_quaternion(value_start);
        let q2 = UnitQuaternion::from_quaternion(value_end);
        q1.slerp(&q2, factor)
    };

    let translation_mat = Matrix4::new_translation(&position);
    let rotation_mat = rotation.to_homogeneous();
    let scaling_mat = Matrix4::new_nonuniform_scaling(&scaling);

    translation_mat * rotation_mat * scaling_mat
}

Please let me know if this question would have been best asked in assimp forums or some discord server somewhere. And sorry in advance if that’s the case.

All help appreciated! Thank you for your time.

2

Of course, after asking, I found the issue.

Nothing to do with assimp of course, just a bug in my code: opengl (or glium at least) takes column major matrices, but when converting from Vec> I made the mistake of converting it wrongly to a row major matrix.

1 Like
3

Hello :wave:
I’m following ogldevs skeletal animation tutorial too i cloned the repo and downloaded all the dependencies when i run it, it works and logs “textures loaded successfully” but nothing is shown only getting empty window. Did you go through something similar?

1 Like
4

Hey. No, I did not run his code, although that was about to be my next debugging step, when I found the issue. So I can’t help you there unfortunately

1 Like
5

It’s very clearly explained, and you can tell it was written with care!

6

What do you mean? OGLDEV’s code? Or this forum topic?

7

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.