Home

Assignment 3 — Skinned Multi-Person Linear Model

Due Wednesday, 3/5 Monday, 3/10 at 23:59:59. You must work individually.

Goals

In this assignment, you will be loading and viewing a Skinned Multi-Person Linear model. The blendshapes and the skeletal structure are provided as plain ASCII files. The output of this assignment is a series of OBJ files, which can be loaded into Blender for visualization. However, grading will be performed by comparing the generated OBJ files against the expected output (122 MB zip file). The generated output files should be placed in different folders (named output1 through output6), depending on the task.

Starting point

Download the data for the assignment. Take a look at the files in the input/ folder. The contents are:

This assignment is composed of multiple tasks, each building on top of each other. As you finish each task, you do not need to keep around the code to generate the required output for the task. Only the highest task will be graded, and the points for all of the previous tasks will be given. You may also skip any of the previous tasks and go straight to the final task.

You may ignore the normal for this assignment. We will rely on Blender to generate them.

Your code can be written in C++, Python, or MATLAB.

For debugging, it may be useful to visualize the obj files. One option is to use the Stop Motion OBJ add-on for Blender.

Task 1: Blendshape meshes

The first task is to load and process the blendshape mesh data. Load the \(11\) obj files (00 through 10). The \(0\)th file corresponds to the base shape, and the rest correspond to the blendshapes. Create the delta versions of these blendshapes by subtracting the base mesh: \(\Delta x_b = x_b - x_0\) for \(b \in [1,10]\). Then use the following two sets of \(\beta\)s to generate two meshes.

beta1 = [-1.711935 2.352964 2.285835 -0.073122 1.501402 -1.790568 -0.391194 2.078678 1.461037 2.297462];
beta2 = [1.573618 2.028960 -1.865066 2.066879 0.661796 -2.012298 -1.107509 0.234408 2.287534 2.324443];

Below, the original mesh (with \(\beta=0\)) is shown in blue. The red and yellow meshes correspond to the two \(\beta\) values above. (The colors are added just for visualization purposes; you do not need to export the colors.) From here on, we’ll call these \(\beta^{(0)}\), \(\beta^{(1)}\), and \(\beta^{(2)}\). If we use \(\beta^{(1)}\), then the mesh can be computed as \(x_\beta = x_0 + \sum_b \beta^{(1)}_b \Delta x_b\) for \(b \in [1,10]\), and the result would be the red mesh below. Similarly, using \(\beta^{(2)}\) should generate the yellow mesh.

Export these three meshes to frame000.obj, frame001.obj, and frame002.obj in the output1 folder. You can test them against the solution by running

> ./objscmp ../output1 ../solution1

Task 2: Absolute Translations & Skinning

Load smpl_skel00.txt. In this dataset, each of these files contains the data corresponding to the bind pose of the base and blendshape meshes. Each set of \(7\) numbers corresponds to a bone; \(4\) for the quaternion, and \(3\) for the position. In this file, the quaternion can be ignored, since the rotations are assumed to be identity for the bind pose. The indexing of the bones for the bind pose is shown in the image below. (Note that \(0\)-indexing is used for this image.)

We now apply the absolute translations to the skeletons corresponding to \(\beta^{(0)}\), \(\beta^{(1)}\), and \(\beta^{(2)}\). First load smpl_skel01.txt through smpl_skel10.txt, which contain the bone positions of the 10 blendshape meshes. These need to be deformed to match the blue, red, and yellow meshes. This should be done in the same way as before—create the delta bone positions by subtracting the blendshapes: \(\Delta p_b = p_b - p_0\) for \(b \in [1,10]\), where \(p\) is the bone position. Then add the weighted sum of these deltas to the base bone positions. For a given \(\beta\), the bone positions corresponding to the blendshape can be computed as \(p_\beta = p_0 + \sum_b \beta_b \Delta p_b\).

After we have the skeleton for \(\beta^{(0)}\), \(\beta^{(1)}\), and \(\beta^{(2)}\), we are ready to apply skinning. Load smpl_skin.txt, which contains the skinning weights and influences. In this dataset, the maximum number of influences is \(4\). Each line corresponds to a vertex. For example, the first data line is

0 0.001881 4 0.000927 14 0.000877 15 0.996315

and this tells us that vertex with index \(0\) is influenced by bones with indices \((0, 4, 14, 15)\) using weights \((0.001881, 0.000927, 0.000877, 0.996315)\), respectively. If skinning is implemented correctly, the resulting skinned meshes look the same as before, since we have not moved the skeleton yet.

Now translate the “L_Elbow” bone by \(0.2\) units up in the Y-direction. Since the translation is not relative (not until the next task), the descendants of L_Elbow should not translate. Using \(\beta^{(0)}\), \(\beta^{(1)}\), and \(\beta^{(2)}\), the generated output should look like this:

Export these meshes as frame000.obj, frame001.obj, and frame002.obj in the output2 folder.

Task 3: Relative Translations

We now switch to relative translations. Load the hierarchy information from smpl_hiearchy.txt and store the relative translation of each bone with respect to its parent. For the root (0:Pelvis), the relative and absolute translations should be the same. Once all the relative translations are computed, the absolute translation of a bone can be reconstructed by traversing the relative translations from the bone all the way to the root (0:Pelvis). For example, the ancestors of 18:L_Elbow are: 16:L_Shoulder, 13:L_Collar, 9:Spine3, 6:Spine2, 3:Spine1, and 0:Pelvis. Therefore, the absolute transformation for the left elbow can be reconstructed by combining the relative transformations: \[ T = T_0 \, T_3 \, T_6 \, T_9 \, T_{13} \, T_{16} \, T_{18}, \] where the \(T_j\) are the relative translation matrices. For example, for \(\beta^{(0)}\), the relative transforms for the 0:Pelvis and 3:Spine1 are: \[ T_0 = \begin{pmatrix} 1 & 0 & 0 & -0.0022 - 0\\ 0 & 1 & 0 & -0.2408 - 0\\ 0 & 0 & 1 & 0.0286 - 0\\ 0 & 0 & 0 & 1 \end{pmatrix} = \begin{pmatrix} 1 & 0 & 0 & -0.0022\\ 0 & 1 & 0 & -0.2408\\ 0 & 0 & 1 & 0.0286\\ 0 & 0 & 0 & 1 \end{pmatrix}, \] \[ T_3 = \begin{pmatrix} 1 & 0 & 0 & 0.0023 - (-0.0022)\\ 0 & 1 & 0 & -0.1164 - (-0.2408)\\ 0 & 0 & 1 & -0.0098 - 0.0286\\ 0 & 0 & 0 & 1 \end{pmatrix} = \begin{pmatrix} 1 & 0 & 0 & 0.0044\\ 0 & 1 & 0 & 0.1244\\ 0 & 0 & 1 & -0.0384\\ 0 & 0 & 0 & 1 \end{pmatrix}. \]

As in Task 2, translate the “L_Elbow” bone by \(0.2\) units up in the Y-direction. Once relative translations are used, the output should be as follows:

Export these meshes as frame000.obj, frame001.obj, and frame002.obj in the output3 folder.

Task 4: Relative Rotations

Now we switch to rotations. Each bone applies a relative translation with respect to its parent, and then a rotation. For example, the transformation for the left shoulder becomes: \[ T = T_0 \, R_0 \, T_3 \, R_3 \, T_6 \, R_6 \, T_9 \, R_9 \, T_{13} \, R_{13} \, T_{16} \, R_{16}, \] where the \(R_j\) are the rotation matrices. Since the character’s skeletal hierarchy contains branching structures, this product can be implemented efficiently with a matrix stack.

Since the mocap data in the next task will use quaternions, we’ll use quaternions for this task as well. Bend the shoulder by applying the quaternion 0, 0, -0.3827, 0.9239. (Here, the quaternion is ordered as \((x,y,z,w)\).) This corresponds to a rotation about the Z-axis by \(-45\) degrees. Apply this rotation with \(\beta\) set to \(\beta^{(0)}\), \(\beta^{(1)}\), and \(\beta^{(2)}\).

Here are some test numbers for \(\beta^{(0)}\) (the blue character):

Export these meshes as frame000.obj, frame001.obj, and frame002.obj in the output4 folder.

Task 5: Mocap Data

Load smpl_quaternions_mosh_cmu_7516.txt. This file has 160 frames, and each data line corresponds to a frame. The first 3 numbers are the root translations, which can be ignored for this assignment. The remaining numbers are the quaternions \((x,y,z,w)\) for the 24 bones. Each quaternion represents the relative rotation of a joint with respect to its parent joint.

Using the three sets of \(\beta\) values (\(\beta^{(0)}\), \(\beta^{(1)}\), and \(\beta^{(2)}\)), play back the animation sequentially. Since there are 160 frames, the total number of obj files generated should be 480. These files should be exported as frame000.obj through frame479.obj in the output5 folder. (Note that in the figure title, the animation index and frame index use \(1\)-indexing, whereas the obj filename uses \(0\)-indexing.)

Task 6: Combined Output

For the final task, generate the following output. (Note that in the figure title, the animation index and frame index use \(1\)-indexing, whereas the obj filename uses \(0\)-indexing.) As a reminder, to linearly interpolate between two quantities \(x_0\) and \(x_1\), use \((1-\alpha) x_0 + \alpha x_1\), where \(\alpha \in [0,1]\). In this task, linear interpolation is performed over 50 frames. On the first frame, \(\alpha\) should be \(0\), and on the 50th frame, \(\alpha\) should be \(1\).

We are now using both animations (smpl_quaternions_mosh_cmu_7516.txt and smpl_quaternions_mosh_cmu_8806.txt).

The summary is shown below.

Rubric

Total: 100 points.

What to hand in

Failing to follow these points may decrease your “general execution” score.

Generated on Mon Mar 3 14:55:51 CST 2025