源码tf_smpl

发表于

SMPL

预处理后的Model.pkl里的数据

参数：params:

\[\begin{array}{c|rlc} param.name & \text{Shape} & \text{含义} \\ \hline J\_regressor & (24, 6890) & 1 \\ weights & (6890, 24) & 权重 \\ posedirs & (6890, 3, 207) & \\ v\_template & (6890, 3) \\ shapedirs & (6890, 3, 10) \\ f & (13776, 3)\\ kintree\_table & (2, 24) & 各个关键点链接的关系 \\ \\ \hline \end{array}\]

类SMPLModel：

\[\begin{array}{r | l} 成员变量 & 初始化值 & 含义 \\ \hline J\_regressor & params.J\_regressor \\ weights & params.weights \\ posedirs & params.posedirs \\ v\_template & params.v\_template \\ shapedirs & params.shapedirs \\ faces & params.f \\ kintree\_table & params.kintree\_table \\ \hline parent & kintree\_table\\ \hline pose\_shape & [24, 3] \\ beta\_shape & [10] \\ trans\_shape & [3] \\ \hline pose & 0 \\ beta & 0 \\ trans & 0 \\ \hline verts & None \\ J & None \\ R & None \end{array}\]

成员方法update：

  • v_shaped = self.shapedirs.dot(self.beta) + self.v_template
    测量$\beta$如何影响body的形状。对应于论文中的公式 $\mathbf B(\overrightarrow \beta;S) = \sum_{n=1}^{\mid\overrightarrow \beta \mid}\beta_nS_n$；可以推测出shapedirs是$S_n$
  • self.J = self.J_regressor.dot(v_shaped)
    pose_cube = self.pose.reshape((-1, 1, 3))
    1. 计算关节点的位置J等于J_regressor与前一步v_shaped点乘
    2. $J(\overrightarrow \beta;\mathcal J, \overline \mathbf T, \mathcal S) = \mathcal J(\overline\mathbf T +B_S(\overrightarrow \beta; \mathcal S))$ 其中$\mathcal J$是学习到的即J_regressor
    3. 将$\beta$进行reshape,shape为(24, 1, 3)
  • self.R = self.rodrigues(pose_cube) ——————– 1
    I_cube = np.broadcast_to(
        np.expand_dims(np.eye(3), axis=0),
        (self.R.shape[0]-1, 3, 3)
    ) —————————————————————— 2
    lrotmin = (self.R[1:] - I_cube).ravel() ————– 3
    v_posed = v_shaped + self.posedirs.dot(lrotmin) — 4
    总体计算$\theta$如何影响body的形状。
    1. $\mathbf B_p(\overrightarrow \theta;\mathscr P) = \sum_{n=1}^{9K}(R_n(\overrightarrow \theta) - R_n(\overrightarrow \theta^*))P_n$
    2. 第(1)行代码计算$R_n(\overrightarrow \theta)$
    3. 第(2)行代码计算$R_n(\overrightarrow \theta^*)$
    4. 第(3)行代码计算$R_n(\overrightarrow \theta) - R_n(\overrightarrow \theta^*)$
    5. 第(4)行代码计算$\theta$对body的影响
    6. 由此可知posedirs存放的是$P_n$
  • # world transformation of each joint
    G = np.empty((self.kintree_table.shape[1], 4, 4))
    G[0] = self.with_zeros(np.hstack((self.R[0], self.J[0, :].reshape([3, 1]))))
    for i in range(1, self.kintree_table.shape[1]):
        G[i] = G[self.parent[i]].dot( # 父关节的轴角乘以关节相对旋度等于当前关节的轴角
        self.with_zeros(
          np.hstack(
            [self.R[i], ((self.J[i, :]-self.J[self.parent[i],:]).reshape([3,1]))] # 计算关节的相对位置
          )
         )
        )
    # remove the transformation due to the rest pose
    G = G - self.pack(
        np.matmul(
            G,
            np.hstack([self.J, np.zeros([24, 1])]).reshape([24, 4, 1])
          )
    )
    1. 其中with_zeros和np.hstack实现$G_k(\overrightarrow \theta, \mathbf J) = \prod_{j \in A(k)}[\begin{array}{r l} exp(\overrightarrow w_j) & j_j \ \overrightarrow 0 & 1\end{array}]$
    2. 看不懂了….
  • T = np.tensordot(self.weights, G, axes=[[1], [0]])
    ` rest_shape_h = np.hstack((v_posed, np.ones([v_posed.shape[0], 1])))<br> v = np.matmul(T, rest_shape_h.reshape([-1, 4, 1])).reshape([-1, 4])[:, :3]<br> self.verts = v + self.trans.reshape([1, 3])`
    1. 对每一个点进行变换。
    2. 实现式子$\overline t_i^{‘} = \sum_{k=1}^Kw_{k,i}G^{‘}k(\overrightarrow \theta, J(\overrightarrow \beta))(\overline t_i + b{S,i}(\overrightarrow \beta) + b_{p,i}(\overrightarrow \theta))$