Shape Analysis and Registration of People Using Dynamic Data – SHARED

Acquisition and analysis of the subjects’ movement data: We use optical motion capture system to acquire a set of 'dynamic mesh', i.e. mesh sequence with fixed topology. We then analyze each of the animation sequence to observe the changes in stretching and bending for each graphical entity (i.e. vertex) over time. After the temporal integration of these changes, which we encode in the form of time-varying dynamic descriptor, we identify local maxima with significant deformation behavior and label them as dynamic characteristic points.

* Tempo-spatial segmentation of movement data: We first temporally segment the sequence of mesh frames based on deformation behavior, in a way that similarities within temporal segment is maximized. Next, for each temporal segment, we compute spatial segmentation. The algorithm labels each triangle in a way that triangle affinities within spatial segment is approximately maximized.

* Registration: The initial, rough matching uses the characteristic points to guide an optimization problem, with a small number of degree of freedom of global rigid transformations. Then, we proceed with fine matching, where affine transformation of each vertex in the source object comprises the degrees of freedom in the optimization problem. The matching is realized by finding the set of transformations so as to maximize the similarity of dynamic descriptors between the matching pairs and to minimize the distortion energy of the source object. One important problem that remains to be resolved is the reduction of computation time, which makes our current implementation impractical at the moment.

Motion capture data on facial movements: The mocal data we have captured are expensive and rare, requiring not only high-cost mocap device and experts but also multiple subjects.
Automatic estimation of Dynamic Skin Tension Lines in Vivo: Defined as the lines of maximal tension, skin tension lines often provide guidelines for surgical incisions because incisions made in the direction of these lines would result in functionally and esthetically pleasing scars. While there exist many comparable lines of static tension, little has been explored on dynamic tension lines. Our method is general enough to be applied to arbitrary postures and individuals, less invasive, and efficient. We demonstrate the dynamic tension lines on various subjects, around the knee and shoulder regions.
* Landmark transfer: We have developed an efficient algorithm for the landmark transfer on 3D meshes that are approximately isometric. Given one or more custom landmarks placed by the user on a source mesh, our method efficiently computes corresponding landmarks on a family of target meshes. The technique is useful when a user is interested in characterization and reuse of application-specific landmarks on meshes of similar shape.
* Registration using dynamic data: In our recent work on registration, we are developing a new registration technique that uses the kinematic properties of the deforming surfaces so as to put similar deformation behaviors in correspondence.
* Tempo-spatial segmentation of movement data: We automatically compute tempo-spatial segmentation of mesh sequences so that within-segment affinity is maximized. It is one of the first methods dealing with both spatial and temporal segmentation simultaneously.

In the coming year, we plan to focus on the aforementioned problem we encountered in the full registration. We plan to speed up the computation time by (1) improving the segmentation so as to obtain consistent spatial segmentation of the source and target dynamic meshes, and (2) by carrying out registration in a per-segment basis. This will divide the current optimization problem of huge size into several number of smaller ones, thus drastically improving the computation time.
When this computation time issue will be resolved, the expected main contributions of our work is a new registration technique that makes use of not only geometric features but also dynamic features, adding a new dimension to existing registration techniques, with a capability of reliable correspondence.

So far we have published one article at the international journal of CAD (Computer-Aided Design). The impact factor of this journal is 1.234. A value of approximately 1.0 or higher indicates a good visibility of the journal. The article will also be presented at an international conference «ACM Symposium on Solid and Physical Modeling«, which will take place at Dijon in October.
- Seo H., Kim S., Cordier F., Choi J. and Hong K., Estimating Dynamic Skin Tension Lines in Vivo using 3D Scans, Computer-Aided Design, (Special issue of ACM Symposium on Solid and Physical Modeling 2012, October 29-31, Dijon, France), Elsevier.

Shape registration and analysis of people's surface dataset has become a new mainstream, gradually replacing conventional methods based on 2-dimensional images. Across a variety of disciplines ranging from anthropometry, computer aided design (CAD) computer graphics, and psychology, adopting 3D laser scanners for surface shape capture and building statistical models from a set of registered surface data is now widely accepted. While there is a large amount of research done on the static datasets with a proliferation of algorithms and a solid theoretical background, this does not seem to be the case for dynamic, time-varying datasets, due to the limited accessibility to the dynamic surface. In most of the shape capture sessions, the person is required to remain motionless during the scanning time. Naturally, current registration techniques (and therefore shape analysis techniques) handle the geometric features of static dataset, and the dynamic behavior of people's skin relatively remain unsaid. This is unfortunate, since dynamic features cannot be captured solely by using geometric features when the target subjects undergo deformation. Although the use of geometric feature based on anatomical knowledge is still a golden standard, it is quite obvious that it may generate results with limited capability of reliable correspondence computation, because some commonly observed subjects like human body are highly mobile and drastically change not only its spatial arrangement but also geometric features over time.
Taking a step beyond the existing methods that use static shape information for shape analysis, project SHARED seeks to investigate novel shape analysis method that exploits large redundancy of information from dynamic or movement data. The main interest of the proposed approach is (1) to acquire and preprocess the subjects’ movement data so as to characterize anatomical or functional landmarks, and (2) to devise a registration technique that makes use of this rich set of information to guarantee reliable correspondence. Appreciably, with the recent advances in imaging technologies we now have growing accessibility to capture the shape and motion of human skin from optical motion capture systems, and of organs from medical imaging devices. (3) Further, we will investigate statistical analysis of deforming shapes, tightly coupling the shape identity and shape change due to movement. A statistical atlas spanning over the variations of shape identity and shape deformation will be constructed, which will be used to (4) revise the registration module with great stability and robustness.