Cloth manipulation is a difficult problem mainly because of the non-rigid nature of cloth, which makes a good representation of deformation essential. We present a new representation for the deformation-state of clothes. First, we propose the dGLI disk representation, based on topological indices computed for segments on the edges of the cloth mesh border that are arranged on a circular grid, similarly as done in literature with dGLI matrices. The heat-map of the dGLI disk uncovers patterns that correspond to features of the cloth state, like the corner positions and the fold locations. These patterns are consistent for different shapes, sizes and cloth positions. We then present a method to abstract these important features from the dGLI disk into a circle, calling it the Cloth StatE representation (CloSE). This representation is compact, continuous, and general for different shapes. Finally, we show the strengths of this representation in two relevant applications: semantic labeling and high- and low-level planning.
We present a few examples of the evolution of th dGLI disk with the state of the cloth. We visualize the cloth border alongside the dGLI disk and the dGLI disk difference. The dGLI disk is the proposed rearrangement of the dGLI Matrix. Notice a curve appearing on the dGLI disk when the cloth is folded. This curve is visible in isolation on the dGLI disk difference.
Below we present a few examples of the ground truth cloth border along with the predicted border from the CloSE representation. We estimate the CloSE representation from the given state of the cloth and fold the start state along the predictaed fold line. That is, reflect the border points on the folded side along the fold line. The folded border points and the fold line is predicted by the CloSE representation.
We continuously move the fold location on the CloSE representation to observe that the reconstructed cloth congiguration from the defined representation is also continuous.
By just using the CloSE representation we can naturally give semantic labels to the state of the cloth. Hover over the image to see the Semantic Label.
High-level planning: Since the CloSE representation tracks the folds, it can reason over its current state and plan the high-level intermediate states. That is, just 2 cases in case of a single fold.
Low-level planning (planning the pick and place locations): Here we predict which corners to grab ad where to place them. We pick at max 2 corners from the folded side such that the area of the trepazoid formed by the fold points with the chosen corners is maximum. The locations for placing the grabbed points can be calculated by reflecting the folded points along the fold line. We perform this operation twice, once for each fold, in the case of same semantic region fold to get the final place location.