CloSE: A Compact Shape- and Orientation-Agnostic Cloth State Representation

1Institut de Robòtica i Informàtica Industrial, CSIC-UPC

Abstract

Cloth manipulation is a difficult problem mainly because of the non-rigid nature of cloth, which makes a good representation of deformation essential. We present a new representation for the deformation-state of clothes. First, we propose the dGLI disk representation, based on topological indices computed for segments on the edges of the cloth mesh border that are arranged on a circular grid, similarly as done in literature with dGLI matrices. The heat-map of the dGLI disk uncovers patterns that correspond to features of the cloth state, like the corner positions and the fold locations. These patterns are consistent for different shapes, sizes and cloth positions. We then present a method to abstract these important features from the dGLI disk into a circle, calling it the Cloth StatE representation (CloSE). This representation is compact, continuous, and general for different shapes. Finally, we show the strengths of this representation in two relevant applications: semantic labeling and high- and low-level planning.

Video

Contributions:

  • We propose a new dGLI representation arranged in a disk instead of a matrix, the dGLI Disk. This representation unravels a hidden structure, and helps us detect folds on clothes of any shape.
  • From the dGLI disk, we abstract out a very compact representation, CloSE, that generalizes to any shape and pose of the cloth.
  • We demonstrate the capabalities of our CloSE representation on two applications:
    1. defining an automatic semantic description of the cloth state
    2. planning manipulation sequences

The dGLI disk

We present a few examples of the evolution of th dGLI disk with the state of the cloth. We visualize the cloth border alongside the dGLI disk and the dGLI disk difference. The dGLI disk is the proposed rearrangement of the dGLI Matrix. Notice a curve appearing on the dGLI disk when the cloth is folded. This curve is visible in isolation on the dGLI disk difference.

Evaluations

Ground Truth vs CloSE reconstruction

Below we present a few examples of the ground truth cloth border along with the predicted border from the CloSE representation. We estimate the CloSE representation from the given state of the cloth and fold the start state along the predictaed fold line. That is, reflect the border points on the folded side along the fold line. The folded border points and the fold line is predicted by the CloSE representation.

CloSE Representation in continuous

We continuously move the fold location on the CloSE representation to observe that the reconstructed cloth congiguration from the defined representation is also continuous.

Applications

Semantic Labelling

By just using the CloSE representation we can naturally give semantic labels to the state of the cloth. Hover over the image to see the Semantic Label.

Planning with CloSE representation

High-level planning: Since the CloSE representation tracks the folds, it can reason over its current state and plan the high-level intermediate states. That is, just 2 cases in case of a single fold.

  • Case 1: Start and the Goal are in the same semantic region
  • Case 2: Start and the Goal are in different semantic regions
In case 1, we directly move to the goal configuration, and in case 2, we first unfold the cloth and then fold along the desired fold line.

Low-level planning (planning the pick and place locations): Here we predict which corners to grab ad where to place them. We pick at max 2 corners from the folded side such that the area of the trepazoid formed by the fold points with the chosen corners is maximum. The locations for placing the grabbed points can be calculated by reflecting the folded points along the fold line. We perform this operation twice, once for each fold, in the case of same semantic region fold to get the final place location.