Graph-Transporter: A Graph-based Learning Method for Goal-Conditioned Deformable Object Rearranging Task

Type: Article

Publication Date: 2022-10-09

Citations: 4

DOI: https://doi.org/10.1109/smc53654.2022.9945180

Abstract

Rearranging deformable objects is a long-standing challenge in robotic manipulation for the high dimensionality of configuration space and the complex dynamics of deformable objects. We present a novel framework, Graph-Transporter, for goal-conditioned deformable object rearranging tasks. To tackle the challenge of complex configuration space and dynamics, we represent the configuration space of a deformable object with a graph structure and the graph features are encoded by a graph convolution network. Our framework adopts an architecture based on Fully Convolutional Network (FCN) to output pixel-wise pick-and-place actions from only visual input. Extensive experiments have been conducted to validate the effectiveness of the graph representation of deformable object configuration. The experimental results also demonstrate that our framework is effective and general in handling goal-conditioned deformable object rearranging tasks.

Locations

  • arXiv (Cornell University) - View - PDF
  • 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC) - View
  • DataCite API - View

Similar Works

Action Title Year Authors
+ PDF Chat Learning visual-based deformable object rearrangement with local graph neural networks 2023 Yuhong Deng
Xueqian Wang
Lipeng Chen
+ Learning visual-based deformable object rearrangement with local graph neural networks 2023 Yuhong Deng
Xueqian Wang
Lipeng Chen
+ PDF Chat Deep Reinforcement Learning Based on Local GNN for Goal-Conditioned Deformable Object Rearranging 2022 Yuhong Deng
Chongkun Xia
Xueqian Wang
Lipeng Chen
+ Learning Language-Conditioned Deformable Object Manipulation with Graph Dynamics 2023 Kai Mo
Yuhong Deng
Chongkun Xia
Xueqian Wang
+ PDF Chat Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks 2021 Daniel Seita
Pete Florence
Jonathan Tompson
Erwin Coumans
Vikas Sindhwani
Ken Goldberg
Andy Zeng
+ PDF Chat Adversarial Object Rearrangement in Constrained Environments with Heterogeneous Graph Neural Networks 2023 Xibai Lou
Houjian Yu
Ross Worobel
Yang Yang
Chang-Hyun Choi
+ Adversarial Object Rearrangement in Constrained Environments with Heterogeneous Graph Neural Networks 2023 Xibai Lou
Houjian Yu
Ross Worobel
Yang Yang
Chang-Hyun Choi
+ Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks 2020 Daniel Seita
Pete Florence
Jonathan Tompson
Erwin Coumans
Vikas Sindhwani
Ken Goldberg
Andy Zeng
+ PDF Chat Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects 2021 Zehang Weng
Fabian Paus
Anastasiia Varava
Hang Yin
Tamim Asfour
Danica Kragić
+ Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects 2021 Zehang Weng
Fabian Paus
Anastasiia Varava
Hang Yin
Tamim Asfour
Danica Kragić
+ Structural Concept Learning via Graph Attention for Multi-Level Rearrangement Planning 2023 Manav Kulshrestha
Ahmed H. Qureshi
+ Efficient and Interpretable Robot Manipulation with Graph Neural Networks 2021 Yixin Lin
Austin S. Wang
Akshara Rai
+ PDF Chat Planning for Multi-Object Manipulation with Graph Neural Network Relational Classifiers 2023 Yixuan Huang
Adam Conkey
Tucker Hermans
+ Planning for Multi-Object Manipulation with Graph Neural Network Relational Classifiers 2022 Yixuan Huang
Adam Conkey
Tucker Hermans
+ Learning Latent Graph Dynamics for Deformable Object Manipulation. 2021 Xiao Ma
David Hsu
Wee Sun Lee
+ PDF Chat Learning Language-Conditioned Deformable Object Manipulation with Graph Dynamics 2024 Yuhong Deng
Kai Mo
Chongkun Xia
Xueqian Wang
+ Graph learning in robotics: a survey 2023 Francesca Pistilli
Giuseppe Averta
+ PDF Chat Efficient and Interpretable Robot Manipulation With Graph Neural Networks 2022 Yixin Lin
Austin S. Wang
Eric Undersander
Akshara Rai
+ Learning Latent Graph Dynamics for Visual Manipulation of Deformable Objects 2021 Xiao Ma
David Hsu
Wee Sun Lee
+ Transporter Networks: Rearranging the Visual World for Robotic Manipulation 2020 Andy Zeng
Pete Florence
Jonathan Tompson
Stefan Welker
Jonathan Chien
Maria Attarian
Travis Armstrong
Ivan Krasin
Dan Duong
Vikas Sindhwani

Works Cited by This (16)

Action Title Year Authors
+ Unsupervised Learning of Object Keypoints for Perception and Control 2019 Tejas Kulkarni
Ankush Gupta
Catalin Ionescu
Sebastian Borgeaud
Malcolm Reynolds
Andrew Zisserman
Volodymyr Mnih
+ PDF Chat Combining self-supervised learning and imitation for vision-based rope manipulation 2017 Ashvin Nair
Dian Chen
Pulkit Agrawal
Phillip Isola
Pieter Abbeel
Jitendra Malik
Sergey Levine
+ PDF Chat TossingBot: Learning to Throw Arbitrary Objects With Residual Physics 2020 Andy Zeng
Shuran Song
Johnny Lee
Alberto Rodríguez
Thomas Funkhouser
+ Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering 2016 Michaël Defferrard
Xavier Bresson
Pierre Vandergheynst
+ Learning Robotic Manipulation through Visual Planning and Acting 2019 Angelina Wang
Thanard Kurutach
Kara Liu
Pieter Abbeel
Aviv Tamar
+ PDF Chat Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environment 2019 Yuhong Deng
Xiaofeng Guo
Yixuan Wei
Kai Lu
Bin Fang
Di Guo
Huaping Liu
Fuchun Sun
+ Learning Predictive Representations for Deformable Objects Using Contrastive Estimation 2020 Wilson Yan
Ashwin Vangipuram
Pieter Abbeel
Lerrel Pinto
+ Learning to Manipulate Deformable Objects without Demonstrations 2020 Yilin Wu
Wilson Yan
Thanard Kurutach
Lerrel Pinto
Pieter Abbeel
+ PDF Chat Manipulating deformable objects by interleaving prediction, planning, and control 2020 Dale McConachie
Andrew Dobson
Mengyao Ruan
Dmitry Berenson
+ PDF Chat SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulation 2020 Chen Wang
Shaoxiong Wang
Branden Romero
Filipe Veiga
Edward H. Adelson