Engineering › Computational Mechanics

3D Shape Modeling and Analysis

Works Count: 33444

Wikipedia

Description

This cluster of papers focuses on the analysis, reconstruction, and representation of three-dimensional shapes using deep learning techniques, point clouds, and mesh processing. It covers topics such as 3D classification, segmentation, object recognition, shape reconstruction from single images, surface parameterization, mesh deformation, and texture mapping.

Keywords

Deep Learning; Point Clouds; 3D Reconstruction; Mesh Segmentation; Shape Representation; Neural Radiance Fields; Surface Parameterization; Mesh Deformation; Texture Mapping; Point Set Surfaces

Discrete Differential-Geometry Operators for Triangulated 2-Manifolds

2003-01-01

Mark Meyer , Mathieu Desbrun , Peter Schröder +1 more

View PDF Chat

SMPL

2015-10-26

Matthew Loper , Naureen Mahmood , Javier Romero +2 more

We present a learned model of human body shape and pose-dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person … We present a learned model of human body shape and pose-dependent shape variation that is more accurate than previous models and is compatible with existing graphics pipelines. Our Skinned Multi-Person Linear model (SMPL) is a skinned vertex-based model that accurately represents a wide variety of body shapes in natural human poses. The parameters of the model are learned from data including the rest pose template, blend weights, pose-dependent blend shapes, identity-dependent blend shapes, and a regressor from vertices to joint locations. Unlike previous models, the pose-dependent blend shapes are a linear function of the elements of the pose rotation matrices. This simple formulation enables training the entire model from a relatively large number of aligned 3D meshes of different people in different poses. We quantitatively evaluate variants of SMPL using linear or dual-quaternion blend skinning and show that both are more accurate than a Blend-SCAPE model trained on the same data. We also extend SMPL to realistically model dynamic soft-tissue deformations. Because it is based on blend skinning, SMPL is compatible with existing rendering engines and we make it available for research purposes.

Reverse engineering of geometric models—an introduction

1997-04-01

Tamás Várady , Ralph R. Martin , Jordan J. Cox

Surface shape and curvature scales

1992-10-01

Jan J. Koenderink , Andrea J. van Doorn

Poisson surface reconstruction

2006-06-26

Michael Kazhdan , Matthew Bolitho , Hugues Hoppe

We show that surface reconstruction from oriented points can be cast as a spatial Poisson problem. This Poisson formulation considers all the points at once, without resorting to heuristic spatial … We show that surface reconstruction from oriented points can be cast as a spatial Poisson problem. This Poisson formulation considers all the points at once, without resorting to heuristic spatial partitioning or blending, and is therefore highly resilient to data noise. Unlike radial basis function schemes, our Poisson approach allows a hierarchy of locally supported basis functions, and therefore the solution reduces to a well conditioned sparse linear system. We describe a spatially adaptive multiscale algorithm whose time and space complexities are proportional to the size of the reconstructed model. Experimenting with publicly available scan data, we demonstrate reconstruction of surfaces with greater detail than previously achievable.

On Visual Similarity Based 3D Model Retrieval

2003-09-01

Ding‐Yun Chen , Xu-hong Tian , Y. SHEN +1 more

Abstract A large number of 3D models are created and available on the Web, since more and more 3D modelling anddigitizing tools are developed for ever increasing applications. The techniques … Abstract A large number of 3D models are created and available on the Web, since more and more 3D modelling anddigitizing tools are developed for ever increasing applications. The techniques for content‐based 3D model retrievalthen become necessary. In this paper, a visual similarity‐based 3D model retrieval system is proposed.This approach measures the similarity among 3D models by visual similarity, and the main idea is that if two 3Dmodels are similar, they also look similar from all viewing angles. Therefore, one hundred orthogonal projectionsof an object, excluding symmetry, are encoded both by Zernike moments and Fourier descriptors as features forlater retrieval. The visual similarity‐based approach is robust against similarity transformation, noise, model degeneracyetc., and provides 42%, 94% and 25% better performance (precision‐recall evaluation diagram) thanthree other competing approaches: (1) the spherical harmonics approach developed by Funkhouser et al., (2) theMPEG‐7 Shape 3D descriptors, and (3) the MPEG‐7 Multiple View Descriptor. The proposed system is on the Webfor practical trial use ( http://3d.csie.ntu.edu.tw ), and the database contains more than 10,000 publicly available3D models collected from WWW pages. Furthermore, a user friendly interface is provided to retrieve 3D modelsby drawing 2D shapes. The retrieval is fast enough on a server with Pentium IV 2.4 GHz CPU, and it takes about2 seconds and 0.1 seconds for querying directly by a 3D model and by hand drawn 2D shapes, respectively. Categories and Subject Descriptors (according to ACM CCS): H.3.1 [Information Storage and Retrieval]: Indexing Methods

View PDF Chat

Mean value coordinates

2003-03-01

Michael S. Floater

A survey of content based 3D shape retrieval methods

2007-12-07

Johan W. H. Tangelder , Remco C. Veltkamp

View PDF Chat

A search engine for 3D models

2003-01-01

Thomas Funkhouser , Patrick Min , Michael Kazhdan +4 more

As the number of 3D models available on the Web grows, there is an increasing need for a search engine to help people find them. Unfortunately, traditional text-based search techniques … As the number of 3D models available on the Web grows, there is an increasing need for a search engine to help people find them. Unfortunately, traditional text-based search techniques are not always effective for 3D data. In this article, we investigate new shape-based search methods. The key challenges are to develop query methods simple enough for novice users and matching algorithms robust enough to work for arbitrary polygonal models. We present a Web-based search engine system that supports queries based on 3D sketches, 2D sketches, 3D models, and/or text keywords. For the shape-based queries, we have developed a new matching algorithm that uses spherical harmonics to compute discriminating similarity measures without requiring repair of model degeneracies or alignment of orientations. It provides 46 to 245% better performance than related shape-matching methods during precision--recall experiments, and it is fast enough to return query results from a repository of 20,000 models in under a second. The net result is a growing interactive index of 3D models available on the Web (i.e., a Google for 3D models).

Using spin images for efficient object recognition in cluttered 3D scenes

1999-05-01

Andrew Johnson , Martial Hebert

We present a 3D shape-based object recognition system for simultaneous recognition of multiple objects in scenes containing clutter and occlusion. Recognition is based on matching surfaces by matching points using … We present a 3D shape-based object recognition system for simultaneous recognition of multiple objects in scenes containing clutter and occlusion. Recognition is based on matching surfaces by matching points using the spin image representation. The spin image is a data level shape descriptor that is used to match surfaces represented as surface meshes. We present a compression scheme for spin images that results in efficient multiple object recognition which we verify with results showing the simultaneous recognition of multiple objects from a library of 20 models. Furthermore, we demonstrate the robust performance of recognition in the presence of clutter and occlusion through analysis of recognition trials on 100 scenes.

A Concise and Provably Informative Multi‐Scale Signature Based on Heat Diffusion

2009-07-01

Jian Sun , Maks Ovsjanikov , Leonidas Guibas

Abstract We propose a novel point signature based on the properties of the heat diffusion process on a shape. Our signature, called the Heat Kernel Signature (or HKS), is obtained … Abstract We propose a novel point signature based on the properties of the heat diffusion process on a shape. Our signature, called the Heat Kernel Signature (or HKS), is obtained by restricting the well‐known heat kernel to the temporal domain. Remarkably we show that under certain mild assumptions, HKS captures all of the information contained in the heat kernel, and characterizes the shape up to isometry. This means that the restriction to the temporal domain, on the one hand, makes HKS much more concise and easily commensurable, while on the other hand, it preserves all of the information about the intrinsic geometry of the shape. In addition, HKS inherits many useful properties from the heat kernel, which means, in particular, that it is stable under perturbations of the shape. Our signature also provides a natural and efficiently computable multi‐scale way to capture information about neighborhoods of a given point, which can be extremely useful in many applications. To demonstrate the practical relevance of our signature, we present several methods for non‐rigid multi‐scale matching based on the HKS and use it to detect repeated structure within the same shape and across a collection of shapes.

View PDF Chat

The ball-pivoting algorithm for surface reconstruction

1999-10-01

Fausto Bernardini , Joshua Mittleman , Holly Rushmeier +2 more

The Ball-Pivoting Algorithm (BPA) computes a triangle mesh interpolating a given point cloud. Typically, the points are surface samples acquired with multiple range scans of an object. The principle of … The Ball-Pivoting Algorithm (BPA) computes a triangle mesh interpolating a given point cloud. Typically, the points are surface samples acquired with multiple range scans of an object. The principle of the BPA is very simple: Three points form a triangle if a ball of a user-specified radius p touches them without containing any other point. Starting with a seed triangle, the ball pivots around an edge (i.e., it revolves around the edge while keeping in contact with the edge's endpoints) until it touches another point, forming another triangle. The process continues until all reachable edges have been tried, and then starts from another seed triangle, until all points have been considered. The process can then be repeated with a ball of larger radius to handle uneven sampling densities. We applied the BPA to datasets of millions of points representing actual scans of complex 3D objects. The relatively small amount of memory required by the BPA, its time efficiency, and the quality of the results obtained compare favorably with existing techniques.

Presentation and validation of the Radboud Faces Database

2010-04-08

Oliver Langner , Ron Dotsch , Gijsbert Bijlstra +3 more

Many research fields concerned with the processing of information contained in human faces would benefit from face stimulus sets in which specific facial characteristics are systematically varied while other important … Many research fields concerned with the processing of information contained in human faces would benefit from face stimulus sets in which specific facial characteristics are systematically varied while other important picture characteristics are kept constant. Specifically, a face database in which displayed expressions, gaze direction, and head orientation are parametrically varied in a complete factorial design would be highly useful in many research domains. Furthermore, these stimuli should be standardised in several important, technical aspects. The present article presents the freely available Radboud Faces Database offering such a stimulus set, containing both Caucasian adult and children images. This face database is described both procedurally and in terms of content, and a validation study concerning its most important characteristics is presented. In the validation study, all frontal images were rated with respect to the shown facial expression, intensity of expression, clarity of expression, genuineness of expression, attractiveness, and valence. The results show very high recognition of the intended facial expressions.

View PDF Chat

ShapeNet: An Information-Rich 3D Model Repository

2023-11-09

Anne Lynn S. Chang , Thomas Funkhouser , Leonidas Guibas +7 more

We present ShapeNet: a richly-annotated, large-scale repository of shapes represented by 3D CAD models of objects. ShapeNet contains 3D models from a multitude of semantic categories and organizes them under … We present ShapeNet: a richly-annotated, large-scale repository of shapes represented by 3D CAD models of objects. ShapeNet contains 3D models from a multitude of semantic categories and organizes them under the WordNet taxonomy. It is a collection of datasets providing many semantic annotations for each 3D model such as consistent rigid alignments, parts and bilateral symmetry planes, physical sizes, keywords, as well as other planned annotations. Annotations are made available through a public web-based interface to enable data visualization of object attributes, promote data-driven geometric analysis, and provide a large-scale quantitative benchmark for research in computer graphics and vision. At the time of this technical report, ShapeNet has indexed more than 3,000,000 models, 220,000 models out of which are classified into 3,135 categories (WordNet synsets). In this report we describe the ShapeNet effort as a whole, provide details for all currently available datasets, and summarize future plans.

Least squares conformal maps for automatic texture atlas generation

2002-07-01

Bruno Lévy , Sylvain Petitjean , Nicolas Ray +1 more

A Texture Atlas is an efficient color representation for 3D Paint Systems. The model to be textured is decomposed into charts homeomorphic to discs, each chart is parameterized, and the … A Texture Atlas is an efficient color representation for 3D Paint Systems. The model to be textured is decomposed into charts homeomorphic to discs, each chart is parameterized, and the unfolded charts are packed in texture space. Existing texture atlas methods for triangulated surfaces suffer from several limitations, requiring them to generate a large number of small charts with simple borders. The discontinuities between the charts cause artifacts, and make it difficult to paint large areas with regular patterns.In this paper, our main contribution is a new quasi-conformal parameterization method, based on a least-squares approximation of the Cauchy-Riemann equations. The so-defined objective function minimizes angle deformations, and we prove the following properties: the minimum is unique, independent of a similarity in texture space, independent of the resolution of the mesh and cannot generate triangle flips. The function is numerically well behaved and can therefore be very efficiently minimized. Our approach is robust, and can parameterize large charts with complex borders.We also introduce segmentation methods to decompose the model into charts with natural shapes, and a new packing algorithm to gather them in texture space. We demonstrate our approach applied to paint both scanned and modeled data sets.

View PDF Chat

A scalable active framework for region annotation in 3D shape collections

2016-11-11

Li Yi , Vladimir G. Kim , Duygu Ceylan +7 more

Large repositories of 3D shapes provide valuable input for data-driven analysis and modeling tools. They are especially powerful once annotated with semantic information such as salient regions and functional parts. … Large repositories of 3D shapes provide valuable input for data-driven analysis and modeling tools. They are especially powerful once annotated with semantic information such as salient regions and functional parts. We propose a novel active learning method capable of enriching massive geometric datasets with accurate semantic region annotations. Given a shape collection and a user-specified region label our goal is to correctly demarcate the corresponding regions with minimal manual work. Our active framework achieves this goal by cycling between manually annotating the regions, automatically propagating these annotations across the rest of the shapes, manually verifying both human and automatic annotations, and learning from the verification results to improve the automatic propagation algorithm. We use a unified utility function that explicitly models the time cost of human input across all steps of our method. This allows us to jointly optimize for the set of models to annotate and for the set of models to verify based on the predicted impact of these actions on the human efficiency. We demonstrate that incorporating verification of all produced labelings within this unified objective improves both accuracy and efficiency of the active learning procedure. We automatically propagate human labels across a dynamic shape network using a conditional random field (CRF) framework, taking advantage of global shape-to-shape similarities, local feature similarities, and point-to-point correspondences. By combining these diverse cues we achieve higher accuracy than existing alternatives. We validate our framework on existing benchmarks demonstrating it to be significantly more efficient at using human input compared to previous techniques. We further validate its efficiency and robustness by annotating a massive shape dataset, labeling over 93,000 shape parts, across multiple model classes, and providing a labeled part collection more than one order of magnitude larger than existing ones.

Geometric Deep Learning: Going beyond Euclidean data

2017-07-01

Michael M. Bronstein , Joan Bruna , Yann LeCun +2 more

Many scientific fields study data with an underlying structure that is a non-Euclidean space. Some examples include social networks in computational social sciences, sensor networks in communications, functional networks in … Many scientific fields study data with an underlying structure that is a non-Euclidean space. Some examples include social networks in computational social sciences, sensor networks in communications, functional networks in brain imaging, regulatory networks in genetics, and meshed surfaces in computer graphics. In many applications, such geometric data are large and complex (in the case of social networks, on the scale of billions), and are natural targets for machine learning techniques. In particular, we would like to use deep neural networks, which have recently proven to be powerful tools for a broad range of problems from computer vision, natural language processing, and audio analysis. However, these tools have been most successful on data with an underlying Euclidean or grid-like structure, and in cases where the invariances of these structures are built into networks used to model them. Geometric deep learning is an umbrella term for emerging techniques attempting to generalize (structured) deep neural models to non-Euclidean domains such as graphs and manifolds. The purpose of this paper is to overview different examples of geometric deep learning problems and present available solutions, key difficulties, applications, and future research directions in this nascent field.

View PDF Chat

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

2017-07-01

Raffaelli Charles , Hao Su , Kaichun Mo +1 more

Point cloud is an important type of geometric data structure. Due to its irregular format, most researchers transform such data to regular 3D voxel grids or collections of images. This, … Point cloud is an important type of geometric data structure. Due to its irregular format, most researchers transform such data to regular 3D voxel grids or collections of images. This, however, renders data unnecessarily voluminous and causes issues. In this paper, we design a novel type of neural network that directly consumes point clouds, which well respects the permutation invariance of points in the input. Our network, named PointNet, provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing. Though simple, PointNet is highly efficient and effective. Empirically, it shows strong performance on par or even better than state of the art. Theoretically, we provide analysis towards understanding of what the network has learnt and why the network is robust with respect to input perturbation and corruption.

View PDF Chat

FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation

2018-06-01

Yaoqing Yang , Chen Feng , Yiru Shen +1 more

Recent deep networks that directly handle points in a point set, e.g., PointNet, have been state-of-the-art for supervised learning tasks on point clouds such as classification and segmentation. In this … Recent deep networks that directly handle points in a point set, e.g., PointNet, have been state-of-the-art for supervised learning tasks on point clouds such as classification and segmentation. In this work, a novel end-to-end deep auto-encoder is proposed to address unsupervised learning challenges on point clouds. On the encoder side, a graph-based enhancement is enforced to promote local structures on top of PointNet. Then, a novel folding-based decoder deforms a canonical 2D grid onto the underlying 3D object surface of a point cloud, achieving low reconstruction errors even for objects with delicate structures. The proposed decoder only uses about 7% parameters of a decoder with fully-connected neural networks, yet leads to a more discriminative representation that achieves higher linear SVM classification accuracy than the benchmark. In addition, the proposed decoder structure is shown, in theory, to be a generic architecture that is able to reconstruct an arbitrary point cloud from a 2D grid. Our code is available at http://www.merl.com/research/license#FoldingNet.

View PDF Chat

PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud

2019-06-01

Shaoshuai Shi , Xiaogang Wang , Hongsheng Li

In this paper, we propose PointRCNN for 3D object detection from raw point cloud. The whole framework is composed of two stages: stage-1 for the bottom-up 3D proposal generation and … In this paper, we propose PointRCNN for 3D object detection from raw point cloud. The whole framework is composed of two stages: stage-1 for the bottom-up 3D proposal generation and stage-2 for refining proposals in the canonical coordinates to obtain the final detection results. Instead of generating proposals from RGB image or projecting point cloud to bird's view or voxels as previous methods do, our stage-1 sub-network directly generates a small number of high-quality 3D proposals from point cloud in a bottom-up manner via segmenting the point cloud of the whole scene into foreground points and background. The stage-2 sub-network transforms the pooled points of each proposal to canonical coordinates to learn better local spatial features, which is combined with global semantic features of each point learned in stage-1 for accurate box refinement and confidence prediction. Extensive experiments on the 3D detection benchmark of KITTI dataset show that our proposed architecture outperforms state-of-the-art methods with remarkable margins by using only point cloud as input. The code is available at https://github.com/sshaoshuai/PointRCNN.

View PDF Chat

On the Continuity of Rotation Representations in Neural Networks

2019-06-01

Yi Zhou , Connelly Barnes , Jingwan Lu +2 more

In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. In this … In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. In this paper, we advance a definition of a continuous representation, which can be helpful for training deep neural networks. We relate this to topological concepts such as homeomorphism and embedding. We then investigate what are continuous and discontinuous representations for 2D, 3D, and n-dimensional rotations. We demonstrate that for 3D rotations, all representations are discontinuous in the real Euclidean spaces of four or fewer dimensions. Thus, widely used representations such as quaternions and Euler angles are discontinuous and difficult for neural networks to learn. We show that the 3D rotations have continuous representations in 5D and 6D, which are more suitable for learning. We also present continuous representations for the general case of the n-dimensional rotation group SO(n). While our main focus is on rotations, we also show that our constructions apply to other groups such as the orthogonal group and similarity transforms. We finally present empirical results, which show that our continuous rotation representations outperform discontinuous ones for several practical problems in graphics and vision, including a simple autoencoder sanity test, a rotation estimator for 3D point clouds, and an inverse kinematics solver for 3D human poses.

View PDF Chat

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

2016-12-02

Charles R. Qi , Hao Su , Kaichun Mo +1 more

Point cloud is an important type of geometric data structure. Due to its irregular format, most researchers transform such data to regular 3D voxel grids or collections of images. This, … Point cloud is an important type of geometric data structure. Due to its irregular format, most researchers transform such data to regular 3D voxel grids or collections of images. This, however, renders data unnecessarily voluminous and causes issues. In this paper, we design a novel type of neural network that directly consumes point clouds and well respects the permutation invariance of points in the input. Our network, named PointNet, provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing. Though simple, PointNet is highly efficient and effective. Empirically, it shows strong performance on par or even better than state of the art. Theoretically, we provide analysis towards understanding of what the network has learnt and why the network is robust with respect to input perturbation and corruption.

Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images

2018-01-01

Nanyang Wang , Yinda Zhang , Zhuwen Li +3 more

View PDF Chat

Learning Implicit Fields for Generative Shape Modeling

2019-06-01

Zhiqin Chen , Hao Zhang

We advocate the use of implicit fields for learning generative models of shapes and introduce an implicit field decoder, called IM-NET, for shape generation, aimed at improving the visual quality … We advocate the use of implicit fields for learning generative models of shapes and introduce an implicit field decoder, called IM-NET, for shape generation, aimed at improving the visual quality of the generated shapes. An implicit field assigns a value to each point in 3D space, so that a shape can be extracted as an iso-surface. IM-NET is trained to perform this assignment by means of a binary classifier. Specifically, it takes a point coordinate, along with a feature vector encoding a shape, and outputs a value which indicates whether the point is outside the shape or not. By replacing conventional decoders by our implicit decoder for representation learning (via IM-AE) and shape generation (via IM-GAN), we demonstrate superior results for tasks such as generative shape modeling, interpolation, and single-view 3D reconstruction, particularly in terms of visual quality. Code and supplementary material are available at https://github.com/czq142857/implicit-decoder.

View PDF Chat

Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models

2017-10-01

Roman Klokov , Victor Lempitsky

We present a new deep learning architecture (called Kdnetwork) that is designed for 3D model recognition tasks and works with unstructured point clouds. The new architecture performs multiplicative transformations and … We present a new deep learning architecture (called Kdnetwork) that is designed for 3D model recognition tasks and works with unstructured point clouds. The new architecture performs multiplicative transformations and shares parameters of these transformations according to the subdivisions of the point clouds imposed onto them by kdtrees. Unlike the currently dominant convolutional architectures that usually require rasterization on uniform twodimensional or three-dimensional grids, Kd-networks do not rely on such grids in any way and therefore avoid poor scaling behavior. In a series of experiments with popular shape recognition benchmarks, Kd-networks demonstrate competitive performance in a number of shape recognition tasks such as shape classification, shape retrieval and shape part segmentation.

View PDF Chat

3D Semantic Segmentation with Submanifold Sparse Convolutional Networks

2018-06-01

Benjamin Graham , Martin Engelcke , Laurens van der Maaten

Convolutional networks are the de-facto standard for analyzing spatio-temporal data such as images, videos, and 3D shapes. Whilst some of this data is naturally dense (e.g., photos), many other data … Convolutional networks are the de-facto standard for analyzing spatio-temporal data such as images, videos, and 3D shapes. Whilst some of this data is naturally dense (e.g., photos), many other data sources are inherently sparse. Examples include 3D point clouds that were obtained using a LiDAR scanner or RGB-D camera. Standard "dense" implementations of convolutional networks are very inefficient when applied on such sparse data. We introduce new sparse convolutional operations that are designed to process spatially-sparse data more efficiently, and use them to develop spatially-sparse convolutional networks. We demonstrate the strong performance of the resulting models, called submanifold sparse convolutional networks (SS-CNs), on two tasks involving semantic segmentation of 3D point clouds. In particular, our models outperform all prior state-of-the-art on the test set of a recent semantic segmentation competition.

View PDF Chat

PointConv: Deep Convolutional Networks on 3D Point Clouds

2019-06-01

Wenxuan Wu , Zhongang Qi , Fuxin Li

Unlike images which are represented in regular dense grids, 3D point clouds are irregular and unordered, hence applying convolution on them can be difficult. In this paper, we extend the … Unlike images which are represented in regular dense grids, 3D point clouds are irregular and unordered, hence applying convolution on them can be difficult. In this paper, we extend the dynamic filter to a new convolution operation, named PointConv. PointConv can be applied on point clouds to build deep convolutional networks. We treat convolution kernels as nonlinear functions of the local coordinates of 3D points comprised of weight and density functions. With respect to a given point, the weight functions are learned with multi-layer perceptron networks and the density functions through kernel density estimation. A novel reformulation is proposed for efficiently computing the weight functions, which allowed us to dramatically scale up the network and significantly improve its performance. The learned convolution kernel can be used to compute translation-invariant and permutation-invariant convolution on any point set in the 3D space. Besides, PointConv can also be used as deconvolution operators to propagate features from a subsampled point cloud back to its original resolution. Experiments on ModelNet40, ShapeNet, and ScanNet show that deep convolutional neural networks built on PointConv are able to achieve state-of-the-art on challenging semantic segmentation benchmarks on 3D point clouds. Besides, our experiments converting CIFAR-10 into a point cloud showed that networks built on PointConv can match the performance of convolutional networks in 2D images of a similar structure.

View PDF Chat

Relation-Shape Convolutional Neural Network for Point Cloud Analysis

2019-06-01

Yongcheng Liu , Bin Fan , Shiming Xiang +1 more

Point cloud analysis is very challenging, as the shape implied in irregular points is difficult to capture. In this paper, we propose RS-CNN, namely, Relation-Shape Convolutional Neural Network, which extends … Point cloud analysis is very challenging, as the shape implied in irregular points is difficult to capture. In this paper, we propose RS-CNN, namely, Relation-Shape Convolutional Neural Network, which extends regular grid CNN to irregular configuration for point cloud analysis. The key to RS-CNN is learning from relation, i.e., the geometric topology constraint among points. Specifically, the convolutional weight for local point set is forced to learn a high-level relation expression from predefined geometric priors, between a sampled point from this point set and the others. In this way, an inductive local representation with explicit reasoning about the spatial layout of points can be obtained, which leads to much shape awareness and robustness. With this convolution as a basic operator, RS-CNN, a hierarchical architecture can be developed to achieve contextual shape-aware learning for point cloud analysis. Extensive experiments on challenging benchmarks across three tasks verify RS-CNN achieves the state of the arts.

View PDF Chat

DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation

2019-06-01

Jeong Joon Park , Pete Florence , Julian Straub +2 more

Computer graphics, 3D computer vision and robotics communities have produced multiple approaches to representing 3D geometry for rendering and reconstruction. These provide trade-offs across fidelity, efficiency and compression capabilities. In … Computer graphics, 3D computer vision and robotics communities have produced multiple approaches to representing 3D geometry for rendering and reconstruction. These provide trade-offs across fidelity, efficiency and compression capabilities. In this work, we introduce DeepSDF, a learned continuous Signed Distance Function (SDF) representation of a class of shapes that enables high quality shape representation, interpolation and completion from partial and noisy 3D input data. DeepSDF, like its classical counterpart, represents a shape's surface by a continuous volumetric field: the magnitude of a point in the field represents the distance to the surface boundary and the sign indicates whether the region is inside (-) or outside (+) of the shape, hence our representation implicitly encodes a shape's boundary as the zero-level-set of the learned function while explicitly representing the classification of space as being part of the shapes interior or not. While classical SDF's both in analytical or discretized voxel form typically represent the surface of a single shape, DeepSDF can represent an entire class of shapes. Furthermore, we show state-of-the-art performance for learned 3D shape representation and completion while reducing the model size by an order of magnitude compared with previous work.

View PDF Chat

SO-Net: Self-Organizing Network for Point Cloud Analysis

2018-06-01

Jiaxin Li , Ben M. Chen , Gim Hee Lee

This paper presents SO-Net, a permutation invariant architecture for deep learning with orderless point clouds. The SO-Net models the spatial distribution of point cloud by building a Self-Organizing Map (SOM). … This paper presents SO-Net, a permutation invariant architecture for deep learning with orderless point clouds. The SO-Net models the spatial distribution of point cloud by building a Self-Organizing Map (SOM). Based on the SOM, SO-Net performs hierarchical feature extraction on individual points and SOM nodes, and ultimately represents the input point cloud by a single feature vector. The receptive field of the network can be systematically adjusted by conducting point-to-node k nearest neighbor search. In recognition tasks such as point cloud reconstruction, classification, object part segmentation and shape retrieval, our proposed network demonstrates performance that is similar with or better than state-of-the-art approaches. In addition, the training speed is significantly faster than existing point cloud recognition networks because of the parallelizability and simplicity of the proposed architecture. Our code is available at the project website1.

View PDF Chat

Occupancy Networks: Learning 3D Reconstruction in Function Space

2019-06-01

Lars Mescheder , Michael Oechsle , Michael Niemeyer +2 more

With the advent of deep neural networks, learning-based approaches for 3D reconstruction have gained popularity. However, unlike for images, in 3D there is no canonical representation which is both computationally … With the advent of deep neural networks, learning-based approaches for 3D reconstruction have gained popularity. However, unlike for images, in 3D there is no canonical representation which is both computationally and memory efficient yet allows for representing high-resolution geometry of arbitrary topology. Many of the state-of-the-art learning-based 3D reconstruction approaches can hence only represent very coarse 3D geometry or are limited to a restricted domain. In this paper, we propose Occupancy Networks, a new representation for learning-based 3D reconstruction methods. Occupancy networks implicitly represent the 3D surface as the continuous decision boundary of a deep neural network classifier. In contrast to existing approaches, our representation encodes a description of the 3D output at infinite resolution without excessive memory footprint. We validate that our representation can efficiently encode 3D structure and can be inferred from various kinds of input. Our experiments demonstrate competitive results, both qualitatively and quantitatively, for the challenging tasks of 3D reconstruction from single images, noisy point clouds and coarse discrete voxel grids. We believe that occupancy networks will become a useful tool in a wide variety of learning-based 3D tasks.

View PDF Chat

Dynamic Graph CNN for Learning on Point Clouds

2019-10-10

Yue Wang , Yongbin Sun , Ziwei Liu +3 more

Point clouds provide a flexible geometric representation suitable for countless applications in computer graphics; they also comprise the raw output of most 3D data acquisition devices. While hand-designed features on … Point clouds provide a flexible geometric representation suitable for countless applications in computer graphics; they also comprise the raw output of most 3D data acquisition devices. While hand-designed features on point clouds have long been proposed in graphics and vision, however, the recent overwhelming success of convolutional neural networks (CNNs) for image analysis suggests the value of adapting insight from CNN to the point cloud world. Point clouds inherently lack topological information, so designing a model to recover topology can enrich the representation power of point clouds. To this end, we propose a new neural network module dubbed EdgeConv suitable for CNN-based high-level tasks on point clouds, including classification and segmentation. EdgeConv acts on graphs dynamically computed in each layer of the network. It is differentiable and can be plugged into existing architectures. Compared to existing modules operating in extrinsic space or treating each point independently, EdgeConv has several appealing properties: It incorporates local neighborhood information; it can be stacked applied to learn global shape properties; and in multi-layer systems affinity in feature space captures semantic characteristics over potentially long distances in the original embedding. We show the performance of our model on standard benchmarks, including ModelNet40, ShapeNetPart, and S3DIS.

View PDF Chat

PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization

2019-10-01

Shunsuke Saito , Zeng Huang , Ryota Natsume +3 more

We introduce Pixel-aligned Implicit Function (PIFu), an implicit representation that locally aligns pixels of 2D images with the global context of their corresponding 3D object. Using PIFu, we propose an … We introduce Pixel-aligned Implicit Function (PIFu), an implicit representation that locally aligns pixels of 2D images with the global context of their corresponding 3D object. Using PIFu, we propose an end-to-end deep learning method for digitizing highly detailed clothed humans that can infer both 3D surface and texture from a single image, and optionally, multiple input images. Highly intricate shapes, such as hairstyles, clothing, as well as their variations and deformations can be digitized in a unified way. Compared to existing representations used for 3D deep learning, PIFu produces high-resolution surfaces including largely unseen regions such as the back of a person. In particular, it is memory efficient unlike the voxel representation, can handle arbitrary topology, and the resulting surface is spatially aligned with the input image. Furthermore, while previous techniques are designed to process either a single image or multiple views, PIFu extends naturally to arbitrary number of views. We demonstrate high-resolution and robust reconstructions on real world images from the DeepFashion dataset, which contains a variety of challenging clothing types. Our method achieves state-of-the-art performance on a public benchmark and outperforms the prior work for clothed human digitization from a single image.

View PDF Chat

Deep Closest Point: Learning Representations for Point Cloud Registration

2019-10-01

Yue Wang , Justin Solomon

Point cloud registration is a key problem for computer vision applied to robotics, medical imaging, and other applications. This problem involves finding a rigid transformation from one point cloud into … Point cloud registration is a key problem for computer vision applied to robotics, medical imaging, and other applications. This problem involves finding a rigid transformation from one point cloud into another so that they align. Iterative Closest Point (ICP) and its variants provide simple and easily-implemented iterative methods for this task, but these algorithms can converge to spurious local optima. To address local optima and other difficulties in the ICP pipeline, we propose a learning-based method, titled Deep Closest Point (DCP), inspired by recent techniques in computer vision and natural language processing. Our model consists of three parts: a point cloud embedding network, an attention-based module combined with a pointer generation layer to approximate combinatorial matching, and a differentiable singular value decomposition (SVD) layer to extract the final rigid transformation. We train our model end-to-end on the ModelNet40 dataset and show in several settings that it performs better than ICP, its variants (e.g., Go-ICP, FGR), and the recently-proposed learning-based method PointNetLK. Beyond providing a state-of-the-art registration technique, we evaluate the suitability of our learned features transferred to unseen objects. We also provide preliminary analysis of our learned model to help understand whether domain-specific and/or global features facilitate rigid registration.

View PDF Chat

KPConv: Flexible and Deformable Convolution for Point Clouds

2019-10-01

Hugues Thomas , Charles R. Qi , Jean‐Emmanuel Deschaud +3 more

We present Kernel Point Convolution (KPConv), a new design of point convolution, i.e. that operates on point clouds without any intermediate representation. The convolution weights of KPConv are located in … We present Kernel Point Convolution (KPConv), a new design of point convolution, i.e. that operates on point clouds without any intermediate representation. The convolution weights of KPConv are located in Euclidean space by kernel points, and applied to the input points close to them. Its capacity to use any number of kernel points gives KPConv more flexibility than fixed grid convolutions. Furthermore, these locations are continuous in space and can be learned by the network. Therefore, KPConv can be extended to deformable convolutions that learn to adapt kernel points to local geometry. Thanks to a regular subsampling strategy, KPConv is also efficient and robust to varying densities. Whether they use deformable KPConv for complex tasks, or rigid KPconv for simpler tasks, our networks outperform state-of-the-art classification and segmentation approaches on several datasets. We also offer ablation studies and visualizations to provide understanding of what has been learned by KPConv and to validate the descriptive power of deformable KPConv.

View PDF Chat

Surface reconstruction from unorganized points

1992-07-01

Hugues Hoppe , Tony DeRose , Tom Duchamp +2 more

We describe and demonstrate an algorithm that takes as input an unorganized set of points {x l , . . . . x n } ⊂ R 3 on or … We describe and demonstrate an algorithm that takes as input an unorganized set of points {x l , . . . . x n } ⊂ R 3 on or near an unknown manifold M, and produces as output a simplicial surface that approximates M. Neither the topology, the presence of boundaries, nor the geometry of M are assumed to be known in advance - all are inferred automatically from the data. This problem naturally arises in a variety of practical situations such as range scanning an object from multiple view points, recovery of biological shapes from two-dimensional slices, and interactive surface sketching.

View PDF Chat

RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds

2020-06-01

Qingyong Hu , Bo Yang , Linhai Xie +5 more

We study the problem of efficient semantic segmentation for large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing approaches are only able … We study the problem of efficient semantic segmentation for large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing approaches are only able to be trained and operate over small-scale point clouds. In this paper, we introduce RandLA-Net, an efficient and lightweight neural architecture to directly infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Although remarkably computation and memory efficient, random sampling can discard key features by chance. To overcome this, we introduce a novel local feature aggregation module to progressively increase the receptive field for each 3D point, thereby effectively preserving geometric details. Extensive experiments show that our RandLA-Net can process 1 million points in a single pass with up to 200x faster than existing approaches. Moreover, our RandLA-Net clearly surpasses state-of-the-art approaches for semantic segmentation on two large-scale benchmarks Semantic3D and SemanticKITTI.

View PDF Chat

PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection

2020-06-01

Shaoshuai Shi , Chaoxu Guo , Li Jiang +4 more

We present a novel and high-performance 3D object detection framework, named PointVoxel-RCNN (PV-RCNN), for accurate 3D object detection from point clouds. Our proposed method deeply integrates both 3D voxel Convolutional … We present a novel and high-performance 3D object detection framework, named PointVoxel-RCNN (PV-RCNN), for accurate 3D object detection from point clouds. Our proposed method deeply integrates both 3D voxel Convolutional Neural Network (CNN) and PointNet-based set abstraction to learn more discriminative point cloud features. It takes advantages of efficient learning and high-quality proposals of the 3D voxel CNN and the flexible receptive fields of the PointNet-based networks. Specifically, the proposed framework summarizes the 3D scene with a 3D voxel CNN into a small set of keypoints via a novel voxel set abstraction module to save follow-up computations and also to encode representative scene features. Given the high-quality 3D proposals generated by the voxel CNN, the RoI-grid pooling is proposed to abstract proposal-specific features from the keypoints to the RoI-grid points via keypoint set abstraction. Compared with conventional pooling operations, the RoI-grid feature points encode much richer context information for accurately estimating object confidences and locations. Extensive experiments on both the KITTI dataset and the Waymo Open dataset show that our proposed PV-RCNN surpasses state-of-the-art 3D detection methods with remarkable margins.

View PDF Chat

Deep Learning for 3D Point Clouds: A Survey

2020-06-29

Yulan Guo , Hanyun Wang , Qingyong Hu +3 more

Point cloud learning has lately attracted increasing attention due to its wide applications in many areas, such as computer vision, autonomous driving, and robotics. As a dominating technique in AI, … Point cloud learning has lately attracted increasing attention due to its wide applications in many areas, such as computer vision, autonomous driving, and robotics. As a dominating technique in AI, deep learning has been successfully used to solve various 2D vision problems. However, deep learning on point clouds is still in its infancy due to the unique challenges faced by the processing of point clouds with deep neural networks. Recently, deep learning on point clouds has become even thriving, with numerous methods being proposed to address different problems in this area. To stimulate future research, this paper presents a comprehensive review of recent progress in deep learning methods for point clouds. It covers three major tasks, including 3D shape classification, 3D object detection and tracking, and 3D point cloud segmentation. It also presents comparative results on several publicly available datasets, together with insightful observations and inspiring future research directions.

View PDF Chat

O-CNN

2017-07-20

Peng‐Shuai Wang , Yang Liu , Yuxiao Guo +2 more

We present O-CNN, an Octree-based Convolutional Neural Network (CNN) for 3D shape analysis. Built upon the octree representation of 3D shapes, our method takes the average normal vectors of a … We present O-CNN, an Octree-based Convolutional Neural Network (CNN) for 3D shape analysis. Built upon the octree representation of 3D shapes, our method takes the average normal vectors of a 3D model sampled in the finest leaf octants as input and performs 3D CNN operations on the octants occupied by the 3D shape surface. We design a novel octree data structure to efficiently store the octant information and CNN features into the graphics memory and execute the entire O-CNN training and evaluation on the GPU. O-CNN supports various CNN structures and works for 3D shapes in different representations. By restraining the computations on the octants occupied by 3D surfaces, the memory and computational costs of the O-CNN grow quadratically as the depth of the octree increases, which makes the 3D CNN feasible for high-resolution 3D models. We compare the performance of the O-CNN with other existing 3D CNN solutions and demonstrate the efficiency and efficacy of O-CNN in three shape analysis tasks, including object classification, shape retrieval, and shape segmentation.

View PDF Chat

PCT: Point cloud transformer

2021-04-10

Meng-Hao Guo , Jun-Xiong Cai , Zheng-Ning Liu +3 more

The irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing. This paper presents a novel framework named Point Cloud Transformer(PCT) for … The irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing. This paper presents a novel framework named Point Cloud Transformer(PCT) for point cloud learning. PCT is based on Transformer, which achieves huge success in natural language processing and displays great potential in image processing. It is inherently permutation invariant for processing a sequence of points, making it well-suited for point cloud learning. To better capture local context within the point cloud, we enhance input embedding with the support of farthest point sampling and nearest neighbor search. Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification, part segmentation and normal estimation tasks.

View PDF Chat

Image Alignment and Stitching: A Tutorial

2007-01-01

Richard Szeliski

View PDF Chat

Point Transformer

2021-10-01

Hengshuang Zhao , Li Jiang , Jiaya Jia +2 more

Self-attention networks have revolutionized natural language processing and are making impressive strides in image analysis tasks such as image classification and object detection. Inspired by this success, we investigate the … Self-attention networks have revolutionized natural language processing and are making impressive strides in image analysis tasks such as image classification and object detection. Inspired by this success, we investigate the application of self-attention networks to 3D point cloud processing. We design self-attention layers for point clouds and use these to construct self-attention networks for tasks such as semantic scene segmentation, object part segmentation, and object classification. Our Point Transformer design improves upon prior work across domains and tasks. For example, on the challenging S3DIS dataset for large-scale semantic scene segmentation, the Point Transformer attains an mIoU of 70.4% on Area 5, outperforming the strongest prior model by 3.3 absolute percentage points and crossing the 70% mIoU threshold for the first time.

EfficientPEAL: Efficient prior-embedded attention learning for partially overlapping point cloud registration

2025-06-25

Junle Yu , Wenhui Zhou , Zhehao Shen +1 more | Expert Systems with Applications

Three-dimensional reconstruction image generation of traditional Chinese painting elements

2025-06-24

Qiyao Hu , Jingyu Wang , Xianlin Peng +2 more | Engineering Applications of Artificial Intelligence

PcBD: A Novel Point Cloud Processing Flow for Boundary Detecting and De-Noising

2025-06-23

Shuyu Sun , Jianqiang Huang , Shuai Zhao +1 more | Applied Sciences

In target detection tasks equipped with depth sensors, it is crucial to adopt the point cloud pretreatment process, which is directly related to the quality of the obtained three-dimensional model … In target detection tasks equipped with depth sensors, it is crucial to adopt the point cloud pretreatment process, which is directly related to the quality of the obtained three-dimensional model of the target. However, there are few methods that can be combined with common preprocessing methods to quickly process ToF camera output. In real-life experiments, the common method is to adopt multiple types of preprocessing methods and adjust parameters separately. We proposed PcBD, a method that integrates outlier removal, boundary detection, and smooth sliders. PcBD does not limit the number of input points, and can remove outliers and predict smooth projection boundaries at one time while ensuring that the total number of points remains unchanged. We also introduced Bound57, a benchmark dataset that contains point clouds with synthetic noise, outliers, and projected boundary labels. Experimental results show that PcBD performs significantly better than state-of-the-art methods in various de-noising and boundary detection tasks.

3D Magic Mirror: Clothing Reconstruction from a Single Image via a Causal Perspective

2025-06-23

Zhedong Zheng , Jiayin Zhu , Wei Ji +2 more

<title>Abstract</title> This paper aims to study a self-supervised 3D clothing reconstruction method, which recovers the geometry shape and texture of human clothing from a single image. Compared with existing methods, … <title>Abstract</title> This paper aims to study a self-supervised 3D clothing reconstruction method, which recovers the geometry shape and texture of human clothing from a single image. Compared with existing methods, we observe that three primary challenges remain:(1) 3D ground-truth meshes of clothing are usually inaccessible due to annotation difficulties and time costs; (2) Conventional template-based methods are limited to modeling non-rigid objects, e.g., handbags and dresses, which are common in fashion images;(3) The inherent ambiguity compromises the model training, such as the dilemma between a large shape with a remote camera or a small shape with a close camera. In an attempt to address the above limitations, we propose a causality-aware self-supervised learning method to adaptively reconstruct 3D non-rigid objects from 2D images without 3D annotations. In particular, to solve the inherent ambiguity among four implicit variables, i.e., camera position, shape, texture, and illumination, we introduce an explainable structural causal map (SCM) to build our model. The proposed model structure follows the spirit of the causal map, which explicitly considers the prior template in the camera estimation and shape prediction. When optimization, the causality intervention tool, i.e., two expectation-maximization loops, is deeply embedded in our algorithm to (1) disentangle four encoders and (2) facilitate the prior template. Extensive experiments on two 2D fashion benchmarks (ATR and Market-HQ) show that the proposed method could yield high-fidelity 3D reconstruction. Furthermore, we also verify the scalability of the proposed method on a fine-grained bird dataset, i.e., CUB.

View PDF Chat

LLMto3D - Generation of parametric, 3D printable objects using large language models

2025-06-23

Bat El Hizmi , Yoav Sterman , Guy Austern | International Journal of Architectural Computing

Recent advancements in Machine Learning (ML) have significantly enhanced the capability to generate 3D objects from textual descriptions, offering significant potential for design and fabrication workflows. However, these models typically … Recent advancements in Machine Learning (ML) have significantly enhanced the capability to generate 3D objects from textual descriptions, offering significant potential for design and fabrication workflows. However, these models typically fail to meet practical requirements like printability or manufacturability. In addition, current approaches often cannot accurately control the dimensions and the interrelations of elements within the generated 3D models. This presents a major challenge in utilizing ML-generated designs in real-world applications. To address this gap, we introduce a novel method for translating natural language descriptions into parametric 3D objects using Large Language Models (LLMs). Our approach employs multiple agents, each agent is LLM pre-trained for a specific task. The first agent deconstructs textual prompts into design elements and describes their geometry and spatial relations. The second agent translates the description into code using the Rhino.Geometry coding library in the Rhino3D-Grasshopper modeling environment. A final agent reassembles the models and adds parametric control interfaces, enabling customizable outputs. In this paper, we describe the method’s architecture and the training methodologies, including in-context learning and fine-tuning. The results demonstrate that the suggested method successfully generates code for variations of familiar objects. However, it still struggles with more complex designs that deviate significantly from the training data. We outline future directions for improvement, including expanding the training dataset and exploring advanced LLM models. This work is a step towards making 3D modeling accessible to a broader audience, using everyday language to simplify design processes.

WITHDRAWN: Three-dimensional sand particle shape characterization by PointConv-Transformer deep learning

2025-06-19

Junbo Xi , Junxing Zheng , Lin Gao +3 more | Research Square (Research Square)

<title>Abstract</title> The full text of this preprint has been withdrawn by the authors in order to comply with an institutional policy on preprints. Therefore, the authors do not wish this … <title>Abstract</title> The full text of this preprint has been withdrawn by the authors in order to comply with an institutional policy on preprints. Therefore, the authors do not wish this work to be cited as a reference.

DefTransNet: A Transformer-based Method for Non-Rigid Point Cloud Registration in the Simulation of Soft Tissue Deformation

2025-06-19

Sara Monji-Azad , Marvin Kinz , Siddharth kothari +5 more | Measurement Science and Technology

Abstract Soft-tissue surgeries, such as tumor resections, are complicated by tissue deformations that can obscure the accurate location and shape of tissues. By representing tissue surfaces as point clouds and … Abstract Soft-tissue surgeries, such as tumor resections, are complicated by tissue deformations that can obscure the accurate location and shape of tissues. By representing tissue surfaces as point clouds and applying non-rigid point cloud registration (PCR) methods, surgeons can better understand tissue deformations before, during, and after surgery. Existing non-rigid PCR methods, such as feature-based approaches, struggle with robustness against challenges like noise, outliers, partial data, and large deformations, making accurate point correspondence difficult. Although learning-based PCR methods, particularly Transformer-based approaches, have recently shown promise due to their attention mechanisms for capturing interactions, their robustness remains limited in challenging scenarios. In this paper, we present DefTransNet, a novel end-to-end Transformer-based architecture for non-rigid PCR. DefTransNet is designed to address the key challenges of deformable registration—including large deformations, outliers, noise, and partial data—by inputting source and target point clouds and outputting displacement vector fields. The proposed method incorporates a learnable transformation matrix to enhance robustness to affine transformations, integrates global and local geometric information, and captures long-range dependencies among points using Transformers. We validate our approach on four datasets: ModelNet, SynBench, 4DMatch, and DeformedTissue, using both synthetic and real-world data to demonstrate the generalization of our proposed method. Experimental results demonstrate that DefTransNet outperforms current state-of-the-art registration networks across various challenging conditions. Our code and data are publicly available.&#xD;

View PDF Chat

Towards automation of warehouse management: Counting boxes on pallets via 3D reconstruction from a single image

2025-06-18

Ikram Eddahmani , Thibault Napoléon , Chi-Hieu Pham +2 more | Applied Intelligence

ClotheDreamer: Text-guided garment generation with 3D gaussians

2025-06-18

Yufei Liu , Junshu Tang , Zheng Chu +3 more | Applied Intelligence

View PDF Chat

Goal-oriented 3D pattern adjustment with machine learning

2025-06-17

Mallikarjun Shastry , Fan Ye , C. De Oliveira Martins +1 more | Graphical Models

Optimal network geometry detection for weak geometry

2025-06-17

NULL AUTHOR_ID | Physical review. E

Geometry-aware multi-task learning network for outlier removal and denoising in scanned point clouds of aerospace thin-walled parts

2025-06-17

Liling Zuo , Jie Zhang , Hui Cheng +1 more | Journal of Intelligent Manufacturing

APPLYING POSE-GUIDED DEEP LEARNING FOR REAL-TIME VIRTUAL TRY-ON

2025-06-17

J. Chandrika , K Khushi , K. Ameenal Bibi +2 more | International Journal For Multidisciplinary Research

This project presents a real-time virtual try-on system that applies pose-guided deep learning techniques to accurately fit and visualize clothing on a user’s 3D body model. Leveraging OpenCV for webcam … This project presents a real-time virtual try-on system that applies pose-guided deep learning techniques to accurately fit and visualize clothing on a user’s 3D body model. Leveraging OpenCV for webcam capture, MediaPipe for real-time pose estimation, TensorFlow for size prediction, and Unity3D for interactive 3D visualization, the system enables users to virtually try on clothes with realistic fitting and 360° viewing capabilities. This seamless integration of cutting-edge computer vision and deep learning technologies enhances online shopping experiences by delivering personalized and engaging virtual try-ons.

Powerful Sample Reduction Techniques for Constructing Effective Point Cloud Object Classification Models

2025-06-16

Chih-Lung Lin , Haiwei Yang , Chi-Hung Chuang | Electronics

Due to the large volume of raw data in 3D point clouds, downsampling techniques are crucial for reducing computational load and memory usage to improve the training of 3D point … Due to the large volume of raw data in 3D point clouds, downsampling techniques are crucial for reducing computational load and memory usage to improve the training of 3D point cloud models. This paper plans to conduct research using the ModelNet40 dataset. Our proposed method is based on the PointNext architecture, an improved version of PointNet++ that significantly enhances performance through optimized training strategies and adjusted receptive fields. During the model training process, we employ the farthest point sampling method for downsampling. Specifically, we use an improved attention-based point cloud edge sampling (APES) method for downsampling, where we compute the density of each point and set the size of the neighbor K value to effectively retain feature points during downsampling. Our improved method captures edge points more effectively than the original APES method. By adjusting the architecture, our method, combined with the farthest point sampling method, not only reduced the average training time by nearly 15% compared to PointNext-s, but also improved accuracy from 93.11% to 93.57%.

Digital restoration of traditional huizhou clothing based on 3D modeling and virtual fitting technology

2025-06-16

Bing Hu , Shikui Wang | Journal of the Textile Institute

TCFNet: Bidirectional face-bone transformation via a Transformer-based coarse-to-fine point movement network

2025-06-16

Runshi Zhang , Bimeng Jie , Yang He +1 more | Medical Image Analysis

Dtm: Density embeded transformer mamba hybird network for point cloud analysis

2025-06-14

Yaojie Chen , Chengyu Deng | The Visual Computer

$$E^{3}DGE$$: Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversion

2025-06-14

Yushi Lan , Xuyi Meng , Shuai Yang +2 more | International Journal of Computer Vision

NESI: Neural Explicit-Shape-Intersection-Based Geometry Representation

2025-06-14

Congyi Zhang , Yang Jin-fan , Eric Hedlin +5 more | ACM Transactions on Graphics

Compressed representations of 3D shapes that are compact, accurate, and can be processed efficiently directly in compressed form, are extremely useful for digital media applications. Recent approaches in this space … Compressed representations of 3D shapes that are compact, accurate, and can be processed efficiently directly in compressed form, are extremely useful for digital media applications. Recent approaches in this space focus on learned implicit or parametric representations. While implicits are well suited for tasks such as in-out queries, they lack natural 2D parameterization, complicating tasks such as texture or normal mapping. Conversely, parametric representations support the latter tasks but are ill-suited for occupancy queries. We propose a novel learned alternative to these approaches, based on intersections of localized explicit , or height-field , surfaces. Since explicits can be trivially expressed both implicitly and parametrically, NESI directly supports a wider range of processing operations than implicit alternatives, including occupancy queries and parametric access. We represent input shapes using a collection of differently oriented height-field bounded half-spaces combined using volumetric Boolean intersections. We first tightly bound each input using a pair of oppositely oriented height-fields, forming a Double Height-Field (DHF) Hull . We refine this hull by intersecting it with additional localized height-fields (HFs) that capture surface regions in its interior. We minimize the number of HFs necessary to accurately capture each input and compactly encode both the DHF hull and the local HFs as neural functions defined over subdomains of $\mathbb {R}^2 $ . This reduced dimensionality encoding delivers high-quality compact approximations. Given similar parameter count, or storage capacity, NESI significantly reduces approximation error compared to the state of the art, especially at lower parameter counts.

IDiff-NeRF: single-view 3D human body reconstruction utilizing identity-based diffusion within implicit neural network framework

2025-06-13

Yansen Huang , Hongji Yang , Jiao Liu +1 more | The Visual Computer

3DGENie: Synthetic point clouds for semantic segmentation in realistic virtual environments

2025-06-13

Anthony Yaghi , Joe Tekli , Marc Kamradt +1 more | Multimedia Tools and Applications

Enhancing primitive segmentation through transformer-based cross-task interaction

2025-06-13

Tao Wang , Weibin Xi , Yong Cheng +3 more | Engineering Applications of Artificial Intelligence

A Survey on Deep Learning in 3D CAD Reconstruction

2025-06-13

Ruiquan Lin , Yonggang Ji , Wanting Ding +3 more | Applied Sciences

Three-dimensional CAD reconstruction is a long-standing and important task in fields such as industrial manufacturing, architecture, medicine, film and television, research, and education. Reconstructing CAD models remains a persistent challenge … Three-dimensional CAD reconstruction is a long-standing and important task in fields such as industrial manufacturing, architecture, medicine, film and television, research, and education. Reconstructing CAD models remains a persistent challenge in machine learning. There have been many studies on deep learning in the field of 3D reconstruction. In recent years, with the release of CAD datasets, there have been more and more studies on 3D CAD reconstruction using deep learning. With the continuous deepening of research, deep learning has significantly improved the performance of tasks in the field of CAD reconstruction. However, this task remains challenging due to data scarcity and labeling difficulties, model complexity, and lack of generality and adaptability. This paper reviews both classic and recent research results on 3D CAD reconstruction tasks based on deep learning. To the best of our knowledge, this is the first investigation focusing on the CAD reconstruction task in the field of deep learning. Since there are relatively few studies related to 3D CAD reconstruction, we also investigate the reconstruction and generation of 2D CAD sketches. According to the different input data, we divide all investigations into the following categories: point cloud input to 3D CAD models, sketch input to 3D CAD models, other input to 3D CAD models, reconstruction and generation of 2D sketches, characterization of CAD data, CAD datasets, and related evaluation indicators. Commonly used datasets are outlined in our taxonomy. We provide a brief overview of the current research background, challenges, and recent results. Finally, future research directions are discussed.

Real-Time Cloth Simulation in Extended Reality: Comparative Study Between Unity Cloth Model and Position-Based Dynamics Model with GPU

2025-06-12

Taeheon Kim , Jun Ma , Min Hong | Applied Sciences

This study proposes a GPU-accelerated Position-Based Dynamics (PBD) system for realistic and interactive cloth simulation in Extended Reality (XR) environments, and comprehensively evaluates its performance and functional capabilities on standalone … This study proposes a GPU-accelerated Position-Based Dynamics (PBD) system for realistic and interactive cloth simulation in Extended Reality (XR) environments, and comprehensively evaluates its performance and functional capabilities on standalone XR devices, such as the Meta Quest 3. To overcome the limitations of traditional CPU-based physics simulations, we designed and optimized highly parallelized algorithms utilizing Unity’s Compute Shader framework. The proposed system achieves real-time performance by implementing efficient collision detection and response handling with complex environmental meshes (RoomMesh) and dynamic hand meshes (HandMesh), as well as capsule colliders based on hand skeleton tracking (OVRSkeleton). Performance evaluations were conducted for both single-sided and double-sided cloth configurations across multiple resolutions. At a 32 × 32 resolution, both configurations maintained stable frame rates of approximately 72 FPS. At a 64 × 64 resolution, the single-sided cloth achieved around 65 FPS, while the double-sided configuration recorded approximately 40 FPS, demonstrating scalable quality adaptation depending on application requirements. Functionally, the GPU-PBD system significantly surpasses Unity’s built-in Cloth component by supporting double-sided cloth rendering, fine-grained constraint control, complex mesh-based collision handling, and real-time interaction with both hand meshes and capsule colliders. These capabilities enable immersive and physically plausible XR experiences, including natural cloth draping, grasping, and deformation behaviors during user interactions. The technical advantages of the proposed system suggest strong applicability in various XR fields, such as virtual clothing fitting, medical training simulations, educational content, and interactive art installations. Future work will focus on extending the framework to general deformable body simulation, incorporating advanced material modeling, self-collision response, and dynamic cutting simulation, thereby enhancing both realism and scalability in XR environments.

VectorMamba: Enhancing point cloud analysis through vector representations and state space modeling

2025-06-11

Zhicheng Wen | Computers & Graphics

SR-CurvANN: Advancing 3D surface reconstruction through curvature-aware neural networks

2025-06-04

Marina Hernández-Bautista , Francisco Javier Melero | Computers & Graphics