3D part segmentation

RePart

Organize Primitives into Semantic Parts:
Reinforcement Reasoning for 3D Segmentation

Xiaoming Gong^1,* Ruoyu Wu^2,* Zhenhong Sun³ Chunlin Chen¹ Daoyi Dong⁴ Huadong Mo² Zhi Wang¹ Hongdong Li³

¹ Nanjing University, China ² University of New South Wales, Canberra, Australia ³ Australian National University, Australia ⁴ University of Technology Sydney, Australia

^* Equal contribution

Code

A three-panel figure: a chair decomposed into primitives, examples of primitive-to-part ambiguity, and sequential grouping as a Markov decision process. — Primitive decomposition supplies compact structure. RePart organizes those primitives into semantic parts through sequential merge-and-stop decisions.

The problem

Abstract

Primitive-based 3D segmentation offers a compact and explicit alternative to dense surface prediction, but geometric decomposition alone does not determine which primitives belong to the same semantic part. One part may span several primitives, while nearby primitives with similar geometry may belong to different parts.

RePart treats primitive-to-part organization as a finite-horizon decision process. A merge-and-stop policy learns to group fine-grained superquadrics according to their effect on the final partition, then transfers the inferred identities back to the original mesh with boundary-aware surface labeling. Training uses part annotations, while the deployed policy relies on geometry alone.

How it works

From a primitive workspace to dense part labels

RePart separates compact structural reasoning from boundary localization on the original surface.

Overview of RePart: SQ workspace construction, progressive merge-and-stop reasoning, and surface labeling of the original mesh. — Overview of the three stages of RePart. The policy uses geometry-only observations at inference; ground-truth part labels supervise training.

Composable Primitive Workspace

Over-segmented superquadrics provide stable, geometric tokens without fixing the final semantic partition.

Reinforcement-Learning Part Reasoning

A policy repeatedly merges groups or stops. PPO evaluates decisions through their downstream effect on the partition.

Boundary-Aware Surface Labeling

Terminal group identities are projected onto the original mesh using surface proximity and volumetric evidence.

Quantitative results

Partition quality across datasets

Strong in-domain performance on PartNet, with metric-dependent transfer to 3DCoMPaT++ without target-dataset fine-tuning.

0.5460PartNet mIoUHighest in the comparison

0.8471PartNet VoILower is better

0.75753DCoMPaT++ RIHighest in the comparison

0.57903DCoMPaT++ SCHighest in the comparison

Average partition quality

Each dataset is evaluated with four complementary partition metrics.

Method	mIoU ↑	RI ↑	VoI ↓	SC ↑
MPS	0.3169	0.5266	1.0213	0.5267
PartField	0.5249	0.7877	1.0810	0.6502
P3-SAM	0.4858	0.6966	1.2238	0.5833
SAMPart3D	0.4229	0.6999	1.1996	0.5862
PriMAny	0.2440	0.5231	2.6506	0.2476
RePart	0.5460	0.7965	0.8471	0.6891

Method	mIoU ↑	RI ↑	VoI ↓	SC ↑
MPS	0.2125	0.4371	1.2043	0.4371
PartField	0.3483	0.6914	1.2998	0.5302
P3-SAM	0.5003	0.7455	1.3783	0.5606
SAMPart3D	0.4144	0.7089	1.5508	0.5304
PriMAny	0.2477	0.5932	2.4509	0.2796
RePart	0.4476	0.7575	1.2838	0.5790

Metrics: mIoU measures part overlap, RI pairwise partition agreement, VoI partition information difference, and SC region covering. Results match the manuscript's average partition-quality table.

On 3DCoMPaT++, RePart's mIoU is below P3-SAM (0.4476 vs. 0.5003), while its RI and SC are higher than the compared methods. Transfer is therefore metric-dependent.

Category view

RePart mIoU by category group

Jointly fine-tuned policy; no 3DCoMPaT++ fine-tuning.

Dataset	Furniture	Containers	Daily-use	Average
PartNet	0.4247	0.6456	0.5432	0.5460
3DCoMPaT++	0.3703	0.5149	0.4607	0.4476

Qualitative results

From fragmented tokens to coherent parts

Examples from PartNet and 3DCoMPaT++ compare RePart with ground truth and five baseline methods.

Ten rows of segmented 3D objects. Columns show ground truth, MPS, PriMAny, PartField, P3-SAM, SAMPart3D, and RePart. — Five PartNet examples appear above five 3DCoMPaT++ examples. Click the expand button for a closer view.