TIGP (SNHCC) -- Weakly Supervised Point Cloud Segmentation using An MIL-Derived Transformer with 2D-3D Interlace Attention
- LecturerProf. Yen-Yu Lin (Department of Computer Science, National Yang Ming Chiao Tung University)
Host: TIGP (SNHCC) - Time2024-03-18 (Mon.) 14:00 ~ 16:00
- LocationAuditorium 106 at IIS new Building
Abstract
In this talk, I will talk about our work, an MIL-derived transformer, which addresses weakly supervised point cloud segmentation by mining additional supervisory signals. First, the transformer model is derived based on multiple instance learning (MIL) to explore pair-wise cloud-level supervision, where two clouds of the same category yield a positive bag while two of different classes produce a negative bag. It leverages not only individual cloud annotations but also pair-wise cloud semantics for model optimization. Second, Adaptive global weighted pooling (AdaGWP) is integrated into our transformer model to replace max pooling and average pooling. It introduces learnable weights to re-scale logits in the class activation maps. It is more robust to noise while discovering more complete foreground points under weak supervision. Third, we perform point subsampling and enforce feature equivariance between the original and subsampled point clouds for regularization. The proposed method is end-to-end trainable and is general because it can work with different backbones with diverse types of weak supervision signals, including sparsely annotated points and cloud-level labels. The experiments show that it achieves state-of-the-art performance on the S3DIS and ScanNet benchmarks.