DriveOSS | WACV 2027 Dataset & Benchmark

DriveOSS Overview — 128-Beam LiDAR + 6× Surround RGB

12,661

Annotated LiDAR Scans

75,966

6-View RGB Images

52

Scenes (19 ID / 33 OOD)

5

Weather Domains

19 + 1

ID Classes + Outlier

1.52M

3D Outlier Points

Motivation & Scope

Abstract

Recent advances in 3D semantic segmentation have significantly improved scene understanding in autonomous driving. However, existing methods are typically developed under a closed-set assumption and evaluated under standard driving conditions, where all categories are known in advance. In real-world driving, autonomous vehicles inevitably encounter previously unseen objects, making closed-set perception insufficient for safe deployment. To address this challenge, we introduce DriveOSS, which, to the best of our knowledge, is the first real-world multimodal dataset and benchmark specifically designed for 3D anomaly segmentation and open-set semantic segmentation. It contains 12,661 synchronized and annotated frames captured using six surround-view RGB cameras, a 128-beam LiDAR, and a GPS/IMU unit. The dataset comprises 52 real-world driving scenes collected under clear, nighttime, cloudy, rainy, and snowy conditions and is organized into ID (in-distribution) and OOD (out-of-distribution) subsets. The ID scenes contain only known classes and are used for training, whereas the OOD scenes additionally contain unknown objects annotated as the outlier class and are used for validation and testing. Extensive experiments show that unseen objects remain a major challenge for current 3D perception systems.

Adverse Weather & Disjoint Splits

Recorded with an Ouster OS1-128 LiDAR (20 Hz), 6 Basler RGB cameras (10 Hz), and RTK-INS (600 Hz) across Clear, Nighttime, Cloudy, Rainy, and Snowy conditions on geographically disjoint ID and OOD routes.

Unified 3D Open-IoU Protocol

Unlike binary anomaly benchmarks that discard background semantics, DriveOSS retains all 19 known ID classes + 1 Outlier class in OOD scenes, enabling joint evaluation via 3D mOpen-IoU.

LiDAR & Multimodal Baselines

Benchmarks P3Former, LCPS-lidar, Mask4Former3D, LIDO, LCPS, and our query-level fusion baseline Mask4Former3D* across MSP, MaxLogit, Energy, Entropy, and RbA scoring.

Literature Context

Comparison with Automotive 3D Datasets

Dataset	Year	Weather & Illumination	Modality	#Beam	Cameras	OOD Scene Annotations	Tasks
SemanticKITTI	2019	Clear only	L+C	64	Front	× None (Closed-set)	SS, PS
Panoptic nuScenes	2021	Diverse	L+C	32	6× Surround	× None (Closed-set)	PS
SemanticSTF	2023	Fog / Rain / Snow	L+C	64	Front	× None (Closed-set)	SS
Panoptic-CUDAL	2025	Rain	L+C	128	Surround	× None (Closed-set)	PS
STU	2025	Clear only	L+C	128	Surround	Binary (Inlier / Outlier)	PS, AS
LIDO (Synthetic Mix)	2026	Clear only	L	32/40/64	—	Binary (Inlier / Outlier)	AS
DriveOSS (Ours)	2027	Clear, Night, Cloudy, Rain, Snow	L+C	128	6× Surround	✓ 19 ID Classes + Outlier	SS, AS, OSS

Hardware & Environmental Diversity

Multimodal Sensor Suite & Weather Distribution

Synchronized Recording Platform

All sensors are hardware-synchronized to a common GPS-derived time reference across urban, rural, downtown, pedestrian, and industrial zones:

1× LiDAR (Ouster OS1-128): 128 beams, 360° H-FOV, −22.5° to +22.5° V-FOV @ 20 Hz.
6× RGB Cameras (Basler ace2R): Surround-view 1920×1080 resolution @ 10 Hz.
1× GNSS/INS (Certus RTK): >600 Hz update rate, 0.01 m horizontal & 0.015 m vertical accuracy.

Data Recording Platform

Surround LiDAR-Camera Calibration

Distribution Across Splits & 5 Weather Domains

Environmental Domain	ID Train Split		ID Validation Split		OOD Validation Split			OOD Test Split
Environmental Domain	Seq.	Frames	Seq.	Frames	Seq.	Frames	OOD Points	Seq.	Frames	OOD Points
Clear	4	809	6	1,701	9	2,315	252,606	8	1,766	885,729
Nighttime	1	184	1	79	2	287	22,132	2	357	42,979
Cloudy	2	636	1	186	2	742	43,548	2	666	71,440
Rainy	1	399	1	165	2	696	35,104	2	618	66,509
Snowy	1	212	1	253	2	238	12,898	2	352	89,803
Overall DriveOSS	9	2,240	10	2,384	17	4,278	366,288	16	3,759	1,156,460

Anomalies & Ground Truth Quality

OOD Hazard Taxonomy & Two-Stage Annotations

Representative OOD Samples

📦 1. Containers & Packaging

Cardboard boxes, plastic boxes, foam blocks, garbage bags, and small trash bins with simple geometries and diverse materials.

🪑 2. Furniture & Fixtures

Office chairs, folding chairs, wooden chairs, clothes racks, and advertising signs with complex structures and varying heights.

🪜 3. Tools & Industrial Equipment

Ladders, shovels, metal barriers, and maintenance equipment featuring rigid structures and occasionally high reflectivity.

🎒 4. Personal Belongings

Backpacks, bicycle helmets, bags, and personal items representing small low-profile hazards with irregular shapes.

🛒 5. Utility & Service Equipment

Shopping carts, waste-collection carts, and mobile bins commonly encountered in urban and commercial driving zones.

🪵 6. Roadside Debris & Misc.

Wooden boards, fallen signs, tires, and unstructured obstacles with diverse appearances and physical properties.

Spatial-Hazard Protocol & Annotation Workflow

To maintain an objective boundary between outlier and other-object, any non-ID object located on or immediately adjacent to the drivable surface (road, parking, lane-marking, or sidewalk curb) is strictly annotated as outlier, while permanent off-road fixtures (bus shelters, benches) are labeled other-object.

100% of OOD objects are manually annotated from scratch. For ID classes, KISS-ICP registered scans are initialized with coarse Mask4Former pre-labels and thoroughly overhauled via plane removal, class-isolation filtering, and independent dual-annotator verification.

Prelabel vs Refined_GT

Quantitative Evaluation

Interactive Benchmark Explorer

Showing the best-performing anomaly scoring variant per architecture across all 19 known classes + Outlier.

Method (Best Variant)	Mod.	car	bicycle	motorcycle	truck	other-veh.	person	bicyclist	motorcyclist	road	parking	sidewalk	other-gnd.	building	fence	vegetation	trunk	terrain	pole	traffic-sign	outlier ↑	mOpen-IoU ↑
P3Former + RbA	L	2.13	0.00	0.00	0.00	0.00	0.00	0.00	0.00	16.62	1.03	0.51	0.72	65.18	0.08	53.47	13.10	4.49	0.35	1.14	1.51	8.36
LCPS-lidar + Entropy	L	41.47	9.79	1.53	11.27	0.00	29.53	10.01	0.00	31.51	0.14	8.84	0.00	75.30	16.50	55.06	5.89	0.32	13.56	65.25	0.61	20.75
M4F3D + RbA	L	34.90	11.11	8.39	11.84	0.53	28.14	27.06	0.00	18.85	13.46	17.87	0.16	4.44	0.26	2.64	3.59	2.43	0.73	1.60	0.32	9.58
LIDO (Joint)	L	38.77	7.64	0.00	10.00	11.32	18.52	17.06	0.00	38.76	11.97	36.27	0.02	93.97	62.56	78.08	13.96	21.36	7.30	0.16	1.07	22.98
LCPS + Entropy	L+C	37.88	1.94	3.60	7.40	0.22	24.64	17.50	0.00	26.08	0.10	5.76	0.04	77.60	26.88	40.30	4.71	1.67	8.49	45.93	0.57	17.41
M4F3D* + RbA	L+C	44.83	5.30	4.88	8.21	2.48	37.81	32.96	0.52	17.18	0.03	18.55	0.65	10.49	2.91	3.45	8.86	7.87	2.94	4.20	0.21	11.27

Method (Best Variant per Backbone)	Modality	AUROC (%) ↑	FPR@95 (%) ↓	AP (%) ↑
P3Former + RbA	LiDAR	89.80	45.94	3.53
LCPS-lidar + Entropy	LiDAR	79.76	46.95	1.38
Mask4Former3D + RbA	LiDAR	67.91	100.00	12.57
LIDO (Joint Feature-Based)	LiDAR	91.70	32.66	4.44
LCPS + Entropy	L + C	78.67	54.44	1.59
Mask4Former3D* + MSP	L + C	70.38	96.85	7.11

Method	Modality	DriveOSS-ID Val mIoU (%) ↑	DriveOSS-OOD Test (ID Points) mIoU (%) ↑	Domain Shift Drop (Δ)
P3Former	LiDAR	43.27	29.58	−13.69
LCPS-lidar	LiDAR	51.94	33.75	−18.19
Mask4Former3D	LiDAR	34.98	31.11	−3.87
LIDO	LiDAR	58.35	40.99	−17.36
LCPS	L + C	54.05	34.00	−20.05
Mask4Former3D* (Query Fusion)	L + C	48.44	28.46	−19.98

Method	Modality	Base Network (ms)	+MSP	+ML	+Energy	+Entropy	+RbA	Total Per-Frame Latency ↓
P3Former	LiDAR	51.29	0.07	0.10	0.13	0.10	0.11	51.36 – 51.42 ms (~19.5 FPS)
LIDO	LiDAR	68.01	End-to-End Joint Semantic + Anomaly Branch					68.01 ms (~14.7 FPS)
LCPS-lidar	LiDAR	85.44	0.12	0.17	0.25	0.19	0.18	85.56 – 85.69 ms
Mask4Former3D	LiDAR	104.21	0.11	0.17	0.23	0.18	0.13	104.32 – 104.44 ms
Mask4Former3D* (Query Fusion)	L + C	106.35	0.10	0.16	0.21	0.17	0.12	106.45 – 106.56 ms (+2.05% vs L-only)
LCPS (Dense Point-Pixel Fusion)	L + C	179.80	0.11	0.16	0.21	0.15	0.16	179.91 – 180.01 ms (+110.4% vs L-only)

Qualitative Comparison Across Benchmark Tasks

Qualitative Benchmark Comparison

citation.bib

@inproceedings{yan2027driveoss,
  title     = {DriveOSS: Dataset and Benchmark for 3D Anomaly Segmentation and Open-set Semantic Segmentation in Autonomous Driving},
  author    = {Yan, Zhiran and Bhanderi, Savankumar and Lolge, Sourabh and Song, Rui and Paneliya, Girdharbhai and Agrawal, Shiva and Elger, Gordon},
  booktitle = {TODO TODO},
  year      = {TODO}
}

DriveOSS: Dataset and Benchmark for 3D Anomaly Segmentation and Open-set Semantic Segmentation in Autonomous Driving

Teaser Figure Placeholder