Privately Owned Vehicle Work Group Meeting - 2025/02/03 - Slot 2

[m-zain-khawaja]

Agenda

• Update on Scene3D training
• Update on EgoPath dataset curation
• EgoPath network design discussion
• Update on EgoLanes dataset curation

Discussion

@m-zain-khawaja :

After completing the second phase of Scene3D training using both simulator and real-world data, it was found that while results were good, there was still room for improvement in the network to achieve sharper edge boundaries and better overall accuracy. To this effect, a modification has been made to the Scene3D Head block. This block is now outputting three separate predictions, and the final depth map is the sum of the predictions. The idea is to allow the network to specialise in depth estimation for natural scenes by exploiting scene priors - typically a real-world road scene has 3 depth distribution types, 1) flat (road, footpath), 2) parallel (buildings, trees, barriers) 3) fronto-planar (foreground objects like cars, pedestrians, traffic cones, signs etc). By predicting three distinct outputs, the network should be able to more accurately model these unique depth distribution priors and more accurately predict depth. The DDAD, UrbanSyn and KITTI datasets are being used for training and the Argoverse Dataset is being used as a large and diverse test set.

The inference class (scene_3d_infer.py) has been written. This will be used to visualize the depth map. Additionally, Open3D will be used to create a PointCloud output visualization from the depth predictions.

EgoPath Dataset Curation Update

@TranHuuNhatHuy has completed bug-fixes for the CurveLanes dataset and aims to upload data this week.

@sarun-hub has completed dataset parsing and auditing of ROADWorks dataset and his pull request has been merged with the main branch.

@docjag has submitted the latest fixes in his PR - he is currently doing data auditing and expects to upload the completed dataset this week.

Dataset curation tracking

• TuSimple - 6,394 training data samples
• CULane - 15,800 training data samples
• CurveLane - (parsing completed by @TranHuuNhatHuy, data to be uploaded soon)
• BDD100K - 24,500 training samples
• ROADWorks - (parsing completed by @docjag, data to be uploaded soon)
• Comma2k19 - 31,962 training data samples

Ego Path Network Design

We are now at the stage where we can begin thinking about the design of the EgoPath network. The upstream blocks including the Backbone and PathContext blocks are fixed. We need to determine the best EgoPath Head block and the corresponding loss function to penalise the networks outputs. In general, keypoint prediction methods and parametric curve prediction methods give the best trade-off between speed vs accuracy and are able to deliver faster than real-time results on commodity hardware.

We should also explore the use of GRU and LSTM blocks to help incorporate temporal information in the network predictions, as there is a continuity in path estimates between consecutive frames.

EgoLanes Dataset Curation Update

• TuSimple - 6,394 training data samples
• CULane - 15,800 training data samples
• CurveLane - (to be started)
• BDD100K - (to be started)

Attendees

TBD

Zoom Meeting Video Recording

Video Meeting Link
Please contact the work group lead (@m-zain-khawaja) to request access to a recording of this meeting.