As developments in robotics, autonomous driving, and spatial computing proceed to unfold, a rising variety of Pc Imaginative and prescient and Machine Studying (CVML) algorithms are incorporating three-dimensional information into their frameworks. Debugging these 3D CVML fashions usually requires going past conventional efficiency analysis strategies, necessitating a deeper understanding of an algorithm’s conduct inside its spatio-temporal context. Nonetheless, the shortage of acceptable visualization instruments presents a major impediment to successfully exploring 3D information and spatial options in relation to key efficiency indicators (KPIs). To handle this problem, we discover the appliance of Immersive Analytics (IA) methodologies to reinforce the debugging strategy of 3D CVML fashions. By way of in-depth interviews with eight CVML engineers, we determine widespread duties and challenges confronted through the improvement of spatial algorithms, and set up a set of design ideas for creating instruments tailor-made to spatial mannequin analysis. Constructing on these insights, we suggest a novel immersive analytics system for debugging an indoor localization algorithm. The system is constructed utilizing internet applied sciences and integrates WebXR to allow fluid transitions throughout the reality-virtuality continuum. We conduct a qualitative research with six CVML engineers utilizing our system on Apple Imaginative and prescient Professional, observing their analytical workflow as they debug an indoor localization sequence. We focus on the benefits of using immersive analytics within the mannequin analysis workflow, emphasizing the position of seamlessly integrating 2D and 3D visualizations throughout various ranges of immersion to facilitate more practical mannequin evaluation. Lastly, we mirror on the implementation trade-offs and focus on the generalizability of our findings for future efforts in immersive 3D CVML mannequin debugging.
- † Harvard College, Cambridge
