Deep Waltz: Neuro-Symbolic Line Drawing Interpretation via Learned Perception and Constraint Satisfaction

Abstract

Interpreting three-dimensional structure from two- dimensional line drawings remains a fundamental challenge in computer vision, cognitive science, and artificial intelligence. Classical symbolic approaches based on constraint-driven junction labelling provide strong geometric interpretability but are highly sensitive to noise, fragmented lines, and missing segments. In contrast, modern deep learning methods are effective at detecting edges, junctions, and local geometric patterns under real-world conditions, yet often lack global consistency, interpretability, and enforcement of physically plausible structural relationships. These limitations motivate hybrid neuro-symbolic approaches that combine learned perception with symbolic reasoning. In this work, we present Deep Waltz, a hybrid vision framework that integrates a compact CNN-based neural refinement module with a Waltz- style constraint satisfaction solver. The proposed pipeline performs end to-end processing from raw images to globally consistent symbolic interpretations, including edge detection, line segment extraction, junction detection, CNN-based patch classification, and constraint-based global inference using legal junction-label tables. An EM-like iterative training scheme is introduced, in which CSP-inferred labels serve as pseudo-labels to refine the neural components and progressively improve global coherence. Experiments on synthetic polyhedral scenes, hand-drawn sketches, and real-image edge maps demonstrate that Deep Waltz substantially improves junction classification accuracy, legal labelling rates, and structural reconstruction quality compared to symbolic-only and neural-only baselines. These results indicate that the proposed framework provides a robust, interpretable, and reproducible solution for structural scene understanding from line drawings.