Advancing AI Spatial Understanding: XGRIDS Research Selected for Oral Presentation at ACM Multimedia 2026

News2026.09.106 min read

XGRIDS advances AI spatial understanding at ACM MM 2026 with SpatialGrammar, coupling language models with compiler-driven physical constraint validation.

Key Takeaways & Technical Highlights

Academic Honor: Selected for Oral Presentation at ACM Multimedia 2026 (jointly developed by HKUST(GZ), XGRIDS, and HIT Shenzhen).
SpatialGrammar [Novel Paradigm] : Decouples spatial relationship reasoning from 3D geometric solving via a Bird’s-Eye-View (BEV) grid representation and deterministic compiler translation.
SG-Mini [Compact Model]: Only 104M parameters, demonstrating that lightweight models can master complex indoor spatial logic.
SG-Agent [Data Loop]: Iteratively generates, validates, and refines scenes using compiler and visual feedback to enforce spatial and collision constraints.

Titled “SpatialGrammar: A Domain-Specific Language for LLM-Based 3D Indoor Scene Generation, the research paper jointly developed by teams from The Hong Kong University of Science and Technology (Guangzhou), XGRIDS, and Harbin Institute of Technology (Shenzhen) has been selected for Oral Presentation at ACM Multimedia 2026, recognizing research that explores a fundamental challenge at the intersection of large language models (LLMs), 3D generation, and spatial reasoning.

At the heart of the work is a fundamental question: can LLMs move beyond generating visually plausible 3D content to reasoning about the structure and spatial relationships of objects within a physical environment?

To address this challenge, the team introduces SpatialGrammar, a domain-specific language designed specifically for LLM-based 3D indoor scene generation, together with SG-Agent, a closed-loop agentic system that iteratively generates and validates scenes, and SG-Mini, a compact 104M-parameter model trained entirely on compiler-validated synthetic data.

More broadly, the work points to a shift in how AI-generated 3D content can be understood: from generating objects and scenes to reasoning about where things belong, how they relate to one another, and whether a generated environment makes physical sense.

AI Can Generate 3D, but Can It Understand Space?

Consider a simple instruction to generate a living room: place a sofa against the wall, position a coffee table in front of it, and arrange two chairs facing the sofa. For humans, these spatial relationships require little explanation. We naturally understand what “against the wall,” “in front of,” and “facing” mean. We also know that a coffee table should not intersect with the sofa and that chairs should not collide with the walls.

For LLMs, however, translating these language-based relationships into precise 3D positions, orientations, and physical constraints is far more challenging. Methods that require models to directly handle raw coordinates, object poses, or complex procedural code can also result in spatial errors, such as floating objects, wall penetration, or overlapping assets.

This reveals a key gap in AI’s evolution from language understanding to spatial understanding.

SpatialGrammar addresses this challenge with a fundamental question: if it is difficult for LLMs to reason directly about complex 3D coordinates, could AI be given a “spatial language” better suited to expressing and reasoning about spatial relationships?

SpatialGrammar: Giving LLMs a Language for Space

SpatialGrammar introduces a domain-specific language for LLM-based 3D indoor scene generation, translating complex 3D layouts into a more structured representation. It uses a bird’s-eye-view (BEV) grid to describe indoor layouts, while the LLM determines object placement, arrangement, and orientation. A compiler then deterministically converts these high-level descriptions into precise 3D geometry and poses.

SpatialGrammar DSL code with a bird's-eye grid translating into rendered 3D indoor scenes for AI scene generation

In this way, SpatialGrammar establishes a clearer spatial representation between natural language and 3D geometry. Instead of requiring the model to directly reason over complex coordinates and geometric relationships, it can use structured rules to describe what goes where and how objects relate to one another, while enabling the resulting layouts to be validated.

In other words, SpatialGrammar is not simply about enabling AI to generate a 3D scene. Rather, it explores how AI can build a structured understanding of three-dimensional environments, grounded in physical commonsense and spatial relationships. As embodied AI continues to advance, this marks an important step toward moving AI beyond text-based interaction and toward a deeper understanding of the physical world.

SG-Agent: From One-Shot Generation to Generate, Validate, and Refine

A single generation does not necessarily produce a physically plausible scene. Objects may collide, face the wrong direction, or intersect with walls.

To address this, the research introduces SG-Agent, transforming 3D scene generation from a one-shot prediction into a generate–validate–refine loop.

The user’s instruction is first translated into SpatialGrammar and compiled into a scene. The system then evaluates the result through symbolic constraint checking, Draft Engine visualization, physics-based collision detection, and visual or multimodal feedback. Based on this feedback, the agent modifies the spatial description and regenerates the scene.

SG-Agent pipeline with in-context learning, collision checking and rendering for closed-loop embodied AI scene generation

The process typically requires only two to three iterations to progressively refine spatial relationships. This shifts AI-based 3D generation beyond simply producing an answer toward checking whether the answer satisfies spatial constraints and refining it based on feedback.

SG-Mini: Compact Models Can Learn Structured Spatial Knowledge

The research also explores an important question: does spatial generation necessarily require increasingly larger language models?

To investigate this, the team developed SG-Mini, a model with just 104 million parameters, trained entirely on compiler-validated synthetic data. Without relying on human annotation, the model learns to generate scenes that satisfy spatial constraints.

SpatialGrammar overview pairing furniture-layout and architectural-structure languages for 3D indoor spatial understanding

Under the evaluation settings reported in the paper, SG-Mini achieves performance comparable to or better than some larger LLM-based baselines. This suggests that for specialized spatial reasoning tasks, a more effective spatial representation and structured supervision can be just as important as simply increasing model size.

Beyond 3D Generation: Toward Spatial Understanding

For VR, gaming, simulation, and embodied AI, generating a visually realistic environment is only the first step. An intelligent system that truly operates in a 3D world also needs to understand what exists in the space, where things are, how they relate to one another, and what constitutes a physically plausible arrangement.

This means 3D should not be viewed merely as content to be generated and rendered. It can also serve as a structured representation through which AI understands the world.

From this perspective, SpatialGrammar explores a connection between language intelligence and spatial intelligence. Natural language can tell AI what is needed, while a spatial language can further express where it should be, how it should be arranged, and whether the result makes sense. By closing the loop between generation, validation, and refinement, AI gains the potential to continuously reason about space.

XGRIDS: Building the Spatial Layer for AI

By combining a spatial language, structured representations, deterministic compilation, physical constraint validation, and agent-based iterative refinement, SpatialGrammar explores a path from 3D generation toward spatial understanding.

For XGRIDS, this research also extends the company’s long-term exploration of spatial intelligence—from reality capture and 3D reconstruction to spatial intelligence applications. XGRIDS is building a technology pipeline that transforms the real world into high-fidelity, computable, and usable digital spaces.

If reality capture enables machines to “see” the world, and 3D reconstruction gives that world a digital representation, spatial understanding enables AI to further understand the relationships within it. This is a critical step toward the next stage of spatial intelligence, where 3D is no longer simply a digital asset for people to view, but a foundation for AI to understand, reason about, generate, and interact with the world.

XGRIDS will continue to explore technologies that connect AI with the 3D real world, advancing 3D from content that machines can generate and render into a foundation for machines to understand, reason about, and interact with the world.