Typed, parallel agentic systems for text-to-scene generation in Unreal Engine with reflection and error feedback

Simulation environments for AI development demand structured, error-resilient generation pipelines. We extend our earlier work on LSCENE by introducing role-based agents that operate in parallel under explicit parent–child constraints. Agentic systems, however, face challenges in scaling functional roles and maintaining reliability under type errors. To address these limitations, each agent is grounded in an independently typed vocabulary and is equipped with reflection access to Unreal Engine methods and assets. To support asset grounding, we construct a vector database of Quixel/Fab assets using a large language model to generate structured semantic descriptors, enabling similarity-based retrieval and filtering. This constrains the candidate asset space and, when combined with type-safe bindings, reduces ambiguity and improves selection fidelity. During graph execution, any error detected by the TypeScript compiler or runtime validation is propagated back to the responsible agent, triggering localized re-execution without disrupting the broader pipeline. Once all agents complete, their outputs are aggregated into a unified LSCENE program. As scene complexity grows, additional agents can be introduced without disrupting existing roles due to agent independence. In general, this division of labor reduces the context of individual agents, task complexity, and generation time while improving robustness and scalability. We evaluate the system along two axes: (i) asset filtering via corrective retrieval-augmented generation, demonstrating improved consistency and relevance in asset selection, and (ii) end-to-end environment construction workflows, illustrating the system’s ability to generate coherent, complex scenes under parallel execution.