A Method for Generating Immersive Narratives in Digital Shadow Puppetry Based on ReAct Planning and Multi-Agent Coordination
Main Article Content
Abstract
Technological developments in the field of artificial intelligence and intelligent algorithms are the foundation of digital shadow puppetry to move towards a dynamic, interactive and multimodal storytelling. This paper aims to solve such problems as inadequate plot continuity, lack of constraints on cultural rules and conflicts between multi-agent outputs, and proposes a method for creating an immersive narrative in digital shadow puppetry based on ReAct planning and multi-agent collaboration. This approach builds an association model between narrative elements and cultural characteristics of digital shadow puppetry, and then it uses the ‘reasoning–action–observation’ cycle to do the narrative goal decomposition, cultural constraint path search and local dynamic re-planning, and finally it achieves the collaborative generation of multimodal content by modelling the director, screenwriter, characters, scenes, actions and music as agents. Plot coherence, cultural conformity and multimodal alignment were achieved at 92.4 per cent, 94.1 per cent and 92.8 per cent respectively while task completion was at 94.5 per cent and overall user satisfaction score of 4.63 in experimental results. The effective combination of cultural expression norms, narrative consistency and real-time interaction requirements has ensured this method can be a viable approach for the diffusion and interactive performance of digital shadow puppetry.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
J. Gao and S. Juluri, “From Idea to Co-Creation: A Planner-Actor-Critic Framework for Agent Augmented 3D Modeling,” in Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems, 2026, pp. 1–5.
H. Marah and M. Challenger, “Adaptive hybrid reasoning for agent-based digital twins of distributed multi-robot systems,” Simulation, vol. 100, no. 9, pp. 931–957, 2024.
M. Younes, E. Kijak, R. Kulpa, et al., “MAAIP: Multi-Agent Adversarial Interaction Priors for imitation from fighting demonstrations for physics-based characters,” Proceedings of the ACM on Computer Graphics and Interactive Techniques, vol. 6, no. 3, pp. 1–20, 2023.
S. Zhang, Y. Xiao, R. Ma, et al., “RPGAgent: Driving Coherent Story-to-Play Generation with an LLM-Based Multi-Agent System,” in Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, 2026, pp. 1–22.
D. Wu, H. Shi, Z. Sun, et al., “Deciphering digital detectives: Understanding LLM behaviors and capabilities in multi-agent mystery games,” in Findings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 8225–8291.
E. K. Tütüncü, Q. Zhou, F. Brudy, et al., “PlayWrite: A Multimodal System for AI Supported Narrative Co-Authoring Through Play in XR,” in Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, 2026, pp. 1–26.
I. Ahmed, M. A. Syed, M. Maaruf, et al., “Distributed computing in multi-agent systems: a survey of decentralized machine learning approaches,” Computing, vol. 107, no. 1, p. 2, 2025.
S. Uddin, B. Hussain, S. Fareed, et al., “A review of fault tolerance techniques in generative multi-agent systems for real-time applications,” International Journal of Ethical AI Application, vol. 1, no. 1, pp. 43–53, 2025.
H. Wu, A. Ghadami, A. E. Bayrak, et al., “Evaluating emergent coordination in multi-agent task allocation through causal inference and sub-team identification,” IEEE Robotics and Automation Letters, vol. 8, no. 2, pp. 728–735, 2022.
Y. Ge, Y. Ren, W. Hua, et al., “Llm as os, agents as apps: Envisioning aios, agents and the aios-agent ecosystem,” arXiv preprint arXiv:2312.03815, 2023.
I. Adabara, B. Olaniyi Sadiq, A. Nuhu Shuaibu, et al., “Trustworthy agentic AI systems: a cross-layer review of architectures, threat models, and governance strategies for real-world deployment,” F1000Research, vol. 14, p. 905, 2025.
D. Parmar, S. Olafsson, D. Utami, et al., “Designing empathic virtual agents: manipulating animation, voice, rendering, and empathy to create persuasive agents,” Autonomous agents and multi-agent systems, vol. 36, no. 1, p. 17, 2022.
J. Wei, X. Wang, D. Schuurmans, et al., “Chain-of-thought prompting elic-its reasoning in large language models,” Advances in neural information processing systems, vol. 35, pp. 24824–24837, 2022.
W. Huang, P. Abbeel, D. Pathak, et al., “Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,” in Proceedings of the 39th International Conference on Machine Learning. PMLR, vol. 162, 2022, pp. 9118–9147.
S. Yao, J. Zhao, D. Yu, et al., “ReAct: Synergizing reasoning and acting in language models,” in The Eleventh International Conference on Learning Representations, 2023, pp. 1–34.
N. Shinn, F. Cassano, E. Berman, et al., “Reflexion: Language agents with verbal reinforcement learning,” in Advances in Neural Information Processing Systems, vol. 36, 2023, pp. 8634–8652.
S. Venkatraman, N. I. Tripto, and D. Lee, “CollabStory: Multi-LLM collaborative story generation and authorship analysis,” in Findings of the Association for Computational Linguistics: NAACL 2025, 2025, pp. 3665–3679.
Y. Ran, X. Wang, T. Qiu, et al., “BOOKWORLD: From novels to interactive agent societies for story creation,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, 2025, pp. 15898–15912.
K. Park, M. Kim, and K. Jung, “A character-centric creative story generation via imagination,” in Findings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 1598–1645.
Y. He, “ShadowPlayVR: Understanding traditional shadow puppetry performance techniques through non-intuitive embodied interactions,” in Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology, 2023, Art. no. 73, pp. 1–2.
Z. Yao, S. Lyu, Y. Lu, et al., “ShadowMaker: Sketch-based creation tool for digital shadow puppetry,” in Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, 2024, Art. no. 417, pp. 1–5.
Y. Liu, R. M. Williams, G. Xie, et al., “Promoting the culture of Qinhuai River Lantern shadow puppetry with a digital archive and immersive experience,” arXiv preprint arXiv:2410.03532, pp. 1–23, 2024.
W. Chen and Y. Guo, “Embodied transmission: Preserving intangible cultural heritage through procedural VR experience—A case study on Chinese shadow puppetry,” in Proceedings of the 2025 5th International Conference on Culture, Design and Social Development. Atlantis Press, 2026, pp. 272–278.
Y. Zhou and S. J. Jong, “Reconfiguring Chinese shadow puppetry in digital media,” SN Social Sciences, vol. 6, p. 216, 2026.
F. Huot, R. K. Amplayo, J. Palomaki, et al., “Agents’ Room: Narrative generation through multi-step collaboration,” in The Thirteenth International Conference on Learning Representations, 2025, pp. 1–34.
T. Yu, K. Shi, Z. Zhao, et al., “Multi-agent based character simulation for story writing,” in Proceedings of the Fourth Workshop on Intelligent and Interactive Writing Assistants. Association for Computational Linguistics, 2025, pp. 87–108.
T. Zhou, Z. Duan, C. Chen, et al., “AgentStory: A multi-agent system for story visualization with multi-subject consistent text-to-image generation,” in Proceedings of the 2025 International Conference on Multimedia Retrieval, 2025, pp. 1894–1902.