A Role-Specialized Retrieval-Augmented Generation Framework for Automated Compliance Checking in Structural Engineering
Main Article Content
Abstract
Automated code compliance checking in structural engineering remains difficult because practical systems must balance accuracy, maintainability, and deployment cost. Pure prompting with large language models is prone to hallucination and unstable numerical judgment, conventional retrieval-augmented generation may fail to assemble dispersed regulatory evidence, and ontology-heavy pipelines can be expensive to extend when codes evolve. We therefore propose a role-specialized retrieval-augmented generation framework composed of a deterministic Parser, a Reviewer guided by an Equilibrium Rule prompt, and a Critic that outputs a structured verdict. On a 211-case benchmark built from real-project building information modeling exports and expert review across slabs, beams, and columns, the framework achieves 96.68% overall accuracy, compared with 57.35% for zero-shot prompting, 79.15% for naive retrieval-augmented generation, 77.25% for naive retrieval-augmented generation with the Equilibrium Rule, and 89.10% for a re-implemented ontology- and rule-based baseline. The benchmark knowledge resources, retrieval corpus, and rule-supporting mappings are extracted from two Chinese national structural design codes covering concrete member design and structural loading. Ablation results suggest that the Equilibrium Rule and the Critic contribute most of the observed gain. We therefore position the framework as a decision-support approach for engineering compliance review rather than as a fully autonomous approval engine.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
J. Zhang et al., “Factors influencing the acceptance of BIM-based automated code compliance checking in the AEC industry in China,” J. Manage. Eng., vol. 39, no. 5, Art. no. 04023036, 2023.
J. Peng and X. Liu, “Automated code compliance checking research based on BIM and knowledge graph,” Sci. Rep., vol. 13, Art. no. 7065, 2023.
R. Zhang, “BIM, NLP, and AI for automated compliance checking,” in Proc. Comput. Civil Eng., Reston, VA, USA, 2022.
C. Zhang and N. M. El-Gohary, “Integrating semantic NLP and logic reasoning into a unified system for fully automated code checking,” Autom. Constr., vol. 73, pp. 45–57, 2017.
S. Li et al., “Automated compliance checking for BIM models based on Chinese NLP and knowledge graph: An integrative conceptual framework,” Eng. Constr. Archit. Manag., 2024.
P. Lewis et al., “Retrieval-augmented generation for knowledge-intensive NLP tasks,” in Proc. Adv. Neural Inf. Process. Syst. 33, Red Hook, NY, USA, 2020.
J. K. Lee et al., “High-level implementable methods for automated building code compliance checking,” Dev. Built Environ., vol. 15, Art. no. 100174, 2023.
N. Chen et al., “Automated building information modeling compliance check through a large language model combined with deep learning and ontology,” Buildings, vol. 14, no. 7, Art. no. 1983, 2024.
J. Peng and X. Liu, “Knowledge-graph-driven automated drawing review and BIM compliance checking,” Sci. Rep., vol. 13, Art. no. 7065, 2023.
S. Fuchs et al., “Using large language models for the interpretation of building regulations,” arXiv preprint arXiv:2407.21060, 2024. [Online]. Available: https://arxiv.org/abs/2407.21060.
S. Madireddy et al., “Large language model-driven code compliance checking in building information modeling,” Electronics, vol. 14, no. 11, Art. no. 2146, 2025.
X. Xu and H. Cai, “Ontology and rule-based natural language processing approach for interpreting textual regulations on underground utility infrastructure,” Adv. Eng. Inform., vol. 48, Art. no. 101288, 2021.
M. Aydin, “Building information modeling based automated building regulation compliance checking ASP.NET web software,” Intell. Autom. Soft Comput., vol. 28, no. 1, pp. 11–25, 2021.
Z. Zhang et al., “Towards fully automated code compliance checking of building regulations: challenges for rule interpretation and representation,” in Proc. 2022 Eur. Conf. Comput. Constr., Rhodes, Greece, 2022.
D. Kampelopoulos et al., “A review of LLMs and their applications in the architecture, engineering and construction industry,” Artif. Intell. Rev., vol. 58, no. 8, Art. no. 250, 2025.
Y. Zheng, “Evaluating the potential of large language models for building information modeling macro generation,” arXiv preprint arXiv:2307.11623, 2023. [Online]. Available: https://arxiv.org/abs/2307.11623.
S. Madireddy et al., “Large language model-driven code compliance checking in building information modeling,” Electronics, vol. 14, no. 11, Art. no. 2146, 2025.
G. Izacard and E. Grave, “Leveraging passage retrieval with generative models for open-domain question answering,” in Proc. 16th Conf. Eur. Chapter Assoc. Comput. Linguist., Stroudsburg, PA, USA, 2021, pp. 874– 880.
L. Gao et al., “Precise zero-shot dense retrieval without relevance labels,” in Proc. 61st Annu. Meet. Assoc. Comput. Linguist., Stroudsburg, PA, USA, 2023, pp. 1762–1777.
L. Wang et al., “Query2Doc: Query expansion with large language models,” in Proc. Conf. Empir. Methods Nat. Lang. Process., Stroudsburg, PA, USA, 2023.
A. Asai et al., “Self-RAG: Learning to retrieve, generate, and critique through self-reflection,” in Proc. Int. Conf. Learn. Represent., 2024.
Y. Zhou et al., “Trustworthiness in retrieval-augmented generation systems: A survey,” arXiv preprint arXiv:2409.10102, 2024. [Online]. Available: https://arxiv.org/abs/2409.10102.
J. Lee et al., “Performance comparison of retrieval-augmented generation and fine-tuned large language models for construction safety management knowledge retrieval,” Autom. Constr., vol. 168, Art. no. 105846, 2024.
J. Wei et al., “Chain-of-thought prompting elicits reasoning in large language models,” in Proc. Adv. Neural Inf. Process. Syst. 35, Red Hook, NY, USA, 2022.
D. Zhou et al., “Least-to-most prompting enables complex reasoning in large language models,” in Proc. Int. Conf. Learn. Represent., 2023.
T. Kojima et al., “Large language models are zero-shot reasoners,” in Proc. Adv. Neural Inf. Process. Syst. 35, Red Hook, NY, USA, 2022.
Q. Wu et al., “AutoGen: Enabling next-generation LLM applications via multi-agent conversation,” arXiv preprint arXiv:2308.08155, 2023. [Online]. Available: https://arxiv.org/abs/2308.08155.
S. Hong et al., “MetaGPT: Meta programming for a multi-agent collaborative framework,” in Proc. Int. Conf. Learn. Represent., 2024.
X. Liu et al., “AgentBench: Evaluating LLMs as agents,” in Proc. Int. Conf. Learn. Represent., 2024.
L. D. Brown, T. T. Cai and A. Dasgupta, “Interval estimation for a binomial proportion,” Stat. Sci., vol. 16, no. 2, pp. 101–133, 2001.
M. W. Fagerland et al., “Statistical analysis of contingency tables,” Boca Raton, FL, USA: CRC Press, 2017.