Using SAM to Extract Semantic Boundaries in Complex Artistic Images to Assist in the Optimization of Graphic Layout Design
Main Article Content
Abstract
Although the Segment Anything Model (SAM) exhibits strong capability in visual boundary extraction, its limited high-level semantic understanding restricts its application to complex artistic images and intelligent design-oriented scenarios. This limitation is particularly evident in tasks requiring semantic interpretation beyond low-level edge detection. To address this issue, this study proposes a semantically enhanced segmentation framework that integrates textual prompts with design priors to improve image decomposition and structural understanding. The proposed framework provides end-to-end support from semantic image analysis to layout generation through interactive optimization and hierarchical boundary construction. Experimental results demonstrate that the method achieves an mIoU of 0.78 and a pixel accuracy of 0.95 on the ADE20K validation set. For visual center recognition and text avoidance, which are critical indicators of layout quality, the proposed approach attains a visual overlap of 0.82 and a text avoidance accuracy of 0.91, with an expert evaluation score of 4.2. Compared with baseline methods, the framework delivers superior segmentation performance, layout rationality, and operational efficiency. Beyond graphic design applications, the proposed semantic enhancement strategy provides an effective computational paradigm for multimodal visual information processing and electromagnetic imaging interpretation, offering valuable methodological insights for intelligent sensing systems and advanced image analysis in engineering applications.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
C. Soreanu, “From media to mediums of expression: Visual art communication and meaning from fine arts to advertising,” Anastasis Research in Medieval Culture and Art, vol. 7, no. 2, pp. 261-276, 2020, doi: 10.35221/armca.2020.2.08.
N. Kolesnyk, O. Piddubna, O. Polishchuk, et al., “Digital art in designing an artistic image,” Ad Alta, vol. 31, no. 12, pp. 128-133, 2022, [Online]. Available: https://eprints.zu.edu.ua/id/eprint/36188.
L. Wang, “Machine learning-based environmental art automated design method,” Journal of Computational Methods in Sciences and Engineering, vol. 25, no. 2, pp. 1866-1879, 2025, doi: 10.1177/14727978241306041.
Y. Wu and K. Kim, “Automatic generation of traditional patterns and aesthetic quality evaluation technology,” Information Technology and Management, vol. 25, no. 2, pp. 125-143, 2024, doi: 10.1007/s10799-022-00356-w.
M. Vijendran, J. Deng, S. Chen, E. S. L. Ho, and H. P. H. Shum, “Artificial intelligence for geometry-based feature extraction, analysis and synthesis in artistic images: A survey,” Artificial Intelligence Review, vol. 58, no. 2, pp. 64, 2024, doi: 10.1007/s10462-024-11051-3.
Sakshi and V. Kukreja, “Image segmentation techniques: Statistical, comprehensive, semi-automated analysis and an application perspective analysis of mathematical expressions,” Archives of Computational Methods in Engineering, vol. 30, no. 1, pp. 457-495, 2023, doi: 10.1007/s11831-022-09805-9.
E. Song, D. Oh, and B. S. Oh, “Visual prompt selection framework for real-time object detection and interactive segmentation in augmented reality applications,” Applied Sciences, vol. 14, no. 22, Art. no. 10502, 2024, doi: 10.3390/app142210502.
P. Ni, X. Li, D. Kong, and X. Yin, “Scene-adaptive 3D semantic segmentation based on multi-level boundarysemantic-enhancement for intelligent vehicles,” IEEE Transactions on Intelligent Vehicles, vol. 9, no. 1, pp. 1722-1732, 2023, doi: 10.1109/TIV.2023.3274949.
Y. Zhang, M. Fjeld, M. Fratarcangeli, A. Said, and S. Zhao, “Affective colormap design for accurate visual comprehension in industrial tomography,” Sensors, vol. 21, no. 14, pp. 4766, 2021, doi: 10.3390/s21144766.
L. P. Yuan, Z. Zhou, J. Zhao, Y. Guo, F. Du, and H. Qu, “Infocolorizer: Interactive recommendation of color palettes for infographics,” IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 12, pp. 4252-4266, 2021, doi: 10.1109/TVCG.2021.3085327.
S. Chan, W. Zhou, Y. Lei, C. Li, J. Hu, and F. Hong, “Sparse point annotations for remote sensing image segmentation,” Scientific Reports, vol. 15, no. 1, Art. no. 27347, 2025, doi: 10.1038/s41598-025-12969-6.
T. Zhang, M. Zhu, Y. Ren, C. Wang, L. Zhang, W. Zhang, et al., “BPA-SAM: Box prompt augmented SAM for traditional Chinese realistic painting,” Journal of Graphics, vol. 46, no. 2, pp. 322, 2025, doi: 10.11996/JG.j.2095-302X.2025020322.
S. Liu, F. Wang, H. You, N. Jiao, G. Zhou, and T. Zhang, “Context-aggregated and SAM-guided network for ViTbased instance segmentation in remote sensing images,” Remote Sensing, vol. 16, no. 13, pp. 2472, 2024, doi: 10.3390/rs16132472.
Z. Wang, Y. Zhang, Z. Zhang, Z. Jiang, Y. Yu, L. Li, et al., “Exploring semantic prompts in the segment anything model for domain adaptation,” Remote Sensing, vol. 16, no. 5, pp. 758, 2024, doi: 10.3390/rs16050758.
C. Li, L. Qi, and X. Geng, “A SAM-guided two-stream lightweight model for anomaly detection,” ACM Transactions on Multimedia Computing, Communications and Applications, vol. 21, no. 2, pp. 1-23, 2025, doi: 10.1145/3706574.
K. Fan, L. Liang, H. Li, W. Situ, W. Zhao, and G. Li, “Research on medical image segmentation based on SAM and its future prospects,” Bioengineering, vol. 12, no. 6, pp. 608, 2025, doi: 10.3390/bioengineering12060608.
Z. Chen and Q. Sun, “Weakly-supervised semantic segmentation with image-level labels: From traditional models to foundation models,” ACM Computing Surveys, vol. 57, no. 5, pp. 1-29, 2025, doi: 10.1145/3707447.
F. Zhao, C. Zhang, and B. Geng, “Deep multimodal data fusion,” ACM Computing Surveys, vol. 56, no. 9, pp. 1-36, 2024, doi: 10.1145/3649447.
Y. E. Leichsenring and F. Baldo, “An evaluation of compression algorithms applied to moving object trajectories,” International Journal of Geographical Information Science, vol. 34, no. 3, pp. 539-558, 2020, doi: 10.1080/13658816.2019.1676430.
K. Mohamed and F. Adiloglu, “Analyzing the role of gestalt elements and design principles in logo and branding,” International Journal of Communication and Media Science, vol. 10, no. 2, pp. 33-43, 2023, doi: 10.14445/2349641X/IJCMS-V10I2P104.
R. Feng, H. Shen, J. Bai, and X. Li, “Advances and opportunities in remote sensing image geometric registration: A systematic review of state-of-the-art approaches and future research directions,” IEEE Geoscience and Remote Sensing Magazine, vol. 9, no. 4, pp. 120-142, 2021, doi: 10.1109/MGRS.2021.3081763.
D. Di Caprio, A. Ebrahimnejad, H. Alrezaamiri, et al., “A novel ant colony algorithm for solving shortest path problems with fuzzy arc weights,” Alexandria Engineering Journal, vol. 61, no. 5, pp. 3403-3415, 2022, doi: 10.1016/j.aej.2021.08.058.
Z. Wang, Y. Lv, R. Wu, and Y. Zhang, “Review of GrabCut in image processing,” Mathematics, vol. 11, no. 8, pp. 1965, 2023, doi: 10.3390/math11081965.
C. Wu, J. Zheng, Z. Feng, H. Zhang, L. Zhang, J. Cao, et al., “Fuzzy SLIC: Fuzzy simple linear iterative clustering,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 6, pp. 2114-2124, 2020, doi: 10.1109/TCSVT.2020.3019109.
Y. Wang, L. Yang, X. Liu, and P. Yan, “An improved semantic segmentation algorithm for high-resolution remote sensing images based on DeepLabv3+,” Scientific Reports, vol. 14, no. 1, pp. 9716, 2024, doi: 10.1038/s41598-024-60375-1.
S. Guo, Q. Yang, S. Xiang, S. Wang, and X. Wang, “Mask2Former with improved query for semantic segmentation in remote-sensing images,” Mathematics, vol. 12, no. 5, pp. 765, 2024, doi: 10.3390/math12050765.