From Mock-Ups to IFML-Like GUI Models: Using Large Language Models in Web Engineering

Authors

Nirumand A., Cabot J.

Reference

Lecture Notes in Computer Science, vol. 15749 LNCS, pp. 253-268, 2026

Description

The development of web applications often requires the translation of design mock-ups into functional code, a process that is time-consuming and inaccessible to those without technical expertise. This research addresses this challenge by leveraging Large Language Models (LLMs) to automate the derivation of IFML-like Graphical User Interface (GUI) models from mock-up images. These models serve as a foundation for the validation and subsequent generation of web applications using available deterministic code-generators. By enhancing the BESSER low-code framework with this extension, we provide an advanced pipeline that integrates LLMs into model-driven web application development relying on the UML (for the structural models) and IFML (for the GUI ones) standards. Our LLM-based approach achieves high accuracy for both digital and hand-drawn mock-ups, demonstrates efficiency with rapid run-time performance, and delivers web pages that have a high structural similarity to their corresponding mock-ups. The complete infrastructure is available in an open-source repository.

Link

doi:10.1007/978-3-031-97207-2_20