GUI-WORLD: A Dataset for GUI-oriented Multimodal LLM-based Agents
GUI-WORLD: A Dataset for GUI-oriented Multimodal LLM-based Agents
Recently, Multimodal Large Language Models (MLLMs) have been used as agents to control keyboard and mouse inputs by directly perceiving the Graphical User Interface (GUI) and generating corresponding code. However, current agents primarily exhibit excellent understanding capabilities in static environments and are predominantly applied in relatively simple domains, such as …