HelpViz: Automatic Generation of Contextual Visual MobileTutorials from Text-Based Instructions

Mingyuan Zhong, Gang Li, Peggy Chi, Yang Li

Type: Preprint

Publication Date: 2021-01-01

Citations: 0

DOI: https://doi.org/10.48550/arxiv.2108.03356

View Publication

Locations

arXiv (Cornell University) - View - PDF
DataCite API - View

Similar Works

Action	Title	Year	Authors
+	HelpViz: Automatic Generation of Contextual Visual Mobile Tutorials from Text-Based Instructions	2021	Mingyuan Zhong Gang Li Peggy Chi Yang Li
+ PDF Chat	Understanding mobile GUI: From pixel-words to screen-sentences	2024	Jingwen Fu Xiaoyi Zhang Yuwang Wang Wenjun Zeng Nanning Zheng
+	SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents	2024	Kanzhi Cheng Qiushi Sun Yougang Chu Fangzhi Xu Yantao Li Jianbing Zhang Zhiyong Wu
+ PDF Chat	Inferring Alt-text For UI Icons With Large Language Models During App Development	2024	Sabrina Haque Christoph Csallner
+ PDF Chat	Bridging Design and Development with Automated Declarative UI Code Generation	2024	Ting Zhou Yanjie Zhao Xinyi Hou Xiaoyu Sun Kai Chen H. Wang
+	GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation	2023	An Yan Zhengyuan Yang Wanrong Zhu Kevin Lin Linjie Li Jianfeng Wang Jianwei Yang Yiwu Zhong Julian McAuley Jianfeng Gao
+ PDF Chat	Vision-driven Automated Mobile GUI Testing via Multimodal Large Language Model	2024	Zhe Liu Cheng Li Chunyang Chen Junjie Wang Boyu Wu Yawen Wang Jun Hu Qing Wang
+ PDF Chat	MobileViews: A Large-Scale Mobile GUI Dataset	2024	Longxi Gao Li Zhang Shihe Wang Shangguang Wang Yuanchun Li Mengwei Xu
+ PDF Chat	VideoGUI: A Benchmark for GUI Automation from Instructional Videos	2024	Kevin Qinghong Lin Linjie Li Difei Gao Qinchen WU Mingyi Yan Zhengyuan Yang Lijuan Wang Mike Zheng Shou
+ PDF Chat	Tell Me What's Next: Textual Foresight for Generic UI Representations	2024	Andrea Burns Kate Saenko Bryan A. Plummer
+	Unblind Text Inputs: Predicting Hint-text of Text Input in Mobile Apps via LLM	2024	Zhe Liu Chunyang Chen Junjie Wang M. Chen Boyu Wu Yue-Kai Huang Jun Hu Qing Wang
+	Extracting Replayable Interactions from Videos of Mobile App Usage	2022	Jieshan Chen Amanda Swearngin Jason Wu Titus Barik Jeffrey Nichols Xiaoyi Zhang
+ PDF Chat	Beyond Pass or Fail: A Multi-dimensional Benchmark for Mobile UI Navigation	2025	Dezhi Ran Ming C. Wu Hao Yu Yuetong Li Jun Ren Yuan Cao Xia Zeng Haochuan Lu Zizhen Xu M. Xu
+ PDF Chat	Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding	2024	Yue Fan Lei Ding Ching-Chen Kuo Shan Jiang Yang Zhao Xinze Guan Jie Yang Yi Zhang Xin Wang
+	Understanding Mobile GUI: from Pixel-Words to Screen-Sentences	2021	Jingwen Fu Xiaoyi Zhang Yuwang Wang Wenjun Zeng Sam Yang Grayson Hilliard
+ PDF Chat	Fill in the Blank: Context-aware Automated Text Input Generation for Mobile GUI Testing	2023	Zhe Liu Chunyang Chen Junjie Wang Xing Che Yue-Kai Huang Jun Hu Qing Wang
+	Fill in the Blank: Context-aware Automated Text Input Generation for Mobile GUI Testing	2022	Zhe Liu Chunyang Chen Junjie Wang Xing Che Yue-Kai Huang Jun Hu Qing Wang
+	Towards Better Semantic Understanding of Mobile Interfaces	2022	Srinivas Sunkara Maria Wang Lijuan Liu Gilles Baechler Yu-Chung Hsiao Jindong Chen Abhanshu Sharma James W. Stout
+ PDF Chat	Automatically Distilling Storyboard With Rich Features for Android Apps	2022	Sen Chen Lingling Fan Chunyang Chen Yang Liu
+	WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics	2023	Jason Wu Siyan Wang Siman Shen Yi‐Hao Peng Jeffrey Nichols Jeffrey P. Bigham

Works That Cite This (0)

Action	Title	Year	Authors

Works Cited by This (1)

Action	Title	Year	Authors
+	Mapping Natural Language Instructions to Mobile UI Action Sequences	2020	Yang Li Jiacong He Xin Zhou Yuan Zhang Jason Baldridge