WebRobot: Web Robotic Process Automation using Interactive Programming-by-Demonstration
Thu 16 Jun 2022 01:30 - 01:50 at Kon-Tiki - Synthesis I
It is imperative to democratize robotic process automation (RPA), as RPA has become a main driver of the digital transformation but is still technically very demanding to construct, especially for non-experts. In this paper, we study how to automate an important class of RPA tasks, dubbed web RPA, which are concerned with constructing software bots that automate interactions across data and a web browser. Our main contributions are twofold. First, we develop a formal foundation which allows semantically reasoning about web RPA programs and formulate its synthesis problem in a principled manner. Second, we propose a web RPA program synthesis algorithm based on a new idea called speculative rewriting. This leads to a novel speculate-and-validate methodology in the context of rewrite-based program synthesis, which has also shown to be both theoretically simple and practically efficient for synthesizing programs from demonstrations. We have built these ideas in a new interactive synthesizer called WebRobot and evaluate it on 76 web RPA benchmarks. Our results show that WebRobot automated a majority of them effectively. Furthermore, we show that WebRobot compares favorably with a conventional rewrite-based synthesis baseline implemented using egg. Finally, we conduct a small user study demonstrating WebRobot is also usable.
Wed 15 JunDisplayed time zone: Pacific Time (US & Canada) change
13:30 - 14:50 | |||
13:30 20mTalk | WebRobot: Web Robotic Process Automation using Interactive Programming-by-Demonstration PLDI Rui Dong University of Michigan, Zhicheng Huang University of Michigan, Ian Iong Lam University of Michigan, Yan Chen University of Michigan, Xinyu Wang University of Michigan DOI Pre-print | ||
13:50 20mTalk | Synthesizing Analytical SQL Queries from Computation DemonstrationDistinguished Paper Award PLDI Xiangyu Zhou University of Washington, Rastislav Bodík University of Washington, Alvin Cheung University of California, Berkeley, Chenglong Wang University of Washington, USA DOI | ||
14:10 20mTalk | Type-Directed Program Synthesis for RESTful APIs PLDI Zheng Guo University of California, San Diego, David Cao University of California, San Diego, Davin Tjong University of California, San Diego, Jean Yang Akita Software, Cole Schlesinger Akita Software, Nadia Polikarpova University of California at San Diego DOI | ||
14:30 20mTalk | Visualization Question Answering Using Introspective Program SynthesisDistinguished Paper Award PLDI Yanju Chen University of California, Santa Barbara, Xifeng Yan University of California, Santa Barbara, Yu Feng University of California, Santa Barbara DOI |
Thu 16 JunDisplayed time zone: Pacific Time (US & Canada) change
01:30 - 02:50 | |||
01:30 20mTalk | WebRobot: Web Robotic Process Automation using Interactive Programming-by-Demonstration PLDI Rui Dong University of Michigan, Zhicheng Huang University of Michigan, Ian Iong Lam University of Michigan, Yan Chen University of Michigan, Xinyu Wang University of Michigan DOI Pre-print | ||
01:50 20mTalk | Synthesizing Analytical SQL Queries from Computation DemonstrationDistinguished Paper Award PLDI Xiangyu Zhou University of Washington, Rastislav Bodík University of Washington, Alvin Cheung University of California, Berkeley, Chenglong Wang University of Washington, USA DOI | ||
02:10 20mTalk | Type-Directed Program Synthesis for RESTful APIs PLDI Zheng Guo University of California, San Diego, David Cao University of California, San Diego, Davin Tjong University of California, San Diego, Jean Yang Akita Software, Cole Schlesinger Akita Software, Nadia Polikarpova University of California at San Diego DOI | ||
02:30 20mTalk | Visualization Question Answering Using Introspective Program SynthesisDistinguished Paper Award PLDI Yanju Chen University of California, Santa Barbara, Xifeng Yan University of California, Santa Barbara, Yu Feng University of California, Santa Barbara DOI |