FSE 2016 All Events

24th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE 2016), November 13–18, 2016, Seattle, WA, USA

Desktop Layout

Session 16: Program Repair
Research Papers
Emerald 1, Chair: Tien Nguyen
Semi-supervised Verified Feedback Generation
Shalini Kaleeswaran, Anirudh Santhiar, Aditya Kanade, and Sumit Gulwani
(Indian Institute of Science, India; Microsoft Research, USA)
Publisher's Version
Abstract: Students have enthusiastically taken to online programming lessons and contests. Unfortunately, they tend to struggle due to lack of personalized feedback. There is an urgent need of program analysis and repair techniques capable of handling both the scale and variations in student submissions, while ensuring quality of feedback. Towards this goal, we present a novel methodology called emph{semi-supervised verified feedback generation}. We cluster submissions by solution strategy and ask the instructor to identify or add a correct submission in each cluster. We then verify every submission in a cluster against the instructor-validated submission in the same cluster. If faults are detected in the submission then feedback suggesting fixes to them is generated. Clustering reduces the burden on the instructor and also the variations that have to be handled during feedback generation. The verified feedback generation ensures that only correct feedback is generated. We implemented a tool, named CoderAssist, based on this approach and evaluated it on dynamic programming assignments. We have designed a novel emph{counter-example guided feedback generation} algorithm capable of suggesting fixes to all faults in a submission. In an evaluation on $2226$ submissions to $4$ problems, CoderAssist could generate emph{verified} feedback for $1911$ ($85%$) submissions in $1.6$s each on an average. It does a good job of reducing the burden on the instructor. Only one submission had to be manually validated or added for every $16$ submissions.

Authors:


Time stamp: 2019-03-26T21:22:35+01:00