OOPSLAB – Journal Issue |
Contents -
Abstracts -
Authors
|
cartokit direct manipulation programming system (cartokitDM+NL). Our key insight is to treat programs as sequences of structured edits and to use an edit language as a shared interface for both direct manipulation and natural language interactions, leveraging constrained decoding to support the latter. Using our instantiation, we conducted a within-subjects study (N=18) to understand how the combination of direct manipulation and natural language as editing modalities changes the programming process compared to each modality alone. Perhaps surprisingly, we found that study participants overwhelmingly chose to edit via direct manipulation when both modalities were available, performing just 6.14% of edits via natural language. Our thematic analysis of study sessions revealed that direct manipulation aided task decomposition, encouraged incremental editing, and helped mitigate known challenges in natural language programming related to understanding model capabilities and interpreting model-generated code. Conversely, natural language editing came into play largely to automate, parameterize, and replay known edits that would otherwise be repeated tediously by hand. Our edit-based framework and study findings lay out a possible pathway for future research on programming systems that blend natural language with alternative editing modalities, building on the foundation of edit languages.
proc time: 2.46