DLS '22: "Execution vs. Parse-Based ..."
Execution vs. Parse-Based Language Servers: Tradeoffs and Opportunities for Language-Agnostic Tooling for Dynamic Languages
Stefan Marr, Humphrey Burchell, and Fabio Niephaus
(University of Kent, UK; Oracle Labs, Germany; University of Potsdam, Germany; Hasso Plattner Institute, Germany)
With the wide adoption of the language server protocol, the desire to have IDE-style tooling even for niche and research languages has exploded. The Truffle language framework facilitates this desire by offering an almost zero-effort approach to language implementers to providing IDE features. However, this existing approach needs to execute the code being worked on to capture much of the information needed for an IDE, ideally with full unit-test coverage.
To capture information more reliably and avoid the need to execute the code being worked on, we propose a new parse-based design for language servers. Our solution provides a language-agnostic interface for structural information, with which we can support most common IDE features for dynamic languages.
Comparing the two approaches, we find that our new parse-based approach requires only a modest development effort for each language and has only minor tradeoffs for precision, for instance for code completion, compared to Truffle's execution-based approach.
Further, we show that less than 1,000 lines of code capture enough details to provide much of the typical IDE functionality, with an order of magnitude less code than ad hoc language servers. We tested our approach for the custom parsers of Newspeak and SOM, as well as SimpleLanguage's ANTLR grammar without any changes to it. Combining both parse and execution-based approaches has the potential to provide good and precise IDE tooling for a wide range of languages with only small development effort. By itself, our approach would be a good addition to the many libraries implementing the language server protocol to enable low-effort implementations of IDE features.
@InProceedings{DLS22p1,
author = {Stefan Marr and Humphrey Burchell and Fabio Niephaus},
title = {Execution vs. Parse-Based Language Servers: Tradeoffs and Opportunities for Language-Agnostic Tooling for Dynamic Languages},
booktitle = {Proc.\ DLS},
publisher = {ACM},
pages = {1--14},
doi = {10.1145/3563834.3567537},
year = {2022},
}
Publisher's Version
Info
DLS '22: "Who You Gonna Call: Analyzing ..."
Who You Gonna Call: Analyzing the Run-Time Call-Site Behavior of Ruby Applications
Sophie Kaleba, Octave Larose, Richard Jones, and Stefan Marr
(University of Kent, UK)
Applications written in dynamic languages are becoming larger and larger and companies increasingly use multi-million line codebases in production. At the same time, dynamic languages rely heavily on dynamic optimizations, particularly those that reduce the overhead of method calls. In this work, we study the call-site behavior of Ruby benchmarks that are being used to guide the development of upcoming Ruby implementations such as TruffleRuby and YJIT. We study the interaction of call-site lookup caches, method splitting, and elimination of duplicate call-targets. We find that these optimizations are indeed highly effective on both smaller and large benchmarks, methods and closures alike, and help to open up opportunities for further optimizations such as inlining. However, we show that TruffleRuby’s splitting may be applied too aggressively on already-monomorphic call-sites, coming at a run-time cost. We also find three distinct patterns in the evolution of call-site behavior over time, which may help to guide novel optimizations. We believe that our results may support language implementers in optimizing runtime systems for large code-bases built in dynamic languages.
@InProceedings{DLS22p15,
author = {Sophie Kaleba and Octave Larose and Richard Jones and Stefan Marr},
title = {Who You Gonna Call: Analyzing the Run-Time Call-Site Behavior of Ruby Applications},
booktitle = {Proc.\ DLS},
publisher = {ACM},
pages = {15--28},
doi = {10.1145/3563834.3567538},
year = {2022},
}
Publisher's Version
DLS '22: "Who You Gonna Call: Analyzing ..."
Who You Gonna Call: Analyzing the Run-Time Call-Site Behavior of Ruby Applications
Sophie Kaleba, Octave Larose, Richard Jones, and Stefan Marr
(University of Kent, UK)
Applications written in dynamic languages are becoming larger and larger and companies increasingly use multi-million line codebases in production. At the same time, dynamic languages rely heavily on dynamic optimizations, particularly those that reduce the overhead of method calls. In this work, we study the call-site behavior of Ruby benchmarks that are being used to guide the development of upcoming Ruby implementations such as TruffleRuby and YJIT. We study the interaction of call-site lookup caches, method splitting, and elimination of duplicate call-targets. We find that these optimizations are indeed highly effective on both smaller and large benchmarks, methods and closures alike, and help to open up opportunities for further optimizations such as inlining. However, we show that TruffleRuby’s splitting may be applied too aggressively on already-monomorphic call-sites, coming at a run-time cost. We also find three distinct patterns in the evolution of call-site behavior over time, which may help to guide novel optimizations. We believe that our results may support language implementers in optimizing runtime systems for large code-bases built in dynamic languages.
@InProceedings{DLS22p15,
author = {Sophie Kaleba and Octave Larose and Richard Jones and Stefan Marr},
title = {Who You Gonna Call: Analyzing the Run-Time Call-Site Behavior of Ruby Applications},
booktitle = {Proc.\ DLS},
publisher = {ACM},
pages = {15--28},
doi = {10.1145/3563834.3567538},
year = {2022},
}
Publisher's Version
DLS '22: "Who You Gonna Call: Analyzing ..."
Who You Gonna Call: Analyzing the Run-Time Call-Site Behavior of Ruby Applications
Sophie Kaleba, Octave Larose, Richard Jones, and Stefan Marr
(University of Kent, UK)
Applications written in dynamic languages are becoming larger and larger and companies increasingly use multi-million line codebases in production. At the same time, dynamic languages rely heavily on dynamic optimizations, particularly those that reduce the overhead of method calls. In this work, we study the call-site behavior of Ruby benchmarks that are being used to guide the development of upcoming Ruby implementations such as TruffleRuby and YJIT. We study the interaction of call-site lookup caches, method splitting, and elimination of duplicate call-targets. We find that these optimizations are indeed highly effective on both smaller and large benchmarks, methods and closures alike, and help to open up opportunities for further optimizations such as inlining. However, we show that TruffleRuby’s splitting may be applied too aggressively on already-monomorphic call-sites, coming at a run-time cost. We also find three distinct patterns in the evolution of call-site behavior over time, which may help to guide novel optimizations. We believe that our results may support language implementers in optimizing runtime systems for large code-bases built in dynamic languages.
@InProceedings{DLS22p15,
author = {Sophie Kaleba and Octave Larose and Richard Jones and Stefan Marr},
title = {Who You Gonna Call: Analyzing the Run-Time Call-Site Behavior of Ruby Applications},
booktitle = {Proc.\ DLS},
publisher = {ACM},
pages = {15--28},
doi = {10.1145/3563834.3567538},
year = {2022},
}
Publisher's Version
DLS '22: "Execution vs. Parse-Based ..."
Execution vs. Parse-Based Language Servers: Tradeoffs and Opportunities for Language-Agnostic Tooling for Dynamic Languages
Stefan Marr, Humphrey Burchell, and Fabio Niephaus
(University of Kent, UK; Oracle Labs, Germany; University of Potsdam, Germany; Hasso Plattner Institute, Germany)
With the wide adoption of the language server protocol, the desire to have IDE-style tooling even for niche and research languages has exploded. The Truffle language framework facilitates this desire by offering an almost zero-effort approach to language implementers to providing IDE features. However, this existing approach needs to execute the code being worked on to capture much of the information needed for an IDE, ideally with full unit-test coverage.
To capture information more reliably and avoid the need to execute the code being worked on, we propose a new parse-based design for language servers. Our solution provides a language-agnostic interface for structural information, with which we can support most common IDE features for dynamic languages.
Comparing the two approaches, we find that our new parse-based approach requires only a modest development effort for each language and has only minor tradeoffs for precision, for instance for code completion, compared to Truffle's execution-based approach.
Further, we show that less than 1,000 lines of code capture enough details to provide much of the typical IDE functionality, with an order of magnitude less code than ad hoc language servers. We tested our approach for the custom parsers of Newspeak and SOM, as well as SimpleLanguage's ANTLR grammar without any changes to it. Combining both parse and execution-based approaches has the potential to provide good and precise IDE tooling for a wide range of languages with only small development effort. By itself, our approach would be a good addition to the many libraries implementing the language server protocol to enable low-effort implementations of IDE features.
@InProceedings{DLS22p1,
author = {Stefan Marr and Humphrey Burchell and Fabio Niephaus},
title = {Execution vs. Parse-Based Language Servers: Tradeoffs and Opportunities for Language-Agnostic Tooling for Dynamic Languages},
booktitle = {Proc.\ DLS},
publisher = {ACM},
pages = {1--14},
doi = {10.1145/3563834.3567537},
year = {2022},
}
Publisher's Version
Info
DLS '22: "Who You Gonna Call: Analyzing ..."
Who You Gonna Call: Analyzing the Run-Time Call-Site Behavior of Ruby Applications
Sophie Kaleba, Octave Larose, Richard Jones, and Stefan Marr
(University of Kent, UK)
Applications written in dynamic languages are becoming larger and larger and companies increasingly use multi-million line codebases in production. At the same time, dynamic languages rely heavily on dynamic optimizations, particularly those that reduce the overhead of method calls. In this work, we study the call-site behavior of Ruby benchmarks that are being used to guide the development of upcoming Ruby implementations such as TruffleRuby and YJIT. We study the interaction of call-site lookup caches, method splitting, and elimination of duplicate call-targets. We find that these optimizations are indeed highly effective on both smaller and large benchmarks, methods and closures alike, and help to open up opportunities for further optimizations such as inlining. However, we show that TruffleRuby’s splitting may be applied too aggressively on already-monomorphic call-sites, coming at a run-time cost. We also find three distinct patterns in the evolution of call-site behavior over time, which may help to guide novel optimizations. We believe that our results may support language implementers in optimizing runtime systems for large code-bases built in dynamic languages.
@InProceedings{DLS22p15,
author = {Sophie Kaleba and Octave Larose and Richard Jones and Stefan Marr},
title = {Who You Gonna Call: Analyzing the Run-Time Call-Site Behavior of Ruby Applications},
booktitle = {Proc.\ DLS},
publisher = {ACM},
pages = {15--28},
doi = {10.1145/3563834.3567538},
year = {2022},
}
Publisher's Version
DLS '22: "Execution vs. Parse-Based ..."
Execution vs. Parse-Based Language Servers: Tradeoffs and Opportunities for Language-Agnostic Tooling for Dynamic Languages
Stefan Marr, Humphrey Burchell, and Fabio Niephaus
(University of Kent, UK; Oracle Labs, Germany; University of Potsdam, Germany; Hasso Plattner Institute, Germany)
With the wide adoption of the language server protocol, the desire to have IDE-style tooling even for niche and research languages has exploded. The Truffle language framework facilitates this desire by offering an almost zero-effort approach to language implementers to providing IDE features. However, this existing approach needs to execute the code being worked on to capture much of the information needed for an IDE, ideally with full unit-test coverage.
To capture information more reliably and avoid the need to execute the code being worked on, we propose a new parse-based design for language servers. Our solution provides a language-agnostic interface for structural information, with which we can support most common IDE features for dynamic languages.
Comparing the two approaches, we find that our new parse-based approach requires only a modest development effort for each language and has only minor tradeoffs for precision, for instance for code completion, compared to Truffle's execution-based approach.
Further, we show that less than 1,000 lines of code capture enough details to provide much of the typical IDE functionality, with an order of magnitude less code than ad hoc language servers. We tested our approach for the custom parsers of Newspeak and SOM, as well as SimpleLanguage's ANTLR grammar without any changes to it. Combining both parse and execution-based approaches has the potential to provide good and precise IDE tooling for a wide range of languages with only small development effort. By itself, our approach would be a good addition to the many libraries implementing the language server protocol to enable low-effort implementations of IDE features.
@InProceedings{DLS22p1,
author = {Stefan Marr and Humphrey Burchell and Fabio Niephaus},
title = {Execution vs. Parse-Based Language Servers: Tradeoffs and Opportunities for Language-Agnostic Tooling for Dynamic Languages},
booktitle = {Proc.\ DLS},
publisher = {ACM},
pages = {1--14},
doi = {10.1145/3563834.3567537},
year = {2022},
}
Publisher's Version
Info