2015 10th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE 2015), August 30 – September 4, 2015, Bergamo, Italy

Desktop Layout

Testing III
Research Papers
Oggioni, Chair: Justyna Petke
Guided Differential Testing of Certificate Validation in SSL/TLS Implementations
Yuting Chen and Zhendong Su
(Shanghai Jiao Tong University, China; University of California at Davis, USA)
Publisher's Version
Supplementary Material
Abstract: Certificate validation in SSL/TLS implementations is critical for Internet security. There is recent strong effort, namely frankencert, in automatically synthesizing certificates for stress-testing certificate validation. Despite its early promise, it remains a significant challenge to generate effective test certificates as they are structurally complex with intricate syntactic and semantic constraints. This paper tackles this challenge by introducing mucert, a novel, guided technique to much more effectively test real-world certificate validation code. Our core insight is to (1) leverage easily accessible Internet certificates as seed certificates, and (2) diversify them by adapting Markov Chain Monte Carlo (MCMC) sampling. The diversified certificates are then used to reveal discrepancies, thus potential flaws, among different certificate validation implementations. We have implemented mucert and extensively evaluated it against frankencert. Our experimental results show that mucert is significantly more cost-effective than frankencert. Indeed, 1K mucerts (i.e., mucert-mutated certificates) yield three times as many distinct discrepancies as 8M frankencerts (i.e., frankencert-synthesized certificates), and 200 mucerts can achieve higher code coverage than 100,000 frankencerts. This improvement is significant as it incurs much cost to test each generated certificate. We have analyzed and reported 20+ latent discrepancies (presumably missed by frankencert), and reported an additional 357 discrepancy-triggering certificates to SSL/TLS developers, who have already confirmed some of our reported issues and are investigating causes of all the reported discrepancies. In particular, our reports have led to bug fixes, active discussions in the community, and proposed changes to relevant IETF’s RFCs. We believe that mucert is practical and effective for helping improve the robustness of SSL/TLS implementations.


Time stamp: 2019-06-24T05:30:37+02:00