FastLane: Test Minimization for Rapidly Deployed Large-Scale Online Services

Year	Authors	Publisher	DOI
2019	Philip, Adithya Abraham Bhagwan, Ranjita Kumar, Rahul Maddila, Chandra Sekhar Nagppan, Nachiappan	IEEE	10.1109/ICSE.2019.00054

Year

Authors

Publisher

DOI

2019

Philip, Adithya Abraham Bhagwan, Ranjita Kumar, Rahul Maddila, Chandra Sekhar Nagppan, Nachiappan

IEEE

10.1109/ICSE.2019.00054

Techniques	Applicability
TSR	Industry Motivation Industry Evaluation Industry Author

Techniques

Applicability

TSR

Industry Motivation Industry Evaluation Industry Author

Experiment subject(s)	Industrial Partner	Programming Language
Microsoft Office 365 (tens of thousands of TCs) Industrial proprietary, very large scale	Microsoft (India)	C#

Experiment subject(s)

Industrial Partner

Programming Language

Microsoft Office 365 (tens of thousands of TCs) Industrial proprietary, very large scale

Microsoft (India)

Effectiveness Metrics	Efficiency Metrics	Other Metrics
Testing time, Accuracy/precision/recall

Effectiveness Metrics

Efficiency Metrics

Other Metrics

Testing time, Accuracy/precision/recall

Information Approach	Algorithm Approach	Open Challenges
	Machine learning-based	n/a

Information Approach

Algorithm Approach

Open Challenges

Machine learning-based

n/a

Abstract

Today, we depend on numerous large-scale services for basic operations such as email. These services, built on the basis of Continuous Integration/Continuous Deployment (CI/CD) processes, are extremely dynamic: developers continuously commit code and introduce new features, functionality and fixes. Hundreds of commits may enter the code-base in a single day. Therefore one of the most time-critical, yet resource-intensive tasks towards ensuring code-quality is effectively testing such large code-bases. This paper presents FastLane, a system that performs data-driven test minimization. FastLane uses light-weight machine-learning models built upon a rich history of test and commit logs to predict test outcomes. Tests for which we predict outcomes need not be explicitly run, thereby saving us precious test-time and resources. Our evaluation on a large-scale email and collaboration platform service shows that our techniques can save 18.04%, i.e., almost a fifth of test-time while obtaining a test outcome accuracy of 99.99%.