Symbolic Basic Block Profiling for Machine Learning Kernels

Abstract

Current basic block profiling techniques obtain the count of executions of each basic block in a program using dynamic instrumentation. These profiling counters create runtime overheads and also require the execution of the program, which, for large input sizes, can take substantial time. We propose symbolic program profiling that generates symbolic formulae for a basic block's count with inputs as the independent variables. Our technique is limited in applicability to a certain class of programs, namely machine learning (ML) kernels. We implement our technique in the LLVM compiler and evaluate it on 78 ML operators from 50 different ML models. These operators are generated by TVM, a machine learning compiler. Our symbolic profiles deliver exactly the same results as dynamic instrumentation for 73 out of 78 kernels with a median speedup of 15093x.