Improving AFL++ CmpLog: Tackling the bottlenecks

Abstract

The performance of the AFL++ CmpLog feature varies considerably for specific programs under test (PUTs). In this paper it is demonstrated that the main cause of the poor performance is low seed entropy, and a lack of deduplication of magic bytes candidates. An improvement is proposed by mapping comparisons to input bytes, in order to track which comparisons are controlled by what input bytes. This mapping is then used to fuzz only the comparison values that are magic byte candidates for that input part. Second, a caching mechanism is introduced to reduce the number of redundant executions. The evaluation of the improved versions shows a significant coverage gain compared to the original AFL++ implementation of CmpLog for all PUTs, without breaking functionality. The proposed solution in this paper provides a solid basis for a redesign of CmpLog.