Functional or imperative? On pleasant semantics for differentiable programming languages

Abstract

In machine learning (ML), researchers and engineers seem to be at odds. System implementers would prefer models to be declarative, with detailed type information and semantic restrictions that allow models to be optimised, rearranged and parallelised. Yet practitioners show an overwhelming preference for dynamic, imperative languages with mutable state, and much engineering effort is spent bridging the resulting semantic divide. Is there a fundamental conflict? This article explores why imperative and functional styles are used, and how future language designs might get the best of both worlds.