Formalising the intentional stance 2: a coinductive approach

Abstract

Given a stochastic process with inputs and outputs, how might its behaviour be related to pursuit of a goal? We model this using 'transducers', objects that capture only the external behaviour of a system and not its internal state. A companion paper summarises our results for cognitive scientists; the current paper gives formal definitions and proofs. To formalise the concept of a system that behaves as if it were pursuing a goal, we consider what happens when a transducer (a 'policy') is coupled to another transducer that comes equipped with a success condition (a 'teleo-environment'). An optimal policy is identified with a transducer that behaves as if it were perfectly rational in the pursuit of a goal; our framework also allows us to model constrained rationality. Optimal policies obey a version of Bellman's principle: a policy that's optimal in one time step will again be optimal in the next time step, but with respect to a different teleo-environment (obtained from the original one by a modified version of Bayesian filtering). This property sometimes also applies to the bounded-rational case; we give a sufficient condition. A policy is deterministic if and only if there exists a teleo-environment for which it is uniquely optimal among the set of all policies; we relate this to classical representation theorems from decision theory. This result need not hold in the bounded-rational case; we give an example related to the absent-minded driver problem. The formalism is defined using coinduction, following the style proposed by Czajka.