Relevant Knowledge First - Reinforcement Learning and Forgetting in Knowledge Based Configuration

Abstract

In order to solve complex configuration tasks in technical domains, various knowledge based methods have been developed. However their applicability is often unsuccessful due to their low efficiency. One of the reasons for this is that (parts of the) problems have to be solved again and again, instead of being "learnt" from preceding processes. However, learning processes bring with them the problem of conservatism, for in technical domains innovation is a deciding factor in competition. On the other hand a certain amount of conservatism is often desired since uncontrolled innovation as a rule is also detrimental. This paper proposes the heuristic RKF (Relevant Knowledge First) for making decisions in configuration processes based on the so-called relevance of objects in a knowledge base. The underlying relevance-function has two components, one based on reinforcement learning and the other based on forgetting (fading). Relevance of an object increases with its successful use and decreases with age when it is not used. RKF has been developed to speed up the configuration process and to improve the quality of the solutions relative to the reward value that is given by users.