Efficiently handling data schema changes in an event sourced system

Abstract

Data schema changes occur as a result of evolving software systems. This is a well-known problem and for relational data models there is a large body of research that addresses this problem. However, this research does not apply to event sourced systems, because in those systems there is no relational model and often there is no explicit data schema. A second problem is that in event sourced system the amount of data is much larger than in conventional systems, because the system does not only store the current state but every event that changed the state. In this research, a framework is created to illustrate which schema changes are to be expected, reflected by operations, executed with data transformation techniques and deployed according to a deployment strategies to execute these operations. When creating the framework, efficiency including zero downtime was. When completed, the framework was evaluated with three Dutch experts. The final framework provides a starting point for developers that are currently building an event sourced system and that are searching for a solution for the problems of the changing data schema.