Automatic World Knowledge-Informed Cloud Distribution of Real-time Streaming Services
22 January 2018
With cloud-integrated networks becoming the distributed canvas for ever more ad-hoc instantiated, hyperpersonalized data streaming services, the deployment of such services as instantiations of data flow graphs composed of streams and stream processing operators is the dominant approach of choice. In this paper, we elaborate a novel method to automatically optimize data streaming service deployments by leveraging probabilistic data and relational knowledge of the surrounding physical world, leading to improved service distributions in terms of stream processing operator selection, placement, and interconnection. After a system overview and discussion of the probabilistic world modeling approach, we formulate the operator selection, placement, and interconnection problem as a minimum cost network flow problem, whose solution leverages stream flow sizing estimates from the probabilistic world model to generate cost-efficient service implementations. We wrap up the paper with an experimental validation of the overall method in a real-world service use case.
