Data I/O provision for Spark applications in a Mesos cluster

01 March 2016

New Image

At present there is a crucial need to take into account the I/O capabilities of commodity servers of clusters in production environments. In order to support such a demand in production environments, we propose a solution that can be integrated into Mesos to control I/O data throughput for Spark applications. The proposed solution takes into account the I/O capability to provide resource mapping, control and enforcement functionality for applications that process HDFS data. The aim is to minimize I/O contention situations that lead to the degradation of quality of service offered to Spark applications and clients.?