Training and Testing of Recommender Systems on Data Missing not at Random

Users typically rate only a small fraction of all possible items. We show that the absence of ratings carries useful information for improving the top-k hit rate, a natural accuracy measure for recommendations. For testing recommender systems, we present two performance measures that are closely related to the top-k hit rate and, under mild assumptions, can be estimated without bias from data even when ratings are missing not at random (MNAR). As to achieve optimal test results, we develop appropriate surrogate objective functions for training on MNAR data. Their main property is to account for all ratings whether observed or missing in the data. Concerning the top-k hit rate on test data, our experiments indicate dramatic improvements over even sophisticated methods that optimize the popular root mean squared error on the observed ratings.