Welfare properties of profit maximizing recommender systems: theory and results from a randomized experiment

Recommender systems have been introduced to help consumers navigate large sets of alternatives. They usually lead to more sales, which may increase consumer surplus and firm profit. In this paper, we ask whether the firms’ choice of recommender system might hurt consumers. We use data from a large-scale field experiment in video-on-demand to measure the price elasticity of demand for movies placed in salient and non-salient slots on the TV screen. During this experiment, the firm randomized the prices and slots in which movies were recommended to consumers. This setting readily allows for identifying the effects of price and slot on demand, and thus computes consumer surplus. We find that consumers are less price-elastic toward movies placed in salient slots. Using the outcomes of this experiment, we simulate how consumer surplus and welfare change when the firm implements several types of recommender systems, namely one that maximizes profit. We show that this system hurts consumer surplus and welfare relative to a system that maximizes welfare. We also show that, in our setting, the system that maximizes profit does not generate less consumer surplus than several recommender systems frequently used in practice. Yet, the amount of extra rent the firm can extract from strategically placing movies in salient slots is still a function of the popularity and quality of movies used to do so. Ultimately, our results question whether recommender systems embed mechanisms that extract excessive surplus from consumers, which may call for better scrutiny.