Recommender Systems II — So, What’s Not to Love?

36-462/662, Data Mining, Fall 2019

Lecture 13/14 (9/14 October 2019)

\[ \newcommand{\Expect}[1]{\mathbb{E}\left[ #1 \right]} \]

Recommender systems are everywhere these days

What happens now that they’re everywhere?

Homogenization (I)

Homogenization (II)

Homogenization (II), cont’d

Homogenization (II), cont’d

Rabbit holes, feedback loops, echo chambers

Feedback loops, echo chambers


Discrimination (cont’d)

Why might this be a problem?

What does the system actually optimize for?

What does the system actually optimize for?

  1. What’s the expected revenue for recommending item \(k\)?
  2. When will the system owner prefer to recommend item \(k\) rather than item \(l\), even though \(p_{ik} < p_{il}\)?

What does the system actually optimize for?


  1. Expected revenue is \(q r_k p_{ik}\)

  2. Prefers item \(k\) to item \(l\) when expected revenue is higher, \[ q r_k p_{ik} > q r_l p_{il} ~ \Rightarrow ~ \frac{r_k}{r_l} > \frac{p_{il}}{p_{ik}} \]

Do recommendation systems do anything?

What would a successful recommendation look like?

How do we answer causal questions?

Summing up

Backup: “Engagement”


(Prof. Rossman is joking, but he’s also an excellent sociologist of mass media and social diffusion, so this isn’t entirely a joke)

Backup: Increasing returns to scale

(If you want to learn to think this way, Shapiro and Varian (1998) is old but still excellent)

References (in addition to the background reading on the course homepage)

Rubin, Donald B., and Richard P. Waterman. 2006. “Estimating the Causal Effects of Marketing Interventions Using Propensity Score Methodology.” Statistical Science 21:206–22.

Salganik, Matthew J., Peter S. Dodds, and Duncan J. Watts. 2006. “Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market.” Science 311:854–56.

Salganik, Matthew J., and Duncan J. Watts. 2008. “Leading the Herd Astray: An Experimental Study of Self-Fulfilling Prophecies in an Artificial Cultural Market.” Social Psychological Quarterly 71:338–55.

Shapiro, Carl, and Hal R. Varian. 1998. Information Rules: A Strategic Guide to the Network Economy. First. Boston: Harvard Business School Press.

Shardanand, Upendra, and Pattie Maes. 1995. “Social Information Filtering: Algorithms for Automating ‘Word of Mouth’.” In Proceedings of ACM CHI’95 Conference on Human Factors in Computing Systems, 1:210–17. New York: ACM Press.