The Machines That Taught Us What to Want

The term collaborative filtering dates to Goldberg, Nichols, Oki and Terry's 1992 Communications of the ACM paper describing Xerox PARC's "Tapestry." The Netflix Prize (2006–2009) made matrix factorization — decomposing the sparse user–item rating matrix into low-dimensional latent factors — the industry default. Modern systems are hybrids: collaborative signals, content embeddings, sequence models, and real-time contextual features operating together in milliseconds.

Each major platform's architecture reflects what it is paid to maximize. YouTube's published system (Covington, Adams and Sargin, RecSys 2016) optimizes expected watch time per impression, not click-through rate — a distinction with large downstream consequences. YouTube's Chief Product Officer confirmed at CES 2018 that more than 70% of watch time comes from recommendations. Netflix (Gomez-Uribe and Hunt, ACM TMIS 2015) openly states its optimization target is "improved member retention" proxied through hours streamed, and values the system at "more than $1 billion per year" in retained subscribers. TikTok's leaked internal "Algo 101" document exposed a simplified scoring formula weighting likes, comments, playtime, and completions — with "retention" and "time spent" as the company's ultimate north-star metrics. Meta's optimization target since January 2018 has been "Meaningful Social Interactions" — a weighted composite that the Facebook Files revealed weighted angry-emoji reactions 5× a Like.

These systems share ML infrastructure with but are architecturally distinct from the ad-tech pipeline. Content recommenders are single-tenant rankers running in 50–300 milliseconds. Real-time bidding is a multi-party auction clearing inside a 100-millisecond budget, governed by the IAB's OpenRTB specification, in which supply-side platforms broadcast impression opportunities with behavioral metadata to dozens or hundreds of demand-side platforms. Large platforms run both stacks simultaneously — recommender engagement generates the inventory that the ad auction monetizes.

The central critique, articulated by Jonathan Stray at Berkeley's Center for Human-Compatible AI and by Stuart Russell in Human Compatible (2019), is a Goodhart's Law problem: engagement is a proxy for value, and once a proxy becomes a target, a sufficiently capable optimizer will achieve it by modifying the user — making them angrier, more addicted, more predictable — rather than by genuinely serving them. Haugen's disclosures are empirical confirmation: Meta's own documents describe MSI rewarding outrage despite a publicly stated goal of "time well spent."

If the recommender is the visible front of the system, the ad-tech stack is its balance sheet. The Irish Council for Civil Liberties estimates 178 trillion real-time-bidding broadcasts per year across the U.S. and Europe. The average American's data is exposed 747 times per day. Google alone broadcasts user information 42 billion times per day in Europe and 31 billion in the U.S., allowing 4,698 companies to receive bid data on U.S. users. Acxiom has publicly claimed 3,000+ "propensities" on nearly every U.S. consumer. The FTC's 2014 "Data Brokers: A Call for Transparency" named categories sold including "Rural and Barely Making It," "diabetes interest," "pregnancy," and "financially vulnerable." In August 2022 the FTC sued Kochava for selling precise location data tracking visits to reproductive health clinics, houses of worship, addiction recovery facilities, and domestic-violence shelters.

Meta's targeting capabilities have been documented by both the company and its critics. A 2017 leaked internal document from Facebook Australia pitched advertisers on the platform's ability to identify teenagers' emotional states — "worthless," "insecure," "defeated," "anxious," "stressed," "useless," "overwhelmed," "failure" — among 6.4 million young people, some as young as 14. Matz, Kosinski and colleagues' PNAS 2017 paper showed psychologically tailored ads derived from Facebook likes increased clicks by up to 40% and conversions by 50% compared with generic copy — a genuine peer-reviewed result and the most defensible version of the Cambridge Analytica thesis, though Cambridge Analytica's actual election impact is now widely regarded as marketing hype.

Dark patterns are now empirically measured. Luguri and Strahilevitz's "Shining a Light on Dark Patterns" (Journal of Legal Analysis, 2021) found that mild dark patterns made subjects 2.1× more likely to stay enrolled in a dubious product, while aggressive patterns produced nearly 4× the enrollment of controls. Less-educated subjects were systematically more susceptible. The FTC has since extracted $245 million from Epic Games and $100 million from Vonage for dark patterns.

The programmatic stack also quietly subsidizes information disorder. NewsGuard and Comscore estimated roughly $2.6 billion per year in programmatic ad spend flows to misinformation sites — including brands like Pepsi, Starbucks, Verizon, and the CDC, almost always unintentionally. Over 21% of programmatic spend — roughly $20 billion — goes to low-quality "made-for-advertising" sites with the average campaign running on about 44,000 websites. The auction is blind to editorial quality unless advertisers actively exclude.

The cleanest causal estimates come from randomized field experiments. Lee and Hosanagar (Information Systems Research, 2019), covering 82,290 products and 1.14 million users at a top North American retailer, found that recommendation exposure caused sales rather than merely revealing latent demand — an important distinction. The marginal lift is roughly 10–20% in most causal studies, real but not overwhelming. The often-repeated claim that "35% of Amazon sales come from recommendations" is a 2013 McKinsey estimate without disclosed methodology and should be treated as directional.

Hosanagar and Fleder's theoretical work ("Blockbuster Culture's Next Rise or Fall," Management Science, 2009) showed collaborative filters' popularity bias tends to reduce aggregate sales diversity — the opposite of Chris Anderson's 2006 long-tail prediction. But their empirical follow-up (2014) found that individual users' consumption actually diversified even as users became more similar to one another. This is the governing paradox of algorithmic personalization: it appears to expand your world while quietly converging it with everyone else's.

Algorithms don't only route attention; they increasingly shape what gets produced. Netflix's 76,897 "altgenres" and its greenlighting of House of Cards are well documented. Spotify's influence on music structure is more intimate — average UK number-one track length fell from 4:16 in 1998 to roughly 3:03 in 2019; intros shrank from ~20 seconds in the 1980s to ~5 today. Spotify's 30-second royalty threshold and TikTok's 15-second hook logic are structural incentives embedded in the economics of platform distribution. Liz Pelly's Mood Machine (2025) documents Spotify's Perfect Fit Content program directly licensing cheap instrumental tracks from production houses under pseudonymous artist names, seeded into mood playlists at reduced royalty — a structural margin play disguised as editorial curation.

Algorithmic awareness rarely translates to algorithmic resistance. Eslami's CHI 2015 study found 62.5% of Facebook users did not know the News Feed was algorithmically curated. At 2–6 month follow-up after disclosure, satisfaction levels were statistically unchanged. Knowing the machine exists does not empirically change what most people end up watching, listening to, or buying.

A rigorous peer-reviewed literature documents mechanisms of algorithmic influence on cognition and psychology — mostly qualitative and small-n, establishing how it works rather than population-scale harms. Jhaver, Karpfen and Antin's 2018 CHI study of Airbnb hosts formalized "algorithmic anxiety" — the double negotiation of appealing to both guests and an opaque evaluator, producing coping rituals with no evidence of efficacy. Register et al. (CSCW 2023) documented Instagram users exhibiting distress, hypervigilance, and a need to appease "the algorithm." Taina Bucher's 2017 "algorithmic imaginary" research shows imagined algorithms generate real affect and behavior — the machine doesn't have to work the way you think it does to shape what you do.

The "creepy ad" literature is one of the more empirically stable areas. When personalization exceeds the user's perceived consent, brand purchase intent falls. Personalization, beyond a threshold, backfires commercially — Petrova et al.'s 2026 Psychology & Marketing four-study result, among others. The personalization uncanny valley is real: too accurate feels like surveillance.

What is not well-supported is the strong form of the behavioral manipulation thesis — the idea that recommendation algorithms have durably altered personality, political beliefs, or fundamental preferences at population scale. The Meta 2020 election studies produced the largest-scale null findings on feed-algorithm political effects in the literature: three months on a chronological feed moved exposure substantially but political attitudes barely at all. Amy Orben and Andrew Przybylski's specification-curve analyses across 355,358 adolescents found that digital technology use explains at most 0.4% of variation in adolescent well-being — comparable, as they noted, to eating potatoes.

Shoshana Zuboff's The Age of Surveillance Capitalism (2019) is the most influential framework for thinking about this space. The descriptive claims — mass behavioral extraction, documented psychological-vulnerability targeting, "behavioral futures markets" — are substantially supported by ICCL, FTC and leaked-document evidence. The sweeping theoretical synthesis is contested by Ball, Morozov, Rinehart, and Brynjolfsson's revealed-preference work estimating users would demand $17,530 per year to give up search and $8,414 to give up email — implying substantial consumer surplus alongside extraction.

If there is one place where careful empirical work has most complicated the popular narrative, it is political effects.

The landmark 2023 Meta Election Studies — a collaboration between Meta and roughly 20 external academics — produced findings that were widely underreported. Guess, Malhotra, Pan, Barberá, Allcott et al. (Science, 381(6656)) randomly assigned users to chronological rather than algorithmic feeds for three months. Exposure to political and untrustworthy content actually increased in the chronological condition, and time on platform dropped substantially — but there was no detectable effect on political attitudes, polarization, or knowledge. Nyhan, Settle, Thorson, Wojcieszak et al. (Nature, 620) reduced users' exposure to like-minded sources and found like-minded content prevalent but "not polarizing" — reducing it did not measurably change attitudes. Allcott, Gentzkow et al.'s PNAS 2024 Facebook deactivation experiment (six weeks, ~36,000 users) produced only modest effects on news knowledge and limited attitudinal change.

At the same time, asymmetric amplification findings are remarkably consistent. Huszár, Ktena et al. (PNAS 119(1), 2022) randomized roughly 2 million Twitter accounts and found that in 6 of 7 countries, the mainstream political right was amplified more than the mainstream left. González-Bailón, Lazer, Barberá et al. (Science, 2023) found similar asymmetric segregation on Facebook. These findings coexist: algorithms amplify asymmetrically, but short-window political attitudes are anchored by demographics, identity, and offline networks in ways even substantial feed interventions struggle to move.

Meta's own internal research — authenticated but not peer-reviewed — told a darker story than the academic work. The 2016 Monica Lee internal presentation found 64% of all extremist-group joins on Facebook came from its own recommendation tools. A 2018 internal slide stated: "Our algorithms exploit the human brain's attraction to divisiveness." The "Common Ground" depolarization initiative was largely shelved. The honest reading: external experiments and internal documents are both partly right. Algorithms amplify divisive content as Meta's own researchers found, but short-window political attitudes resist even substantial feed interventions.

Cambridge Analytica's election impact, meanwhile, is not supported by peer-reviewed persuasion research. Eitan Hersh (Tufts) has argued consistently that public voter-file data predicts voting behavior better than Facebook-derived personality inferences, and Trump's 2016 digital director publicly denied psychographics mattered. Matz et al.'s PNAS 2017 work shows psychographic targeting can work at small scale; the gap between that capability and Cambridge Analytica's deployed impact on the 2016 election is large.

The most important generational finding cuts against the common story. Guess, Nagler and Tucker's "Less than you think" (Science Advances, 5(1), 2019) linked YouGov survey data to actual Facebook activity in 2016 and found users over 65 shared nearly 7× more fake-news articles than 18–29s — a ratio that persisted after controlling for partisanship, ideology, and education. Only 3% of 18–29s shared fake news versus 11% of those over 65. Grinberg et al. (Science, 2019) found the parallel pattern on Twitter. Older Americans compressed their social-media onboarding from 8% adoption in 2008 to 40% by 2019 with limited digital literacy scaffolding.

Gen Z is genuinely the first cohort whose taste, social cognition, and self-understanding developed inside recommendation systems rather than being retrofitted onto them. Kyle Chayka's Filterworld (2024) argues for cultural homogenization — the global sameness of Instagram face, Airbnb aesthetics, Spotify playlists. W. David Marx's Status and Culture (2022) calls it a "pluralistic monoculture." The reconciliation: the public algorithmic layer homogenizes while the sub-algorithmic layer — group chats, Discord servers, private Substacks — fragments further than ever. Gen Z navigates both simultaneously.

No single rigorous study establishes that Gen Z has measurably lower preference diversity than Millennials or Gen X at the same age. The claim rests on circumstantial convergence — compressed microtrend cycles, Luminate figures showing catalog tracks hold 70–72% of on-demand audio consumption, qualitative observation — not a controlled diversity index. Gen Z's striking political polarization by gender (young women leftward, young men rightward, a pattern that appears across Western democracies) suggests algorithms in some domains amplify rather than flatten divergence.

Gen Alpha is the first cohort forming preferences before language acquisition is complete under machine-learning mediation. Common Sense Media's 2025 census shows 40% of U.S. children have their own tablet by age 2, daily reading among 5–8s dropping from 64% in 2017 to 52% in 2025. The 2017 Elsagate case — YouTube Kids' algorithm rewarding bizarre content optimized for watch-time signals children produce — remains the paradigmatic failure of algorithmic curation for preverbal users. What recommenders do to preference formation before language is fully acquired is largely unknown. The natural experiment is running in real time on tens of millions of children.

An intellectually honest synthesis requires taking seriously the substantial body of evidence that complicates the dominant "algorithms are destroying us" frame.

Recommenders do not monolithically reduce diversity. Lee and Hosanagar's randomized experiment found individuals' variety expanded under collaborative filtering, even as aggregate concentration modestly increased. Filter bubbles are weaker than claimed — Bakshy (Science, 2015), Flaxman (Public Opinion Quarterly, 2016), Dubois and Blank (ICS, 2018), and Gentzkow and Shapiro (QJE, 2011) all point in the same direction: individual choice and homophilic social networks do more work than algorithms. Long-tail democratization is real — Brynjolfsson's work estimated $731M–$1.03B in annual consumer welfare from expanded online book variety, with niche books reaching 36.7% of Amazon sales. The Meta 2020 election studies produced the largest-scale null findings on feed-algorithm political effects in the literature. Boxell, Gentzkow and Shapiro (PNAS 2017) showed U.S. polarization grew fastest among demographic groups least likely to use social media — hard to square with a clean algorithm-to-polarization story.

Christopher Ferguson's historical work places current algorithmic-harm claims in a long pattern of media panics — waltzes, dime novels, comic books, D&D, violent video games — whose empirical foundations later eroded. Allcott, Gentzkow, Budak, Nyhan and colleagues (Nature, 630, 2024) argue the harms of online misinformation are widely overstated: "fake news" constitutes roughly 0.15% of Americans' daily media diet. Hosseinmardi et al.'s 2024 counterfactual-bot PNAS study found YouTube recommendations on average pull users toward more moderate content, not more extreme.

The skeptic position is not "algorithms do nothing." It is that individual preferences and offline institutions explain most of what is blamed on machines; that causal effects, where identified, are typically small; that platforms have genuinely democratized production and access to niche culture; and that audit methodologies often overstate algorithmic pull. A proper epistemic stance holds that algorithms are consequential tools whose effects are context-dependent, frequently smaller than headlines suggest, occasionally net-positive — while concentrated real harms on superconsumers, already-vulnerable users, and through the structural funding of misinformation merit targeted rather than panicked responses.

Four conclusions have strong empirical backing. The ad-tech infrastructure is industrial in scale — 178 trillion RTB broadcasts per year, documented psychological-vulnerability targeting, $2.6 billion annually funding misinformation — and regulators have ruled major pieces of it unlawful. Recommenders causally shift consumption by 10–20%, shape what Netflix and Spotify produce (altgenres, ghost artists, song-length compression), and homogenize taste across users while appearing to personalize — the Hosanagar paradox. Dark patterns quantifiably manipulate, with experimental effect sizes of 2–4×. And the misinformation-susceptibility asymmetry runs toward older, not younger, users — the 7× fake-news sharing ratio among over-65s is the cleanest generational finding in the literature.

Four claims central to the popular narrative are weaker on inspection. The strong filter-bubble thesis is not supported by the best causal evidence. The YouTube "rabbit hole" is rare in representative data, especially post-2019. Cambridge Analytica's election impact is not supported by peer-reviewed persuasion research. And the "algorithms caused a Gen Z mental-health crisis" thesis, in Jonathan Haidt's strong form, is currently ahead of the evidence — Candice Odgers's March 2024 Nature review described the "rewiring" claim as "not supported by science."

The most important conceptual point: the machines are not hypnotists, and the consumers are not zombies. What algorithmic systems reliably do — and what is genuinely novel in human history — is mediate taste formation, identity construction, political exposure, and commercial decision-making through continuously optimizing proxies chosen by private firms accountable primarily to advertisers. The harm is not always that they make us want things we wouldn't otherwise want. It is that they foreclose the quiet cognitive labor of noticing what we want on our own.

The first generation for whom that foreclosure is native rather than imposed is now entering adulthood. Whether their epistemic condition is catastrophe or simply a new mode of human being is not yet empirically decidable — and anyone who tells you otherwise, from either direction, is selling something the peer-reviewed literature does not support.