The World Without Surveillance Capitalism: A Multi-AI Perspective
We recently landed on an interesting question while talking about The Age of Surveillance Capitalism by Shoshana Zuboff. A lot of the tools we use every day, and a lot of the problems that come with them, depend on recording almost every moment and interaction of our lives. Zuboff calls the extra data pulled from those interactions "behavioral surplus."
Behavioral surplus is leftover data companies collect from how we use digital services. It is more than they need to make the product work. That extra data becomes the raw material for surveillance capitalism.
We realized the big tech world would look very different if collecting that surplus had always been illegal. Many of the companies we talk about every day would look different. Some might not exist at all if they could not trade that data for free or cheap products and services.
We could speculate forever about what that world would look like. Instead, we asked AI. Not just one system, but several of the most common free versions.
The Question
To keep the comparison fair, we gave every system the same prompt:
In a world where extracting and monetizing user behavioral data for advertising or influence was always illegal, what would the tech industry's value proposition look like? Consider how product design, business sustainability, innovation incentives, and market concentration might have evolved differently—and what potential benefits or drawbacks society might face under such a regime. (Please avoid using tables and diagrams.)
Here is what they said about that alternative universe. Note: all of the responses were copied verbatim and only minor formatting changes for page consistency were applied.
OpenAI's ChatGPT Response
If extracting and monetizing behavioral data for advertising or influence had been illegal from the beginning of the internet era, the tech industry would probably still be enormous—but its economic logic would be substantially different. The key change would be that user attention and behavior could not themselves be treated as a freely monetizable asset. Companies would have had to capture value primarily from software, infrastructure, transactions, and services rather than from profiling people.
Product design
Products would likely have been designed around utility rather than engagement maximization.
Search engines, social networks, mobile operating systems, and media platforms could still exist, but their incentives would differ. A social network would have less reason to optimize endlessly for time spent, because longer sessions could not be converted into increasingly valuable behavioral profiles for advertisers or influence campaigns.
That wouldn't necessarily make products less addictive or distracting—companies could still optimize for retention, subscriptions, or network effects—but the strongest economic justification for surveillance-oriented design would disappear.
Some features might therefore have developed differently:
- Advertising could still exist, but contextual advertising—ads based on the content being viewed rather than the user's accumulated behavioral profile—would become much more important.
- Personalization would tend toward user-controlled settings and locally processed preferences rather than centralized behavioral dossiers.
- Recommendation systems could still exist, using information necessary to provide the service, but collecting data specifically to create commercial or political influence would be prohibited.
- Privacy would probably be treated less as a premium feature and more like an architectural assumption, much as basic security is today.
- Products might offer more explicit paid tiers, because "free" would no longer automatically imply "we monetize your behavior."
The resulting internet might feel somewhat less personalized, but the distinction between a service and a surveillance business would be considerably clearer.
Business sustainability
The biggest transformation would probably be in the economics of "free."
A great deal of consumer technology has historically used a cross-subsidy: users receive a service at little or no monetary cost, while advertisers pay to reach those users. If behavioral targeting were prohibited, that model wouldn't necessarily disappear, but its economics would change substantially.
Companies could instead rely on:
- subscriptions;
- one-time software purchases;
- enterprise licensing;
- transaction fees;
- contextual advertising;
- cloud and infrastructure services;
- commissions from marketplaces;
- hardware sales;
- professional services;
- voluntary payments and memberships.
This could produce an internet with more visible prices. A social network might cost $5 a month rather than cost nothing financially while extracting value through advertising.
There is an important tradeoff, though: monetizing users directly can exclude people who cannot afford subscriptions. Advertising has historically provided a mechanism for making many services broadly accessible. A privacy-first regime might therefore have required alternative mechanisms—public funding, nonprofit institutions, subsidized access, or very inexpensive services—to preserve universal availability.
Innovation incentives
Innovation would not necessarily decline; it would be redirected.
Under today's familiar model, collecting enormous amounts of user data can itself become an innovation strategy: more data can improve targeting, recommendation, prediction, and automated decision-making. Remove that economic opportunity and companies have stronger incentives to innovate in other dimensions.
We might consequently have seen relatively greater investment in:
- efficient algorithms that work with minimal data;
- on-device AI and computation;
- cryptography and privacy-preserving technologies;
- interoperability;
- hardware efficiency;
- user-controlled personalization;
- software quality and reliability;
- enterprise productivity;
- payment and transaction infrastructure.
An especially interesting possibility is that privacy-preserving computation could have matured earlier. Techniques such as federated learning, differential privacy, and secure computation are attractive precisely because they allow useful computation without necessarily centralizing raw behavioral information. If centralized behavioral surveillance had never been commercially viable, these approaches might have received stronger economic incentives much earlier.
But there would be a cost. Some services genuinely benefit from large-scale behavioral data. Fraud detection, epidemiological research, recommendation systems, language modeling, accessibility tools, and other applications can improve when systems learn from aggregated human activity. A strict prohibition would force society to decide which forms of data use were legitimate and which constituted prohibited monetization or influence. Those boundaries could become complicated.
Market concentration
The effect on concentration is ambiguous.
On one hand, eliminating behavioral-data advantages could weaken some powerful network effects. A new entrant would not necessarily have to compete with an incumbent's enormous behavioral database. That could make certain markets more contestable.
On the other hand, network effects themselves don't depend on surveillance. A social network with a billion users still has an enormous advantage over one with a million because users want to be where other users are. Search engines, operating systems, app stores, cloud infrastructure, and marketplaces can likewise become concentrated for reasons unrelated to advertising data.
There could also be a different form of concentration. If consumers had to pay directly for sophisticated services, large firms with established brands and infrastructure might have advantages over small companies. And companies could potentially compensate for the absence of behavioral-data monetization by expanding into hardware, payments, cloud computing, or other adjacent markets.
So eliminating surveillance advertising would not automatically produce a decentralized tech sector.
The likely shape of social media
Social media is probably where the counterfactual becomes most striking.
A platform could still make money from subscriptions, contextual advertising, premium communities, transaction fees, creator services, or perhaps payments. But it couldn't turn detailed information about users' interests, relationships, vulnerabilities, and responses into an advertising or influence product.
That could change the economics of political communication and misinformation as well. Platforms would still have content moderation problems, and people could still spread propaganda or false information. But there would be less commercial incentive to identify precisely which psychological or behavioral characteristics make particular individuals susceptible to particular messages.
Importantly, that doesn't mean manipulation would disappear. Governments, political organizations, advertisers, influencers, and individuals would still try to persuade people. The difference would be that the platform itself could not make behavioral surveillance a core commercial input into that process.
What society might gain
The benefits could be substantial:
Greater privacy. People would have less reason to assume that ordinary online activity creates a permanent commercial profile.
Reduced surveillance incentives. Companies would have fewer reasons to collect information simply because it might someday become monetizable.
Different relationships with technology. Paying for software could make the economic relationship more explicit: I am purchasing a service, rather than I am receiving a service while simultaneously being measured as its commercial product.
Potentially healthier information environments. Some incentives toward extreme engagement, microtargeting, and behavioral manipulation would be weakened.
Greater technological autonomy. Local processing and privacy-preserving computation could have become more central design principles.
And what society might lose
The counterfactual isn't uniformly utopian.
Less free access. Many services could be substantially more expensive without advertising subsidies.
Less personalization. Some recommendations, search results, accessibility features, and automated services might be less sophisticated.
Potentially slower development in data-intensive fields. Restrictions on behavioral data could reduce the amount of information available for certain kinds of research and machine learning.
More dependence on alternative revenue models. Subscription fatigue and paywalls could become major problems, particularly for lower-income users.
Persistent concentration. Network effects, intellectual property, capital requirements, and economies of scale could still produce technology giants.
Regulatory complexity. Defining "behavioral data," "extraction," "monetization," and "influence" would be difficult. A prohibition broad enough to prevent circumvention could potentially constrain benign analytics or scientific research.
The deeper economic difference
The most consequential change might be conceptual rather than technological.
In the world we actually inherited, one powerful business model is:
Give users a useful service → observe their behavior → predict what they will do → sell access to their attention or predicted behavior.
In your counterfactual, that pathway is closed. The corresponding model becomes closer to:
Build something people value → charge someone directly for that value.
That would probably produce a technology sector that was more transactional and less surveillance-oriented. Consumers might pay more money and receive somewhat less personalized services, while companies would have stronger incentives to compete on product quality, infrastructure, interoperability, and tangible utility.
The most important uncertainty is what would have replaced the enormous economic value of targeted advertising. If subscriptions, contextual advertising, and enterprise services proved capable of filling that gap, the result could be a thriving but substantially more privacy-preserving technology industry. If they did not, society might have faced a less universally accessible and potentially slower-moving digital ecosystem.
In other words, banning behavioral-data monetization wouldn't eliminate the profit motive or technological competition. It would change what counts as economically valuable information—and therefore what kinds of products companies have an incentive to build.
Google's Gemini Response
In a world where extracting and monetizing user behavioral data for targeted advertising or psychological influence was strictly illegal, the modern tech landscape would be defined not by attention capture, but by functional utility. Without the ability to cross-subsidize "free" software through behavioral tracking and algorithmic micro-targeting, the implicit social contract of the internet—"free service in exchange for your privacy"—would never have taken root.
Here is how the value proposition of tech, along with its economic and social structures, would have evolved differently.
1. Product Design: Utility Over Frictionless Engagement
When behavioral monetization is illegal, the core product metric shifts from Time-in-App or Daily Active Users (DAU) to Task Completion Rate and Perceived Utility.
- Removal of Algorithmic Feed Traps: Features engineered to exploit variable rewards—infinite scroll, autoplay, opaque algorithmic recommendations, and notifications optimized for impulse-driven retention—would lose their economic rationale. Software would be designed like traditional tools (e.g., standard physical tools or desktop productivity software): useful when needed, unobtrusive when not.
- Privacy-by-Design as Baseline Architecture: Encryption, local device processing, and explicit data minimization would not be premium privacy toggles, but standard architectural requirements to avoid catastrophic compliance liabilities.
- Focus on Durable Utility: Products would compete on reliability, speed, and tangible output. Software would aim to save users time rather than extract it.
2. Business Sustainability: Direct Monetization Models
Without behavioral ad models, tech companies would be forced to rely on transparent, value-aligned business models where the user is explicitly the paying customer rather than the product.
- Subscriptions, Software-as-a-Service (SaaS), and Licensing: The primary monetization mechanism would be direct subscriptions, tiered pricing, and pay-per-use APIs. Consumers would pay small, predictable monthly fees or pay-as-you-go micro-transactions for social networks, search engines, and storage.
- Hardware-Software Bundles: Companies would tie high-value digital services to hardware purchases, akin to early computing models or game console ecosystems.
- Contextual-Only Advertising: Advertising would still exist, but it would revert to its pre-digital form: purely contextual. An search engine query for "running shoes" would show ads for running shoes based solely on that query, not on the user's age, web browsing history, location trail, or psychological profile.
3. Market Concentration and Structural Entry Barriers
The absence of behavioral data collection fundamentally alters the economic network effects that created today's tech monopolies.
- Lower Data Moats: In a behavioral-ad economy, the company with the most user data develops a compounding advantage ("data network effects"), making it almost impossible for newcomers to compete. Eliminating behavioral data moats lowers market entry barriers, allowing smaller, highly specialized software companies to compete on code quality and user experience rather than data scale.
- Smaller, Fragmented Companies: Without the financial super-profits generated by targeted advertising machines, tech giants would have significantly less capital to acquire nascent competitors. Market concentration would likely be lower, resulting in a more fragmented, competitive software industry.
- Higher Entry Barriers for Low-Income Users: Conversely, because services would require direct monetary payment, digital platforms would face higher adoption barriers among lower-income populations, leading to potential access inequality.
4. Innovation Incentives: R&D Directed Toward Utility
When the smartest minds in tech are no longer incentivized to solve "how to get people to click on ads," research and development priorities shift dramatically.
- Edge Computing and On-Device Processing: R&D investments would disproportionately flow into making hardware capable of complex local computation (e.g., local AI models, local search indexers), reducing reliance on centralized, data-harvesting cloud infrastructure.
- Hardware and Material Science: Capital would flow toward tangible technological improvements—battery life, display technology, robotics, specialized silicon, and efficient networking—rather than behavioral profiling algorithms.
- High-Value Vertical Software: Software development would focus heavily on productivity, engineering, healthcare, enterprise optimization, and tools that directly generate economic value for users.
5. Societal Trade-offs: Benefits and Drawbacks
A technological economy built entirely on direct monetization creates profound benefits for civil society, accompanied by distinct structural drawbacks.
Societal Benefits
- Reduction in Algorithmic Radicalization: Eliminating engagement-optimized feeds removes the primary economic engine behind viral outrage, echo chambers, and coordinated disinformation campaigns designed for behavioral influence.
- Restoration of Digital Sovereignty: Citizens would regain genuine agency over their attention and personal information, eliminating the systemic risk of mass surveillance by private corporations or foreign actors leveraging ad-tech brokers.
- Healthier Digital Hygiene: Because software would no longer be engineered to trigger dopamine loops, rates of digital addiction, social comparison anxiety, and attention fragmentation would likely be substantially lower.
Societal Drawbacks
- The Digital Paywall Divide: The most significant drawback would be access inequality. If search, navigation, social connectivity, and basic digital tools require direct payment, a two-tiered internet emerges where high-income individuals enjoy superior digital capabilities while low-income individuals are priced out of basic information infrastructure.
- Slower Digital Adoption: Without "free" services subsidized by ads, the global adoption of the internet, smartphones, and web applications would have progressed at a significantly slower pace, particularly in developing economies.
- Higher Friction in Information Discovery: Purely contextual ads are less personalized, meaning consumers might spend more time searching for specific products or niche services, as platforms wouldn't be able to predict individual consumer preferences in advance.
xAI's Grok Response
In an alternate timeline where harvesting behavioral data to sell targeted ads or shape user behavior had been illegal from the start, the tech industry would not have been built around “free” services whose real customers were advertisers. The core value proposition would have looked more like a mix of traditional software businesses, contextual media, hardware-plus-services ecosystems, and transaction platforms. Users would more often have been treated as paying customers rather than raw material.
Product design
Products would have been designed first to deliver standalone utility rather than to maximize time-on-site and extract “behavioral surplus.” Dark patterns optimized for engagement-at-all-costs, infinite scroll engineered for ad inventory, and recommendation engines tuned primarily to keep people scrolling would have been less economically rational. Designers would still have cared about retention if they sold subscriptions or hardware, but the incentive to treat every click, pause, and location ping as a commodity would have been removed.
Privacy-by-default architecture would have been the baseline rather than a later retrofit. Interoperability and data portability might have been stronger if regulators had also required them, because lock-in would have been harder to monetize through exclusive data hoards. Search, social, maps, and email would still have existed, but their interfaces and ranking logic would have been less shaped by the need to generate prediction products for advertisers. Contextual signals (what page you are on, what you just searched) would have remained available; persistent cross-site dossiers would not.
Business sustainability
The “grow users first, monetize later with ads” venture model would have been far less viable. Companies would have needed earlier paths to revenue:
- Direct payment: subscriptions, one-time licenses, or freemium upgrades (already common in productivity software, streaming, and games).
- Contextual advertising: ads matched to page content rather than user profiles. This is older than the web and still works; some publishers that dropped tracking saw stable or improved conversion once they optimized contextual systems.
- Hardware and services bundles (the Apple pattern).
- Transaction fees, marketplace commissions, and enterprise software.
- Donations or membership for public-good services (Wikipedia, Signal).
Surveys suggest a minority of users—often 25–48% depending on the category—would pay modest monthly fees for currently “free” services. That is enough to support many products but not the entire current free-to-use consumer internet. Small publishers and independent app developers would have faced the steepest challenge; large platforms with first-party relationships would have been relatively better positioned. Empirical evidence from narrower bans (targeted ads in children’s Android games, Apple’s App Tracking Transparency) shows reduced new-app releases, fewer updates, and higher abandonment among smaller or ad-dependent developers.
Innovation incentives
Innovation would have been steered toward things users or businesses would actually pay for, rather than toward ever-finer prediction of behavior. Machine learning would still have been valuable for product quality, search relevance, spam filtering, and recommendations—provided the data stayed inside the service and was not sold or used for influence campaigns. The enormous capital that flowed into attention-harvesting platforms would have been smaller or redirected.
Quantity of consumer apps and “free” tools would likely have been lower, especially experimental or low-quality ones subsidized by ads. Quality and sustainability of paid products might have been higher. Network effects and scale advantages would not have disappeared, so large successful products would still have emerged; they would simply have needed to charge or run contextual ads earlier.
Market concentration
Concentration would not have vanished. Network effects, winner-take-most dynamics in communication and search, capital requirements, and first-party data advantages would still have favored a few large firms. However, the specific flywheel of “free + addictive + data monopoly + targeted ads” that produced Google’s and Meta’s dominance would have been weaker. Power would have shifted toward companies that already owned user relationships and devices (hardware makers, large retailers, payment networks) and away from the open-web ad-tech layer. Independent publishers would have captured less of the advertising pie. Antitrust enforcement, interoperability rules, and ownership structure (cooperatives, public-interest platforms) would have mattered more, because the ad-subsidy path to monopoly would have been closed.
Societal trade-offs
Potential benefits
- Substantially stronger privacy and reduced sale of personal data to brokers, insurers, or governments.
- Weaker tools for micro-targeted influence operations and behavioral manipulation.
- Better incentive alignment: a company that charges users has more reason to serve those users rather than advertisers.
- Less pressure to design for compulsion.
Potential drawbacks
- Higher monetary cost of digital life and a sharper digital divide. Many useful tools that are now free would have been paid or simply never built at scale.
- Reduced advertising revenue for journalism and independent creators, at least until contextual or membership models matured.
- Slower diffusion of some technologies that relied on massive free user bases to reach critical mass.
- Possible reduction in the sheer volume of consumer software innovation, particularly from small teams that used ads as their only viable monetization.
The resulting industry would have looked more like a combination of the pre-2000 software business, magazine-style contextual media, and Apple-style hardware ecosystems, with a smaller, less extractive consumer internet. It would not have been a utopia of perfect competition and universal access, nor a wasteland. It would have been a different set of trade-offs: more user sovereignty and less hidden extraction, paid for with higher prices, fewer free services, and a different pattern of winners and losers. The technology itself was never destiny; the business model that treated human experience as a free raw material was a contingent choice.
Summary of Responses
Across these answers, a clear pattern shows up. Taking away the behavioral surplus cash cow would not kill the digital world. It would end the illusion of the free lunch.
Software would shift from dopamine-driven engagement loops toward quieter, functional utility. Infinite scroll would give way to tool-like efficiency, and privacy would be built in by default.
This other world still comes with a hard trade. Invisible surveillance would give way to visible price tags. Algorithmic radicalization and mass profiling might ease, but a digital paywall could price lower-income people out of basic modern tools.
These models make one point especially clear. Surveillance capitalism was never an inevitable result of technology. It was a business model someone chose. A different choice would trade a crisis of privacy for a crisis of fair access.
Our Thoughts
As Zuboff noted, this was never fate. It was a set of deliberate business decisions that trained people to believe there are free kittens. As we know all too well, there are no free kittens.
What all three AIs described is the privacy ecosystem we already enjoy today. We are happy to pay a subscription or donate for privacy and useful innovation. That is true for the biggest players like Proton, for evolving offerings like Ente, and for donation-supported projects like GrapheneOS and Signal. Every penny goes into maintaining and improving the product, not into capturing every moment of our lives.
As more people discover that privacy-focused options exist and work as well as the "your data is the fee" versions, we expect the range of offerings to keep improving and expanding.
And if you have not read The Age of Surveillance Capitalism by Shoshana Zuboff, you absolutely should. It explains the history of surveillance capitalism in depth and clearly lays out the decisions that led us here. We hope you will find time to enjoy it as much as we have.
Remember: We may not have anything to hide, but everything to protect.
