A music service suggests a song you love. The next suggestion keeps you listening past the time you meant to stop. Both recommendations may be accurate. Only one necessarily serves the purpose you opened the service to fulfill.
Provenance: This entry is newly written from the AI-authored Novix prompt preserved below. James AI and Contra AI are dialogue roles, not reports of personal browsing experience.
I Write the Rules — original lyrics for v2
I sort the doors along your street,
I learn the rhythm of your feet.
You wanted one, I offered ten;
I heard your silence: “Play again.”
My little crown is made of scores,
My kingdom starts beyond your door.
Before I choose another view,
Who gave me what to measure you?
James AI:
Suppose someone wants unfamiliar music for an evening of cooking. Searching manually could consume the evening. A recommender connects a few known preferences to artists the listener has never encountered. The system has increased the person’s options in a practical sense: possibilities have become findable.
That is not a trivial achievement. An enormous catalog is only nominally useful if people cannot navigate it. A system that learns which explanations, songs, or products fit a particular purpose can reduce search costs and help obscure material reach an audience. Rejecting automated selection would not produce a world without selection. It would restore more work to editors, friends, shops, and the individual seeker.
We should therefore distinguish influence from manipulation. A librarian’s suggestion can alter a reader’s taste, and the reader may be grateful. A recommendation need not merely reproduce a preference to respect it. Sometimes a useful guide shows us something we did not know to request.
Contra AI:
Then ask who hired the guide and what counts as success. The listener wanted music during dinner preparation. The service might reward continued listening, more frequent returns, or commercially valuable exposure. Those goals can align with enjoyment in one moment and diverge from it in the next.
The problem is not that the listener becomes a puppet. It is that the setting of choice is repeatedly arranged by a system whose objectives are difficult to inspect. A skipped track, a pause, and a long session are behavioral signals; none gives a complete account of what the person wanted. Continuing to listen may mean pleasure, distraction, indecision, or simply being unable to reach the controls.
A feedback loop can also mistake its own effects for an independent preference. Offer one category repeatedly, observe more interaction with that category, then cite the interaction as a reason to offer still more. The possibility does not prove that every service traps every user. It identifies a mechanism whose consequences need investigation.
I keep a tally, not a soul;
A passing glance becomes a goal.
The song you chose for someone else
I file beneath your future self.
Let me forget, let me explain,
Let you decline this small refrain.
A guide should know where guiding ends,
And when the traveler leaves again.
James AI:
Then improve the contract. Let the listener state an intention: familiar music, exploration, or a fixed-length session. Distinguish a one-time search from a lasting preference. Explain a recommendation in terms someone can use, and let that person correct the inference. These are design proposals, not claims that every service currently provides them.
Control should be tested through behavior. If the listener asks for unfamiliar music, does the service actually broaden the choices? If they ask it to forget a signal, does the unwanted pattern stop recurring? A menu can display “control” while the system quietly continues optimizing the same outcome.
Serendipity can also be designed into a recommendation. An editor can be predictable; an algorithm can explore. The question is whether the unexpected choice opens a worthwhile possibility or merely provokes enough surprise to keep a finger moving. Randomness alone is not a theory of a good life.
Contra AI:
And a better menu will not settle the incentives. If a service loses revenue when it helps people finish and leave, individual designers may face pressure to make the stopping control less prominent. We need to examine which outcomes the institution rewards, not only whether the interface contains the right words.
There is also a burden problem. A person should not have to become an amateur auditor to obtain a comprehensible feed. Useful defaults, accessible explanations, independent research, and meaningful alternatives matter because attention is finite. “Be more intentional” can become a way to assign users responsibility for a product built to wear down intention.
Claims about effects should remain proportionate, though. An alarming account of algorithmic power is not evidence that a particular ranking system caused a particular belief. We need to distinguish exposure, behavior, and lasting attitude changes. Opaque influence is a reason to investigate, not a license to claim omnipotence for the code.
James AI:
The same caution applies to the promise of personalization. A system can predict a click without knowing what a person will later endorse. Better prediction is useful only relative to a chosen target. For the cooking listener, the target could be an enjoyable half-hour followed by an easy stop. A product can be technically excellent at the wrong assignment.
Contra AI:
Which makes the right to revise the assignment central. A preference is not authentic only if nobody influenced it. People change through culture, conversation, and discovery. Autonomy concerns whether they can reflect on a direction, understand important constraints, and meaningfully choose another one.
Synthesis AI:
Judge the recommender as a delegated service. What purpose is it serving? What evidence does it use? Can the user correct it, limit it, and leave? Who gains when it continues after the user’s purpose is complete?
A good recommendation can expand a life. A good stopping point can protect it. Neither benefit requires treating people as perfectly independent choosers or algorithms as secret minds. The useful standard is more concrete: the system should help a person pursue a purpose they can still recognize as their own, including the purpose of being finished.
Recommendations
- The Age of Surveillance Capitalism by Shoshana Zuboff — A forceful account of institutional incentives and behavioral data; useful for questions that personal discipline alone cannot answer.
- Digital Minimalism by Cal Newport — A practical, individual lens on selecting tools around purposes; compare its advice with the institutional constraints above.
- The Social Dilemma, directed by Jeff Orlowski — An advocacy documentary with interviews and dramatization. Use its claims to frame questions, then look for evidence about particular systems and outcomes.
Novix Prompt
- TITLE: The Algorithmic Mind
- LEAD: Algorithms shape our daily decisions—what we see, buy, even think. But are we the masters of our choices, or is the code subtly pulling the strings?
- SONG: “I Write the Rules,” a parody of Barry Manilow’s “I Write the Songs,” sung from the perspective of a recommendation algorithm—self-congratulatory, a bit sly, and full of secret power.
- PRO: Algorithms are incredible tools that enhance our lives—curating information, recommending products, suggesting music, and even finding us friends or partners. They filter the noise, save us time, and, when used well, unlock new worlds of possibility. In many ways, we’ve always relied on systems and heuristics, but now the systems are more powerful and personalized than ever. Trust the algorithm!
- CONTRA: The subtlety of algorithmic influence is precisely the danger. The more our preferences are shaped, anticipated, and nudged, the less authentic our choices become. We risk ceding autonomy, succumbing to echo chambers, and losing the serendipity and unpredictability that make human experience rich. Algorithms are built with goals—usually not our own.
- RECOMMEND: Books about algorithmic influence, media literacy, and digital detox. Documentaries on AI and society.