Designing AI’s Boundaries: Respecting Human Agency
Four days with no papers and no spectators, built around one question: where should the line fall between the work we hand to machines and the work we keep? What we built to hold that question is now a method and three tools, and the conversation it started is still going.
What Happened
Most conferences end with a proceedings volume. This one ended with three tools for public release, and by declaring that it would not end. The lenses stay open, new participants can join, and the conversation continues asynchronously.
Over four days, from Tongji University to SUES Changning, fifty people in the room and twenty-five online, from seventeen countries, held one conversation with a shared memory. Every session was recorded and transcribed, and catch-ups were generated on the same day. Every participant could ask a personal AI lens what a session looked like from where they stand. The question stayed the same throughout: when, how much, and what kind of AI to use, and when to hold back for the human.
Nobody read a paper, keynotes included. The room was asked to do work, not to applaud.
The AI Dramaturg
In theater, the dramaturg holds the intellectual architecture of a production: researching context, questioning choices, keeping the play coherent. The dramaturg doesn’t direct, act, or write, but holds the space in which meaning is made.
AI is becoming every discipline’s dramaturg. It can retrieve, synthesize, pattern-match and propose. It cannot make the situated judgment that turns information into understanding. Deciding how much AI to use, and where, is a dramaturg’s decision: what keeps the play moving, and what only the people on stage can do.
In Shanghai the AI Dramaturg had a concrete form: the Catch-up. Each session’s same-day account supplied the 60%, and the people reading it supplied the rest. The same concept shaped the interface of Help me decide.
The line does not run between the biological and the artificial. It runs between participants who have something at stake in whether their anticipations hold, and processes that do not care whether they do.
Structural isomorphism, not identity. The model and the anticipation it stands in for occupy the same role in an architecture. They are not the same thing, and the difference is where human judgment lives.
A Conference Method, Not Only a Conference
We set out to test whether a conference could be an instrument rather than an occasion. Five components did the work. Each one either adds to the conference’s shared memory or reads from it.
Buberian dialogues and provocations
Protagonists in sustained exchange, then roles reversed. Learned listeners who hear rather than respond. An audience given a task: say what the speakers held in common, and what they left out in common. A presentation transcribes as what the speaker already knew; a dialogue transcribes as thinking.
An AI that listens and never speaks
Each participant had a lens: a compact description of their intellectual stance, built from their public work. It was demonstrated live on the first afternoon. After any session you could ask your lens what it looked like from where you stand: what you would agree with, where you would push back, and what to raise with the group. Participants sent corrections to the team, who folded them into the lens.
Transcripts and catch-up, the same day
Sessions were recorded, transcribed, cleaned and loaded within hours. The catch-up for each block gave questions to discuss, a narrative account of what happened, and the reader’s own lens reading beneath it. The historian’s mode of that account was added after the second day, at the request of the remote participants. Anyone who missed a session could rejoin the conversation where it had got to.
Working groups formed in the room
On the first afternoon the room named the problems it wanted to work on. The next morning, coloured hexagons sorted people into groups around them. The groups were recorded as dialogue, and their transcripts entered the same memory as every other session.
A second shift for other time zones: our weakest link
Each night at 22:00 Shanghai time, Ken Friedman hosted a remote session from Sweden: fifteen people the first night, nine the second, and five the third, which fell on a Monday morning in Europe and the Americas. Each session opened with the day’s AI synthesis, the neutral baseline catch-up, on a shared screen so the group had something to talk from. The time differences proved too difficult to overcome. Only three remote participants ran their own catch-ups, and the nightly talk turned more to how to use the tools than to the conference itself. But the remote cohort changed the tools. After the second day, the nine at that night’s session asked for a different way to read the catch-up: a historian’s account of what happened, alongside the original summary, agree, disagree and question format. It was built and added before the next session. With the full record now loaded, we are restarting asynchronous participation, this time without a live event to keep pace with.
Two commitments ran through all five
Every AI output was labeled as substrate, never as verdict. And the system was built against the echo chamber: every lens reading had to say where its owner would push back, not only where they would agree, and every collective synthesis was labeled as one reading among N+1.
Consent, built in from the start
The consent instrument was written to China’s Personal Information Protection Law. Recording consent sat in the speaker agreement, and no voiceprint attribution was used. Speakers who declined transcription could take part as silent participants with full access. Any use of lens data beyond the conference required a separate opt-in; every participant gave it, and some chose to remain anonymous. The study was approved in advance by the Second Order Science Foundation’s institutional review board. We practiced responsible AI on ourselves before recommending it to anyone else.
Four Days
- Each participant wrote on hexagons what would make the conference a success by Tuesday, and the room clustered them, with one rule: you could move anyone’s hexagon but your own.
- A welcome from the Dean of the College of Design and Innovation, who described AI as a projection of human intentions and an extension of human capabilities.
- Provocations from the work of Sherry Turkle, on what “human in the loop” loses when the practices that form judgment erode, and Markus Buehler, on computation that proposes its own investigations but cannot supply its own warrant.
- Keynote by Harry Collins on tacit knowledge. Fifty years ago, no laboratory that worked only from the published literature managed to build a working laser; everything turned on who had visited a lab where one already worked.
- The lens system was demonstrated live on the convener’s own lens.
- A Buberian exchange in two rounds: Owen Matson and Feng Tao (Nankai), then Thomas Ploug (Aalborg) and Alice Jing Shan, with the audience reporting what each pair held in common.
- The room named the problems it would work on the next day.
- Dave Snowden closed the day live from Wales, answering a provocation Michael Lissack composed from his work.
- A video address recorded by Don Norman, on AI as a tool that augments versus a substitute that erodes, and a provocation from the work of Joanna Bryson, on why delegating a task leaves the obligation to justify it with the human.
- Five working groups: academic publishing; education; misinformation; value-added uses; and a fifth that read the discourse about AI itself, through Lacan, Merleau-Ponty, Stiegler and Latour.
- Each table brought back a question to carry into its own AI practice. Hugo Letiche put the Dutch childcare-benefits scandal to the room, and individual and structural accountability stayed in tension from then on.
- Two overnight tasks. First, act as the advisory board of a multi-tool AI provider and come back with three questions its interface should ask before the user goes on. Second, name the question two days had not yet raised.
- The room took stock of the group work: abstraction is easier than a specific question, because a specific question demands a commitment you have to defend.
- The convener traced what “AI” has meant decade by decade: expert systems, search, GPS, large language models. A participant wrote the four Chinese characters for the term on the board and traced their meanings.
- A provocation from the work of Andy Clark, on cognition beyond the skull, and a second on how algorithmic systems carry their designers’ moral choices forward in time.
- The Buberian dialogue. Lorenzo Magnani (Pavia) spoke on abduction beyond the machine: models master the chain of signifiers but lack eco-cognitive openness. Ricardo Baeza-Yates (Northeastern / Pompeu Fabra) spoke on what the model cannot be held to, from the Dutch and Post Office scandals to the ceremonial human in the loop. Harry Collins added that societies work because people break rules when the rules stop serving.
- An open morning. Two models working together can do worse than one. Baeza-Yates proposed a “right number zero”: the right not to use AI. Collins observed that glasses are always full; the question is full of what.
- A provocation from the work of Ken Friedman, on why an instruction is not the experience it occasions, and one made by Cameron Tonkinwise, who called for a Brechtian dramaturgy that interrupts creation to show its workings.
- The anticipation cycle was put to the groups as their instrument, with a real fatality as the case.
- Aaron Kagan showed an unfinished prototype for human–AI co-design, built around “who is leading now” and the argument that inclusive design begins with deciding whom you are intentionally designing for.
- The convener closed by declaring that the conference would continue in asynchronous form.
Two provocations were made by the thinkers themselves: Don Norman and Cameron Tonkinwise. Michael Lissack composed the rest for the conference, and how they were made is part of what they put to the room:
- Find. Claude and Gemini located publicly available recorded talks by each thinker that, as those models understood it, bore on the conference’s question.
- Summarize. ChatGPT drafted a summary of the thinker’s position from the transcripts of those talks.
- Check. Michael Lissack read, corrected and cleaned up every summary. This was the human in the loop.
- Treat. Claude wrote a video treatment for each summary.
- Produce. The summary and its treatment went to Gemini Notebook (formerly NotebookLM), which made the narrated film.
The narration is a machine voice; no thinker’s voice or likeness was synthesized. The composed provocations are not statements by the people named; their names record whose thinking was put to the room. A conference about where machines may stand in for people opened its days with exactly such stand-ins, made largely by machines with one human checkpoint. We put them forward as instances to examine, not as illustrations.
Since the conference, several of the thinkers whose work was put to the room, Dave Snowden and Ken Friedman among them, have said they intend to use the provocation made from their work themselves.
Three Tools, Released Publicly
No single line came out of four days, and none was meant to. What came out is something people can use: three tools, all free to use and released by the Second Order Science Foundation. For other conferences, the Catch-up may prove the most valuable of the three. The code is not open source; its developers retain ownership.
Help me decide
Give it a task or a situation, and it helps you decide how much AI to use, in what role, and which one. A prototype was shown in the closing session, working through the design of a child’s stroller. It surfaced regulations, human-factors research and parents’ feedback, turned them into design direction, and showed at every step who was leading, you or the AI, through a boundary you redraw in plain language. Its interface follows the conference’s AI Dramaturg concept: the tool supplies options and their grounds, and you make the call. The alpha is complete, the beta is in testing, and the release version is due in October 2026.
The Catch-up
Built for anyone who missed a session, or was asleep in another time zone while it happened. In Shanghai it turned out to serve the people who were there. Within hours of each session, it gives questions to discuss, an account of what was said, by whom and in what order (the historian’s mode, added at the remote participants’ request), and, if you have a Lens, what the session means from where you stand. No profile is needed for the account itself, which is why it travels to any conference. In Shanghai, about 40 participants ran their own catch-up each day, all but three of them people who had been in the room.
The Lens
A personal AI reader built from your own published stance. It listens and never speaks. Give it a transcript, and it tells you what the session looks like from where you stand: what you would agree with, where you would push back, and what to raise with others. About a hundred were built for Shanghai, and more than a third of their owners corrected how theirs read them. The system is now opening to anyone.
Built on what the room found
The decision tool answers to findings that separate groups reached from different directions.
Most of what we blame on AI was already there.
Working separately, the groups reached the same observation. Metric gaming in publishing, assessment built on the essay, and the erosion of trust at scale all predate AI. AI makes them harder to ignore.
Accountability was the one answer every table accepted.
That means transparency at each step and ownership of each result, built into the process rather than left to individual vigilance. A reviewer under normal load misses the fabricated reference that one attentive reviewer catches.
Telling people how the machine works does not protect them.
Under cognitive load and time pressure, people trust AI output more, not less. A single human placed at the end of a workflow can be present but powerless. The protection has to be designed into the structure.
“AI” is too broad a word to argue with.
Expert systems, search, GPS and language models are different things. Contextual knowledge, not abstract principle, is the condition for ethical judgment about any one of them.
There is a right not to use it.
The right to opt out, and the people a digital-only process excludes, belong in the design from the start.
Prediction is not anticipation.
Language models predict well. Anticipators have something at stake in whether their anticipations hold, and so stay open to the encounter that proves them wrong. The closing morning gave the groups that distinction as their working instrument.
The reasoning behind it
The cycle runs: anticipate, encounter, abduce, interpret, decide the next action. Each turn begins with an anticipation that matters to the one who holds it.
The case was a driver whose anticipation, shaped by the label “self-driving,” diverged fatally from what the system was actually built to do. Nothing in the design forced the gap into view before it mattered. The lesson the room drew is a design lesson: build the forcing question into the work itself, at the point where time pressure makes individual vigilance least reliable.
That is also where the second night’s homework pointed. The best protection is not a warning label. It is the question the interface asks before you go on.
The Conversation Continues
The convener closed by announcing that the conference would carry on in asynchronous form. Because the record is the dialogue itself, it can keep being read, answered, and written from.
Bringing the absent in
The time differences from Shanghai proved too difficult to overcome during the conference. With every session now transcribed, catch-ups written and lenses open, we are restarting asynchronous participation for those who could not keep Shanghai hours.
An independent test of Help me decide
David Ing, President of the International Society for the Systems Sciences, and Gary Metcalf, former President of ISSS and of the International Federation for Systems Research, are running an evaluation of the decision tool, independently of its developers. Participants bring a real task, decide, use the tool, and decide again. The test is whether the call stays theirs.
The record, open to research
Anonymized transcripts of the sessions are available for bona fide research on request from the Second Order Science Foundation, from 15 October 2026.
Help me decide
The decision tool that came out of the conference: how much AI, in what role, and which one. Developed by Aaron Kagan of GraspingAI and released free by the Second Order Science Foundation at aihowmuch.app. The alpha is complete and the beta is in testing, with the release version due in October 2026.
The Catch-up
Same-day accounts of every session, readable through your own Lens. Developed by Michael Lissack and released free by the Second Order Science Foundation at aqlens.com/catchup, for use at other conferences.
The Lens stays open
About a hundred personal lenses were built for the conference, and they remain available. The Lens, developed by Michael Lissack, is being released free by the Second Order Science Foundation at aqlens.com, so the conversation is not limited to the people who were in Shanghai.
Abduction Beyond the Machine
A journal article developed from Lorenzo Magnani’s strand of the 28 September dialogue, with Magnani as lead author and Hugo Letiche and Michael Lissack situating his argument among the conference’s other strands.
The method, reported
A research article by the organizing team describing the dialogic, lens-mediated, asynchronous conference method and its first full deployment in Shanghai.
A Version of You in the Room
A design-and-use study of the per-participant lenses: how scholars across time zones used an AI that listens on their behalf, and what they corrected. A second paper on the corrections themselves will follow.
She Ji theory issue: “The AI Dramaturg”
A companion issue of She Ji: The Journal of Design, Economics, and Innovation. Participants from the working groups and the overnight cohort may develop contributions, supported by the same research infrastructure.
What they would say to Shanghai
Long-form essays by Michael Lissack, composed from the public talks and writing of Lucy Suchman, Joanna Bryson, Andy Clark, Don Norman, Markus Buehler and Sherry Turkle, putting each thinker’s position to the conference’s question. Like the composed provocations, they are his prose, not statements by the people named.
Designing Human Agency
Shanghai asked where AI’s boundaries should fall. The successor conference turns to the other side of the same line: what it takes to design for judgment that stays one’s own. It will be held in 2027 with a host institution, and the host gets more than an event.
A signature convening
Dialogue-first, provocation-led, and built to produce something usable: conversation that produces work rather than panels that produce applause.
A working AI layer, with its ethics already done
Personal lenses, same-day catch-up, nightly sessions for remote participants built on an AI synthesis of the day, and a consent framework written to a demanding national privacy law. Every output is labeled as substrate, not verdict.
An international network
Scholars and practitioners from seventeen countries, with partnership routes through Tongji University, Shanghai University of Engineering Science, and ISTEC Paris.
The tools, and a practice students can learn
The Lens, the Catch-up and the decision tool, and the anticipation cycle behind them, adapt directly to studios, capstones, and executive education as a disciplined practice of better questioning: given this task, how much AI, in what role, and which one.
Michael R. Lissack
Michael Lissack conceived and co-chaired the conference, designed its method with a small team, and developed the Lens and the Catch-up. He is Professor of Design and Innovation at Tongji University and Executive Director of the Second Order Science Foundation. He was President of the American Society for Cybernetics from 2014 to 2020, and founded the Institute for the Study of Coherence and Emergence and its journal, Emergence: Complexity & Organization.
At the 2026 World Design Cities Conference he also chaired a main-stage roundtable on human–machine symbiosis, and he was appointed an Expert Committee Member of the Shanghai International Design 100 Think Tank for 2026–2029, by the Shanghai Municipal Commission of Economy and Informatization and Tongji University. He has organized more than twenty-five international conferences and published more than a hundred works, including Questioning Understanding, Understanding Questioning (2025). His current research develops an account of anticipatory agency: what separates agents with something at stake in their anticipations from systems that predict without caring. Two companion articles on it are accepted at AI and Ethics and AI & Society.
Before academia he spent fourteen years in public finance at Smith Barney and was CFO of Webmind, one of the first AI companies. The common thread is building institutions that help people keep their judgment in the presence of powerful technology.
lissack.com · anticipatoryagent.com · ORCID · Google Scholar
The Room, and Beyond It
Keynote, dialogues and live address
- Harry CollinsCardiff University · keynote
- David SnowdenThe Cynefin Company · live from the UK
- Lorenzo MagnaniUniversity of Pavia
- Ricardo Baeza-YatesNortheastern University · Pompeu Fabra
- Owen MatsonBuberian exchange
- Feng TaoNankai University
- Alice Jing ShanScholarLand Ltd
- Thomas PlougAalborg University Copenhagen
Working sessions
- Hugo LeticheDean, ISTEC Paris
- Lola WoetzelMcKinsey & Company
- Marco PalombiHuman-centric platforms
- Aaron KaganGraspingAI
- Brenden MeagherSecond Order Science Foundation
- Dan ZhuGenerative-AI artist
- David IngPresident, ISSS
- Gary MetcalfFormer President, ISSS and IFSR
Scholars and participants
- Brent Que
- Frank Chen
- Giorgio Zampirolo
- Joachim Wiewiura
- Ludovic Curtil
- Peter Friess
- Shuqin Ma
- Terry Daniel
- Faculty and invited researchers of Tongji University and SUES
Remote participants
- Aleksandar Fatic
- Anca-Simona Horvath
- Andrew Rixon
- Denisa Reshef Kera
- Ezio Manzini
- Fabio Anza
- George Por
- Gualtiero Piccinini
- Ian Tepoot
- John Hawkins
- Magda Romanska
- Marianna Simonyan
- Moti Mizrahi
- Natasha Vita-More
- Piero Dominici
- Ratnish Malhotra
- Robert Lowe
- Roger Hunt
- Ruth Glendinning
- Spyros Aleiferis
- Thomas Biedermann
- Thomas Powers
- Umer Asgher
- Viviana Polisena
Thinkers whose work was put to the room
- Sherry TurkleMIT
- Markus BuehlerMIT
- Dave SnowdenThe Cynefin Company · also live
- Don NormanUC San Diego · made his own
- Joanna BrysonHertie School
- Andy ClarkUniversity of Sussex
- Ken FriedmanTongji University · She Ji
- Cameron TonkinwiseUTS Sydney · made his own
Apart from the provocations Don Norman and Cameron Tonkinwise made themselves, and Dave Snowden’s live session, these were provocations composed by Michael Lissack for the conference; listing here records whose thinking was examined, not participation in the event.
See the Conversation
Invitation
The two-minute invitation that opened the conversation.
Watch →Intellectual Foundations
Tacit knowledge, abduction, trust, and the boundaries of the machine.
Watch →The Academic Case
The conference’s intellectual framework.
View →The Original Page
The conference as it was announced, before it ran.
View →Program Overview
The four days as planned.
Download →