Society-in-the-Loop

Iyad Rahwan’s society-in-the-loop proposal extends human-in-the-loop oversight to the people affected by algorithmic systems. Its social contract involves negotiating stakeholder values and monitoring compliance.

The 2017 paper presents a governance framework and research agenda, not a finished method for encoding society’s values.

For Social Alignment, the question is who participates in setting the terms and who can challenge the outcome. One operator’s approval is not automatically agreement from everyone affected.

Iyad Rahwan distinguishes negotiation about benefits and costs from simple operator approval. Training Socially Aligned Language Models in Simulated Human Society draws on simulated interactions, whose agreement does not itself establish consent by affected people.

Institutions as part of the risk model

Current and Near-Term AI as a Potential Existential Risk Factor proposes impaired collective decision-making as an intermediate risk factor (§5.1).^[2] An editorial question for a social contract is therefore whether affected people retain the capacity to investigate a problem, deliberate and revise the terms. Negotiation on paper needs institutions capable of acting on it.