What the machine is made of

Early in development, while we were still working the kinks out of things, I was showing our image system to a friend who runs campaigns at a feminist environmental organization. She wanted to see what it would do with an image of her. It put her on the screen with a whole lot of cleavage. In front of her. While I was presenting.

She is not somebody I can imagine ever having worn, or ever wanting to wear, anything that showed cleavage. My face was red, really red..

This is why my developer friends always warn: never do a live demo.

Nobody had asked the image generator for that. It had learned from the data image generators are trained on, which is, of course, the internet. 

The same machinery shows up wherever you look for it. For a long stretch, asking an image generator for a CEO returned a white man, every time, and it took a while for anybody outside the labs to notice how or why. That got fixed, for most models. Ask now and what comes back is so contrived, whether it is a carefully balanced act of tokenism or an immaculate calculation of race, gender and identity, that it reads as a performance of itself.

The correction though was also built on an assumption we’re not stating out loud. It assumes you want to admire these people. A campaign making an image of a CEO is usually making it because they intend to confront one. We would be happy to leave the half-attempt at representation aside and represent these folks as they mostly are: white, male, straight, and profiting off the backs of us and the planet.

You cannot fix any of this with a simple rule, either. Strip cleavage out of every image of a woman, then try to make a memorial picture of Dolly Parton. I tried last week, after she died on the twenty-fifth of August. She built her entire public self out without shame for her body, on purpose, and spent fifty years being in on the joke. She told the story her whole life. The woman she modeled herself on was the one her town called the town tramp: red lipstick, long red nails, and high heels. People said she was trash. Dolly (may she rest in peace) said that was what she wanted to be when she grew up, a sentiment us occasional drag queens have always looked up to her for. A rule written to protect women erases her, and any of us who aspire to the high fashion of high femme fabulousness.

The first output, a group of white male CEOs, was accurate. Women are enormously underrepresented in the C-suite, and the machine was showing the world as it is. The corrected one shows a world that does not exist yet.

Finding this all confusing? Looking for a simple solution to AI bias? I have some bad news for you. There is none.

What worries me about bias is not mainly what it gets wrong about the present. It is whether we can use these tools to represent a future we are trying to bring about. Politics is the exercise of a society deciding what it wants to become.

That is a different problem from a machine getting something wrong.

A large language model is built out of the prevailing consensus. It is trained on what has already been written, weighted heavily toward the Global North, and it returns the middle of all that with enormous confidence. It is not a neutral instrument with a bias problem to be patched. It is a consensus engine, working exactly as designed.

Transformational politics means moving the consensus. An LLM is a machine built out of the consensus.

Transactional politics can live with that. If the job is to find the people who already agree and move them efficiently through a funnel, a machine made of the current consensus is a reasonable instrument for it. Transformational work is the other thing. It is not content to meet people where the consensus currently sits, because underneath the campaign battles we fight the societal ground is always moving, and not always in the direction we want. A tool made of the present, handed to people whose entire job is moving it, points backwards by construction.

This is not a communications problem. It is the delegation of creative participation to a biased machine. The bias is not in the output. It is in who is doing the imagining. Who is filling in the blanks as the work is being done. 

Organizing needs to name lived experience that the public consensus does not carry, which is most of what a community does when it finally gets a microphone. It needs to imagine arrangements that do not exist yet, which is the reason anybody conducts politics instead of administration. A model can do neither, because a model can only return the present.

None of which is an argument for staying away from it.

We have to separate fear of what this technology does from the reality that not training and encouraging its use, particularly among communities already disenfranchised, carries a danger of its own. Refusal is certainly not neutral. The organizations that opt out are disproportionately the ones with the least to spare, and they opt out on behalf of people not in the room. The gates were shut to a small group of insiders for a very long time before any of this arrived.

If the problem is what the machine is made of, the answer is to change what it is made of.

There are two layers underneath what we have built. The first is a movement intelligence layer: fourteen years of organizing work, six hundred campaigns, forty countries, encoded into what the tools ask and what they push back on. Organizing logic where a commercial model has market logic. It is not in any public training data, and it is not the kind of thing that could be, because it was never written down anywhere to be taken. It lived in the consultant class and in people's heads.

The second layer belongs to the organization. Their strategy, their language, their campaign history, the lines they will not cross. The local knowledge stays local and the specialist knowledge arrives from outside, which is the shape the work has had for fourteen years, one client at a time. This is the first time it has been available so broadly. 

Most of what the tools refuse is set there, by them, deliberately and on the record with a reason attached. A short list - mostly a duty of care around the safety for the users and those they have contact with - stays with us and cannot be overridden by anyone, and we publish that list along with why each line is on it. This is while almost no AI company will tell you what it blocks or why.

Knowledge is the easy half of a layer. The other half is refusals.

We went through the image system laboriously, and we are still going through it. Not to remove the bias, which we cannot do. To reduce what can be reduced, to make what remains visible to the person using it, and to leave the choice with them. Where something matters and nobody has said what they want, the system asks rather than filling the blank. Nobody told it to give my friend cleavage. It filled a blank, the way it always does, out of the middle of the data. The fix was to stop defaulting.

A blanket rule would have been faster, cheaper and much easier to defend in public, and it would have produced a system incapable of making an image of Dolly Parton. Lifting a block does not lift the coaching either. The tool still says what it notices, and then it does what you told it.

There is a prior decision underneath all of this, and it is the one that makes the rest workable. The same act means different things depending on who does it and against what. A chain across a factory door means one thing locked by the owner and another locked by workers on strike. No system can read that from a prompt, and we wouldn’t pretend to try. We decide instead who gets an account, which is organizations and communities working against unjust power differentials. That gate does the work that policing a text box never could.

We used to call the thing in that first layer strategic data, and it is not quite the right word for it. Data is a set of things you know. What is in there is a method.

Strategy is not knowledge of a subject. Knowing every campaign poster ever printed is not strategy. It is relational and it is dynamic, lived experience held against one particular situation: this organization, this moment, this town, and the judgment to see the shape of it and then work alongside somebody until the direction comes out of them. The direction was already theirs. Getting it out is the job.

None of that was ever written down, and not because anybody was hoarding it. Articulating your own judgment is a separate skill from having it, and most people who are very good at something cannot tell you how they do it. You cannot scrape a process. Somebody has to be able to say how they think, out loud, in order.

The system does not do strategy. It has no context large enough to point a direction. What the system holds is the method that helps a person do the strategy themselves.

Personal data was the currency of the social media age. Method is the currency of this one. Those are not the same claim, and the difference is most of what this book has been arguing for a month: one was taken from people who were not included, and the other has to be given, deliberately, by somebody willing to explain how they work. Answers arrive faster now. Framing the question in the right direction does not, and it is worth more for that. The designer becomes a strategic designer.

None of this makes a model unbiased. It changes what it draws on and it changes who decided. We have not solved bias. We have made a decision about whose method the machine starts from and where that method points: lived experience of people fighting for a more just world. 

The question was never whether the machine is biased. It is whose thinking it carries, and whether anybody in the room chose. The room where it was built, and the room where an everyday person sits down to make something with this new technology.