Sort by Controversial: training machine learning to sow irreparable divisions

Scott Alexander continues to delight with his works of short, sharp science fiction (previously): this time, it's 'Sort by Controversial,' a teachnolovecraftian story of training a machine learning system to recognize (and then produce) 'controversial' stories by exploiting Reddit's 'sort by controversial' feature to obtain training data.

Alexander's mcguffin is something called 'Shiri's Scissor,' a machine learning system that produces polarizing statements whose deceptive obvious rightness (or, alternately, wrongness) pits people against one another so violently that once you've been scissored, your peace is forever fractured.

It's a lovely tale in the tradition of Lexicon and Snow Crash, turning on the use of algorithms to locate 'spells' whose utterances destroy our ability to think clearly -- and as such, it's a wonderful metaphor for the engagement-maximized political climate we find ourselves imprisoned by.

Shiri’s English wasn’t great, so I thought this was a communication problem. I corrected her. The program was spitting out obviously false statements. She stuck to her guns. I still thought she was confused. I walked her through the meanings of the English words “true” and “false”. She looked offended. I tried to confirm. She thought this abysmal programming decision, this plan of combining every bad design technique together and making it impossible to ever fix, was the right way to build our codebase? She said it was. Worse, she was confused I didn’t think so. She thought this was more or less what we were already doing; it wasn’t. She thought that moving away from this would take a total rewrite and make the code much worse.

At this point I was doubting my sanity, so we went next door to Blake and David, who were senior coders in our company and usually voices of reason. They were talking about their own problem, but I interrupted them and gave them the Scissor statement. Blake gave the reasonable response – why are you bothering me with this stupid wrong garbage? But David had the same confusion Shiri did and started arguing that the idea made total sense. The four of us started fighting. I still was sure Shiri and David just misunderstood the question, even though David was a native English-speaker and the question was crystal-clear. Meanwhile David was feeling more and more condescended to, kept protesting he wasn’t misunderstanding anything, that Blake and I were just crappy programmers who couldn’t make the most basic architecture decisions. He kept insisting the same thing Shiri had, that the Scissor statement had already been the plan and any attempt to go in a different direction would screw everything up. It got so bad that we decided to go to Brad for clarification.

Sort By Controversial [Scott Alexander/Slate Star Codex]

(via Fipi Lele)