In this essay, Cory Doctorow discussed why content moderation should be treated like other security measures such as encryption, where security through obscurity is not security. Although some parts of content moderation are open to debate, others are more like adversarial attacks: deliberate attempts to circumvent policies, similar to attempts to break encryption algorithms.
With that, Cory Doctorow recommends to ditch the idea of relying on hidden policies to prevent attackers from finding holes, since harassers have the time and resources to uncover them through trial-and-error anyways. Instead, those policies should be made available to the public, and be battle-tested until they reach the point that even if the harassers know the rules, they can’t find any loopholes no matter how much effort they pour.
I can see some parallels here between search engines and SEO. As mentioned by @Colin before, search engines like Mojeek currently rely on security through obscurity to prevent SEO’s deliberate attempts to game the ranking. I wonder if a similar method can be applied, to make search engine’s algorithm public and have it battle-tested against SEO until such a point where it becomes impossible to game the ranking.
I would like to hear Mojeek and other users thoughts on these.
Do you agree with this article, that it’s time to move away from security through obscurity for content moderation?
Do you think that a similar approach would work against SEO garbage? Or if not, what factors do you think make it impossible for search engines to employ that tactic?
If yes, would Mojeek be able to create such a system?
Do you have ideas or suggestions on how such a system could be implemented on content moderation or against SEO garbage?
Thanks for posting this. I read it back in 2022 and as ever, from Cory, it’s thought provoking.
Has anyone actually tried such an approach? If such an innovation can succeed perhaps it will arise. Personally I have massive doubts that it would at scale. But I am not an expert at all.
My quick take is that content moderation extends well beyond security and any approach should consider that wider context. But those handling content moderation, notably on social media, are better placed to comment on whether such an approach can succeed. I suggest you read Yishan, who once ran Reddit, and has written about this challenge. As he eloquently points out content moderation is in fact a signal-to-noise management problem, not a content problem. Seen that way one can see more of the parallels to SEO. But still the differences between tackling SEO spam and content moderation spam are very significantly different.
Thanks! That’s a pretty long thread but a very insightful one. So it seems that the challenge in creating a successful algorithm is less about the technology, more about the acceptance. That people need to change their thinking on content moderation, that it’s less about the content, more about the behavior, to have a consensus of what constitutes an effective content moderation tool.
Can you enumerate in what ways do they differ? So far I see the challenge to be quite similar, that SEO garbage have patterns that can be detected, just that some would protest loudly if you did so (even though it would be beneficial to almost all users). Perhaps a method like what Block Party did could work, i.e, stash the sites filtered on a box that users can check? That would avoid censoring content, allowing users to compare the sites filtered to the search results, and let users provide feedback if there are mistakes.
Or are there other issues that I have overlooked? Like is it because contents in sites are longer than tweets and you lack computing power as @Josh has mentioned before regarding snippets?
“Content moderation” implies on-page SEO. But SEO involves both on-page and off-page (incoming links) optimization. A search engine has to pay attention to both.
I don’t think it matters if the ranking algo for a given search engine is published or secret because SEO’s will look for patterns as to why one page ranks well for a given, money-shot, term and another page does not and then will try to reverse engineer a competitor’s success.