
RIGHT- AND LEFT-WING VIOLENT EXTREMIST ABUSE OF DIGITAL TECHNOLOGIES IN SOUTH AMERICA, AFRICA AND ASIA
33
In response to these threats – to user safety, to their own reputation, and to their revenue
90
– the
larger technology companies each employ as many as tens of thousands of people focused on
content moderation,91 and there are a growing toolkit of technical approaches92 and a burgeoning
industry of third-party vendors.93 Moderators working for Facebook, the world’s largest social
network, reportedly made three million daily moderation decisions in 2020, while by 2024 its
users were uploading billions of pieces of content per day.
94
Recently, however, some social
media companies, such as X (formerly Twitter), have severely cut their internal resources for
content moderation, and their safety and public policy personnel, with potential repercussions
for existing measures to tackle terrorist and violent extremist content and online hate overall.95
Some experts argue that ongoing advances in AI-driven tools, which have long played a role
in content moderation systems, are likely to revolutionise the ability of technology companies
effectively to detect and remove illegal content posted on their services.
96
In their current form,
however, such technologies do not appear to have comprehensively resolved issues around
interpreting contextual linguistic nuance, reliable sound, weapon detection, or the detection of
content that has been deliberately modied.97
Some platforms appear to have dedicated disproportionate resources to moderating English-lan-
guage content compared with other languages. According to internal company les leaked by a
whistleblower in 2021, for example, Facebook spent 87% of its budget on combating misinfor-
mation on English-language content at a time when just 9% of its users were English-speaking.98
Transparency reports published in 2023, as required by the European Union’s Digital Services
90 Ryan Mac and Kate Conger, “X May Lose up to $75 Million in Revenue as More Advertisers Pull Out”, The New York Times, 24 No-
vember 2023, available at: https://www.nytimes.com/2023/11/24/business/x-elon-musk-advertisers.html; Tiffany Hsu and Elea-
nor Lutz, “More than 1,000 companies boycotted Facebook. Did it work?”, The New York Times, 1 August 2020, available at: https://
www.nytimes.com/2020/08/01/business/media/facebook-boycott.html.
91 Paul M. Barrett, “Who moderates the social media giants?”, Center for Business and Human Rights, New York Universi-
ty, June 2020, available at: https://static1.squarespace.com/static/5b6df958f8370af3217d4178/t/5ed9854bf618c710cb-
55be98/1591313740497/NYU+Content+Moderation+Report_June+8+2020.pdf.
92 T.G. Thorley and E. Saltman, “GIFCT tech trials: combining behavioural signs to surface terrorist and violent extremist content
online”, Studies in Conict and Terrorism, 1-26 (2023), available at: https://doi.org/10.1080/1057610X.2023.2222901; “GIFCT tech-
nical approaches working group: Gap Analysis and recommendations for deploying technical solutions to tackle the terrorist use
of the internet”, Tech Against Terrorism and the Global Internet Forum to Counter Terrorism, July 2021, available at: https://gifct.org/
wp-content/uploads/2021/07/GIFCT-TAWG-2021.pdf.
93 Tim Bernard, “The evolving trust and safety vendor ecosystem”, Tech Policy Press, 24 July 2023, available at: https://www.techpol-
icy.press/the-evolving-trust-and-safety-vendor-ecosystem/.
94 John Koetsier, “Report: Facebook makes 300,000 content moderation mistakes every day”, Forbes, 30 June 2021, available at:
https://www.forbes.com/sites/johnkoetsier/2020/06/09/300000-facebook-content-moderation-mistakes-daily-report-says/; “41
up-to-date Facebook facts and stats”, Wishpond, available at: https://blog.wishpond.com/post/115675435109/40-up-to-date-
facebook-facts-and-stats.
95 “Report reveals the extent of deep cuts to safety staff and gaps in Twitter/X’s measures to tackle online hate”, eSafetyCommission-
er, Australian Government, January 2024, available at: https://www.esafety.gov.au/newsroom/media-releases/report-reveals-the-
extent-of-deep-cuts-to-safety-staff-and-gaps-in-twitter/xs-measures-to-tackle-online-hate.
96 Interview with Adam Hadley, Tech Against Terrorism, 20 June 2024; Interview with Senior Tech Company Representative, 04 July 2024.
97 Interview with Nicole Matejic, Charles Sturt University, 12 June 2024; Billy Perrigo, “Social media companies vowed to stop videos
of terror attacks. Buffalo showed they have more work to do”, Time, 17 May 2022, available at: https://time.com/6177640/buffa-
lo-shooting-twitch-social-media/; Stuart Macdonald, Ashley Mattheis and David Wells, ”Using Articial Intelligence and Machine
Learning to Identify Terrorist Content Online”, Tech Against Terrorism Europe, 15 January 2024, available at: https://tate.techagain-
stterrorism.org/news/tcoaireport.
98 Dan Milmo, “Facebook revelations: what is in cache of internal documents?”, The Guardian, 25 October 2021, available at: https://
www.theguardian.com/technology/2021/oct/25/facebook-revelations-from-misinformation-to-mental-health.