New report tackles malicious flagging and de-platforming across social media platforms

A report by a Northumbria University researcher, detailing policy recommendations to tackle platform governance inequalities in the enforcement of ‘flagging’ and ‘de-platforming’ on social media, has been launched today.

Dr Carolina Are, whose research specialises in the intersection between online abuse and censorship, has penned the Co-designing platform governance policies report in collaboration with The World Wide Web Foundation and Superbloom, as part of a wider research project she is leading on at the Centre for Digital Citizens (CDC).

As a user with personal experiences of online abuse and de-platforming, Dr Are is interested in re-designing platform policies surrounding de-platforming, also known as account deletion, and malicious flagging, or the misuse of the reporting tool to silence specific accounts.

Her research has shown that the flagging or reporting tool on social media platforms allows malicious users with opportunities to de-platform or ‘ban’ social media users with whom they disagree. This has disproportionately affected marginalised users like LGBTQIA+ (lesbian, gay, bisexual, transgender, queer, questioning, intersex, or asexual), BIPOC (black, indigenous, and other people of colour), nude and body-positive content creators, pole dancers, but also journalists and activists.

The idea underpinning this report is that content moderation often fails to take the human experience into account, lacking in the necessary empathy for users who are experiencing abuse, censorship, loss of livelihood and network as well as emotional distress. According to the report, being de-platformed from social media often leaves users unable to access work opportunities, information, education as well as their communities and networks – and research has found this has adverse mental health and wellbeing impacts.

Dr Are, who is an Innovation Fellow in the Department of Psychology at Northumbria, said: “In my research and personal interactions with social media companies I have learnt that, too often, the time, resources and attention allocated to engagement with the stakeholders who are directly affected by technology are awarded sparingly.

“The idea underpinning this report is that content moderation often fails to take the human experience into account to prioritise speed and platform interests, lacking in the necessary empathy for users who are experiencing abuse, censorship, loss of livelihood and network as well as emotional distress.

“As a result, this report is a free resource for both users to feel seen in a governance process that often erases them and, crucially, for platform workers to avoid escaping stakeholder engagement in the drafting of their policies.”

Co-designed with 45 end-users, who Dr Are says are often ignored when drafting the rules governing the spaces they depend on for their social and work lives, the report’s recommendations were gathered through a series of workshops structured according to The World Wide Web Foundation’s Tech Policy Design Labs (TPDLs) playbook.

Because users found that current legislation falls short of protecting them on social media, participants in the workshops pushed for radical transparency and for a duty of care by platform conglomerates, demanding information, workers’ rights, and compensation when platforms fall short of protecting their users from censorship and/or from abuse.

The report, therefore, provides social media platforms with user-centred and research-informed recommendations to improve the design and effectiveness of their flagging and appeals tools.

The workshops were supported by Northumbria University’s Policy Support Fund and by the Engineering and Physical Sciences Research Council and ran together with Ageless Citizen Innovation Fellow and product designer, Dr Henry Collingham.

Download the full report here.