Journal of Online Trust and Safety

Protecting Young Users on Social Media: Evaluating the Effectiveness of Content Moderation and Legal Safeguards on Video-Sharing Platforms

Fatmaelzahraa Eltaher — 2026-03-23

Video-sharing platforms such as TikTok, YouTube, and Instagram implement content moderation policies to reduce the exposure of minors to harmful videos. As video has become the dominant and most immersive form of online content, assessing how effectively these systems protect younger users is increasingly important. This study evaluates the effectiveness of video moderation for different age groups on TikTok, YouTube, and Instagram, based on a focused set of experimental accounts. Accounts were created for simulated users aged 13 and 18, and 3,000 recommended videos were analyzed in two interaction modes: passive scrolling and search-based scrolling. Each video was manually assessed for the severity of the harm using a unified harm classification framework. While low-severity harm was the most prevalent form encountered, the results show that accounts configured as 13-year-olds encountered harmful videos more frequently and rapidly than accounts configured as 18-year-olds. On YouTube, 15% of videos recommended to 13-year-old accounts during passive scrolling were classified as harmful, compared to 8.17% for adult accounts, with exposure occurring within an average of 3:06 minutes. This exposure appeared without user-initiated searches, highlighting weaknesses in algorithmic filtering. Results from our targeted study point to gaps in video moderation systems, suggesting the need for more effective safeguards to better protect minors from harmful online content.

Public Support for Misinformation Interventions Depends On Perceived Fairness, Effectiveness, and Intrusiveness

Catherine King — 2026-03-23

The proliferation of misinformation on social media has concerning possible consequences, such as the degradation of democratic norms. While recent research on countering misinformation has largely focused on analyzing the effectiveness of interventions, the factors associated with public support for these interventions have received little attention. We asked 1,010 American social media users to rate their support for and perceptions of ten misinformation interventions implemented by the government or social media companies. Our results indicate that the perceived fairness of the intervention is the most important factor associated with support, followed by the perceived effectiveness of that intervention and then the intrusiveness. Interventions that supported user agency and transparency, such as labeling content or fact-checking ads, were more popular than those that involved moderating or removing content or accounts. We found some demographic differences in support levels, with Democrats and women supporting interventions more and rating them as more fair, more effective, and less intrusive than Republicans and men do, respectively. It is critical to understand which interventions are supported and how they are perceived, as public opinion can play a key role in the rollout and effectiveness of policies.

“I Tend to Run to Problems That People Run Away From”

Toby Shulruff — 2026-04-22

Platform-side Trust and Safety (T&S) is the crucial paid work of responding to and mitigating harmful content and behavior online and beyond. It is characterized by complexity, ambiguity, urgency, and trade-offs based on competing values across a constantly morphing landscape of technologies, abuses, and actors. Interviews with 47 T&S professionals suggest that their expertise is rooted in affective-relational skills: seeking multiple perspectives, reflexivity, curiosity, and collaboration. Furthermore, our findings suggest T&S professionals are motivated by a desire to protect users and spaces, the intellectual challenges inherent in the work, and the caliber of their colleagues. However, the fundamental challenges of the work are compounded by other conditions: both internal and external misperceptions about T&S, responsibility with limited autonomy, and organizational structures. In contributing a more nuanced and grounded perspective of platform-side T&S, we argue that emotion is not a liability, but rather an essential asset in T&S work. We call for valuing the affective skills and motivations T&S professionals bring to their work with the aim of a shift from coping toward well-being and from individually-borne responsibility toward organizational support.

Uncovering and Overcoming Offender Tactics for Distributing Child Sexual Abuse Material on File-Hosting Services

Kelly Barker — 2026-05-01

File-hosting services play a major role in facilitating the online distribution of child sexual abuse material and child sexual exploitation material (CSAM/CSEM). For example, hundreds of file hosts across the globe have received millions of removal notices issued by the Canadian Centre for Child Protection since 2018. Yet no known research has investigated how offenders exploit file hosts for CSAM/CSEM distribution purposes, or the characteristics of the services they exploit. To address these gaps, we thematically analyzed offenders' posts on 15 Tor-based child sexual abuse and exploitation forums (Study 1) and quantified the characteristics of 93 clear web file hosts known to have hosted CSAM/CSEM (Study 2). Results bring to light several tactics offenders use to distribute CSAM/CSEM stored on file hosts, including sharing tutorials on how to safely upload CSAM/CSEM, "link protection" methods to hide CSAM/CSEM from automated detection (e.g., encryption, altering URLs), and creating Tor-based file hosts designed to store CSAM/CSEM. They have also created Tor-based web applications where offenders can employ all these tactics at once. Further, results demonstrate that offenders tend to use file hosts that facilitate easy, anonymous, and enduring distribution. As such, offenders preferred file hosts that retain files for long periods of time, accept archive files, allow uploads originating from the Tor network, and do not require that users enable JavaScript. These findings highlight several platform design risk factors and point to data-driven, practical, and cost-effective measures that policymakers and online service providers (including file hosts and the Tor Project) can implement to reduce the availability of CSAM/CSEM and the revictimization of children and survivors.

“The Report Button is Just for Decoration”

Victoria Zhong — 2026-05-11

For content creators, whose careers rely on digital visibility, hate and harassment are an occupational hazard that impacts creators’ mental health, and in cases where online harassment spills offline, physical safety. We interviewed 19 YouTube creators on their experiences with and strategies used to combat hate and harassment, focusing on platform-provided tools, to understand their needs and identify areas for improvement. While participants did report offensive content, they did not find the platform’s reporting feature useful and felt they could not rely on it for remediation or support. Instead, they primarily used platform-provided moderation tools, social media hygiene practices, and other creators’ influence to manage the abuse they receive. Additionally, we found that harassment extended beyond the overt abuse perpetrated by bad actors and included seemingly innocuous interactions from their audience. Creators thus had to factor both external threats and intracom-munity dynamics into their threat model. The persistence of these issues across years of research suggests that, absent changes in incentives or policy reforms, it is unlikely that platform improvements alone will meet user safety needs. We discuss how external factors contribute to these challenges or constrain solutions.

Trust and Safety in Social XR

Dennis Redeker — 2026-05-11

Social Extended Reality (XR) platforms introduce new challenges for content moderation. Unlike traditional social media, XR enables embodied, immersive interaction—intensifying the psychological and social impacts of online harms such as violence, sexual harassment, manipulation, and impersonation. Drawing on an analysis of platform policies and moderation practices, this paper examines how social XR platforms govern these risks. We find that legacy content moderation strategies, such as algorithmic content moderation, are insufficient for the novel characteristics of XR, where harmful material can consist of non-verbal, spatial, and highly engaging behavior. Comparing VRChat’s structured policy framework with Horizon Worlds’ (now Worlds’) more fragmented approach, we highlight gaps in policy clarity, enforcement transparency, and user protection. The paper contributes to emerging debates on platform governance in immersive media, arguing that both state and platform actors should recalibrate their approaches to accountability, real-time moderation, and jurisdictional oversight. We argue that content moderation in XR is not merely a technical challenge—it is a socio-political dilemma requiring participatory, rights-respecting solutions rooted in human rights norms.

Social Proof is in the Pudding: The (Non)-Impact of Social Proof on Software Downloads

Lucas Shen — 2026-05-13

Open-source software is widely used in commercial applications. Paired with the fact that developers often use social proof as a cue when choosing which open-source software to use, these two facts raise concerns that bad actors can game social proof metrics to induce the use of malign software. Here we study questions around the effects of such gaming using two field experiments on the largest developer platform, GitHub. To examine the impact of social proof, we bought ‘stars’ for a random set of GitHub repositories containing recently created Python packages, and estimated their impact on package downloads and broader repository activity. We find no discernible impact on downloads, nor on forks, pull requests, issues, or other measures of developer engagement. In our second field experiment, we manipulated the number of human downloads for Python packages. Again, we find no detectable effect on subsequent downloads or on any measure of repository activity. Our findings suggest that modest manipulation of social proof does not detectably shift developer adoption, though the threat may grow at higher manipulation intensities and in less-scrutinized contexts such as agentic coding, motivating platform signals that are harder to fake.