Journal of Online Trust and Safety

Trust and Safety of Human- and AI-Powered Visual Description: A Case Study of Be My Eyes

Natalie Grace Brigham — 2026-09-30

Be My Eyes provides a volunteer-powered visual description service that connects over one million blind and low vision users with more than ten million volunteers for everyday visual assistance tasks as well as Be My AI, an AI-powered counterpart. In this work, we qualitatively analyze 1,428 one-, two-, and three-star reviews from the Apple App Store and Google Play Store to holistically examine the concerns and harmful experiences of both users and volunteers. Our analysis surfaces online harassment and abuse (trolling, cyberflashing, reporting failures), privacy concerns beyond information exposure (data use for model training), and accessibility/safety problems (inaccurate descriptions, paternalistic guardrails). We additionally analyze platform policies to examine whether and how these issues are addressed by them. Based on challenges we find related to accessibility, security, privacy, and AI, we provide recommendations for strengthening trust and safety on visual description services that serve blind and low vision people’s access needs.

AI and Agency: Benefiting Users by Understanding and Serving Explicit Aspirations

Emily Xing — 2026-09-30

This paper introduces a framework for understanding how generative AI affects user agency, drawing on psychological research distinguishing between expressed preferences and immediate, often-regretted desires. We conducted a study examining how AI usage relates to users’ self-reported capacity for accomplishing goals along with the potential benefits of providing users with a sense of control over AI tools. This study with a nationally representative sample of U.S. generative AI users examined users’ aspirational preferences for AI usage; where they perceive capability (outcome control) gains from AI; users’ perceived control over AI tool experiences (process control); the relationship between perceived control over AI experiences and reported capability gains; where users locate their concerns about AI; and whether reported gains differ across demographic groups. Results indicate that users primarily both value and report benefit from AI in instrumental domains (e.g., task completion, information gathering, and learning). Aspirations for, and experiences of, benefits related to social and affective domains were less common. Users most commonly reported concerns about AI eroding independent thinking and agency. Importantly, users who felt more empowered to shape AI outputs (process control) also reported greater capability gains (outcome control), although exploratory analyses suggest these gains are unevenly distributed across demographic groups. We argue that centering user agency, encompassing control over both product experience and product outcomes, should be a primary design principle.

The Backbone of Abuse: How Infrastructure Providers Enable the Proliferation of AI-Generated Non-Consensual Intimate Imagery

Sarah Morgan — 2026-09-30

Advances in, and democratization of, generative-AI technologies have facilitated a phenomenal and troubling rise in the creation and sharing of AI-generated non-consensual intimate imagery (NCII). Although NCII is illegal in several countries, we identified 88 publicly available sites hosting it, with 38 of them fully dedicated to synthetic imagery. We found that a handful of the world’s biggest internet companies provide the underpinning infrastructure (web hosting, content delivery network, etc.). Cloudflare, for example, provides web hosting for over half of the identified sites, and the vast majority of sites can be surfaced via a Google search. We call on these companies to do better: they must revise their policies to ensure they are not facilitating the creation and sharing of NCII. More broadly, we enumerate specific regulatory reforms to mandate that companies take responsibility for the illegal content they allow to proliferate online. Content warning: this paper contains sexually violent language and describes sexually explicit content.

“It Doesn’t Violate Our Community Standards”: Examining Social Media Platforms’ Moderation Practices for User-Reported Rule-Breaking Tobacco Content

George Pearson — 2026-09-30

Growing concerns regarding harmful content on social media platforms underscore the need for systematic examination of these platforms’ advertised safeguards. User reports are a key mechanism through which platforms can be apprised of and address harmful content. This study uses tobacco content as an important public health context to examine how strictly social media platforms moderate content that violates their own community guidelines after it is reported through user-reporting mechanisms. Using a replicable observational study, we tested the responsiveness to user reports across the three platforms most popular with youth: Instagram, TikTok, and YouTube. We identified 294 cases (between 97 and 99 per platform) violating platforms’ community guidelines covering five content areas: insufficient age-gating, disallowed promotions, illegal sales, underage use, and dangerous activities. Cases were reported through the platform’s reporting mechanisms and monitored for at least two weeks for moderation actions. Of a total of 294 cases, 221 (75%) were unmoderated. Instagram moderated 51 of 98 (52%) cases, YouTube 14 of 97 (14%), and TikTok 8 of 99 (8%). Insufficient age-gating was the only category with more than 50% of cases moderated (29 of 56). Less than 10% of dangerous activity (1 of 29) and disallowed promotion (4 of 79) cases were moderated. Results show that obvious violations of community guidelines reported to the platforms through their own mechanisms were rarely addressed, subjecting users to harmful, and sometimes illegal tobacco-, nicotine-, and cannabis-related content. Findings call into question whether the platforms can be relied upon to self-regulate to protect users from harmful tobacco content online.

Regulating the “Unregulatable”: Why Policymakers Are in Paralysis When Regulating AI

Sarah Barrington — 2026-09-30

Policymakers confronting generative AI have often accepted a striking premise: that AI is too complex, too fast-moving, and too “unprecedented” to be governed by existing frameworks. We argue that this premise is itself part of the problem. Many harms associated with generative AI, including fraud, impersonation, deceptive advertising, and exploitative synthetic media, are not wholly new categories of threat. They are familiar harms, now produced and distributed more cheaply, more quickly, and at greater scale. To move beyond the current policy paralysis, we propose decomposing AI into three layers: the model layer, the deployment layer, and the distribution layer. Once disaggregated in this way, AI no longer appears as an ungovernable monolith, but instead as a supply chain of software products and infrastructures that already sit within consumer protection, product liability, sectoral, and platform-governance regimes. Governing actors should stop treating the absence of legal authority as the obstacle. The real challenge, we argue, is the persistence of AI exceptionalism and the resulting under-enforcement of ordinary law.

Abuse Is Not Love: Failure to Recognize Technology-Facilitated Intimate Partner Violence

Nina Jane Patel — 2026-09-30

Technology-facilitated intimate partner violence (TFIPV) refers to the use of digital technologies, including smartphones, social media, messaging applications, location tracking services, and artificial intelligence (AI)-powered tools, to monitor, coerce, harass, or control a current or former intimate partner. Across 10 countries, nongovernmental organization (NGO) partners of the YSL Beauty Abuse Is Not Love program consistently find that young people aged 16 to 24 do not recognize TFIPV. Drawing on semi-structured interviews with 10 NGO experts working in domestic violence prevention and survivor support, this commentary suggests that surveillance may have become normalized to a degree that makes it difficult for many young people to distinguish love from abuse. The analysis identifies four thematic areas of recognition failure: normalization, parent/caregiver modeling, a language gap in naming abuse, and legal enforcement failures. A fifth emerging theme, AI-generated harms, is also discussed. The commentary presents the nine warning signs of abuse adapted to TFIPV as a practical heuristic for making digital control visible. A surveilled relationship is not a safe relationship. Ending TFIPV requires shifting the social, legal, and technological structures that have erased the distinction between love and abuse.

Mandating Data Interoperability as a Tool for Improving Youth Wellbeing on Social Media

Alexis Hiniker — 2026-09-30

Many social media platforms are filled with problematic designs that range from annoying (e.g., excessive notifications) to devastating (e.g., bulk-add features that have connected children to pedophiles). Users express frustration with these designs and yet continue to use these platforms. Why? If an email client sent constant clickbait notifications and intermixed sponsored content with true emails, many users would not tolerate it; they would switch to a different email platform and continue emailing the same contacts. This flexibility is only possible because email obeys a standardized protocol allowing for data interoperability across platforms. If all social platforms were required to make user-posted content consumable by third parties, new and existing social media companies alike would be forced to compete for their users and could only succeed by offering users experiences they value. Users would no longer be held captive by the platforms they join and could continue connecting with the same network of people from any platform they choose. Mandated interoperability on social media platforms is not a new concept and is already under consideration in state legislatures across the United States. The purpose of this commentary is to distill its relevance to youth wellbeing online.

The Far-Reaching Implications of Brazil’s Supreme Court Decision for Platform Governance

Beatriz Kira — 2026-09-30

In October 2025, Brazil’s Federal Supreme Court declared Article 19 of the Marco Civil da Internet partially unconstitutional, fundamentally restructuring the country’s intermediary liability regime. For nearly a decade, platforms could only be held liable for third-party content upon failing to comply with a specific judicial removal order, subject to limited statutory exceptions. Concluding that this model no longer afforded adequate protection to fundamental rights and democratic institutions, the Court replaced it with four distinct liability regimes while introducing a duty of care alongside new transparency and due-process obligations. This commentary examines the ruling and situates it within Brazil’s institutional and political context, arguing that this decision reaches far beyond the allocation of liability in individual cases to influence broader governance arrangements applicable to digital platforms. Although adopted through constitutional adjudication rather than changes in legislation, the decision responded to pressing structural concerns and shifted Brazilian platform regulation toward a preventive, risk-oriented model that has helped to shape a hybrid framework combining judicial standards with regulatory supervision.

Beyond Content-Based Enforcement

Jeffrey Howard — 2026-09-30

Behavior-based interventions involve restricting users based on what they do, rather than what they post. This practice is already established at major platforms. However, it has received little explicit philosophical scrutiny. This essay explores behavior-based interventions from the perspective of ethics and political philosophy. It first distinguishes interventions targeting behavior that is itself a rule violation (such as impersonation) from the harder class of cases in which no single action breaches a rule, but a combination of signals indicates misuse. It then argues that, because such interventions are best understood as defensive, the philosophical literature on defensive ethics yields the relevant criteria they must satisfy. Interventions should respect user autonomy by being triggered by conduct for which the user is responsible, ruling out demographic profiling and purely association-based signals. They should be suitable, necessary, and proportionate, with particular attention to the risk of false positives. The essay also defends the importance of transparency, graduated interventions, and meaningful appeals mechanisms. It flags the distinctive danger of governments seeking platforms’ behavioral predictions. And it closes by urging platforms to institutionalize engagement with scholars, civil society, and affected rights-holders.