Note: This post contains the text of my opening keynote at Deloitte’s Digital Platform Trust Forum 2025 (US Edition), which was held in Mountain View, California on November 13, 2025.

tl;dr: Digital platforms are in their “OSHA moment.” Just as the passage of the OSH Act of 1970 moved workplace safety from voluntary codes to enforceable standards, recent frameworks like the EU’s Digital Services Act are compelling platforms to make their safety systems auditable while still holding them accountable for safety outcomes. Meeting this challenge requires integrating the Trust & Safety and Compliance disciplines into product design from the start, an investment that’s becoming both legally unavoidable and strategically essential. The platforms that will lead aren’t the ones resisting this shift; rather, the leaders will be the ones turning accountability into a competitive advantage by engineering trust as deliberately as they engineer features.

Introduction

It’s a bit surreal to be speaking after seeing a fireside chat with a version of myself that doesn’t technically exist. 

But that deepfake demonstration makes the point better than any slide could: we’ve reached a moment where even seeing isn’t necessarily believing.

Likewise, we can’t assume trust. It has to be built, proven, and continually reinforced. That shift – from assuming trust to deliberately engineering it – is exactly what I want to talk about today. 

Expectations for digital platforms have changed. It’s no longer enough to make promises about safety, integrity, or responsibility; regulators now expect proof that those promises are being kept. 

Accountability has become a complement to reputation – a way to show that integrity isn’t just claimed, but confirmed. 

While that may feel new in the digital world, we’ve seen it before in another industry, in another era, and that’s where I want to start. 

Workplace Safety and OHSA

When the Occupational Safety and Health Administration, also known as OSHA, was created in 1970, it wasn’t because companies suddenly discovered empathy. 

It was because in the late 1960s, workplace accidents in the United States were killing about 14,000 people a year and injuring millions more.

Before OSHA, workplace safety was a patchwork of state laws, voluntary codes, and after-the-fact investigations. Every factory, refinery, or mine was essentially on its own, and the result was grim. It was clear that voluntary self-regulation wasn’t working and, instead, a federal standard was required to ensure every worker, regardless of state or industry, was protected (as the law says) “so far as possible.”

OSHA didn’t set out to make companies more caring; it set out to make them more accountable.

It established enforceable national standards, created a framework for inspections, and backed it with penalties. While OSHA didn’t make safety self-sustaining overnight, it did move safety from being an afterthought to being an expectation. Today, you don’t need to convince any serious company that safety matters. The only question is how far beyond baseline compliance they choose to go. The rulebook still matters; it’s just no longer the whole story. 

Of course, that progress hasn’t made OSHA obsolete. Violations still happen, enforcement still matters, and culture still drifts. But since OSHA began in 1971, workplace fatality rates in the United States have fallen by roughly eighty percent. The progress hasn’t been perfect, but the long-term trend is unmistakable: safety became a core expectation.

OSHA showed how clear standards, consistent accountability, and transparency can shift industry norms. In the digital realm, we now have the chance to apply the same principles proactively through the systems and safeguards we design ourselves.

From Safety to Digital Trust

For the past two decades, online safety has operated primarily through self-regulation. Trust & Safety teams built policies, published transparency reports, ran investigations, and created the world’s first frameworks for digital accountability largely because it was the right thing to do, not because anyone required it. Though let’s be honest: it was also strategic. It’s better to build accountability proactively than to have it imposed reactively.

Where regulation did exist, it reinforced fragmentation. Privacy over here, payments over there, child safety here, content moderation here, and here, and here . . . The Communications Decency Act (CDA) in 1996, the Children’s Online Privacy Protection Act (COPPA) in 1998, the Payment Card Industry Data Security Standard (PCI DSS) in 2004 – each added a layer of responsibility, but each was still narrow and focused on a single category of risk. None required platforms to examine how their systems created or managed risk. That’s what’s changing now. 

Around the world, governments are moving toward comprehensive frameworks like the EU Digital Services Act (DSA), the EU Artificial Intelligence (AI) Act, the UK Online Safety Act (OSA), Australia’s Industry Codes of Practice for the Online Industry, and Singapore’s Online (Relief and Accountability) Bill. A number of other countries are proposing similar legislation. 

Under the DSA, for instance, very large platforms are now required to assess and publish the systemic risks their services create — from how content spreads to how their algorithms influence what people see and share. 

We are moving from a world that judged platforms only by visible outcomes to a world that judges them by how safely their systems are designed to operate. 

It’s the difference between “Don’t sell spoiled food” and “Show us your entire supply chain, explain how contamination gets detected, and prove your safety protocols actually prevent it.” 

One regulates outcomes. The other regulates systems.

Let’s be honest: most platforms are still developing the processes and infrastructure needed to understand how their systems create risk, let alone to demonstrate how those risks are being managed. 

Here’s what that looks like in practice: 

• a content moderation team removes a video for violating harassment policies;
• That same day, the recommendation algorithm promotes fifty more videos with identical characteristics because the enforcement signal never reached the ranking system;
• Meanwhile, the policy team is analyzing user appeals in one database;
• The machine learning (ML) team is evaluating model performance in another database;
• The compliance team is trying to report on “effectiveness” without having access to either. 

Each team is doing its job. But the platform has no unified view of whether its safeguards are actually working. That fragmentation isn’t just an operational inconvenience; it’s a structural vulnerability that makes harm both more likely and harder to detect. Digital platforms are fundamentally different from factories – they’re not static. An algorithm update can undo a safety feature overnight. Code ships constantly. The platform you audit today isn’t the platform running tomorrow. 

The opportunity in front of us isn’t speed of response; it’s quality of systems – making accountability consistent, reliable, and built-in. Regulators are asking for the same kind of rigor that platforms have been working to develop internally for years – frameworks that make risk management systemic instead of situational. 

Trust & Safety’s frameworks are dynamic and operational, designed to detect emerging risks, evaluate harm, and respond in real time. Compliance’s are structural and evidentiary, designed to align those actions with laws, standards, and audit requirements. 

The real opportunity now is to connect these frameworks. When those systems are integrated, accountability becomes continuous rather than episodic, and the organization gains a clear, defensible understanding of how risk is identified, managed, and improved over time. That’s the same evolution we saw in physical safety half a century ago: the shift from patchwork fixes to accountable systems. 
This is our turning point.

The platforms that learn to do this won’t just meet the new expectations; they’ll define what “trustworthy” looks like for everyone else.

From Reactive to Preventive

Compliance isn’t automatic. Most organizations start reactively, interpreting new requirements, fixing issues as they arise, and only gradually learning how to anticipate and prevent them. It’s still rare for systems to be built where robust, effective safeguards are integrated from the beginning. 

That was true in workplace safety, too. The hazards that led to OSHA’s standards weren’t mysteries; most were known long before regulation forced change. The problem wasn’t awareness, but rather, priority. It took loss, followed by accountability, for prevention to become the default expectation. 

Digital platforms are in a similar place now. Trust & Safety and Compliance have built extraordinary capacity to respond – to manage incidents, investigate harms, engage with regulators, and explain what happened. We can evolve that same capability into something systemic and use what we already know about risk not just to validate the past, but also to shape the design of what comes next. 

Integrating systems, anticipating risk, embedding accountability into design – these aren’t separate, unrelated tasks. They’re different dimensions of a single transition: from compliance as a function to trust as an operating principle.

That transition starts with how we handle what we learn. Every investigation, audit, and user report reveals something about how a system behaves under pressure. When that information is used to inform design and policy rather than just ticking a box, incidents lead to insights that shape how systems work. 

Addressing Cost and Resistance

But here’s what I haven’t said yet, and what everyone in this room is thinking: this shift isn’t free and it isn’t easy. Building systemic accountability requires resources, creates friction, and slows some things down. 

When OSHA was created, industry groups predicted it would cripple American manufacturing. They argued that compliance costs would make companies uncompetitive and that federal overreach would stifle innovation.

They were wrong about the ultimate outcome, but they weren’t wrong about the immediate cost. Safety did require investment. It did change how products got made. Some companies struggled. Some failed. But the industry that emerged was stronger, more sustainable, and more trustworthy. The ones that survived weren’t the ones that fought hardest against accountability; instead, they were the ones that learned to integrate it fastest.

Digital platforms face a similar moment. Building the infrastructure for systemic accountability isn’t cheap. Integrating Trust & Safety with product development adds complexity. Documenting risk assessments takes time. But the alternative – operating without that infrastructure – is becoming both legally untenable and strategically unsustainable. 

The platforms that will lead five years from now aren’t the ones resisting this shift. They’re the ones figuring out how to make accountability a competitive advantage.

If We Were Starting from Scratch

If we were starting fresh today, knowing everything we know now, what would we build?

We’d build platforms where every algorithmic change – every new ranking signal, every content recommendation tweak – would have a documented impact assessment before deployment, not after a crisis. Where enforcement actions automatically generate signals that feed back into the systems that created the risk in the first place. Where product teams have real-time visibility into how their features are being exploited, and policy teams can see immediately whether their rules are having the intended effect.

We’d design organizations where Compliance isn’t just the people who write the audit report six months after launch, but also the people in the room when architecture decisions get made. Where “Can we demonstrate this is safe?” isn’t a question you ask at the end of the development cycle, but a gate you pass through before you build.

We aren’t lacking insight into where things go wrong. Ask any Trust & Safety team, and they can tell you exactly where the gaps are. The gap isn’t visibility – it’s priority. It’s whether those insights reach the people making resource decisions, architecture choices, and product roadmaps.

If we can turn that awareness into shared responsibility, we can address risks before they become crises and show, in practice, what it means to design for trust. 

That’s what trust by design looks like: not perfect, but provable. Not assumed, but engineered. 

The Human Core

At its center, all of this – accountability, compliance, regulations, safety – has never really been about “frameworks” or “systems.” It’s been about people.

People say OSHA regulations are written in blood — that every regulation exists because someone, somewhere, got hurt. Many of our digital safeguards were born the same way. We have the opportunity to shift our approach – to build not just safeguards designed to recover from harm, but also safeguards that anticipate the need to protect. Making that shift requires applying the same rigor to anticipating and mitigating impact as we do to driving progress, so that innovation remains sustainable. 

That is the next step in our evolution. Not waiting for the next crisis to force change, but building systems that prevent it from happening. Not explaining after the fact what went wrong, but demonstrating in advance what’s been made right. 

Proving, through evidence, design, and practice, that safety and trust can be engineered as intentionally as anything else we build. 

That’s not the future of platform accountability; it’s the present. The only question is how quickly we move.

Thank you so much.

Note: The 7 Rules of Causation are part of my upcoming Root Cause Analysis guide, a resource in the Trust & Safety Toolkit.

Introduction

We’ve all been there: an incident has occurred. Something that should not have happened did happen, and now you’re faced with explaining how, exactly, that situation came to pass.

A retro, a post-mortem, a debriefing—whatever you call it, it’s got to be done.

Why Causation Matters in Trust & Safety

Robust and accurate evaluations of the root causes and contributing factors that led to an incident are essential for informed decision-making, preventing future incidents, and improving communication. They also play a key role in maintaining user trust by demonstrating accountability, transparency, and a commitment to safety.

When done right, a Root Cause Analysis strengthens the integrity and effectiveness of Trust & Safety (T&S) operations. As the name implies, an effective Root Cause Analysis identifies and documents the often complex and interconnected sequence of causes and effects that led to the incident.

Determining Causation:
7 Rules for Better Incident Reports

Adapted from David Marx’s 5 Rules of Causation for aviation safety incidents¹, below are the 7 Rules of Causation for documenting the root causes and contributing factors of a T&S incident.

While aviation safety and Trust & Safety may seem quite different at a glance, they share common challenges: complex systems, human decision-making under pressure, and the need to prevent recurring incidents.

When followed, these 7 rules ensure that incident descriptions are accurate, precise, and unemotional. They reflect human factors engineering (HFE) principles because they require us to examine how people interact with systems and to determine how we can make these interactions more reliable, efficient, and error-resistant.

Collectively, the 7 Rules of Causation help us analyze incidents through the lens of human-system interaction rather than treating technical and human factors separately.

Rule 1. Causal statements must clearly show the “cause and effect” relationship

Ensure clear identification of the chain of events that led to the incident.

SCENARIO: A violative post remained visible on the platform for several hours.

• Incorrect: “The system failed.”
• Correct: “The violative post remained visible because the moderation algorithm was not updated, which caused the violative content to go undetected.”

Why It Matters:

• The rule forces clear thinking about the actual chain of events;
• The rule helps us avoid vague conclusions (e.g., “the system failed”) that don’t help prevent future incidents; and
• The rule enables teams to identify specific points of failure that can be addressed.

Rule 2. Negative descriptors are not used in causal statements

Replace subjective or vague terms with factual and specific language.

SCENARIO: User reports are not being resolved in a timely manner.

• Incorrect: “Poor user communication.”
• Correct: “User reports were not resolved promptly because the notification system did not send alerts to the moderation team on time.”

Why It Matters:

• Negative descriptors such as “poor,” “inadequate,” or “bad” are subjective and don’t describe what actually happened;
• The rule keeps the focus on specific, actionable issues rather than judgmental language; and
• The rule keeps the analysis professional and factual rather than emotional or blame-oriented.

Rule 3. Each human error must have a preceding cause

Highlight the systemic or environmental factors that led to human mistakes rather than blaming individuals.

SCENARIO: A moderator incorrectly flags a harmless post as harmful.

• Incorrect: “The moderator made an error.”
• Correct: “The moderator incorrectly flagged the post because they were not properly trained on the new policy guidelines.”

Why It Matters:

• Humans rarely make mistakes for no reason;
• The rule forces us to identify systemic issues (such as training gaps or unclear procedures) that can be fixed; and
• The rule moves us beyond blaming individuals to understanding why they acted or handled the situation the way they did.

Rule 4. Each violation of procedure must have a preceding cause

Examine why a procedure was not followed, addressing unclear or impractical guidelines.

SCENARIO: A reviewer skips a step in the content review process.

• Incorrect: “Procedure was not followed.”
• Correct: “The reviewer skipped a step in the content review process because the guidelines were unclear and the training provided was incomplete.”

Why It Matters:

• People rarely deviate from a prescribed procedure without a reason;
• The rule helps identify if procedures are unclear, impractical, or poorly communicated; and
• The rule can reveal when “workarounds” have become normalized due to system issues.

Rule 5. Failure to act is only causal when there is a pre-existing duty to act

Assign responsibility only where there is a legitimate expectation for the person or group to act, avoiding unwarranted blame based on hindsight.

SCENARIO: Harmful content was not escalated for review.

• Incorrect: “The reviewer failed to escalate the content.”
• Correct: “The harmful content was not escalated because the reviewer was not aware of the escalation protocol even though they had a duty to monitor for and escalate such content.”

Why It Matters:

• We can’t accurately assess causation if we attribute duties that are not specifically assigned to a role;

• The rule helps us distinguish between actual responsibilities and hindsight bias;
• The rule prevents unfairly attributing blame when someone wasn’t responsible; and
• The rule helps identify gaps in role definitions and responsibilities.

Rule 6. Causal search must focus on information transfers, control actions, and human or system interfaces

Identify points where communication, controls, or system interfaces broke down.

SCENARIO: Harmful content is not removed promptly.

• Incorrect: “The content was not removed because of a system error.”
• Correct: “The harmful content was not removed because the user interface of the agent tool’s review queue did not highlight the violation to the moderator, leading to a delay in action.”

Why It Matters:

• The rule directs attention to specific points where interventions can be made;
• The rule helps identify breakdowns in communication or system design; and
• The rule focuses attention on actionable elements rather than abstract concepts.

Rule 7. Specific descriptors are necessary to identify what failed

Avoid vague descriptions like “system error” and provide detailed explanations.

SCENARIO: A violative post is not removed.

• Incorrect: “The system had a bug.”
• Correct: “The violative post was not removed because the content filtering system had a bug that misclassified the post, preventing it from being flagged.”

Why It Matters:

• Vague descriptions like “had a bug” don’t go into enough specific detail to understand why things went wrong;
• The rule forces us to identify precisely what, why, and how things went wrong; and
• The rule enables us to deploy targeted fixes rather than broad, ineffective solutions.

Applying the Rules: An Example

Let’s take an incident write-up and see how applying the 7 Rules of Causation makes a world of difference. 

Initial Draft

Here’s the initial write-up:

The content moderation tool was misclassifying user posts, leading to a significant number of false positives for harmful content. Users reported being wrongly flagged for content violations, resulting in temporary account restrictions. The issue was first identified due to an influx of user complaints received on September 2, 2024, across multiple regions.

The moderation team immediately halted auto-classification and reverted to manual review. Users affected by the misclassification were notified, and account restrictions were lifted after verifying that no actual violations had occurred.

A cursory read shows us what has happened, but it has not explained why and how it happened.

Reviewing the Draft

Let’s review how well the write-up abides by the rules of causation. 

In the annotated image below, we see right away how Rule 1 and Rule 7 are not fully implemented.

We also note that nowhere in this write-up was there a mention of system-level factors (Rule 6) that may have contributed to the incident.

Improving the Draft

To improve the write-up, we first apply Rule 7 (Specificity)

By examining the specifics, we can better understand the incident’s scope (number of users) and what was misclassified (new allow-listed terms). 

On 28 August 2024, the Policy team published OPSUPDATE-037 to add a new set of slang/terms to the Allow List. This policy change is intended to reflect how different marginalized communities have reappropriated these terms. 

The policy change was scheduled to take effect on 2 September 2024 at noon PT. Unexpectedly, the auto-classification bots continued to classify user posts that included the slang/terms as violative.

From noon PT on 2 September 2024 until the auto-classification bots were paused, a total of 94,673 posts were wrongly flagged for content violations, affecting 43,862 unique users and resulting in these user accounts being placed under temporary account restrictions.

The issue was first identified because Anjali Bahri (T&S Policy) noticed on 3 September that user complaints about their accounts being restricted due to these slang/terms continued despite the planned policy change.

Next, we apply Rule 1 (Cause and Effect)

We dig deeper into what happened and identify the surprising series of events that led to the misclassification:

The moderation tool’s algorithm was not updated to handle the new slang and context variations documented in OPSUPDATE-037 because the Heuristics team member who was auto-assigned the ticket based on the on-call rotation had started parental leave earlier than expected.

There is no established process to ensure that already-assigned tickets are reassigned when the on-call rotation is retroactively updated. As such, OPSUPDATE-037 was not bundled with other planned updates and did not ship as scheduled.

There is also no established process within the Policy team for confirming that planned policy updates shipped as scheduled.

Assessing the Result

Notice how the improved write-up now abides by the 7 Rules:

• [Rule 1] Causal statements show the cause of the misclassification (approved policy change was not implemented as expected);
• [Rule 2] There are no negative descriptors used;
• [Rule 3] The human error (not deploying the allowlist update) has a preceding cause (ticket was not reassigned despite early parental leave);
• [Rule 4] The violation of procedure (failure to deploy the allowlist changes) has a preceding cause;
• [Rule 5] The lack of a pre-existing duty to act (no established process within the Policy team to confirm deployment of new policies) is noted explicitly;
• [Rule 6] The search for the cause identified gaps in hand-offs between people and systems; and
• [Rule 7] Specific descriptors now indicate who, when, how many, how long, and why.

More importantly, the write-up now describes the root causes and contributing factors in sufficient detail so we can easily identify the specific actions needed to prevent a similar incident from occurring again.

Wrapping Up

The 7 Rules demystify the “art” of writing incident reports. Instead of fatalistically believing that “only a good writer can produce decent incident reports,” we now have a rigorous framework for analysis that moves beyond simple blame to an in-depth understanding of system issues. 

While the rules are listed as separate statements, our example clearly shows how applying a handful of rules leads to improvements across all seven.

By applying the 7 Rules of Causation to any incident write-up, we can:

• Better understand what really happened during incidents;
• Identify concrete points for tactical interventions and improvements;
• Share what we’ve learned effectively across teams; and
• Build more robust systems and processes.

The next time you need to write an incident report, use these rules as your guide. They’ll help you move from “What went wrong” to “Here’s how we can make it better.” Ultimately, that’s what effective Trust & Safety work is all about.

Author’s note: Special thanks and everlasting gratitude to mdy, who’s helped me both with this and so much else over the years.

Resources

Downloadable Rules

[ Join this Google Group (free) to gain access. ]
Download the 7 Rules of Causation as a compact, two-page PDF.

Causify It! Custom GPT

Need a little help getting used to the 7 Rules of Causation? Try Causify It, an experimental Custom GPT powered by this blog post. See a sample chat. See also OpenAI’s Privacy Policy.

Related Template

Root Cause Analysis: A Lightweight Template

References

1. Marx, D. (1999). Maintenance error causation. Chapter 2. In Human Factors in Aviation Maintenance – Phase XI, Progress Report. Washington, D.C: FAA, Office of Aviation Medicine. ↩︎

Note: I presented a version of this post as the opening keynote (see keynote deck) at TrustCon’s inaugural conference in 2022. Have feedback on this post? @ me on Bluesky.

Introduction

Back in 2018, I began looking at other fields that dealt with challenges similar to Trust & Safety (T&S) – multidisciplinary, high velocity, high stakes, murky information environments, global in reach – to see if I could draw any inspiration for the issues we were struggling with.

I found myself increasingly intrigued by some of the overlaps I noticed with epidemiology (the study of how diseases and health-related conditions spread, occur, and are controlled within populations), and, in turn, with the field of preventative healthcare and with public health more broadly.

The more I learned, the more similarities I found, and I eventually came to the realization that at its core, Trust & Safety’s purpose is to work in service of health – of the company, the platform, the people who use it, the people whose lives are impacted by it, and (possibly idealistically) the world writ large.

Now, when I say Trust & Safety’s purpose is to work in service of health, I’m not just talking about defining what is bad and addressing the bad things. In healthcare, “health” isn’t just the absence of disease, disorder, illness, or injury. Rather, in its ideal state, health refers to complete physical, mental, and social well-being. Ultimately, we must work in service of that broader meaning of health.

As such, T&S professionals have five important and distinct responsibilities, each of which tracks closely to what is called an “intervention level” in preventative healthcare.

T&S Responsibilities and Healthcare Intervention Levels

The diagram below provides an overview of the five responsibilities that we must fulfill in service of Health.

As T&S professionals, we must:

• Prevent Risk Factors. We identify and target situations and conditions that are likely to create the factors that put people at risk of harm and work to stop the risk factors from ever developing. This work maps to the Primordial intervention level in healthcare. 
• Reduce Risk. When risk factors already exist, we work to reduce or remove them, empower people to control their experiences, and increase protective factors to build people’s resilience in situations where they encounter risk. This work maps to the Primary intervention level in healthcare. 
• Detect Harm and Intervene. We proactively identify harms and intervene to stop them when they occur. This work maps to the Secondary intervention level in healthcare.
• Mitigate Harm. We reduce the impact of experienced harms through direct action and provide remediations to avoid reoccurrences. This work maps to the Tertiary intervention level in healthcare.
• Do It the Right Way. Last but not least, we strive to fulfill these responsibilities the right way. As both a responsibility and a cross-cutting principle, we balance potential risks and harms against the benefits of a given intervention or approach. We make sure we don’t inadvertently do more harm than good. This work maps to the Quaternary intervention level in healthcare. 

Mapping Healthcare Interventions to a Trust & Safety Scenario

Let’s consider a specific scenario – say, losing log-in access to your account – and examine the goals and actions we can take at each intervention level.

A. Primordial

Our first responsibility – preventing risk factors from developing – tracks closely to preventative healthcare’s primordial level. By identifying the underlying conditions that lead to risk factors or that prevent protective factors from developing, we can design and implement system-wide interventions to either inhibit or promote the development of a given factor.

Healthcare Example: In healthcare, increasing neighborhood walkability to encourage physical activity is an example of a primordial intervention. The broader population benefits because physical activity is more easily accessible and, as a result, more likely to become a consistent part of someone’s lifestyle.

T&S Example: In our account access scenario, we want an intervention that will cut down on the number of people who inadvertently lose account log-in access and can’t regain it without human assistance. At this level, we aren’t trying to identify specific groups or individuals in need; rather, we’re looking for broader conditions that could lead to risk factors developing that can be targeted across the system. For example, we might try requiring that people confirm their email addresses as part of the sign-up process to ensure that the email addresses they provide don’t have any inadvertent typos. Another option could be to display a user prompt informing them of the required correction when a known invalid email address is entered – say, an email address that has a comma instead of a period or no @ sign.

B. Primary

Now, realistically, there is no way that we can prevent every possible risk factor from developing. In preventative healthcare, primary prevention maps to our responsibility to reduce risk. By reducing or eliminating existing risk factors, we can lower the likelihood of people experiencing a given harm. If risk reduction is not possible, we instead must work to enhance protective factors and build capacity for and resilience to harm.

Healthcare Example: In preventative healthcare, a vaccine that lowers your likelihood of getting a specific disease is an example of risk reduction, while a law mandating the use of seatbelts in cars is an example of an intervention that enhances protective factors.

T&S Example: In our account access scenario, we want to reduce the number of people who inadvertently lose log-in access to their accounts and can’t regain it without human assistance. However, we’re no longer trying to address underlying conditions; rather, we’re attempting to either reduce risk or enhance protective factors. Let’s say, for example, that we’ve seen a significant increase in account compromises lately. We might consider requiring people to confirm they’re still in control of their account via an in-app or email prompt whenever we see an account login attempt from a new device. This intervention provides an additional layer of protection against account compromise.

C. Secondary

Much as we can’t prevent every possible risk factor from developing, there’s also no feasible way to eliminate all existing risk factors or prevent all harm. As such, we also need to detect harm and intervene – actions that fall within the secondary-level interventions in preventative healthcare. We want to take action at the earliest possible point where doing so can be effective, ideally before the full impact of the harm is experienced. 

Healthcare Example: In preventative healthcare, these interventions involve screening for and identifying unrecognized diseases in healthy-appearing people and treating them before the onset of symptoms or at the earliest possible point where detection and intervention can be effective, as well as reversing the communicability of infectious disease. These interventions are not implemented for the whole population; rather, the focus is on specific individuals or groups who are likely at risk. Regular mammography screenings to detect early-stage breast cancer are an example of a secondary-level intervention. 

T&S Example: In our account access scenario, the regular monitoring of credential dumps resulting from various data breaches and cross-referencing those credentials with our records to identify accounts that have direct password exposure will allow us to implement forced password resets before an account is compromised. This intervention identifies and halts the progression of harm for at-risk accounts before the full impact of the harm is experienced.

D. Tertiary

With the previous level, we focused on halting the progression of harm. To effectively mitigate harm – known as tertiary interventions in healthcare – we must not only reduce the damage caused by the harm, with the goal of restoring the account to a healthy state, but also devise ways to prevent the harm from reoccurring or causing further damage. 

Healthcare Example: In healthcare, tertiary interventions take place once a disease has entered the clinical stage and symptoms are present. These interventions focus on reducing the damage caused by symptomatic disease, including mitigating current pain or damage, preventing additional pain or damage, and restoring the health and functions of individuals affected by disease. Examples of tertiary interventions in healthcare include stroke rehabilitation programs and outpatient support programs. 

T&S Example: In our Account Access scenario, a tertiary intervention could be to have a guided account restoration process in place so that people can easily regain access to their accounts. Similarly, having multi-factor authentication (MFA) reset tools so that people can reset their MFA settings after losing access to their authentication device (e.g., a lost phone or a broken hardware token) would also be a tertiary intervention.

E. Quaternary

Finally, as both a responsibility and a cross-cutting principle, we want to ensure that any actions we take and interventions we deploy are done the right way – preventative healthcare’s quaternary level. We need to weigh benefits against risks and potential ramifications before an action is taken. If the risks and potential ramifications outweigh the benefits, we need to figure out a different approach.

Healthcare Example: In healthcare, these interventions include actions taken to identify and protect individuals from medical interventions that are more likely to cause harm than good. These steps include identifying patients at risk of overtesting or overmedicalization, protecting them from excessive medical invasion, preventing the use of therapies that have not been adequately assessed, and suggesting interventions that are ethically acceptable.

Ultimately, such initiatives are informed by the principle of nonmaleficence – the obligation to not inflict harm on others – which is closely tied to the maxim of “first do no harm.” An example of a quaternary intervention in healthcare is avoiding excessive antibiotic use because overuse could lead to the development of drug-resistant diseases.

T&S Example: How might this play out for our account access scenario? Let’s say we’re planning to force password resets for all accounts that have had their passwords exposed through data breaches elsewhere. As we’re evaluating potential risks and ramifications, we realize that a significant number of affected accounts don’t have confirmed email addresses. If there is no way for people to regain access to their account without access to their email, we could consider either implementing a special account access flow for those accounts or limiting our forced password reset to those that have confirmed email addresses, then falling back on in-app notifications for other accounts to let them know that they need to change their password.

By providing effective and reliable mechanisms for people to regain access to their accounts if and when they need help – particularly if they need that help because of an intervention we’ve initiated – we give ourselves more freedom to implement other types of interventions because there is still a path back to health for those affected.

When we keep the five intervention levels in mind as we evaluate how to tackle a given T&S challenge, we can identify optimal intervention points and, when possible, shift our focus to earlier levels to maximize their impact.

Using Intervention Objectives to Identify Gaps in T&S Work

In the preceding section, I’ve focused primarily on Trust & Safety interventions that aim to either preserve health (i.e., health protection) or prevent, slow, or repair harm (i.e., harm reduction). 

To use a video game metaphor, I’ve described actions that are designed to prevent our character from taking damage (shield), slow the rate of damage experienced (armor), or heal them (potion).

For Trust & Safety work to be successful and sustainable, however, our efforts must also include interventions that promote health and build resiliency – or, to go back to the video game analogy, interventions that will make our character’s life meter longer and give us more health capacity (i.e., health promotion).

Why does this matter? 

Health promotion interventions empower people to engage in responsible behaviors, which enhances harm prevention and strengthens protection measures. Because health promotion encourages individuals to adopt healthy behaviors before issues arise, these types of interventions increase resilience, further strengthening harm prevention and health protection interventions. 

By developing interventions that span all three areas, we can break the cycle of being trapped in the endless whack-a-mole of reacting to harm after it has already occurred and instead begin to shift towards a more holistic pursuit of health.

Consider the Account Access examples we covered in the previous section. The table below maps the aforementioned interventions (denoted with a green checkbox) to the three intervention objectives and five intervention levels. A quick glance at the table below shows that our previously identified interventions all fall squarely under the health protection and harm prevention objectives only. It’s clear that we have gaps in health promotion and have an opportunity to expand our work on the first two objectives.

What might a comprehensive set of interventions look like? Let’s fill out the rest of the table and consider how other interventions might work in service of preserving Account Access for our rightful users. 

Conclusion

As Trust & Safety professionals, our focus cannot solely be on mitigating harm. Instead, our interventions must be in service of all three objectives:

• Health protection  –  eliminate risk factors and reduce the number of potential risks people encounter.
• Harm prevention  –  maintain current health capacity, mitigate the impact of harm, and repair damage.
• Health promotion  –  empower people to take control of their own experiences, build resilience, and increase their capacity to handle encountered risks.

When our work spans the three intervention objectives and is across all five intervention levels, we create a virtuous cycle that allows us to identify optimal intervention points, discover gaps in our T&S work, and better support the development of a balanced and sustainable path in our pursuit of health.

Author’s note: Special thanks and everlasting gratitude to mdy, who’s helped me both with this and so much else over the years.

References

Association of Faculties of Medicine of Canada. “Introduction to Epidemiology.” Public Health Primer. Accessed August 9, 2024. https://phprimer.afmc.ca/en/part-i/chapter-4/.

Mendes, René, and Elizabeth Costa Dias. “Health Protection, Health Promotion, and Disease Prevention at the Workplace.” In Global Occupational Health, edited by Tee L. Guidotti. Oxford University Press, 2011. https://doi.org/10.1093/acprof:oso/9780195380002.003.0018. Accessed August 9, 2024.