This is personal. I think my first experience of online bullying was probably 15 years ago when someone actually created a hate site dedicated to me during high school! But today the problem’s grown out of control: the surface area over which you can be attacked is just huge and the incivility is overwhelming.
I interned at Facebook in 2008, I was at Pinterest when it had just 10 employees and I was the second engineer hired to work on Quora. My time at these places taught me that you can’t just pull out content moderation from the platform as a separate problem. It’s built in to how these communities work: poorly designed mechanisms open up the potential for abuse.
I also saw what the lack of diversity and inclusion in Silicon Valley meant. It’s a place that has a habit of solving the problems of its own demographic. It seems like it’s trying to figure out: “What did mom do for me that I don’t want to do myself? Cook for me, do my laundry, drive me around.” It’s problematic to phrase it that way, but you get the point. If there aren’t folks who have had to deal with the bad stuff, that means solutions to these problems aren’t prioritised.
So, ultimately, I wanted to build the Block Party app to help everyone who deals with harassment and bullying on the internet. We didn’t need sophisticated AI to build something helpful. The way Block Party currently filters out harassment or bullying is via simple heuristics [rules].
For example, there’s an ‘I need a break mode’ where you won’t see mentions from people unless you follow them or they’re followed by the people you follow, they’re a verified user or you’ve interacted with them recently.
We collect everything else and put it in a folder which you can review later. This way, you don’t have to worry about missing out on anything but you do get to choose how and when you see things.
This approach means we don’t have to be perfect in classifying what’s okay or not, with some kind of machine-learning algorithm that reaches whatever accuracy. And how do you even define ‘correct’: what’s bullying and what’s not? Each person has their own threshold, and it changes depending on how you’re feeling. There are times when I’m ready to hear from the world and others where I’m just so exhausted with dealing with these people.
This strategy also means we can add other features. For instance, you can give others access to that folder, like friends or even the authorities, to look into the harassment. This is preferable to giving them my full Twitter credentials so that they see everything.
A lot of the current design of social platforms puts the onus on the victim to deal with the harassment. I did this Reddit AMA [Ask Me Anything] last summer and it ended up with 4,000 trolls brigading me. Reddit’s response was that they don’t condone harassment, and that I should report it. But I’m not going to sit there and report thousands of comments and accounts. It’s frustrating and potentially traumatising to put that burden on the person who has to deal with the abuse. A sharing system like ours, with delegated access, means you can have others help you.
We have a watchlist feature as well that lets you keep an eye on people. I once had someone who replied every time I tweeted and I didn’t want to see it, so I muted him. But then at some point he escalated to physically stalking me and I didn’t see it because of the muting. From that point on I had to regularly check his account to make sure I was going to be physically safe.
Ultimately, our goal is to get to a point where we won’t just build on top of existing software for platforms like Twitter or Instagram, but provide tools for new platforms that will help them avoid the same pitfalls. Think of it like payment processing in apps – a start-up doesn’t need to build that software, they just plug in a monetisation tool from someone else. Block Party could be that tool, and hopefully solve this problem for everyone.
Interviewed by Dan Bennett
- This article first appeared inissue 361ofBBC Science Focus Magazine–find out how to subscribe here
More on how to make the internet great again:
- How a “new kind of physics” could track down extremists online
- Interview: Web inventor Tim Berners-Lee thinks his creation is out of control. Here's his plan to save it
- Why the internet can survive without ads
- Social networks are built to turn us against each other. Can we fix them?
- Why scientists don’t actually know if social media is bad for you
- The simple reasons online disinformation may never be fixed
- Breaking up the power: Does the internet’s future have to depend on a handful of megacorps?