Facebook grapples with removing live-streamed content in real-time, while also preventing or reactively removing re-uploads of violating content.
By the end of the shooting, the shooter had killed 51 people and injured 49. Only the first shooting was live-streamed, but Facebook was unable to end the stream before it had been viewed by a few hundred users and shared by a few thousand more.
The stream was removed by Facebook almost an hour after it appeared, thanks to user reports. The moderation team began working immediately to find and delete re-uploads by other users. Violent content is generally a clear violation of Facebook's terms of service, but context does matter. Not every video of violent content merits removal, but Facebook felt this one did.
The delay in response was partly due to limitations in Facebook's automated moderation efforts. As Facebook admitted roughly a month after the shooting, the shooter's use of a head-mounted camera made it much more difficult for its AI to make a judgment call on the content of the footage.
Facebook's efforts to keep this footage off the platform continue to this day. The footage has migrated to other platforms and file-sharing sites -- an inevitability in the digital age. Even with moderators knowing exactly what they're looking for, platform users are still finding ways to post the shooter's video to Facebook. Some of this is due to the sheer number of uploads moderators are dealing with. The Verge reported the video was re-uploaded 1.5 million times in the 48 hours following the shooting, with 1.2 million of those automatically blocked by moderation AI.
Decisions to be made by Facebook:
Questions and policy implications to consider:
Resolution: Facebook reacted quickly to user reports and terminated the livestream and the user's account. It then began the neverending work of taking down uploads of the recording by other users. It also changed its rules governing livestreams in hopes of deterring future incidents. The new guidelines provide for temporary and permanent bans of users who livestream content that violates Facebook's terms of service, as well as prevent these accounts from buying ads. The company also continues to invest in improving its automated moderation efforts in hopes of preventing streams like this from appearing on users' timelines.
We just sent you an email. Please click the link in the email to confirm your subscription!