Facebook now identifies potential suicides for authorities to take action

Please consider donating to Behind the Black, by giving either a one-time contribution or a regular subscription, as outlined in the tip jar to the right. Your support will allow me to continue covering science and culture as I have for the past twenty years, independent and free from any outside influence.

Reason #3,434,389 why I don’t use Facebook: Facebook has developed software that identifies what it thinks are suicidal thoughts by a user, then sends that information to the government so it can take immediate action.

The social network has been testing the tool for months in the US, but is now rolling out the program to other countries. The tool won’t be active in any European Union nations, where data protection laws prevent companies from profiling users in this way.

In a Facebook post, company CEO Mark Zuckerberg said he hoped the tool would remind people that AI is “helping save peoples’ lives today.” He added that in the last month alone, the software had helped Facebook flag cases to first responders more than 100 times. “If we can use AI to help people be there for their family and friends, that’s an important and positive step forward,” wrote Zuckerberg. “The AI looks for comments like “are you ok?” and “can I help?””

Despite this emphasis on the power of AI, Facebook isn’t providing many details on how the tool actually judges who is in danger.

The potential for abuse here is beyond words. Worse, Facebook’s unwillingness to be transparent about this software makes it even more suspect. From the article:

TechCrunch writer Josh Constine noted that he’d asked Facebook how the company would prevent the misuse of this AI system and was given no response.

As I’ve written previously, companies like Google, Facebook, and Microsoft might be providing their customers some good products, but they are also doing so from a very amoral position, abusing the privacy of their customers in ways that are simply wrong. While this software is likely being used today in a totally correct way, I have strong doubts about it in the long term. As the politics of our time become even more heated, partisan, and childish, the temptation to use this software to target and eliminate those who disagree with either Facebook or its allies in the government will certainly grow. And then, how does one protect oneself from this abuse?

Hat tip to reader Max Hunt.



  • wayne

    yes, bad idea, all around.

  • Edward

    From the article: “He added that in the last month alone, the software had helped Facebook flag cases to first responders more than 100 times.

    The article fails to mention how many of those 100 times were not false alarms.

    What happens when the first responders themselves ask questions such as “are you OK?” and “can I help?” to the falsely accused — er — whatever they are called. (Did I just initiate a false alarm by using both of those phrases? Oops. My bad.)

    Doesn’t just knowing that a machine-intelligence is second guessing our attitudes make us depressed enough to drive us suicidal? Certainly having the first responders sicced on you unnecessarily makes you angry, if not suicidal.

  • Cotour

    People ask me: “Are you on Facebook?”.

    And I answer: “No, I call Facebook Lucifers Living room”.

    I can see the value in Facebook for promoting a business but what I see mostly in people who use it for personal reasons is a lot of wasted time digging into the past and comparing ones self to others and enjoying a kind of a Schadenfreude and it develops a negative kind of competition. I see it as a kind of dead end in some ways, and I know many disagree with me as the massive numbers of participants demonstrate.

    It can have its positive communication and organizational aspects but certainly a double edged sword.

  • Mark

    Liability protection.

  • Lee S

    @Cotour, I consider Facebook and other forms of social media very useful tools, used properly they are a life improving asset. As an ex-pat they are invaluable in keeping me and my kids in touch with friends and family back in the UK.
    However I’m fully aware that everything I decide to share on these platforms is “out there” for perpetuity, and I’m fully aware that I am the product in the “contract” between Facebook and myself. I can live with the focused advertising I receive in my feed, and they can mine my data to their hearts content… they will learn nothing that they couldn’t learn with a phone call or a chat over a pint.
    This whole “suicide watch” idea is admittedly creepy, but this is the world we live in, Facebook has to be seen to be doing something, but if it goes too far people will move to a different platform. ( indeed, observing my kids I believe Facebook is already loosing the younger generation, it’s fine to chat with grandma, but they don’t use it to stay in contact with their pals!)
    Like any tool, used properly social media is useful, but misused without an understanding of the consequences it can be dangerous.
    Like so many problems, the answer is education.

  • pzatchok

    Today they are announcing a suicide watch.

    Tomorrow its a watch for terrorists.

    Then drug dealers.

    Then gang members.

    Then those nasty politically incorrect statements.

    Oh, these social median companies are not restricting your freedoms and rights. But they are reporting everything to the appropriate authorities.

    Next thing you know your pulled over for a legitimate speeding ticket and the cop asks if you have a gun with you. (Face book reported you posted about having one.) The cop is just asking for his own safety you know.
    By the way he now knows your a member of the NRA, have posted a few anti Islam treats, are a republican, have a hunting license, are divorced and dating, who you are dating, several of your sexual kinks…

    All because he clicked that little ‘voluntary information’ link provided by Social Watch (a new tech company that sources open information from multiple social media sights for law enforcement).

    The whole stop is a little creepy but all you get is a ticket. And an odd snicker from the cop.

    Now what if some mistake or a hacker happens to send law enforcement on a drug/firearm raid of your home and family? Don’t move funny, jump or move slow/ to fast or they might just shoot you accidentally.

    Or your employer asks Social Watch if you have any ‘inappropriate’ posts.

    Add in the general list of friends and relatives you have and they now your getting grief for who they are.
    What gay magazine is going to hire a gay editor who has republican friends and a loving supporting family?

  • ken anthony

    Why get a warrant when they already have your info? It starts with the fiction of consent of the governed which is not just fiction but impossible. Where it ends is where ever the connected care to take it. Don’t worry about tyranny… that ship sailed a long time ago.

Leave a Reply

Your email address will not be published. Required fields are marked *