- cross-posted to:
- technology@beehaw.org
- cross-posted to:
- technology@beehaw.org
Jackson soon discovered that Amazon suspended his account because a Black delivery driver who’d come to his house the previous day had reported hearing racist remarks from his video doorbell. In a brief email sent to Jackson at 3 a.m., the company explained how it unilaterally placed all of his linked devices and services on hold as it commenced an internal investigation.
The accusations baffled Jackson. He and his family are Black. When he reviewed the doorbell’s footage, he saw that nobody was home at the time of the delivery. At a loss for what could have prompted the accusation of racism, he suspected the driver had misinterpreted the doorbell’s automated response: “Excuse me, can I help you?”
In this case it’s the “suspect” of racism, in which I think we all agree should not directly lock you out of your account, but perhaps give you a warning.
But what if it’s suspected illegal actions or content? Like them catching the home server being part of a DDoS attack, or overhearing signs of child abuse / identifying possible child pornography content or noticing illicit gun ownership. Their AI will determine that there’s a 97% chance of that being the case.
I wonder if that would change things.
If such a system is not allowed to block usage, it will probably at least inform the local police.Your home setup will in most cases eventually act as a panopticon.