MENLO PARK, Calif. — Meta told its own oversight board that it relies on “media reports” to add images to a permanent database of banned content that parent company Instagram and Facebook have compiled.
The disclosure took place in the context of a rack issued this week by the company’s appointed, reportedly independent Oversight Board, criticizing Meta for its inconsistent handling of “explicit AI imagery,” commonly known as deepfakes.
In response to questions about two specific cases of deepfakes involving an Indian and an American celebrity respectively, Meta acknowledged the practice of adding explicit images to a Media Matching Service (MMS) bank.
These MMS banks “automatically find and delete images that human reviewers have already determined violate Meta’s rules,” the Council explained.
When the Council noted that the image resembling an Indian public figure “was not added by Meta to any MMS bank until the Council asked why,” Meta responded by saying that “it relied on media reports to identify the image which seemed to add the American public figure to the bench, but in the first case there were no such media signals.”
According to the Council, this is “concerning because many victims of deepfake intimate images are not in the public eye and are forced to accept the distribution of their non-consensual images or to seek and report any case. One of the existing signals of lack of consent under adult sexual exploitation policies are media reports of intimate images being leaked without mutual consent. This can be useful when posts concern public figures, but is not useful for private individuals. Therefore, Meta should not be too dependent on this signal.”
The Council also suggested that “context indicating that the nude or sexualized aspects of the content are AI-generated, photoshopped, or otherwise manipulated should be considered a signal of non-consent.”
Meta has been repeatedly challenged by sex workers, adult performers, and many others to shed light on widespread shadow banning policies and practices, but access to the details of those processes has been scarce. Meta’s response to its own Oversight Board is a rare example of lifting the veil of secrecy over its often arbitrary and confusing moderation practices.
As XBIZ reported, the Oversight Board has already criticized Meta for its policies regarding content it deems sexual, though its recommendations do not appear to have had a meaningful impact on its still-opaque moderation practices.
The Oversight Board made non-binding recommendations that Meta should add the ban on “derogatory sexualized photoshop” to the Adult Sexual Exploitation Community Standard; change the word ‘derogatory’ in the ban on ‘derogatory sexualized photoshop’ to ‘non-consensual’; replace the word ‘photoshop’ in the ban on ‘derogatory sexualized photoshop’ with a more general term for manipulated media; and generally “harmonize its policy on non-consensual content by adding a new signal for lack of consent in the adult sexual exploitation policy” for “the context of content being generated or manipulated by AI.”
For content with this specific context, the Council recommended that the new policy should also specify that it does not have to be “non-commercial or produced in a private setting” to violate Meta’s Terms of Service.