In the year 2023, the widespread growth of generative Artificial Intelligence (AI) tools globally became a notable phenomenon. This led to the swift evolution and reshaping of the content creation landscape, driven by the widespread accessibility of numerous generative AI tools on the internet. Today these advancements have empowered the rapid generation of animated, visual, and textual content. Generative AI tools and applications are rapidly finding extensive use across diverse sectors, encompassing healthcare, marketing, gaming, education, and communication.
Nevertheless, the myriad advantages of generative AI have come hand in hand with a plethora of challenges, leaving numerous countries to grapple with the complexities that accompany its adoption. Among them, is the concern raised about generative AI makes combatting online gender-based violence (OGBV) more difficult and complex (1). As an illustration, the widespread use of open-source generative AI technology for producing homemade, non-consensual pornography and the generation of manipulated images, commonly referred to as “deep fakes,”(2) has become a prevalent issue. This involves the incorporation of the faces of women of various statuses to produce nude and sexually explicit images, highlighting a disturbing trend in contemporary society.
The initial deep fakes were easily identifiable due to glitches and often unrealistic appearance. However, the evolution of this technology has reached a point where individuals with a modest understanding of computers and AI, coupled with access to a decent computer, can effortlessly create, distribute, and even sell convincing deep fakes.
In November 2023, popular Indian actress Rashmika Mandanna became the victim of an AI-generated video. In highly conservative India, the video created shockwaves as many viewers believed the woman in the video garbed in a revealing outfit was indeed the actress herself. In fact, the video was watched more than 2.5 million times on the social media platform X alone. The actress called the incident ‘a great misuse of technology’ and the experience ‘extremely scary’(3). However, the creator remains elusive to date and the incident led to calls for greater scrutiny of the legal and regulatory frameworks used to control the use of AI and fake generated images.
In Sri Lanka, while indistinguishable deep fakes such as in the Rashmika Mandanna incident continue to be a rarity, in 2023, Hashtag Generation(4) – a movement led and run by a group of tech-savvy and socially conscious youth in the country – was truly troubled to observe the emerging trend of using AI to generate deep fakes in Sri Lanka as well as its potential to perpetrate gender-based abuse.
While the deep fakes observed in many of these instances were easy to spot, the rapid development of AI and the rising threat cannot be negated. Hashtag Generation’s monitoring efforts conducted as part of the project “Get The Trolls Out! Sri Lanka” confirmed that the repercussions of advanced technology were felt by various individuals, encompassing local actresses, singers, YouTube content creators, and ordinary women.
The Sensity AI report titled “The State of Deepfakes 2019: Landscape, Threats, and Impact”(5) revealed that a staggering 96% of deepfakes are employed for the creation of non-consensual sexual content. Additionally, within this category, a striking 99% consisted of manipulated images of women.
The story in Sri Lanka appears to be the same. The creators behind these deep fakes observed by Hashtag Generation showed no discrimination regarding the ages of the victims, yet notably, their targets were exclusively female. The women featured in the deep fakes were also subjected to verbal harassment and unsavory comments by social media users, and commentators included the creators masking their identities behind fake user profiles.
The creation of a deep fake is clearly a violation of one’s sexual autonomy and posting deep fakes on the internet under the guise of authentic sexual content amounts to the non-consensual distribution of intimate images.
Nevertheless, nations worldwide are contending with the challenges posed by this emerging phenomenon. The legal frameworks in many countries currently lack the necessary tools to effectively address cases of OGBV perpetrated through generative AI tools and applications. For example, Canada’s criminal law does not presently incorporate a provision holding content creators liable for generating deep fakes. Consequently, pressing charges in cases of non-consensual release of sexual images requires the content to feature one’s actual naked body for legal recourse. Presently, certain pornography websites have implemented content moderation rules dictating who can post and what can be posted. However, despite the existence of explicit content moderation guidelines on platforms like Meta, Instagram, and TikTok, Hashtag Generation’s monitoring efforts show that these rules are not consistently enforced. Therefore the question remains as to how Sri Lanka will face the challenge of deep fakes in the foreseeable future.
Referring to concerns related to deep fakes and comparable content creation, among other reasons, the Sri Lankan Government introduced the highly debated Online Safety Act to the Parliament on January 23, 2024. Emphasizing the widespread occurrence of online abuse, the Minister of Public Security Tiran Alles highlighted data indicating that in 2023, approximately 3,000 out of 8,000 filed complaints for child safety concerns were traced back to OGBV perpetrated through social media platforms. This significant statistic supposedly served as a pivotal motivator for the government’s forthcoming Online Safety Bill, he said.(6)
Nevertheless, the newly proposed legislation is not without its challenges and has faced substantial criticism for its draconian and arbitrary provisions, raising concerns about potential infringements on personal liberties. As for non-consensual image sharing, the new legislation has merely one provision referring to the sharing of such images on social media which may bring into question the efficacy of the legislation against the threat of generative AI being used to commit OGBV.
Critics also argue that the structure of the commission established by the law, along with its extensive powers, could potentially stifle freedom of expression, particularly impacting those utilizing generative AI for creative purposes.
Given the complexity of generative AI and the issues surrounding it, it remains uncertain whether the government’s measures will adequately tackle the threat of online gender-based violence (OGBV) perpetrated through such means and ensure the safety of women and children in the country.
- https://ethicalsource.dev/blog/ai-and-gender-based-violence/
- https://www.techtarget.com/whatis/definition/deepfake
- https://timesofindia.indiatimes.com/city/mumbai/hurt-by-deepfake-video-rashmika-big-b-seeks-legal-action-on-misuse/articleshow/105025495.cms
- Hashtag Generation
- https://regmedia.co.uk/2019/10/08/deepfake_report.pdf
- https://english.newsfirst.lk/2024/1/4/social-media-abuse-spurs-online-safety-bill-in-sri-lanka-minister-promises-child-protection
_Article By : Maneshka Borham _
Maneshka Borham, is a print media journalist based in Colombo. She currently works as a reporter for the Daily Financial Times and focuses mainly on subjects including human rights, defence, leftist politics, development, police and crime.
