AI 'Ragebait': How synthetic videos fuel anti-migrant rhetoric on Facebook

A recent video on Facebook shows a couple outside a British pub. The woman complains that there were no halal options and they were offered pork.

 She argues that tourist pubs should have halal menus in order to be respectful of Muslim patrons, News.Az reports, citing North West by Lines.

The poster invites people to comment with their thoughts, under the guise of debate. The comments quickly devolve into anti-migrant rhetoric. ‘Go back to your own country’ says one user, and ‘if you cannot integrate with our culture you should not be here’’ says another. However, this is not a real couple outside a real pub. They are AI-generated, and designed to create engagement amongst those with anti-migrant views.

AI-generated anti-migrant videos 

Across Facebook, many accounts publish similar videos and have spread across the platform. The jumbled text, glitches in footage and an ultra ‘smooth’ quality to the footage, should make it clear that this content is inauthentic. Some users even disclose the use of  AI, claiming it has been used to recreate ‘real-life’ scenarios. 

Dr Daniël De Zeeuw, University of Amsterdam, has researched the right-wing social media eco-system, tracking its shift from more fringe sites like 4Chan to mainstream social media networks. He said that the right has always been ”super good at creating memes to create a lot of engagement online” by using “trolling to influence the public conversation on topics”. AI videos spreading across Facebook is just another example of this ‘trolling’ entering mainstream social media spaces.

When interviewed, Dr de Zeeuw explained how AI-generated videos promoting anti-migrant rhetoric fit into a ‘post-truth conspiracism’ framework. ‘Post-truth’ refers to a wider movement where arguments are made based on emotions and beliefs, rather than objective facts or ‘truths’. However, modern-day conspiracy movements presenting themselves as ‘truth-seekers’ and ‘heroes of truth-finding’, Dr de Zeeuw told us, create the framework he calls ‘post-truth conspiracism’. 

These AI videos incite anger against immigrants

AI videos with anti-migrant rhetoric rely on inciting emotions of anger in order to create engagement, instead of providing objective statistics on immigration. Dr De Zeeuw explained how “AI has a flimsy relationship to truth”, with it producing false and biased information to fit a prompt and elicit expected behaviour. 

AI content can reinforce conspiratorial thinking because it is not bound to ‘restraints’ and can reinforce ‘fantasy’ without ‘any natural barrier’ of the real world. 

Some people in the comment section of this content seem unaware that these videos are not real. Yet, Dr de Zeeuw explained why it is that whether or not people believe the content is authentic is not always the ‘relevant’ question to ask. 

He said:

It’s not often that users genuinely believe in it. The content resonates with them and reflects their political point of view.

Sharing the videos to show where you stand

Engaging with this content is, for some, a performative gesture. By sharing this content, the ‘sharer’ is saying ‘I belong to this community’ and “signalling to people that you’re on their side”, Dr de Zeeuw added. 

To explain this, he compared AI videos to the concept of ‘Mar a Lago face’, which refers to the ultra-smooth, bold features (often achieved by cosmetic surgery) of many women in Trump’s circle. “Artificiality is the point”, he said. There is an expression of power in being able to ‘manipulate reality’ and ‘normalise’ certain ‘aesthetic norms’. The surreal ‘look’ of AI is a political signal in itself.

Not all Facebook users see AI videos for what they are

However, there is a concern that Facebook’s main age demographics are not as digitally literate as their younger counterparts. This means they are less able to detect AI-generated content. In 2024, Meta shared that they would be including an ‘AI info’ disclaimer on posts made with AI. However, this relies on Facebook detecting the content or users self-declaring that their post was made with AI. With the speed that this content is created and shared, it is difficult for all posts to be labelled accordingly.

Dr de Zeeuw said:

In an ideal world you would have systems in place for content moderation.

However, when it comes to social media companies, Dr de Zeeuw explained that they “don’t seem to have any intrinsic value systems”. Instead, they behave opportunistically only to make a profit. 

Right-wing AI content keeps people engaged so it is not removed from apps

Right-wing social media accounts use tactics of post-truth to create engagement, as viewers become very emotionally involved in the content. Therefore, it is within the social media companies’ best interests to let this content spread across these platforms, as it keeps people on their apps for longer. Recently, a US jury ruled in favour of a woman who sued Meta, claiming its platforms are intentionally addictive. So, awareness of these engagement tactics is growing and accountability is being demanded. 

AI-generated content is impossible to escape on social media, yet, even in its most harmless forms, it bends the boundaries of reality. How do we bring people back to reality when truth matters less than emotion?

News.Az 

By Leyla Şirinova