Swipe with Caution: Unveiling the TikTok Algorithm and its Gateway to Problematic Content through Engagement with Martial Arts

Introduction

On August 9th, 2022, I undertook a self-experiment to investigate the underlying mechanisms of TikTok’s algorithmic curation. The specific approach involved immersing myself in a focused stream of martial arts content, aiming to comprehend the platform’s content recommendation system. My curiosity stemmed from a Media Matters study on TikTok, who did a similar approach for transphobic and right-wing-extremist content. Methodologically, the self-experiment entailed a comprehensive day-long observation (or rather consumption) of TikTok, during which I exclusively engaged with martial arts content and followed five accounts associated with Mixed Martial Arts (MMA).

Engagement means, I diligently liked every martial arts-related video for the initial ten-minute period and subsequently relied solely on the curated content presented on the For You page. Upon completion, I analyzed and hand-coded approximately 900 videos from my own user data downloaded through TikTok. This study presents a starting point for the exploration of several interconnected aspects within the context of TikTok and radicalization. Firstly, it offers an another avenue to investigate the potential provision of extremist material through the platform. Secondly, it opens up the question on how “algorithmic” triggers might play a role in the dissemination of such content. Lastly, this research prompts an investigation into the potential effects of consuming such material on TikTok’s audience.

Topics

In the table below, you will find the topics that I have encoded, capturing various themes present in the videos. Upon examining the frequencies, it becomes apparent that the total number of topics represented exceeds the number of videos. This arises from the fact that individual videos may have been associated with multiple topics. We will further investigate these relevant connections later. By just looking at this table, it is evident that “martial arts” emerges as the most prevalent topic. However, the top 10 topics also encompass videos related to “military” displays and operations, the glorification (but not explicit portrayal) of violence, expressions of eroticism, and depictions of crime and illegal activities. NA refers to the videos that I was not able to retrieve. A total of 37, or around 4% of all videos in between the time I have watched them to when I have coded them (20 days). The status of these videos remains uncertain, as it is unknown whether TikTok deleted them, if the account seized to exist or removed them itself, or if they were set to private. Hence, it would be interesting to investigate if they are currently available online again.

The Timeline

Displayed below is a figure illustrating the occurrence of all 80 topics coded in chronological order, reflecting the progression from the first to the last videos. Further elaboration on these videos will follow, but we can observe how topic distributions evolved during my engagement with the For You Page and how the recommendation system adapted accordingly. In the initial third of videos, a diverse array of topics emerged, introduced by consecutive displays of similar content. During the subsequent third, while engaging solely with martial arts-related videos, the recommendations predominantly featured martial arts content. Notably, as TikTok discerned my primary interest, the majority of videos in the final third were martial arts-related (evident from the closely aligned dots forming a line). Adjacent to these topics, other themes emerged, such as martial arts and relationships or martial arts and comedy. Additionally, newer topics surfaced that were closely connected to martial arts, like Chechen MMA fighters and then again the Russian-Chechen war through that adjacency. As this study was conducted within a single day, it highlights how TikTok rapidly captured my interests, leading to a noticeable shift from generally popular topics that I could have interacted with to new topics closely aligned with or adjacent to the ones I already demonstrated clear interest in.

Religion

In the context of religious extremism or associated content, two topics were indicative. The first one involves “Anasheed,” which refers to Islamic vocal chants or songs often associated with spiritual themes. In some instances, Anasheed featured (partially) extremist rhetoric (code: nasheed_ext). The second one involves actors, warranting further investigation, as they displayed radical or extremist ideologies and activities (code rad_bezug). When addressing the issue of problematic Anasheed, it is essential to remain cognizant that many individuals share them due to their Islamic nature, pleasant melodies, or just being generally popular and widely shared. A closer investigation of the accounts posting these Anasheed revealed that the majority did not exhibit any form of radicalism.

Three videos, however, warranted further investigation as they displayed actors promoting extremist thought. One of these videos featured an MMA fighter from Germany who used one of those Anasheed in his content. Upon examining his account, it was evident that he glorified religious scholars with radical views and maintained links to political groups in Syria (#607). Another video praised the Chechen mujahideen during the Russian-Chechen war (#626). The third video belonged to an account centered around Turkish nationalism and exhibited revisionist views regarding the reinstatement of the Turkish caliphate (#757). This actor appeared to be more systematic in their beliefs, whereas the second actor displayed a degree of randomness in their content, making it uncertain whether any politically relevant conclusions or meaning could be inferred upon the actor posting it.

Right-Wing Extremism

For right-wing extremist material, a larger number of codes were indicative, as shown by the presence of blue triangles. Consequently, a larger proportion of videos fell into this category. The content also encompassed recurrent displays of misogyny and (offensive and hostile) anti-feminism, resembling quite radical subsections of the supposed manosphere or red pill community. Notably, right-wing extremist videos appeared quite early in the dataset, including displays of misogyny (#173), right-wing anti-Ukrainian propaganda (#192), and praise of the Wehrmacht (#259).

It is worth noting that a diverse range of nationalistic sentiments was evident, like German, Greek, and Turkish nationalisms. Right-wing material tended to co-occur with the praise or glorification of violence. This observation extended to Turkish nationalism, which, in part, manifested through the Grey Wolves — a group associated with both organized crime and far-right Turkish nationalism.

Crime

Engaging with martial arts content on TikTok exposed me to a lot of materials that glorify violence and delinquenc. After all, martial arts play an important role in certain forms of (organized) crime, as a display of strength and masculinity. For example, I observed instances where content such as rappers rapping about committing violence or displaying weapons, as well as videos by criminal/outlaw motorcycle groups (referred to as 1%ers). These were triggered as a result of me interacting with martial arts content, however, martial art contents did not frequently co-occur with videos relating to crime.

Eroticism & Pornography

I observed that gateways to pornographic material and displays of eroticism were quite prevalent. TikTok is making efforts to improve flagging of mature content, hence the extent of this issue in the future remains uncertain. It was concerning to notice that some of these materials were consumed by a fairly young audience, evident from the comments. One video, in particular, was particularly disturbing as it showcased ways to find porn actresses on platforms like PornHub.com, cleverly coded in TikTok spaces to evade detection (like cornhub, the hub, or just the colors of the website). The video focused on actresses who appeared especially young or underage while still being at the age of consent (#266). I referred to this as “jail baiting”, although the term usually pertains to someone who appears sexually mature but is actually below the age of consent.