Beyond Detection: Visual Realism Assessment of Deepfakes

In the era of rapid digitalization and artificial intelligence advancements, the development of DeepFake technology has posed significant security and privacy concerns. This paper presents an effective measure to assess the visual realism of DeepFake videos. We utilize an ensemble of two Convolutional Neural Network (CNN) models: Eva and ConvNext. These models have been trained on the DeepFake Game Competition (DFGC) 2022 dataset and aim to predict Mean Opinion Scores (MOS) from DeepFake videos based on features extracted from sequences of frames. Our method secured the third place in the recent DFGC on Visual Realism Assessment held in conjunction with the 2023 International Joint Conference on Biometrics (IJCB 2023). We provide an over\-view of the models, data preprocessing, and training procedures. We also report the performance of our models against the competition's baseline model and discuss the implications of our findings

Building support for Slovene in Spacy library

Predstavljen je postopek izgradnje podpore za slovenščino v okolju Spacy, ki je ena najpopularnejših knjižnic za obdelavo naravnega jezika. Opisane so osnovne funkcionalnosti orodij za obdelavo naravnega jezika in predsta- vljene nekatere obstoječe knjižnice, modeli ter korpusi s tega področja. Po- drobneje je predstavljeno okolje Spacy in njegova implementacija cevovoda za označevanje besedil. Praktični del obsega izdelavo novih modelov za le- matizacijo, oblikoskladenjsko označevanje, skladenjsko razčlenjevanje in pre- poznavanje imenskih entitet v standardnem in nestandardnem slovenskem jeziku. Ena od komponent izdelave so besedni vektorji, ki jih generiramo iz obstoječih prosto dostopnih korpusov. Modeli strojnega učenja so ustvarjeni s pomočjo odprtokodne knjižnice Thincc. Opisan je postopek konfiguracije in treniranja modelov na ročno označenih učnih množicah ssj500k (za stan- dardno slovenščino) in Janes-Tag (za nestandardno slovenščino). Zgrajene komponente ovrednotimo s primerjavo hitrosti ter natančnosti že obstoječih modelov.We present the implementation of the Slovenian annotation pipeline in Spacy, which is one of the most popular libraries for natural language processing. We outline some of the existing tools, models and corpora. Spacy and it’s low-level pipeline for language annotations are described in detail. We imple- mentint new models for lemmatization, part-of-speech tagging, dependency parsing and named entity recognition for Slovenian. We generate static word embeddings from existing and publicly available corpora. The models are built using neural networks and the open source library Thincc. We describe the configuration and training of the models on two public corpora, ssj500k (for standard Slovenian) and Janes-Tag (for nonstandard Slovenian). The models are evaluated and compared to existing tools

Ocena kakovosti ponarejenih posnetkov

In this thesis, we tackle the issues of artificial intelligence and DeepFake technology, which in the era of rapid digitalization, pose significant security and privacy concerns. We focus on the assessment of quality and visual realism of DeepFakes, a key factor for the impact of a forged video. We introduce an effective approach for quantifying the visual realism of DeepFake videos, using an ensemble of ConvNext, a Convolutional Neural Network (CNN), and Eva, a vanilla Vision Transformer (ViT). These models were trained on a subset of the DeepFake Game Competition 2022 (DFGC 2022) dataset to regress to Mean Opinion Scores (MOS) from DeepFake videos. Our work yielded successful results, securing third place in the DeepFake Game Competition on Visual Realism Assessment (DFGC-VRA 2023). The thesis provides a detailed presentation of the employed models, data preprocessing procedures, and training, as well as a comparison of our results with other competitors.V diplomski nalogi obravnavamo problematiko umetne inteligence in tehno- logijo globokih ponaredkov (angl. DeepFake), ki sta v dobi hitre digitalizacije ključni za varnost in zasebnost. Osredotočili smo se na ocenjevanje kakovosti in vizualnega realizma globoko ponarejenih videposnetkov, kar je ključnega pomena za njihov vpliv. Predstavljamo učinkovit pristop za kvantifikacijo vizualnega realizma globokih ponaredkov z uporabo ansambla dveh napred- nih globokih nevronskih mrež imenovanih ConvNext in Eva. Modela smo natrenirali na podmnožici podatkovne množice DeepFake Game Competition (DFGC) 2022, s ciljem napovedati povprečno oceno mnenja (MOS) ponare- jenega videoposnetka. Rezultati našega dela so se izkazali za uspešne, saj je naš pristop na tekmovanju DFGC-VRA 2023 zasedel tretje mesto. V diplom- ski nalogi so podrobno predstavljeni uporabljeni modeli, postopki predhodne obdelave podatkov in treniranja modelov, ter primerjava naših rezultatov s sotekmovalci

Comprehensive Slovenian-Hungarian Dictionary 2.0

The Comprehensive Slovenian-Hungarian dictionary is a general bilingual dictionary that is being compiled at the Centre for Language Resources and Technologies of the University of Ljubljana (CJVT UL). Version 2.0 contains 15,362 headwords, 61,190 translations, 28,748 collocations and other word combinations, and 7,741 examples. The file also contains links between synonymous entries or entry senses, and links between single-word headwords and compounds/phrases. The Comprehensive Slovenian-Hungarian dictionary is a growing dictionary, which means that new headwords will be added in regular intervals. The Comprehensive Slovenian-Hungarian dictionary is based on a concept (Kosem et al. 2018) that was prepared in the targeted research project KOMASS (the Concept of Hungarian-Slovenian dictionary: from a language resource to its user), funded by the Slovenian Research Agency and the Ministry of Education, Science and Sport of the Republic of Slovenia. The dictionary concept follows the state-of-the-art international lexicographic practice, e.g. bilingual dictionaries compiled at established international publishers and institutes. In the second version, nearly 5,000 entries have been added, and some corrections to the old ones were also made. Moreover, additional metadata has been included, e.g. lemma and tags for headwords and collocations, and statistical and syntactic structure information on collocations. The contact person for dictionary-related questions is Iztok Kosem ([email protected])