Breaking Audio Captcha using Machine Learning/Deep Learning and Related Defense Mechanism

CAPTCHA is a web-based authentication method used by websites to distinguish between humans (valid users) and bots(attackers). Audio captcha is an accessible captcha meant for the visually disabled section of users such as color-blind, blind, near-sighted users. In this project, I analyzed the security of audio captchas from attacks that employ machine learning and deep learning models. Audio captchas of varying lengths (5, 7 and 10) and varying background noise (no noise, medium noise or high noise) were analyzed. I found that audio captchas with no background noise or medium background noise were easily attacked with 99% - 100% accuracy. Whereas, audio captchas with high noise were relatively more secure with breaking accuracy of 85%. I also propose that adversarial example attacks can be used in favor of audio captcha, that is, adversarial example attacks can be used to defend audio captcha from attackers. I explored two adversarial examples attack algorithms: Basic Iterative Method (BIM) and DeepFool method to create new adversarial audio captcha. Finally, I analyzed the security of these newly created adversarial audio captcha by simulating Level I and Level II defense scenarios. Level I defense is a defense against pre- trained models that have never seen adversarial examples before. Whereas a Level II defense is a defense against models that have been re-trained on adversarial examples. My experiments show that Level I defense can prevent nearly 100% of attacks from pre-trained models. It also proves that Level II defense increases security of audio captcha by 57% to 67%. Real world scenarios such as multi-retries are also studied and related defense mechanism are suggested

Detecting and indexing moving objects for Behavior Analysis by Video and Audio Interpretation

2012 - 2013In the last decades we have assisted to a growing need for security in many public environments. According to a study recently conducted by the European Security Observatory, one half of the entire population is worried about the crime and requires the law enforcement to be protected. This consideration has lead the proliferation of cameras and microphones, which represent a suitable solution for their relative low cost of maintenance, the possibility of installing them virtually everywhere and, finally, the capability of analysing more complex events. However, the main limitation of this traditional audiovideo surveillance systems lies in the so called psychological overcharge issue of the human operators responsible for security, that causes a decrease in their capabilities to analyse raw data flows from multiple sources of multimedia information; indeed, as stated by a study conducted by Security Solutions magazine, after 12 minutes of continuous video monitoring, a guard will often miss up to 45% of screen activity. After 22 minutes of video, up to 95% is overlooked. For the above mentioned reasons, it would be really useful to have available an intelligent surveillance system, able to provide images and video with a semantic interpretation, for trying to bridge the gap between their low-level representation in terms of pixels, and the high-level, natural language description that a human would give about them. On the other hand, this kind of systems, able to automatically understand the events occurring in a scene, would be really useful in other application fields, mainly oriented to marketing purposes. Especially in the last years, a lot of business intelligent applications have been installed for assisting decision makers and for giving an organization’s employees, partners and suppliers easy access to the information they need to effectively do their jobs... [edited by author]XII n.s

Elaborazione audio dei Segnali con reti neurali profonde per la rilevazione di situazioni di pericolo

Nei sistemi di sorveglianza moderni, soluzioni composte dall’unione di telecamere a circuito chiuso e tecniche di intelligenza artificiale, rappresentano lo strumento principale per fronteggiare minacce e pericoli in diversi ambienti: ambienti pubblici, abitazioni private, uffici, strutture critiche come ospedali o scuole. Questi sistemi vengono equipaggiati da robuste tecniche di computer vision, le quali permettono di riconoscere e rilevare oggetti e persone, attraverso sequenze di immagini in maniera automatica. L’obiettivo è predire l’azione degli elementi osservati in un determinato scenario per aumentare l’efficienza globale di un sistema di sorveglianza. Tuttavia, l’analisi delle immagini può subire importanti cali di prestazioni in diverse circostanze, dovuti alla natura dei sensori video e dalle limitazioni che essi introducono. Nel progetto di tesi presentato, si discute lo sviluppo di un sistema di riconoscimento di situazioni di pericolo i cui dati elaborati sono acquisiti da sensori audio. Negli ultimi anni, la sorveglianza audio ha riscosso un grande interesse grazie alla flessibilità di utilizzo, sia per la diversità delle situazioni in cui può essere impiegata, sia per la possibilità di essere combinata con la controparte video in sistemi ibridi. Il sistema proposto è costituito da una rete neurale convoluzionale, la cui architettura si ispira fortemente alla VGG19. Al suo ingresso vengono fornite immagini costruite a partire da porzioni di stream audio e trasformate in rappresentazioni tempo-frequenza quali: spettrogramma, spettrogramma in scala Mel e gammatonogramma. L’obiettivo è stato quello di costruire un modello di classificazione di eventi audio di pericolo, per i quali si sono considerati suoni come: vetri che si infrangono, colpi di pistola e urla. Successivamente si è condotto un confronto sia tra le performance indotte dall’utilizzo delle tre rappresentazioni, sia tra la rete neurale e una tecnica di classificazione standard quale l’SV

Application of machine learning to quantify forest cover loss in the Congo Basin and its implications for large mammal habitat suitability

Machine learning (ML) models are a powerful tool for land use and land cover (LULC) mapping. In the African tropics, and particularly in the Congo Basin, there is a need to better assess the performance and reliability of ML-based LULC classification using coarse-resolution satellite images. In the context of ongoing climate change and socioeconomically-driven forest disturbances, it is important to understand and quantify the extent of forest cover loss in the Congo Basin, as well as the impact of this loss on suitable habitat for key wildlife species. In this dissertation, I address these key issues in three manuscript-based chapters. In Chapter 2, I compared the classification performance of four ML algorithms (k-nearest neighbor (kNN), support vector machines (SVM), artificial neural networks (ANN), and random forests (RF)) for LULC mapping within a tropical region in Central Africa (the Mayo Rey department of northern Cameroon). All four classification algorithms produced high accuracy (overall classification accuracy > 80%), with the RF model (> 90% classification accuracy) outperforming the other algorithms. In Chapter 3, I used the RF model, together with the Idrissi TerrSet land change modeler, to map and project LULCC for the Congo Basin under historical and future scenarios of socioeconomic impacts and climate change. I found that over 352642 km2 of dense forests have been lost in this region between 1990 and 2020, with projected continued loss of about 174860 - 204161 km2 by the year 2050. In Chapter 4, I produced spatially explicit species distribution models to map habitat suitability for great apes (chimpanzees and gorillas) and elephants within the Dzanga Sangha Protected Areas (DSPA) of the Congo Basin. I found that priority habitat areas for the three mammal species mostly occurred and overlapped spatially within the DSPA national parks. However, priority habitat areas for the three species declined by 4, 4.5 and 9.8 percentage points respectively between 2015 and 2020, mostly due to increased human pressures. This research provides a new understanding of the extend and implications of forest cover loss in the Congo Basin, highlighting the critical conservation challenges that remain in this region