17,354 research outputs found
Identifying Implementation Bugs in Machine Learning based Image Classifiers using Metamorphic Testing
We have recently witnessed tremendous success of Machine Learning (ML) in
practical applications. Computer vision, speech recognition and language
translation have all seen a near human level performance. We expect, in the
near future, most business applications will have some form of ML. However,
testing such applications is extremely challenging and would be very expensive
if we follow today's methodologies. In this work, we present an articulation of
the challenges in testing ML based applications. We then present our solution
approach, based on the concept of Metamorphic Testing, which aims to identify
implementation bugs in ML based image classifiers. We have developed
metamorphic relations for an application based on Support Vector Machine and a
Deep Learning based application. Empirical validation showed that our approach
was able to catch 71% of the implementation bugs in the ML applications.Comment: Published at 27th ACM SIGSOFT International Symposium on Software
Testing and Analysis (ISSTA 2018
DeepSaucer: Unified Environment for Verifying Deep Neural Networks
In recent years, a number of methods for verifying DNNs have been developed.
Because the approaches of the methods differ and have their own limitations, we
think that a number of verification methods should be applied to a developed
DNN. To apply a number of methods to the DNN, it is necessary to translate
either the implementation of the DNN or the verification method so that one
runs in the same environment as the other. Since those translations are
time-consuming, a utility tool, named DeepSaucer, which helps to retain and
reuse implementations of DNNs, verification methods, and their environments, is
proposed. In DeepSaucer, code snippets of loading DNNs, running verification
methods, and creating their environments are retained and reused as software
assets in order to reduce cost of verifying DNNs. The feasibility of DeepSaucer
is confirmed by implementing it on the basis of Anaconda, which provides
virtual environment for loading a DNN and running a verification method. In
addition, the effectiveness of DeepSaucer is demonstrated by usecase examples
Evolution and Detection of Polymorphic and Metamorphic Malwares: A Survey
Malwares are big threat to digital world and evolving with high complexity.
It can penetrate networks, steal confidential information from computers, bring
down servers and can cripple infrastructures etc. To combat the threat/attacks
from the malwares, anti- malwares have been developed. The existing
anti-malwares are mostly based on the assumption that the malware structure
does not changes appreciably. But the recent advancement in second generation
malwares can create variants and hence posed a challenge to anti-malwares
developers. To combat the threat/attacks from the second generation malwares
with low false alarm we present our survey on malwares and its detection
techniques.Comment: 5 Page
Exploration of Finite 2D Square Grid by a Metamorphic Robotic System
We consider exploration of finite 2D square grid by a metamorphic robotic
system consisting of anonymous oblivious modules. The number of possible shapes
of a metamorphic robotic system grows as the number of modules increases. The
shape of the system serves as its memory and shows its functionality. We
consider the effect of global compass on the minimum number of modules
necessary to explore a finite 2D square grid. We show that if the modules agree
on the directions (north, south, east, and west), three modules are necessary
and sufficient for exploration from an arbitrary initial configuration,
otherwise five modules are necessary and sufficient for restricted initial
configurations
- …