Identifying Implementation Bugs in Machine Learning based Image Classifiers using Metamorphic Testing

We have recently witnessed tremendous success of Machine Learning (ML) in practical applications. Computer vision, speech recognition and language translation have all seen a near human level performance. We expect, in the near future, most business applications will have some form of ML. However, testing such applications is extremely challenging and would be very expensive if we follow today's methodologies. In this work, we present an articulation of the challenges in testing ML based applications. We then present our solution approach, based on the concept of Metamorphic Testing, which aims to identify implementation bugs in ML based image classifiers. We have developed metamorphic relations for an application based on Support Vector Machine and a Deep Learning based application. Empirical validation showed that our approach was able to catch 71% of the implementation bugs in the ML applications.Comment: Published at 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2018

Cloud based testing of business applications and web services

This paper deals with testing of applications based on the principles of cloud computing. It is aimed to describe options of testing business software in clouds (cloud testing). It identifies the needs for cloud testing tools including multi-layer testing; service level agreement (SLA) based testing, large scale simulation, and on-demand test environment. In a cloud-based model, ICT services are distributed and accessed over networks such as intranet or internet, which offer large data centers deliver on demand, resources as a service, eliminating the need for investments in specific hardware, software, or on data center infrastructure. Businesses can apply those new technologies in the contest of intellectual capital management to lower the cost and increase competitiveness and also earnings. Based on comparison of the testing tools and techniques, the paper further investigates future trend of cloud based testing tools research and development. It is also important to say that this comparison and classification of testing tools describes a new area and it has not yet been done

A scalable application server on Beowulf clusters : a thesis presented in partial fulfilment of the requirement for the degree of Master of Information Science at Albany, Auckland, Massey University, New Zealand

Application performance and scalability of a large distributed multi-tiered application is a core requirement for most of today's critical business applications. I have investigated the scalability of a J2EE application server using the standard ECperf benchmark application in the Massey Beowulf Clusters namely the Sisters and the Helix. My testing environment consists of Open Source software: The integrated JBoss-Tomcat as the application server and the web server, along with PostgreSQL as the database. My testing programs were run on the clustered application server, which provide replication of the Enterprise Java Bean (EJB) objects. I have completed various centralized and distributed tests using the JBoss Cluster. I concluded that clustering of the application server and web server will effectively increase the performance of the application running on them given sufficient system resources. The application performance will scale to a point where a bottleneck has occurred in the testing system, the bottleneck could be any resources included in the testing environment: the hardware, software, network and the application that is running. Performance tuning for a large-scale J2EE application is a complicated issue, which is related to the resources available. However, by carefully identifying the performance bottleneck in the system with hardware, software, network, operating system and application configuration. I can improve the performance of the J2EE applications running in a Beowulf Cluster. The software bottleneck can be solved by changing the default settings, on the other hand, hardware bottlenecks are harder unless more investment are made to purchase higher speed and capacity hardware

Software testing education and training in Hong Kong

While the use of computer applications is widely spread in every business and, hence, the reliability of software is critical, it is believed that many organizations involved in software development do not take software testing sufficiently seriously as an important task. It is worthwhile to find out how far organizations are carrying out software testing in a systematic and structured manner or still taking on an ad-hoc approach. A survey was conducted to understand the software testing practices and the level of related education and training in Hong Kong. It was found that most testing team members did not have formal training in software testing. University curricula generally did not prepare graduates with enough coverage in software testing. It is proposed that a review of the current software engineering curricula in the universities to examine the coverage of software testing will be useful to the development of quality software. © 2005 IEEE.published_or_final_versio

COBOL to Java and Newspapers Still Get Delivered

This paper is an experience report on migrating an American newspaper company's business-critical IBM mainframe application to Linux servers by automatically translating the application's source code from COBOL to Java and converting the mainframe data store from VSAM KSDS files to an Oracle relational database. The mainframe application had supported daily home delivery of the newspaper since 1979. It was in need of modernization in order to increase interoperability and enable future convergence with newer enterprise systems as well as to reduce operating costs. Testing the modernized application proved to be the most vexing area of work. This paper explains the process that was employed to test functional equivalence between the legacy and modernized applications, the main testing challenges, and lessons learned after having operated and maintained the modernized application in production over the last eight months. The goal of delivering a functionally equivalent system was achieved, but problems remained to be solved related to new feature development, business domain knowledge transfer, and recruiting new software engineers to work on the modernized application.Comment: 4 pages, Accepted to be Published in: Proceedings of the 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), September 23-29, 2018, Madrid, Spai

Innovative Applications of Artificial Intelligence Techniques in Software Engineering

International audienceArtificial Intelligence (AI) techniques have been successfully applied in many areas of software engineering. The complexity of software systems has limited the application of AI techniques in many real world applications. This talk provides an insight into applications of AI techniques in software engineering and how innovative application of AI can assist in achieving ever competitive and firm schedules for software development projects as well as Information Technology (IT) management. The pros and cons of using AI techniques are investigated and specifically the application of AI in IT management, software application development and software security is considered. Organisations that build software applications do so in an environment characterised by limited resources, increased pressure to reduce cost and development schedules. Organisations demand to build software applications adequately and quickly. One approach to achieve this is to use automated software development tools from the very initial stage of software design up to the software testing and installation. Considering software testing as an example, automated software systems can assist in most software testing phases. On the hand data security, availability, privacy and integrity are very important issues in the success of a business operation. Data security and privacy policies in business are governed by business requirements and government regulations. AI can also assist in software security, privacy and reliability. Implementing data security using data encryption solutions remain at the forefront for data security. Many solutions to data encryption at this level are expensive, disruptive and resource intensive. AI can be used for data classification in organizations. It can assist in identifying and encrypting only the relevant data thereby saving time and processing power. Without data classification organizations using encryption process would simply encrypt everything and consequently impact users more than necessary. Data classification is essential and can assist organizations with their data security, privacy and accessibility needs. This talk explores the use of AI techniques (such as fuzzy logic) for data classification and suggests a method that can determine requirements for classification of organizations' data for security and privacy based on organizational needs and government policies. Finally the application of FCM in IT management is discussed

Testing as a Service on Cloud A Review

Software testing is an important part of software engineering life cycle. Software testing is a process used for evaluating an attributes or capability of program and makes sure that it meets the requirements. The application building techniques have changed and has adapted to newly emerging technology of cloud. Cloud computing has changed t he way of obtaining computing resources, and also has given a new direction to manage and deliver computing services, technologies, and solutions. Cloud computing not only brings new business opportunities, but also causes some majo r impacts on software testing and maintenance. Cloud computing creates an opportunity that offer s testing as a service (TaaS) for SaaS and Cloud s. This lead to a new phase shift in conventional testing thereby identifying new issues, challenges and needs in software testing, particular in testing Cloud s and Cloud - based applications. This paper gives a comprehensive view on Testing as a Service. Also a comparative view towards conventional testing and Cloud testing is also considered

TOWARDS UTILIZATION OF LEAN CANVAS IN THE DEVOPS SOFTWARE

The growth of technology made human to depend more on the software applications in his daily life and nowadays software companies focused more on building robust error free software to end customers in very short time. Software development companies facing one side growth of technological complexity and another side build the products fast to win a competition in business. In recent years growth of a DevOps given lot of new growth opportunity for the software companies. DevOps basic principles focused on the collaboration and communication as a key in between software development information technology professional. It is concentrated on the automating the most of the routine tasks such as development, delivery, infrastructure, support, software testing in software development process. DevOps also emphasize on the building, testing and releasing the software more quickly and in a reliable way