International Peer Review of the Performance Audit Practice of the United States Government Accountability Office

Other written product issued by the Government Accountability Office with an abstract that begins "An international peer review team with representatives from the supreme audit institutions of Canada, Australia, Mexico, the Netherlands, Norway, South Africa, and Sweden reviewed the quality assurance system that the United States Government Accountability Office (GAO) has established for managing its performance audit practice. The GAO's quality assurance system encompasses its organizational structure and the policies and procedures established to provide it with reasonable assurance of complying with Government Auditing Standards. The GAO is responsible for the design of its quality assurance system and compliance with it, including the quality of its products. The responsibility of the peer review team is to express an opinion on whether the system is suitably designed and operating effectively to meet its objective. The criteria the peer review team used to assess the GAO's quality assurance system were drawn from GAO legislative authorities, Government Auditing Standards, and the GAO performance audit manual. The peer review team conducted the review in accordance with the peer review standards in Government Auditing Standards, and in a manner consistent with the Code of Ethics and auditing standards issued by the International Organization of Supreme Audit Institutions (INTOSAI). The peer review team examined the GAO's documented policies and procedures relative to applicable professional standards, reviewed documentation for a representative sample of 2004 audits, and interviewed professional and administrative staff.

A Framework for Exploiting Emergent Behaviour to capture 'Best Practice' within a Programming Domain

Inspection is a formalised process for reviewing an artefact in software engineering. It is proven to significantly reduce defects, to ensure that what is delivered is what is required, and that the finished product is effective and robust. Peer code review is a less formal inspection of code, normally classified as inadequate or substandard Inspection. Although it has an increased risk of not locating defects, it has been shown to improve the knowledge and programming skills of its participants. This thesis examines the process of peer code review, comparing it to Inspection, and attempts to describe how an informal code review can improve the knowledge and skills of its participants by deploying an agent oriented approach. During a review the participants discuss defects, recommendations and solutions, or more generally their own experience. It is this instant adaptability to new 11 information that gives the review process the ability to improve knowledge. This observed behaviour can be described as the emergent behaviour of the group of programmers during the review. The wider distribution of knowledge is currently only performed by programmers attending other reviews. To maximise the benefits of peer code review, a mechanism is needed by which the findings from one team can be captured and propagated to other reviews / teams throughout an establishment. A prototype multi-agent system is developed with the aim of capturing the emergent properties of a team of programmers. As the interactions between the team members is unstructured and the information traded is dynamic, a distributed adaptive system is required to provide communication channels for the team and to provide a foundation for the knowledge shared. Software agents are capable of adaptivity and learning. Multi-agent systems are particularly effective at being deployed within distributed architectures and are believed to be able to capture emergent behaviour. The prototype system illustrates that the learning mechanism within the software agents provides a solid foundation upon which the ability to detect defects can be learnt. It also demonstrates that the multi-agent approach is apposite to provide the free flow communication of ideas between programmers, not only to achieve the sharing of defects and solutions but also at a high enough level to capture social information. It is assumed that this social information is a measure of one element of the review process's emergent behaviour. The system is capable of monitoring the team-perceived abilities of programmers, those who are influential on the programming style of others, and the issues upon III which programmers agree or disagree. If the disagreements are classified as unimportant or stylistic issues, can it not therefore be assumed that all agreements are concepts of "Best Practice"? The conclusion is reached that code review is not a substandard Inspection but is in fact complementary to the Inspection model, as the latter improves the process of locating and identifying bugs while the former improves the knowledge and skill of the programmers, and therefore the chance of bugs not being encoded to start with. The prototype system demonstrates that it is possible to capture best practice from a review team and that agents are well suited to the task. The performance criteria of such a system have also been captured. The prototype system has also shown that a reliable level of learning can be attained for a real world task. The innovative way of concurrently deploying multiple agents which use different approaches to achieve the same goal shows remarkable robustness when learning from small example sets. The novel way in which autonomy is promoted within the agents' design but constrained within the agent community allows the system to provide a sufficiently flexible communications structure to capture emergent social behaviour, whilst ensuring that the agents remain committed to their own goals

Software Engineering for Millennials, by Millennials

Software engineers need to manage both technical and professional skills in order to be successful. Our university offers a 5.5 year program that mixes computer science, software and computer engineering, where the first two years are mostly math and physics courses. As such, our students' first real teamwork experience is during the introductory SE course, where they modify open source projects in groups of 6-8. However, students have problems working in such large teams, and feel that the course material and project are "disconnected". We decided to redesign this course in 2017, trying to achieve a balance between theory and practice, and technical and professional skills, with a maximum course workload of 150 hrs per semester. We share our experience in this paper, discussing the strategies we used to improve teamwork and help students learn new technologies in a more autonomous manner. We also discuss what we learned from the two times we taught the new course.Comment: 8 pages, 9 tables, 4 figures, Second International Workshop on Software Engineering Education for Millennial