A brief history of software engineering

Abstract We present a personal perspective of the Art of Programming. We start with its state around 1960 and follow its development to the present day. The term Software Engineering became known after a conference in 1968, when the difficulties and pitfalls of designing complex systems were frankly discussed. A search for solutions began. It concentrated on better methodologies and tools. The most prominent were programming languages reflecting the procedural, modular, and then object-oriented styles. Software engineering is intimately tied to their emergence and improvement. Also of significance were efforts of systematizing, even automating program documentation and testing. Ultimately, analytic verification and correctness proofs were supposed to replace testing. More recently, the rapid growth of computing power made it possible to apply computing to ever more complicated tasks. This trend dramatically increased the demands on software engineers. Programs and systems became complex and almost impossible to fully understand. The sinking cost and the abundance of computing resources inevitably reduced the care for good design. Quality seemed extravagant, a loser in the race for profit. But we should be concerned about the resulting deterioration in quality. Our limitations are no longer given by slow hardware, but by our own intellectual capability. From experience we know that most programs could be significantly improved, made more reliable, economical and comfortable to use. The 1960s and the Origin of Software Engineering It is unfortunate that people dealing with computers often have little interest in the history of their subject. As a result, many concepts and ideas are propagated and advertised as being new, which existed decades ago, perhaps under a different terminology. I believe it worth while to occasionally spend some time to consider the past and to investigate how terms and concepts originated. I consider the late 1950s as an essential period of the era of computing. Large computers became available to research institutions and universities. Their presence was noticed mainly in engineering and natural sciences, but also in business they soon became indispensable. The time when they were accessible only to a few insiders in laboratories, when they broke down every time one wanted to use them, belonged to the past. Their emergence from the closed laboratory of electrical engineers into the public domain meant that their use, in particular their programming, became an activity of many. A new profession was born; but the large computers themselves became hidden within closely guarded cellars. Programmers brought their programs to the counter, where a dispatcher would pick them up, queue them, and where the results could be fetched hours or days later. There was no interactivity between man and computer. 2 Programming was known to be a sophisticated task requiring devotion and scrutiny, and a love for obscure codes and tricks. In order to facilitate this coding, formal notations were created. We now call them programming languages. The primary idea was to replace sequences of special instruction code by mathematical formulas. The first widely known language, Fortran, was issued by IBM (Backus, 1957), soon followed by Algol (1958) and its official successor in 1960. As computers were then used for computing rather than storing and communicating, these languages catered mainly to numerical mathematics. In 1962 the language Cobol was issued by the US Department of Defense for business applications. But as computing capacity grew, so did the demands on programs and on programmers: Tasks became more and more intricate. It was slowly recognized that programming was a difficult task, and that mastering complex problems was non-trivial, even when -or because -computers were so powerful. Salvation was sought in "better" programming languages, in more "tools", even in automation. A better language should be useful in a wider area of application, be more like a "natural" language, offer more facilities. PL/1 was designed to unify scientific and commercial worlds. It was advertised under the slogan "Everybody can program thanks to PL/1". Programming languages and their compilers became a principal cornerstone of computing science. But they neither fitted into mathematics nor electronics, the two traditional sectors where computers were used. A new discipline emerged, called Computer Science in America, Informatics in Europe. In 1963 the first time-sharing system appeared (MIT, Stanford, McCarthy, DEC PDP-1). It brought back the missing interactivity. Computer manufacturers picked the idea up and announced time-sharing systems for their large mainframes (IBM 360/67, GE 645). It turned out that the transition from batch processing systems to time-sharing systems, or in fact their merger, was vastly more difficult then anticipated. Systems were announced and could not be delivered on time. The problems were too complex. Research was to be conducted "on the job". The new, hot topics were multiprocessing and concurrent programming. The difficulties brought big companies to the brink of collapse. In 1968 a conference sponsored by NATO was dedicated to the topic (1968 at Garmisch-Partenkirchen, Germany) [1]. Although critical comments had occasionally been voiced earlier Programming as a Discipline In the academic world it was mainly E.W.Dijkstra and C.A.R.Hoare, who recognized the problems and offered new ideas. In 1965 Dijkstra wrote his famous Notes on Structured Programming Furthermore, in 1966 Dijkstra wrote a seminal paper about harmoniously cooperating processes Of course, all this did not change the situation, nor dispel all difficulties over night. Industry could change neither policies nor tools rapidly. Nevertheless, intensive training courses on structured programming were organized, notably by H. D. Mills in IBM. None less than the US Department of Defense realized that problems were urgent and growing. It started a project that ultimately led to the programming language Ada, a highly structured language suitable for a wide variety of applications. Software development within the DoD would then be based exclusively on Ada UNIX and C Yet, another trend started to pervade the programming arena, notably academia, pointing in the opposite direction. It was spawned by the spread of the UNIX operating system, contrasting to MIT's MULTICS and used on the quickly emerging minicomputers. UNIX was a highly welcome relief from the large operating systems established on mainframe computers. In its tow UNIX carried the language C [8], which had been explicitly designed to support the development of UNIX. Evidently, it was therefore at least attractive, if not even mandatory to use C for the development of applications running under UNIX, which acted like a Trojan horse for C. From the point of view of software engineering, the rapid spread of C represented a great leap backward. It revealed that the community at large had hardly grasped the true meaning of the term "high-level language" which became an ill-understood buzzword. What, if anything, was to be "high-level"? As this issue lies at the core of software engineering, we need to elaborate. Abstraction Computer systems are machines of large complexity. This complexity can be mastered intellectually by one tool only: Abstraction. A language represents an abstract computer whose objects and constructs lie closer (higher) to the problem to be represented than to the concrete machine. For example, in a high-level language we deal with numbers, indexed arrays, data types, conditional and repetitive statements, rather than with bits and bytes, addressed words, jumps and condition codes. However, these abstractions are beneficial only, if they are consistently and completely defined in terms of their own properties. If this is not so, if the abstractions can be understood only in terms of the facilities of an underlying computer, then the benefits are marginal, almost given away. If debugging a program -undoubtedly the most pervasive activity in software engineeringrequires a "hexadecimal dump", then the language is hardly worth the trouble

Table driven prediction for recursive descent LL(k) parsers

Programming languages are typically described in BNF or some extension of BNF, and the process of converting these descriptions into parsers is performed by parser generators. Some of the parser generators that convert LL grammars into parsers construct them to use recursive descent that gives them context during execution. The context is provided by the execution stack as the parser descends into the grammar and this is what allows the full expressiveness of LL grammars. Table driven parsers can be generated instead but restrictions are placed on the LL grammars that can be accepted. The benefit of tables is that they facilitate a separation of syntax analysis and semantic code written by a language designer. They are also faster and they simplify language implementation and modification. This paper proposes the possibility of a hybrid system that makes decisions using tables but once decisions are made recursive descent is employed to maintain context. The benefits of each system are maintained, and the drawbacks are mitigated. Also discussed are the modifications made to an existing parser generator, oops (version 2), so that it accepts LL(k) grammars and builds parsers using this system as proof-of-concept

Several types of types in programming languages

Types are an important part of any modern programming language, but we often forget that the concept of type we understand nowadays is not the same it was perceived in the sixties. Moreover, we conflate the concept of "type" in programming languages with the concept of the same name in mathematical logic, an identification that is only the result of the convergence of two different paths, which started apart with different aims. The paper will present several remarks (some historical, some of more conceptual character) on the subject, as a basis for a further investigation. The thesis we will argue is that there are three different characters at play in programming languages, all of them now called types: the technical concept used in language design to guide implementation; the general abstraction mechanism used as a modelling tool; the classifying tool inherited from mathematical logic. We will suggest three possible dates ad quem for their presence in the programming language literature, suggesting that the emergence of the concept of type in computer science is relatively independent from the logical tradition, until the Curry-Howard isomorphism will make an explicit bridge between them.Comment: History and Philosophy of Computing, HAPOC 2015. To appear in LNC