Dunne, PE

University of Liverpool Repository

Book Introduction by the AuthorsInvited byPaul SpirakisP.Spirakis@liverpool.ac.ukPresident of EATCSLiverpool, UKandPatras, GreeceComputation Counts –An Introduction to Analytic Concepts inComputer Science⇤Paul E. DunneDept. of Computer ScienceUniversity of Liverpoolped@csc.liv.ac.ukOverviewThe study of Computer Science requires some familiarity with many mathemat-ical topics. Traditionally undergraduate programmes have focused on what aredubbed “Discrete Methods”: Boolean algebra and logic; sets and relations; com-binatorics; graph theory; probability, etc. One consequence has been the availabil-ity of numerous excellent and carefully presented textbooks having the commonaim of providing clear expositions of Discrete Methods and taking particular carerespecting the needs and background of first-year CS students.When it comes to fields such as Calculus, Vector and Matrix algebra, Com-plex Number Theory, and Spectral Methods, however, a rather di↵erent pictureemerges. Although in modern CS, research areas such as Optimization, Compu-tational Game Theory, Data Science, Machine Learning, etc., often exploit math-ematical machinery originating in what I shall refer to as “Analytic Concepts”,there appear to be very few resources o↵ering accessible and relevant introduc-tions. Such texts as are available being either intended for specialist mathematicsstudents or directed towards motivating applications in non-computing disciplinessuch as Chemistry or Biological Sciences.It is increasingly the case that Analytic Concepts as described in the precedingparagraph feature as part of a first or second year undergraduate CS degree pro-gramme. This may especially hold true of institutions o↵ering specialist modulesin Data Science, advanced AI, or Optimization. The text of the present book isbased on material from a module (Analytic Techniques for CS) taught by the au-⇤Vol. 23 of Texts in Computing, College Publications, 2019, 540pp, ISBN 978-1-84890-310-4thor to first-year students at the University of Liverpool. This was introduced inJanuary 2018 and adds to an existing first semester Discrete Methods module.This book is intended to address the gap between the widespread availabilityof Discrete Methods textbooks and the comparative lack of relevant texts withrespect to Analytic Concepts. Its rationale is twofold: firstly, to present the basicconcerns and techniques of a number of mathematical topics not usually dealt withon Discrete Methods courses; secondly, to do so in a manner which makes clearthrough examples why the specific themes are essential elements of CS.A key theme promoted throughout the text (and emphasized in the volume’stitle) is that the fundamental computational question is “How do I count and mea-sure this object?”. This, of course, is a rather di↵erent perspective from, mostimportantly, that of Claude Shannon in [3], whereby the fundamental compu-tational question is posed as “What is information?”. I think both of these areperfectly justifiable claims and I have no intention of disputing Shannon’s view.Where there is a distinction between the two lies in the interests of the audiencesto whom the question is posed. The challenge raised by “What is information?”asks for a solution to an abstract issue of huge importance to the formal basis ofComputer Science as a Science. The question “How do I count and measure thisobject?” deals with rather more mundane, but no less important, matters: namelywith how computation is used to assist with practical issues. Thus, on one side“What is information?” concerns computation as a science; on the other “‘Howdo I count and measure?” is the currency of computation as a day-to-day activity.OrganizationThe book consists of eight chapters comprising an Introduction, six specialisttechnical chapters, and a concluding reprise.The structure of each technical chapter has a similar pattern.a. An introduction to the topic under consideration that in some cases (e.g.Chapters 2, 4, and 5) includes an overview of historical development, whilein others (Chapters 3, 6 and 7) presents some contextualization with regardto CS applications.b. The main technical presentation (including some worked examples of therelevant techniques).c. A short summary of the main ideas discussed including pointers to wherethese will be developed later in the text. For example Chapter 2, amongwhose concerns are properties of polynomials, does not discuss the matterof the roots of x2 + 1 but signals its treatment and further discussion inChapter 5.d. An outline of projects describing various software activities.e. A section of “Endnotes”. These provide further commentary on some pointshighlighted in the main body. The matters discussed are not essential tounderstanding (being included solely for the benefit of interested readers)and would be likely to disrupt the flow of the main text. In general, thesemay be ignored without losing essential supporting material.It should be noted that reinforcement of practical experience is done throughthe, sometimes rather large scale, activities o↵ered within the Projects sections.There are no lengthy screeds of variation-on-a-theme exercises such as “enumer-ate the coe cients of (followed by twenty or more polynomials)”, or “find thefirst derivative and classify the critical points of (followed by another catalogueof increasingly obscure functions)”. My personal belief (concerning which I canappreciate others may disagree) is that such hand-cranking repetition does littleto enhance practical understanding. Actually implementing an algorithm to formthe quotient and remainder of two polynomials described through their respectivecoe cients seems, to me, to provide at least as great an appreciation of the me-chanics of polynomial division as would be given by laboriously working througha dozen or more specific examples.Chapter 1 – IntroductionIn this chapter a high-level overview of the specialist topics addressed in the re-mainder of the text is presented. In order to place these in context a brief surveyof how the methods considered have been applied in a selected number of otherscientific disciplines is presented. The chapter concludes by summarizing the re-quired background knowledge assumed.Chapter 2 – The Idea of NumberThis builds on the consequences of two critical observations. Firstly that the con-cept of number and its use was a hugely significant cultural development on apar with the evolution of spoken and written language. Secondly that the no-tion of “number” in purely abstract terms is computationally powerless: somemeans of representing numerical information is also needed. Pursuing the sec-ond of these leads to a review of classical number representation methods (tallyschemes, Greek and Roman systems) together with a discussion of their problem-atic aspects. The growth of modern styles through the watershed discovery ofpositional schemes including an explicit representation of zero is then discussed.An important distinction is made between the concept of some numericalquantity (x say) and the representation of that quantity in some formalism, e.g.⇢(x) where ⇢ could be anything from Natural language, Roman numerals or con-temporary decimal schemes. As a simple illustration of computational aspectssome basic algorithms for translating between x (a numerical quantity) and (x)base(the number x described in some finite base) are given.The next section of this chapter moves on to describe ideas behind di↵eringtypes of number (Natural, Whole, Integer, Rational and Real) discussing originsand notational conventions. This leads to a treatment of the formal structure ofpolynomials of a given degree and their properties. In addition to introducingideas relating to polynomial roots a range of operations on polynomial forms(product, division, factorization etc.) are o↵ered. The principal ideas are de-scribed using univariate forms, however, a brief summary of multivariate struc-tures is included.Chapter 3 – Vectors and MatricesThe premise of this chapter is that there are contexts requiring some form of or-dered structure to be added to the basic notion of number type. It presents vectors(distinguishing some di↵erences from the relational algebra concept of n-tuple),operations on vectors such as scalar and cross product together with an introduc-tion to the notion of vector space. The concept of di↵ering types of vector trans-formation is then introduced. The relation between linear transformations andmatrix-vector product is described followed by a description of a ne transfor-mations. The class of projective transformations is mentioned, however, detaileddescription is avoided.A core application of vector and matrix methods in CS arises in realizing ba-sic graphics e↵ects. After describing simple (2 ⇥ 2, 3 ⇥ 3) matrix forms these areillustrated together with homogenous coordinates by giving matrix-vector imple-mentations of e↵ects such as scaling, rotation, translation and shearing.Chapter 4 – Basic introduction to CalculusThis introduces elements of Di↵erential Calculus emphasizing its importance inOptimization Theory. After motivating the idea of first derivative in geometricterms, the second derivative test is described and applied to some examples.In real environments it is rarely the case that Optimization involves findinga setting for just a single variable. In order to give some flavour of how mul-tivariable optimization could be handled, the concept of partial di↵erentiation isintroduced with an overview of how optimization problems can be tackled throughthe analysis of partial derivatives. Some example cases are then worked throughand potential issues with extension to large numbers of variables discussed.The concept of roots of a polynomial had been described in Chapter 2 andthe importance of Di↵erential Calculus to root finding techniques is considered.This is via comparison of two widely-used methods: Halley’s which subject tosome technical conditions can be used to find roots (zeros) of arbitrary functionsand Laguerre’s which is specialized to discovering roots of polynomials. TheJenkins-Traub polynomial root finding method (from [1, 2]) while referred to isnot discussed in depth.The chapter concludes with a short overview of Integral Calculus concentrat-ing on its application as a measurement technique. The connection between In-tegral and Di↵erential Calculus is presented through the concept of antiderivativeso that (as a sophisticated extension of the so-called “Method of Exhaustion”) thearea spanned by a function between two values may be developed using the an-tiderivative. Discussion of the more creative techniques applied to find antideriva-tives is avoided, however, the extension of basic area measurement to line lengthsand volumes is briefly considered.Chapter 5 – Complex NumbersHere an introduction to Complex Numbers is o↵ered. The starting point concernsthe (apparent) lacuna in the review of polynomial properties from Chapter 2: sincepolynomials of degree k have k roots (allowing for multiplicity) what are the tworoots of x2 + 1?I am sure I am not alone in noticing the level of suspicion that is sometimesencountered among students (especially those with minimal previous mathemat-ical background) when these are just baldly stated to be {ı, ı}. Since this levelof scepticism was felt by figures as distinguished as René Descartes whose com-ments gave rise to the term “Imaginary” number, some detailed discussion of thebasis for this suspicion and an attempt at rebuttal are presented.The chapter continues by giving the standard alternative representations ofComplex Numbers (including Argand diagrams, polar coordinates and Euler form)together with basic operations on these.One of the problematic aspects of Complex analysis is the care needed whendealing with Complex powers (zw). A standard example of fallacious treatmentis given: evaluating (e2⇡ı)ı so that both 1 and e 2⇡ appear to be justifiable. A fullbreakdown of cases is not given: instances of raising a Complex number to aComplex power being uncommon in CS. The applicable analyses where at leastone element of zw is Real are presented. Discussion of Complex powers leads tothe important idea of primitive roots of unity.Computational applications arising from concepts grounded in Complex num-bers are seen to be of considerable importance. In order to emphasize this fact thechapter includes a variety of such applications. Among other topics this presents:an overview of quaternion algebra and its role in sophisticated 3-dimensionalgraphics rendering; a detailed discussion of the Fourier Transform (in its discreteform) together with its application to image analysis and integer multiplication.One final application is found in iterating Complex functions in such a way thatthese lead to intricate graphical objects such as Mandelbrot and Julia-Fatou sets.Perhaps less well-known is the use of Julia-Fatou sets in the AI field of Algorith-mic Composition, e.g. Walker [4].Chapter 6 – Statistics and Data AnalysisThis focuses on Statistics and methods in Data Analysis (regression and correla-tion coe cients). Its aim is to highlight the importance of experimental methodin CS. After reviewing the distinctive aims of Probability Theory and Statistics,some foundational ideas (population, random variable, expectation etc.) are given.This chapter then looks at the Normal Distribution and some selected discretedistributions (Binomial, Geometric, Poisson). The summary of discrete distribu-tions is followed by a review of how moments of a distribution have been used inCS. Here Markov’s inequality is presented together with one of the standard firstmoment arguments applied in analyzing 3-colourability. After discussing somecommon statistical fallacies the concept of Statistical Confidence is presented to-gether with the elements of some tools used to analyze confidence (Student t-test,Welch’s Test) and a small number of example studies. The chapter concludes withan overview of regression methods and other data analysis techniques. This in-cludes a presentation of linear least squares regression, its extension to a smallnumber of non-linear functions (quadratic regression being dealt with separately),polynomial interpolation via Lagrange interpolants, and measures such as Spear-man’s and Pearson’s correlation coe cients.Chapter 7 – Introduction to Spectral MethodsChapter 7 returns to the topic of Matrix Theory with a particular emphasis onspectral methods. It describes various computational views of matrix determi-nant, introduces the eigenvalue problem and the Perron-Frobenius Theorem. Anumber of computational methods for approximating eigenvalues and eigenvec-tors are discussed: Power and Inverse Power methods as well as Hotelling’s De-flation. Jacobi’s transformation of symmetric matrices to triangular is describedat a high-level only.The principal focus, however, is towards important applications of spectralanalysis in CS. Among those presented are Google’s page rank technique usingnot only the basic ranking method under the assumption that no pages form “dead-ends” but also the modifications adopted in order to ensure the presence of a dom-inant eigenvalue.Further applications are o↵ered through the matrix description form knownas Singular Value Decomposition (svd). The use of svd as a method of (lossy)Data Compression is illustrated by showing its e↵ect when applied to a series ofimages.As another application, included to emphasize that spectral methods continueto find new uses in CS, some recent work from Computational Argument usingspectral techniques to model unruly debate is described.Chapter 8 – EpilogueThis concluding part reviews the material presented over the previous chapterswith the aim of placing the di↵erent topics in the context of developments inCS. Thus the treatment of polynomials is considered with respect to influence onComputer Algebra and Automated Theorem proving; the application of ComplexNumbers to novel ideas in AI (such as Algorithmic Composition). This chapteralso discusses two significant areas which have not featured in the main body:Numerical Methods and Information Theory and the rationale for their omission.SummaryThe aim of this book is to provide an introduction to mathematical ideas whichare of significance in more advanced CS studies. It does not assume mathemat-ical background beyond what would have been covered at school or within anintroductory Discrete Methods course. In order to promote accessibility, technicaland detailed mathematical exposition is kept to a minimum. The core rationale isto show why specific mathematical disciplines matter and to provide support forsuch claims through illustrative applications in modern CS.CaveatIt is only fair to point out (as will, undoubtedly, be confirmed by many studentswho have had to contend with it) that the author has what may best be describedas some “eclectic eccentricities” and an even more exaggerated, eccentric senseof humour. Some readers may consider that one or even both of these have beenrather too freely indulged in the text. This would be unfortunate and I can onlyrefer those of such an opinion to the comment on p. 460 (or to that terse rebut-tal of “you-ought-not-to-have” critics embodied within John 19:22). That said,most of the more heinous examples are confined to discussion in endnotes (e.g.the “achievements” of Paulo Valmes in Chapter 2; the discussion of lateralismin Chapter 3). As I have indicated earlier, these are included for the benefit ofinterested readers and are not essential to understanding of the main text.There are, however, a few worked examples (e.g. one of the multivariable opti-mization examples, that involving the Geometric Distribution and that concerningconfidence estimates in Poisson distributions with small sample sets) where thetemptation to describe the setting used proved impossible to resist.About the authorPaul Dunne is a Professor of CS at the University of Liverpool where he hasworked since 1985. He studied CS at the University of Edinburgh (1977–1981)and completed his PhD research in the area of Boolean function complexity atWarwick University (1981–1984) under Mike Paterson’s supervision. In his timeat Liverpool he has had experience in teaching all levels of undergraduate fromfirst year through to Honours year presenting courses on Computability and Com-plexity Theory, Algorithms, Operating Systems, and the topic of the present book.He has published research in several fields including Boolean Function complex-ity, phase transition phenomena, AI and Law, complexity in multiagent systems,and models of Computational Argument.References[1] M. A. Jenkins. Algorithm 493: zeros of a real polynomial [c2]. ACM Transactionson Mathematical Software (TOMS), 1(2):178–189, 1975.[2] M. A. Jenkins and J. F. Traub. Algorithm 419: zeros of a complex polynomial [c2].Comm. of the ACM, 15(2):97–99, 1972.[3] C. E. Shannon. A mathematical theory of communication. Bell System TechnicalJournal, 27(3):379–423, 1948.[4] E. Walker. Chaos Melody Theory. Music in Music Technology New York University,Master’s thesis, 2001.

Computation counts - an Introduction to Analytic Concepts in Computer Science (Invited Book Introduction)

http://livrepository.liverpool.ac.uk/3049233/1/583-2311-1-PB.pdf

Computation counts - an Introduction to Analytic Concepts in Computer Science (Invited Book Introduction)

Abstract

Similar works

Full text

Available Versions

University of Liverpool Repository