Big data and the SP theory of intelligence

This article is about how the "SP theory of intelligence" and its realisation in the "SP machine" may, with advantage, be applied to the management and analysis of big data. The SP system -- introduced in the article and fully described elsewhere -- may help to overcome the problem of variety in big data: it has potential as "a universal framework for the representation and processing of diverse kinds of knowledge" (UFK), helping to reduce the diversity of formalisms and formats for knowledge and the different ways in which they are processed. It has strengths in the unsupervised learning or discovery of structure in data, in pattern recognition, in the parsing and production of natural language, in several kinds of reasoning, and more. It lends itself to the analysis of streaming data, helping to overcome the problem of velocity in big data. Central in the workings of the system is lossless compression of information: making big data smaller and reducing problems of storage and management. There is potential for substantial economies in the transmission of data, for big cuts in the use of energy in computing, for faster processing, and for smaller and lighter computers. The system provides a handle on the problem of veracity in big data, with potential to assist in the management of errors and uncertainties in data. It lends itself to the visualisation of knowledge structures and inferential processes. A high-parallel, open-source version of the SP machine would provide a means for researchers everywhere to explore what can be done with the system and to create new versions of it.Comment: Accepted for publication in IEEE Acces

What May Visualization Processes Optimize?

In this paper, we present an abstract model of visualization and inference processes and describe an information-theoretic measure for optimizing such processes. In order to obtain such an abstraction, we first examined six classes of workflows in data analysis and visualization, and identified four levels of typical visualization components, namely disseminative, observational, analytical and model-developmental visualization. We noticed a common phenomenon at different levels of visualization, that is, the transformation of data spaces (referred to as alphabets) usually corresponds to the reduction of maximal entropy along a workflow. Based on this observation, we establish an information-theoretic measure of cost-benefit ratio that may be used as a cost function for optimizing a data visualization process. To demonstrate the validity of this measure, we examined a number of successful visualization processes in the literature, and showed that the information-theoretic measure can mathematically explain the advantages of such processes over possible alternatives.Comment: 10 page

Developing geometrical reasoning in the secondary school: outcomes of trialling teaching activities in classrooms, a report to the QCA

This report presents the findings of the Southampton/Hampshire Group of mathematicians and mathematics educators sponsored by the Qualifications and Curriculum Authority (QCA) to develop and trial some teaching/learning materials for use in schools that focus on the development of geometrical reasoning at the secondary school level. The project ran from October 2002 to November 2003. An interim report was presented to the QCA in March 2003. 1. The Southampton/Hampshire Group consisted of five University mathematicians and mathematics educators, a local authority inspector, and five secondary school teachers of mathematics. The remit of the group was to develop and report on teaching ideas that focus on the development of geometrical reasoning at the secondary school level. 2. In reviewing the existing geometry curriculum, the group endorsed the RS/ JMC working group conclusion (RS/ JMC geometry report, 2001) that the current mathematics curriculum for England contains sufficient scope for the development of geometrical reasoning, but that it would benefit from some clarification in respect of this aspect of geometry education. Such clarification would be especially helpful in resolving the very odd separation, in the programme of study for mathematics, of ‘geometrical reasoning’ from ‘transformations and co-ordinates’, as if transformations, for example, cannot be used in geometrical reasoning. 3. The group formulated a rationale for designing and developing suitable teaching materials that support the teaching and learning of geometrical reasoning. The group suggests the following as guiding principles: • Geometrical situations selected for use in the classroom should, as far as possible, be chosen to be useful, interesting and/or surprising to pupils; • Activities should expect pupils to explain, justify or reason and provide opportunities for pupils to be critical of their own, and their peers’, explanations; • Activities should provide opportunities for pupils to develop problem solving skills and to engage in problem posing; • The forms of reasoning expected should be examples of local deduction, where pupils can utilise any geometrical properties that they know to deduce or explain other facts or results. • To build on pupils’ prior experience, activities should involve the properties of 2D and 3D shapes, aspects of position and direction, and the use of transformation-based arguments that are about the geometrical situation being studied (rather than being about transformations per se); • The generating of data or the use of measurements, while playing important parts in mathematics, and sometimes assisting with the building of conjectures, should not be an end point to pupils’ mathematical activity. Indeed, where sensible, in order to build geometric reasoning and discourage over-reliance on empirical verification, many classroom activities might use contexts where measurements or other forms of data are not generated. 4. In designing and trialling suitable classroom material, the group found that the issue of how much structure to provide in a task is an important factor in maximising the opportunity for geometrical reasoning to take place. The group also found that the role of the teacher is vital in helping pupils to progress beyond straightforward descriptions of geometrical observations to encompass the reasoning that justifies those observations. Teacher knowledge in the area of geometry is therefore important. 5. The group found that pupils benefit from working collaboratively in groups with the kind of discussion and argumentation that has to be used to articulate their geometrical reasoning. This form of organisation creates both the need and the forum for argumentation that can lead to mathematical explanation. Such development to mathematical explanation, and the forms for collaborative working that support it, do not, however, necessarily occur spontaneously. Such things need careful planning and teaching. 6. Whilst pupils can demonstrate their reasoning ability orally, either as part of group discussion or through presentation of group work to a class, the transition to individual recording of reasoned argument causes significant problems. Several methods have been used successfully in this project to support this transition, including 'fact cards' and 'writing frames', but more research is needed into ways of helping written communication of geometrical reasoning to develop. 7. It was found possible in this study to enable pupils from all ages and attainments within the lower secondary (Key Stage 3) curriculum to participate in mathematical reasoning, given appropriate tasks, teaching and classroom culture. Given the finding of the project that many pupils know more about geometrical reasoning than they can demonstrate in writing, the emphasis in assessment on individual written response does not capture the reasoning skills which pupils are able to develop and exercise. Sufficient time is needed for pupils to engage in reasoning through a variety of activities; skills of reasoning and communication are unlikely to be absorbed quickly by many students. 8. The study suggests that it is appropriate for all teachers to aim to develop the geometrical reasoning of all pupils, but equally that this is a non-trivial task. Obstacles that need to be overcome are likely to include uncertainty about the nature of mathematical reasoning and about what is expected to be taught in this area among many teachers, lack of exemplars of good practice (although we have tried to address this by lesson descriptions in this report), especially in using transformational arguments, lack of time and freedom in the curriculum to properly develop work in this area, an assessment system which does not recognise students’ oral powers of reasoning, and a lack of appreciation of the value of geometry as a vehicle for broadening the curriculum for high attainers, as well as developing reasoning and communication skills for all students. 9. Areas for further work include future work in the area of geometrical reasoning, include the need for longitudinal studies of how geometrical reasoning develops through time given a sustained programme of activities (in this project we were conscious that the timescale on which we were working only enabled us to present 'snapshots'), studies and evaluation of published materials on geometrical reasoning, a study of 'critical experiences' which influence the development of geometrical reasoning, an analysis of the characteristics of successful and unsuccessful tasks for geometrical reasoning, a study of the transition from verbal reasoning to written reasoning, how overall perceptions of geometrical figures ('gestalt') develops as a component of geometrical reasoning (including how to create the links which facilitate this), and the use of dynamic geometry software in any (or all) of the above.10. As this group was one of six which could form a model for part of the work of regional centres set up like the IREMs in France, it seems worth recording that the constitution of the group worked very well, especially after members had got to know each other by working in smaller groups on specific topics. The balance of differing expertise was right, and we all felt that we learned a great deal from other group members during the experience. Overall, being involved in this type of research and development project was a powerful form of professional development for all those concerned. In retrospect, the group could have benefited from some longer full-day meetings to jointly develop ideas and analyse the resulting classroom material and experience rather than the pattern of after-school meetings that did not always allow sufficient time to do full justice to the complexity of many of the issues the group was tackling

Geoscience after IT: Part J. Human requirements that shape the evolving geoscience information system

The geoscience record is constrained by the limitations of human thought and of the technology for handling information. IT can lead us away from the tyranny of older technology, but to find the right path, we need to understand our own limitations. Language, images, data and mathematical models, are tools for expressing and recording our ideas. Backed by intuition, they enable us to think in various modes, to build knowledge from information and create models as artificial views of a real world. Markup languages may accommodate more flexible and better connected records, and the object-oriented approach may help to match IT more closely to our thought processes

Knowledge-based systems and geological survey

This personal and pragmatic review of the philosophy underpinning methods of geological surveying suggests that important influences of information technology have yet to make their impact. Early approaches took existing systems as metaphors, retaining the separation of maps, map explanations and information archives, organised around map sheets of fixed boundaries, scale and content. But system design should look ahead: a computer-based knowledge system for the same purpose can be built around hierarchies of spatial objects and their relationships, with maps as one means of visualisation, and information types linked as hypermedia and integrated in mark-up languages. The system framework and ontology, derived from the general geoscience model, could support consistent representation of the underlying concepts and maintain reference information on object classes and their behaviour. Models of processes and historical configurations could clarify the reasoning at any level of object detail and introduce new concepts such as complex systems. The up-to-date interpretation might centre on spatial models, constructed with explicit geological reasoning and evaluation of uncertainties. Assuming (at a future time) full computer support, the field survey results could be collected in real time as a multimedia stream, hyperlinked to and interacting with the other parts of the system as appropriate. Throughout, the knowledge is seen as human knowledge, with interactive computer support for recording and storing the information and processing it by such means as interpolating, correlating, browsing, selecting, retrieving, manipulating, calculating, analysing, generalising, filtering, visualising and delivering the results. Responsibilities may have to be reconsidered for various aspects of the system, such as: field surveying; spatial models and interpretation; geological processes, past configurations and reasoning; standard setting, system framework and ontology maintenance; training; storage, preservation, and dissemination of digital records