A debugging approach for live Big Data applications

International audienceMany frameworks exist for programmers to develop and deploy Big Data applications such as Hadoop Map/Reduce and Apache Spark. However, very little debugging support is currently provided in those frameworks. When an error occurs, developers are lost in trying to understand what has happened from the information provided in log files. Recently, new solutions allow developers to record & replay the application execution, but replaying is not always affordable when hours of computation need to be re-executed. In this paper, we present an online approach that allows developers to debug Big Data applications in isolation by moving the debugging session to an external process when a halting point is reached. We introduce IDRA MR , our prototype implementation in Pharo. IDRA MR centralizes the debugging of parallel applications by introducing novel debugging concepts, such as composite debugging events, and the ability to dynamically update both the code of the debugged application and the same configuration of the running framework. We validate our approach by debugging both application and configuration failures for two driving scenarios. The scenarios are implemented and executed using Port, our Map/Reduce framework for Pharo, also introduced in this paper

On-stack replacement, distilled

On-stack replacement (OSR) is essential technology for adaptive optimization, allowing changes to code actively executing in a managed runtime. The engineering aspects of OSR are well-known among VM architects, with several implementations available to date. However, OSR is yet to be explored as a general means to transfer execution between related program versions, which can pave the road to unprecedented applications that stretch beyond VMs. We aim at filling this gap with a constructive and provably correct OSR framework, allowing a class of general-purpose transformation functions to yield a special-purpose replacement. We describe and evaluate an implementation of our technique in LLVM. As a novel application of OSR, we present a feasibility study on debugging of optimized code, showing how our techniques can be used to fix variables holding incorrect values at breakpoints due to optimizations

Bringing Back-in-Time Debugging Down to the Database

With back-in-time debuggers, developers can explore what happened before observable failures by following infection chains back to their root causes. While there are several such debuggers for object-oriented programming languages, we do not know of any back-in-time capabilities at the database-level. Thus, if failures are caused by SQL scripts or stored procedures, developers have difficulties in understanding their unexpected behavior. In this paper, we present an approach for bringing back-in-time debugging down to the SAP HANA in-memory database. Our TARDISP debugger allows developers to step queries backwards and inspecting the database at previous and arbitrary points in time. With the help of a SQL extension, we can express queries covering a period of execution time within a debugging session and handle large amounts of data with low overhead on performance and memory. The entire approach has been evaluated within a development project at SAP and shows promising results with respect to the gathered developer feedback.Comment: 24th IEEE International Conference on Software Analysis, Evolution, and Reengineerin

Multiple-View Tracing for Haskell: a New Hat

Different tracing systems for Haskell give different views of a program at work. In practice, several views are complementary and can productively be used together. Until now each system has generated its own trace, containing only the information needed for its particular view. Here we present the design of a trace that can serve several views. The trace is generated and written to file as the computation proceeds. We have implemented both the generation of the trace and several different viewers

Challenges for the Adoption of Model-Driven Web Engineering Approaches in Industry

Model-driven web engineering approaches have become an attractive research and technology solution for Web application development. However, after 20 years of development, they have attracted little attention from the Industry due to the mismatch between technical versus research requirements. In this joint work between academia and industry, the authors present the current problems of using these approaches in scale and provide guidelines to convert them into viable industry solutions.Ministerio de ciencia e Innovación TIN2016-76956-C3-2-RMinisterio de Economía y Competitividad TIN2015-71938-RED