Learning Generalized Reactive Policies using Deep Neural Networks

We present a new approach to learning for planning, where knowledge acquired while solving a given set of planning problems is used to plan faster in related, but new problem instances. We show that a deep neural network can be used to learn and represent a \emph{generalized reactive policy} (GRP) that maps a problem instance and a state to an action, and that the learned GRPs efficiently solve large classes of challenging problem instances. In contrast to prior efforts in this direction, our approach significantly reduces the dependence of learning on handcrafted domain knowledge or feature selection. Instead, the GRP is trained from scratch using a set of successful execution traces. We show that our approach can also be used to automatically learn a heuristic function that can be used in directed search algorithms. We evaluate our approach using an extensive suite of experiments on two challenging planning problem domains and show that our approach facilitates learning complex decision making policies and powerful heuristic functions with minimal human input. Videos of our results are available at goo.gl/Hpy4e3

Introduction in IND and recursive partitioning

This manual describes the IND package for learning tree classifiers from data. The package is an integrated C and C shell re-implementation of tree learning routines such as CART, C4, and various MDL and Bayesian variations. The package includes routines for experiment control, interactive operation, and analysis of tree building. The manual introduces the system and its many options, gives a basic review of tree learning, contains a guide to the literature and a glossary, and lists the manual pages for the routines and instructions on installation

To increase predictability in complex engineering and fabrication projects : construct of a framework for planning and production control in FMC technologies

Masteroppgave i industriell økonomi og teknologiledelse IND 590 projects : construct of a framework for planning and production control in FMC technologiesHow to increase predictability in complex engineering and fabrication projects is what it is allabout. The case studied, FMC Technologies, is located in Norway, but is part of a globalcompany. FMC has grown quickly, and solving issues with tacit knowledge and personalexperience, as was done earlier, is challenging. The unit of analysis within the organization isthe department of Well Access Systems (WAS). WAS is concerned with connecting subseawells to surface rigs or vessels. A typical project consists of complex subsea equipment forwork over and intervention of established wells.The research method is based on a constructive research design for analyzing the case(Lukka, 2003). The constructive research approach is a research procedure for developingconstructions that in turn can contribute to the theory connected to the field of research. Inaddition, constructive research relates to design science research, which according to Simon(1996) is concerned with devising artifacts, e.g. tools, techniques, and methods, to attaingoals. Constructive research is a form of prescriptive research aiming at improving theperformance of the case being studied. Furthermore, our approach is based on action research(Reason & Bradbury, 2008), as we have been working closely with FMC.During our exploratory study, we got a comprehensive view of the organization, as well asmanagement processes and tools. Within WAS, two tools are used to plan and follow up onengineering activities. However, the utilization of them does not seem satisfactory to ensure asmooth project execution. The tools are the well-established “Eplan” and the newly developed“PPM tool”. However, we have found that neither Eplan nor PPM tool are planning tools;they are merely progress reporting tools. The PPM tool is based on frequent progressreporting for each task, and Eplan is based on a few milestone dates within each task. ThePPM tool was implemented as Eplan does not include all engineering activities, only puredeliverables that are sent to the client. Consequently, Eplan does not capture the actual usageof hours or remaining hours, thus failing to visualize the actual status of projects. Further, theinitial planning, which serves as input for both tools, is performed at the startup of the project,usually without sufficient emphasis on the importance of “doing it right the first time”. Thus,inconsistent milestone dates1 and infeasible resource allocations are frequent. In addition,activities are often planned in parallel with long durations and without dependency links.Consequently, on-time delivery (OTD) of documentation and drawings is found to be low atFMC. In March 2014, the OTD was as low as 38 % on average for all the ongoing projects.1 Urgent activities are planned too late, and non-urgent activities are planned too early.IVProjects within the subsea oil and gas industry tend to be large-scale, and the financial impactof delays and deviations is significant (Kalsaas, 2013). Thus, increased predictability in thedesign and engineering phase may reduce the risk of potential outburst from the initial budget.However, due to the nature of the design process, planning serves as a challenging task.Traditionally, several planning strategies used in the design process are based on linearapproaches, such as “Stage Gate” and “Waterfall” (Kalsaas, 2013). In addition, complexprojects tend to perform concurrent engineering, i.e. a number of engineering activities areunderway simultaneously and the entire set of activities converges to the design solution atonce (Hoedemaker, Blackburn, & Van Wassenhove, 1999). Yet, traditional planningtechniques take little account of the interdisciplinary, iterative nature of the design process(Austin, Baldwin, Waskett, & Li, 1999). Inevitably, this leads to cycles of rework, known asnegative iterations (Ballard, 2000b), as well as time and cost penalties in both design andfabrication. Against this background, iterative and inclusive methods for planning design andengineering, such as the Last Planner System (LPS), Critical Chain (CC) and Scrum, must besought in order to increase predictability and quality of the deliverables. The thesis presents aconstruct on how the initial planning and subsequent production control can be strengthenedby adapting ideas from these methods.Planning of design processes serves as a challenging task: the design emerges through acomplex process where solutions, and thus activities, evolve as the process progress (Ballard,1999), i.e. reciprocal dependencies (Thompson, 1967/2003). The main idea of the framework,or construct, is to postpone the documentation and drawing phase to the end of the designphase. As such, the design can be fully completed before the production of documents anddrawings commences. Further, the two distinct phases can be handled separately, as illustratedin Figure 1. Today, the design and documentation are often conducted concurrently, thusleading to several parallel activities with long durations due to reciprocal dependenciesbetween them. With several designers and engineers working in parallel, this often results inrework, i.e. negative iterations, due to late changes and poor communication. Thus, it isimportant to freeze the design at some point, in order to make the documentation phase sound.Figure 1: The design phase and the documentation and drawing phase.VThe first aspect to consider is the initial planning of the documentation and drawing phase.The main goal of the initial planning is to sequence the engineering activities in the rightorder to avoid both inconsistent delivery dates and parallel activities with unnecessary longdurations. The planning must be executed in accordance with the principles of collaborativeplanning in LPS, where different disciplines attend to unveil constraints and evaluate thebudgeted amount of hours. Based on ideas from CC, resources are allocated in advance toavoid parallel activities on individual resources. Further, the problem of infeasible resourceallocations is reduced, while the visibility is increased. The latter removes the necessity of thefrequent progress reporting done today, which further renders the PPM tool unnecessary.Today, parallel activities on individual resources must be reported frequently in order toforesee any off-track activities potentially threatening the delivery or to track cost measures,while sequential activities are more visible and easier to track, as illustrated in Figure 2.Figure 2: Parallel vs. sequential progress measurement.By structuring activities according to CC, the problems related to multitasking, StudentSyndrome, and Parkinson’s Law will be structurally mitigated (Koskela, Stratton, &Koskenvesa, 2010). Herroelen and Leus (2001) point out that multitasking is quite common inmulti-project environments where resources often have more than one significant taskrunning. However, such multitasking results in individuals who bounce back and forth,whereas the flow time in individual activities increases. Further, activities stretched over along period does not motivate the resource to go with full thrust from start, or even begin onthe task immediately after the start date, i.e. the Student Syndrome (Leach, 1999). Longdurations also affect Parkinson’s Law, stating that work expands to fill the time available(Shen & Chua, 2008).In accordance with CC, buffers are postponed to the end of each activity chain in order tovisualize off-track activities. Since several deliverables are subjected to an internal reviewbefore delivery to the client, we propose to add buffers at the end of these chains, asillustrated in Figure 3. The size of these buffers must be evaluated collaboratively at the initialplanning. No existing method seems satisfactory (Tukel, Rom, & Eksioglu, 2006): however,VIShen and Chua (2008) point out that the soundness of the tasks should be of guidance, i.e. thedegree of prerequisite work serving as input. Figure 3 illustrates how the logical sequencingof tasks visualizes the upcoming activities for the resource and determines the start date. Theblue bars represent the budgeted amount of hours, while the internal review marks thedelivery date to client.Figure 3: Sequencing of activities with postponed buffer.For progress measurement, each participating engineer reports progress in accordance withthe milestones in Eplan. Further, Eplan updates the Product Plans automatically, which servesas a holistic management tool to control cost, progress, and quality. As suggested by Shen andChua (2008), the CC framework acts as a linear controlling feature. This is also in accordancewith the addressed need of such system in LPS (Junior, Scola, & Conte, 1998; Kalsaas, 2013).However, as LPS demonstrates, planning and production control2 are strongly related.Thus, besides the framework for initial planning and progress control, a proper frameworksecuring corrective actions is necessary. The principles of production control from LPS isimplemented complimentary to CC, to allow more detailed handling of assignments, flows,and constraints (Shen & Chua, 2008). This is also supported by Koskela et al. (2010), whosuggest weekly and daily planning across all tasks, as an extension of CC. We propose weeklyforward-looking meetings, where key personnel meet and evaluate upcoming activitiesspanning six weeks ahead. An important part of this meeting is to make sure that prerequisiteinputs are available, or that actions can be taken in advance of the scheduled startup dates, tomake tasks ready for execution (Hamzeh, Ballard, & Tommelein, 2008). The mostchallenging prerequisite in FMC is human resources. Even though the initial planning securesproper resource allocation and workload distribution, the resources might have beenreassigned to other projects, or the workload of an ongoing activity might have increased dueto variation orders. Thus, it is important to look ahead and see if the upcoming workload isfeasible for the resources. Further, it is of interest to evaluate reasons for non-completion ofongoing activities as proposed by Ballard (2000a), in order to improve future planning.Forward-looking meetings are required weekly in order to get frequent updates on ongoingactivities, input on the planned workload and commitment to upcoming activities through2 Production control is monitoring of performance against project specifications (budget, plans, etc.) andcorrective actions needed to conform performance to the specifications (Ballard & Howell, 1998).VIIpublic promises, public checking of task status, and evaluation of reasons for non-completion(Koskela et al., 2010). Drawing on the ideas from Scrum, all meeting arenas should be timeboxedand standardized to reduce complexity (Schwaber & Sutherland, 2013). Thus, themeetings should be held at the same time and location, and have a fixed duration and agenda.Based on the ideas of Scrum and LPS, a framework for planning and production control ofthe design phase is further described. The Sprints in Scrum are in many ways similar to thephase scheduling in LPS, where activities and their sequence are determined. Handoffsbetween trades are identified as a part of the process to determine the sequence. The Sprintscan be considered as these handoffs, where an increment of the design serves as input forother products’ designs. In LPS, the tasks themselves are the central unit of analysis, butScrum focus on the achievement, or goal, within the phase. This is more suitable whenplanning future design activities, since it is easier to determine the preferred outcome, than theway of achieving that outcome. In contrast to Scrum, these Sprint Goals must be planned priorto commencement of the design phase. Thus, ensuring fulfillment of the total scope within theplanned period, synchronization with other product designs, and providing transparency interms of progress and cost to project management and the client. A generic set of incrementswas evaluated for one product. However, it proved impossible to make a generic set of goalsbecause the design is completely project specific, e.g. water depth, field age, installationspace, equipment interfaces, etc. Consequently, budgeted hours and percentage of total scopefor each Sprint becomes project specific as well. However, our investigation revealed thepossibility to either divide into sub-product increments or interface increments3, depending onthe product and project. This must be done as a collaborative process prior to the startup ofthe design phase. In addition, documentation and specification from systems engineering mustbe present in order to set the Sprint Goals and ensure soundness of the design phase. In Figure4, the Sprints are illustrated as sub-milestones within the 3D modelling. Each Sprint’sduration should be less than one month to reduce complexity and risk (Schwaber &Sutherland, 2013). These are implemented in the Product Plan for cost and progress measures.Figure 4: Sprints in the design phase.The Sprints connected to one product is performed by a Sprint Team, which is self-organizingand multifunctional with the ability to perform all necessary tasks. These are solely3 Control areas based on a completed part design, or interface verification between several parts.VIIIresponsible for the product design, unlike today where several participants may interfere withthe current design. The team collaborates jointly on how the specifications from the client canbe implemented in the concept design. However, the team must also collaborate with otherteams and representatives from the workshop, suppliers, etc., in order to adjust the designearly, and reduce the amount of negative iterations. As Macomber and Howell (2003) pointedout, it is of great importance for the project to use multiple sources to ensure more accurateinformation. Thus, weekly forward-looking meetings are arranged in the design phase as well.It is important to arrange these meetings weekly, and not only at the startup of each Sprint, inorder to secure an arena for frequent mutual adjustment. This is supported by Kalsaas (2013),who claims that the planning period must be shortened, and actions and decisions related tothe actual engineering activities must be detailed on a rolling basis with a short-termperspective. The team can invite different disciplines to discuss the current design, thus get aview of any upcoming obstacles, and follow up on these in order to make future tasks soundprior to commencement. An action list, and a design review document (DRM) is usedthroughout the entire design phase for each product to follow up on hindrances and documentthe process of the design, i.e. decision points and increment freezes. When the Sprint ends, aretrospective meeting is held in order to freeze the increment and update a register of lessonslearned. The learning perspective is important in order to improve future projects and is partof both Scrum and LPS.The proposed process for the design phase has an additional value for the costumer.Today, if a variation order occurs, it proves difficult to determine the impact on ongoing orcompleted work. However, if increments are frozen, it is easier to determine the effect onsubsequent work and invoice the client accordingly. Also, the DRM serves as guidance to seehow the variation order affects previously made decisions, and other products and workpackages, thus increasing flexibility. Whenever a retrospective meeting is held, possibleextensions in the scope may be proposed, in order to enhance the product beyond thecontractual provisions.In addition to a more comprehensive explanation of the construct presented, an in-depthdescription of FMC and relevant processes, a better understanding of different types ofdependencies and coordination methods suitable to control the dependencies, and acomprehensive evaluation of the framework is provided. The thesis contributes in the broaderperspective to the understanding of how increased predictability can be achieved, exemplifiedby the practical relevance in FMC