Action selection in modular reinforcement learning

textModular reinforcement learning is an approach to resolve the curse of dimensionality problem in traditional reinforcement learning. We design and implement a modular reinforcement learning algorithm, which is based on three major components: Markov decision process decomposition, module training, and global action selection. We define and formalize module class and module instance concepts in decomposition step. Under our framework of decomposition, we train each modules efficiently using SARSA($\lambda$) algorithm. Then we design, implement, test, and compare three action selection algorithms based on different heuristics: Module Combination, Module Selection, and Module Voting. For last two algorithms, we propose a method to calculate module weights efficiently, by using standard deviation of Q-values of each module. We show that Module Combination and Module Voting algorithms produce satisfactory performance in our test domain.Computer Science

Parabolic Dish Stirling Module

The design, manufacture, and assembly of a commercially designed parabolic dish Stirling 25 kWe module is examined. The cost, expected performance, design uniquenesses, and future commercial potential of this module, which is regarded as the most technically advanced in the parabolic dish industry is discussed

A Hands-On, Active Learning Approach to Increasing Manufacturing Knowledge in Engineering Students

This paper describes a new learning module implemented as part of the senior capstone design course at Marquette University to teach engineering students about basic manufacturing processes, lean manufacturing principles, and design for manufacturability. The module includes several examples of active and student centered learning as part of an in-class assembly line simulation exercise. Students reflected on this experience, and suggested process improvements to save time, reduce cost and waste, and improve the assembly line process. They learned of the importance of manufacturing documentation, process design, and design for assembly. At the end of the module, students understood the importance of designing a product not only for the end user, but also for the assemblers and inspectors. Details of the module design and implementation will be presented along with comments from students

Block 5 module design summary

The Flat Plate Solar Array Block 5 Module design efforts of six manufacturers are summarized. Residential designs and module designs for intermediate-load applications are described. It is planned that a few prototype modules of each design will be fabricated. The modules incorporate advanced concepts, are larger than earlier ones and have high power output

Development of Combined Opto-Acoustical Sensor Modules

The faint fluxes of cosmic neutrinos expected at very high energies require large instrumented detector volumes. The necessary volumes in combination with a sufficient shielding against background constitute forbidding and complex environments (e.g. the deep sea) as sites for neutrino telescopes. To withstand these environments and to assure the data quality, the sensors have to be reliable and their operation has to be as simple as possible. A compact sensor module design including all necessary components for data acquisition and module calibration would simplify the detector mechanics and ensures the long term operability of the detector. The compact design discussed here combines optical and acoustical sensors inside one module, therefore reducing electronics and additional external instruments for calibration purposes. In this design the acoustical sensor is primary used for acoustic positioning of the module. The module may also be used for acoustic particle detection and marine science if an appropriate acoustical sensor is chosen. First tests of this design are promising concerning the task of calibration. To expand the field of application also towards acoustic particle detection further improvements concerning electromagnetic shielding and adaptation of the single components are necessary.Comment: 4 pages, 2 figures, ARENA2010 proceeding

A Student-Centered Learning Approach to Design for Manufacturability: Meeting the Needs of an Often- Forgotten Customer

A hands-on learning module was implemented at Marquette University in 2012 to teach biomedical engineering students about basic manufacturing processes, lean manufacturing principles, and design for manufacturability. It incorporates active and student-centered learning as part of in-class assembly line simulations. Since then, it has evolved from three class periods to five. The module begins with two classroom presentations on manufacturing operations and electronics design, assembly, and testing. Students then participate in an in-class assembly line simulation exercise where they build and test an actual product per written work instructions. They reflect on this experience and suggest design and process changes to improve the assembly line process and quality, save time, and reduce cost and waste. At the end of the module students implement their suggested design and process improvements and repeat the exercise to determine the impact of their improvements. They learn of the importance of Design for Manufacturability, well-written work instructions, process design, and designing a product not only for the end user, but also for the assemblers and inspectors. Details of the module, and its implementation and assessment are presented along with student feedback and faculty observations

Discrete event simulation tool for analysis of qualitative models of continuous processing systems

An artificial intelligence design and qualitative modeling tool is disclosed for creating computer models and simulating continuous activities, functions, and/or behavior using developed discrete event techniques. Conveniently, the tool is organized in four modules: library design module, model construction module, simulation module, and experimentation and analysis. The library design module supports the building of library knowledge including component classes and elements pertinent to a particular domain of continuous activities, functions, and behavior being modeled. The continuous behavior is defined discretely with respect to invocation statements, effect statements, and time delays. The functionality of the components is defined in terms of variable cluster instances, independent processes, and modes, further defined in terms of mode transition processes and mode dependent processes. Model construction utilizes the hierarchy of libraries and connects them with appropriate relations. The simulation executes a specialized initialization routine and executes events in a manner that includes selective inherency of characteristics through a time and event schema until the event queue in the simulator is emptied. The experimentation and analysis module supports analysis through the generation of appropriate log files and graphics developments and includes the ability of log file comparisons

Flow characteristics of various swirl-can module designs

Flow measurements were performed on each of six swirl-can combustor module designs under simulated combustor operating conditions to find the design which exhibited a small recirculation zone, intense air mixing, and good fuel distribution in its wake. Conditions that are favorable for producing low oxides of nitrogen emissions and high combustion efficiency were investigated. The recirculation zone, the turbulence intensity and the fuel distribution pattern are obtained in the wake region of the center module of a three module array. The most promising swirl-can module design incorporates two air swirlers which discharge air in opposite directions (contraswirl), mixes the fuel and air upstream of the inner swirler, and has a flow area blockage of 64.3% for the three module array

Development of basic quantity surveying learning module through 21st century learning activity

Technical and Vocational Education in Malaysia (TVET) is one of the most important branches of education to produce a balanced human capital. Not only the skills in the field need to be emphasized, but the skills also need to be enhanced under the 'Eleventh Malaysia Plan ' (RMK-11). 21st century learning is a student centered learning process based on 4C’s elements which are communication, collaborative, critical thinking and creativity. The importance of producing 21st century learning based modules is essential to ensure that TVET students, especially at the Vocational College, are exposed to 21st century learning activities and can apply the 4C’s element. Therefore, this study was conducted to develop Basic Quantity Surveying Learning Module Through The 21st Century Learning Activity that tested its validity and credibility as reference material or teaching aids to assist teachers and students taking this course. This module design has two stages beginning with goal building and ending with a complete module that is ready to use. The module validation process involves using the instrument and involving three experts for verification. The evaluation of the module validation consists of a questionnaire consisting of 28 items based on the format and content of the module. The findings show that Basic Quantity Surveying Learning Module Through The 21st Century Learning Activity is highly validated. Analyst views are presented as a percentage. Based on the result, 96% agree on the design of the module while 98.46%. agree on the content of the module developed. Through the analysis of the findings, the Basic Quantity Surveying Module Through The 21st Century Learning Activities has high content validity