Inspector Gadget: A Data Programming-based Labeling System for Industrial Images

As machine learning for images becomes democratized in the Software 2.0 era, one of the serious bottlenecks is securing enough labeled data for training. This problem is especially critical in a manufacturing setting where smart factories rely on machine learning for product quality control by analyzing industrial images. Such images are typically large and may only need to be partially analyzed where only a small portion is problematic (e.g., identifying defects on a surface). Since manual labeling these images is expensive, weak supervision is an attractive alternative where the idea is to generate weak labels that are not perfect, but can be produced at scale. Data programming is a recent paradigm in this category where it uses human knowledge in the form of labeling functions and combines them into a generative model. Data programming has been successful in applications based on text or structured data and can also be applied to images usually if one can find a way to convert them into structured data. In this work, we expand the horizon of data programming by directly applying it to images without this conversion, which is a common scenario for industrial applications. We propose Inspector Gadget, an image labeling system that combines crowdsourcing, data augmentation, and data programming to produce weak labels at scale for image classification. We perform experiments on real industrial image datasets and show that Inspector Gadget obtains better performance than other weak-labeling techniques: Snuba, GOGGLES, and self-learning baselines using convolutional neural networks (CNNs) without pre-training.Comment: 10 pages, 11 figure

Carpe Diem: On the Evaluation of World Knowledge in Lifelong Language Models

In an ever-evolving world, the dynamic nature of knowledge presents challenges for language models that are trained on static data, leading to outdated encoded information. However, real-world scenarios require models not only to acquire new knowledge but also to overwrite outdated information into updated ones. To address this under-explored issue, we introduce the temporally evolving question answering benchmark, EvolvingQA - a novel benchmark designed for training and evaluating LMs on an evolving Wikipedia database, where the construction of our benchmark is automated with our pipeline using large language models. Our benchmark incorporates question-answering as a downstream task to emulate real-world applications. Through EvolvingQA, we uncover that existing continual learning baselines have difficulty in updating and forgetting outdated knowledge. Our findings suggest that the models fail to learn updated knowledge due to the small weight gradient. Furthermore, we elucidate that the models struggle mostly on providing numerical or temporal answers to questions asking for updated knowledge. Our work aims to model the dynamic nature of real-world information, offering a robust measure for the evolution-adaptability of language models.Comment: 14 pages, 5 figures, 5 tables; accepted at NeurIPS Syntheticdata4ML workshop, 202

FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets

Evaluation of Large Language Models (LLMs) is challenging because aligning to human values requires the composition of multiple skills and the required set of skills varies depending on the instruction. Recent studies have evaluated the performance of LLMs in two ways, (1) automatic evaluation on several independent benchmarks and (2) human or machined-based evaluation giving an overall score to the response. However, both settings are coarse-grained evaluations, not considering the nature of user instructions that require instance-wise skill composition, which limits the interpretation of the true capabilities of LLMs. In this paper, we introduce FLASK (Fine-grained Language Model Evaluation based on Alignment SKill Sets), a fine-grained evaluation protocol that can be used for both model-based and human-based evaluation which decomposes coarse-level scoring to an instance-wise skill set-level. Specifically, we define 12 fine-grained skills needed for LLMs to follow open-ended user instructions and construct an evaluation set by allocating a set of skills for each instance. Additionally, by annotating the target domains and difficulty level for each instance, FLASK provides a holistic view with a comprehensive analysis of a model's performance depending on skill, domain, and difficulty. Through using FLASK, we compare multiple open-sourced and proprietary LLMs and observe highly-correlated findings between model-based and human-based evaluations. FLASK enables developers to more accurately measure the model performance and how it can be improved by analyzing factors that make LLMs proficient in particular skills. For practitioners, FLASK can be used to recommend suitable models for particular situations through comprehensive comparison among various LLMs. We release the evaluation data and code implementation at https://github.com/kaistAI/FLASK

Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering

This publication is the Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering from July 6-8, 2022. The EG-ICE International Workshop on Intelligent Computing in Engineering brings together international experts working on the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolution of challenges such as supporting multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways. &nbsp

Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering

This publication is the Proceedings of the 29th EG-ICE International Workshop on Intelligent Computing in Engineering from July 6-8, 2022. The EG-ICE International Workshop on Intelligent Computing in Engineering brings together international experts working on the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolution of challenges such as supporting multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways. &nbsp

Wireless Inchworm-like Compact Soft Robot by Induction Heating of Magnetic Composite

Microrobots and nanorobots have been produced with various nature-inspired soft materials and operating mechanisms. However, freely operating a wirelessly miniaturized soft robot remains a challenge. In this study, a wireless crawling compact soft robot using induction heating was developed. The magnetic composite heater built into the robot was heated wirelessly via induction heating, causing a phase change in the working fluid surrounding the heater. The pressure generated from the evaporated fluid induces the bending of the robot, which is composed of elastomers. During one cycle of bending by heating and shrinking by cooling, the difference in the frictional force between the two legs of the robot causes it to move forward. This robot moved 7240 μm, representing 103% of its body length, over nine repetitions. Because the robot’s surface is made of biocompatible materials, it offers new possibilities for a soft exploratory microrobot that can be used inside a living body or in a narrow pipe

Snakeskin-Inspired 3D Printable Soft Robot Composed of Multi-Modular Vacuum-Powered Actuators

A modular soft actuator with snakeskin-inspired scales that generates an anisotropic friction force is designed and evaluated in this study. The actuator makes it possible to fabricate soft robots that can move on various surfaces in the natural environment. For existing modulus soft robots, additional connectors and several independent pneumatic pumps are required. However, we designed precise connection and snake-scale structures integrated with a single pneumatic modular actuator unit. The precise structure was printed using a DLP 3D printer. The movement characteristics of the soft robot changed according to the angle of the scale structure, and the movement distance increased as the number of modular soft actuator units increased. Soft robots that can move in operating environments such as flat land, tubes, inclined paths, and water have been realized. Furthermore, soft robots with modularization strategies can easily add modular units. We demonstrate the ability to deliver objects 2.5 times heavier than the full weight of the soft robot by adding tong-like structure to the soft robot. The development of a soft robot inspired by snakeskin suggests an easy approach to soft robots that enables various tasks even in environments where existing robots have limited activity

Wireless Micro Soft Actuator without Payloads Using 3D Helical Coils

To receive a greater power and to demonstrate the soft bellows-shaped actuator’s wireless actuation, micro inductors were built for wireless power transfer and realized in a three-dimensional helical structure, which have previously been built in two-dimensional spiral structures. Although the three-dimensional helical inductor has the advantage of acquiring more magnetic flux linkage than the two-dimensional spiral inductor, the existing microfabrication technique produces a device on a two-dimensional plane, as it has a limit to building a complete three-dimensional structure. In this study, by using a three-dimensional printed soluble mold technique, a three-dimensional heater with helical coils, which have a larger heating area than a two-dimensional heater, was fabricated with three-dimensional receiving inductors for enhanced wireless power transfer. The three-dimensional heater connected to the three-dimensional helical inductor increased the temperature of the liquid and gas inside the bellows-shaped actuator while reaching 176.1% higher temperature than the heater connected to the two-dimensional spiral inductor. Thereby it enables a stroke of the actuator up to 522% longer than when it is connected to the spiral inductor. Therefore, three-dimensional micro coils can offer a significant approach to the development of wireless micro soft robots without incurring heavy and bulky parts such as batteries

Evaluation of Weather Information for Short-Term Wind Power Forecasting with Various Types of Models

The rising share of renewable energy in the energy mix brings with it new challenges such as power curtailment and lack of reliable large-scale energy grid. The forecasting of wind power generation for provision of flexibility, defined as the ability to absorb and manage fluctuations in the demand and supply by storing energy at times of surplus and releasing it when needed, is important. In this study, short-term forecasting models of wind power generation were developed using the conventional time-series method and hybrid models using support vector regression (SVR) based on rolling origin recalibration. For the application of the methodology, the meteorological database from Korea Meteorological Administration and actual operating data of a wind power turbine (2.3 MW) from 1 January to 31 December 2015 were used. The results showed that the proposed SVR model has higher forecasting accuracy than the existing time-series methods. In addition, the conventional time-series model has high accuracy under proper curation of wind turbine operation data. Therefore, the analysis results reveal that data curation and weather information are as important as the model for wind power forecasting

Evaluation of Weather Information for Short-Term Wind Power Forecasting with Various Types of Models

The rising share of renewable energy in the energy mix brings with it new challenges such as power curtailment and lack of reliable large-scale energy grid. The forecasting of wind power generation for provision of flexibility, defined as the ability to absorb and manage fluctuations in the demand and supply by storing energy at times of surplus and releasing it when needed, is important. In this study, short-term forecasting models of wind power generation were developed using the conventional time-series method and hybrid models using support vector regression (SVR) based on rolling origin recalibration. For the application of the methodology, the meteorological database from Korea Meteorological Administration and actual operating data of a wind power turbine (2.3 MW) from 1 January to 31 December 2015 were used. The results showed that the proposed SVR model has higher forecasting accuracy than the existing time-series methods. In addition, the conventional time-series model has high accuracy under proper curation of wind turbine operation data. Therefore, the analysis results reveal that data curation and weather information are as important as the model for wind power forecasting