Machine learning (ML) applications in Internet of Things (IoT) scenarios face the issue that supervision signals, such as labeled data, are scarce and expensive to obtain. For example, it often requires a human to manually label events in a data stream by observing the same events in the real world. In addition, the performance of trained models usually depends on a specific context: (1) location, (2) time and (3) data quality. This context is not static in reality, making it hard to achieve robust and transferable machine learning for IoT systems in practice. In this paper, we address these challenges with an envisioned method that we name Knowledge Infusion. First, we present two past case studies in which we combined external knowledge with traditional data-driven machine learning in IoT scenarios to ease the supervision effort: (1) a weak-supervision approach for the IoT domain to auto-generate labels based on external knowledge (e.g., domain knowledge) encoded in simple labeling functions. Our evaluation for transport mode classification achieves a micro-F1 score of 80.2%, with only seven labeling functions, on par with a fully supervised model that relies on hand-labeled data. (2) We introduce guiding functions to Reinforcement Learning (RL) to guide the agents' decisions and experience. In initial experiments, our guided reinforcement learning achieves more than three times higher reward in the beginning of its training than an agent with no external knowledge. We use the lessons learned from these experiences to develop our vision of knowledge infusion. In knowledge infusion, we aim to automate the inclusion of knowledge from existing knowledge bases and domain experts to combine it with traditional data-driven machine learning techniques during setup/training phase, but also during the execution phase
