GeXSe (Generative Explanatory Sensor System): An Interpretable Deep Generative Model for Human Activity Recognition in Smart Spaces

We introduce GeXSe (Generative Explanatory Sensor System), a novel framework designed to extract interpretable sensor-based and vision domain features from non-invasive smart space sensors. We combine these to provide a comprehensive explanation of sensor-activation patterns in activity recognition tasks. This system leverages advanced machine learning architectures, including transformer blocks, Fast Fourier Convolution (FFC), and diffusion models, to provide a more detailed understanding of sensor-based human activity data. A standout feature of GeXSe is our unique Multi-Layer Perceptron (MLP) with linear, ReLU, and normalization layers, specially devised for optimal performance on small datasets. It also yields meaningful activation maps to explain sensor-based activation patterns. The standard approach is based on a CNN model, which our MLP model outperforms.GeXSe offers two types of explanations: sensor-based activation maps and visual domain explanations using short videos. These methods offer a comprehensive interpretation of the output from non-interpretable sensor data, thereby augmenting the interpretability of our model. Utilizing the Frechet Inception Distance (FID) for evaluation, it outperforms established methods, improving baseline performance by about 6\%. GeXSe also achieves a high F1 score of up to 0.85, demonstrating precision, recall, and noise resistance, marking significant progress in reliable and explainable smart space sensing systems.Comment: 29 pages,17 figure

Mechanisms for sharing interpersonal context and social titles as awareness information

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007.Includes bibliographical references (p. 67-70).Mobile awareness systems aim to convey personal context information between people in a way that is less intrusive, somewhat automatic, and often much more persistent than a phone call. Although there are many ways in which awareness information can be useful, the privacy of the individual can become more of an issue as more information is made available. Prototype awareness systems developed to date have therefore often been aimed at familiar cliques. In this thesis proposal, we introduce a mobile address book based awareness system called Look Who's Talking (LWT) that aims to transcend the user's various social settings and social groups. Among its novel features is the ability to grant 'episodic access' to the device owner's context information in circumstances where their day-today access settings do not suffice. This is achieved by way of an SMS-like message called a LookAtMe. LWT also introduces a new type of awareness information for communicating aspects of the user's social attention (called Social Titling). Additionally, the system has a user interface that is geared towards mobile use, including mechanisms for on-the-fly input and a glanceable summary of incoming awareness information.by Matthew Graham Adcock.S.M

The magic window : balancing privacy and awareness in office settings

Co-workers who are physically distributed in the same building often obtain information about others through the windows in office doors. Using the information gathered by looking through the window, they can determine whether it is a good time to initiate a conversation with the occupant. There are, however, two problems with ordinary glass windows. First, there are times when the window does not provide enough information, such as when the occupant is away. Second, there is potential to violate the occupant’s privacy; as a result of the privacy risk, people often cover their windows entirely. If office windows are to work efficiently as a support for collaboration, there must be a balance between awareness and privacy. In this research, I augmented the functions of a physical office window with a computer-mediated replacement called the Magic Window. The Magic Window collects video of the occupant, mediates the signal in various ways, and then presents the altered view on a screen that replaces the glass window. The Magic Window provides a better balance of awareness and privacy in office settings by allowing occupant to differentiate the amount of awareness information based on the viewer. The Magic Window system was tested in an eight-month field trial. The trial showed that the augmented window did provide a balance of privacy and awareness, and also raised a number of issues that will aid the design of future design of co-present media spaces

Social Feedback: Social Learning from Interaction History to Support Information Seeking on the Web

Information seeking on the Web has become a central part of many daily activities. Even though information seeking is extremely common, there are many times when these tasks are unsuccessful, because the information found is less than ideal or the task could have been completed more efficiently. In unsuccessful information-seeking tasks, there are often other people who have knowledge or experience that could help improve task success. However, information seekers do not typically look for help from others, because tasks can often be completed alone (even if inefficiently). One of the problems is that web tools provide people with few opportunities to learn from one another’s experiences in ways that would allow them to improve their success. This dissertation presents the idea of social feedback. Social feedback is based on the theory of social learning, which describes how people learn from observing others. In social feedback, observational learning is enabled through the mechanism of interaction history – the traces of activity people create as they interact with the Web. Social feedback systems collect and display interaction history to allow information seekers to learn how to complete their tasks more successfully by observing how other people have behaved in similar situations. The dissertation outlines the design of two social-feedback systems, and describes two studies that demonstrate the real world applicability and feasibility of the idea. The first system supports global learning, by allowing people to learn new search skills and techniques that improve information seeking success in many different tasks. The second system supports local learning, in which people learn how to accomplish specific tasks more effectively and more efficiently. Two further studies are conducted to explore potential real-world challenges to the successful deployment of social feedback systems, such as the privacy concerns associated with the collection and sharing of interaction history. These studies show that social feedback systems can be deployed successfully for supporting real world information seeking tasks. Overall, this research shows that social feedback is a valuable new idea for the social use of information systems, an idea that allows people to learn from one another’s experiences and improve their success in many common real-world tasks

Using relationship to control disclosure in awareness servers

Awareness servers provide information about a person to help observers determine whether they are available for contact. A tradeoff exists in these systems: more sources of information, and higher fidelity in those sources, can improve people’s decisions, but each increase in information reduces privacy. In this paper, we look at whether the type of relationship between the observer and the person being observed can be used to manage this tradeoff. We conducted a survey that asked people what amount of information from different sources that they would disclose to seven different relationship types. We found that in more than half of the cases, people would give different amounts of information to different relationships. We also found that the only relationship to consistently receive less information was the acquaintance – essentially the person without a strong relationship at all. Our results suggest that awareness servers can be improved by allowing finergrained control than what is currently available