Event-based Vision: A Survey

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world

Exploring the Evolution of Node Neighborhoods in Dynamic Networks

Dynamic Networks are a popular way of modeling and studying the behavior of evolving systems. However, their analysis constitutes a relatively recent subfield of Network Science, and the number of available tools is consequently much smaller than for static networks. In this work, we propose a method specifically designed to take advantage of the longitudinal nature of dynamic networks. It characterizes each individual node by studying the evolution of its direct neighborhood, based on the assumption that the way this neighborhood changes reflects the role and position of the node in the whole network. For this purpose, we define the concept of \textit{neighborhood event}, which corresponds to the various transformations such groups of nodes can undergo, and describe an algorithm for detecting such events. We demonstrate the interest of our method on three real-world networks: DBLP, LastFM and Enron. We apply frequent pattern mining to extract meaningful information from temporal sequences of neighborhood events. This results in the identification of behavioral trends emerging in the whole network, as well as the individual characterization of specific nodes. We also perform a cluster analysis, which reveals that, in all three networks, one can distinguish two types of nodes exhibiting different behaviors: a very small group of active nodes, whose neighborhood undergo diverse and frequent events, and a very large group of stable nodes

Endless construction : occupant activism and authorship

Though we don’t think of it this way, “Adaptive Reuse” often happens naturally, guided by users who modify their physical environment to satisfy urgent living needs. While these modifications may appear haphazard, they show the user’s individualism and life philosophy. We, as designers, should embrace these interventions to explore how to participate with the occupants’ adaptive reuse process over time. Given the richness of adaptation within the old residences in the ancient Chinese city of Suzhou, it is an appropriate place to explore both the skill of self–modification and the potential for interaction of designers in this process. The old residences in Suzhou have two fates ultimately, either being destroyed for new construction or being restored for tourism and commercial space. I suggest a third option: introducing occupant activism and authorship to acknowledge the ignored history of these places, as a demonstration of how people adapt to these buildings and how these buildings have survived over time. The old buildings cannot quickly adapt to shifting life habits caused by the industrial revolution and housing policy. However, through my observations of the sites, people are designing their living space spontaneously, incorporating household appliances, building technology, and customs within a limited cost. Thus, I ascribe value to the organic change that has been crafted over time to align with larger societal shifts and acknowledging the user’s actions as adaptive reuse design. Assuming the role of curator, without any intent to disturb the existing condition, this thesis proposes a speculative reality where I live in the site and encourage continued change. Setting Mr. Shen’s house as a container to implement my reinterpretations of the modifications, I’m using photos, hand drawings, plans, and creating categories through programs, building elements, and technologies to envision the fiction scenario. In 2042, children grow up and become masters of adaptive reuse. Come 2062, old families move out and new families come in, the spirit of adaptive reuse will be inherited, and the site will become a mecca for spreading self-design. What’s the boundary between everyday life and design? I try to use the untrained version of adaptive reuse to push design more to the side of everyday life. We, as designers, should practice our perspective of seeing the space and reconsider our role in design. And I encourage residents who live in the old residences to recognize and celebrate what they have done together wisely and carefully

Behavioral, morphological, and genomic analyses of population structure in brood parasitic indigobirds (Vidua spp.)

The African indigobirds (Vidua spp.) are exceptional among avian brood parasites in that mimicry of host vocalizations plays an integral role in their social behaviors and evolutionary history. Young indigobirds imprint on the vocalizations of their hosts during development, adult males include mimicry of these vocalizations in their own repertoire, and adult females use these songs to choose both their mates and the nests they parasitize. Imprinting on the host during development therefore results in assortative mating and host fidelity, but also provides a mechanism for rapid, sympatric speciation via host shift. Host shifts require some degree of host infidelity, however, and the same behavioral mechanisms may thus lead to hybridization if eggs are laid in the nest of a host species already "occupied" by another indigobird species. Thus, it is not clear if the morphological and genetic similarity of most indigobird species is due to recent common ancestry or ongoing hybridization. I addressed this uncertainty by studying indigobirds in East Africa, a region that was colonized by West African ancestors in the late Pleistocene and is currently home to four indigobird species. I analyzed variation among species in: vi1) the responses of territorial males to playbacks of conspecific and heterospecific vocalizations; 2) temporal and frequency traits of chatter calls and complex non-mimicry songs; 3) morphological characters; and 4) genomic polymorphisms. The playback experiment shows that host mimicry is an important cue in species recognition, and suggests that it may contribute to species cohesion when juveniles or adults disperse beyond the boundaries of their dialect neighborhood. Analyses of both non-mimetic vocalizations and morphological characters (i.e., plumage color and body size) reveal that they are shaped by divergence among species as well as local ecology. Analyses of thousands of "double-digest" restriction site-associated DNA (ddRAD) loci scattered across the genome indicate that both species identity and geographic divergence contribute to population structure. Taken together, the results show that the tempo of speciation and morphological divergence among indigobirds associated with different hosts is likely variable, depending on geographic context, and the breeding ecology and morphology of alternative hosts

Placing Color: Architectural Color & Facade Improvement Programs in Commercial Corridor Revitalization in Philadelphia

Exterior changes to existing and historic buildings in depressed historic areas signify financial investment and change, and have the potential to play a significant role in both urban preservation and revitalization. Changes to exterior architectural colors are perhaps the most visible signifier of such revitalization, as the use of color changes as a tool in façade improvement programs nationwide demonstrates. As part of façade improvement programs, changes to the exteriors of existing buildings in historic cities are most often a visual indication of adaptive reuse. Urban façade improvement programs that are dedicated to commercial corridor revitalization provide a variety of funding and services to business and property owners. The existence of such programs suggests that beautifying neighborhoods through changes to existing building exteriors will benefit business and property owners economically. This thesis proposes to explore and analyze the various government-sponsored façade improvement programs in Philadelphia to determine their impact on commercial corridor revitalization (or economic success as determined by the City) and on their long-term sustainability after initial investment. If façade improvement programs are indeed successful as indicators of change within a community, and thereby adaptive reuse of existing buildings, then the application of such programs within historic districts has the potential to dramatically impact the way we view, use, and reuse the existing built fabric of our cities. This thesis examines these programs and their application citywide through Philadelphia-specific case studies that are exclusive to façade improvements or upgrades rather than those that include façade improvements as part of a more comprehensive rehabilitation strategy for individual structures or neighborhoods

Heart rates estimation using rPPG methods in challenging imaging conditions

Abstract. The cardiovascular system plays a crucial role in maintaining the body’s equilibrium by regulating blood flow and oxygen supply to different organs and tissues. While contact-based techniques like electrocardiography and photoplethysmography are commonly used in healthcare and clinical monitoring, they are not practical for everyday use due to their skin contact requirements. Therefore, non-contact alternatives like remote photoplethysmography (rPPG) have gained significant attention in recent years. However, extracting accurate heart rate information from rPPG signals under challenging imaging conditions, such as image degradation and occlusion, remains a significant challenge. Therefore, this thesis aims to investigate the effectiveness of rPPG methods in extracting heart rate information from rPPG signals in these imaging conditions. It evaluates the effectiveness of both traditional rPPG approaches and rPPG pre-trained deep learning models in the presence of real-world image transformations, such as occlusion of the faces by sunglasses or facemasks, as well as image degradation caused by noise artifacts and motion blur. The study also explores various image restoration techniques to enhance the performance of the selected rPPG methods and experiments with various fine-tuning methods of the best-performing pre-trained model. The research was conducted on three databases, namely UBFC-rPPG, UCLA-rPPG, and UBFC-Phys, and includes comprehensive experiments. The results of this study offer valuable insights into the efficacy of rPPG in practical scenarios and its potential as a non-contact alternative to traditional cardiovascular monitoring techniques

Predicting human behavior in smart environments: theory and application to gaze prediction

Predicting human behavior is desirable in many application scenarios in smart environments. The existing models for eye movements do not take contextual factors into account. This addressed in this thesis using a systematic machine-learning approach, where user profiles for eye movements behaviors are learned from data. In addition, a theoretical innovation is presented, which goes beyond pure data analysis. The thesis proposed the modeling of eye movements as a Markov Decision Processes. It uses Inverse Reinforcement Learning paradigm to infer the user eye movements behaviors