Optimizing Vision Transformers: Leveraging Max and Min Operations for Efficient Pruning

The research on Deep Neural Networks (DNNs) continues to enhance the performance of these models over a wide spectrum of tasks, increasing their adoption in many fields. This leads to the need of extending their usage also on edge devices with limited resources, even though, with the advent of Transformer-based models, this has become an increasingly complex task because of their size. In this context, pruning emerges as a crucial tool to reduce the number of weights in the memory-hungry Fully Connected (FC) layers. This paper explores the usage of neurons based on the Multiply-And-Max/min (MAM) operation, an alternative to the conventional Multiply-and-Accumulate (MAC), in a Vision Transformer (ViT). This enhances the model prunability thanks to the usage of Max and Min operations. For the first time, many MAM-based FC layers are used in a large state-of-the-art DNN model and compressed with various pruning techniques available in the literature. Experiments show that MAM-based layers achieve the same accuracy of traditional layers using up to 12 times less weights. In particular, when using Global Magnitude Pruning (GMP), the FC layers following the Multi-head Attention block of a ViT-B/16 model, fine-tuned on CIFAR-100, count only 560000 weights if MAM neurons are used, compared to the 31.4 million that remain when using traditional MAC neurons

Human Bone-Marrow-Derived Stem-Cell-Seeded 3D Chitosan–Gelatin–Genipin Scaffolds Show Enhanced Extracellular Matrix Mineralization When Cultured under a Perfusion Flow in Osteogenic Medium

Tissue-engineered bone tissue grafts are a promising alternative to the more conventional use of natural donor bone grafts. However, choosing an appropriate biomaterial/scaffold to sustain cell survival, proliferation, and differentiation in a 3D environment remains one of the most critical issues in this domain. Recently, chitosan/gelatin/genipin (CGG) hybrid scaffolds have been proven as a more suitable environment to induce osteogenic commitment in undifferentiated cells when doped with graphene oxide (GO). Some concern is, however, raised towards the use of graphene and graphene-related material in medical applications. The purpose of this work was thus to check if the osteogenic potential of CGG scaffolds without added GO could be increased by improving the medium diffusion in a 3D culture of differentiating cells. To this aim, the level of extracellular matrix (ECM) mineralization was evaluated in human bone-marrow-derived stem cell (hBMSC)-seeded 3D CGG scaffolds upon culture under a perfusion flow in a dedicated custom-made bioreactor system. One week after initiating dynamic culture, histological/histochemical evaluations of CGG scaffolds were carried out to analyze the early osteogenic commitment of the culture. The analyses show the enhanced ECM mineralization of the 3D perfused culture compared to the static counterpart. The results of this investigation reveal a new perspective on more efficient clinical applications of CGG scaffolds without added GO

Event-based Classification with Recurrent Spiking Neural Networks on Low-end Micro-Controller Units

Due to its intrinsic sparsity both in time and space, event-based data is optimally suited for edge-computing applications that require low power and low latency. Time varying signals encoded with this data representation are best processed with Spiking Neural Networks (SNN). In particular, recurrent SNNs (RSNNs) can solve temporal tasks using a relatively low number of parameters, and therefore support their hardware implementation in resource-constrained computing architectures. These premises propel the need of exploring the properties of these kinds of structures on low-power processing systems to test their limits both in terms of computational accuracy and resource consumption, without having to resort to full-custom implementations. In this work, we implemented an RSNN model on a low-end, resource-constrained ARM-Cortex-M4-based Micro Controller Unit (MCU). We trained it on a down-sampled version of the N-MNIST event-based dataset for digit recognition as an example to assess its performance in the inference phase. With an accuracy of 97.2%, the implementation has an average energy consumption as low as 4.1μJ and a worst-case computational time of 150.4μs per time-step with an operating frequency of 180 MHz, so the deployment of RSNNs on MCU devices is a feasible option for small image vision real-time tasks

Haptoglobin preserves the CD163 hemoglobin scavenger pathway by shielding hemoglobin from peroxidative modification

Detoxification and clearance of extracellular hemoglobin (Hb) have been attributed to its removal by the CD163 scavenger receptor pathway. However, even low-level hydrogen peroxide (H(2)O(2)) exposure irreversibly modifies Hb and severely impairs Hb endocytosis by CD163. We show here that when Hb is bound to the high-affinity Hb scavenger protein haptoglobin (Hp), the complex protects Hb from structural modification by preventing alpha-globin cross-links and oxidations of amino acids in critical regions of the beta-globin chain (eg, Trp15, Cys93, and Cys112). As a result of this structural stabilization, H(2)O(2)-exposed Hb-Hp binds to CD163 with the same affinity as nonoxidized complex. Endocytosis and lysosomal translocation of oxidized Hb-Hp by CD163-expressing cells were found to be as efficient as with nonoxidized complex. Hp complex formation did not alter Hb's ability to consume added H(2)O(2) by redox cycling, suggesting that within the complex the oxidative radical burden is shifted to Hp. We provide structural and functional evidence that Hp protects Hb when oxidatively challenged with H(2)O(2) preserving CD163-mediated Hb clearance under oxidative stress conditions. In addition, our data provide in vivo evidence that unbound Hb is oxidatively modified within extravascular compartments consistent with our in vitro findings

Forest ecosystem monitoring in Tuscany (Italy): past activities, present status and future perspectives

Since 1987 the Region of Tuscany has been actively monitoring crown status in its forests, in order to protect them from atmospheric pollution, biotic factors and environmental change. Over this period the Region has performed periodical inventories on crown condition in publicly-owned forests (Level I network) and established a network of permanent plots (MON.I.TO., Level II – III) to study long-term changes occurring in forest ecosystems. Some of these permanent plots were later included in the national programme CONECOFOR, managed by the Ministry for Policy in Agriculture and Forest. Currently a further development of MON.I.TO. is being implemented, called MONITO III – TOpModel, the aim of which is to broaden the information potential of the monitoring system to include carbon stocks and biodiversity evaluation. This paper provides an up-to-date report on the status of the various surveys and recommends a closer connection between MON.I.TO. and the other regional information systems, especially the Regional Forest Inventory, in order to produce information that may be useful in forest planning and in Sustainable Forest Management

Detection of 'Candidatus Neoehrlichia mikurensis' and other Anaplasmataceae and Rickettsiaceae in Canidae in Switzerland and Mediterranean countries

'Candidatus Neoehrlichia mikurensis' is an emerging tick-borne zoonotic agent that primarily affects immunocompromised human patients. Dogs and foxes are frequently exposed to ticks, and both species are in close proximity to humans. This is the first study to systematically investigate the occurrence of 'Candidatus Neoehrlichia mikurensis' in Canidae in Europa. We analyzed 1'739 blood samples from dogs in Switzerland, Italy, Spain and Portugal and 162 blood samples from free-ranging red foxes (Vulpes vulpes) in Switzerland. All samples were tested using a previously described multiplex real-time PCR for the Anaplasmataceae family, the 'Candidatus Neoehrlichia' genus and the 'Candidatus Neoehrlichia mikurensis' species. All Anaplasmataceae positive samples were subsequently tested using specific real-time PCRs for Anaplasma phagocytophilum, Anaplasma platys, Ehrlichia canis and Rickettsia helvetica. Among the tested animals, one dog from Zurich tested positive for 'Candidatus Neoehrlichia mikurensis'. The 12-year old West Highland white terrier had been splenectomized 3 months prior to the blood collection and presented with polyuria/polydipsia. Fanconi syndrome was diagnosed based on glucosuria with normoglycemia and hyperaminoaciduria. A. platys and E. canis were detected in 14/249 dogs from Sicily and Portugal; two of the dogs were coinfected with both agents. Four Swiss foxes tested positive for A. phagocytophilium. R. helvetica was detected for the first time in a red fox. In conclusion, 'Candidatus Neoehrlichia mikurensis' infection should be considered in sick dogs, particularly when immunocompromised. The pathogen seems not to be widespread in Canidae in the investigated countries. Conversely, other Anaplasmataceae were more readily detected in dogs and foxes

Investigation of waste heat recovery system at supercritical conditions with vehicle drive cycles

Waste heat recovery (WHR) for internal combustion engines in vehicles using Organic Rankine cycle (ORC) has been a promising technology. The operation of the ORC WHR system in supercritical conditions has a potential to generate more power output and thermal efficiency compared with the conventional subcritical conditions. However, in supercritical conditions, the heat transfer process in the evaporator, the key component of the ORC WHR system, becomes unpredictable as the thermo-physical properties of the working fluid change with the temperature. Furthermore, the transient heat source from the vehicle’s exhaust makes the operation of the WHR system difficult. We investigated the performance of the ORC WHR system at supercritical conditions with engine’s exhaust data from real city and highway drive cycles. The effects of operating variables, such as refrigerant flow rates, evaporator and condenser pressure, and evaporator outlet temperature, on the performance indicators of the WHR system in supercritical conditions were examined. Simulation of operating parameters and the boundary of the WHR system are also included in this paper

Dynamic model of supercritical Organic Rankine Cycle waste heat recovery system for internal combustion engine

The supercritical Organic Rankine Cycle (ORC) for the Waste Heat Recovery (WHR) from Internal Combustion (IC) engines has been a growing research area in recent years, driven by the aim to enhance the thermal efficiency of the ORC and engine. Simulation of a supercritical ORC-WHR system before a real-time application is important as high pressure in the system may lead to concerns about safety and availability of components. In the ORC-WHR system, the evaporator is the main contributor to thermal inertia of the system and is considered to be the critical component since the heat transfer of this device influences the efficiency of the system. Since the thermo-physical properties of the fluid at supercritical pressures are dependent on temperature, it is necessary to consider the variations in properties of the working fluid. The wellknown Finite Volume (FV) discretization method is generally used to take those property changes into account. However, a FV model of the evaporator in steady state condition cannot be used to predict the thermal inertia of the cycle when it is subjected to transient heat sources. In this paper, a dynamic FV model of the evaporator has been developed and integrated with other components in the ORC-WHR system. The stability and transient responses along with the performance of the ORC-WHR system for the transient heat source are investigated and are also included in this paper

The importance of interacting climate modes on Australia’s contribution to global carbon cycle extremes

The global carbon cycle is highly sensitive to climate-driven fluctuations of precipitation, especially in the Southern Hemisphere. This was clearly manifested by a 20% increase of the global terrestrial C sink in 2011 during the strongest sustained La Niña since 1917. However, inconsistencies exist between El Niño/La Niña (ENSO) cycles and precipitation in the historical record; for example, significant ENSO-precipitation correlations were present in only 31% of the last 100 years, and often absent in wet years. To resolve these inconsistencies, we used an advanced temporal scaling method for identifying interactions amongst three key climate modes (El Niño, the Indian Ocean dipole, and the southern annular mode). When these climate modes synchronised (1999-2012), drought and extreme precipitation were observed across Australia. The interaction amongst these climate modes, more than the effect of any single mode, was associated with large fluctuations in precipitation and productivity. The long-term exposure of vegetation to this arid environment has favoured a resilient flora capable of large fluctuations in photosynthetic productivity and explains why Australia was a major contributor not only to the 2011 global C sink anomaly but also to global reductions in photosynthetic C uptake during the previous decade of drought

Increased masticatory activity and quality of life in elderly persons with dementia-a longitudinal matched cluster randomized single-blind multicenter intervention study.

Background: Worldwide, millions of people are suffering from dementia and this number is rising. An index of quality of life (QoL) can describe the impact a disease or treatment has on a person's wellbeing. QoL comprises many variables, including physical health and function, and mental health and function. QoL is related to masticatory ability and physical activity. Animal studies show that disruption of mastication due to loss of teeth or a soft diet leads to memory loss and learning problems. Since these are common complaints in dementia, it is hypothesized that improvement of masticatory function and normalization of diet consistency can increase QoL in elderly persons suffering from dementia. Therefore, the goal of the present study is to examine whether an increase in masticatory activity, achieved by increased food consistency and enhancement of masticatory function through improved oral health care has a positive effect on QoL, including cognition, mood, activities of daily living (ADL), and circadian rhythm in elderly persons with dementia.Methods and design: The described study is a prospective longitudinal matched cluster randomized single-blind multicenter study. Participants are elderly persons living in the Netherlands, suffering from dementia and receiving psychogeriatric care. An intervention group will receive improved oral health care and a diet of increased consistency. A control group receives care as usual. Participants will be assessed four times; outcome variables besides QoL are cognition, mood, independence, rest-activity rhythm, blood pressure, and masticatory function.Discussion: This research protocol investigates the effect of an intervention executed by daily caregivers. The intervention will increase masticatory activity, which is achieved by three different actions, (providing oral health care, increasing food consistency, or a combination of both). There is a certain amount of variety in the nature of the interventions due to local differences in nursing homes. This might be a scientific weakness in the study design; however, a practical implementation of any findings will be subject to the same factors, making this study design clinically relevant.Trial registration: NTR1561. © 2013 Weijenberg et al.; licensee BioMed Central Ltd