Towards Frame Rate Agnostic Multi-Object Tracking

Multi-Object Tracking (MOT) is one of the most fundamental computer vision tasks which contributes to a variety of video analysis applications. Despite the recent promising progress, current MOT research is still limited to a fixed sampling frame rate of the input stream. In fact, we empirically find that the accuracy of all recent state-of-the-art trackers drops dramatically when the input frame rate changes. For a more intelligent tracking solution, we shift the attention of our research work to the problem of Frame Rate Agnostic MOT (FraMOT). In this paper, we propose a Frame Rate Agnostic MOT framework with Periodic training Scheme (FAPS) to tackle the FraMOT problem for the first time. Specifically, we propose a Frame Rate Agnostic Association Module (FAAM) that infers and encodes the frame rate information to aid identity matching across multi-frame-rate inputs, improving the capability of the learned model in handling complex motion-appearance relations in FraMOT. Besides, the association gap between training and inference is enlarged in FraMOT because those post-processing steps not included in training make a larger difference in lower frame rate scenarios. To address it, we propose Periodic Training Scheme (PTS) to reflect all post-processing steps in training via tracking pattern matching and fusion. Along with the proposed approaches, we make the first attempt to establish an evaluation method for this new task of FraMOT in two different modes, i.e., known frame rate and unknown frame rate, aiming to handle a more complex situation. The quantitative experiments on the challenging MOT datasets (FraMOT version) have clearly demonstrated that the proposed approaches can handle different frame rates better and thus improve the robustness against complicated scenarios.Comment: 21 pages; Author versio

A multi-view CNN-based acoustic classification system for automatic animal species identification

Automatic identification of animal species by their vocalization is an important and challenging task. Although many kinds of audio monitoring system have been proposed in the literature, they suffer from several disadvantages such as non-trivial feature selection, accuracy degradation because of environmental noise or intensive local computation. In this paper, we propose a deep learning based acoustic classification framework for Wireless Acoustic Sensor Network (WASN). The proposed framework is based on cloud architecture which relaxes the computational burden on the wireless sensor node. To improve the recognition accuracy, we design a multi-view Convolution Neural Network (CNN) to extract the short-, middle-, and long-term dependencies in parallel. The evaluation on two real datasets shows that the proposed architecture can achieve high accuracy and outperforms traditional classification systems significantly when the environmental noise dominate the audio signal (low SNR). Moreover, we implement and deploy the proposed system on a testbed and analyse the system performance in real-world environments. Both simulation and real-world evaluation demonstrate the accuracy and robustness of the proposed acoustic classification system in distinguishing species of animals

Argon-helium knife cryoablation plus programmed cell death protein 1 inhibitor in the treatment of advanced soft tissue sarcomas: there is no evidence of the synergistic effects of this combination therapy

BackgroundEffective treatment for advanced soft tissue sarcomas (STSs) is necessary for improved outcomes. Previous studies have suggested that cryoablation can have a synergistic effect with programmed cell death protein-1 (PD-1) inhibitor in the treatment of malignancy. This study aimed to clarify the efficacy and safety of argon-helium knife cryoablation in combination with PD-1 inhibitor in the treatment of STSs.MethodsRetrospectively collected and analyzed the clinical data of patients with advanced STS who underwent cryoablation and PD-1 inhibitor between March 2018 and December 2021.ResultsThis study included 27 patients with advanced STS. In terms of target lesions treated with cryoablation, 1 patient achieved complete response, 15 patients had partial response (PR), 10 patients had stable disease, and 1 patient had progressive disease. This corresponded to an overall response rate of 59.3% and a disease control rate of 96.3%. In terms of distant target lesions untreated with cryoablation, only two patients had a PR compared to the diameter of the lesion before ablation. The combination therapy was relatively well tolerated. None of the patients experienced treatment-related death or delayed treatment due to adverse events.ConclusionCryoablation combined with PD-1 inhibitors in the therapy of advanced STS is safe and can effectively shrink the cryoablation-target lesion. However, there is no evidence of the synergistic effects of this combination therapy

Frozen inactivated autograft replantation for bone and soft tissue sarcomas

BackgroundThe frozen inactivation of autologous tumor bones using liquid nitrogen is an important surgical method for limb salvage in patients with sarcoma. At present, there are few research reports related to frozen inactivated autograft replantation.MethodsIn this study, we retrospectively collected the clinical data of patients with bone and soft tissue sarcoma treated with liquid nitrogen-frozen inactivated tumor bone replantation, and analyzed the safety and efficacy of this surgical method. The healing status of the frozen inactivated autografts was evaluated using the International Society of Limb Salvage (ISOLS) scoring system. Functional status of patients was assessed using the Musculoskeletal Tumor Society (MSTS) scale.ResultsThis study included 43 patients. The average length of the bone defect after tumor resection is 16.9 cm (range 6.3–35.3 cm). Patients with autograft not including the knee joint surface had significantly better healing outcomes (ISOLS scores) (80.6% ± 15% vs 28.2% ± 4.9%, P<0.001) and limb function (MSTS score) (87% ± 11.6% vs 27.2% ± 4.4%, P<0.001) than patients with autografts including the knee joint surface. The healing time of the end of inactivated autografts near the metaphyseal was significantly shorter than that of the end far away from the metaphyseal (9.8 ± 6.3 months vs 14.9 ± 6.3 months, P=0.0149). One patient had local recurrence, one had an autograft infection, five (all of whom had an autograft including the knee joint surface) had joint deformities, and seven had bone non-union.ConclusionFrozen inactivated autologous tumor bone replantation is safe and results in good bone healing. But this method is not suitable for patients with autograft involving the knee joint surface

Rewiring Neuronal Glycerolipid Metabolism Determines the Extent of Axon Regeneration

How adult neurons coordinate lipid metabolism to regenerate axons remains elusive. We found that depleting neuronal lipin1, a key enzyme controlling the balanced synthesis of glycerolipids through the glycerol phosphate pathway, enhanced axon regeneration after optic nerve injury. Axotomy elevated lipin1 in retinal ganglion cells, which contributed to regeneration failure in the CNS by favorably producing triglyceride (TG) storage lipids rather than phospholipid (PL) membrane lipids in neurons. Regrowth induced by lipin1 depletion required TG hydrolysis and PL synthesis. Decreasing TG synthesis by deleting neuronal diglyceride acyltransferases (DGATs) and enhancing PL synthesis through the Kennedy pathway promoted axon regeneration. In addition, peripheral neurons adopted this mechanism for their spontaneous axon regeneration. Our study reveals a critical role of lipin1 and DGATs as intrinsic regulators of glycerolipid metabolism in neurons and indicates that directing neuronal lipid synthesis away from TG synthesis and toward PL synthesis may promote axon regeneration.This study was supported by grants from the Hong Kong Research Grant Council (AoE/M-09/12, AoE/M-604/16, C6004-17G, 16103315, 16149316, and 16102519 to K.L. and C5031-14E to Z.Y.); Innovation and Technology Commission (ITCPD/17-9) of Hong Kong SAR; National Natural Science Foundation of China (81671214); Shenzhen Knowledge Innovation Program (JCYJ20160428145818099 and JCYJ20160427185601855); Guangdong Provincial Key S&T Program (2018B030336001); Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions; Nan Fung Group; and Wellcome Trust Seed award (108042 to S.S.). K.L. is Cheng Associate Professor of Science

Efficacy and safety of sintilimab plus doxorubicin in advanced soft tissue sarcoma: A single-arm, phase II trial

Background: Chemoimmunotherapy is safe and efficacious in treating many types of malignant tumors. However, clinical data demonstrating the effect of this combination treatment in patients with metastatic soft tissue sarcoma (STS) are currently limited. This study evaluated the safety and efficacy of a programmed cell death protein 1 (PD-1) inhibitor plus doxorubicin in patients with advanced STS who failed previous systemic therapy.Methods: This was a single-center, single-arm, open-label phase II trial. Patients with unresectable or metastatic STS who had previously failed systemic therapy were enrolled. Patients received up to six cycles of doxorubicin and sintilimab (a PD-1 inhibitor), while sintilimab treatment continued for up to 2 years. Primary outcomes were objective response rate (ORR) and safety. Univariate Cox proportional hazards model was used to analyze the relationship between clinicopathological parameters and progression-free survival (PFS).Results: A total of 38 patients (20 men and 18 women) were enrolled in this study. The overall ORR was 39.5%, disease control rate was 71.1%, and the median PFS was 4.5 months [95% confidence interval (CI), 3.0–8.5 months]. The adverse events (AEs) associated with the combined treatment were mild, manageable, and well-tolerated. The most common grade 3 or higher AEs were hematologic, including leukopenia (21.1%), anemia (18.4%), and thrombocytopenia (18.4%). Patients with undifferentiated pleomorphic sarcoma (UPS) or dedifferentiated liposarcoma had a significantly longer PFS than those with other pathological subtypes [hazard ratio (HR) = 0.42, 95% CI 0.21–0.83; p = 0.013]. There was no significant difference in the median PFS between patients who had previously received anthracycline-based chemotherapy and those who had not (HR = 0.74, 95% CI 0.34–1.58, p = 0.43).Conclusion: Sintilimab plus doxorubicin is a safe and promising treatment for patients with advanced STS who have failed previous systemic therapy (including anthracycline-based chemotherapy). The efficacy of this combination therapy in UPS and dedifferentiated liposarcoma is superior to that in other sarcomas.Clinical Trial Registration:https://www.chictr.org.cn, registration number: ChiCTR1900027009

Metabolomics combined with transcriptomics analyses of mechanism regulating testa pigmentation in peanut

Peanut testa (seed coat) contains large amounts of flavonoids that significantly influence seed color, taste, and nutritional qualities. There are various colors of peanut testa, however, their precise flavonoid components and regulatory mechanism of pigmentation remain unclear. In this study, a total of 133 flavonoids were identified and absolutely quantified in the seed coat of four peanut cultivars with different testa color using a widely targeted metabolomic approach. Black peanut skin had more types and substantial higher levels of cyanidin-based anthocyanins, which possibly contribute to its testa coloration. Procyanidins and flavan-3-ols were the major co-pigmented flavonoids in the red, spot and black peanuts, while flavanols were the most abundant constitutes in white cultivar. Although the concentrations as well as composition characteristics varied, the content ratios of procyanidins to flavan-3-ols were similar in all samples except for white peanut. Furthermore, MYB-like transcription factors, anthocyanidin reductases (ANR), and UDP-glycosyltransferases (UGT) were found to be candidate genes involved in testa pigmentation via RNA-seq and weighted gene co-expression network analysis. It is proposed that UGTs and ANR compete for the substrate cyanidin and the prevalence of UGTs activities over ANR one will determine the color pattern of peanut testa. Our results provide a comprehensive report examining the absolute abundance of flavonoid profiles in peanut seed coat, and the finding are expected to be useful for further understanding of regulation mechanisms of seed coat pigmentation in peanut and other crops

Modeling, stability analysis and advanced control of cyber-physical microgrid systems

Microgrids (MGs) are becoming vital parts of the future smart grid with the higher penetration of renewable energy sources. Compared with the traditional power system, MGs have the advantages such as being eco-friendly, flexible and enhancing grid resilience. For the islanded MGs, a typical hierarchical control framework including primary, secondary, and tertiary control is commonly implemented. The droop-based primary control ensures the MG system's rapid stability. The secondary control aims to eliminate the frequency and voltage deviation. The tertiary control is to achieve economic dispatch. Besides, considering a single MG has a limited generation capacity and specific geographical boundaries, the networked-microgrid (NMG) system is also proposed by interconnecting multiple MGs to enhance its resilience against extreme events. MG/NMG is a cyber-physical system with coupled electrical and cyber networks, such a system could involve new stability issues due to the intrinsic characteristics and non-ideal cyber networks, such as low-inertia characteristics, time delays, and cyber-attacks. Hence, the modeling, stability analysis and advanced control methods for cyber-physical MG/NMG systems need to be in-depth investigated. This thesis aims to solve the above-mentioned stability issues, and presents integrated modeling, analysis, and advanced control for MG/NMG systems. The focus is mainly on the following aspects: small-signal modeling of MG and NMG system, advanced control method to enhance the time-delay margin for MG system, cyber-resilient control against cyber-attacks for MG system, layered distributed control framework, stability analysis and time-delay compensation control for NMG system. The detailed descriptions will be respectively provided hereinafter. The time delay in MG control will significantly impact the system's stability. To enhance the time-delay margin for MG system, a weight-average-prediction (WAP) controller is first designed to compensate for the delayed system state. By introducing a time-delayed differential term in the designed control law, the traditional time-delayed small-signal model is transformed into a neutral time-delayed mathematic model. Based on the developed model, the stability analysis is conducted considering both fixed-time and time-varying delays. For the former, a novel graphic analytical method is proposed to evaluate the time delay margin, which eliminates the conservatism compared with existing time-domain methods. For the latter, the stability condition is established by a Lyapunov-Krasovskii function and linear matrix inequalities. In addition, some non-linear WAP control methods are discussed to guide the parameter tuning with a higher resolution. Lastly, the designed method and analytical result are verified in the OPAL-RT real-time test platform. Cyber-attack is another stability issue for cyber-physical MG systems. Considering the different attack intensities for data availability, an attacker can choose denial-of-service (DoS) attacks or latency attacks to impact the stability of MG system. In this thesis, the above two kinds of attack consequences including network jamming and time-varying latency in the communication network are simultaneously studied. Firstly, a metric is defined to quantify the DoS attacks. Then, the time-domain stability study is conducted. Next, a cyber-resilient control strategy is designed with two control modes: (i) An adaptive-gain resilient controller to sustain the fast stabilization of MG systems under the non-uniform time-varying latency attack. (ii) An event-trigger topology reconfiguration controller against excessive latency and damaged cyber connectivity caused by DoS attacks. A switching mechanism for coordinating the above control modes is also designed. The effectiveness of the designed controller under different attack scenarios is verified by OPAL-RT real-time tests. NMG system, as a specific MG system, has a much more complex structure with larger control difficulties and complicated dynamic behaviors. To address this issue, a layered distributed control framework for NMG system is designed in this thesis, which is comprised of two layers: the individual MG control layer and NMG control layer. The NMG control layer generates the voltage/frequency reference for the MG control layer. The MG layer controls each DG to ensure system stability. Besides, a small-signal model for a generalized single-bus NMG system is derived to analyze the impact of coupling among MGs and various parameters on NMG dynamic stability. It reveals that the coupling relationship among MGs will weaken the stability of the whole NMG system while the proper control parameters can enhance the stability. Lastly, the designed control framework and the analytical result are verified by time-domain simulations. The above-designed distributed control framework in NMG systems relies on the information exchange among distributed generators and sub-MGs, which inevitably contain heterogenous time delays that can deteriorate system stability. Hence, the multiple time-delays small-signal stability analysis of an NMG with the above-designed distributed layered control architecture is further studied, and a lead-lag compensation controller is designed. Firstly, a generalized time-delayed DC NMG small-signal model is developed. Then, stability analysis is conducted considering multiple time delays from communication and measurement stages in NMG layer and MG layer. The impact of multiple time delays on NMG stability is quantified. It reveals that both the electrical coupling among MGs and different kinds of time delays in distributed control loops will impact the NMG stability. Subsequently, based on the detailed analytical results, the critical eigenvalue is identified, and a lead-lag compensation controller is designed to enhance the system stability by compensating for the phase lag of the critical eigenvalue. Lastly, the effectiveness of the designed method and the accuracy of the analytical findings are verified by OPAL-RT-based real-time tests. In summary, the overall research problems of this thesis focus on modeling, stability analysis, and advanced control of cyber-physical MG/NMG systems. Various stability issues including time delays, cyber-attacks, and low-inertia characteristics are separately studied. The relevant stability characteristics are analyzed and revealed in this thesis. Based on the findings, the novel control framework and advanced controllers are separately designed to enhance stability, compensate for time-delayed or attacked effects, and achieve multiple control objectives in MG/NMG system.Doctor of Philosoph

Distributed energy storage system for power system control

Due to the intermittent nature of solar PV, the increasing deployment of solar energy may cause a series of issues such as frequency fluctuation and active power unbalance. Distributed energy storage systems (DESSs) could be a feasible solution for the mitigation of large power fluctuation by acting as “power buffer” to absorb and release power. This dissertation based on Singapore-Malaysia power system model analyzes DESSs’ influence in frequency regulation considering PV’s penetration. Firstly, a comprehensive power system which consists of modern power grid, PV power plant, energy storage system and baseload is modeled. Then based on this comprehensive power system, from load’s perspective, it provides an effectively mathematic approach to analyze load forecast based on MATLAB software tool. In addition, three frequency control algorithms which contain ramp rate control, local frequency regulation and centralized frequency regulation are provided to prove the effect of DESSs. Finally, this dissertation considers different latency time, capacity and power rating to give some advice on how to optimize frequency control algorithms.Master of Science (Power Engineering