3D Hand Pose Estimation from Single RGB Images with Auxiliary Information

3D hand pose estimation from monocular RGB inputs is critical for augmented and virtual reality applications, and has achieved remarkable progress due to the revolution of deep learning. Existing deep-learning-based hand pose estimation systems target learning good representations for hand poses, requiring a large amount of accurate ground truth labels, which are difficult to obtain. We turn to explore different auxiliary information to aid representation learning and reduce the reliance on data annotation. This dissertation explores different auxiliary information, i.e., image factors, multi-modal data, and synthetic data, for 3D hand pose estimation. Motivated by the image rendering that requires a number of image factors of variation, we propose to learn disentangled representations to better analyze these factors of variation. The disentangled representations enable explicit control over different factors of variation for synthesizing hand images and training with hand factors as weak labels for hand pose estimation. Besides labelled or shared hand factors, different modalities (e.g., RGB images and depth maps) of the same hand should have shared information. Therefore, we present multi-modalities as auxiliary information for RGB inputs. Specifically, we explore multi-modal alignment in three aspects: latent space alignment based on variational autoencoder and product of Gaussian expert, pixel-level alignment via attention fusion, and low-dimensional subspace alignment via contrastive learning. Besides multi-modal alignment, the auxiliary modalities can also serve as weak labels for hand pose estimation. To further remove the requirements of image factors or different modalities, we emphasize the importance of synthetic data. Synthetic data is flexible, infinite, and easy to achieve. With synthetic data as auxiliary information, we can significantly reduce the number of labelled real-world data. Therefore, we introduce a challenging scenario that learns only from labelled synthetic data and fully unlabelled real-world data. To address this challenging scenario, we present a semi-supervised framework with pseudo-labelling and consistency training, and try to address noisy pseudo-labels using modules like label correction and self-distillation. This dissertation advances the state-of-the-art 3D hand pose estimation, explores representation learning, weakly- and semi-supervised learning for pose estimation, and paves a path forward for learning pose estimation with diverse auxiliary information

Spore photoproduct lyase: the known, the controversial, and the unknown

Spore photoproduct lyase (SPL) repairs 5-thyminyl-5,6-dihydrothymine, a thymine dimer that is also called the spore photoproduct (SP), in germinating endospores. SPL is a radical S-adenosylmethionine (SAM) enzyme, utilizing the 5′-deoxyadenosyl radical generated by SAM reductive cleavage reaction to revert SP to two thymine residues. Here we review the current progress in SPL mechanistic studies. Protein radicals are known to be involved in SPL catalysis; however, how these radicals are quenched to close the catalytic cycle is under debate

Insights into the Activity Change of Spore Photoproduct Lyase Induced by Mutations at a Peripheral Glycine Residue

UV radiation triggers the formation of 5-thyminyl-5,6-dihydrothymine, i.e., the spore photoproduct (SP), in the genomic DNA of bacterial endospores. These SPs, if not repaired in time, may lead to genome instability and cell death. SP is mainly repaired by spore photoproduct lyase (SPL) during spore outgrowth via an unprecedented protein-harbored radical transfer pathway that is composed of at least a cysteine and two tyrosine residues. This mechanism is consistent with the recently solved SPL structure that shows all three residues are located in proximity and thus able to participate in the radical transfer process during the enzyme catalysis. In contrast, an earlier in vivo mutational study identified a glycine to arginine mutation at the position 168 on the B. subtilis SPL that is >15 Å away from the enzyme active site. This mutation appears to abolish the enzyme activity because endospores carrying this mutant were sensitive to UV light. To understand the molecular basis for this rendered enzyme activity, we constructed two SPL mutations G168A and G168R, examined their repair of dinucleotide SP TpT, and found that both mutants exhibit reduced enzyme activity. Comparing with the wildtype (WT) SPL enzyme, the G168A mutant slows down the SP TpT repair by 3~4-fold while the G168R mutant by ~ 80-fold. Both mutants exhibit a smaller apparent (DV) kinetic isotope effect (KIE) but a bigger competitive (DV/K) KIE than that by the WT SPL. Moreover, the G168R mutant also produces a large portion of the abortive repair product TpT-[Formula: see text]; the formation of which indicates that cysteine 141 is no longer well positioned as the H-donor to the thymine allylic radical intermediate. All these data imply that the mutation at the remote glycine 168 residue alters the enzyme 3D structure, subsequently reducing the SPL activity by changing the positions of the essential amino acids involved in the radical transfer process

Optimal ALOHA-like random access with heterogeneous QoS guarantees for multi-packet reception aided visible light communications

There is a paucity of random access protocols designed for alleviating collisions in visible light communication (VLC) systems where carrier sensing is hard to be achieved due to the directionality of light. To resolve the problem of collisions, we adopt the successive interference cancellation (SIC) algorithm to enable the coordinator to simultaneously communicate with multiple devices, which is referred to as the multi-packet reception (MPR) capability. However, the MPR capability could be fully utilized only when random access algorithms are accordingly designed. Considering the characteristics of the random access VLC system with SIC, we propose a novel effective capacity (EC)-based ALOHA-like random access algorithm for MPR-aided uplink VLC systems having heterogeneous quality-of-service (QoS) guarantees. Firstly, we model the VLC network as a conflict graph and derive the EC for each device. Then, we formulate the VLC QoS-driven random access problem as a saturation throughput maximization problem subject to multiple statistical QoS constraints. Finally, the resultant non-concave optimization problem (OP) is solved by a memetic search algorithm relying on invasive weed optimization and differential evolution (IWO-DE). We demonstrate that our derived EC expression matches the Monte Carlo simulation results accurately, and the performance of our proposed algorithm is competitive

Three-dimensional structure of the milky way dust: modeling of LAMOST data

We present a three-dimensional modeling of the Milky Way dust distribution by fitting the value-added star catalog of LAMOST spectral survey. The global dust distribution can be described by an exponential disk with scale-length of 3,192 pc and scale height of 103 pc. In this modeling, the Sun is located above the dust disk with a vertical distance of 23 pc. Besides the global smooth structure, two substructures around the solar position are also identified. The one located at $150^{\circ}<l<200^{\circ}$ and $-5^{\circ}<b<-30^{\circ}$ is consistent with the Gould Belt model of \citet{Gontcharov2009}, and the other one located at $140^{\circ}<l<165^{\circ}$ and $0^{\circ}<b<15^{\circ}$ is associated with the Camelopardalis molecular clouds.Comment: 15 pages, 6 figure, accepted by Ap