DynamicISP: Dynamically Controlled Image Signal Processor for Image Recognition

Image signal processor (ISP) plays an important role not only for human perceptual quality but also for computer vision. In most cases, experts resort to manual tuning of many parameters in the ISPs for perceptual quality. It failed in sub-optimal, especially for computer vision. Aiming to improve ISPs, two approaches have been actively proposed; tuning the parameters with machine learning, or constructing an ISP with DNN. The former is lightweight but lacks expressive powers. The latter has expressive powers but it was too heavy to calculate on edge devices. To this end, we propose DynamicISP, which consists of traditional simple ISP functions but their parameters are controlled dynamically per image according to what the downstream image recognition model felt to the previous frame. Our proposed method successfully controlled parameters of multiple ISP functions and got state-of-the-art accuracy with a small computational cost

Rawgment: Noise-Accounted RAW Augmentation Enables Recognition in a Wide Variety of Environments

Image recognition models that work in challenging environments (e.g., extremely dark, blurry, or high dynamic range conditions) must be useful. However, creating training datasets for such environments is expensive and hard due to the difficulties of data collection and annotation. It is desirable if we could get a robust model without the need for hard-to-obtain datasets. One simple approach is to apply data augmentation such as color jitter and blur to standard RGB (sRGB) images in simple scenes. Unfortunately, this approach struggles to yield realistic images in terms of pixel intensity and noise distribution due to not considering the non-linearity of Image Signal Processors (ISPs) and noise characteristics of image sensors. Instead, we propose a noise-accounted RAW image augmentation method. In essence, color jitter and blur augmentation are applied to a RAW image before applying non-linear ISP, resulting in realistic intensity. Furthermore, we introduce a noise amount alignment method that calibrates the domain gap in the noise property caused by the augmentation. We show that our proposed noise-accounted RAW augmentation method doubles the image recognition accuracy in challenging environments only with simple training data.Comment: Accepted to CVPR202

Diarylethene Isomerization by Using Triplet–Triplet Annihilation Photon Upconversion

Green-to-blue triplet–triplet annihilation photon upconversion with the well-studied upconversion pair 9,10-diphenylanthracene (DPA)/platinum octaethylporphyrin (PtOEP) was used to reversibly drive the photoisomerization of diarylethene (DAE) photoswitches by using visible light. By carefully selecting the kinetic and spectral properties of the molecular system as well as the experimental geometry, a single green light source can be used to selectively trigger both the ring-opening and the ring-closing reactions, whilst also inducing fluorescence from the colored closed isomer that can be used as a readout to monitor the isomerization process in situ. The upconversion solution and the DAE solution are kept physically separated, allowing them to be characterized both concomitantly and individually without further separation processes. The ring-closing reaction using upconverted photons was quantified and compared to the efficiency of direct isomerization with ultraviolet light

Near-infrared two-photon absorption and excited state dynamics of a fluorescent diarylethene derivative

The version of record of this article, first published in Photochemical and Photobiological Sciences, is available online at Publisher’s website: https://doi.org/10.1007/s43630-024-00573-y.Abstract: Near-infrared two-photon absorption and excited state dynamics of a fluorescent diarylethene (fDAE) derivative were investigated by time-resolved absorption and fluorescence spectroscopies. Prescreening with quantum chemical calculation predicted that a derivative with methylthienyl groups (mt-fDAE) in the closed-ring isomer has a two-photon absorption cross-section larger than 1000 GM, which was experimentally verified by Z-scan measurements and excitation power dependence in transient absorption. Comparison of transient absorption spectra under one-photon and simultaneous two-photon excitation conditions revealed that the closed-ring isomer of mt-fDAE populated into higher excited states deactivates following three pathways on a timescale of ca. 200 fs: (i) the cycloreversion reaction more efficient than that by the one-photon process, (ii) internal conversion into the S1 state, and (iii) relaxation into a lower state (S1’ state) different from the S1 state. Time-resolved fluorescence measurements demonstrated that this S1’ state is relaxed to the S1 state with the large emission probability. These findings obtained in the present work contribute to extension of the ON–OFF switching capability of fDAE to the biological window and application to super-resolution fluorescence imaging in a two-photon manner

Emission color tuning and white-light generation based on photochromic control of energy transfer reactions in polymer micelles

We encapsulate a fluorescent donor molecule and a photochromic acceptor unit (photoswitch) in polymer micelles and show that the color of the emitted fluorescence is continuously changed from blue to yellow upon light-induced isomerization of the acceptor. Interestingly, white-light generation is achieved in between. With the photoswitch in the colorless form, intense blue emission from the donor is observed, while UV-induced isomerization to the colored form induces an energy transfer reaction that quenches the donor emission and sensitizes the yellow emission from the colored photoswitch. The process is reversed by exposure to visible light, triggering isomerization to the colorless form

The New Type of Lectures on the Development of Science Teaching Plans and Materials Tried in the Department ofEducation

岡山大学教育学部の理科教育講座に所属する3年生を対象に,理科教材･授業案開発を指向した新しい講義を始めたのでその報告を行う｡本講義では,近隣の小,中学校や公民館,科学館等の協力も得て実践の場を確保し,開発した教材や授業案を実践することで生きた教材開発の訓練を行なうことを目指している｡教科内容学担当教員と教科教育学担当教員の協働を模索すると同時に,大学教員の教科内容の専門知識や技量を教材や授業案に有効に活用できる講義を意識した。This is a report of the new type of lectures on the development of teaching plans and teaching materials attempted in the science class for 3rd grade students in the Department of Education Okayama Universlty. The students practiced the development of teaching plans and teaching materials, and they conducted their activities not only at university but also at the neighboring schools, the public hall, the science museum etc. We expected the development of further collaboration in the academic staffs in the science education course, and the science abilities will be helpful for the development of teaching plans and teaching materials

An all-photonic full color RGB system based on molecular photoswitches

On-command changes in the emission color of functional materials is a sought-after property in many contexts. Of particular interest are systems using light as the external trigger to induce the color changes. Here we report on a tri-component cocktail consisting of a fluorescent donor molecule and two photochromic acceptor molecules encapsulated in polymer micelles and we show that the color of the emitted fluorescence can be continuously changed from blue-to-green and from blue-to-red upon selective light-induced isomerization of the photochromic acceptors to the fluorescent forms. Interestingly, isomerization of both acceptors to different degrees allows for the generation of all emission colors within the red-green-blue (RGB) color system. The function relies on orthogonally controlled FRET reactions between the blue emitting donor and the green and red emitting acceptors, respectively

PD-1 blockade therapy promotes infiltration of tumor-attacking exhausted T cell clonotypes

PD-1 blockade exerts clinical efficacy against various types of cancer by reinvigorating T cells that directly attack tumor cells (tumor-specific T cells) in the tumor microenvironment (TME), and tumor-infiltrating lymphocytes (TILs) also comprise nonspecific bystander T cells. Here, using single-cell sequencing, we show that TILs include skewed T cell clonotypes, which are characterized by exhaustion (T-ex) or nonexhaustion signatures (Tnon-ex). Among skewed clonotypes, those in the T-ex, but not those in the Tnon-ex, cluster respond to autologous tumor cell lines. After PD-1 blockade, non-preexisting tumor-specific clonotypes in the T-ex cluster appear in the TME. Tumor-draining lymph nodes (TDLNs) without metastasis harbor a considerable number of such clonotypes, whereas these clonotypes are rarely detected in peripheral blood. We propose that tumor-infiltrating skewed T cell clonotypes with an exhausted phenotype directly attack tumor cells and that PD-1 blockade can promote infiltration of such T-ex clonotypes, mainly from TDLNs