Improving Retrieval-Augmented Large Language Models via Data Importance Learning

Retrieval augmentation enables large language models to take advantage of external knowledge, for example on tasks like question answering and data imputation. However, the performance of such retrieval-augmented models is limited by the data quality of their underlying retrieval corpus. In this paper, we propose an algorithm based on multilinear extension for evaluating the data importance of retrieved data points. There are exponentially many terms in the multilinear extension, and one key contribution of this paper is a polynomial time algorithm that computes exactly, given a retrieval-augmented model with an additive utility function and a validation set, the data importance of data points in the retrieval corpus using the multilinear extension of the model's utility function. We further proposed an even more efficient ({\epsilon}, {\delta})-approximation algorithm. Our experimental results illustrate that we can enhance the performance of large language models by only pruning or reweighting the retrieval corpus, without requiring further training. For some tasks, this even allows a small model (e.g., GPT-JT), augmented with a search engine API, to outperform GPT-3.5 (without retrieval augmentation). Moreover, we show that weights based on multilinear extension can be computed efficiently in practice (e.g., in less than ten minutes for a corpus with 100 million elements)

Ground-based remote sensing cloud detection using dual pyramid network and encoder–decoder constraint

Many methods for ground-based remote sensing cloud detection learn representation features using the encoder–decoder structure. However, they only consider the information from single scale, which leads to incomplete feature extraction. In this article, we propose a novel deep network named dual pyramid network (DPNet) for ground-based remote sensing cloud detection, which possesses an encoder–decoder structure with dual pyramid pooling module (DPPM). Specifically, we process the feature maps of different scales in the encoder through dual pyramid pooling. Then, we fuse the outputs of the dual pyramid pooling in the same pyramid level using the attention fusion. Furthermore, we propose the encoder–decoder constraint (EDC) to relieve information loss in the process of encoding and decoding. It constrains the values and the gradients of probability maps from the encoder and the decoder to be consistent. Since the number of cloud images in the publicly available databases for ground-based remote sensing cloud detection is limited, we release the TJNU Large-scale Cloud Detection Database (TLCDD) that is the largest database in this field. We conduct a series of experiments on TLCDD, and the experimental results verify the effectiveness of the proposed method

Integration transformer for ground-based cloud image segmentation

Recently, convolutional neural networks (CNNs) dominate the ground-based cloud image segmentation task, but disregard the learning of long-range dependencies due to the limited size of filters. Although Transformer-based methods could overcome this limitation, they only learn long-range dependencies at a single scale, hence failing to capture multiscale information of cloud images. The multiscale information is beneficial to ground-based cloud image segmentation, because the features from small scales tend to extract detailed information, while features from large scales have the ability to learn global information. In this article, we propose a novel deep network named Integration Transformer (InTransformer), which builds long-range dependencies from different scales. To this end, we propose the hybrid multihead transformer block (HMTB) to learn multiscale long-range dependencies and hybridize CNNs and HMTB as the encoder at different scales. The proposed InTransformer hybridizes CNNs and Transformer as the encoder to extract multiscale representations, which learns both local information and long-range dependencies with different scales. Meanwhile, in order to fuse the patch tokens with different scales, we propose a mutual cross-attention module (MCAM) for the decoder of InTransformer which could adequately interact multiscale patch tokens in a bidirectional way. We have conducted a series of experiments on the large ground-based cloud detection database TJNU Large Scale Cloud Detection Database (TLCDD) and Singapore Whole sky IMaging SEGmentation Database (SWIMSEG). The experimental results show that the performance of our method outperforms other methods, proving the effectiveness of the proposed InTransformer

Penyelesaian Tindak Pidana Perjudian yang Dilakukan oleh Anak Menurut UU No.11 Tahun 2012

The title of this legal writing is "The Completion of the Crime of Gambling Carried Out by minors based on the law Number 11 of 2012 on the Juvenile Justice system". This type of research is normative legal research. Normative legal research is a research conducted or focusing on norm of positive law in the form of legislation. Legal issues raised is whether the completion of the crime of gambling by children is in conformity with the law Number 11 of 2012 about the juvenile justice system. The purpose of this research is to determine and analyze the completion of the crime of gambling by children under the law of the juvenile justice system. The result showed that the efforts made to prevent criminal acts of a child is an attempt preventive and repressive efforts. Juvenile justice system is closely related to restorative justice. Regarding the obligation to make a diversion conducted by law enforcement officials, in particular under Article 7 and 96 of the law number 11 of 2012 on the Juvenile Justice System

Grading of MRI–detected skull-base invasion in nasopharyngeal carcinoma with skull-base invasion after intensity-modulated radiotherapy

Abstract Background The aim of this study is to evaluate the prognostic value of grading MRI–detected skull-base invasion in nasopharyngeal carcinoma (NPC) with skull-base invasion after intensity-modulated radiotherapy (IMRT). Methods This study is a retrospective chart review of 469 non-metastatic NPC patients with skull-base invasion. Patients were classified as extensive skull-base invasion (ESBI) group and limited skull-base invasion (LSBI) group. Results Multivariate analysis showed that the skull-base invasion (LSBI vs. ESBI) was an independent prognostic predictor of progression free survival (PFS). The estimated 5-year local failure free survival (LFFS), distant metastasis free survival (DMFS), PFS, and overall survival (OS) rates for patients in the T3-LSBI and T3-ESBI group were 92.9% versus 93.5, 89.8% versus 86.1, 81.6% versus 76.4, and 93.5% versus 86.3%, respectively (P > 0.05). Conclusion Grading of MRI-detected skull-base invasion is an independent prognostic factor of NPC with skull-base invasion. It is scientific and reasonable for skull-base invasion as a single entity to be classified as T3 classification

Cross-Domain Ground-Based Cloud Classification Based on Transfer of Local Features and Discriminative Metric Learning

Cross-domain ground-based cloud classification is a challenging issue as the appearance of cloud images from different cloud databases possesses extreme variations. Two fundamental problems which are essential for cross-domain ground-based cloud classification are feature representation and similarity measurement. In this paper, we propose an effective feature representation called transfer of local features (TLF), and measurement method called discriminative metric learning (DML). The TLF is a generalized representation framework that can integrate various kinds of local features, e.g., local binary patterns (LBP), local ternary patterns (LTP) and completed LBP (CLBP). In order to handle domain shift, such as variations of illumination, image resolution, capturing location, occlusion and so on, the TLF mines the maximum response in regions to make a stable representation for domain variations. We also propose to learn a discriminant metric, simultaneously. We make use of sample pairs and the relationship among cloud classes to learn the distance metric. Furthermore, in order to improve the practicability of the proposed method, we replace the original cloud images with the convolutional activation maps which are then applied to TLF and DML. The proposed method has been validated on three cloud databases which are collected in China alone, provided by Chinese Academy of Meteorological Sciences (CAMS), Meteorological Observation Centre (MOC), and Institute of Atmospheric Physics (IAP). The classification accuracies outperform the state-of-the-art methods

Photovoltaic Array Fault Diagnosis Based on Gaussian Kernel Fuzzy C-Means Clustering Algorithm

In the fault diagnosis process of a photovoltaic (PV) array, it is difficult to discriminate single faults and compound faults with similar signatures. Furthermore, the data collected in the actual field experiment also contains strong noise, which leads to the decline of diagnostic accuracy. In order to solve these problems, a new eigenvector composed of the normalized PV voltage, the normalized PV current and the fill factor is constructed and proposed to characterize the common faults, such as open circuit, short circuit and compound faults in the PV array. The combination of these three feature characteristics can reduce the interference of external meteorological conditions in the fault identification. In order to obtain the new eigenvectors, a multi-sensory system for fault diagnosis in a PV array, combined with a data-mining solution for the classification of the operational state of the PV array, is needed. The selected sensors are temperature sensors, irradiance sensors, voltage sensors and current sensors. Taking account of the complexity of the fault data in the PV array, the Kernel Fuzzy C-means clustering method is adopted to identify these fault types. Gaussian Kernel Fuzzy C-means clustering method (GKFCM) shows good clustering performance for classifying the complex datasets, thus the classification accuracy can be effectively improved in the recognition process. This algorithm is divided into the training and testing phases. In the training phase, the feature vectors of 8 different fault types are clustered to obtain the training core points. According to the minimum Euclidean Distances between the training core points and new fault data, the new fault datasets can be identified into the corresponding classes in the fault classification stage. This strategy can not only diagnose single faults, but also identify compound fault conditions. Finally, the simulation and field experiment demonstrated that the algorithm can effectively diagnose the 8 common faults in photovoltaic arrays

Multi-View Ground-Based Cloud Recognition by Transferring Deep Visual Information

Since cloud images captured from different views possess extreme variations, multi-view ground-based cloud recognition is a very challenging task. In this paper, a study of view shift is presented in this field. We focus both on designing proper feature representation and learning distance metrics from sample pairs. Correspondingly, we propose transfer deep local binary patterns (TDLBP) and weighted metric learning (WML). On one hand, to deal with view shift, like variations of illuminations, locations, resolutions and occlusions, we first utilize cloud images to train a convolutional neural network (CNN), and then extract local features from the part summing maps (PSMs) based on feature maps. Finally, we maximize the occurrences of regions for the final feature representation. On the other hand, the number of cloud images in each category varies greatly, leading to the unbalanced similar pairs. Hence, we propose a weighted strategy for metric learning. We validate the proposed method on three cloud datasets (the MOC_e, IAP_e, and CAMS_e) that are collected by different meteorological organizations in China, and the experimental results show the effectiveness of the proposed method