101 research outputs found
Automated Testing and Improvement of Named Entity Recognition Systems
Named entity recognition (NER) systems have seen rapid progress in recent
years due to the development of deep neural networks. These systems are widely
used in various natural language processing applications, such as information
extraction, question answering, and sentiment analysis. However, the complexity
and intractability of deep neural networks can make NER systems unreliable in
certain circumstances, resulting in incorrect predictions. For example, NER
systems may misidentify female names as chemicals or fail to recognize the
names of minority groups, leading to user dissatisfaction. To tackle this
problem, we introduce TIN, a novel, widely applicable approach for
automatically testing and repairing various NER systems. The key idea for
automated testing is that the NER predictions of the same named entities under
similar contexts should be identical. The core idea for automated repairing is
that similar named entities should have the same NER prediction under the same
context. We use TIN to test two SOTA NER models and two commercial NER APIs,
i.e., Azure NER and AWS NER. We manually verify 784 of the suspicious issues
reported by TIN and find that 702 are erroneous issues, leading to high
precision (85.0%-93.4%) across four categories of NER errors: omission,
over-labeling, incorrect category, and range error. For automated repairing,
TIN achieves a high error reduction rate (26.8%-50.6%) over the four systems
under test, which successfully repairs 1,056 out of the 1,877 reported NER
errors.Comment: Accepted by ESEC/FSE'2
Changes and driving forces analysis of alpine wetlands in the first meander of the Yellow River based on long-term time series remote sensing data
IntroductionAs a vital component of the ecosystem of the Qinghai-Tibet Plateau, alpine wetlands coexist with their vulnerability, sensitivity, and abundant biodiversity, propelling the material cycle and energy flux of the entire plateau ecosystem. In recent decades, climate change and human activities have significantly altered the regional landscape. Monitoring and assessing changes in the alpine wetlands on the Qinghai-Tibet Plateau requires the efficient and accurate collection of long-term information.MethodsHere, we interpreted the remote sensing data of the first meander of the Yellow River of alpine wetlands from 1990 to 2020 based on Google Earth Engine (GEE) platform, using geographic information system (GIS) and landscape pattern index to analyze the spatial and temporal evolution of wetland landscape patterns, and the primary drivers of changes in wetland area were explored by GeoDetector.ResultsOur result showed that most wetland areas were found in regions with gradients less than 12° and elevations between 3315 and 3600 m. From 1990 to 2010, the area of alpine wetland in the study area decreased by 25.43%. During the period between 2010 and 2020 to the 1990s, the wetland area decreased by 322.9 km2. Conversion to and from grassland was the primary form of wetland transfer out and in, respectively. The overall migration of the wetland centroid in the study area was to the southwest between 1990 and 2010 and to the north between 2010 and 2020. The geometry of the wetland landscape was relatively simple, the landscape was relatively intact, and patches retained a high level of agglomeration and connectivity. However, their level of agglomeration and connectivity was disrupted. A quantitative analysis of the factor detector in GeoDetector revealed that the DEM, slope, and evaporation were the most important driving factors influencing the change of wetland area, with socioeconomic development also influencing changes in the wetland area to a lesser extent.DiscussionUsing interaction detectors, it was discovered that the interaction of various driving factors could better explain the long-term variations in wetland areas, with a greater degree of explanation than that of each driving factor alone
- …