29 research outputs found
Improving utility of brain tumor confocal laser endomicroscopy: objective value assessment and diagnostic frame detection with convolutional neural networks
Confocal laser endomicroscopy (CLE), although capable of obtaining images at
cellular resolution during surgery of brain tumors in real time, creates as
many non-diagnostic as diagnostic images. Non-useful images are often distorted
due to relative motion between probe and brain or blood artifacts. Many images,
however, simply lack diagnostic features immediately informative to the
physician. Examining all the hundreds or thousands of images from a single case
to discriminate diagnostic images from nondiagnostic ones can be tedious.
Providing a real-time diagnostic value assessment of images (fast enough to be
used during the surgical acquisition process and accurate enough for the
pathologist to rely on) to automatically detect diagnostic frames would
streamline the analysis of images and filter useful images for the
pathologist/surgeon. We sought to automatically classify images as diagnostic
or non-diagnostic. AlexNet, a deep-learning architecture, was used in a 4-fold
cross validation manner. Our dataset includes 16,795 images (8572 nondiagnostic
and 8223 diagnostic) from 74 CLE-aided brain tumor surgery patients. The ground
truth for all the images is provided by the pathologist. Average model accuracy
on test data was 91% overall (90.79 % accuracy, 90.94 % sensitivity and 90.87 %
specificity). To evaluate the model reliability we also performed receiver
operating characteristic (ROC) analysis yielding 0.958 average for the area
under ROC curve (AUC). These results demonstrate that a deeply trained AlexNet
network can achieve a model that reliably and quickly recognizes diagnostic CLE
images.Comment: SPIE Medical Imaging: Computer-Aided Diagnosis 201
BIQ2021: A Large-Scale Blind Image Quality Assessment Database
The assessment of the perceptual quality of digital images is becoming
increasingly important as a result of the widespread use of digital multimedia
devices. Smartphones and high-speed internet are just two examples of
technologies that have multiplied the amount of multimedia content available.
Thus, obtaining a representative dataset, which is required for objective
quality assessment training, is a significant challenge. The Blind Image
Quality Assessment Database, BIQ2021, is presented in this article. By
selecting images with naturally occurring distortions and reliable labeling,
the dataset addresses the challenge of obtaining representative images for
no-reference image quality assessment. The dataset consists of three sets of
images: those taken without the intention of using them for image quality
assessment, those taken with intentionally introduced natural distortions, and
those taken from an open-source image-sharing platform. It is attempted to
maintain a diverse collection of images from various devices, containing a
variety of different types of objects and varying degrees of foreground and
background information. To obtain reliable scores, these images are
subjectively scored in a laboratory environment using a single stimulus method.
The database contains information about subjective scoring, human subject
statistics, and the standard deviation of each image. The dataset's Mean
Opinion Scores (MOS) make it useful for assessing visual quality. Additionally,
the proposed database is used to evaluate existing blind image quality
assessment approaches, and the scores are analyzed using Pearson and Spearman's
correlation coefficients. The image database and MOS are freely available for
use and benchmarking
Blind Quality Assessment for in-the-Wild Images via Hierarchical Feature Fusion and Iterative Mixed Database Training
Image quality assessment (IQA) is very important for both end-users and
service-providers since a high-quality image can significantly improve the
user's quality of experience (QoE) and also benefit lots of computer vision
algorithms. Most existing blind image quality assessment (BIQA) models were
developed for synthetically distorted images, however, they perform poorly on
in-the-wild images, which are widely existed in various practical applications.
In this paper, we propose a novel BIQA model for in-the-wild images by
addressing two critical problems in this field: how to learn better
quality-aware feature representation, and how to solve the problem of
insufficient training samples in terms of their content and distortion
diversity. Considering that perceptual visual quality is affected by both
low-level visual features (e.g. distortions) and high-level semantic
information (e.g. content), we first propose a staircase structure to
hierarchically integrate the features from intermediate layers into the final
feature representation, which enables the model to make full use of visual
information from low-level to high-level. Then an iterative mixed database
training (IMDT) strategy is proposed to train the BIQA model on multiple
databases simultaneously, so the model can benefit from the increase in both
training samples and image content and distortion diversity and can learn a
more general feature representation. Experimental results show that the
proposed model outperforms other state-of-the-art BIQA models on six
in-the-wild IQA databases by a large margin. Moreover, the proposed model shows
an excellent performance in the cross-database evaluation experiments, which
further demonstrates that the learned feature representation is robust to
images with diverse distortions and content. The code will be released publicly
for reproducible research
Kuvanlaatukokemuksen arvionnin instrumentit
This dissertation describes the instruments available for image quality evaluation, develops new methods for subjective image quality evaluation and provides image and video databases for the assessment and development of image quality assessment (IQA) algorithms. The contributions of the thesis are based on six original publications.
The first publication introduced the VQone toolbox for subjective image quality evaluation. It created a platform for free-form experimentation with standardized image quality methods and was the foundation for later studies.
The second publication focused on the dilemma of reference in subjective experiments by proposing a new method for image quality evaluation: the absolute category rating with dynamic reference (ACR-DR).
The third publication presented a database (CID2013) in which 480 images were evaluated by 188 observers using the ACR-DR method proposed in the prior publication. Providing databases of image files along with their quality ratings is essential in the field of IQA algorithm development.
The fourth publication introduced a video database (CVD2014) based on having 210 observers rate 234 video clips. The temporal aspect of the stimuli creates peculiar artifacts and degradations, as well as challenges to experimental design and video quality assessment (VQA) algorithms. When the CID2013 and CVD2014 databases were published, most state-of-the-art I/VQAs had been trained on and tested against databases created by degrading an original image or video with a single distortion at a time. The novel aspect of CID2013 and CVD2014 was that they consisted of multiple concurrent distortions.
To facilitate communication and understanding among professionals in various fields of image quality as well as among non-professionals, an attribute lexicon of image quality, the image quality wheel, was presented in the fifth publication of this thesis. Reference wheels and terminology lexicons have a long tradition in sensory evaluation contexts, such as taste experience studies, where they are used to facilitate communication among interested stakeholders; however, such an approach has not been common in visual experience domains, especially in studies on image quality.
The sixth publication examined how the free descriptions given by the observers influenced the ratings of the images. Understanding how various elements, such as perceived sharpness and naturalness, affect subjective image quality can help to understand the decision-making processes behind image quality evaluation. Knowing the impact of each preferential attribute can then be used for I/VQA algorithm development; certain I/VQA algorithms already incorporate low-level human visual system (HVS) models in their algorithms.Väitöskirja tarkastelee ja kehittää uusia kuvanlaadun arvioinnin menetelmiä, sekä tarjoaa kuva- ja videotietokantoja kuvanlaadun arviointialgoritmien (IQA) testaamiseen ja kehittämiseen. Se, mikä koetaan kauniina ja miellyttävänä, on psykologisesti kiinnostava kysymys. Työllä on myös merkitystä teollisuuteen kameroiden kuvanlaadun kehittämisessä. Väitöskirja sisältää kuusi julkaisua, joissa tarkastellaan aihetta eri näkökulmista.
I. julkaisussa kehitettiin sovellus keräämään ihmisten antamia arvioita esitetyistä kuvista tutkijoiden vapaaseen käyttöön. Se antoi mahdollisuuden testata standardoituja kuvanlaadun arviointiin kehitettyjä menetelmiä ja kehittää niiden pohjalta myös uusia menetelmiä luoden perustan myöhemmille tutkimuksille.
II. julkaisussa kehitettiin uusi kuvanlaadun arviointimenetelmä. Menetelmä hyödyntää sarjallista kuvien esitystapaa, jolla muodostettiin henkilöille mielikuva kuvien laatuvaihtelusta ennen varsinaista arviointia. Tämän todettiin vähentävän tulosten hajontaa ja erottelevan pienempiä kuvanlaatueroja.
III. julkaisussa kuvaillaan tietokanta, jossa on 188 henkilön 480 kuvasta antamat laatuarviot ja niihin liittyvät kuvatiedostot. Tietokannat ovat arvokas työkalu pyrittäessä kehittämään algoritmeja kuvanlaadun automaattiseen arvosteluun. Niitä tarvitaan mm. opetusmateriaalina tekoälyyn pohjautuvien algoritmien kehityksessä sekä vertailtaessa eri algoritmien suorituskykyä toisiinsa. Mitä paremmin algoritmin tuottama ennuste korreloi ihmisten antamiin laatuarvioihin, sen parempi suorituskyky sillä voidaan sanoa olevan.
IV. julkaisussa esitellään tietokanta, jossa on 210 henkilön 234 videoleikkeestä tekemät laatuarviot ja niihin liittyvät videotiedostot. Ajallisen ulottuvuuden vuoksi videoärsykkeiden virheet ovat erilaisia kuin kuvissa, mikä tuo omat haasteensa videoiden laatua arvioiville algoritmeille (VQA). Aikaisempien tietokantojen ärsykkeet on muodostettu esimerkiksi sumentamalla yksittäistä kuvaa asteittain, jolloin ne sisältävät vain yksiulotteisia vääristymiä. Nyt esitetyt tietokannat poikkeavat aikaisemmista ja sisältävät useita samanaikaisia vääristymistä, joiden interaktio kuvanlaadulle voi olla merkittävää.
V. julkaisussa esitellään kuvanlaatuympyrä (image quality wheel). Se on kuvanlaadun käsitteiden sanasto, joka on kerätty analysoimalla 146 henkilön tuottamat 39 415 kuvanlaadun sanallista kuvausta. Sanastoilla on pitkät perinteet aistinvaraisen arvioinnin tutkimusperinteessä, mutta niitä ei ole aikaisemmin kehitetty kuvanlaadulle.
VI. tutkimuksessa tutkittiin, kuinka arvioitsijoiden antamat käsitteet vaikuttavat kuvien laadun arviointiin. Esimerkiksi kuvien arvioitu terävyys tai luonnollisuus auttaa ymmärtämään laadunarvioinnin taustalla olevia päätöksentekoprosesseja. Tietoa voidaan käyttää esimerkiksi kuvan- ja videonlaadun arviointialgoritmien (I/VQA) kehitystyössä