Image based crop row detection using global optimization methods

U ovoj doktorskoj disertaciji naglasak je stavljen na razvoj metoda za prepoznavanje redova usjeva na slici. Cilj je razviti metodu koja posjeduje sljedeca svojstva: robusnost s obzirom na prisutnost korova i sjene, mogucnost prepoznavanja redova usjeva za razlicite kulture u razlicitim stadijima rasta, mogucnost prepoznavanja razlicitog broja proizvoljno razmaknutih redova usjeva te mogucnost prepoznavanja ravnih i zakrivljenih redova usjeva. Razvijene su dvije metode koje su nazvane CRDI metoda i TMGEM metoda. CRDI metoda zasnovana je na inkrementalnoj metodi za trazenje priblizno optimalne particije skupa podataka te primjeni DIRECT algoritma za globalnu optimizaciju. Metoda je pogodna za prepoznavanje ravnih redova usjeva, pri cemu broj redova usjeva mora biti unaprijed poznat. TMGEM metoda zasnovana je racunanju podudaranja s predloskom i minimizaciji funkcije globalne energije primjenom dinamickog programiranja. Funkcija globalne energije, uz podatke sa slike, koristi i prethodno znanje o geometrijskoj strukturi redova usjeva, cime se postize veca tocnost prepoznavanja redova usjeva. Predlozena metoda ima mogucnost prepoznavanja proizvoljnog broja redova usjeva za razlicite kulture u razlicitim stadijima rasta te pri razlicitim geometrijama polja. Nadalje, predlozena TMGEM metoda robusna je na zakrivljenost redova usjeva. Osim navedenih metoda, u radu je predlozen novi evaluacijski okvir za usporedbu metoda prepoznavanja redova usjeva koji ukljucuje bazu slika redova usjeva, postupak rucnog generiranja referentnih vrijednosti redova usjeva te kriterije za odredivanje tocnosti prepoznavanja redova usjeva. Na slikama u bazi snimljeni su redovi usjeva kukuruza, celera, krumpira, luka, suncokreta i soje. Koristenjem predlozenog evaluacijskog okvira moguce je ekasnije i objektivnije usporediti novo razvijene metode za prepoznavanje redova usjeva s postojecim metodama. U eksperimentalnoj evaluaciji CRDI metoda je usporedena s IMLD i HT metodom na umjetno generiranim skupovima podataka. Tocnost prepoznavanja odredena je primjenom CRHID kriterija, a rezultati pokazuju da CRDI metoda prepoznaje redove usjeva sa znatno vecom tocnoscu u odnosu na ostale razmatrane metode. TMGEM metoda testirana je na 225 stvarnih slika redova usjeva iz baze slika te je usporedena s HT, HTT i LR metodom. Tocnost prepoznavanja redova usjeva odredena je koristenjem CRDA kriterija, a rezultati pokazuju da predlozena metoda znacajno nadmasuje ostale tri razmatrane metode pri prepoznavanju ravnih redova usjeva te je robusna na zakrivljenost redova usjeva.The topic of this doctoral thesis is the development of image based crop row detection methods. The main goal of the research is to develop a method which is: highly insensitive to the presence of weeds and shadows, capable of detecting crop rows of dierent crop types at dierent stages of growth, capable of detecting straight and curved crop rows and insensitive to the number and spacing of crop rows. Two methods are proposed entitled CRDI and TMGEM. CRDI method is based on incremental method of searching for an approximate globally optimal partition of a set of data points and on the DIRECT algorithm for global optimization. The method is capable of detecting straight crop rows, wherein the number of crop rows must be known in advance. TMGEM method is based on template matching followed by global energy minimization with dynamic programming technique. For accurate crop row detection, the global energy function combines image evidence and prior knowledge about the geometric structure of crop rows. The proposed method is insensitive to the number and spacing of crop rows and is capable of detecting crop rows of dierent crop types at dierent stages of growth. Furthermore, the proposed TMGEM method is capable of detecting curved crop rows. A new evaluation framework is proposed that consists of a crop row image database, manual ground truth image creation approach and two crop row detection performance measures. The image database includes images of dierent crop types including maize, celery, potato, onion, sun ower and soybean. The proposed evaluation framework enables ecient and objective comparison of new crop row detection methods with existing ones. The experimental evaluation of CRDI method includes comparison with IMLD and HT method on synthetic datasets, based on the proposed CRHID performance measure. The results show that CRDI method outperforms other considered methods. TMGEM method is evaluated on a set of 225 real-world crop row images from the image database and it is compared with HT, HTT and LR method. The proposed CRDA measure is used as a performance measure in the comparison. The results show that TMGEM signicantly outperforms the other considered methods in straight crop row detection and is capable of detecting curved crop rows

Detect or Track: Towards Cost-Effective Video Object Detection/Tracking

State-of-the-art object detectors and trackers are developing fast. Trackers are in general more efficient than detectors but bear the risk of drifting. A question is hence raised -- how to improve the accuracy of video object detection/tracking by utilizing the existing detectors and trackers within a given time budget? A baseline is frame skipping -- detecting every N-th frames and tracking for the frames in between. This baseline, however, is suboptimal since the detection frequency should depend on the tracking quality. To this end, we propose a scheduler network, which determines to detect or track at a certain frame, as a generalization of Siamese trackers. Although being light-weight and simple in structure, the scheduler network is more effective than the frame skipping baselines and flow-based approaches, as validated on ImageNet VID dataset in video object detection/tracking.Comment: Accepted to AAAI 201

Automatic Brain Tumor Segmentation by Deep Convolutional Networks and Graph Cuts

Brain tumor segmentation in magnetic resonance imaging (MRI) is helpful for diagnostics, growth rate prediction, tumor volume measurements and treatment planning of brain tumor. The difficulties for brain tumor segmentation are mainly due to high variation of brain tumors in size, shape, regularity, location, and their heterogeneous appearance (e.g., contrast, intensity and texture variation for different tumors). Due to recent advances in deep convolutional neural networks for semantic image segmentation, automatic brain tumor segmentation is a promising research direction. This thesis investigates automatic brain tumor segmentation by combining deep convolutional neural network with regularization by a graph cut. We investigate several deep convolutional network structures that have been successful in semantic and medical image segmentation. Since the tumor pixels account for a very small portion in the whole brain slice, segmenting the tumor from the background is a highly imbalanced dense prediction task. We use a loss function that takes the imbalance of the training data into consideration. In the second part of the thesis, we improve the segmentation results of a deep neural network by using optimization framework with graph cuts. The graph cut framework can improve segmentation boundaries by making them more smooth and regular. The main issue when using the segmentation results of convolutional neural networks for the graph cut optimization framework is to convert tumor probabilities learned by a convolutional network into data terms. We investigate several possible ways that take into consideration the segmentation artifacts by convolutional neural networks. In experiments, we present the segmentation results by different deep convolutional neural network structures, e.g., fully convolutional neural network, dilated residual network and UNet. Also, we compare the combination of U-Net with different data terms for graph cut regularization to improve the neural network segmentation results. Experimental results show that the U-Net performs best with the intersection over union (IoU) for tumors of 0.7286. The IoU for tumors is improved to 0.7530 by training on three slices. Also, the IoU for tumors is improved to 0.7713 by U-Net with balanced loss function. The IoU for tumors is further improved to 0.8078 by graph cut regularization