Dimov, \u2702, Alexander

Tiede, Faculty Advisor, Hans-Joerg

Digital Commons @ Illinois Wesleyan University

Illinois Wesleyan UniversityDigital Commons @ IWUJohn Wesley Powell Student ResearchConference 2002, 13th Annual JWP ConferenceApr 21st, 10:00 AM - 11:00 AMThe Use of a Genetic Algorithm to Evolve NeuralNetworks for a Natural Language Processing TaskAlexander Dimov, '02Illinois Wesleyan UniversityHans-Joerg Tiede, Faculty AdvisorIllinois Wesleyan UniversityFollow this and additional works at: http://digitalcommons.iwu.edu/jwprcThis Event is brought to you for free and open access by The Ames Library, the Andrew W. Mellon Center for Curricular and FacultyDevelopment, the Office of the Provost and the Office of the President. It has been accepted for inclusion in Digital Commons @ IWU bythe faculty at Illinois Wesleyan University. For more information, please contact digitalcommons@iwu.edu.©Copyright is owned by the author of this document.Alexander Dimov, '02 and Hans-Joerg Tiede, Faculty Advisor, "The Use of a Genetic Algorithm to Evolve Neural Networksfor a Natural Language Processing Task" (April 21, 2002). John Wesley Powell Student Research Conference. Paper 1.http://digitalcommons.iwu.edu/jwprc/2002/oralpres3/1Oral Presentation 03. 1 THE USE OF A GENETIC ALGORITHM TO EVOLVE NEURAL NETWORKS FOR A NATURAL LANGUAGE PROCESSING TASK Alexander Dimov and Hans-Joerg Tiede* Department of Mathematics and Computer Science, lllinois Wesleyan University In this project a novel approach was taken for performing a natural language task. The task requires a neural network to predict the grammatical category of the next word in a stream of sentences. There are two main reasons why this task is interesting. In natural language processing, it is sometimes very difficult to determine the grammatical category of a word in a sentence when that word could belong to different grammatical categories depending on the context. For example, the word run can either be a noun or a verb in a certain sentence. The ability to correctly determine the category of the word can help a computer process natural language. In addition, the approach taken here to solve this task can lead to insights about the way the human brain learns andlor understands language. A Genetic Algorithm, which is conceptually based on simple principles known from Genetics, was developed and utilized to evolve neural networks that were used to perform the task. Genetic Algorithms have been used with remarkable success to solve complex problems in a number of fields but not for this type of problem. In addition, networks were trained via a classic learning algorithm, called back-propagation; to perform the same task. Since a Genetic Algorithm has not been used for this type of task, an implicit goal of this project was to show that it can be used. One of the other main questions addressed is whether learning (as in the case of training a neural network via back-propagation) and a search for an optimal solution (as in the case of the use of a Genetic Algorithm to evolve neural networks) differ and if so, how. Also, the underlying properties of the two different types of networks (depending on the approach taken to obtain them) were compared. Finally, issues about the computational complexity of the Genetic Algorithm were studied and discussed. These issues included the relationship between the input size (for ex. 1 0000 sentences) and the performance of the network developed via the Genetic Algorithm approach, as well as the way the network must change as the input changes in size and the task changes in complexity (i.e. as the grammar and lexicon change) while the optimal parameters (of the Genetic Algorithm) are used. 

The Use of a Genetic Algorithm to Evolve Neural Networks for a Natural Language Processing Task

http://digitalcommons.iwu.edu/cgi/viewcontent.cgi?article=2250&context=jwprc

The Use of a Genetic Algorithm to Evolve Neural Networks for a Natural Language Processing Task

Abstract

Similar works

Full text

Available Versions

Digital Commons @ Illinois Wesleyan University