Knowledge-based document retrieval with application to TEXPROS

Document retrieval in an information system is most often accomplished through keyword search. The common technique behind keyword search is indexing. The major drawback of such a search technique is its lack of effectiveness and accuracy. It is very common in a typical keyword search over the Internet to identify hundreds or even thousands of records as the potentially desired records. However, often few of them are relevant to users\u27 interests. This dissertation presents knowledge-based document retrieval architecture with application to TEXPROS. The architecture is based on a dual document model that consists of a document type hierarchy and, a folder organization. Using the knowledge collected during document filing, the search space can be narrowed down significantly. Combining the classical text-based retrieval methods with the knowledge-based retrieval can improve tremendously both search efficiency and effectiveness. With the proposed predicate-based query language, users can more precisely and accurately specify the search criteria and their knowledge about the documents to be retrieved. To assist users formulate a query, a guided search is presented as part of an intelligent user interface. Supported by an intelligent question generator, an inference engine, a question base, and a predicate-based query composer, the guided search collects the most important information known to the user to retrieve the documents that satisfy users\u27 particular interests. A knowledge-based query processing and search engine is presented as the core component in this architecture. Algorithms are developed for the search engine to effectively and efficiently retrieve the documents that match the query. Cache is introduced to speed up the process of query refinement. Theoretical proof and performance analysis are performed to prove the efficiency and effectiveness of this knowledge-based document retrieval approach

A more efficient document retrieval method for TEXPROS

Document processing is a critical element of office automation. Through document classification, extraction and filing, documents are automatically placed into a knowledge base according to certain rules. Document retrieval is a process to get a document back according to a user\u27s requirements and to show the results to the user. Hence, a good user-interface and an efficient retrieval algorithm become core parts of document retrieval. Unlike previous browsers that have been proposed for this purpose, this dissertation develops a new browser that has a user interface with more tools, and one that has a more efficient retrieval algorithm that can deal with a wide variety of retrieval situations. In this dissertation, from the view of an interface, the new browser provides more functions such as zoom in and zoom out , (i.e. automatic scaling of the portion of a graph that is of interest to a user), and help. These functions give users an easier way to view a large graph in one window and provide users with help during the retrieval process. The new browser also provides an algorithm that makes retrieval more efficient by using a reusable base. The Reusable Base is used to hold information that is most related to the user previous desires and the information stored in the Reusable Base is more easily used to form the OP-Net than that in the System Catalog. Hence, it eliminates the need to go to the System Catalog to find the results. This speeds up the retrieval significantly -at least two times faster than without the Reusable Base. Further, the new browser provides information about the folder organization and the document type hierarchy that is in addition to the OP-Net. If users know the type of documents they want, or which folder they are interested in, they can go to the particular document type or the particular folder directly

Knowledge-based document filing for texpros

This dissertation presents a knowledge-based document filing system for TEXPROS. The requirements of a. personal document processing system are investigated. In order for the system to be used in various application domains, a flexible, dynamic modeling approach is employed by getting the user involved in document modeling. The office documents are described using a dual-model which consists of a document type hierarchy and a folder organization. The document type hierarchy is used to capture the layout, logical and conceptual structures of documents. The folder organization, which is defined by the user, emulates the real world structure for organizing and storing documents in an office environment. The document filing and retrieval are predicate-driven. The user can specify filing criteria and queries in terms of predicates. The predicate specification and folder organization specification are described. It is shown that the new specifications can prevent false drops which happen in the previous approach. The dual models are incorporated by a three-level storage architecture. This storage architecture supports efficient document and information retrieval by limiting the searches to those frame instances of a document type within those folders which appear to be the most similar to the corresponding queries, Specifically, a. three-level retrieval strategy is used in document and information retrieval. Firstly, a knowledge-based query preprocess is applied for efficiently reducing the search space to a small set of frame instances, using the information in the query formula. Secondly, the knowledge and content-based retrieval on the small set of frame instances is applied. Finally, the third level storage provides a platform for adopting potential content-based multimedia document retrieval techniques. A knowledge-based predicate evaluation engine is described for automating document filing. The dissertation presents a knowledge representation model. The knowledge base is dynamicly created by a learning agent, which demonstrates that the notion of flexible and dynamic modeling is applicable. The folder organization is implemented using an agent-based architecture. Each folder is monitored by a filing agent. The basic operations for constructing and reorganizing a folder organization are defined. The dissertation also discusses the cooperation among the filing agents, which is needed for implementing the folder organization

Knowledge management for TEXPROS

Most of the document processing systems today have applied Al technologies to support their system intelligent behaviors. For the application of Al technologies in such systems, the core problem is how to represent and manage different kinds of knowledge to support their inference engine components\u27 functionalities. In other words, knowledge management has become a critical issue in the document processing systems. In this dissertation, within the scope of the TEXt PROcessing System (TEXPROS), we identify knowledge of various kinds that are applicable in the system. We investigate several problems of managing this knowledge and then develop a knowledge base for TEXPROS. In developing this knowledge base, we present approaches to representing and managing different kinds of knowledge to support its inference engine components\u27 functionalities. In TEXPROS, a dual-model paradigm is used, which contains the folder organization and the document type hierarchy, to represent and manage documents. We introduce a new System Catalog structure to represent and manage the knowledge for TEXPROS. This knowledge includes the system-level information of the folder organization and the document type hierarchy, and the operational level information of the document base itself. A unified storage approach is employed to store both the operational level information and system level information. Such storage is to house the frame template base and frame instance base. An enhanced two-level thesaurus model is presented in this dissertation. When dealing with special kinds of data in processing documents, a new structure DataDomain is presented, which supports the extended thesaurus functionalities, pattern recognition and data type operations. Based on the dual-model paradigm of TEXPROS, a concept of “Semantic Range” is presented to solve the sense ambiguity problems. In this dissertation, we also present the approaches to implement the general KeyTerm transformation and approximate term matching of TEXPROS. Finally, a new component “Registration Center” at the knowledge management level of TEXPROS is presented. The registration center aims to help users handle knowledge packages for specific working domain and to solve the knowledge porting problem for TEXPROS. This dissertation is concluded with the future research work

e-DOCSPROS : exploring TEXPROS into e-business era

Document processing is a critical element of office automation. TEXPROS (TEXt PROcessing System) is a knowledge-based system designed to manage personal documents. However, as the Internet and e-Business changed the way offices operate, there is a need to re-envision document processing, storage, retrieval, and sharing. In the current environment, people must be able to access documents remotely and to share those documents with others. e-DOCPROS (e-DOCument PROcessing System) is a new document processing system that takes advantage of many of TEXPROS\u27s structures but adapts the system to this new environment. The new system is built to serve e-businesses, takes advantage of Internet protocols, and to give remote access and document sharing. e-DOCPROS meets the challenge to provide wider usage, and eventually will improve the efficiency and effectiveness of office automation. It allows end users to access their data through any Web browser with Internet access, even a wireless network, which will evolutionarily change the way we manage information. The application of e-DOCPROS to e-Business is considered. Four types of business models re considered here. The first is the Business-to-Business (B2B) model, which performs business-to-business transactions through an Extranet. The Extranet consists of multiple Intranets connected via the Internet.The second is the Business-to-Consumer (B2Q model, which performs business-to-consumer transactions through the Internet. The third is the Intranet model, which performs transactions within an organization through the organization\u27s network. The fourth is the Consumer-to-Consumer (C2C) model, which performs consumer-to consumer transactions through the Internet. A triple model is proposed in this dissertation to integrate organization type hierarchy and document type hierarchy together into folder organization. e-DOCPROS introduces new features into TEXPROS to support those four business models and to accommodate the system requirements. Extensible Markup Language (XML), an industrial standard protocol for data exchange, is employed to achieve the goal of information exchange between e-DOCPROS and the other systems, and also among the subsystems within e-DOCPROS. Document Object Model (DOM) specification is followed throughout the implementation of e-DOCPROS to achieve portability. Agent-based Application Service Provider (ASP) implementation is employed in e-DOCPROS system to achieve cost-effectiveness and accessibility

Intelligent information retrieval from the World Wide Web using fuzzy user modelling.

This thesis investigates the application. of fuzzy logic techniques and user modelling to the process of information retrieval (IR) from the World Wide Web (WWW). The research issue is whether this process can be improved through such an application. The exponential rise of information itself as an invaluable global commodity, coupled with .acceierating development in. computing and telecommunications, and boosted by networked information sources such as the WWW, has led to the development of tools, such as search engines, to facilitate information search and retrieval. However, despite their sophistication, they are unable effectively to. address users' information. needs. Also, as the-WWW can be seen as a dynamic, continuously changing global information corpus, these tools suffer from the problems of irrelevancy and redundancy. Therefore, in order to overcome these problems and remain effective, IR systems need to become 'intelligent' in some way. It is from this premise that the focus of this research has developed. Initially, theoretical and investigative research into the areas ofIR from electronic sources and the nature of the Internet (including the WWW) revealed that highly sophisticated systems are being developed and there is a drive towards the integration of, for example, electronic libraries, COROM networks, and the WWW. Research into intelligent IR, the use of AI techniques to improve the IR process, informed an evaluation of various approaches. This revealed that a munber of techniques, for example, expert systems, neural networks and semantic networks, have been employed, with limited success. Owing to the nature of the WWW, though, many of the previous AI approaches are inapplicable as they rely too much on extensive knowledge of the retrieval corpus. However, the evaluation suggested that fuzzy logic, with its inherent ability to capture partial knowledge within fuzzy sets, is a valid approach. User modelling research indicated that adaptive user stereotypes are a fruitful way to represent different types of user and their information need. Here, these stereotypes are represented as fuzzy sets, ensuring flexibility and adaptivity. The goal of the reported research. then, was not to. develop an 'intelligent agent' but to apply fuzzy logic techniques and user modelling to the process of user query formulation, in order to test the research issue. This issue was whether the application of these techniques could improve the IR process. A prototype system, the Fuzzy Modelling Query Assistant (FMQA), was developed that attempts intelligently to assist the user in capturing their information need. The concept was to refine the user's query before submitting it to an existing search engine, in order to improve upon the IR results of using the search tool alone. To address the research issue, a user study of the FMQA was performed. The design and conduct is reported in depth. The study results were analysed and the findings are given. The results indicate that,. for certain types of user especially, the FMQA does provide improvement in the IR process, in terms of the results. There is a critical review of the research aims in the light of the results, conclusions are drawn and recommendations for future research given