Ankolekar A, Cimiano P, Hitzler P, et al. SmartWeb: Mobile Access to the Semantic Web. In: Wache H, ed. Proceedings of the ESWC2006 poster and demo session. 2006: 3-4

Ankolekar, Anupriya

Cimiano, Philipp 

Hitzler, Pascal

Kiesel, Malte

Krötzsch, Markus

Ladwig, Günter

Lewen, Holger

Oberle, Daniel

Sintek, Michael

Studer, Rudi

Wache, Holger

English

Publications at Bielefeld University

SmartWeb: Mobile Access to the Semantic Web A. Ankolekar, P. Cimiano, P. Hitzler, M. Krötzsch, G. Ladwig, H. Lewen, D. Oberle, R. Studer AIFB, Universität Karlsruhe, Germany  M. Kiesel, M. Sintek DFKI Kaiserslautern, Germany  ABSTRACT We present the SmartWeb Demonstrator for multimodal and mobile querying of semantic resources and the open WWW.  The end-user interface consists of a Pocket Data Assistant which accepts written or spoken questions as input and delivers answers based on a multitude of resources including a semantic knowledge base, semantically annotated online web services, and semi-automatically created knowledge from text-based web pages. If answers cannot be found using these structured resources, then the system returns answers based on linguistic query-answering techniques on the open WWW.  1. INTRODUCTION Recent progress in mobile broadband communication and semantic web technology is enabling innovative internet services that provide advanced personalization and localization features. The goal of the SmartWeb (http://www.smartweb-project.de) project is to lay the foundations for multimodal user interfaces to distributed semantic web resources and services on mobile devices. The SmartWeb consortium brings together experts from various research communities: mobile services, intelligent user interfaces, language and speech technology, information extraction, and semantic web technologies.  SmartWeb is based on two parallel efforts that have the potential of forming the basis for an advancement of the web. The first effort is the Semantic Web, which provides the tools for the explicit markup of the content of web pages. The second effort is the development of semantic web services which results in a web where programs act as autonomous agents to become the producers and consumers of information and enable automation of transactions. The appeal of being able to ask a question to a mobile internet terminal and receive an answer immediately has been renewed by the broad availability of information on the web. Ideally, a spoken dialogue system that uses the web as its knowledge base would be able to answer a broad range of questions. SmartWeb exploits the machine-understandable content of semantic web pages for intelligent question-answering as a next step beyond today's search engines. Since semantically annotated web pages are still very rare due to the time-consuming and costly manual markup, SmartWeb is using advanced language technology and information extraction methods for the automatic annotation of traditional web pages encoded in HTML or XML. SmartWeb provides a context-aware user interface, so that it can support the user in different roles, e.g. as a car driver, a motor biker, a pedestrian or a sports spectator. One of the demonstrators of SmartWeb is a personal guide for the 2006 FIFA world cup in Germany, that provides mobile infotainment services to soccer fans, anywhere and anytime, using a PDA as user-interface. We will present this demonstrator at the conference.    The academic partners of SmartWeb are the research institutes DFKI (consortium leader, Prof. Dr. Wolfgang Wahlster), FhG FIRST, and ICSI together with university groups from Erlangen, Karlsruhe, Munich, Saarbrücken, and Stuttgart. The industrial partners of SmartWeb are BMW, DaimlerChrysler, Deutsche Telekom, and Siemens as large companies, as well as EML, Ontoprise, and Sympalog as small businesses. The German Federal Ministry of Education and Research (BMBF) is funding Demos and Posters of the 3rd European Semantic Web Conference (ESWC 2006), Budva, Montenegro, 11th - 14th June, 2006 the SmartWeb consortium from 2004 to 2007. We acknowledge the funding under project number 01 IMD01 B. 2. MULTIMODAL RECOGNITION AND MODELING  Access to SmartWeb is either gained on the field via a PDA/Smartphone (UMTS) with a server-based speech-recognizer or in a mobile car-scenario via a built-in speech-recognizer in the car. On the PDA, the interface is supplemented with multimodal input, e.g. with a pen. In addition, a camera monitors the face and recognizes whether the user is addressing the system.  For the development of a mobile multimodal dialogue assistant in SmartWeb that is usable in open domains and thematically wide-ranging areas, spoken language is the central mode of communication. In communications that are situation dependent and technically allow the use of the whole range of multimodal functionalities, the phonological output of the dialogue assistant using different output modalities – for example music as another form of acoustic output or visual (text, graphic, picture, video) and haptic representations – has to be spatially and timely synchronized.   3. ONTOLOGICAL INFRASTRUCTURE  The SmartWeb project comprises the definition, implementation, and application of ontologies for various parts of the system. Using ontologies enables the formalization of concepts that are understood and accepted by a wide user basis. They lay the foundation for the dialogue with the user as well as for the flexible communication between applications from various, wide-ranging areas. The ontologies also enable important basic tasks such as the formulation of structural queries and inferencing. The SmartWeb integrated ontology consists of several domain ontologies which are aligned by means of an adaptation of SUMO and DOLCE. The domain ontologies describe sport events, navigation information, multimodal interaction discourses, multimedia data, and linguistic information. The purpose of the domain ontologies is not only to provide accurate answers to queries. They are also used in other parts of the system, e.g. for semantic annotation of web services, for modelling the multimodal interaction between the user and the SmartWeb system, and for handling linguistic information about objects and classes.  Different ontology representation languages are used within different subsystems. RDFS is being used dominantly, in particular for the domain ontologies. OWL is being used for representing more complex information, e.g. for the foundational ontology which was utilized for aligning the domain ontologies. Reasoning support and intelligent knowledge processing is provided by the Ontobroker system which is based on F-Logic. 4. ON- AND OFFLINE EXTRACTION OF SEMANTIC STRUCTURES Extraction from syntactic web pages deals with the development and application of techniques that render answers to arbitrary, domain-independent user requests online and in real-time. These methods are rather shallow but fast and robust. They depend on already existing search engines like Google and only take classic ("syntactic") web pages into account that are written in pure HTML (without semantic annotation). The Offline-Extraction component SOBA consists of a web crawler, linguistic annotation components and a component for the transformation of linguistic annotations into an ontology-based representation which can be used to answer user queries. Currently, SOBA is applied to FIFA web pages to extract information related to the world cup 2006. 5. WEB SERVICES The SmartWeb system utilizes existing web services, including the T-Info web services. They include navigational and weather information. For use within the SmartWeb system, the web services are semantically annotated. Certain queries invoke calls to corresponding web services whose responses can be fed back to the user.   6. DEMONSTRATION We will demonstrate the SmartWeb system on the PDA, which accepts input in spoken form, by keyboard, or by pen. Users can ask open questions which are linguistically analysed and passed on to the semantic mediator subsystem. The semantic mediator queries all available knowledge sources, including the manually created domain ontologies, semantically annotated web services, the knowledge which was automatically extracted from web pages, and a linguistic query-answering subsystem on the open internet.  Answers returned by the knowledge sources come accompanied by multimedia data and are endowed with certainty estimates. They are integrated in order to obtain the system response, consisting of text and multimedia objects. Further queries by the user are then processed in the context of the previous interactions.    

SmartWeb: Mobile Access to the Semantic Web

Ankolekar A, Buitelaar P, Cimiano P, et al. SmartWeb: Mobile Access to the Semantic Web. In:  Proceedings of the ISWC 2006 Poster and Demo Session. 2006

Buitelaar, Paul

Neumann, G.

Tserendorj, Tuvshintur

SmartWeb: Mobile Access to the Semantic Web1A. Ankolekar1, P. Buitelaar2, P. Cimiano1, P. Hitzler1, M. Kiesel3, M. Krötzsch1,  H. Lewen1, G. Neumann2, M. Sintek3, T. Tserendorj1, R. Studer1  1 AIFB, Universität Karlsruhe, Germany  2 DFKI GmbH Saarbrücken, Germany 3 DFKI GmbH Kaiserslautern, Germany Abstract. We present the SmartWeb Demonstrator for multimodal and mobile querying of semantic resources and the open WWW.  The end-user interface consists of a Pocket Data Assistant which accepts written or spoken questions as input and delivers answers based on a multitude of resources including a semantic knowledge base, semantically annotated online web services, and semi-automatically created knowledge from text-based web pages. If answers cannot be found using these structured resources, then the system returns answers based on linguistic query-answering techniques on the open WWW.  Keywords: question answering, ontology engineering, semantic web services, multimodality, mobile and ubiquitous computing 1   Introduction Recent progress in mobile broadband communication and semantic web technology is enabling innovative internet services that provide advanced personalization and localization features. The goal of the SmartWeb (http://www.smartweb-project.de) project is to lay the foundations for multimodal user interfaces to distributed semantic web resources and services on mobile devices.  SmartWeb is based on two parallel efforts that have the potential of forming the basis for an advancement of the web. The first effort is the Semantic Web, which provides the tools for the explicit markup of the content of web pages. The second effort is the development of semantic web services which results in a web where programs act as autonomous agents to become the producers and consumers of information and enable automation of transactions. SmartWeb exploits the machine-understandable content of semantic web pages for intelligent question-answering as a next step beyond today's search engines. Since semantically annotated web pages are still very rare due to the time-consuming and costly manual markup, SmartWeb is using advanced language technology and information extraction methods for the automatic annotation of traditional web pages encoded in HTML or XML. SmartWeb provides a context-aware user interface, so that it can support the user in different roles, e.g. as a car driver or a sports spectator. One of the demonstrators of                                                           1 The academic partners of SmartWeb are the research institutes DFKI (consortium leader, Prof. Dr. Wolfgang Wahlster), FhG FIRST, and ICSI together with university groups from Erlangen, Karlsruhe, Munich, Saarbrücken, and Stuttgart. The industrial partners of SmartWeb are BMW, DaimlerChrysler, Deutsche Telekom, and Siemens as large companies, as well as EML, Ontoprise, and Sympalog as small businesses. The German Federal Ministry of Education and Research (BMBF) is funding the SmartWeb consortium from 2004 to 2008. We acknowledge the funding under project number 01 IMD01. SmartWeb is a personal guide for the 2006 soccer world cup in Germany, that provides mobile infotainment services to soccer fans, anywhere and anytime, using a PDA as user-interface. We will present this demonstrator at the conference. 2   Multimodal Recognition and Modelling Access to SmartWeb is either gained on the field via a PDA/Smartphone (UMTS) with a server-based speech-recognizer or in a mobile car-scenario via a built-in speech-recognizer in the car. On the PDA, the interface is supplemented with multimodal input, e.g. with a pen. In addition, a camera monitors the face and recognizes whether the user is addressing the system.  3   Ontological Infrastructure The SmartWeb project comprises the definition, implementation, and application of ontologies for various parts of the system. Using ontologies enables the formalization of concepts that are understood and accepted by a wide user basis. They lay the foundation for the dialogue with the user as well as for the flexible communication between applications from various, wide-ranging areas. The ontologies also enable important basic tasks such as the formulation of structural queries and inferencing. 4 On- and Offline Extraction of Semantic Structures for Question Answering SmartWeb derives answers through open-domain question answering (QA) or from an automatically generated knowledge base. The QA component extracts answers from web pages that are retrieved and analysed in real-time by use of statistical and shallow linguistic techniques. The SmartWeb Ontology-Based Annotation (SOBA) subsystem automatically populates a knowledge base by information extraction from semi-structured and free text soccer match reports from the WWW. 5   Web Services The SmartWeb system utilizes existing web services, including the T-Info web services. They include navigational and weather information. For use within the SmartWeb system, the web services are semantically annotated. Certain queries invoke calls to corresponding web services whose responses can be fed back to the user. References P. Buitelaar, P. Cimiano, S. Racioppa, M. Siegel, Ontology-based Information Extraction with SOBA. In: Proceedings of LREC 2006. G. Neumann, B. Sacaleanu, Experiments on Cross-Linguality and Question-type driven Strategy Selection for Open-Domain Question Answering. In: C. Peters et al. (Eds.), Proceedings of CLEF 2005, LNCS 4022, Springer, Berlin, pp. 429–438, 2006. D. Oberle et al., DOLCE ergo SUMO: On Foundational and Domain Models in SWIntO (SmartWeb Integrated Ontology). Technical Report, 2006. Available from http://www.aifb.uni-karlsruhe.de/Publikationen/showPublikation?publ_id=1246. N. Reithinger, D. Sonntag, An Integration Framework for a Mobile Multimodal Dialogue System Accessing the Semantic Web. In: Proceedings of Interspeech 2005. 

An Integration Framework for a Mobile Multimodal Dialogue System Accessing the Semantic Web. In:

DOLCE ergo SUMO: On Foundational and Domain Models in SWIntO (SmartWeb Integrated Ontology).

Ontology-based Information Extraction with SOBA. In:

http://pub.uni-bielefeld.de/publication/2497570

SmartWeb: Mobile Access to the Semantic Web

Abstract

Similar works

Full text

Available Versions

Publications at Bielefeld University

Publications at Bielefeld University