Automatic Web Information Extraction Based on Maximal and Frenquent Equivalence Classes

Abstract

在定义模板的基础上,提出了页面创建模型。该模型描述了如何使用模板将来自于后台数据库的值编码生成页面。基于这个模型,设计了一个基于最大频繁等价类的抽取算法EBMFEC,通过分析给定的数据导向型页面的终端符号的出现情况,找出最大频繁等价类,并推导出用于生成页面的未知模板。然后使用推导出的模板,从输入页面中提取出相关信息。在大量实际HTML页面上的实验证明,EBMFEC在大部分情况下都可以从给定页面中推导出模板,并正确抽取出数据信息。A novel approach based on MFEC(Maximal and Frenquent Equivalence Classes)is proposed to solve the problem of automatically extracting data from data-intensive Web pages.A template is defined and a model of page creation is proposed to describe how values are encoded into pages using the defined template. We present an algorithm,EBMFEC that takes,as input,a set of template-generated pages,analyzes the page-tokens of given pages to discover MFEC,deduces the unknown template used to generate the pages and extracts,as output,the values encoded in the pages. Experiments on a large number of HTML pages indicate that our algorithm correctly extracts data in most cases and the results are also provided.国家自然科学基金(50474033);; 福建省自然科学基金(A0310008);; 福建省重点科技项目(2003H043)

    Similar works