A new metric for patent retrieval evaluation

Patent retrieval is generally considered to be a recall-oriented information retrieval task that is growing in importance. Despite this fact, precision based scores such as mean average precision (MAP) remain the primary evaluation measures for patent retrieval. Our study examines different evaluation measures for the recall-oriented patent retrieval task and shows the limitations of the current scores in comparing different IR systems for this task. We introduce PRES, a novel evaluation metric for this type of application taking account of recall and user search effort. The behaviour of PRES is demonstrated on 48 runs from the CLEF-IP 2009 patent retrieval track. A full analysis of the performance of PRES shows its suitability for measuring the retrieval effectiveness of systems from a recall focused perspective taking into account the expected search effort of patent searchers

Discovering shifts in competitive strategies in probiotics, accelerated with TechMining

[EN] Profiling the technological strategy of different competitors is a key element for the companies in a given industry, as well to technology planners and R&D strategists. The analysis of the patent portfolio of a company as well as its evolution in the time line is of interest for technology analysts and decision makers. However, the need for the participation of experts in the field of a company as well as patent specialists, slows down the process. Bibliometrics and text mining techniques contribute to the interpretation of specialists. The present paper tries to offer a step by step procedure to analyze the technology strategy of several companies through the analysis of their portfolio claims, combined with the use of TechMining with the help of a text mining tool. The procedure, complemented with a semantic TRIZ analysis provides key insights in disclosing the technological analysis of some competitors in the field of probiotics for livestock health. The results show interesting shifts in the key probiotic and prebiotic ingredients for which companies claim protection and therefore offers clues about their technology intention in the life sciences industry in a more dynamic, convenient and simple way.The authors would like to thank the contribution of the research institute IRTA, to the TRIZ company triz XXI and to Fernando Palop and their wise insights and guidance. The authors thank the usage of Search Technology s VantagePoint and IHS-Markit s Goldfire.Vicente Gomila, JM.; Palli, A.; De La Calle, B.; Artacho Ramírez, MÁ.; Jimémez, S. (2017). Discovering shifts in competitive strategies in probiotics, accelerated with TechMining. Scientometrics. 111(3):1907-1923. https://doi.org/10.1007/s11192-017-2339-5S190719231113Abbas, A., Zhang, L., & Khan, S. (2014). A literature review on the state-of-the-art in patent analysis. World Patent Information, 37, 3–13.Allen, H., Levine, T., Bandrick, M., & Casey, T. (2012). Treatment, promotion, commotion: Antibiotic alternatives in food-producing animals. Trends in Microbiology, 21(3), 114–119.Animal Task Force. (2013). Research & innovation for a sustainable livestock sector in Europe. http://www.animaltaskforce.eu/Portals/0/ATF/horizon2020/ATF%20white%20paper%20Research%20priorities%20for%20a%20sustainable%20livestock%20sector%20in%20Europe.pdf . Accessed September 4, 2016.Abramson, D. (2011). Patent strategies for life sciences companies to navigate the changing patent landscape. Journal of Commercial Biotechnology, 17, 358–364.Banan-Mwine Daliri, E., & Lee, B. H. (2015). New perspectives on probiotics and disease. Food Science and Human Wellness, 4, 56–65.Bubela, T., Gold, R., Gregory, G., Cahoy, D., & Castle, D. (2013). Patent landscaping for life sciences innovation: Toward consistent and transparent practices. Nature Biotechnology, 31, 202–206.Chih-Hung, H. (2013). Patent value assessment and commercialization strategy. Technology forecasting & Social Change, 80, 307–319.Choi, S., Yoon, J., Kim, K., Lee, J. Y., & Kim, C.-H. (2011). SAO network analysis of patents for technology trends identification: A case study of polymer electrolyte membrane technology in proton exchange membrane fuel cells. Scientometrics, 88, 863–883.Collins, M. D., & Gibson, G. (1999). Probiotics, prebiotics, and synbiotics: Approaches for modulating the microbial ecology of the gut. American Journal of Clinical Nutrition, 69(suppl), 1052S–1057S.Ernst, H. (1998). Patent portfolio for strategic technology management. Journal of Engineering Technology Management, 15, 279–308.Ferraro, G., & Wanner, L. (2011). Towards the derivation of verbal content relations from patent claims using deep syntactic structures. Knowledge-Based Systems, 24, 1233–1244.Foligné, B., Daniel, C., & Pot, B. (2013). Probiotics from research to market: The possibilities, risks and challenges. Current Opinion in Microbiology, 16(3), 284–292.Gerken, J., & Moehrle, M. (2012). A new instrument for technology monitoring: Novelty in patents measured by semantic patent analysis. Scientometrics, 91, 645–670.Grant, R. (2006). Contemporary strategic analysis (5th ed.). ISBN 1-405-1999-3.Grant, E., Van den Hof, M., & Gold, R. (2014). Patent landscape analysis: A methodology in need of harmonized standards. World Patent Information, 39, 3–10.He, J., Yamanaka, T., & Kano, S. (2016). Mapping university receptor based on claim embodiment quantitative analysis: A study of 31 cases form the University of Tokio. World Patent Information, 46, 49–55.IHS Goldfire. www.ihsmarkit.com . Accessed November 2016.Kaushik, G. (Ed.) (2015). Applied environmental biotechnology: Present scenario and future trends. Springer. ISBN 978-81-322-2122-7.Kim, B., Miller, D., & Mahoney, J. (2016). The impact of the timing of patents on innovation performance. Research Policy, 45(2016), 914–928.Kume, H. (2010). From low power to no power through energy harvesting: Powering up the battery-free world. Nikkei Elctronics Asia; October 31, 2010; Accessed November 2011.Lanjouw, J., & Schankerman, M. (1999). The quality of ideas: Measuring innovation with multiple indicators. 7345. National Bureau for Economic Research, Cambridge, MA, USA. http://www.nber.org . Accessed September 2016.Lee, C., Kim, J., Kwon, O., & Woo, H. G. (2016). Stochastic technology life cycle analysis using multiple patent indicators. Technological Forecasting and Social Change, 106(2016), 53–64.Mogee, M. E. (1991). Using patent data for technology analysis and planning. Research-Technology Management, 34(4), 43–49.Niwa, S. (2016). Patent claims and economic growth. Economic Modelling, 54, 377–381.Noh, H., Jo, Y., & Lee, S. (2015). Keyword selection and processing strategy for applying text mining to patent analysis. World Patent Information, 42, 4348–4360.O’Callaghan, T. F., Ross, R. P., Stanton, C., & Clarke, G. (2016). The gut micorbiome as a virtual endocrine organ with implicaitons for farm and domestic animal endocrinology. Domestic Animal Endocrinology, 56, S44–S55.Pargaonkar, Y. (2016). Leveraging patent landscape analysis and IP competitive intelligence. World Patent Information, 45, 10–20.Park, H., Yoon, J., & Kim, K. (2012). Identifiying patent infringement using SAO based semantic technological similarities. Scientometrics, 90, 515–529.Park, H., Yoon, J., & Kim, K. (2013). Identification and evaluation of corporations for merger and acquisition strategies using patent information and text mining. Scientometrics, 97, 883–909.Porter, M. (2008). The five competitive forces that shape strategy. Harvard Business Review. January 2008. 1–17. Reprint R0801E. www.hbrreprints.org .Porter, A. L., & Cunningham, S. (2005). Tech Mining. Hoboken: Wiley Interscience.Porter, A., & Newman, N. (2011). Mining external R&D. Technovation, 31, 171–176.Regulation (EC) No 1831/2003 of the European Parliament and of the Council of 22 September 2003 on additives for use in animal nutrition Regulation (EC) No 1831/2003 of the European Parliament and of the Council. http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32003R1831 .Rose, C., Cronin, J., & Schwartz, R. (2007). Communicating the value of your intellectual property to Wall Street. Research Technology Management, 50(2), 36–40.Schrezenmeir, J., & De Vrese, M. (2001). Probiotics, prebiotics, and synbiotics—Approaching a definition. The American Journal of Clinical Nutrition, 73(2), 361s–364s.Soranzo, B., Nosella, A., & Filippini, R. (2016). Managing firm patents: A bibliometric investigation into the state of the art. Journal of Engineering and Technology Management, 42, 15–30.Teece, D. J. (1986). Profiting from technological innovation: Implications for integration, collaboration, licensing and public policy. Research Policy, 15, 285–305.The patent guide; A handbook for analyzing and interpreting patent data. UK Intellectual patent office.Tong, X., & Frame, D. (1994). Technological performance with patent claims data. Research Policy, 23, 133–141.VantagePoint. www.theVantagePoint.com . Accessed September 20, 2016.Verberne, S., D’hondt, E., & Oostdijk, N. (2010). Quantifying the challenges in parsing patent claims. In The 1st international workshop on Advances in Patent Information Retrieval (AsPIRe’10), Milton Keynes, UK.Verbitsky, M. (2004). Semantic TRIZ, triz-journal.com. http://www.triz-journal.com/archives/2004/ .Vicente-Gomila, J. M. (2014). The contribution of syntactic-semantic approach to the search for complimentary literatures for scientific or technical discovery. Scientometrics. doi: 10.1007/s11192-014-1299-2 .Vicente-Gomila, J. M., & Palop, F. (2013). Combining tech-mining and semantic-TRIZ for a faster and better technology analysis: A case in energy storage systems. Technology Analysis & Strategic Management, 25(6), 725–743.Wang, M., Chiu, T., & Chen, W. (2009). Exploring potential R&D collaborators based on patent portfolio analysis: The case of biosensors. In PICMET 2009 Proceedings, August 2–6, Portland, Oregon, USA.Wang, J., Lu, F., & Loh, H. (2015). A two-level parser for patent claim parsing. Advanced Engineering Informatics, 29, 431–439.Weenen, T. C., Pronker, E. S., Commandeur, H. R., & Claasen, E. (2013). Patenting in the European medical nutrition industry: Trends, opportunities and strategies. PharmaNutrition, 1, 13–21.Xie, Z., & Miyazaki, K. (2013). Evaluating the effectiveness of keyword search strategy for patent identification. World Patent Information, 35(1), 20–30.Yang, Y., & Choct, M. (2009). Dietary modulation of gut microflora in broiler chickens: A review of the role of six kinds of alternatives to in-feed antibiotics. World’s Poultry Science Journal, 65, 97–114.Yang, S.-Y., & Soo, V.-W. (2012). Extract conceptual graphs from plain texts in patent claims. Engineering Applications of Artificial Intelligence, 25, 874–887.Yoon, J., Park, H., & Kim, K. (2013). Identifying technological competition trends for R&D planning using dynamic patent maps: SAO-bassed content analysis. Scientometrics, 94, 313–331

Retrieval for Extremely Long Queries and Documents with RPRS: a Highly Efficient and Effective Transformer-based Re-Ranker

Retrieval with extremely long queries and documents is a well-known and challenging task in information retrieval and is commonly known as Query-by-Document (QBD) retrieval. Specifically designed Transformer models that can handle long input sequences have not shown high effectiveness in QBD tasks in previous work. We propose a Re-Ranker based on the novel Proportional Relevance Score (RPRS) to compute the relevance score between a query and the top-k candidate documents. Our extensive evaluation shows RPRS obtains significantly better results than the state-of-the-art models on five different datasets. Furthermore, RPRS is highly efficient since all documents can be pre-processed, embedded, and indexed before query time which gives our re-ranker the advantage of having a complexity of O(N) where N is the total number of sentences in the query and candidate documents. Furthermore, our method solves the problem of the low-resource training in QBD retrieval tasks as it does not need large amounts of training data, and has only three parameters with a limited range that can be optimized with a grid search even if a small amount of labeled data is available. Our detailed analysis shows that RPRS benefits from covering the full length of candidate documents and queries.Comment: Accepted at ACM Transactions on Information Systems (ACM TOIS journal

Toward higher effectiveness for recall-oriented information retrieval: A patent retrieval case study

Research in information retrieval (IR) has largely been directed towards tasks requiring high precision. Recently, other IR applications which can be described as recall-oriented IR tasks have received increased attention in the IR research domain. Prominent among these IR applications are patent search and legal search, where users are typically ready to check hundreds or possibly thousands of documents in order to find any possible relevant document. The main concerns in this kind of application are very different from those in standard precision-oriented IR tasks, where users tend to be focused on finding an answer to their information need that can typically be addressed by one or two relevant documents. For precision-oriented tasks, mean average precision continues to be used as the primary evaluation metric for almost all IR applications. For recall-oriented IR applications the nature of the search task, including objectives, users, queries, and document collections, is different from that of standard precision-oriented search tasks. In this research study, two dimensions in IR are explored for the recall-oriented patent search task. The study includes IR system evaluation and multilingual IR for patent search. In each of these dimensions, current IR techniques are studied and novel techniques developed especially for this kind of recall-oriented IR application are proposed and investigated experimentally in the context of patent retrieval. The techniques developed in this thesis provide a significant contribution toward evaluating the effectiveness of recall-oriented IR in general and particularly patent search, and improving the efficiency of multilingual search for this kind of task