Query expansion for the legal domain: a case study from the JUMAS project Fabio Sartori, Matteo Palmonari

Transcript

1 ONTOSE th June CAISE, Hammamet, Tunisia Query expansion for the legal domain: a case study from the JUMAS project Fabio Sartori, Matteo Palmonari Disco, University of Milan-Bicocca, Italy Paper appears in: M.-A. Sicilia, Ch. Kop, F. Sartori, ONTOSE 2010, Springer, Lecture Notes in Business Information Processing (LNBIP), Vol. 62

2 Outline JUMAS: Background & Motivation Query Expansion Methods Platform for Qualitative Evaluation of Methods

4 Libraries in the Juridical Domain The legal domain has interested AI for many years Knowledge representation systems for reasoning about the legal domain Large and heterogeneous data sources: Documents Video and audio of trials Need for effective techniques to manage libraries in the juridical domain Semantic processing and search The JUMAS project addresses these issues

5 The JUMAS project JUdicial MAnagement by Digital Libraries Semantics JUMAS envisages a system for the embedded semantic extraction from multimedia data that join into an advanced knowledge management system

6 Jumas Objectives Objectives: Knowledge Models and Spaces: search directly in the audio and video source without a verbatim transcription of the proceedings Knowledge and Content Management: exploit hidden semantics in audiovisual digital libraries in order to facilitate search and retrieval, intelligent processing and effective presentation of multimedia information Sensor and Multimedia Integration: information fusion deriving from multi-modal sources (audio/video) in order to improve accuracy in automatic transcription and annotation phases Effective Information Management: streamline and optimise the document workflow allowing the analysis of (un) structured information for document search and evidence base assessment ICT Infrastructure: Service Oriented Architecture supporting a largescale audiovideo retrieval system focusing on scalability, interoperability and modularity.

7 Jumas Objectives Objectives: Knowledge Models and Spaces: search directly in the audio and video source without a verbatim transcription of the proceedings Knowledge and Content Management: exploit hidden semantics in audiovisual digital libraries in order to facilitate search and retrieval, intelligent processing and effective presentation of multimedia information Sensor and Multimedia Integration: information fusion deriving from multi-modal sources (audio/video) in order to improve accuracy in automatic transcription and annotation phases Effective Information Management: streamline and optimise the document workflow allowing the analysis of (un) structured information for document search and evidence base assessment ICT Infrastructure: Service Oriented Architecture supporting a largescale audiovideo retrieval system focusing on scalability, interoperability and modularity.

8 Jumas Functional Architecture Automatic transcription Semantic annotation Search services, including semantic search

9 Semantic Search and Query Expansion (QE) Problems with search for semantically annotated content: poor recall Query expansion (QE) is the process of reformulating a query to improve retrieval performance. Evaluates a query (i.e. the keywords typed into the search query form) and expands them to match additional relevant documents, using a domain ontology for query expansion Many different approaches: Difficult to evaluate which is better in a given domain

10 Specific Goal & Objectives Main goal Testing and comparison of existing ontology-based methods for query expansion to be used in the JUMAS project Objectives Develop a configurable platform for query expansion that: implements and integrate existing methods can be adopted by domain experts in the context of the JUMAS project

12 Query Expansion Explained Query expansion Knife, trial, murder Knife phrase-based single-keyword often Econsists of a set of weighted keywords Two main appraches in literature: Probabilistic QE Ontology-based QE knife -1.0 blade -0.6 melee weapon -0.5 bayonet machete Scalpel Sword -0.2

13 Query Expansion Explained Query expansion Knife, trial, murder Knife phrase-based single-keyword often Econsists of a set of weighted keywords Two main appraches in literature: Probabilistic QE Ontology-based QE knife -1.0 blade -0.6 melee weapon -0.5 bayonet machete Scalpel Sword -0.2

14 Methodology 1.Analysis of proposed approaches 2.Implementation of the platform, including a set of selected methods 3.Testing e qualitative method comparison

15 Proposed and Selected QE Methods There are many classifications according to: Type of method (relevance feedback / similarity function) User involvement (manual / interactive / automatic) Usage of adjacent words (single word/ phrase based) Kind of query (short / long, structured / not structured, navigational / information / transactional) Application domain documents / tagged resources/ontologies/ source code / ) Kind of domain (generic / specific) Voorhees 94 [Voorhees] Tuominen et al. 09 Alani et al. 00[ATJ] Navigli & Velardi 03 Hirst & St-Onge 97[HO] Andreou 05 Xhu et al. 06 Calegari & Pasi 08 + Wu & Palmer 94 [Ancestor]

16 Adopted Ontology Model and Web-compliant Representation Thesaurusas the ontological models (i.e. lexical ontologies) focus on four relations among terms: SYN(Sinonimy); NT(Narrower Term); BT(Broader Term); RT(Related Term). Representation with SKOS(Simple Knowledge Organization System) W3C recommandation for thesauri definition based on RDF (+-OWL)

17 Platform s functional architecture Three main blocks: query expansion testing interface query expansion engine ontology management

22 QE workflow

23 Query Expansion Engine in JUMAS QE exposed as a Web service

24 Query Expansion Platform s GUI Queries and results with explanations Retrieved word and distance Type of relation Word to expand

25 Query Expansion Platform s GUI Ontology Management Interface Add/delete terms and relations to/from the thesaurus Retrieved word and distance Word to expand Export the ontology

26 Query Expansion Platform s GUI Platform configuration Configuration of parameters for each method Example for HO

27 Conclusions & Future Work Development of an environment for testing and discovering potentiality of expansion through lexical ontologies (i.e. thesauri) a query expander service that can be invoked by a Web service The platform allowed the expert from the court to select a method that they felt it fit better their needs (Hirst & St- Onge 97) Future work More quantitative experiments Axiomatic ontologies Adaptation of other similarity measures for ontologies (e.g. in the Simpack library)

28 Thanks!

29

30 Thesaurus adopted Thesaurus PICO, from (the website of Italian Ministery of Cultural Resources Management) Why Pico? Developed in SKOS; complete; Well-known domain. Completed by means of RT addition; Synonyms addition; Terms addition.

31 Set di query The query adopted are single word They have been selected in order to guarantee that each term can distignuishe within the thesaurus. In particular they are different according to: genericity; Number of NT; Number of RT; Presence of synonyms; Numbr of parents;

32 Evaluation Criteria Qualitative, not quantitative! Based on reasonable research scenarios

33 Output

34 Query Output

35 Output Query Expansion By means of Ancestor

36 Output Query Expansion By means of Ancestor Expansion By means of Voorhees (vers.1)

37 Results Indications on the better expansion for different scenarios: Contextual definition: expansion by menas of brothers or NT Components definition: the expansion by mean of NT is ideal, it is better to avoid the expansion by menas of brother nodes Search for general information: RT are very usefule, better to avoid NT methods Search for specific information: better to avoid brothrhood based methods, synonyms are very useful Causal relationships among terms: it is ideal to expand by means of children instead of brothers

38 Results Indications on general features of QE: Importance numerosity: relationships among terms seem to be inverse proportional with respect to their number Expansion Explosion : some terms, if reached during the expansion, cause the addition of many other terms which are irrelevant These result are starting points for new tests!!

39 Platform Extension: Sviluppi futuri Addition of new QE methods Addition of new kinds of ontolgies Addition of a suitable GUI Application to the JUMAS and CRESM-Milano Antica projects Analisys and research: looking for new methods, phrase-based methods, techniques to detect the kind of queries and to configure the method to apply according to them

40 Four variants: 1 Expansion only by means of synonyms 2 Expansion by means of synonyms and hyponyms 3 Expansion by means of synonyms, hypnyms and hyperonyms 4 Expansion by means of all the adjacent nodes Metodo 1: Voorhees

41 Metodo 2: ATJ Quattro varianti: espansione con iponimi e iperonimi, pesando le relazioni in base a tipo e profondità; come prima, con l'aggiunta delle relazioni RT; limitazione topologica all'espansione delle RT; limitazione semantica all'espansione delle RT (non implementata).

46 Metodo 3: HO Vengono aggiunti soli i termini che sono raggiunti attraverso pattern specifici: Una relazione BT può essere preceduto solo da altre relazioni BT Può esserci un unico cambio di direzione, due se il pattern è composto da serie di BT, serie di RT, serie di NT La lunghezza massima del percorso è5

47 Metodo 3: HO, pattern Uno o più RT Uno o più BT Uno o più NT

48 Vengono aggiunti soli i termini che sono raggiunti attraverso pattern specifici: Una relazione BT può essere preceduto solo da altre relazioni BT Metodo 3: HO Può esserci un unico cambio di direzione, due se il pattern è composto da serie di BT, serie di RT, serie di NT La lunghezza massima del percorso è 5

49 Metodo 4: Ancestor Calcola la similaritàdi due termini tramite una formula che tiene conto della distanza dal genitore comune e dalla profondità di quest'ultimo: ConSim C1,C2 = 2N3 N1+N2+ 2N3 Vengono aggiunti i termini che hanno distanza minore dal termine iniziale.

50 Metodo 4: Ancestor Calcola la similarità di due termini tramite una formula che tiene conto della distanza dal genitore comune e dalla profondità di quest'ultimo: ConSim C1,C2 = 2N3 N1+N2+ 2N3 Vengono aggiunti i termini che hanno distanza minore dal termine iniziale.