Abstract

This report builds approaches towards evaluating lexico-semantic networks by studying evaluation strategies applied to ontologies. It shows the lack of such methods for networks such as WordNet, and so builds a case for such evaluations. A brief introduction to lexico-semantic networks, a mention of the principles of evaluation and the successes in Machine Translation evaluation are also included in this report.

Lexico-semantic networks such as WordNet have burst into prominence be cause applications, especially those targeted for the Web, now aim to en hance the semantic dimensions of their performance. An example of such an application is in Information Retrieval where a lexical resource can help provide easy query keyword disambiguation and improve the quality of search results retrieved. This is especially due to the fact that the quantity of documents now available via the Web is extremely large, resulting in the nee for further sophistication. Alternatively, consider automatic generation of content for certain contexts such as tourist phrasebooks, or automatic sensing of emotion from text.

Lexical resources that can potentially reveal, generate or help infer such content are being developed by various research groups. These are no longer simple dictionaries; rather they are rich in \semantic content" going far beyond the scope of mere lexicons. Lexico-semantic networks can also be viewed as a reservoir of common sense concepts arranged ontologically, hence describing the real-world through lexical knowledge. The bottomline is that such resources are being increasing co-opted in applications involving language technology, and not just in English. Almost every major language now has a WordNet project, and efforts such as ConceptNet attempt to include those aspects not covered by WordNet.

The increasing production of such networks and their application in diverse areas call for evaluation methods to describe the quality of rival networks as well to set expectations about their likely performance in applications. This covers a gamut of criteria, which unfortunately have not been studied in detail. This report sets the stage for an investigation into evaluation strategies for lexico-semantic networks.

<< back