Automatic Assessment of Coverage Quality in Intelligence Reports

Samuel Brody and Paul Kantor
Rutgers University


Abstract

Common approaches to assessing document quality look at shallow aspects, such as grammar and vocabulary. For many real-world applications, deeper notions of quality are needed. This work represents a first step in a project aimed at developing computational methods for deep assessment of quality in the domain of intelligence reports. We present an automated system for ranking intelligence reports with regard to coverage of relevant material. The system employs methodologies from the field of automatic summarization, and achieves performance on a par with human judges, even in the absence of the underlying information sources.




Full paper: http://www.aclweb.org/anthology/P/P11/P11-2086.pdf