Evaluation of IR and IE Systems

20 outubro 2017, 12:30 • Bruno Emanuel Da Graça Martins

• Why and what to evaluate?
• The Cleverdon evaluation methodology
• Measuring precision and recall in terms of retrieving relevant documents
• Precision-recall curves and interpolated curves
• Other evaluation metrics:
• P@N and R-precision
• F1-metric
• Mean Average Precision (MAP)
• Normalized Discounted Cumulative Gain (NDCG)
• Mean Reciprocal Rank (MRR)
• Metrics for comparing Rankins (Spearman's Rho and Kendall's Tau)
• Methods for collecting reference data