Sumários

Lab 04: Evaluation in Information Extraction and Retrieval

18 outubro 2019, 11:00 • Danielle Caled Vieira

Programing exercises:

  • Indexing documents using the Whoosh library.
  • Search over the index, using similarity ranking.
  • Implementation of evaluation metrics (precision, recall and F1).
Pen and Paper Exercises:
  • Information Retrieval System evaluation.
  • Classifier evaluation.
  • Clustering evaluation.
Support to the project.

Lab 04: Evaluation in Information Extraction and Retrieval

18 outubro 2019, 09:30 • Danielle Caled Vieira

Programing exercises:

  • Indexing documents using the Whoosh library.
  • Search over the index, using similarity ranking.
  • Implementation of evaluation metrics (precision, recall and F1).
Pen and Paper Exercises:
  • Information Retrieval System evaluation.
  • Classifier evaluation.
  • Clustering evaluation.
Support to the project.

Evaluating IR and IE Systems

17 outubro 2019, 17:00 • João Miguel Cordeiro Monteiro

  • Python exercises on evaluation IR systems (using the Whoosh library)
  • Pen-and-paper exercises about IR/IE evaluation metrics

IR/IE Evaluation (cont.)

17 outubro 2019, 15:30 • Bruno Emanuel Da Graça Martins

  • Ranking Comparison
    • Spearman Coefficient
    • Kendall’s Tau
  • Obtaining the Ground Truth and Methodological Aspects
    • Reference Collections
    • Human Experimentation in the Lab and A/B Testing
    • Crowdsourcing
    • Evaluation using Clickthrough Data and Click Models
  • Evaluation of Classifiers
    • Confusion Matrices and Accuracy
    • Macro- and Micro-Averaged Precision and Recall
    • Precision and Recall in Multi-Label Scenarios
    • Cross-Validation Methodologies
  • Evaluation of Clustering
    • Purity
    • Normalized Mutual Information
    • Rand Index
    • The B3 Family of Metrics

IR/IE Evaluation (cont.)

17 outubro 2019, 15:30 • Bruno Emanuel Da Graça Martins

  • Ranking Comparison
    • Spearman Coefficient
    • Kendall’s Tau
  • Obtaining the Ground Truth and Methodological Aspects
    • Reference Collections
    • Human Experimentation in the Lab and A/B Testing
    • Crowdsourcing
    • Evaluation using Clickthrough Data and Click Models
  • Evaluation of Classifiers
    • Confusion Matrices and Accuracy
    • Macro- and Micro-Averaged Precision and Recall
    • Precision and Recall in Multi-Label Scenarios
    • Cross-Validation Methodologies
  • Evaluation of Clustering
    • Purity
    • Normalized Mutual Information
    • Rand Index
    • The B3 Family of Metrics