• Home
• General Information
• Events
• Publications
• The Forum
• SIG-IRList
• Resources

SIGIR Museum contents

Go directly to the SIGIR Digital Museum of Information Retrieval Research.

A new batch of historic documents is available now.

Report on the first stage of an investigation onto the comparative efficiency of indexing systems

Cyril W. Cleverdon
The College of Aeronautics, Cranfield, England, 1960

• Preface
  
• Chapter 1, Introduction
  
• Chapter 2, General objectives of the investigation
  
• Chapter 3, General organisation of programme
  
• Chapter 4, Detailed analysis of the problems encountered
  
• Chapter 5, Supplumentary procedures
  
• Chapter 6, Statistical details
  
• Chapter 7, Supplementary indexing
  
• Chapter 8, Conclusions
  
• Appendix A, List of project documents 14001 - 14100
  
• Appendix B, U.D.C. schedules and index
  
• Appendix C and D, Alphabetical subject headings and facet schedules
  
• Appendix E, List of uniterms
  
• Appendix F, Sample sheet of documents for externam indexing and questions
  
• Appendix G, List of co-operating organisations
  

Factors determining the performance of indexing systems; Volume 1: Design

Cyril W. Cleverdon, Jack Mills, E. Michael Keen
The College of Aeronautics, Cranfield, England, 1966

• Preface
  
• Chapter 1, General considerations
  
• Chapter 2, Test design
  
• Chapter 3, Documents and questions
  
• Chapter 4, Indexing procedures
  
• Chapter 5, Formation of index languages
  
• Chapter 6, Testing Techniques
  
• Chapter 7, Additional tests
  
• Chapter 8, Comments
  
• References
  

Factors Determining the performance of indexing systems. Volume 2

Cyril W. Cleverdon, E. Michael Keen
The College of Aeronautics, Cranfield, England, 1966

• Summary and Preface
  
• Chapter 1, Introduction
  
• Chapter 2, Test environment
  
• Chapter 3, Methods for presentation of results
  
• Chapter 4, Main test results efficiency
  
• Chapter 5, simulated ranking and output cut-off
  
• Chapter 6, Supplementary tests and results
  
• Chapter 7, Citation indexing and bibliographic coupling
  
• Chapter 8, Conclusions
  
• References
  
• Appendix 3A, Tables of generality number, and fallout, recall and precision ratios
  
• Appendix 4A, Set of individual test results
  
• Appendix 5A, Formula for document ranking based on probability considerations
  G. H. Stearman
• Index
  

Factors Determining the Performance of Indexing Systems. Part 2, Appendices

Cyril W. Cleverdon, Jack Mills, E. Michael Keen
The College of Aeronautics, Cranfield, England, 1966

• Introduction to appendices
  
• List of Index Terms
  
• First letter and enclosures sent to authors of research papers
  
• Examples of various computer print-outs
  

Information Retrieval Experiment

Karen Sparck Jones
Butterworths, 1981

• Preface
  
• Chapter 1, Introduction
  Karen Sparck Jones
• Part 1, Testing in general
  Karen Sparck Jones
• The methodology of information retrieval experiment
  Stephen E. Robertson
• Retrieval effectiveness
  Cornelis J. van Rijsbergen
• Ineffable concepts in information retrieval
  Nicholas J. Belkin
• The pragmatics of information retrieval experimentation
  Jean M. Tague
• Part 2, Types of test
  
• Evaluation within the environment of an operating information service
  F. Wilfrid Lancaster
• Opportunities for testing with online systems
  Elizabeth D. Barraclough
• Laboratory tests of manual systems
  E. Michael Keen
• Laboratory tests: automatic systems
  Robert N. Oddy
• Simulation, and simulation experiments
  Michael D. Heine
• Gedanken experimentation: An alternative to traditional system testing?
  William S. Cooper
• Part 3, Actual tests
  
• Retrieval system tests 1958-1978
  Karen Sparck Jones
• The Cranfield tests
  Karen Sparck Jones
• An experiment: search strategy variations in SDI profiles
  Lynn Evans
• The Smart environment for retrieval system evaluation — advantages and problem areas
  Gerard Salton
• Bibliography
  
• Index
  
• Back cover
  

Document Retrieval System - Optimization and Evaluation

Joseph John Rocchio
PhD Thesis, Harvard University, Cambridge, Massachusetts, 1966

• Preface
  
• Introduction
  
• The Indexing Function
  
• Search Request Formulation
  
• The Query-Document Matching Function
  
• Evaluation of Document Retrieval Systems
  
• Appendix A, The SMART System
  

Information Storage and Retrieval: Scientific Report No. ISR-11

Gerard Salton
The National Science Foundation, 1966

• Summary
  Gerard Salton
• The SMART System — Retrieval Results and Future Plans
  Gerard Salton
• Operating Instructions for the SMART Text Processing and Document Retrieval System
  Michael E. Lesk
• SOCCER - A Concordance Program
  Guy T. Hochgesang
• Information Analysis and Dictionary Construction
  Gerard Salton, Michael E. Lesk
• Design Criteria for Automatic Information Systems
  Michael E. Lesk, Gerard Salton
• Relevance Feedback in an Information Retrieval System
  W. Riddle, T. Horwitz, R. Dietz
• A Modified Two-Level Search Algorithm Using Request Clustering
  V.R. Lesser
• An Experimental Investigation of Automatic Hierarchy Generation
  G. Blomgren, A. Goodman, L. Kelly
• On Some Clustering Techniques for Information Retrieval
  J. D. Broffit, H. L. Morgan, J. V. Soden
• Design Considerations for Time Shared Automatic Documentation Centers
  Michael E. Lesk

Information Storage and Retrieval: Scientific Report No. ISR-12

Gerard Salton
The National Science Foundation, 1967

• Preface
  
• I. The SMART project: Status Report and Plans
  Gerard Salton
• II. SIG - The Significance Programs for Testing the Evaluation Output
  Michael E. Lesk
• III. Computer Valuation of Indexing and Text Processing
  Gerard Salton, Michael E. Lesk
• IV. The Cornell Programs for Cluster Searching and Relevance Feedback
  E. Ide, R. Williamson, D. Williamson
• V. Search Strategy and the Optimization of Retrieval Effectiveness
  Gerard Salton
• VI. An Evaluation of Rocchios Clustering Algorithm
  Robert T. Grauer, Michel Messier
• VII. Information Retrieval: Dictionary Representations and Cluster Evaluation
  P.C. Leech, R.C. Matlack
• VIII. User Interaction with an Automated Information Retrieval System
  Eleanor Ide
• IX. Negative Response Relevance Feedback
  John Kelly
• X. A Relevance Feedback System Based on Document Transformations
  S.R. Friedman, J.A. Maceyak, S.F. Weiss
• XI. An Experiment in the Use of Bibliographic Data as a Source of Relevance Feedback in Information Retrieval
  M. Amreich, G. Grissom, D. Michelson, E. Ide
• XII. The Evaluation Problem in Relevance Feedback Systems
  Harold A. Hall, Nelson H. Weiderman

Information Storage and Retrieval: Scientific Report No. ISR-13

Gerard Salton
The National Science Foundation, 1968

• Preface
  
• I. Test Environment
  E. Michael Keen
• II. Evaluation Parameters
  E. Michael Keen
• III. Search Matching Functions
  E. Michael Keen
• IV. Correlation Measures
  K. Reitsma, J. Sagalyn
• V. Document Length
  E. Michael Keen
• VI. Suffix Dictionaries
  E. Michael Keen
• VII. Thesaurus, Phrase and Hierarchy Dictionaries
  E. Michael Keen
• VIII. An Experiment in Automatic Thesaurus Construction
  R.T. Dattola, D.M. Murray
• IX. Word-Word Associations in Document Retrieval Systems
  E. Michael Keen
• X. An Analysis of the Documentation Requests
  E. Michael Keen
• Appendix A, Recall-Precision Tables
  
• Appendix B, Request Texts
  

Measures and Averaging Methods used in Performance Testing of Indexing Systems

E. Michael Keen
The College of Aeronautics, Cranfield, England, 1966

• Preface
  Cyril Cleverdon
• Measures of Retrieval Performance
  

Evaluation of the MEDLARS Demand Search Service

F.W. Lancaster
U.S. Department of Health, Education, and Welfare, 1968

• Preface
  
• Design and Execution of the Evaluation Program
  
• The Test Results
  
• Conclusions and Recommendations
  
• Appendices
  

Automatic Indexing: A State-of-the-Art Report

Mary Elizabeth Stevens
National Bureau of Standards Monograph 91, 1965

• Foreword
  A.V. Astin, Lewis M. Branscomb
• 1. Introduction
  
• 2. Indexes Compiled by Machine
  
• 3. Indexes Compiled by Machine - Automatic Derivative Indexing
  
• 4. Automatic Assignment Indexing Techniques
  
• 5. Automatic Classification and Categorization
  
• Other Potentially Related Research
  
• Problems of Evaluation
  
• Operational Considerations
  
• 9. Conclusion Appraisal of the State of the Art in Automatic Indexing
  
• Appendix A: List of References Cited and Selected Bibliography
  
• Appendix B: Progress and Prospects in Mechanized Indexing
  
• References
  
• Appendix C: Selective Bibliography of Additional References
  

Relevance Feedback in an Automatic Document Retrieval System

Eleanor Ide
Cornell University, 1969

• Abstract
  
• Preface
  
• I. Automatic Document; Retrieval Systems
  
• II. User Interaction With An On-Line Retrieval System
  
• III. Prior Investigations Of The Relevance Feedback Retrieval Algorithm
  
• IV. Environment Of The Reported Experiments
  
• V. Evaluation Of Retrieval Performance
  
• VI. Experimental Results
  
• VII. Recommendations Based on Present and Prior Experiments
  
• Summary
  
• Bibliography
  

Report on the testing and analysis of an investigation into the comparative efficiency of indexing systems

Cyril W. Cleverdon
The College of Aeronautics, Cranfield, England, 1962

• Preface
  
• Chapter 1, Introduction
  
• Chapter 2, Main test programme
  
• Chapter 3, Test results
  
• Chapter 4, Statistical Analysis
  
• Chapter 5, Analysis of failures
  
• Chapter 6, Supplementary test programmes
  
• Chapter 7, Testing of existing systems
  
• Chapter 8, Supplementary indexing
  
• Chapter 9, Comments on the results
  
• Chapter 10, Basic problems of information retrieval
  
• Appendix 2A
  
• Appendix 2B, Programme rules
  
• Appendix 3A, Statistical analysis of results of first two rounds of Aslib Cranfield research project
  
• Appendix 4A, 100 Questions based on source documents listed in Appendix 4B
  
• Appendix 4B, Titles of source documents for question listed in Appendix 4A
  
• Appendix 4C, Postings and results for documents P12001-P14000
  
• Appendix 4D, Correlation of terms used in 300 searches for indexing and searching
  
• Appendix 4E, Results for documents P12001-P14000 showing number of search results required
  
• Appendix 5A, Analysis of failures
  
• Appendix 5B, Examples of Analysis of Failures
  
• Appendix 6A, Letter to those assisting in compilation of bibliographies
  
• Appendix 7A, Analysis of results of test on English electric facet catalogue
  
• Appendix 7B, Analysis of failures for facet index in W.R.U test
  
• Appendix 8A, Instruction for supplementary indexers
  
• Backmatter
  

The effect of varation in relevance assessments in comparative experimental tests of indexing languages

Cyril W. Cleverdon
The College of Aeronautics, Cranfield, England, 1970

• Summary
  
• Contents
  

---

Send questions or comments to infodir_sigir@acm.org or see the SIGIR contact page.