As we saw in the previous section, there are many algorithmic steps that have an impact on the results produced by a search engine. Among these steps, the results ranking step is probably one of the most complex and difficult to understand. Fortunately, as explained in OER7, digital library search engines such as the NewsEye platform are often different from commercial search engines. They are often based on classification systems, and they provide full-text search options that allow for fine-tuning of searches.

It is not necessary to understand all the details of these methods, which are reserved for computer scientists. However, knowing the most general principles allows you to be aware of the biases. It can help you develop a more efficient querying strategy, particularly if the data are difficult or impossible to categorize by subject, which is the case of historical newspapers. For example, we will see that the search engine does not necessarily take into account the order of the words in a query. Thus, using the syntax that allows one to search for words in a precise order can be essential.

Let’s define the entities that are concerned with the ranking problem.

• A user (U) has an information need (I).
• He/She wants to identify a set of relevant documents (D) among a collection (C).
• Every document is composed of a list of terms (T).
• The first step for the user is to translate his/her information need into a language that the system can understand, that is, create a query. Such a query is often constituted of a list of keywords that the search system will try to match to the content of documents in the collection.

Furthermore, to measure the performance of an information retrieval system and to understand its effects, we need appropriate measures. More specifically, we need to be able to compute how well it can match documents against queries. The goal is not to make an absolute evaluation of the system performance but rather to be able to compare different systems or different parameters. After a query, a set of matching documents is returned (matching according to the system, of course). We want to evaluate how many of these documents really are relevant and how many relevant documents were missed by the system. We often use the following terms when evaluating this.

• True Positive (TP): number of relevant documents returned by the system
• False Positive (FP): number of non-relevant documents returned by the system
• True Negative (TN): number of non-relevant documents not returned by the system
• False Negative (FN): number of relevant documents not returned by the system.

Let's now present some examples of information retrieval models to better understand how a search engine works.

An IR model can mathematically represent the entities at play in an information retrieval task. That is the user need (a query) and how relevant documents are selected and ordered. The first model to have been studied is the most simple: the Boolean Model. In this model, document d is represented as a set of terms t_i${\mathrm{<br>}}_{}$. Imagine, for example, a collection C$\mathrm{<br>}$ composed of a set of documents as follow:

A boolean query can then be expressed using the terms t_i and a list of boolean operators: AND ($\wedge$), OR ($\vee$) and NOT ($\mathrm{\neg }$). For example, the following query will match documents ${\mathrm{<em>d₂</em><br>}}_{}$ and ${\mathrm{<em>d₃</em><br>}}_{}$.
This model is quite simple but has many drawbacks. First, it is not always easy for users to express boolean queries, which can sometimes be quite complex. Moreover, there is no relevancy order in the resulting documents: either the document matches the query or it does not. The Boolean Model can still be useful in some cases but is still too coarse, especially when dealing with unstructured textual documents such as historical newspapers.

The next model we will study is the Vector Space Model. Its basic idea is to represent documents as a vector of terms in the vector space generated by the list of terms ${\mathrm{<em>t_i</em>}}_{}$ in the whole collection (where one term corresponds to one dimension in this space). For example, in the collection $\mathrm{<em>C</em><br>}$ defined previously, there is a total of 6 terms, thus generating a vector space of 6 dimensions. The terms in every document and every query are associated with a weight w representing their importance. This weight is computed using statistical methods, the most common one being TF-IDF. The Vector Space Model is considered a Bag-of-Words model because the order of terms in the documents does not matter. Using this representation, the document "the cat eats the mouse" has the exact same representation as the document "the mouse eats the cat". This can appear as a huge drawback, but in many cases, it is often enough to extract relevant documents from a collection. This flaw can be alleviated using different analysis methods like n-grams, for example. To sum up, the previously defined collection C can be represented as the following matrix, where each row represents a document in our collection.


The weights w_i,j can be computed using TF-IDF.

The idea behind TF-IDF is to associate a weight with each term according to its importance in the document. Two aspects are considered to evaluate the importance of a term:

• Term frequency (TF): the more a term appears in a document, the more important it is. There are several ways of computing this (TF computations). The most intuitive one is just to count the number of times a term t,i appear in a document. Other computation methods exist, for example, to normalize this frequency against the length of the document. However, we can’t just take into account term frequency as a measure of the importance of a word because a lot of terms are very common but do not convey much information (articles and pronouns are often quite frequent in every document, for example). One way of alleviating this is to consider how rare a term is in the entire collection; this is the inverse document frequency.
• Inverse document frequency (IDF): in most languages, most of the information is conveyed by words that are quite rare. The IDF measures how rare these words are in a collection, which is equivalent to how much information is conveyed by a term. For example, if the article ‘the’ is very frequent in a particular document (high TF), it is also very frequent in the entire collection (low IDF). Its TF-IDF weight will thus be quite low, which is a good thing because it is indeed not very important. On the contrary, if a term appears in a document but does not appear in any other document of the collection, the IDF will be high. See this page for various ways of computing the IDF.

The final weight is computed by multiplying the TF and the IDF. A query defined by the user will also be represented as a vector in the same vector space. The relevance of every document towards the query can then be computed using a vector similarity function (the most common one being the cosine distance, but others exist).

As you can see, these methods are quite complex and have flaws that can be understood without being a specialist. Most of the time, the user does not know which method is used by the search engine. It is, therefore, important to control as much as possible his search for information by using the means we described in section 2.2. Using precise terms to indicate what you want to obtain allows you to create corpora and datasets that will also be more precise and relevant.