Evaluating information & data: Evaluate search results

You have searched for relevant literature, for instance in Scopus. You now have a manageable selection of publications for the purpose of your assignment (e.g. more than 10 titles and less than 50).

How do you select the best resources in the results list? Which publications should you read first and which publications can you leave unread for the time being or maybe even ignore? Keep in mind the focus of your research questions.

Review the short list on the basis of:

• The titles. How relevant is the title for your research topic?
• Check for your selection of publications (in the left column of f.i. Scopus) the subject terms, keywords and when available the subject area (academic disciplines, interdisciplinary study). You may decide to discard titles that do not seem relevant to your topic. You may also discover angles to your research problem that you were not aware of.
• View the abstract of each publication. Does it provide (partial) information that would help to answer your research question? (see also the video How to read a scholarly article).
• Scopus and also Google Scholar provide you with the number of times a publication has been cited (cited by). In combination with the publication date this will tell you how much attention the publication has attracted and is an indication of relevance to the academic debate. Do bear in mind this is not synonym with quality.
• Apart from relevance to your research questions and to the theoretical developments (Coverage and Relevance), you may decide to limit your scope to peer reviewed publications. Peer-reviewed publications have undergone a quality check prior to publication by experts in the field.

Watch the video Peer Review in 3 Minutes (3:15) to find out more about the peer-review process.

Searching for peer reviewed articles and books might accidently take out work which will be useful for your research, because even esteemed scholars publish non-peer reviewed articles and books. Let’s take for example Noble Laureate Sir Roger Penrose. If we search for his work in sEURch we find 2600 results ranging from articles to books. But if we turn on peer review only for the search, the results are cut down to 400. So, by looking for only peer reviewed materials, we cut out well more than half of the work of a Noble Laureate, and this is something you need to consider when setting up your searches.

In the exercise How to evaluate your search results you rank publications according to three important criteria with a ranking form. 

Email the Information skills team