Always keep in mind that, as the old pessimist philosopher Arnold Schopenhauer stated, “The truth will set you free–but first it will make you miserable.”
The RICE Method of Evaluation
Use the RICE method can help you decide on how to respond to information or intelligence:
- R for reliability. The basic truthfulness or accuracy of the information you are evaluating.
- I for the importance of the data based upon its relevance
- C for the cost of the data (and your possible reactions or actions relating to the information)
- E for the effectiveness of your actions if based upon this information. Would actions based upon this information solve the problems you face?
This evaluation format is useful for summarizing collected data and for analyzing how you might apply the data in a broad range of situations. However, it does not delve into specific metrics used to evaluate a particular piece of information. The next article will provide 13 evaluation metrics to help you do that.
The term metrics means in this context: a system of related measures that facilitates the quantification of some particular characteristic.
Reliability is the basic truthfulness or accuracy of the information. When evaluating data from journals you may not have the technical knowledge to evaluate the content itself. However, you do have the ability to compare the date of publication to the ownership of the journal. You also have the ability to compile a list of the author’s previous journal articles, their date of publication, the names of the journals and the publisher or owner of the journal. You can also identify conferences that the author attended or at which he was a speaker. You can then determine who ran or supported these conferences.
Predatory publishers often create or sponsor conferences to showcase their authors who may pay to be published. The conference may be part of a paid for publication package. Sometimes, they advertise a conference and take in attendance fees, and then the conference never occurs.
It is sometimes difficult to link a conference to a predatory publisher. You have to look at the page source code and the conference domain registrations and the list of speakers to tie the conference to a predatory publisher.
Cost is an important factor to consider from two perspectives. First, the obvious cost/benefit relationship or more precisely, the question, is this worth the cost? Second, if the data seem too good to be offered free, then you have to ask, why is this free?
As you now know, open access journals don’t make their money from subscriptions. You must uncover and then evaluate the relationship between how the journal makes its money and the authors who produce the content. Of course, this evaluation metric is not unique to academic and open access journals.
Part 5 will discuss a more detailed evaluation matrix.