Google Scholar peer reviewing

Many web users participate in online debates, on a large variety of topics, and the discussion may often reach very technical scientific domains. Since not everyone is an expert in every possible discipline, the reasonable use of outside sources is obviously required in order to properly argue in favor of one’s position.

In this post we will explore the possibilities offered by one of the most efficient tools available to obtain sources and research material on the web: Google Scholar, a search engine with just that precise ambition.

Google Scholar

Google Scholar was launched in 2004. Its motto, “Stand on the shoulders of giants”, is a reference to the famous quote by Isaac Newton, which means that one can only make a contribution to science by understanding the great thinkers of the past. The index of the website contains mostly online papers that have been peer-reviewed, and even though it may not be as precise and complete as professional databases, it is free and relatively easy to use. It is a powerful tool which can – and should – be used as frequently as necessary, not only by scientists and academicians, but also by the rest of us.

But scientific methodology and a great many terms and concepts associated with it (even when they are used frequently, like “theory” or “scientific study”) aren’t always clearly understood by the general public. So in this post we will also try to offer some simple explanations to help users use Google Scholar with the utmost efficiency, by focusing on the concept of “peer-reviewed articles”. We will broadly explain what they are, how to recognize them and the main methods to properly refute invalid sources in a debate.

The Peer Review process :

What is a peer reviewed article? In a word, it is the raison d’être of scientific teams. There are no unpublished scientists : even rookies usually have signed at least one peer reviewed article as part of their PhD.

Peer reviewed articles are published in scientific journals. Their “peer reviewed” denomination comes from the fact that these papers are evaluated by a committee of experts on the subject. This committee has many rights: it can ask for any data from the scientific team, it can ask for material and funds to reproduce experiments detailed in the paper, ask for modifications in the content of the paper, and, last but not least, almost always emit comments, objections and criticism which must be taken into account in order for the paper to be published. After evaluation, the paper may be accepted, rejected for scientific reasons or simply because the journal does not think the subject is interesting enough, or accepted under conditions (usually small or large modifications to the articles).

How are these experts chosen? It’s not so rare for the author of an article to suggest names. The actual method varies according to the journal. In the most common case, the editor knows competent researchers working in the field concerned by the article (since he publishes such articles), and will contact from four to six of them. Usually, the editor will choose specialists with a diverse set of skills, according to the contents of the article. For example, if the article contains very technical mathematical demonstrations, the editor may try to find a mathematician working in the field to evaluate those sections of the article.

The “reviewers” are not employed by the editor. They are simply researchers in the same field. These specialists don’t earn anything, except for exclusive access to a state-of-the-art article in their field before its publication, the feeling of prestige associated with becoming a reviewer worthy of trust by a renowned journal, the satisfaction of helping research by stopping absurd ramblings and sloppy work from being published, and sometimes valuable contacts.

Additionally, reviewers are not supposed to know each other. In practice, since there are usually very few specialists in any specific field of research, it is more than likely that they know each other. Still, they should not be coworkers from the same institution and their opinions are independent from those of the other reviewers.

Of course the system isn’t perfect. The main problem is that the opinion of a sole specialist may become too important, especially in very obscure and unpopular fields. Another problem that may occur is that several scientists representing the majority of specialists in their fields may form “clans”, and block the publication of articles that do not conform to their theories. The nineteenth century in particular was plagued by this problem and many great thinkers were silenced in their times (for example, the first doctor to suggest that having surgeons wash their hands after autopsies and before delivering babies may reduce the number of deaths during childbirth was shamed and discredited by the scientific community for suggesting that fine upper class gentlemen may be dirty).

But every journal doesn’t have the same requirements in matters of precision and exactitude. Some journals can’t be trusted blindly, whilst others are known for their very rigorous screening process. Both kinds are useful to science : the latter provide solid, reliable science that is almost unanimously endorsed by the scientific community, whether the other journals help publish more speculative but usually more ambitious content which may turn out to be actual breakthroughs, or at least give interesting ideas to fellow scientists (even though they are mostly “work in progress”, they have still been peer-reviewed and published by serious scientists, and quoting these is still a better idea than providing a link to a 5mn YouTube video, even if it was uploaded by someone who calls himself an “MIT-PhD-scientist”…).

A scientific peer reviewed article is usually divided into six parts: the “abstract” is a short summary of the contents of the article. It exposes the problem the research team has examined, and the method used to solve it. It may also propose a brief contextualization (why they tried to do it). The next part, the introduction, explains the point of the article and the applications its conclusions are supposed to enable. These two parts can be easily read by anyone, no need to be a scientist. The third part exposes the method used: this is for experts only and is usually completely incomprehensible for non initiates. The fourth part, results and discussion, deals with the results of the study: what was proven, and how reliable data is. Some of it may be confusing but it should make sense to the profane.  The fifth part is the conclusion, and, just as the introduction, is understandable by anyone.

Finally, at the bottom of the article, comes a list of references. Scientific rigor requires that each and every fact mentioned in the article should be checked. Each argument in the article is based on either a research conducted by the research team, or on a previously published peer reviewed study. Since there isn’t a lot of room in the article, sometimes the methodology isn’t completely detailed in the text but can be found in another publication. References constitute one of the most consulted parts of the article, and the only one that is absolutely mandatory.

How Google Scholar comes in handy:

Now that what makes an article an actual scientific publication is established, let’s try a simple test with Google Scholar. Let’s assume you are online, debating about, let’s say, global warming. Your opponent cites an article he claims comes from a reputed scientist who found evidence that there is no such thing. By using regular ol’ Google, you find nothing but regular newspapers commenting his speeches, posts of supporters and opponents on forums and blogs, but nothing to properly evaluate this man’s scientific credentials or the reliability of his study.

A simple test to check whether he is a fraud or an actual scientist is to run Google Scholar and type the name of the article. If it appears in the results, it doesn’t look good for you: this simple fact is in itself relevant, but not sufficient. You should also check the journal’s credentials. Some of the most trustworthy are : Elsevier and Science Direct. If the article does not show up at all, it is probable that the article is not peer reviewed.

You can also use Google Scholar to test the credibility of a specialist: if you come across the interview of a so-called « scientist » on a website, a newspaper or a TV show, you might want to be sure he or she is serious. Of course, the best case scenario would be for the specialists to mention their references (the peer-reviewed articles they have used), but it is a rare case since there is usually no time for this during in an interview. Journalists should ask specialists for their references and integrate them in the article for the two or three science geeks that might be interested, but, well…

Anyway, by looking up the name of the scientist in Google Scholar, you can find all peer-reviewed articles he contributed to, and his field of study. Scientific papers are evaluated each year by the ISI, which ranks them by impact. A weak impact rating means that the paper is relatively obscure, and may not be the most reliable source of scientific information.

The apparition of Google Scholar, and of other websites such as Scirus, CiteSeerX and getCITED, raises the question of the actual necessity of scientific papers for the scientific process. Information can now be easily shared by anyone practically for free. New forms of communication are starting to emerge, but for the moment the traditional peer review organization is still the go-to system for the scientific community.