De Facto Adoption of Blog and Wikipedia in Research Reporting

Assessing references in the review process of academic research is a common requirement. One of the criteria in the credibility of the works is to support arguments with solid and peer-reviewed resources from the literature. Having non peer-reviewed resources in the references can degrade the work and this may be a reason for rejection or retraction, later. Recently, works taking such resources as reference appear more often with increasing number of blogs and Wikipedia usage; hence, the motivation of this study is to investigate given credit to such works. As the case study, investigation is done in the huge online database of ScienceDirect. Queries to extract number of works housing such references are run per research fields classified in the database. The results show there are considerable number of examples where blogs and Wikipedia are seen as resources to be used in academic papers. This finding is limited to the case study and it is too early to declare that blogs and Wikipedia can be used as references; however, a de facto adoption exits in some research fields.


Introduction and literature survey
While doing research, one of the rationales that academicians are educated is to find primary sources of the information in the literature surveyed for their works.This stands as one of the principles in the code of conduct in the academia.When looking off the academia, one can easily notice increasing number of works published without peer-review.Obviously, such works' credibility is open to discussion; however, they are getting large in number especially with the boost of the WWW.However, it is not uncommon to see some contradictive examples in the journals with considerable reputation in the academia.With this perceived, we have primary intention of this work as to attract attention to the usage of non peer-reviewed resources in the peer-reviewed works.As this aim scopes academic publications not only the academics at the universities but also librarians are addressed as the audience.
Just to see how much such resources are used in the academic publications, we have purposed to find out number of papers taking such resources as references; hence, this work concerns with the influence of such information resources at the academic research reported to the literature.As the scope of the case study, we have chosen ScienceDirect's online available database for querying references from Wikipedia and web blogs.Before jumping to the data investigation, it is beneficial to visit, quickly, concept of blog and wiki as our case study further focuses referencing blog sites and Wikipedia as information/knowledge sources.
When it comes to the credibility among the readers, peer-review is not given credit that much.
For instance, the authors in (Johnson & Kaye, 2009), find out that the credibility of the blogs were even higher than candidate web sites and bulletin boards in the 2004 elections in USA.
In another work (Yang, 2007), the author investigates credibility of the blogs; the work takes Taiwanese news blogs as case study and concludes that readers believe and trust such blogs.
Although both blogging and contribution to Wikipedia remain non peer-reviewed, we can find academic papers taking such sources as references.This is not that much welcomed as non peer-reviewed are more likely report untrue or biased contributions to the literature; a work (Chip, 2009) underlines that about 2% of the Wikipedia users contributes 75% of the whole content; this is reported in 2009, about 8 years before our time of writing.The credibility of Wikipedia is in discussion and has been a separate topic for many works such as (Rector, 2008) where the author represents examples of wrong entries.The author randomly picks topics from Wikipedia and compares the content against Dictionary of American History and online and printed versions of Encyclopaedia Britannica.
There are also contradicting examples, for instance in (Black, 2008), the author defends the idea of counting on Wikipedia as an academic source.The author discusses examples within which knowledge can be produced off the academic places.Additionally, he argues that articles published on academic journals are addressing audience similar to the authors of these articles.The work concludes that peer-reviewing system should be discussed and reformed as number of works reported to the academic literature are far behind the development of the new ideas and they fail to be public.
While these discussions are on, Wikipedia did not stop growing thanks to its contributors and visitors.Works such as (Xiao & Askin, 2012) underline this growth and quasi academic usage and discuss challenges in recognizing Wikipedia in academic media.These challenges are not only about the credibility but also about format that would let the site to be integrated by the digital libraries; hence, such discussions give a quick sign that Wikipedia can be adopted in the academia in future.This idea is supported in and earlier work (Eijkman, 2010) where the author argues that instructors at the universities agree to use Wikipedia at some level by students; this includes start of a research or having an idea on a topic; a similar finding, using blogs, Twitter, Facebook cannot be solid sources to be used in an academic work, is reported in (Volentine & Tenopir, 2013).
Nevertheless, Wikipedia is treated as a website with a disruptive effect.There are other examples with stronger arguments such as Wikipedia has a strong power to educate, for example in chemistry (Ertl, Patiny, Sander, Rufener, & Zasso, 2015), especially in teaching chemical compounds.When it comes to referencing, Wikipedia is criticized as it is not compulsory to support the arguments there and contributors do not have to rely on peerreviewed resources.In (Serrano-López, Ingwersen, & Sanz-Casado, 2017), the authors take wind power as the scope topic and investigate Wikipedia entries on the subject.They conclude that credibility of the site is questionable.
When it comes to consider blog entries in an academic work, still, trust and credibility is questioned.For example, in (Doyle, Heslop, Ramirez, & Cray, 2012), the authors identify both blog and blogger characteristics to influence trust of the reader.Moreover, trust is not enough alone, as the question includes whether to consider blogs as valid references, or not.In (Hendricks, 2010), the author investigates this question over academic librarians; the findings reveal that blogs are given credits; however, their credibility is not as high as journal articles and they would not be easily at the same level in future because a peer-review system does not exist; the publish year of (Hendricks, 2010) is 2010 and still we do see recognized reviewing process for blogs, yet, there are reference styles to include them as reference.
Next subsections visit blog and Wikipedia as non peer-reviewed sources, quickly.The dataset, method and results are detailed in method and results section with more attention to the computers science, social sciences and business management and accounting fields; finally, conclusion section puts forward that blogs and Wikipedia are increasingly shown as reference in academic papers.

Blog
As a rule of thumb, showing references is a commonly used method to support arguments in the academic writing.The quality of references depends on the credibility hence the sources.Obviously, references with low credibility would open the support of the research into discussion.Two of the most common non peer-reviewed sources are blogs and Wikipedia.Before we question their credibility, we shall visit them to understand their nature.
Blogs, in the sense that we take for this work, are personal websites fed by individuals' writings as posts.The content is not edited, not checked; additionally, not rare that one can find articulation mistakes and/or literal errors; hence, the quality is open to discussion (Wilkins, 2008).Publishing blogs has been in presence before the booming of the www, however, the name was not coined those days; simple platforms such as USENET, CompuServe to share ideas and talks have started with the Internet, mid 1980s; for instance, mod.ber newsgroup by Redmond over USENET can be given as an early example in 1984 (Schwartz, Ranlett, & Draper, 2009).
The popularity of blog is well known for the last 13-15 years.Today, weblogging is expected to have chronological order of writing.Basically, volunteers are posting their comments/ experiences on different topics on the Internet and some of are achieved with high similarity to chronologically published journals/magazines.Blogging become a phenomenon with the increasing number of visitors on such sites.One of the purposes of blogging is to increase visiting traffic to the site so that value of the site gets higher both in popularity and in money.
Examples include overblog.com,which lets their premium account blog owners to share revenue; in this way, the traffic of the blog site is evaluated and converted into money (Overblog, 2017).However, creating revenue is not the only motivator of a person to become a blogger.An earlier study (Hsu & Lin, 2008) investigates the reasons behind intention to blog; basically, the researchers have conducted surveys in Taiwan based on a technology acceptance, knowledge sharing and social influence factors model; their results show that people find blogs easy to publish; additionally, blog followers give credit to enjoyment, fun, curiosity, and exploration hence hosting services are recommended to support the blog owners with related tools.
As the number of blogs are getting higher hence the ideas, knowledge and comments, information spreads considerably fast over the blogs; hot topics are open for commenting as well as making "gossips".In (Kazama, Imada, & Kashiwagi, 2012), the researchers investigate the diffusion of information among the blogs to understand the dynamics of the spread and to have better idea about the information source.As seen from this example, the blogosphere is seen as a huge place to collect wisdom of crowds.For example, in (Noh, Park, Park, & Lee, 2010), the authors propose a system that searches blogs to collect emergent knowledge from social web data.
While the academic journals and conferences are seen as places for exposing and discussing academic findings, researchers from academia are posting to blogs as well.In (Bonetta, 2007), about a decade ago, the author highlights that academic staff from universities with considerable traffic on their blogs, find the platform a suitable place to discuss their ideas and even for education.In another work (Kim, 2008), the power of blog is acknowledged and audience is attracted attention to discussions on replacing computer-mediated communication applications such as e-mail, by blogs, which did not happen yet.Nevertheless, the scientists have started and continue to blog; one of the reasons can be discussed as, academic writing rules and articulation and scientific supports are not necessary followed in blog publishing (Wilkins, 2008).

Wikipedia
Wikipedia is the most commonly known example of the wikis on the Internet.Basically, it is a big sac of definitions and explanations of potentially anything on everything.This extensive content of the Wikipedia is due to its nature of being publicly available for both contribution and editing.Its origin lays back to the end of 1999 with the idea of creating a web based encyclopedia by Jimmy Wales and Larry Sanger (Anderson, 2011).The site has started under Nupedia name and had a protocol of peer-review process.Later, same people have started Wikipedia with the same idea of web-based encyclopedia but this time, the site would not bother peer-review processes.This critical difference has made Wikipedia to grow rapidly; any volunteer can make or edit any entry.On the other side, this flexibility has attracted not only vandals but also so-called contributors with questionable knowledge.This effect has been in discussion also in the early ages of Wikipedia as reported examples in (Sidener, 2006).
Checking Wikipedia has become a frequent way of looking for information, quickly.However, this practice is also warned in the universities such as in Harvard University (Harvard College Writing Program, 2017).The students are given examples where fictionist information can be put easily.

Method and results
We have queried the database for the last 12 years, between 2005 and 2016.We have chosen this time interval because prior to 2005, taking blogs and Wikipedia as sources in academic papers is insignificant and 2017 is not completed at the time of writing.As mentioned in the introduction section, the ScienceDirect's site has extensive advanced search tools for basic, advanced and expert querying that allows us to pinpoint papers at niche; hence, this database is eligible for our work in both the volume and extensive query tools.
The works we have considered as dataset are listed on the A more detailed table is at Appendix 1 where the reader may find number of publications per year per field; apparently, there are papers fitting into more than one research field; for this reason, numbers presented at the Appendix are not to be read as number of unique publications.Nevertheless, medicine and dentistry field has the first rank in the total number of publications.
Over this raw dataset, we have investigated further, to find blog/Wikipedia references per research field.We revisit the table at the analysis section for further detail.
As we have mentioned earlier, we have chosen ScienceDirect database to retrieve data.The database has extensive tools to enable us to run our queries in searching not only title, abstract and keywords but also references.For each research field on Appendix 1, we have excluded books; so, these numbers do not include books.We have prepared graphs for the followings:

•
Publications that use blogs as reference but not researching on blogs  Where, "Research Field" refers to, for instance, "computer science", for which we ran the query.
At figures 1 and 2, the results are represented graphically.The results on Wikipedia reference show similar but slightly different figure.Engineering, computer science, medicine and dentistry, social science and energy fields prefer Wikipedia as reference more than the rest while veterinary science and veterinary medicine, immunology and microbiology, neuroscience and psychology fields are among the most conservative fields that do not take Wikipedia as reference.
Apparently, a general rise in both blog and Wikipedia references is remarkable.After this result, we have been motivated to check number of publications that research on or whose research include blogs and Wikipedia.Basically, we ran the following queries to identify papers with title, abstract or keyword that include blogs and Wikipedia.

TITLE-ABSTR-KEY(blog)[All Sources(Research Field)]
TITLE-ABSTR-KEY(wikipedia)[All Sources(Research Field)] Similarly, "Research Field" is the field of research we ran the query for; the figures 3 and 4 shows the results, graphically.Recalling the figures 1 and 2, the number of publications taking blogs and Wikipedia as reference can be seen incomplete if we do not consider their ratios over the total number of publications per field.For this reason, we have checked these ratios and presented on figures 5 and 6.Basically, Figure 5 present percentages of publications with blog references over total number of publication for each field.Similarly, Figure 6 is presents for Wikipedia.Similar to blog case, medicine and dentistry shows low percentage despite the number is high in Figure 2; at its peak in 2013, medicine and dentistry field's ratio of Wikipedia reference is %0.093.
As the figures 1-6 attract more attention to the computers science, social sciences and business management and accounting fields, we have focused more on them.The usage of both blogs and Wikipedia as reference in these three fields are increasing in general while social sciences and business management and accounting fields are at steady level in recent years in taking Wikipedia as reference.Researching on blogs and Wikipedia is not only low in number but it does not show any sign of increase, either.

Conclusion and Discussion
Including a reference from Wikipedia or from a personal blog page may not be disastrous depending on the purpose of this reference; for instance, such a web page may be given as an example to the subject in research.However, the results show that there is a strong signal in the adoption of such references while academic research requires solid and reviewed resources.
A considerable difference between blogs and Wikipedia is that you may know the writer of a blog however Wikipedia's authors remain anonymous.One can get in touch with the writer and start a discussion hence can assess the knowledge/experience of the source.However, this way is still far from an acceptable peer-review process.When it comes to the assessment of the articles in Wikipedia, it is not obligatory to give reference and the writer can be unreachable.
A misinformation can be corrected by another person however this is open to discussion of when and to what extent it can be "corrected".
Although both blogging and contribution to Wikipedia remain non peer-reviewed, we can find academic papers taking such sources as references.It is too early to classify such papers depending on how these references are included in the research because these references are used for several purposes including to refer definitions or examples.
We believe that trusting non peer-reviewed sources may open discussions on the reliability of the produced work even if it is accepted by an indexed or with reputation journal.The discussion becomes more considerable while considering ethics in sharing knowledge on open platforms in blurred code of conduct.Our analyses show that such resources are used hence given credit in the academia; however, it is far below a dominant level for now; it is still too early to make an accurate projection but infiltration of such resources to the references is promising to continue increasing.
Blogs and Wikipedia are taken as examples in this study however, other non peer-reviewed resources such as other wikis and forums are also used as references.A quick example is reported with (Budge, Lemon, & McPherson, 2016) where micro blogs by academicians on Twitter are investigated.The picture gets even bigger when we include video streaming sites such as the popular YouTube.This brings a further research question of, shall we increase barriers in reviewing references as referees, which is left for a future work.Another limitation comes with the scope, which takes research areas in general but not topics, specifically.Apparently, a more focused follow-up research that takes just certain topics may reveal more about the validity, reasons and consequences of taking blogs and Wikipedia as reference.
(references (http PRE/1 wikipedia) OR references (www PRE/1 wikipedia) OR references (wikipedia PRE/1 org)) AND NOT Abstract (wikipedia) AND NOT Title (wikipedia) AND NOT Keywords (wikipedia)[All Sources(Research Field)]

Fig. 1 .
Fig. 1.Publications that do not research on blogs but take blogs as reference

Fig. 2 .
Fig. 2. Publications that do not research on Wikipedia but take Wikipedia as reference

Fig. 3 .Fig. 4 .
Fig. 3. Publications that take blogs as part of research

Fig. 5 .
Fig. 5. Percentages of publications with blog reference