Lecture on the new Journal ranking metrices

1. Journal Ranking Metrics
new perspective in Journal performance
management
Professor Aboul Ella
Hassanien
Cairo Univesrity
Faculty of Computers and Information
Chairman of Scientific Research Group in Egypt (SRGE)
http://www.egyptscience.net @ Aboitcairo@gmail.com

2. Using the Impact factor alone to judge
a journal is like using weight to judge
a person health
The title is a sentence taken from a report on citation
statistics prepared by the International Mathematical Union (IMU).
Another such take-home message is "Research is too important to
measure its value with only a single coarse tool." Given that citation
statistics are heavily used in assessing research and researchers,
young scientists might gain a lot from investing some time to
familiarize themselves with this subject.

3. Science is a game
 Science is a game and the
best players appreciate not
only the beauty of a
discovery by a precise and
logical reasoning, but also the
taste of making a guess
 There is no way to get
experience except through
experience

4. Writing a paper is …
 A lot like chip
design
 You need
 Conceptualization
 Floor planning and
layout
 Interconnections
 Testing

5. Why write and publish research
papers?
Ideally
 to share research
findings and
discoveries with the
hope of improving
some things.
Practically –
 to get funding
 to get promoted
 to get a job
 to keep your job!

6. Academic publishing
 Academic publishing describes a system that
is necessary in order for academic scholars to
peer review the work and make it available
for a wider audience

7. Multiple Trends Impact The Research
Landscape
Competition / Collaboration
Accountability /
Government policies
Improve
research
outcomes
Multidisciplinarity
High mobility
Research
Strategy
&
Research
Performance

8. What makes a good
research paper?
 Good science
 Good writing
 Publication in good journals

9. Deadly Sins - Plagiarism
"Plagiarism is defined as the act of taking the specific substance
of another and offering it as one's own without giving credit to the
sources.

10. Major Metrics: Research Impact
 Publication count
 Number of publications produced by individual,
school or university
 Citation count
 Number of times publication is cited by other
publications

11. Major Metrics: Journal quality
Impact factor –
 average number of times
published papers are cited
up to two years after
publication
Immediacy Index
 average number of times
published papers are cited
during year of publication.

12. Journal Evaluation
Some challenges
 I’m a publishing author in a niche
area, how can I compare with
researchers in another scientific field?
 How can I get a value that reflects
citation behaviour in my research
area?
 It’s 2010 – why can I only get a
ranking relevant to 2008?
 Where can I find the data that have
been used to generate this number?
 The Impact Factor does not cover the
journal I’m publishing in. What now?
 Difficult to compare
 A Not recent enough
 Lack of clarity of data
origin
 Lack of coverage

13. Journal Impact Factors,
Immediacy Index and
the Author h-index:
tools used for benchmarking the quality of
research

14. Journal Impact Factors:
Why?
 Evaluate the scholarly worth of a journal
 Rank journals within a discipline
 Help you decide where to publish your article for
maximum impact
 Evaluation for promotion / tenure / grants, or in
some countries, even government funding of an
institution
 Frequently used as an evaluation source by
librarians during journal cancellations or new
purchases

15. Journal Impact Factors:
Why?
 In Egypt, researchers in Cairo University are
rewarded for publishing in journals defined
by ISI as prestigious
 In England, hiring panels routinely consider
impact factors
 By Spanish law, researchers are rewarded for
publishing in journals defined by ISI as
prestigious
 In China, scientists get cash bonuses for
publishing in high-impact journals. In some
schools, physics students must publish at least 2
articles with a combined Impact Factor of 4 to
get their PhD

16. Journal Impact Factors:
Where?
 Impact factors are listed in Journal Citation
Reports JCR
 You can easily get to the JCR from the Web of
Science, so let’s start there, since understanding the
Web of Science will help us better understand where
the data for the JCR comes from.
Tip: Use Web of Science for article-level
information; JCR for journal-level information. Both
the Web of Science and the JCR are based on the
same database of journal citations and cited
references

17. Web of Science®
Thomson Reuters (formerly ISI) has one, huge
database, Web of Science, that…
 Indexes selected journals
> 8,000 science; > 3,000 social science journals;
> 1,800 Arts & Humanities
 Tracks “cited references” and “times cited”
 Sample topic/author search: “impact factor”
 Activity: Search for an article in your field that
has been highly cited.

18. Journal Citation Reports
 Journal Citation Reports (JCR) keeps track of citations at the
journal level (same data as in Web of Science, just a different
presentation of the data). A new edition comes out every Spring.
Tip: In addition to accessing the JCR via a Web of Science
citation, you can access the JCR from our Online Journals &
Databases resource: openurl.library.uiuc.edu/sfxlcl3/az?
 Activity: Get into the JCR and then:
 Choose the Science or Social Sciences edition, and year
 Search by individual journal title, or by subject category
 Example:
 Subject: Biochemistry & Molecular Biology
 Then, “View Journal Data”, sorted by Impact Factor

19. Journal Impact Factors:
What?
What is the Journal Impact Factor?
How is it calculated?
Only references in articles within the MORE than 13,000 journals
indexed in Web of Science are counted; does not include citations
that may cite the articles in Cell from book chapters, proceedings,
or other journals that are not indexed in Web of Science
Citable articles are just research articles and reviews – not news
articles, commentary, etc.

20. 20
Impact Factor =
Cites in 2009 to 2007 or 2008 papers
Papers published in 2007 or 2008
The impact factor is calculated by dividing the number of
citations in the JCR year by the total number of articles
published in the two previous years.
Impact Factor Calculation
Journal: ACADEMY OF MANAGEMENT JOURNAL

21. 21
The immediacy index is calculated by dividing the number
of citations to articles published in a given year by the
number of articles published in that year.
Immediacy Index Calculation
Journal: ACADEMY OF MANAGEMENT JOURNAL

22. EFFICIENCY JOURNAL IMPACT
FACTOR
2008 Impact
Factor
200820072006
Source paper – published in 2008
Cited reference – published in 2006 or
2007
Citations
All
Previous
Years
2005 2009

23. 23
Implications of Impact Factor
 Tell us how frequently has the average article in a journal
been cited in a particular year.
 Tell us something about a journal as a whole
e.g. the extent to which its recently published papers were
cited in a given year.
 Impact factor > 1 implied a journal is frequently cited
 Higher citations rate means your article has higher
chances of getting cited or read by researchers.
 Tells us NOTHING concrete about any specific paper or
specific author.

24. Implications of Impact Factor
 Only a limited subset of journals is indexed by ISI
 Only uses the articles cited by the ~13,000 “ISI journals”
 Some disciplines are especially poorly covered
 Biased toward English-language journals
 ISI has recently added several hundred non-English journals
 Short (two year) snapshot of journal
 Some disciplines use older material more or take time to cite new
research
 JCR now also includes the 5-year data
 Is an average; not all articles are equally well-cited

25. Implications of Impact Factor
 Includes self-citations, that is articles in which the
article cites other papers in the same journal
 Only includes “citable” articles in the denominator of
the equation, i.e., articles and reviews
 Editors may skew IF by increasing the number of review
articles, which bring in more citations (increases the
numerator)
 Or by increasing the number of “news” items (e.g., Science,
general medical journals) , which are cited (appear in
numerator) but not considered “citable” (and so aren’t in the
denominator)
 It is expensive to subscribe to the JCR
http://www.slideshare.net/danrholden/bibliometrics-
12605810?next_slideshow=1

26. Publish or perish (POP)
 Publish or Perish is a free analysis software that retrieves and
analyzes academic citations.
The results are available on-screen and can also be copied to the Windows clipboard
(for pasting into other applications) or saved to a variety of output formats

27. Download and install Publish or
Perish
 http://www.harzing.com/pop_win.htm

28. Google Scholar disadvantage

29. Scopus

30. Scopus - Most comprehensive Abstracts and
Citations database
Increased
prestige,
ranking and
funding
because you . .
.
. . . improve
your
institutional
productivity . .
.
. . . by
enabling the
management
to make
better
decisions . .
.
. . . and
improving
student and
researcher
productivity
. . .
…because
Scopus (a)
enables you to
evaluate
performance
and quality
accurately . .
.
. . . (b)
offers you
the most
efficient
and
effective
search . .
.
. . . and (c)
provides
access to the
most
comprehen
sive
content . . .
. . . thereby
unlocking the
library’s full
potential
enabled by . . .
Best evaluation
of performance
and quality:
• Broadest
coverage of
peer-reviewed,
high quality
journals
• Globally as well
as locally
• More accurate
citation count,
fairer
recognition for
work
Improve
productivity of
Researcher and
Student:
• Most efficient
and effective
search
• Access to most
comprehensive
content
Secure institutional
success . . .

31. Scopus vs Web of Science

32. http:/www.slideshare.net/danrholden/bibliometrics-
12605810?next_slideshow=1

33. h-Index…
for Evaluating Authors (or Journals)
 Proposed by JE Hirsch as an index to quantify an
individual's scientific research output
Remember, Impact Factor is just for journals, though it’s often used to evaluate
authors.
 Combines an assessment of both quantity (number of
papers) and quality (impact, or citations to these papers).
 h-index is automatically calculated:
 Web of Science
 Scopus
 Publish or Perish (free download), based on data in Google Scholar
 h-index can also be manually calculated for an author
based on the number of papers authored and the number
of times each paper has been cited.
 See Wikipedia article for overview of h-index including
criticisms, alternatives http://en.wikipedia.org/wiki/H-
index

34. Determining h-index Manually
From h-index,
http://en.wikipedia.org/wiki/H-index

35. Finding a h-index value in
Web of Science

36. Scopus new bibliometric indicators
to measure the influence of journals
 IPP (Impact per Publication)
 The Impact per Publication measures the ratio of
citations per article published in the journal.
 SNIP (Source Normalized Impact per Paper)
 Source Normalized Impact per Paper measures
contextual citation impact by weighting citations based
on the total number of citations in a subject field
 SJR (SCImago Journal Rank).
 SCImago Journal Rank is a prestige metric based on the
idea that 'all citations are not created equal'. With SJR,
the subject field, quality and reputation of the journal
have a direct effect on the value of a citation. It is a
size-independent indicator and it ranks journals by their
'average prestige per article' and can be used for
journal comparisons in the scientific evaluation process

37. SJR (SCImago Journal Rank).
 Developed by Professors Félix
de Moya, Research Professor at
Consejo Superior de
Investigaciones Científicas and
Vicente Guerrero Bote at
University of Extremadura,
 SCImago Journal Rank (SJR) is
a prestige metric based on the
idea that 'all citations are not
created equal'.
 With SJR, the subject field,
quality and reputation of the
journal have a direct effect on
the value of a citation.
 SJR is a measure of
scientific influence of
scholarly journals that
accounts for both the
number of citations
received by a journal
and the importance or
prestige of the journals
where such citations
come from.
 It is a variant of the
eigenvector centrality
measure used in
network theory.
Research Paper: "The SJR indicator: A new indicator of journals' scientific prestige"

38. SJR (SCImago Journal Rank).
 The SCImago Journal &
Country Rank is a portal
that includes the journals
and country scientific
indicators developed from
the information contained in
the Scopus® database
 These indicators can be
used to assess and
analyze scientific
domains.

 This platform takes its name
from the SCImago Journal
Rank (SJR) indicator,
developed by SCImago from
the widely known
algorithmGoogle
PageRank™. This indicator
shows the visibility of the
journals contained in
the Scopus® database from
1996.

39. IPP (Impact per Publication)
 The IPP measures the ratio of
citations in a year (Y) to
scholarly papers published in the
three previous years (Y-1, Y-2,
Y-3) divided by the number of
scholarly papers published in
those same years (Y-1, Y-2, Y-
3). The IPP metric is using a
citation window of three years
which is considered to be the
optimal time period to accurately
measure citations in most subject
fields.



40. SNIP (Source Normalized Impact
per Paper)
 Source Normalized
Impact per Paper
measures contextual
citation impact by
weighting citations
based on the total
number of citations
in a subject field.
http://www.sciencedirect.com/science/article/pii/S17511577100000
39

41. SNIP/IPP
 Source-Normalized Impact per Paper (SNIP) measures
contextual citation impact by weighting citations based
on the total number of citations in a subject field.
 This unique aspect allows direct comparison of sources in
different subject fields. The impact of a single citation is
given higher value in subject areas where citations are
less likely, and vice versa.
 It is a ratio, with a numerator and a denominator. SNIP's
numerator gives a journal's impact per publication (IPP).
This is simply the average number of citations received
in a particular year (e.g. 2013) by papers published in
the journal during the three preceding years (e.g. 2010,
2011 and 2012).


42. SNIP/IPP
 SNIP's denominator, the Database
Citation Potential (DCP) is calculated as
follows.
 We know that there are large differences in the
frequency at which authors cite papers between
various scientific subfields. In view of this, for each
journal an indicator is calculated of the citation
potential in the subject field it covers. This citation
potential is included in SNIP's denominator.


43. SNIP is IPP divided by DCP.
 It aims to allow direct comparison of sources in
different subject fields. Citation potential is shown to
vary not only between journal subject categories -
groupings of journals sharing a research field - or
disciplines (e.g., journals in Mathematics,
Engineering and Social Sciences tend to have lower
values than titles in Life Sciences), but also between
journals within the same subject category. For
instance, basic journals tend to show higher citation
potentials than applied or clinical journals. Likewise,
journals covering emerging topics tend to be higher
than periodicals in classical subjects, or more general
journals.

44. SNIP
 SNIP helps authors to identify which journals
are performing best within their subject field
and where to publish. Mouse over the circles
in the visualization and click on the years to
view the metrics. The size of the circles are
compared to the highest values in the 5-year
range. This highest value has a closed circle
whereas the open circles indicate the value
compared to this highest value.


45. Example

46. References
 http://journalinsights.elsevier.com/journals/003
1-9422/snip
 http://www.sciencedirect.com/science/article/pii
/S1751157710000039
 http://www.harzing.com/pop.htm#about
 http://www.journalmetrics.com/ipp.php
 http://pitt.libguides.com/c.php?g=12107&p=64
718
 http://lib.guides.umd.edu/content.php?pid=569
182&sid=5022406