Bibliometrics

Showcase

Here are some examples of projects by type of activity or need expressed by customers

MEASUREMENT AND BENCHMARKING OF SCIENCE AT NATIONAL AND INTERNATIONAL LEVELS

Bibliometric Analysis for the Expert Panel on the State of Science and Technology in Canada (2011)

Client: Council of Canadian Academies (CCA)

Description of project: The Council of Canadian Academies (CCA) mandated Science-Metrix to produce bibliometric data that would update and expand the bibliometric analyses from its 2006 report on the state of science and technology (S&T) in Canada. The major theme being addressed by an Expert Panel established by the CCA relates to determining the current state of S&T in Canada. The following questions were developed to explore this theme:

  • Considering both basic and applied research fields, what are the scientific disciplines and technological applications in which Canada excels?
  • How are these strengths distributed geographically across the country (by provinces)?
  • How do these trends compare with what has been taking place in comparable countries?
  • In which scientific disciplines and technological applications has Canada shown the greatest improvement/decline in the last five years?
  • What major trends have emerged?
  • Which scientific disciplines and technological applications have the potential to emerge as areas of prominent strength for Canada?

To answer these questions, Science-Metrix prepared timeline measures of the quantity, specialisation, quality and impact of research (based on publications) and innovation (based on patents) performed by countries, with a particular emphasis on Canada and its provinces. The analyses were performed at the field and subfield levels (patent classes were matched as much as possible to scientific disciplines), and additional dimensions of research were investigated, such as scientific collaboration and the cross-country mobility of Canadian researchers. To better rank Canada’s performance by scientific area and to better benchmark its performance relative to other countries, Science-Metrix used its new approach to the multicriteria ranking of alternatives.

During this project a large set of bibliometric data was produced. The performance of Canada relative to other leading countries in the world for all fields of science, the performance of the provinces and territories and the performance of countries and provinces regarding patents were assessed using the following indicators:

  • Number of papers (full and fractional counts);
  • Average of relative citations (ARC);
  • Average of relative impact factor (ARIF);
  • Number of international and inter-provincial collaborations;
  • Growth index (GI);
  • Collaboration index (CI); and
  • Collaboration affinity of Canada towards other countries
  • Collaboration affinity of other countries towards Canada;
  • Number of patent
  • Specialization index;
  • Flow of IP.

A composite indicator combining four aforementioned indicators (Number of papers in full counting, ARC, ARIF and CI) was also calculated to assess the relative performance of leading countries, which were then ranked based on the indicator. An identical weight was awarded to each of the four indicators. In addition, a graphical presentation of a collaboration network was produced to examine existing connections between leading countries along with a positional analysis to outline Canada’s performance in 22 fields of science based on Science-Metrix’ ontology.

 

Bibliometric Study to Support the Assessment of Canadian Ocean Science

Client: Council of Canadian Academies

Description: Science-Metrix was mandated by the Council of Canadian Academies (CCA) to provide bibliometric indicators related to the Canadian performance in ocean science research as well as related sub-topics of high interest.

In its analysis, Science-Metrix provided bibliometric data for the top 25 countries and top 200 institutions publishing the most in the field of ocean science, which provided insights to the CCA regarding the positioning of Canada as well as Canadian institutions at the international level. Furthermore, the top 100 publishing Canadian institutions were also identified to provide a complete overview of ocean science research in Canada. The results of the study highlighted the high quality of the research lead in this field in Canada and in these sub-topics, with noticeable strengths identified in the sub-topics of arctic, human impact and human health research associated to ocean science.

The results from this study were incorporated in a CCA report prepared by its Expert Panel on Canadian Ocean Science. This report presents a comprehensive portrait of the state of Ocean Science research in Canada.

Read the report

 

MEASUREMENT AND BENCHMARKING OF SCIENCE FOR SPECIFIC RESEARCH PERFORMERS

Scientific Output and Collaboration of Companies Publishing the Most in the ERA (2013)

Client: European Commission

Description of project: Science-Metrix has been selected as the provider of bibliometric indicators for the European Commission’s Directorate-General for Research & Innovation (DG Research) that will be integrated into the European Commission’s evidence-based monitoring of progress towards the objectives set forth in the Lisbon framework and the post-Lisbon Strategy for the European Research Area (ERA). The bibliometric component of this monitoring system is part of a package of six complementary studies reporting on the dynamics of research activities along the whole route of knowledge, from R&D investments to publications, patents, and licensing.

This report

More specifically, this report aims to measure the scientific production of research-intensive companies within the ERA zone, portray the geographic distribution of their scientific activities, and characterise their collaboration profiles. Data is presented for the top 100 firms which published the most within the ERA zone from 2007 to 2011; this includes multinational companies with activities outside the ERA, although only their production within this zone is considered.

Standardisation of Author Addresses for Firms and Subsidiaries

The method used to identify and standardise the author addresses for the leading companies is based on two approaches. First, a matching approach was applied based on the list of firms found in the 2012 EU Industrial R&D Scoreboard. The second, and complementary, approach is based on a frequency ranking of the publication output of companies found in Scopus. This involves cleaning the preliminary lists of institutions identified as companies, beginning with those that publish the most. The merger of those two approaches results in a list of identified companies, from which subsidiaries are identified, extracted from the publication database, and associated with their parent companies.  The steps involved in these two approaches are described below.

When the addresses of leading firms and their subsidiaries have been standardised and validated via quality control processes, the unique and standardised organisation names are reincorporated into the database, where they can be used to produce robust bibliometric data and statistics at the institutional level.

Bibliometrics indicators

The production and collaboration profiles are based on the following set of indicators:

  • Number of publications
  • Number of exclusively intra-company publication
  • Number of exclusively intra-company publications with a single affiliate
  • Number of exclusively intra-company publications with multiple affiliates
  • Number of co-publications
  • Number of co-publications (Acad or RPO)
  • Number of co-publications (Firm)
  • Number of co-publications (Unclassified)

The report is primarily descriptive, focusing on the salient points relevant to the report’s two main parts: geographic distribution of the scientific production of the most publishing companies within the ERA and their collaboration patterns.

Read the report

 

MAPPING OF SCIENTIFIC COLLABORATION

Bibliometric Study in Support of Norway’s Strategy for International Research Collaboration

Client: Research Council of Norway

Description

Science-Metrix has been commissioned to provide the Research Council of Norway (RCN) with bibliometric indicators of scientific performance and collaboration for Norway and its international partners. The presentation of these indicators was organised with a view to guiding the RCN’s efforts to establish long-term institutional cooperation between Norwegian institutions and similar institutions in other countries under its International Strategy.

More specifically, the report aims at characterising the scientific production profiles and collaboration patterns of Norway and 57 selected countries in Europe and elsewhere around the world.

Firstly, the report provides a brief comparative analysis of the current scientific performance of Norway and the 57 selected countries, along with their collaboration patterns in the sciences in general. Collaboration is measured from both an international (co-authorship with any foreign country) and a bilateral (co-authorship specifically with Norway) perspective. The analysis of collaboration with Norway is also performed across the organisational sectors of Higher Education, Government, and Business Enterprise. The key questions addressed include:

  • Which are the leading scientific nations and those that co-publish most with foreign partners and with Norway in particular?
  • What has changed in the past decade — in terms of countries’ relative performance and collaboration patterns — and what is likely to change in the near future?

Secondly, the report analyses in more detail these countries’ performance and collaboration in 15 scientific themes of significant importance to Norway. This analysis is intended to provide strategic information to the RCN about the scientific partnerships that are most beneficial to the country in each strategic field. It also identifies which of those promising partnerships appear to be under-exploited with a view to providing recommendations as to potential candidates with which Norway could strengthen partnerships by thematic area.

Thirdly, the report presents further information at the micro-level (i.e., organisational level) to help the RCN target organisations — within the countries identified as potential candidates with which Norway could strengthen partnerships — in the context of setting up specific bilateral agreements. Information on the scientific performance and collaboration patterns of organisations is synthesised in the form of collaboration networks.

Finally, the report presents a detailed analysis of various types of multilateral scientific partnerships to address the following questions: Which type of scientific partnership is most beneficial to a country’s scientific impact and what are the mechanisms underlying such gains in impact? These analyses include:

  • Network analysis, which presents international collaboration networks for each of the themes selected for this study on the organisational level.
  • Analysis of the effect of multilateral co-authorship on the scientific impact of research output
  • Regional analysis
  • Cluster analysis

Constitution of dataset

For this project, data was produced, first, for the sciences in general. More specifically, only documents published in refereed scientific journals (mostly articles, reviews and conference proceedings) were retained, as these documents were reviewed by peers prior to being accepted for publication.

Data was also produced by field of science using Science-Metrix’s journal-based and mutually exclusive classification scheme. Moreover, 15 themes of strategic interest to Norway were delineated by retrieving relevant publications using keyword-based queries, specialist journals, and entire subfields from Science-Metrix’s classification where appropriate.

Data was also produced for the following regional aggregates: the Nordic countries, EU-15 countries, EU-28 countries, and the world. Because some of the included countries have large R&D systems characterised by regional differentiations and strongholds, Science-Metrix also performed a regional analysis for these countries. Data for organisations and researchers was also produced. As well, for organisations, data was produced for a number of clusters or networks of highly interconnected organisations as part of optional deliverable.

Bibliometric indicators

Several indicators were computed throughout the course of this project:

  • Number of publications
  • International co-publications
  • International co-publications with Norway
  • International collaboration rate
  • Collaboration rate with Norway
  • Specialisation Index (SI)
  • Growth ratio (GR)
  • Relative Citations (RC)
  • Relative Number of Authors
  • Average of Relative Citations (ARC)
  • Average of Relative Impact Factors (ARIF)
  • Transdisciplinarity (TD)
  • Collaboration Index (CI)
  • Affinity Index (AI
  • PageRank
  • Betweenness  Centrality
  • Share of output available in Open Access (OA)

Read the report

Intra-European Cooperation Compared to International Collaboration of the ERA Countries (2013)

Client: European Commission

Description: This report is part of the six complementaries studies tha Science-Metrix has performed for the European Commission’s Directorate-General for Research & Innovation (DG Research). The work involves the collection, analysis and updating of bibliometric data that will be integrated into the European Commission’s evidence‑based monitoring of progress towards the objectives set forth in the Lisbon Framework and the post-Lisbon Strategy for the European Research Area (ERA).

The present report

Within the FP7, a strong emphasis has been placed on sponsoring collaborative research within Europe (and beyond) as a way to increase Europe’s competitiveness within the global knowledge based economy.  In particular, the cooperation actions implemented under FP7 are foreseen to foster the achievement of complex research challenges requiring pan-European efforts, to improve research excellence in Europe, as well as to reduce the fragmentation of the European research landscape, thereby contributing to the realisation of the European Research Area (ERA)—an open space for knowledge and growth.

To support the monitoring of collaborative research and its effect under FP7, the current report first provides in depth analyses of the collaboration patterns of European countries (i.e., those included within the ERA; Israel is also covered), as well as of a selected set of international comparables within and outside of the ERA. It also investigates whether the integration of European countries has progressed since the start of FP7 both within and outside of the ERA as well as attempts to determine whether the cooperation actions implemented under FP7 appear to have triggered an increased integration of European countries by comparing the magnitude of changes both within and outside of the ERA.

Subsequently, the report characterises the effect of scientific partnerships on scientific excellence by comparing the scientific impact of various types of publications/co-publications for European countries and a selected set of international comparables. It then provides insights into whether the intensification of cross border cooperation within Europe promoted under the FP7 can be expected to increase the excellence of European research by comparing the scientific impact of various types of publications/co-publications at the aggregate level for EU27 countries. This analysis also sheds light on which types of partnerships are likely to be most beneficial to scientific excellence.

Read the report

ASSESSMENT OF RESEARCH FUNDING PROGRAMS

Bibliometric Technical Report of the Canada Research Chairs Program (2010)

Client: Social Sciences and Humanities Research Council (SSHRC)

Description: The Canada Research Chairs Program (CRCP) is a Tri-Agency program that aims to strengthen Canadian research performance in universities, as well as their affiliated institutes and hospitals, in order to position Canada amongst the world leaders in research. Science-Metrix was mandated to perform the tenth-year evaluation of the CRCP; part of this evaluation involves performing a bibliometric assessment of research funded by the program. The present bibliometric assessment is only one of the multiple lines of evidence that are used to address the following question pertaining to the program’s success:

  • What has been the CRCP’s contribution to the capacity of universities to produce new knowledge?
  • To what extent has the CRCP contributed to universities developing a comparative advantage in strategic areas of research?

The CRCP funds two groups of researchers based on their career status—namely, established researchers who lead in their field (tier 1 Chairs) and emerging researchers who have the potential to lead in their field (tier 2 Chairs). Therefore, the study provides data that will address the following questions for each group of scholars within the research areas of each of the three councils:

  1. Are CRCP-Chairholders more productive, on average, than researchers from comparable groups in terms of their number of published journal articles?
  2. Are the research outputs produced by CRCP-Chairholders of high impact relative to the average output, published in peer-reviewed journals, of comparable groups?
  3. Are CRCP-Chairholders collaborating more often (nationally and internationally), on average, than researchers from comparable groups?
  4. Has CRCP funding contributed to increasing in the scientific performance of supported researchers in terms of their number of published journal articles?
  5. Has CRCP funding contributed to increasing in the scientific performance of supported researchers in terms of the scientific impact of their journal articles?
  6. Has CRCP funding contributed to increasing in the collaboration rate (national and international) of supported researchers?

Ultimately, the findings of the current study will be integrated with those of the other assessment instruments to be used in this evaluation in what is known as the “method of converging partial indicators”. When multiple indicators converge, the conclusions drawn from the evaluation are regarded as being more reliable than those based on a single indicator.

Questions 1, 2 and 3 assess the scientific performance of CRCP-Chairholders relative to those of comparable groups of researchers, whereas Questions 4, 5 and 6 assess the effect of the program on the scientific performance of CRCP-Chairholders through a comparison of their scientific output produced prior to and after receiving their Chairs (pre-post comparisons). The indicators used to answer these questions are:

  • the number of papers produced by each researcher;
  • the Average of Relative Citations (ARC), an indicator of scientific impact that is normalized by scientific disciplines, ensuring that the indicator can be compared across disciplines;
  • the Average of Relative Impact Factors (ARIF), a proxy for the quality of scientific publications that is also normalized by scientific disciplines; and
  • the national and international collaboration rate.
 

Bibliometric Analysis of Individuals Supported by Alberta Ingenuity Fund (2008)

Client: Alberta Ingenuity Fund (AIF)

[PDF] – Study Report

Description:  This study examined the scientific production of researchers and graduate students who received financial support from the Alberta Ingenuity Fund (AIF). The scientific output of about 120 researchers and 300 graduate students was analyzed based on three bibliometric indicators of scientific performance: (1) number of papers, (2) average of relative citations, and (3) average of relative impact factors.

The selection of high-calibre researchers and academically superior graduate students is a crucial component of AIF funding programs. As such, by comparing the scientific output of successful researchers and students to comparable groups, the analysis provided insight as to whether AIF has effectively selected outstanding researchers and highly promising students.

In addition, the effect of AIF funding on the volume and impact of the scientific production of funded applicants was investigated. In the case of graduate students, the analysis also looked at the effect of AIF funding on mobility.