Counter culture: 15 years of Science-Metrix

Published on July 13 2017

 

From June 2017 to May 2018, Science-Metrix is celebrating its 15th year. To kick off our series of posts saluting this important milestone, let’s take a look at who we are, what we’ve done and where we’re going, as recounted by company co-founders Éric Archambault and Grégoire Côté.

Early days

Science-Metrix was formed in 2002, when Éric Archambault saw the opportunity to create a professional services company centred on science and technology. Even before the company was incorporated—in fact, while writing the initial business plan—Éric secured and started performing a series of studies for Genome Canada. Then-president of Genome Canada Martin Godbout was a champion of bibliometric studies, and these early projects helped plant the seed of what became Science-Metrix. A second contract, with the now-defunct Québec Biophotonique, soon followed, and on the last day of May 2002, Éric incorporated the company. A month later he was joined by Grégoire Côté, and in early 2003 Frédéric Bertrand came onboard. Vincent Larivière, who is now a professor at the Université de Montréal, also joined the company for a few months, before deciding to concentrate on his studies.

The initial business plan targeted three types of activities: analytic services, evaluation services and the performing of due diligence studies for venture capital firms. In the end, the provision of analytic and evaluation services prevailed—a few environmental scans were produced, but not a single due-diligence study was ever performed.

From the start, the rookie entrepreneurs performed scientometric and technometric studies. Data were acquired online from Medline, custom datasets were acquired from Thomson ISI, and the team were also helped by colleagues at the Observatoire des sciences et des technologies, as well as Professor Yves Gingras at the Université du Québec à Montréal. The setup was a “somewhat unsightly bricolage,” Éric recalls. “We didn’t have any tools of our own and we did everything we could by cleaning freely available data to make it suitable for bibliometrics.”

In 2005, Natural Resources Canada granted Science-Metrix our first ever evaluation contract, which was conducted under Frédéric’s management. From that point until his departure in 2013, Frédéric assured the growth of the evaluation function of the company, while Grégoire assured the growth of the company’s expertise in analytics. Although analytic and evaluation services have been pursuing their own growth paths, it has always been a tenet of Science-Metrix that complementary excellence in qualitative and quantitative methods would constitute the company’s unique strength and would enable us to provide a vertically integrated offering in the assessment of S&T programs, supporting activities and policy-oriented decision-making.

Establishing in-house databases

An important change occurred in 2007, when Science-Metrix became one of the first companies to be a bibliometric partner using Elsevier’s Scopus database. At the time, few of our clients had heard of Scopus, and they had to make a leap of faith to use this data source rather than the then predominant data from Thomson ISI. For three years, Science-Metrix offered Scopus data nearly exclusively. We also maintained a good relationship with Thomson ISI though, where Henry Small and David Pendlebury helped us acquire custom datasets from time to time, until, in 2010, Science-Metrix had grown enough to be able to acquire data from the Web of Science and the accompanying licence to conduct bibliometric studies. This business relationship continued with Thomson Reuters and, more recently, with Clarivate Analytics, the new owner of the Web of Science. Today, Science-Metrix feels privileged to be a steward of both Scopus and the Web of Science, and we cherish our cooperation with these smart business partners.

In addition to these two major bibliographic databases, we also have in-house versions of Medline, PatentsView and many bespoke databases, which are regularly updated in the pursuit of complex studies. The advantages of having in-house databases are that they can be conditioned to fulfill clients’ specific needs, normalized statistics are computed using the whole content of the database, and each new round of data cleaning we perform can be carried forward to the benefit of all future clients.

Growth and innovation

In 2010, with these databases on our servers, Science-Metrix successfully won a four-year contract with the European Commission to conduct the largest-ever bibliometric study of European scientific production. This was a significant turning point for the company for several reasons. First, this was our debut major contract for a client outside of North America, and it established Science-Metrix as a world leader in bibliometrics. Second, the contract prompted analysts to develop the Science-Metrix journal-based classification of science, a hierarchical, three-level classification tree based on best-practice taxonomies and featuring 6 domains, 22 fields and 176 subfields. The classification—which was designed to be as inclusive as possible of newer fields of enquiry, general and multidisciplinary journals, and the full range of arts and humanities disciplines—is used in nearly every Science-Metrix bibliometric project.

Further European Commission contracts followed for the Directorate-General for Research and Innovation (DG RTD). These, too, inspired further innovations in Science-Metrix’ tools and practices. For example, work conducted under the contract Study to develop a set of indicators to measure open access led to the realization that more than 50% of academic papers were available in open access—more than twice previous estimates. This in turn inspired the launch of our sister company, 1science, which has developed a suite of products for institutions to facilitate the discovery, analysis and diffusion of open access papers published in peer-reviewed journals. Likewise, a network analysis of participation in the Commission’s Seventh Framework Programme pushed our development of research network and collaboration indicators, and our work on She Figures 2015 led to the development of five new gender-based indicators for measuring women’s participation in science and innovation activities. Most recently, our two-year study on data mining for the Commission led to the development of a framework for managing data mining projects and two inventories: one of relevant data sources and the other of indicators for measuring innovation. Few other companies from outside the European Research Area have performed as many studies for the DG RTD, and Science-Metrix is extremely glad that the European Commission has been open to trade in services in the pursuit of excellence.

In 2015, Science-Metrix was contracted by SRI International and the US National Science Foundation (NSF) to develop measures and indicators of research and patent activity using bibliometrics and patent data for inclusion in the US Science and Engineering Indicators 2016. This contract triggered and supported our analysts to refine an algorithm to match patents to their cited publications, in order to measure one way in which research leads to innovation. This was an element of pride for the scientometric team as this statistical work had been initially undertaken by Francis Narin at CHI Research in the 1970s. Science-Metrix feels privileged to follow in the footsteps of this great practitioner and researcher in the fields of scientometrics and technometrics. That initial contract with SRI and the NSF was followed by further collaboration on the production of future iterations of the Science and Engineering Indicators through to 2021. Because these contracts entail preparing official national statistics, they represent the realization of an ambition held by Grégoire and Éric since the founding of the company.

Growth of Evaluation Services

Science-Metrix’ evaluation activities have also made invaluable contributions to the company’s development. The Evaluation team has always used a rigorous and disciplined approach to qualitative analysis and to performing complex evaluation projects. Evaluating research funding policies, institutions, programs and initiatives in the science, technology and innovation space has been the team’s continued core business. Much work has been performed in designing evaluation and performance monitoring frameworks such as the Development of an Evaluation Framework and a Performance Audit and Evaluation Strategy for CIFAR. Many contracts also touched upon support for research in universities in prestigious programs, such as the Tenth-year Evaluation of the Canada Research Chairs Program and the 10-year evaluation of the Caltech Initiative. The latter, a $300 million gift to Caltech from the Gordon and Betty Moore Foundation, was at the time the largest gift ever made to a single university.

In response to shifting Canadian government priorities, the team progressively evolved to evaluate more applied research programs and programs with an industrial research, innovation or commercialization aspect, such as the Evaluation of the Business Innovation Access Program, the Formative Evaluation of Collaborative Research and Innovation Opportunities and the Evaluation of the Earth Observation Business Line of the Canadian Space Agency. The 2017 release of the Naylor Report outlines a pathway to rebalance Canada’s research funding priorities; our Evaluation team’s broad experience means it is well positioned to evaluate programs at all points along the research continuum, including emerging areas such as programs supporting incubators and accelerators.

The human factor

In 15 years, Science-Metrix has become a respected professional services company in the field of S&T analytics and evaluation. It is an important company in the Canadian sphere and also an increasingly important player at the international level, where the majority of our revenues are earned. Science-Metrix has worked hard, innovated and made bold moves. We have also benefitted greatly from the trust of clients who took a chance with us when we were inexperienced and then continued to place their faith in us to realize increasingly complex mandates. We feel blessed to have created so many healthy working relationships with our clients, several of whom we have built extraordinarily cordial relationships with despite the need for the arm’s-length, professional distance we always maintain in the interest of sound governance and client–evaluator independence.

The success of Science-Metrix is due in very large part to the dedication and innovativeness of its entire personnel. Staff members are given latitude to investigate the problems at hand, yet the company has always been composed of a tightly knit team working collaboratively. One key employee deserve a special mention, not only for his longevity at Science-Metrix, but also for all the innovations we collectively owe to him. In 2004, David Campbell began work as an analyst at Science-Metrix. Now Chief Scientist, David has contributed immeasurably to the company’s success over the ensuing 13 years and is constantly developing and refining indicators to ensure Science-Metrix remains a world leader in S&T analytics.

Another important contributor to the company’s growth was partner and Head of Finances and Operations Jean-François Bergeron. Jean-François joined Science-Metrix in 2009, but passed away in 2014 following a fight with leukemia and is deeply missed. In 2013, founding partner Frédéric Bertrand left the company to pursue other interests after building our Evaluation Services team from the ground up. In 2015, Werner Meier officially took on the role of VP Evaluation Services after several months serving as a consultant.

After 15 years of solid growth, with strategic partnerships in place with Elsevier and Clarivate Analytics, and with a healthy cooperation with our sister company, 1science, 2017 is an exciting year for Science-Metrix. With several multi-year contracts in effect, we aim to refine our internal processes, conserve our collegial company culture and continue the development of innovative practices.