The vast and splendid hall in the state-of-the-art building of the Hyderabad International Convention Centre was filled with delegates from all over the world, gathered there to witness the spectacular inaugural ceremony of the International Congress of Mathematicians (ICM). They were waiting for the President of India, Shrimati Pratibha Devisingh Patil, to come and declare the proceedings open. The host on the dais was introducing the audience to the charms and wonders India offers to its visitors. “Where in the world would you see one-third of the people driving on the left side of the road, one-third on the right and one-third in the middle of the road?” she asked and the hall of about 3,000 participants roared with laughter.
ICMs are among the oldest scientific meetings, starting around 120 years ago, and are held every four years. This forum was established to review the current trends in research in Mathematics and to discuss future developments in the field. It was at this meeting in 1900 in Paris that David Hilbert, the German mathematician, presented his famous 23 problems meant to set the future of research in Mathematics, many of which are still unsolved. It was around this time that another famous problem, the Poincare Conjecture, after the name of the French mathematician Henri Poincare, was posed. This problem remained unsolved for about 100 years. Another highlight of the ICMs is the Fields Medals in Mathematics that are announced and awarded at these meetings. This medal in Mathematics is often compared to the Nobel Prize in other areas (there is no Nobel Prize in Mathematics). In the Madrid ICM of 2006, Grigori Perelman of Russia was awarded the Fields Medal for solving the 100-year-old Poincare Conjecture. It is interesting to note that at the time of this award he did not have any publications, nor has he anything published today!
An important session at the 2010 Hyderabad ICM was devoted to the discussion on the use of metrics to evaluate research, particularly on the uncritical use of the Impact Factor (IF) for the same purpose. The IF is the average number of citations made in a given year to a journal’s papers from the preceding two years. Their practical application arises from the need of assessing research by simple and objective methods. The panel included Professor Douglas Arnold from the University of Minnesota, the president of the Society of Industrial and Applied Mathematics (SIAM) also. He had highlighted the issues and instances of fraudulently increasing the IF of journals and other blatant misuses of this measure in his articles ‘Nefarious Numbers’ and ‘Integrity under Attack: The State of Scholarly Publishing’. He analysed the cases of IF manipulation for a few journals in detail. He described, for example, how the IF of the International Journal of Nonlinear Sciences and Numerical Simulation (IJNSNS) rose to 8.9, more than double the next highest journal in applied Mathematics. He told the audience that this journal was in his area of research but that he never knew about it. Explaining the reason for this large IF he said that most of the citations came from IJNSNS itself or special issues of other journals edited by someone on the IJNSNS board. Only in the year 2008, the journal’s editor-in-chief himself cited the journal 243 times within the last two years (the crucial window for calculating the IF). Apart from him two other editors cited this journal 114 and 58 times. With IJNSNS, 72 percent of their citations were in the two years that count for the IF and only 28 percent in all the other years. For normal journals it is usually the opposite. With this glaring performance, the editor-in-chief was among the world’s most highly cited mathematicians and was named the “rising star” and the “hottest researcher of the year”. He repeated this performance the next year as well and was considered worthy of many awards and honours. As the grand hall burst into laughter, I started sinking into my seat; it seemed as if they were discussing the situation in Pakistan!
Professor Arnold concluded by mentioning Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure.” He further elaborated by mentioning “an example used in economics, that if a nail factory in a centralised economy is judged on the number of nails produced, pretty soon they will figure out they should make lots and lots of tiny nails. If it is judged on the weight of the output, they will start making very big nails.”
Professor Malcolm MacCallum of Queen Mary University, London was another panelist. He shared his experience of the Research Assessment Exercise that was carried out in the UK in which he played a prominent role. He told the audience that the assessment was done not by any bibliometric data but by actually reading the research papers! His conclusion was: “I do not believe one can judge a paper by where it appears.” This seemed to be the bottom line of the discussion to which everyone agreed. Let me repeat: a paper cannot be judged by where it appears. If someone wants to evaluate research he will have to read the papers, or get them read by somebody.
(To be continued)
The writer is a faculty member at Quaid-e-Azam University, Islamabad. He may be contacted at [email protected]