अनिल एकलव्य ⇔ Anil Eklavya

April 13, 2008

Two Laws of Reviewing

After a few years in research, I have discovered two laws which the process of reviewing (of research papers) follows. Not very original, but here they are:

  1. You can always find some reasons for accepting any paper.
  2. You can always find some reasons for rejecting any paper.

February 29, 2008

English is Language Independent

It’s the Global Language, right? So how can it be language dependent? You propose a theory based on English. It has to apply to all languages. You propose a Natural Language Processing (NLP) or Computational Linguistics (CL) technique for a particular problem. For English. It applies to all languages. You build a software for some purpose. For English. It has to be useful for all languages. You build a dictionary…

Never mind.

But the vice versa is not true. You propose a theory based on Hindi. It is language specific. It doesn’t count for much. You propose an NLP technique for a particular problem. For Hindi. It is language specific. It doesn’t count for much. You build a software for some purpose. For Hindi. It is language specific. It doesn’t count for much.

That’s how it works in practice, if not theory. Or may be even in theory, with some help from the (very valid) idea of Universal Grammar (except that the UG may be the UG of English).

Even today I have got a review of a paper on a problem which is like one of the holy grails of NLP or CL. One of the comments is that the approach has been evaluated on Hindi so it can’t be compared to other techniques that already exist. True. But what is the number of papers published in the ‘first class’ NLP/CL conferences and journals in which the approach has been tried only on English? Doesn’t matter, because English is language independent. If you only evaluate your technique on English, that’s OK. But if you evaluate on only Hindi, that’s not acceptable. Because Hindi is language specific.

We know this very well in India. The Elite talks about (Indian) literature. And sometimes the Elite magnanimously (or dismissively) talks about (Indian) literature in languages. The first, of course, refers to literature in English. The second refers to literature in other languages. Indian languages.

The Elite talks of media. And the Elite (rarely and mostly negatively) talks of language media.

Hindi is a language. English is not a language.

Pardon me.

Hindi is a language. English is the language.

English is above being merely a language.

That’s why all the work done in English is language independent. Not just research. Not just in NLP/CL. Anything. Movies, literature, music.

I am guilty of the sin of indulging too much in mere languages. I should be working mostly on English. Not just writing blog posts in English. Sometimes, of course, I can bestow a bit of my attention on languages. Like Hindi.

But I won’t do that. I will do the opposite. I am incurable.

February 18, 2008

A Comment on an Influential Article

A colleague has been sending me links to articles by Philip Greenspun. When I got another link today and just finished reading it (a rather long article), I thought I needed to comment on that article. So here it is (I have posted it at his site too):

A great looking intellectual construction, but it is based on some fundamental flaws. So, even though a lot of the things said are correct and sensible, the most important ones are not.

For example, let’s take the practical implications: You first suggest that it is poverty that is increasing the ranks of the suicide bombers. But then you conclude that if we keep these third world incompetent Muslims poor for eternity, we might just save ourselves from terrorism. A dead giveaway I would say.

That’s the trouble with people like you. You ask others to look in the mirror, but you yourself don’t.

What about America’s record in general? I mean active participation in or encouragement of mass murder: Chile, Vietnam, Cambodia, Laos, etc.? Could that have something to do with the fact a lot of people around the world ‘hate the US’?

The ‘conventional wisdom’ that you quote (“Nations don’t have friends. They have interests.”) is from a person who is actually a mass murderer and a war criminal. You seem to have no problem with these ideas. And this person happened to be a Jew.

But so is Noam Chomsky. So was Spinoza. So was Einstein. So was Joseph Heller. So is Woody Allen.

Like most ‘Experts’, though in a slightly better way, you have presented a mixture of true facts and unjustified simplifications to come up with a theory that is sufficiently complex to bore most people into accepting it as true. It is coming from an Expert after all. Why should we bother to look deeper into it? In fact, most people will be overawed by just the MIT label.

You look hard enough at everyone else: Muslims, Europeans, Third Worlders, etc. but you are unwilling to look that hard at the deeds of the Americans, i.e., the establishment of the USA. You put the USA and Canada in the same category, but the facts, if you look deep enough, wouldn’t allow you to do this. Canada has hardly any record of imperialism and attempts of dominating the world as an unchangeable policy that can justify even mass murder, assassinations, drug trafficking to fund terrorism against enemies (as in Afghanistan against the Soviet Union) as long as it is hidden and there is scope for plausible denial.

You even refer to decolonization as if it was only a bad thing. I come from a country where more people died at the time of independence and the partition (of India at the time of ‘decolonization’) than did in the Holocaust. There is no way you are going to confuse me into thinking that the independence (decolonization) was the same as (or the cause of) the horrible events that followed. Decolonization was a good thing. A lot of the events that followed were horrible. There are two different things we ought to be talking about. But, of course, you are not interested in that. It might show the flaws in your theorizing. For example, did colonialism have anything to do with the fact that a lot of non-westerners ‘hate’ westerners even if they try their best to get into the western paradise? And the fact that the US now represents what the UK did in an earlier age. The empire that seeks to rule the whole world and won’t be satisfied until it has risen enough and then falls down (perhaps to be replaced by another empire that would also be hated by the rest of the world). At a huge cost to be paid by people other than you.

January 20, 2008

On Blind Reviewing

This is something about which I have wanted to write for a long time. Since, like many other things about which I want to write, it is quite an important matter, I didn’t want to write in a hurry. Which meant that I had to wait for a time when I could write at enough leisure to be able to write at enough length with enough time for making it rigorous enough. Now, since it is very difficult (for me at least) to get enough of all these, this effectively meant that writing about this topic was postponed indefinitely.

But I don’t want this to be postponed indefinitely. I want to write about this now. So, I would just write and try to be as rigorous as it is possible to be in a blog post written in one or two short sittings. This applies to many other posts, whether written already or to be written in future. You can take it as an apology or you can take it as a disclaimer.

What is the problem? Well, the problem, or rather the question, is whether what is called ‘blind reviewing’ is a good thing or not. And, of course, this is in the context of peer reviewing of scientific (or claimed to be scientific) research papers or articles for the purpose of selection for inclusion in the proceedings of a conference or workshop or for inclusion in a journal.

Excuse the legal sounding language.

First of all, let me list all the reasons in favour (‘favor’ for the dominant party) of the so-called ‘blind reviewing’ process, so that no one can jump and dismiss the whole affair as trivialization by saying you don’t know what you are talking about:

  1. Human beings can be biased. So, if a reviewer knows that a research paper is written by a person she doesn’t like or has strong disagreement with, she can get biased against the paper and will not be able to review the paper fairly.
  2. Apart from the above kind of biases, there can be the bias in terms of the weights associated with the names of the authors, their institutions, their countries, their group, even their academic background. Most of the people who have been working in NLP/CL[1] for some time know about the linguistics vs. statistics or machine learning bias. This kind of bias increases the chance of your paper being rejected or accepted depending on whether you seem to be in favour (or favor) of a linguistics heavy approach to NLP/CL or of a statistics (or machine learning) heavy approach. There are variants of this bias in other fields too. For the closest example, we can consider Linguistics. Where your paper is perceived to be situated along the Chomskyan or Empiricist or Cognitive or Computational axes with respect to the chosen position of the reviewer can have a large impact on the decision about your paper, irrespective of what else your paper says. And the chances of such a perception can be increased if the identities are known.
  3. Human beings can be unduly confrontational and they can also be unduly wary of confrontation. So, if the identity of the reviewer is not withheld, the author(s) may be offended by the reviewer and they may also become confrontational and carry on this confrontation with the reviewer, thus making the process of reviewing difficult and something which a lot of people would like to run away from. Also, the reviewer may avoid making adverse comments, especially if the reviewer doesn’t want to offend the author(s).
  4. If the author(s) don’t know who the reviewer is and vice versa, the whole reviewing process may be more fair for the above specified reasons and because of the general association between anonymity and fairness. If you don’t know who is criticizing and the person criticizing also doesn’t know who is being criticized, then you can expect more fairness.
  5. If the Program Committee (PC) chair(s) also don’t know who the authors are and who the reviewers are, then they can assign equal weight to all the reviews for making the final decision about a paper.
  6. If the author(s) don’t know who the reviewer is, then they won’t have any reason to attribute bias or prejudice to the comments made or ratings given by the reviewer.
  7. Peer reviewing of research papers, like the administration of justice, should not just be fair, but seen to be fair. And this can only happen with blind reviewing.
  8. Blind reviewing, through the use of the device of anonymity, gives a true meaning to the idea of ‘peer reviewing’, because if the identities are not known, all the people involved can be treated as peers, even if some of them are senior most pioneering researchers or Directors of first class institutions in first world countries, while some others are graduate students in second class institutions in third world countries.
  9. If the identities are not known, both the reviewer and the author can focus on the content of the paper and the review, respectively.
  10. Finally, the very practical reason that blind reviewing provides a reasonably fair mechanism to ensure the selection of the best research papers such that everyone can be more or less satisfied with the outcome and no one will have valid reasons to complain.

I think the above list makes as strong a case for blind reviewing as can be made. I mean in a blog post, not in a book.

Now, in the next post (that means in some future post) I will discuss what is or can be wrong with blind reviewing and will try to draw some conclusions. You must have guessed that the reason I am writing all this is that I am not sure whether blind reviewing is the best thing possible. But by writing all this, I am also trying to get things straight in my own mind.

[1]: With apologies to Martin Kay and others, I am using NLP and CL as interchangeable terms because I think my arguments in this matter are not affected by the distinction between the two, a distinction which may be important in many but not all contexts (i.e., in my opinion).

Blog at WordPress.com.