NIST Machine Translation Workshop

I was in DC last week to attend the follow-up workshop on the shared task that consumed the better part of my first 2 months this year. It is quite humbling to go to such a conference. A lot of people in the room had contributed something very significant to the area of MT in last 10 years and it is a pleasure to see them discussing things with each other and talk about the direction of the field.

The workshop had a rather competitive tinge to it with Google, Microsoft, BBN, IBM and other companies being there apart from many universities and everybody trying to win the bragging rights for next one year at least. What was funny however that most of the people doing MT at these places are relatively young and so most of them have worked together and published joint papers in last 4-5 years. I can count at least one joint paper between most pairs of companies. There were many advisor student pairs that are now in different competing companies. It is a nice little well knit community.

One depressing thing from my point of view was that most MT developers didn't seem too enthusiastic about newer methods of MT Evaluation something on which I have been working for part two years. To give a little background, MT Evaluation tries to build automatic metrics that can tell you how good a MT system is by comparing the output of the system with a gold standard translation done by humans. The field was pioneered by IBM and their paper of 2002 remains a de facto standard till date. There have been tons of papers on the topic and many short comings of IBM metric have been shown but nothing has been able to replace it. Primary reason seems to be that there are known techniques to tune the MT systems towards that metric that are easy, stable and fast and no MT developer seems willing to let go of that luxury. NIST is starting a evaluation campaign for MT Evaluation Metrics this year (I know it is getting ridiculous) and hope is to systematically compare existing metrics and develop better ones. The workshop for this year is in Hawaii but by that time I would be back in India. Too bad :(

Let me end by saying that for languages like Arabic, MT systems are actually getting very good and usable. You should definitely give them a try if you have not yet ! Google's online systems should give you a taste of state of the art pretty much.

Comments

Post a Comment

Popular posts from this blog

बिछड़ते दोस्तों के नाम

यादों की माला के मनके खोते जाएंगे, हम खुद से हर रोज़ बेगाने होते जाएंगे। मिलना जुलना, हंसना रोना, दुनिया भर के हिज्जे, रफ्ता रफ्ता सब अफ़साने होते जाएंगे। "आते हैं उस तरफ कभी तो तुमसे मिलते हैं", न मिलने के यूँही बहाने होते जाएंगे। इंक़लाब की बू है अब पुर-कैफ हवाओं में, बेवजह ही लोग दीवाने होते जाएंगे। भिंची मुट्ठियों, उठे कदम, लहराती बाहों से, बिछड़े साथी का हम साथ निभाते जाएंगे। गले नही  ल गते हैं, अब बस हाथ मिलाते हैं, यार अभागा सभी सयाने होते जाएंगे।
Read more

क्या लिखूं?

छूटे कोई दीन दुखी ना, सबको करो शुमार लिखो, दबी हुई चीखों को बुन कर, कर्कश इक ललकार लिखो। धरम जो पूछे शरणागत का, जात जो पूछे साधो की, हिंदुस्तानी दिल ऐसा भी हुआ नहीं लाचार लिखो। लिखने दो उनको BSE, NSE और GDP, तुम आम आदमी की जेबों में बची चवन्नी चार लिखो। इस शोर शराबे में सहमा सा सच जो तुमको मिल जाए, एक बार लिखो, दस बार लिखो, तुम उसको बारम्बार लिखो। है वक़्त अभी कुछ कहने का, है वक़्त नहीं चुप रहने का, नफरत के तूफानों में घिरते, इंसानों को प्यार लिखो। सुनो अभागा बदल गए हैं यहाँ मायने शब्दों के, देशभक्त ऐसे हैं गर तो, खुद को तुम गद्दार लिखो।
Read more

To bend and not to fold

There are thousands of things in my mind right now that I would like to put on this blog, share with the world. But I hardly manage to find time or the kind of tranquility that I want in order to write. There is one thing however, that never fails to push me to the emotional state where words just seem to flow. Few minutes ago, I watched a video prepared by IITK students called "The Way We Were". There was nothing much in the video. Large part of the video is a music video produced in house at IITK which, I must say, is quite good. The rest of the video is brief snippets of some junta talking about how they feel about IITK and what makes it special for them. It is this part that prompted me to write this post. First to quote a dear friend of mine, the later part in the video looks rather like a "boisterous attempt at self-glorification". I somewhat agree with that and it is this feeling of agreement that forces me to think what would I have said if I was there in th...
Read more