Abi See recently hosted a debate between Yann LeCun (Facebook/NYU) and Chris Manning (Stanford) on the importance of linguistic structure and innate priors in systems for natural language understanding. The video, along with a nice write-up from Abi herself, are available here.
I used to have strong opinions on the use of linguistic structure in NLP systems.1 I’m no longer so passionate about that debate, but I still found something interesting in this particular discussion. Yann made a striking remark near the end of the debate (beginning in the video at 59:54):
Language is sort of an epiphenomenon [of human intelligence] … it’s not that complicated … There is a lot to intelligence that has absolutely nothing to do with language, and that’s where we should attack things first. … [Language] is number 300 in the list of 500 problems that we need to face.2
Wow. Those words come from the man who regularly claimed a few years ago (circa 2014) that language was the “next frontier” of artificial intelligence.3
Four years is quite a long time in the world of AI — enough to legitimately change your mind in the light of evidence (or lack thereof). Recall that NLP in 2014 was awash in the first exciting results in neural machine translation.4 LeCun rode the wave, too: in 2015 he and one of his students put up a rather ambitiously titled preprint called “Text Understanding from Scratch.” (No, they didn’t solve “text understanding.”)
Yann seems to have had a change of heart since those brave words. I think the natural language processing community as a whole has begun to brush off the deep learning hype, too.
One can hope.
-
Heck, it was a central motivation for my research at that time. I suppose that was a natural consequence of being directly advised by Chris. :) ↩
-
The “we” in this quote is ambiguous. I’d guess from context that he was referring to Facebook AI, but he could have also meant to refer to the larger AI research community. ↩
-
I recall this distinctive phrasing from several public talks, but we also have some text records. Source 1, Yann’s response to a 2014 AMA: “The next frontier [sic] for deep learning are language understanding, video, and control/planning.” Source 2, quoted in a 2015 article from Cade Metz: “Yann LeCun … calls natural language processing ‘the next frontier.’” ↩
-
See e.g. Kalchbrenner & Blunsom (2013); Sutskever, Vinyals, & Le (2014); and Bahdanau, Cho, & Bengio (2014). ↩