NTU HIGHLIGHTS December 2016  

Leaders Profile


Lin-Shan Lee: Making Computer Understand Chinese


Prof. Lin-Shan Lee, a distinguished professor of the Department of Computer Science and Information Engineering, was elected as an Academia Sinica academician in the Division of Engineering Sciences this year.

In the early 1980s, Prof. Lee and his students at the Department of Electrical Engineering conducted pioneering basic research in the area of Chinese-language speech recognition. Over the more than three decades since then, Lee has stood out as the scholar who devoted the most time and energy to this field that now provides such great convenience to Chinese-speaking smartphone and computer users around the world.

Globally recognized for his influential research, Prof. Lee is also a respected lecturer who has accepted the NTU Outstanding Teaching Award numerous times. He has educated many exceptional professionals for society and encouraged countless students of the Department of Electrical Engineering to pursue careers in research.

During a recent interview, when asked about his students, Prof. Lee casually scans the campus outside his office window as he reveals, "Though I haven't actually calculated the figures, probably more than half of the professors of the Department of Electrical Engineering and Department of Computer Science and Information Engineering have been my students."

When Prof. Lee started his teaching and research career at NTU, the Department of Electrical Engineering was strapped for funding and was hardly comparable to the advanced research institute it has become. Prof. Lee is forever grateful to his students at the time who spent half a semester with Lee tracking down parts and formulating a plan to assemble on their own integrated circuit board capable of processing speech into a binary code. Flashing a smile, Prof. Lee admits, "The machine could only record the first half of a sentence because its memory capacity was just too small."

In those early years, Prof. Lee established three conditions for computer speech recognition: 1) each word must be spoken separately, 2) one computer recognizes the voice of one user exclusively, and 3) recognition mistakes are displayed on a monitor for human correction. These apparently simple criteria served as an important foundation for research in those days and are the reason Chinese speakers enjoy speech recognition functions on their smartphones today.

Recalling that he had only the one computer and that it could record just one person, Prof. Lee says, "That computer would recognize my voice alone. I figured the students would leave eventually, and I would be the only one to remain."

Commercial virtual assistants, including Apple Siri, Google Now, and Microsoft Cortana, offer Chinese-language speech recognition. Prof. Lee, who has been referred to as a technology promoter behind Google's voice search and Apple Siri in the media recently, says it is extremely difficult to develop applications that the average user will accept with the limited resources of an academic institution. He points out that he and his students have found that the basic framework of commercial-grade applications corresponds completely to their past research.

In the early 1990s, Prof. Lee and his students confronted hardships in terms of inadequate equipment and facilities, on one hand, and limited availability of international research relevant to Chinese-language speech recognition, on the other hand.

In addition to facing these challenges, Prof. Lee even reached the bounds of his own knowledge. Following a failed attempt to design a computer capable of producing speech, Lee gained valuable insight while discussing his troubles with a linguist at Academia Sinica who had recently returned from abroad. The linguist reminded Prof. Lee that each character in a Chinese language not only has its basic phonetic sound but that its pronunciation is influenced by the characters coming before and after in a sentence it as well.

Reflecting on the circumstances under which he received his letter of appointment to teach at NTU, Lee says, "No one supported my return to Taiwan at the time." He faced not only the opposition of his family and close friends, but moreover an international situation that was unfavorable to his return to Taiwan to begin a teaching career.

"I received my letter of appointment in November of 1978. Then, in mid-December, United States President Carter ended diplomatic relations with Taiwan, and even more people advised me not to be foolish," says Prof. Lee, emphasizing, "Regardless, contrary to the American Dream embraced by many people, I held a Taiwanese Dream that I could never give up."

Prof. Lee took up his teaching post at NTU in 1979, a time when the Department of Electrical Engineering occupied only one building. "We five professors shared the same office," remembers Lee.

While the department now has four buildings, Prof. Lee's office remains in Building Two, where it has been in use for over thirty years.

Question and Answer Session with Prof. Lin-Shan Lee

What were your primary interests when you were young?
The activity I liked the most during my university years was mountain climbing. For three years from my second year to my fourth, I visited many famous mountains around Taiwan with the Mountain Climbing Club. On our climbing expeditions, we would inevitably pass through local villages and tribal areas and experience a different cultural atmosphere. The sense of homeland I feel for Taiwan was perhaps cultivated in this way.

What aspirations did you hold as a young person?
During my third and fourth years of university I began to want to study in the United States, earn a solid doctorate degree, and return to Taiwan to teach the students following me. So, during classes at Stanford, I would not just strive to gain knowledge but also pay attention to the teaching skills of the professors. I would imitate their strong points and remind myself of areas in which I needed to be especially careful, all for the purpose of becoming a good teacher.

What suggestions do you have for students of the Department of Electrical Engineering?
Compared to my generation, students now have stronger foundations, enjoy a greater abundance of information, and have more agile minds. However, they also pay more attention to the things happening around them and lack the determination to patiently pursue major long-term goals.
Life is like mountain climbing; there are many things going on around us that we must handle, but we can also remind ourselves at the same time that we need to continue advancing towards the major long-term goals and not become stuck in the same place.
I want to tell students: Do the hardest things. The world presents many easy things; let the people who want to do those things do them. If you are among Taiwan's most exceptional young people, then you must try to do the hardest things and make the greatest contributions.

What is the direction of you present research?
Voice search. You could call it the voice version of Google. If it were possible to search the Internet for videos using a designated voice, then users could locate the most vital information in a vast field of noise. For instance, I say "Obama, Trump" and the software helps me find video clips that mention both of these names.
Beyond that is to generate personalized query results. Suppose I identify myself as "a third year student of the Department of Electrical Engineering who hasn't studied machine learning" and who wishes to find "a three-hour machine learning course." Software could search the thousands of videos and internet courses on the Internet and help me locate useful clips, download the clips, and assemble them into a course I can use. I believe the treasures of knowledge of modern humanity are accessible through internet courses.

Introduction to Prof. Lin-Shan Lee
31st Class of Academia Sinica Academicians

NTU, Department of Electrical Engineering, BS, 1974
Stanford University, PhD in Electrical Engineering, 1977

Chairperson of the Department and Graduate Institute of Computer Science and Information Engineering (1982-1987), Director of the Commission on Research and Development (2002-2005), and Dean of the College of Electrical Engineering and Computer Science (2009-2012)
At Academia Sinica:
Director of the Institute of Information Science (1991-1997)

Digital processing of speech signals

Academic honors
Presidential Science Prize of Taiwan, 2015
National Chair Professorship of Taiwan, 2004 and 2007
Outstanding Scholar Award, Foundation for the Advancement of Outstanding Scholarship, 2002
Distinguished Academic Contribution Award (in Engineering), Ministry of Education, 1993
Medal for Distinguished Contributions in Electrical Engineering, Chinese Institute of Electrical Engineers, 1991
Distinguished Research Award, National Science Council, 1985-1999 (7 times in 14 years)
Exemplary Global Service Award, IEEE Communications Society, 2014
Meritorious Service Award, IEEE Signal Processing Society, 2010
Fellow, International Speech Communication Association, 2010
Distinguished Lecturer, IEEE Computer Society/IEEE Signal Processing Society, 1995/2006
Fellow, IEEE, 1992
Board Member, International Speech Communication Association, 2001-2005 and 2005-2009
Chair of Awards Committee, IEEE Communications Society, 1998-1999
Vice President for International Affairs, IEEE Communications Society, 1996-1997
Member of Board of Governors, IEEE Communications Society, 1995-1997