A NEW INTERFACE PARADIGM: AUTOMATIC RECOGNITION OF INTEGRATED SPEECH AND HANDWRITING INFORMATION

Because of the difficulties inherent to the analysis of such complex processes as speaking and (hand)wriling, the machine recognition of the two most natural ways of communicating has so far met with only limited success. In this paper we consider a new paradigm: llic integrated use of speech and handwriting information to improve llic overall accuracy of an automatic recognizer. This approach is made possible by lhe complementarity of the Iwo sources of information yielding the acoustic and pen stroke evidence. It leads to the development of ISWI, the Integrated Speech and Writing Interface. Preliminary resulte indicate llic viability of ISWI, particularly in applications such as human factor studies. These results also bring useful insights into lite feasibility of ISWI given the present level of speech and handwriting recognition performance. It seems reasonable to expect ISWI to become the logical choice for the ultimate user-friendly man-machine interface.