Shumin Zhai  PhD

Pattern recognition: The trace of each word in a large lexicon (e.g. 50,000 words) forms a geometric pattern on a keyboard. Using computational intelligence ShapeWriter finds the closest word whose pattern matches the input stroke gesture on keyboard in real time (with imperceptible time delay). Due to the large amount of white space in a lexicon, an intended word can still be recognized although irrelevant letters between intended letters are crossed or even if some of the letters in a word are missed in the stroke gesture. ShapeWriter is therefore fundamentally error tolerant. The best way to do shape writing recognition is to take multiple sources of information (shape, location, path, context etc) and integrate them in a probabilistic framework.

​

Ease of error correction: alternative words that closely match the user gesture stroke are presented to the user so even if the user stroke is in fact closer to another word, the intended can still be easily selected.

​

Unified command and text input: the same mechanism for shape writing text is also used for issuing commands by Command Strokes. For example if the user swipes a stroke from a command key to c-o-p-y, ShapeWriter copies the selected text.

​

Dynamic and adaptive lexicon: ShapeWriter gives preference to the most common active vocabulary of 10,000 or so words in the 50,000 (or more) words lexicon. In fact only active words can come out by default. A passive word in the lexicon can only be presented as in a suggestion list. However, if the suggestion is selected, it will instantly become an active word. This quick expansion scheme means that ShapeWriter can rapidly adapt to an individual user's writing vocabulary. For a name or a word not in the lexicon, the user can tap it once and add it to the lexicon for shape writing next time.

​

Keyboard layout optimization: The layout of ShapeWriter's soft keyboard defines the ideal trace of each word. The familiar Qwerty layout is currently the default ShapeWriter keyboard but it is also possibly the worse possible layout. Shape writing tends to zig-zag left and right in a similar fashion for many words. In the long run, an optimized layout has great potential in further improving shape writing efficiency and experience. Learning shape writing on a new layout is much easier than learning typing on a new layout because of human sensitivity to geometric patterns.

 

Publications

This list also includes papers on soft keyboard and gesture interface related to ShapeWriter. Recommended readings on ShapeWriter are marked with ****.

 

2009

*** Zhai, S., Kristensson, P.O., Gong, P., Greiner, M., Peng, S., Liu, L. Dunnigan, A., Shapewriter on the iPhone: from the laboratory to the real world. ACM CHI 2009 Conference on Human Factors in Computing Systems Extended Abstracts. pp. 2667-2670.

​

Lee, S., Zhai, S., The performance of touch screen soft buttons. Proc. of ACM CHI 2009 Conference on Human Factors in Computing Systems. pp. 309-318.

​

Appert, C., Zhai, S., Using strokes as command shortcuts: cognitive benefits and toolkit support. Proc. ACM CHI 2009 Conference on Human Factors in Computing Systems. pp. 2289-2298.

         

2008

Zhai, S., On the ease and efficiency of human-computer interfaces. Proc ACM ETRA 2008: ACM Eye Tracking Research & Applications Symposium, pp 9-10 (keynote address abstract, slides)

​

Zhai, S. Kristensson, P.O, Interlaced QWERTY: accommodating ease of visual search and input flexibility in shape writing. Proc of CHI 2008: ACM Conference on Human Factors in Computing Systems, pp 593-596.

 

**** Per Ola Kristensson, Shumin Zhai. Improving word-recognizers using an interactive lexicon with active and passive words. Proc ACM Intelligent User Interfaces 2008, pp 353-356

         

2007

**** Kristensson, P.O., Zhai, S. Command Strokes with and without Preview: Using Pen Gestures on Keyboard for Command Selection,  Proc. CHI 2007: ACM Conference on Human Factors in Computing Systems, 28 April - 3 May, 2007, San Jose, California.

​

Cao. X, Zhai, S. Modeling Human Performance of Pen Stroke Gestures, Proc. CHI 2007: ACM Conference on Human Factors in Computing Systems, 28 April - 3 May, 2007, San Jose, California.

​

Cockburn, A. Kristensson, P.O., Alexander, J. Zhai, S. Hard Lessons: Effort-Inducing Interfaces Benefit Spatial Learning, Proc. CHI 2007: ACM Conference on Human Factors in Computing Systems, 28 April - 3 May, 2007, San Jose, California.

 

Kristensson, P.O., Zhai, S. Learning Shape Writing by Game Playing (Interactivity Paper), CHI 2007: ACM Conference on Human Factors in Computing Systems, 28 April - 3 May, 2007, San Jose, California.

​

Moran, T. P., Zhai, S. Beyond the Desktop Metaphor in Seven Dimensions, closing chapter in V. Kaptelinin and M. Czerwinski (Eds), Designing Integrated Digital Work Environments: Beyond the Desktop Metaphor.

​

** Kristensson, Per Ola, Discrete and Continuous Shape Writing for Text Entry and Control, Linköping University, Department of Computer and Information Science, PhD thesis, 200 pages, Supervisor: Zhai, Shumin,  Opponent: Buxton, William, 2007.  ISBN: 978-91-85831-77-7

        

 2006

**** Zhai, S., Kristensson, P.O., Introduction to Shape Writing, IBM Research Report RJ10393 (A0611-006), November1, 2006 (also as a Chapter 7 of by I. S. MacKenzie and K. Tanaka-Ishii (eds), Text Entry Systems: Mobility, Accessibility, Universality, Morgan Kaufmann Publishers, pp 139, 158)

         

2005

*** Kristensson, P-O., Zhai, S. Relaxing Stylus Typing Precision by Geometric Pattern Matching. Proc. IUI 2005, ACM Conference on Intelligent User Interfaces, 2005, pp. 151 - 158, ACM Press.

​

Zhai, S., Kristensson, P-O, Smith, B.A., In search of effective text interfaces for off the desktop computing, Interacting with Computers, 17(3):229-250, 2005.

​

2004

**** Kristensson, P.O., Zhai, S., SHARK2: A Large Vocabulary Shorthand Writing System for Pen-based Computers, Proc. ACM Symposium on User Interface Software and Technology (UIST 2004), Oct 24-27, Santa Fe, New Mexico, CHI Letters 6(2), pp. 43 - 52, ACM Press.

​

2003

**** Zhai, S., Kristensson, P.O., Shorthand Writing on Stylus Keyboard, in Proceedings of CHI 2003, ACM Conference on Human Factors in Computing Systems, Fort Lauderdale, Florida, April 5-10, 2003. pp 97-104.

 

2002

Zhai, S., Sue, A., Accot, J., Movement Model, Hits Distribution and Learning in Virtual Keyboarding, in Proc. of CHI'2002: ACM Conference on Human Factors in Computing Systems, Minneapolis, Minnesota, April 2002.  pp 17-24.

 

Zhai, S., Hunter, M., Smith, B.A., Performance Optimization of Virtual Keyboards,  Human-Computer Interaction. Vol. 17, No2&3. 2002.  pp 229-269.

         

2001

Zhai, S., Smith. B. A. Alphabetically Biased Virtual Keyboards Are Easier to Use - Layout Does Matter,  in Extended Abstracts of CHI 2001, ACM Conference on Human Factors in Computing Systems, Seattle, Washington, 31 March-5 April 2001 (Short Talk), p321-322.

 

Smith, B.A., Zhai, S. Optimized Virtual Keyboards with and without Alphabetical Ordering - A Novice User Study,  in Proc. of INTERACT 2001: Eight IFIP Conference On Human-Computer Interaction, Tokyo, Japan, July 9-13, 2001. p92-99.

 

2000

Zhai, S., Hunter, M., Smith, B.A., The Metropolis Keyboard -- An Exploration of Quantitative Techniques for Virtual  Keyboard Design. In the Proceeding of the 13th Annual ACM Symposium on User Interface Software and Technology (UIST 2000), November 5-8, 2000, San Diego, California. pp 119-128.

 

Hunter, M., Zhai, S., Smith, B.A., Physics-based Graphical Keyboard Design, CHI'2000. April 1-6, 2000, The Hague, The Netherlands. 157-158.