ECTcomplex - Assistant technology

Assistant Technology for Alternative and Augmentative Communication

Eye-tracking software complexes are intended to allow quadriplegic persons, or people with reduced mobility, the possibility of letter-by-letter text typing. One of such complexes is the set of ECTCamera, ECTtracker and ECTkeyboard programs. This set is a ready-to-use product that allows users to enter text letter by letter using eye movement (e.g. blinking) or other possible movements using a laptop or PC with a connected video capturing device (e.g. camera) and special software (virtual keyboard).

Each program is intended to complete only one operation:

ECTCamera (see Fig. 1) allows the user to view any video captured by camera and also allows the user to change the scale of the image and select the position of necessary fragments (e.g. to focus on user's eyes). This software supports different types of video capturing devices, automatically identifies them and allows the user to select different modes and resolution of images.

(Fig. 1. Interface and settings panel of ECTcamera)

ECTtracker matches images from the video received from ETCcamera by using special identifying structures. The user creates a table of image samples (the Matrix of Samples), which contains images with eyes shown in different states (e.g. both eyes are open or closed, one eye is open or closed) and, subsequently, ECTtracker software determines the degree of similarity of images with samples from the table. Each group of samples corresponds with key codes, which the software sends to the controlling application – ECTkeyboard. By opening or closing one or both eyes, the user can simulate the pressing of keys and send this information to the receiving software, such as a virtual keyboard, thus entering text letter by letter.

(Fig. 2. ECTtracker interface, debugging window and settings panel)

ECTkeyboard contains a special matrix of letters that have variants of an alternating selection of fragments, rows and columns and characters. By receiving control codes from ECTtracker, this software allows the user to select any symbols from the matrix individually, creating words and sentences which can be shown either in a separate form of this software or can be exported as a file.

(Fig. 3. ECTkeyboard interface, text output form and settings panel)

Interface and functionality of this software can be configured for personal needs. In ECTCamera the user can select any scale and position of captured video images, format and compression of screenshots and automatic capturing with program start up.

There are more settings in ECTtracker, where the user can select the size of the target window for video capturing, choose the level of samples comparing tolerance, set any name of the desired receiving software and key codes, choose one of available identifying structures (for one or two eyes) and enable automatic recognition with software start-up, as well as having the ability to change other settings.

In the ECTkeyboard software, the user can choose any size and position of the matrix of letters, the amount of symbols, font, size and text of matrix, and countdown scale. The user can choose colors of inactive, selected and active elements of the matrix and set any desired delay time for selection and cancellation of the desired buttons. There are more than 80 different parameters in ECTkeyboard, which can be configured for comfortable work of any user.

All software elements have several interface languages available: Chinese, German, English, Spanish, French, and Russian. ECTCamera also supports Arabic and Italian. You can select any desired language by choosing the main menu and clicking on Localization – Set Language File item. A quick selection for the English language is available through the main menu Localization – Set English.

To work correctly, all three elements of the complex must be open at the same time. Common interface and window location are represented on figure 4.

(Fig. 4. Common interface of the eye-tracking software)

Numbers 1-3 in this figure show all elements of ECTtracker: 1 – this is the main window of the program, with an image fragment and one eye identifying structure; 2 – this is the Matrix of Samples; 3 – the debugging window ("Stripes" window) with a real-time recognition diagram. Number 4 stays for ECTCamera. The focus is set to the eye of the user. Numbers 5 and 6 show windows of ECTkeyboard: 5 shows the matrix of text, for sequential selection of the desired characters; 6 is the text output window.

For comfortable work the user needs to make some initial settings within the software:

Select working resolution of video capturing device
Place video capturing window
Choose desired identifying structure
Create the Matrix of Samples
Set key codes for receiving software
Set parameters and delay the time for the matrix of letters.

We will look at all these settings more closely further in this manual.