Sign languages are visual languages. The message is not only transferred via hand gestures (manual signs) but also head/body motion and facial expressions (non-manual signs). In order to test the efficiency of algorithms to analyze and classify the non-manual gestures, we present a database of non-manual signs in this web page. The non-manual signs which are frequently used in Turkish Sign Language (TSL) and those changing the meaning of the performed sign considerably are selected as the sign classes in the database. There are also additional signs which we use in daily life during speaking. The database contains the videos of the selected 8 different classes of signs as well as a ground truth data of 52 manually landmarked points of the face.
The non-manual gesture classes used in the database
database is formed of the following 8
different classes of signs (each class name has a link to a sample
In the figure left, some frames captured from different sign classes can be seen.
Properties of the Database
- It involves 11 different subjects (6 female, 5 male).
- Each subject performs 5 repetitions for each of 8 classes. So there are a total number of 440 videos in the database.
- Each video lasts about 1-2 seconds.
- Philips SPC900NC web cam is used with choice of 640×480 resolution and 30 fps.
- The recording is done in a room eliminated from sunlight and illuminated by using daylight halogen and fluorescent lights.
- The videos are compressed with “Indeo 5.10” video codec.
- Each video starts in neutral state, the sign is performed and again ends in neutral state.
- No subjects have beard, moustache or eyeglasses.
- There is no occlusion or motion blur.
- 48 of the videos are annotated as ground truth data.
In order to satisfy different experiments on the database, a preferably large number of facial landmarks are chosen for manual annotation. The selected 52 points can be seen here. Due to the difficulty in manually annotating these landmarks in all frames, only 3 repetitions of 4 classes (Head L-R, Head Up, Head F, Happiness) performed by 4 subjects (2 male, 2 female) are annotated in the database. So, there are a total of 48 annotated videos. In total 2880 (48 videos × 60 average frames per video) frames are annotated.
The video files are named as "[subjectname]_[#classId]_[#repetitionNo].avi". For example, if you download the Head F sample video ("ismail_4_1.avi"), you see that this video belongs to "ismail" and this is the first repetition of him performing the sign of 4th class.
The ground truth files involve the location of the landmarks. That is, ith row in a file involves the landmark locations (x1, y1, x2, y2, ..., xL, yL) in the ith frame of the corresponding video. To illustrate, this MATLAB code snippet reads and animates the landmarks in this sample file.
- Arı, İ., Gao, H., Ekenel, H. K., and Akarun, L., "Yüz Nirengi Noktalarının Zamansal Öz-benzerliğine ve Kelime Çantasına Dayalı Yüz İfadesi ve Kafa Hareketi Tanıma", IEEE 18. Sinyal İşleme ve Uygulamaları Konferansı, Diyarbakır, 2010.
- Saraswat, M. and Arya, K. V., “Automatic facial landmark detection in video sequences of non-manual sign languages”, IEEE International Conference on Industrial and Information Systems (ICIIS), pp.358-361, 2009.
- Akakin, H. C. and Sankur, B., "Analysis of Head and Facial Gestures Using Facial Landmark Trajectories", in Biometric ID Management and Multimodal Communication, Volume 5707/2009, pp.105-113, 2009.
- Arı, İ. and Akarun, L., "Yüz Özniteliklerinin Takibi ve İşaret Dili için İfade Tanıma", IEEE 17. Sinyal İşleme ve Uygulamaları Konferansı, Antalya, 2009. (in Turkish)
- Arı, İ., Uyar, A. and Akarun, L., "Facial Feature Tracking and Expression Recognition for Sign Language", International Symposium on Computer and Information Sciences (ISCIS), Istanbul, Turkey, 2008
- Arı, İ., "Facial Feature Tracking and Expression Recognition for Sign Language" , MS Thesis, Boğaziçi University, Istanbul, Turkey, 2008.
- Aran, O., Arı, İ., Güvensan, M. A., Haberdar, H., Kurt, Z., Türkmen, H. İ., Uyar, A., Akarun, L., "Türk İşaret Dili Yüz İfadesi ve Baş Hareketi Veritabanı",IEEE 15. Sinyal İşleme ve Uygulamaları Konferansı, Eskişehir, 2007. (in Turkish)
- Güvensan, M. A., Haberdar, H. : "Türk İşaret Dili Tanıma İçin Yüz ve Yüz Özniteliklerinin Tanınması ve Takibi", IEEE 15. Sinyal İşleme ve Uygulamaları Konferansı, Eskişehir, 2007. (in Turkish)
- Kurt, Z., Uyar, A., Türkmen, H. İ., "Türk İşaret Diline Yönelik Yüz İfadesi ve Kafa Hareketi Tanıma Sistemi", IEEE 15. Sinyal İşleme ve Uygulamaları Konferansı, Eskişehir, 2007. (in Turkish)
If you use the database, please cite the BUHMAP-DB as:
For Turkish: Aran, O., Arı, İ., Güvensan, M. A., Haberdar, H., Kurt, Z., Türkmen, H. İ., Uyar, A., Akarun, L., "Türk İşaret Dili Yüz İfadesi ve Baş Hareketi Veritabanı", Sinyal İşleme ve Uygulamaları Konferansı (SİU2007), Eskişehir, 2007.
For English: Aran, O., Arı, İ., Güvensan, M. A., Haberdar, H., Kurt, Z., Türkmen, H. İ., Uyar, A., Akarun, L., "A Database of Non-Manual Signs in Turkish Sign Language", Signal Processing and Communications Applications (SIU2007), Eskişehir, 2007.
The BUHMAP database is collected with the collobarion of the project members (Amaç Güvensan, Aslı Uyar, Hakan Haberdar, İsmail Arı, İrem Türkmen, Rüştü Derici, and Zeyneb Kurt) under the supervision of Prof. Lale Akarun and kind helps of Abuzer Yakaryılmaz, Didem Çınar, Neşe Alyüz, Onur Güngör, Oya Aran, Öner Zafer, and Pınar Santemiz,
Last update: Dec '09