Pembangunan Dataset Gestur Angka (1-5) yang Terkontekstualisasi Budaya sebagai inisiasi Basis Data Pengenalan untuk Alat Bantu Pembelajaran Matematika

Abstract

The rapid advancement of Artificial Intelligence (AI) has transformed human–computer interaction toward more natural and intuitive vision-based gesture systems. Hand Gesture Recognition (HGR) plays a crucial role in the development of interactive learning media, particularly for recognizing finger gestures representing numbers 1–5. However, conventional HGR systems are predominantly built upon culturally biased datasets, especially those derived from American Sign Language, which may lead to misinterpretation and normative mismatch across different cultural contexts. This study initiates the development of a culturally inclusive hand gesture dataset for numbers 1–5 by incorporating diverse cultural finger-counting habits. Feature extraction is performed using the MediaPipe Hands framework to obtain accurate and efficient hand landmarks. The constructed dataset is then utilized to train a gesture classification model for interactive mathematics learning applications. Experimental results show that the model achieves strong performance in certain classes, particularly for classes 1 and 5, with a recall of 1.00 and the highest f1-score of 0.85. However, challenges remain in classifying classes 3 and 4 due to class imbalance, resulting in lower recall and f1-scores. Although the overall accuracy reaches 66%, this imbalance indicates the need for further optimization in training strategies to improve model generalization. The proposed dataset is expected to serve as a more relevant and culturally inclusive foundation for developing safe, accurate, and adaptive HGR-based interactive mathematics learning systems.