HAND GESTURE-BASED VIDEO GAME CONTROLLER

 Status : Completed

Tags: OpenCV Python Tkinter MediaPipe PyAutoGui

AIM

To develop a hand gesture-based video game controller that allows users to interact with and control games using real-time hand movements detected through a webcam.

COMPONENTS AND TECHNOLOGIES USED

  • Computer Vision

  • Hand Tracking

  • Gesture Recognition

  • AI

  • Python

  • OpenCV

  • MediaPipe

  • PyAutoGui

  • Tkinter

OVERVIEW

Contributors :-

S.N.

Name

Branch

Reg. No.

Year

1

Satyam Shivam Yadav

MED

20236120

2nd Year

 

Mentors :-

S.N.

Name

Branch

Year

1

Bhanu Pratap Singh

BT

Final Year

2

Rishi Mishra

ECED

Final Year

3

Kushagra Verma

ECED

Pre-Final Year

 

Tech stack:-

  • Python
  • OpenCV
  • MediaPipe
  • PyAutoGui
  • Tkinter

Components Used :-

 

  1. Computer Vision
  2. Hand Tracking
  3. Gesture Recognition
  4. AI
  5. Python

Introduction :-

The aim of this project is to develop an innovative video game controller that replaces traditional input devices like keyboards and joysticks with hand gestures. This system utilizes computer vision techniques and machine learning models to track and interpret hand movements, translating them into game control commands. The project domain is Computer Vision and Human-Computer Interaction (HCI), built using OpenCV and Mediapipe. The graphical user interface (GUI) is developed using Tkinter.

 

Methodology :-

  1. Learning and understanding Python language and its libraries such as OpenCV and Mediapipe
  2. Developing a program for real-time hand tracking using the Mediapipe library.
  3. Defining hand gestures and mapping them to game controls using PyAutoGUI.
  4. Testing and optimizing the system for different lighting conditions and hand orientations.
  5. Integrating the program into a GUI for an interactive experience.

 

Description :-

The Hand Gesture-Based Video Game Controller is a revolutionary approach to gaming that enables users to control video games without physical contact. Instead of pressing buttons or moving a joystick, players can use hand gestures to trigger in-game actions. This system offers an immersive and intuitive gaming experience using advanced computer vision techniques.

Features:

  1. Hand Tracking: Uses real-time computer vision techniques to track hand positions and finger movements.
  2. Gesture Recognition: Recognizes predefined hand gestures and translates them into game controls.
  3. Latency Optimization: Ensures minimal lag between gesture recognition and game response.
  4. Cross-Platform Compatibility: Works with various PC games by simulating keyboard and mouse inputs.
  5. GUI Integration: Provides a user-friendly interface to configure and test gestures before gaming.

Benefits:-

  • Enhances gaming experience with an intuitive and interactive control system.
  • Reduces the need for physical controllers, making gaming more accessible.
  • Encourages physical activity while playing games.
  • Can be extended to other applications like virtual reality (VR) and robotic control.
Real-life applications :-
  1. Gesture-based Gaming.
  2. Virtual Reality and Augmented Reality Controls.
  3. Assistive Technology for Individuals with Disabilities.
  4. Contactless Interaction for Public Devices.
  5. Interactive Learning and Training Simulations.
  6. Smart Home Control Systems.

 

Problem faced :-
  1. Accurate hand tracking under different lighting conditions.
  2. Defining a robust set of gestures that do not interfere with natural hand movements.
  3. Reducing latency between gesture detection and game response.
  4. Ensuring compatibility with a wide range of games and applications.

                                                                                                         

Thank you,

Team Hand Gesture-Based Video Game Controller