Introduction -

 An online proctoring system is an advanced AI-integrated tool that has been created for ensuring a cheat-proof test environment when the candidate is attempting an online test from a remote location. It increases the scope for the administrator to conduct online exams from any remote location without worrying about any sort of misleading act or attempt during the test.

Tech stack - 

    Python

       Detection : 

• OpenCV
• MediaPipe
• Selenium
• Matplotlib
• Subprocess
• time
• pyautogui

      GUI : 

• Tkinter
• TTkTheme
• FPDF

      Database :  SQL

• PyMySQL 

Role of each library:

MediaPipe :

The MediaPipe Face Landmarker task lets you detect face landmarks and facial expressions in  images. This library is used in head pose detection modules where we have recognized the user's face by analyzing the facial landmarks. Then the nose tip of users is detected and a line is directed from the nose whose position is analyzed in a 2D plane to get the direction in which the user is looking.

OpenCV :

OpenCV is a Python library that allows you to perform image processing and computer vision tasks. It provides a wide range of features, including object detection, face recognition, and tracking.

Selenium:

The selenium package is used to automate web browser interaction from Python. This library is used in our window switch detection modules and is also used to open the Chrome window with the test URL. The current URL method is used to get the current URL of the opened browser tab.

Subprocess:

Subprocess in Python is a module used to run new codes and applications by creating new processes. It lets you start new applications right from the Python program you are currently writing.Also this library we have used to run the background check on the user's system to check for all the background apps running on the pc.

Pyautogui:

This library has been used to take screenshots in the user's system.

Tkinter:  

Tkinter is the standard GUI library for Python. Python when combined with Tkinter provides a fast and easy way to create GUI applications. Tkinter provides a powerful object-oriented interface to the Tk GUI toolkit. Creating a GUI application using Tkinter is an easy task.

The entire GUI of our app is based on Tkinter.

FPDF:

PYFPDF is a library in Python, it is used for pdf file document generation. FPDF is a PHP class which allows you to generate PDF files and does not depend on additional PHP libraries

This library has been used to generate the users report after the proctoring.

PyMySQL:

It is a Python package that creates an API interface for us to access MySQL relational databases. We have used this library to make our application interact with the SQL database.

TTKThemes:

This  library is used to make the GUI of our application . This is a extended version of tkinter with inbuilt GUI themes that can be easily used to modify and make the GUI of any application.

Other important Utilities

HaarCascade:

Haar Cascade is a feature-based object detection algorithm to detect objects from images. A cascade function is trained on lots of positive and negative images for detection. The algorithm does not require extensive computation and can run in real-time.

TensorFlow:

The TensorFlow Object Detection API is an open-source framework built on top of TensorFlow that makes it easy to construct, train and deploy object detection models. There are already pre-trained models in their framework, which are referred to as Model Zoo.

SQL :

SQL stands for Structured Query Language. SQL lets you access and manipulate databases. It consists of several built-in commands like Update, Delete, Insert, create, and alter that is used to communicate with the database from our application.

Working of the Application - 

• Firstly there is a user validation window that redirects the user to our proctify app if the user is validated.
• Also there is also a registration form for each test session in which the user has to fill his/her details to be able to start proctoring and attempt the test.
• When the user registers and clicks start proctoring then only the real proctoring starts.
• Now the app accesses the camera of the user's device and then sends frame-by-frame data to each of the detection modules.
• Firstly the frame is sent to head pose detection modules that make a face mesh on user's face and direct a line from the nose and analyze that line's position on a 2D plane to get the direction in which the user is looking.
• Then the same frame is sent to detection modules that analyzes the presence or absence of any user in the frame and reports the user missing if no one is recognized in the frame.
• Then the frame is sent to the Multiple face detection module that uses the haarcascade frontal face to detect multiple faces in the frame.
• The frame is then sent to object detection modules that analyzes the frame to look for any objects like cell phone laptop and other cheating equipments
• Also the app takes the current url of the browser window at very short intervals  to match it with the actual test url so as to detect any browser tab switches.
• If any sort of suspicious activity is recorded in  any frame then the data is stored in a data file in real time. 
• The stored data is then presented in a well ordered manner in the form of a pdf as a report.

Program Flow Diagram-

Source Code -

• https://github.com/roboclub-mnnit/Proctoring_system-2022-23-Project

Video link -

• https://youtu.be/_3bf6AxE9Po

Real-life applications -

Online Proctoring

• This application can be used to conduct online exams in a more cheating-free environment.
• Can be used on sites that conduct short quizzes for certification programs.
• This can also be used in online interviews and viva to add another layer of security to ensure cheating free conduct.

Problem faced -

• MediaPipe that only works in older versions of python (<3.11).This problems lead to shifting of the whole project from Python 3.11 to 3.8 .
• Haarcascade not able to detect faces not facing screen.This problem with haarcascade created new challenges and thus at last we have dropped it for single face detection and head pose detection.
• Tkinter seems slow to build applications with modern looking GUI.

Resources

  Python:

• https://docs.python.org/3/library/tkinter.html

 OpenCV :

• https://docs.opencv.org/4.x/d6/d00/tutorial_py_root.html

  Selenium:

• https://www.selenium.dev/

   MediaPipe:

• https://developers.google.com/mediapipe

 Subprocess:

• https://docs.python.org/2/library/subprocess.html

  Tkinter:

• https://docs.python.org/3/library/tk.html

CONTRIBUTORS -

MENTORS -

1. Anurag Gupta  
2. Purushotam Kumar Agrawal  
3. Gautam Kumar