The AI-Powered Virtual Drawing and 3D Modeling Platform is an innovative system designed to provide a touchless and interactive digital drawing experience using Computer Vision, Artificial Intelligence, and Hand Gesture Recognition. Traditional digital drawing systems often require expensive hardware such as graphic tablets, stylus pens, or special sensors, making them less accessible for students, artists, and general users. This project addresses that limitation by developing a low-cost virtual drawing platform that works using only a standard webcam and hand gestures. The system captures real-time video input through a camera and uses MediaPipe for accurate hand landmark detection and tracking. OpenCV is used for video processing, image rendering, and creating the interactive virtual canvas where users can draw without physically touching any device. The platform supports multiple functionalities such as freehand 2D drawing, erasing, color selection, brush thickness control, undo operations, and saving the final output. Different hand gestures are assigned to specific commands like drawing, erasing, selecting tools, and clearing the canvas, creating a smooth and natural interaction between the user and the system. In addition to 2D virtual drawing, the project also includes AI-based shape detection and auto-correction, where rough hand-drawn shapes such as circles, rectangles, triangles, and straight lines are automatically converted into perfect geometric figures using machine learning techniques and mathematical fitting algorithms. The system further extends its capabilities with a gesture-controlled 3D viewer module developed using OpenGL and PyOpenGL, allowing users to rotate, zoom, translate, and switch between 3D objects like cubes, spheres, pyramids, and cylinders. A machine learning model is trained using synthetic datasets for improved gesture recognition and intelligent shape prediction. The system introduces a gesture-based shape repositioning feature, where users can grab any drawn element using a closed-fist gesture and drag it to a new location on the canvas — eliminating the need to redraw. A built-in voice command module lets users issue drawing instructions entirely through speech, which makes the platform genuinely usable for people with motor impairments. Collaborative drawing support allows multiple users to share a live canvas over a local network. Multiple machine learning models including a trained MLP shape detector, a CNN-based gesture classifier, and a Universal RL Classifier are combined through confidence-weighted ensemble voting for stronger real-world performance.
Pranay et al. (Tue,) studied this question.