Overview

AIDE is an anatomy explorer you control with your hands. A Python backend reads the webcam with OpenCV, tracks hand landmarks with MediaPipe, turns them into gestures and streams them to the browser over Socket.IO, where a Three.js scene rotates, zooms, highlights or explodes a 3D organ model.

My role

The code was first uploaded by another developer as a flat set of files that would not start. In one commit I restructured it into a working Flask application, fixed the dependency and startup problems, and reworked the Three.js scene around real organ models. The hand-tracking and gesture-detection modules are the original author's.

What it does

• Starts the webcam on request and runs a loop that mirrors each frame and extracts hand landmarks.
• Detects one-hand gestures (point, rotate) and two-hand gestures (zoom, explode, implode) and pushes them to the page in real time.
• Shows heart, brain, lungs and kidney models with labelled parts on a holographic grid, with an exploded view that pulls a model apart.
• Falls back to a mock tracking mode when MediaPipe's hands solution is missing, so the interface still loads.

How it's built

Flask serves the page and Flask-SocketIO on eventlet carries camera control and gesture events. The front end is plain JavaScript with Three.js, GLTFLoader, OrbitControls and CSS2D labels. One honest limit: on Python 3.14, MediaPipe ships without the legacy hands solution, so the app runs in mock mode there; real tracking needs Python 3.10 to 3.12.

Highlights

• Getting it to run: moved files into the layout the imports expected, raised an eventlet pin that still imported the removed distutils module, moved eventlet's monkey-patching ahead of every other import, and forced UTF-8 console output so emoji logging stopped crashing on Windows. Each fix is written up in the repo.
• Correct-looking materials: sRGB output with ACES tone mapping, a generated environment map for reflections, rebalanced lighting, and a pass that repairs common glTF export problems such as black, near-transparent or metallic-without-environment surfaces.
• Exploding single-mesh models: scanned models often ship as one mesh, so the scene splits its triangles into a 2x2x2 grid of chunks by centroid and recentres each one, giving the explode animation real pieces to fan out.