AirPointer Documentation — AI Hand Tracking, Landmarks & Calibration

AirPointer Virtual Mouse

AirPointer is a high-performance computer vision system that transforms webcam video feeds into real-time, low-latency, touchless cursor input. Built on top of Google MediaPipe Hands and OpenCV, it tracks 21 three-dimensional anatomical hand landmarks to calculate coordinate deltas, pinch state transitions, and gesture triggers.

Core FrameworkOpenCV 4.x + MediaPipe Hands

Target OSLinux (X11/Wayland), Windows, macOS

FPS & Latency60 FPS @ 4.2ms Processing Delta

LicenseMIT Open Source

Installation & Environment Setup

AirPointer requires Python 3.9 or higher and access to an active webcam device.

# Clone the repository
git clone https://github.com/dev-hints/AirPointer.git
cd AirPointer

# Setup a clean virtual environment
python3 -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# Install core Python dependencies
pip install -r requirements.txt

# Run with default webcam (index 0)
python airpointer.py

System Architecture

The internal pipeline processes frames in four synchronized stages without blocking the main event loop:

[ Webcam Frame (60 FPS) ]
          │
          ▼
[ Frame Preprocessing ] ───> Normalization & Contrast Enhancement
          │
          ▼
[ MediaPipe Hands DNN ] ───> Extracts 21 3D Spatial Landmark Vectors (x, y, z)
          │
          ▼
[ Coordinate Mapper ] ────> Screen Dimension Normalization & Deadzone Clipping
          │
          ▼
[ Kalman Filter ] ────────> Removes Micro-Tremors & Jitter
          │
          ▼
[ Gesture State Machine ] ──> Left/Right Click, Double Click, Drag, Scroll
          │
          ▼
[ OS Input Dispatcher ] ───> PyAutoGUI / pynput Kernel Event Injection

21 MediaPipe Hand Landmarks

AirPointer tracks 21 landmark nodes across the palm, wrist, and five fingers:

Index ID	Landmark Name	Role in AirPointer
`0`	Wrist	Global anchor for hand orientation
`4`	Thumb Tip	Click gesture pinch pairing
`8`	Index Fingertip	Primary cursor position reference
`12`	Middle Fingertip	Right click (V-Sign) & Scroll detection
`16`	Ring Fingertip	Resting state confirmation
`20`	Pinky Tip	Special macro trigger

Gesture Recognition State Machine

State transitions are calculated using Euclidean distance formulas between normalized landmarks:

Distance = sqrt((x1 - x2)² + (y1 - y2)²)

Action	Landmark Trigger Condition	Debounce Window
Move Cursor	Index tip (8) extended, Thumb (4) > 40px away	0 ms (Continuous)
Left Click	Distance(Thumb 4, Index 8) < 30px	120 ms
Right Click	Distance(Index 8, Middle 12) in V-Shape pose	200 ms
Drag & Drop	Sustained pinch hold > 350 ms	Continuous until release
Scroll Up / Down	Index (8) and Middle (12) parallel vertical delta	40 ms

Cursor Smoothing Algorithms

Webcams naturally introduce 1-3 pixel noise on stationary fingers. AirPointer employs a dual-stage smoothing filter: an Exponential Moving Average (EMA) for high-speed tracking and a 2D Kalman Filter for stationary sub-pixel stabilization.

# Smoothing calculation formula
smoothed_x = prev_x + (raw_x - prev_x) * smoothing_factor
smoothed_y = prev_y + (raw_y - prev_y) * smoothing_factor

CLI Flags & Options

CLI Flag	Default	Description
`--camera`	`0`	Webcam device index
`--smoothing`	`0.75`	Smoothing coefficient (0.1 to 1.0)
`--deadzone`	`5`	Pixel deadzone to prevent drift
`--debug`	`False`	Display HUD overlays and landmark lines
`--flip`	`True`	Mirror webcam feed horizontally

Troubleshooting & FAQ

Camera fails to open Check permissions or specify another index: python airpointer.py --camera 1

Cursor movement feels sluggish Increase smoothing factor: --smoothing 0.9 or lower webcam capture resolution in config.

Linux Wayland permission error Run inside an X11 session or configure xdotool permissions for user.

AirPointer Virtual Mouse Documentation