facemood

a web app that guesses your mood from your webcam. it finds 468 points on your face with tensorflow.js, turns them into a few signals (eyes open, smiling, brow raise, head tilt), adds how much you're moving and typing, and picks a mood. it all runs in the browser, so nothing gets uploaded.

it won me some money and a spot on cash app's teen advisory council.

a webcam frame with a green box drawn around a face, a column of labelled bars down the left side, and the text mood: chill at the bottom — an early test, running on daniel. this version scored eight emotions and picked the top one. the version that shipped reads the face signals below instead

reading a face

the face model hands back 468 landmarks every frame, and each signal is a ratio between a few of them. eye openness is the eye aspect ratio, the two vertical gaps across the eye divided by its width, averaged over both eyes. the mouth works the same way, height over width. using ratios means it doesn't matter how close you sit to the camera.

const LEFT_EYE = [33, 160, 158, 133, 153, 144];
const RIGHT_EYE = [263, 387, 385, 362, 380, 373];

function eyeOpenness(points, eye) {
  const [corner1, top1, top2, corner2, bottom2, bottom1] = eye.map((i) => points[i]);
  return (dist(top1, bottom1) + dist(top2, bottom2)) / (2 * dist(corner1, corner2));
}

function eyeAspectRatio(points) {
  return (eyeOpenness(points, LEFT_EYE) + eyeOpenness(points, RIGHT_EYE)) / 2;
}

function mouthAspectRatio(points) {
  return dist(points[13], points[14]) / dist(points[61], points[291]);
}

picking a mood

motion is how much the picture changes from one frame to the next, and typing speed is keys per minute. those two plus the face signals go through a few rules, and the first one that matches wins. the rules are just thresholds.

function inferMood({ eyesOpen, smiling, motion, keysPerMinute }) {
  const arousal = Math.max(motion, eyesOpen ?? 0);
  if (keysPerMinute > 220 || arousal > 0.8) return "energetic";
  if (keysPerMinute > 120 || (arousal > 0.5 && smiling < 0.3)) return "focus";
  if ((smiling ?? 0) < 0.15 && arousal < 0.25) return "melancholy";
  if (arousal < 0.45) return "chill";
  return "happy";
}

download the code (35 kb zip)