Mirror Learning

Project Context

Why I choose AI?

Artificial intelligence (AI), also named as machine learning, was founded on the assumption that human intelligence "can be so precisely described that a machine can be made to simulate it", and nowadays, AI has reached peak hype.
So, I felt it significant to really understand how the technology works, and had the ideas to use ML5 language to create a project for action mimic: An AI Experiment with Pose Estimation in the Browser using TensorFlow.js

What is Mirror Learning

The main part of the project looks like this:

You turn on your webcam and move around, and the computer pulls up pictures of poses that match yours in realtime. The image database is made of more than 80,000 pictures - of people dancing, doing karate, cooking, walking, skiing and so on.

Target

I would use this kind of human-computer interaction way to develop a website for self-learning like dance and fitness, the uses can go along with the poses and been tested to verify the motion accuracy. I believe this would add so much fun and make self-learning more professional, more convienient and more flexible for free.

I was inspired from Move Mirror which allows you explore pictures in a fun new way.

Try its demo here:

Pre-learning

What will I use?

I will use ml5.js, a JS library providing access to AI algorithms and models in the browser, to create the main function.
The function will be built on Human motion capture function of TensorFlow.js, to make this kind of self-learning approachable for a broad users
For running real-time pose estimation in the browser using TensorFlow.js., I will use a package that contains a standalone model called PoseNet, as well as some demos,
The human motion capture function is a real-time motion capture basing on a browser and a web camera, the capture motions will be analyzed by model library (like tfjs-models / posenet or tfjs-models / body-pix).

How to code

First step-make reference to the needed files :

we can use this as standalone es5 bundle like this:

HTML

Before step of pose estimation, we need to load a pre-trained PoseNet Model

Next steps:

Initialize the Webcam

Generate Posenet Objects

Simultaneous analysis

Go further

More information on study:

For new starters, you can use ml5: Friendly Open Source Machine Learning Library for the Web to study.

TensorFlow.js is a library for machine learning in JavaScript

Here are informations about my inspiration and the funny interface. Now, you can start to create your own AI with magic imagiantion!
this video below may give you a basic idea on how to get started with ml5.js: