We’re here to help in the event of a school closure. See our resources.
Face Tracking OpenCV

Face Tracking OpenCV, Python, & Arduino

We sometimes use affiliate links in our content. This won't cost you anything, but it helps us to offset the costs of paying our writing team. You can support us directly on BuyMeACoffee. Thank you!

Online Robotics Course by Learn Robotics
  • Facebook
  • Twitter
  • Pinterest
  • Buffer
  • reddit
  • LinkedIn
In a previous article, I showed you how you can establish communication between Arduino and Python.  In this tutorial, I will show you how you can use OpenCV, Python, and Arduino to detect and track faces. Face tracking can be used in a variety of robotics projects and applications.

Once you learn the basics from this face tracking OpenCV project, you can use your imagination to put these skills to work! So without wasting any more time, let’s get right into it.

Face Tracking OpenCV Project Materials

For this project, you’ll need an Arduino Uno, servos, pan-tilt kit, breadboard kit, and webcam. (Laptop built-in camera also works.) Along with the hardware components, you will also need the following software:

Set Up the Python Environment

First, we need Python 2.7 up and running. To do this first download and install python 2.7.14.
To check if it is installed correctly Goto: Windows Search >> Type “IDLE” >> Hit Enter.
You should see a Python shell that looks something like the photo below.

python opencv
  • Facebook
  • Twitter
  • Pinterest
  • Buffer
  • reddit
  • LinkedIn

You can also use the Command Prompt and script from the terminal. In search bar type ‘CMD,’ and hit enter to open Command Prompt. In CMD type >> python and hit enter. The Python interface will look similar to the photo below.

face tracking opencv
  • Facebook
  • Twitter
  • Pinterest
  • Buffer
  • reddit
  • LinkedIn

If you see an error in CMD, do not worry. You probably need to set the environment variable. You can follow this tutorial here to set up the environment variable.

Robot servo 25kg RDS3225 Metal Gear Digital servo...
equipped with strong copper & aluminum gears,CNC aluminium middle Shell; Professional servo kit easy to design and do all kinds of robot
$28.00

Install ‘pyserial’, ‘OpenCV” and “NumPy” in Python

To install these modules we will use use pip install. First, open CMD and type the following codes:

>pip install serial
>pip install opencv-python
>pip install numpy

Install each of the modules one by one. These modules will help us detect and recognize objects (face tracking in this case). And, they’ll help us connect our Arduino board to Python.

Face Tracking with Python

Before starting to write code first thing to do is make a new folder as all of the code needs to be stored at the same location. Next, create a new folder, and name it anything you want. Now download the ‘Haarcascade‘ paste it in the folder. Next, download the script given below. Save it as ‘face.py’ in the same folder as haarcascade.

Note: Instead of just downloading the code, I recommend you read through and understand the code yourself. Pay attention to all the comments in the code. These provide necessary instructions.

Arduino Face Tracking Code

After the Python script is ready, we need to create an Arduino sketch to control the servos. Refer to the code below. Copy and paste it in a new Arduino sketch.

Save the sketch as ‘servo.ino‘ in the same folder as face.py and haarcascade. Upload the code and move onto the next step to make the connections.

#include<Servo.h>

Servo servoVer; //Vertical Servo
Servo servoHor; //Horizontal Servo

int x;
int y;

int prevX;
int prevY;

void setup()
{
  Serial.begin(9600);
  servoVer.attach(5); //Attach Vertical Servo to Pin 5
  servoHor.attach(6); //Attach Horizontal Servo to Pin 6
  servoVer.write(90);
  servoHor.write(90);
}

void Pos()
{
  if(prevX != x || prevY != y)
  {
    //tune this range to generate map
    int servoX = map(x, 600, 0, 70, 179); 
    //tune this range to generate map
    int servoY = map(y, 450, 0, 179, 95); 

    servoX = min(servoX, 179);
    servoX = max(servoX, 70);
    servoY = min(servoY, 179);
    servoY = max(servoY, 95);
    
    servoHor.write(servoX);
    servoVer.write(servoY);
  }
}

void loop()
{
  if(Serial.available() > 0)
  {
    if(Serial.read() == 'X')
    {
      x = Serial.parseInt();
      if(Serial.read() == 'Y')
      {
        y = Serial.parseInt();
       Pos();
      }
    }
    while(Serial.available() > 0)
    {
      Serial.read();
    }
  }
}

Pan-Tilt Mechanism

I have used a readily available kit for the Pan-Tilt. If you want you can make one yourself using wood or plastic, or you can even 3D print one. I used a kit you can get on Amazon, Gearbest, or Banggood.

pan-tilt servos
  • Facebook
  • Twitter
  • Pinterest
  • Buffer
  • reddit
  • LinkedIn

You can use our guide to assemble the Pan-Tilt mechanism properly.  Once you finish that, it’s time to move on to the electronics.

Sale
American Robotic Supply Pan/Tilt Kit with Servos
Lightweight - good for aircraft cameras; Proven Operation - Simple and Effective; Kit comes with two Authentic Tower Pro SG-92R servos - Strong and Reliable
$19.95 −$2.00 $17.95

Wiring Diagram for Face Tracking OpenCV

The circuit is pretty simple. Just attach two servos to Arduino as shown in the wiring diagram below.

wiring diagram face tracking opencv
  • Facebook
  • Twitter
  • Pinterest
  • Buffer
  • reddit
  • LinkedIn

The vertical servo signal pin goes to digital pin 5. The horizontal servo signal pin goes to digital pin 6. Power goes to +5V and Ground goes to Ground.

Note: To reduce any jitters, connect an electrolytic capacitor in parallel to the power supply.

Here’s what the final assembly looks like.

final setup face tracking arduino opencv
  • Facebook
  • Twitter
  • Pinterest
  • Buffer
  • reddit
  • LinkedIn

Test the Face Tracking OpenCV project

After everything is done, the last thing to do is to test to see if it works. First, make sure that the servos are properly connected to the Arduino and sketch has been uploaded.

After the sketch is uploaded, make sure to close the Arduino IDE so that the Serial port is free to connect to Python.

Next, open face.py with Python IDLE and press ‘F5’ to run the code. You can also navigate to the project directory in terminal and run face.py code directly from the command line.

Sale
ELEGOO UNO R3 Project Most Complete Starter Kit...
All the modules come soldered so they are easy to wire up.
$58.99 −$5.00 $53.99
Sale
Miuzei 10 pcs SG90 9G Servo Motor Kit for RC Robot...
Stall Torque (4.8V): 17.5oz /in (1kg/cm) Dead band width: 7usec; Amplifier type:Analog controller Operating voltage: 4.8V~6.0V
$22.99 −$4.10 $18.89

It will take a few seconds to connect to Arduino. Then you should be able to see a window streaming the webcam. Now the code will detect your face, and the servos will track it. You may have to tune the pan and tilt servo range to best suit the coordinates given by Python & OpenCV. Check out the video to see how you can tune the readings for more dynamic control.

If everything is set up correctly, the servos should move as you move. As an added feature, you can mount the camera directly on the servos. Then the mechanism will mimic your movements while you’re looking directly at it.

That’s all for this tutorial. If you have any questions, be sure to find us on Facebook.

You can also check out our online Arduino Course. It’s designed to help students learn the fundamentals of Arduino to be successful with more advanced projects like this one.

You May Also Like

Have a question? Need Help?

Thanks for dropping by! Comments are no longer moderated daily.
Before posting your comment, please consider buying me a coffee. If you need immediate support, you can schedule an appointment. Providing customized information for your individual scenario takes a lot of my time. I’m happy to help, but I can no longer provide individualized advice pro bono.
Thank you for understanding.

78 comments

  1. Hi again Liz,
    I checked everything, the com port, the code and everything is hooked up correctly. But It doesn’t seem to work still I know the program is uploaded to the arduino but when I play the python code, the coordinates show up but the servos don’t move. Thanks again

    1. Liz Miller

      Hi Dan, Unfortunately, I can’t provide complementary debugging via the blog comments. Maybe try to see if you can control the servos using code from Arduino to verify that the servos are working properly. Thanks for understanding. Good luck ~Liz from Learn Robotics

  2. MANU MOHAN

    how to tune the mapping from OpenCV to the servo output range
    can you explain plz

    1. Liz Miller

      Hi Manu, Unfortunately, I can’t provide complementary debugging via the blog comments. Thanks for understanding. Good luck ~Liz from Learn Robotics

  3. Hi Liz,
    I have everything set up and the servos work and are connected properly, the python code runs and shows the coordinates. but the Servos don’t move. How can I get the python data from VSCode to communicate with the Arduino IDE?

    1. Liz Miller

      Dan, make sure you have the correct Serial port set up in the Python code. If you found value in this post, please consider sending us a coffee! Good luck! ~Liz from Learn Robotics

  4. I get this error message when I upload the sketch to arduino board.
    Using Port : COM3
    Using Programmer : arduino
    Overriding Baud Rate : 115200
    An error occurred while uploading the sketch
    avrdude: ser_open(): can’t open device “\\.\COM3”: The system cannot find the file specified.

    1. Liz Miller

      Divya, if you’re using an Arduino Uno, the baud rate is 9600 baud. Make sure you also change the code to point to the correct serial port that the Arduino is connected to. If you’re still having trouble, you can schedule a coaching appointment and we’ll help you get things sorted out. Unfortunately, we can’t provide complementary debugging via the blog comments. Good luck ~Liz from Learn Robotics

  5. the Face.py file which I downloaded is not commented.

    comments would be extremely helpful

    1. Liz Miller

      Hi James, thanks for your recommendation. We do our best to add comments for critical steps. If you’re having trouble reading the code, you can reference the tutorial for an explanation of what each section does. ~Liz from Learn Robotics

  6. Aayush Lakkad

    Hey Liz,
    I am trying to use this arduino code to track objects based on color, I wrote the codefor color detection and finding its center, and I am communicating with python in the same way as you are.
    But as soon as I bring object infront of the camera it starts moving continously in random direction. can you tell me why this is happening?

    Thank you in advance
    Aayush Lakkad

    1. Liz Miller

      Aayush, thanks for dropping by. I recommend trying to map the servo movements to match the size/range of the detected object. Since this is considered a custom application, I can’t provide a complementary solution via the blog comments. Thanks for understanding. Good luck ~Liz from Learn Robotics

  7. Mit Mistry

    Hey,

    how would I covert the face.py program if I wanted it to recognize objects?

    Thank you

    1. Liz Miller

      Hi Mit, Since this is an individualized project request, I cannot provide a complementary solution via the blog comments. However, I recommend looking at OpenCV’s documentation for Object Detection. That would be a good starting point. Good Luck ~Liz from Learn Robotics

  8. The newest opencv doesn’t work on macOS. But, the errors don’t clearly point you to this reason. Try:

    pip install opencv-python==4.1.2.30

  9. Hi Liz, thx for the tutorial.
    I have a problem… wen i run on idle on the “Getting camera image…” the app crashd and do not work… Thx

    1. Liz Miller

      Hi Lorenzo, thanks for dropping by. I appreciate your comment; however, since it is an individual project, I cannot provide a custom solution via the blog comments. Feel free to run through the setup again and make sure that you have configured the hardware and software as outlined in the tutorial. Thanks for understanding! ~Liz from Learn Robotics

  10. Ben Jack Guru

    it shows error:
    arduino = serial.Serial(‘COM3’, 9600)
    AttributeError: module ‘serial’ has no attribute ‘Serial’

    1. Liz Miller

      Ben, try adding this line with the import statements at the top of the code:
      from serial import Serial
      Make sure serial is installed on your computer. (on Mac: pip install pyserial)
      You can also look through previous comments, as this error was resolved in a previous thread. ~Liz from Learn Robotics

  11. Serhat Saday

    code laptop 1.3 mp webcam also works. But why Raspberry pi and standard 5Mp camera arduino make servos different movements.

    1. Liz Miller

      Serhat, you’ll have to run some tests! Unfortunately, since this is individualized work, I can’t provide a complementary solution via the blog comments. Thanks for understanding! ~Liz from Learn Robotics

  12. Serhat Saday

    Why this project does not work under raspberry pi raspbian buster. Servos move irrationally

    1. Liz Miller

      Serhat, are you using the GPIO to control the servos or an Arduino?

  13. Serhat Saday

    Hi zik i am using ubuntu 18.04 os .
    I tried the mac version of face.py

    use #cv2.resizeWindow(‘img’, 500,500)

    I changed it to #

    open usb permissions on terminal
    like ‘sudo chmod 666 /dev/ttyUSB0’

    it working

    1. Liz Miller

      Thanks, Serhat for your input!

  14. Richard

    Hola a todos los que no logran mover los servos en python 3.7 el problema se encuentra en tipo de variable al cambiar de doble a floar hay problemas entonces deben colocar 0.f En la siguiente parte del codigo

    Hello to all who do not manage to move the servos in python 3.7 the problem is in variable type when changing from double to float there are problems then you must place 0.f
    In the next part of the code

          ("X{0:.0f}Y{1:.0f}Z".format(xx, yy))
            arduino.write(data.encode())
    
    1. Liz Miller

      Thank you Richard for your input!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Free Learn Robotics Swag

Patch/Sticker with every LR Kit

Work with an Engineer

Live Chat with an Instructor

365 Day Course Access

1 year access to every course

100% Secure Checkout

PayPal / MasterCard / Visa

Become a Learn Robotics Insider

Sign up and receive product announcements and discounts!

 

Success! Check your email for a surprise!

Pin It on Pinterest