Arduino Data Logger (CSV) with Sensors and Python

If you ever wanted to generate CSV files from Arduino Sensor readings, then you'll want to read this tutorial.

Disclosure: Some of the links in this post are affiliate links. This means that, at zero cost to you, Learn Robotics will earn an affiliate commission if you click through the link and finalize a purchase. Learn Robotics is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a way for websites to earn advertising revenues by advertising and linking to Amazon.com.

Table of Contents

Update (August 2022): This tutorial has been updated to work with Python3. I’ve added the Starter Files in Step 0 to help you get this project working faster.

What good is reading analog sensor data from an Arduino if you can’t store it for later? In this tutorial, I’m going to show you how to collect and display data from multiple analog sensors using an Arduino Uno. We’re going to create an Arduino Data Logger to generate CSV files.

We’ll do this by reading multiple analog sensors on the Arduino and displaying the information to the Serial Monitor. Then, we’ll use a Python script to capture real-time data, serially, and log it into a CSV (Comma Separated Value) file.

How to generate log files for Arduino projects using Python and Excel (Arduino Data Logger CSV)

At the end of this project, you’ll have a CSV file that you can use in Excel for a variety of data analytics and decision-making projects. Essentially, we’re creating a way for you to connect Arduino Sensors to Excel to capture environment data for processing.

This project can be applied to many applications in Industrial Manufacturing, Smart Homes (IoT), and more. If you have a project that you want to add data logging to, then adding this Python script will allow you to strategically collect data for future analysis.

Step 0. Download the Starter Code Files

Note: After downloading the code, I recommend you read through and understand the code yourself. Pay attention to all the comments in the code. These provide necessary instructions.
Arduino Data Logger Starter Files
Download Now!


Heads Up, this Project is More Advanced

Props to landing on this tutorial! That means you either A) have an interesting project you want to build or B) you’re looking to gain intermediate skills with Arduino and Python. Both are great places to be in, but I want to set some expectations before you head into this tutorial.

While this is an in-depth tutorial, it does require prior experience with simple analog circuits, Arduino sketches, and Python programming.

That means I’ll be covering high-level steps on what you need to do to get this working. Unfortunately, I will not cover these topics in-depth.

If you’re brand-new to Arduino, sensors, circuits, and coding, I highly recommend enrolling in my Robotics Specialist Certification, before attempting to piece this project together. As a result, you’ll gain professional skills in hardware I/O, data collection, decision-making, and mechatronics systems.

If you’re new to Python, here are a few Python courses I recommend checking out.

Now, are you ready to get started?

Step 1. Connect Analog Sensors to Arduino

First, wire up the analog sensors that you want to use for this project. This project requires at least two analog sensors. I’ve chosen a rotational potentiometer and a photoresistor.

Here’s the Fritzing Diagram for the circuit used in this example.

wire multiple analog sensors to Arduino Uno

The potentiometer is connected to A0 and the photoresistor is connected to A1. You can use whichever open pins you’d like.

Once you have your circuit wired, it’s time to write the Arduino Sketch.

Recommended Kit: Learn Robotics Mini Kit

Step 2. Write Arduino Sketch to Read & Send Data Serially

Next, open up a new Arduino sketch and assign two global variables to pins A0 and A1 (or whatever pins your sensors are connected to). Then, begin the Serial Monitor at 9600 baud, and set these sensors as INPUTS in the setup() method.

int sensor1 = A0;
int sensor2 = A1;

void setup(){
  // put your setup code here, to run once:
  Serial.begin(9600);
  pinMode(sensor1, INPUT);
  pinMode(sensor2, INPUT);
}

Now we can develop our loop() method. Create a few globals to store data from both of the sensors. Then, use the analogRead() method to read the pin. Add a delay so that we’re not collecting data too fast.

//globals
int data1, int data 2; //store data from both sensors
int freq = 1000; //data collection frequency ~x milliseconds

void loop(){
    data1 = analogRead(sensor1);
    data2 = analogRead(sensor2);

    delay(freq);
}

We have the data, now we need to display it on the Serial Monitor.

We’re going to use the CSV format for our data, so the data should look like this: data1,data2

//globals
int data1, int data 2; //store data from both sensors
int freq = 1000; //data collection frequency ~x milliseconds

void loop(){
    data1 = analogRead(sensor1);
    data2 = analogRead(sensor2);

    //Display Data to Serial Monitor
    Serial.print(data1);
    Serial.print(",");
    Serial.println(data2);

    delay(freq);
}

This code works as-is, and you can upload it to your Arduino and test it out. Open a Serial Monitor, and you should have readings that look like the following.

Print out values from Arduino as a CSV file

Idea A: Display Data Only if the Sensor Readings are Different

Sometimes, you’ll want to impose conditions on your data collection. A good example of this is if the current reading is the same value as the previous reading.

In some cases, you may not want to print out the same reading every single time. You’ll only want to log a reading if it’s different than previously.

To do this, we can set some global variables equal to the current reading and then collect a new reading for comparison.

//globals
int curr1, int curr2;

void loop(){
    data1 = analogRead(sensor1);
    data2 = analogRead(sensor2);
    
    if (curr1 != data1 || curr2 != data2){

        //Display Data to Serial Monitor
	Serial.print(data1);
	Serial.print(",");
	Serial.println(data2);

	//set the current equal to the data
	curr1 = data1;
	curr2 = data2;
	
	delay(freq);
    }
}

Idea B: Print Data based on Conditional Ranges

Furthermore, sensor readings can be a bit picky and oscillate between +/- 1 of the same reading. So, if you run the code above, and you’re not happy with how it’s working, you may need to set up a conditional range.

For example, you can display data if the new reading is +/- 5% of the current reading. To set this up, you can create a couple of globals to easily adjust your thresholds. 1024 comes from our analog pin, which has a 10-bit resolution. (2^10 = 1024). Therefore 5% = 1024*.05 = 51.2, which is rounded to 51.

Note: This will trigger the next increment of the threshold the next time the sensor meets this requirement unless you add a delay.
//globals
float percent = 0.05;
int threshold = 1024*percent; //within x% in either direction

We can use this threshold in our conditional statement.

//globals
float percent = 0.05;
int threshold = 1024*percent; //within x% in either direction

void loop(){
    data1 = analogRead(sensor1);
    data2 = analogRead(sensor2);

    if((curr1 >=data1+threshold || curr1 <=data1-threshold) || (curr2>=data2+threshold || curr2<=data2-threshold) ){

        //Display Data to Serial Monitor
	Serial.print(data1);
	Serial.print(",");
	Serial.println(data2);

	//set the current equal to the data
	curr1 = data1;
	curr2 = data2;
    }
    delay(freq);
}

Rather than checking to see if the current value is different than the new value, we can check to see if the current value is greater than or equal to the new value + 5% or if the current value is less than or equal to the new value – 5%.

And that’s basically it! Upload the code to your board, and open the Serial Monitor. You can adjust the percentage and frequency so that data is displayed most appropriately for your application.

Idea C: Show Column Headers in the CSV File

Now, this section is completely optional, but if you want to stylize your CSV file, you’ll want to send over your column headers. This is what will be displayed in the first row of Excel.

I used the names of the sensors as my column headers, but you can use whatever you like.

To do this, you can use some global variables to store the header names. Then, set up a loop that runs once and prints them out. Make sure you use the CSV format!

I also set a boolean so that we only run this loop once.

//gloabls
String dataLabel1 = "Potentiometer";
String dataLabel2 = "Photoresistor";
bool label = true;

void loop(){
	
    //print out column headers
    while(label){ //runs once
        Serial.print(dataLabel1);
        Serial.print(",");
        Serial.println(dataLabel2);
        label=false;
  }

    data1 = analogRead(sensor1);
    data2 = analogRead(sensor2);

    if((curr1 >=data1+threshold || curr1 <=data1-threshold) || (curr2>=data2+threshold || curr2<=data2-threshold) ){

	//Display Data to Serial Monitor
	Serial.print(data1);
	Serial.print(",");
	Serial.println(data2);

	//set the current equal to the data
	curr1 = data1;
	curr2 = data2;
    }
    delay(freq);
}

This completes the code on the Arduino side. Feel free to make modifications or adjust it to capture things that best suit your project.

Now, it’s time to move into Python and develop the code that can read and log data from our serial connection.

Step 3. Develop Python Code to Read Serial Data from Arduino

The third step is to create a new Python file and import the serial module and the csv module. Then, set a few variables for the port the Arduino is on the baud rate, and the CSV file name.

import serial
import csv

arduino_port = "/dev/cu.usbmodem14201" #serial port of Arduino
baud = 9600 #arduino uno runs at 9600 baud
fileName="analog-data.csv" #name of the CSV file generated

The Arduino serial port will be in the format “COMX” on Windows or “/dev/cu.usbmodemxxx” on Mac. Use the same address that you selected in the Arduino IDE under Tools > Port.

Next, set up the serial connection and create the file. You can use the input parameter “w” to write a new file or “a” to append to an existing file.

ser = serial.Serial(arduino_port, baud)
print("Connected to Arduino port:" + arduino_port)
file = open(fileName, "a")
print("Created file")

I added a few print statements to let the user know what’s happening in the code.

Step 4. Create an Arduino Data Logger: Send Serial Data into a CSV File

After that, you can read the serial port, parse the data, and print it to the terminal. Then, add the data to the file.

#display the data to the terminal
getData=ser.readline()
dataString = getData.decode('utf-8')
data=dataString[0:][:-2]
print(data)

readings = data.split(",")
print(readings)

sensor_data.append(readings)
print(sensor_data)

Technically, we’re finished collecting readings, here but if you’d like to set the number of samples to collect, you can throw everything into a while loop, like so.

samples = 3 #how many samples to collect
print_labels = False
line = 0 #start at 0 because our header is 0 (not real data)
sensor_data = [] #store data

# collect the samples
while line <= samples:
    getData=ser.readline()
    dataString = getData.decode('utf-8')
    data=dataString[0:][:-2]
    print(data)

    readings = data.split(",")
    print(readings)

    sensor_data.append(readings)
    print(sensor_data)

    line = line+1

You’ll notice that I’ve added a boolean that allows you to choose if you want to display the current line number or if you want to just see raw data. This, again, is all optional, but I think it’s a nice addition.

Step 5. Add the Arduino Serial Data to a CSV File

Now that we have the readings, it’s time to add them to our CSV file. We use the CSV module to take each reading from the sensor_data variable and generate new rows in the CSV file.

There are more complex ways of doing this, such as adding the data to a dictionary and then using key/value pairs. You can add to this starter code to make it more complex and have data in the right format you need for your project.

Here’s how this part of the code is written:

# create the CSV
with open(fileName, 'w', encoding='UTF8', newline='') as f:
    writer = csv.writer(f)
    writer.writerows(sensor_data)

print("Data collection complete!")
file.close()

Once you have the code written, be sure to save it. I named mine read-serial.py.

Step 6. How to use the Python Code for your Arduino Data Logger (CSV)

It should go without saying, but make sure you have the code written from Step 2 uploaded to your Arduino before running the Python script.

Additionally, be sure that your Arduino is connected to your computer’s USB port. Otherwise, none of this will work.

Next, open up a terminal window, navigate to the folder where your code is saved, and run the code using the following command.

python read-serial.py

You should see 10 samples flow through, so long as the data has changed based on your conditions (from Step 3).

Use Python to read Arduino Sensors and log to a CSV

You’ll also have another file called analog-data.csv added in the same directory as your Python script.

Create an Analog Data CSV

You can open this file with a text editor or with Excel to view the data readings.

How to view Arduino Data using Excel

And that’s how you get real-time readings from the Arduino Uno into Excel using Python and serial communication.

I hope you found this tutorial useful for your next project!

What project are you working on that requires Arduino data logging? Let me know in the comments below!

Related Articles

11 Responses

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Invest in Your Future Tech Career & Get Matched with an Expert Robotics Mentor

Connect with your Advisor and select the right Learn Robotics Program to boost your Tech Career

Wait,

Learn Robotics Online
— Get 2 months free!

Exclusive Limited Offer for Serious Beginners Ready to take their Hobby to Engineering Internships, $100k+ Careers, and Beyond!

Enroll now, and earn your first robotics certificate in the next 7 days.

👇 Click below to claim this deal.