You can use the following syntax in pandas to append data to an existing CSV file:
df.to_csv['existing.csv', mode='a', index=False, header=False]
Here’s how to interpret the arguments in the to_csv[] function:
- ‘existing.csv’: The name of the existing CSV file.
- mode=’a’: Use the ‘append’ mode as opposed to ‘w’ – the default ‘write’ mode.
- index=False: Do not include an index column when appending the new data.
- header=False: Do not include a header when appending the new data.
The following step-by-step example shows how to use this function in practice.
Step 1: View Existing CSV File
Suppose we have the following existing CSV file:
Step 2: Create New Data to Append
Let’s create a new pandas DataFrame to append to the existing CSV file:
import pandas as pd #create DataFrame df = pd.DataFrame[{'team': ['D', 'D', 'E', 'E'], 'points': [6, 4, 4, 7], 'rebounds': [15, 18, 9, 12]}] #view DataFrame df team points rebounds 0 D 6 15 1 D 4 18 2 E 4 9 3 E 7 12
Step 3: Append New Data to Existing CSV
The following code shows how to append this new data to the existing CSV file:
df.to_csv['existing.csv', mode='a', index=False, header=False]
Step 4: View Updated CSV
When we open the existing CSV file, we can see that the new data has been appended:
Notes on Appending Data
When appending data to an existing CSV file, be sure to check whether the existing CSV has an index column or not.
If the existing CSV file does not have an index file, you need to specify index=False in the to_csv[] function when appending the new data to prevent pandas from adding an index column.
Additional Resources
How to Export Pandas DataFrame to CSV
How to Export Pandas DataFrame to Excel
How to Export Pandas DataFrames to Multiple Excel Sheets
How to append .csv files with Python pandas
Suppose you have a set of .csv files that you need to combine into one file, while keeping the header information.
Luckily, using the pandas package in Python, you can do this relatively easily.
In this example, you have two .csv files. homes.csv and homes1.csv
homes.csv
| 123 Main St | 99,000 | 1 |
| 4981 Anytown Rd | 199,000 | 4 |
| 132 Walrus ave | 299,001 | 2 |
| 1506 Guido St | 784,049 | 3 |
homes1.csv
| 491 Python St | 293,923 | 4 |
| 938 Zeal Rd | 148,398 | 2 |
| 247 Fort St | 299,238 | 3 |
| 992 Settled St | 823,049 | 4 |
How to append multiple .csv files with pandas
import pandas as pd
# Read in your .csv files as dataframes using pd.read_csv[]
df_homes = pd.read_csv["C:/Users/kennethcassel/homes.csv"]
df_homes1 = pd.read_csv["C:/Users/kennethcassel/homes1.csv"]
# This method combines a list of pandas dataframes into one dataframe
pd.concat[[df_homes, df_homes1]]
# pd.concat accepts a list as an argument. If you had another homes.csv
# file to append you could add it to the list
# like pd.concat[[df_homes, df_homes1, df_homes2]]
If you'd like to export the file as a new csv, your code will look like this:
import pandas as pd
df_homes = pd.read_csv["C:/Users/kennethcassel/homes.csv"]
df_homes1 = pd.read_csv["C:/Users/kennethcassel/homes1.csv"]
pd.concat[[df_homes, df_homes1]].to_csv['homes_complete.csv', index=False]
Resulting combined .csv file
homes_complete.csv
| 123 Main St | 99,000 | 1 |
| 4981 Anytown Rd | 199,000 | 4 |
| 132 Walrus ave | 299,001 | 2 |
| 1506 Guido St | 784,049 | 3 |
| 491 Python St | 293,923 | 4 |
| 938 Zeal Rd | 148,398 | 2 |
| 247 Fort St | 299,238 | 3 |
| 992 Settled St | 823,049 | 4 |
🐼 Get pandas recipes straight to your inbox!
Edit this page on GitHub