Migrating 750.000 Emails from FreshSales into HubSpot

Challenge

Our client provided us with a sizable dataset that included 750.000 folders from FreshSales, each of which contained a number of subfolders that held important email data. The first and most difficult task was to convert this convoluted structure into a structured and readable format, more precisely, a CSV file.
In this complex migration effort, processing multiple email circumstances (including adding CC and BCC emails), validating data, generating contacts in HubSpot, and appropriately associating emails to the appropriate contacts constituted the second challenge we faced.

Solution

Converting the folder structure into a CSV file

We designed and implemented a PHP script that not only facilitated the transformation process but also added a layer of data validation and organization. The script performed the following actions:

Traversed the folder structure from FreshSales, extracting email data
Validated the data to ensure the accuracy of the extracted information
Structured it into a CSV file

The process of migration into HubSpot

We implemented an unique ID system for each email within the dataset
A comprehensive validation of data was established for the email upload process in HubSpot
Batches were used to send data to HubSpot asynchronously, greatly boosting the effectiveness of contact and email creation
Multiple recipients’ emails, including those in the CC and BCC fields, were logically linked to their respective contacts
The emails were linked to the contact object that matched it, if any. A new contact was created in circumstances where there was no matching contact

Highlights and results

Data validation

Ensuring data integrity and correctness by conducting thorough validation checks

Complex data transformation

Creating a structured CSV format out of complex data structures

Programming and SQL Integration

Data processing was optimized, increasing both speed and reliability by using different programming approaches and SQL integration.

Successful migration

750.000 emails were successfully transferred into HubSpot as the project's result, meeting the client's expectations.

Optimized processes

Data handling and storage were optimized through the use of a PHP script, SQL integration, and database management, creating more effective processes.