Optimize marketing campaigns with Hazlo

Knowing how a customer responds to a sales campaign is essential in evaluating it’s rollout — did they send in a query, or not? Completed a final purchase? Cost of acquisition? However, it would be even more beneficial if this could be done pre-emptively: knowing the likelihood of a customer responding to a campaign before we’ve even rolled it out to them. That’s exactly what we’re going to do through Hazlo.

Obtaining & uploading a dataset

Like always, we’ll be obtaining a .csv file & uploading it to Hazlo. This time, we’ll use a public domain dataset available at the Hazlo’s GitHub Datasets page — https://bit.ly/3bM1gmg (the .csv is available as marketing-dataset.csv). Here you’ll also find feature-specific information about the dataset: it includes information about customers with their habits on an ecommerce platform & if they responded to a marketing campaign or not.

A feature-rich dataset will lend itself to some great models down the road.

Post downloading the dataset, we can simply upload it to Hazlo. Go to the “Datasets” page & simply press “Upload”.

Data tags & context help Hazlo better augment the dataset during model training.

Understanding the data

After the upload is complete, we can view relevant meta-data & columnar information in our datasets page. At first glance, it seems that users who had an above average number of catalogue purchases had a higher chance of responding to a campaign.

Users with >1 NumCatalogPurchases seem more likely to respond.

As for dataset augmentation & cleaning, though there were quite a few outliers, sparse columns & some missing values, Hazlo was able to successfully impute & take them into consideration. This has saved us tons of work in the data preparation part of the pipeline.

All feature were successfully imputed & identified.

Creating a project

So now we’ve uploaded our dataset, it’s time for the fun part — building our model. We’ll go to the projects page, press “Deploy”, choose “Response” as the target feature & put “ID” down as a de-selected column (though you can choose not to do this).

That’s it, that’s all we have to do.

Our model’s training :)

…and trained in just under 1/2 a minute

Evaluating our model

Our model has an accuracy of slightly over 95% on non-leaky holdout test data —with similar values for other metrics like precision, recall and F-score. The cluster logs show that Hazlo did some over-sampling to balance the dataset & the model was trained comprehensively on both outcomes. Likewise, similar test & train accuracies also indicate that over-fitting was minimized.

The support values were balanced by Hazlo through over-sampling

Furthermore, we can have a look at how the different features are impacting the target. It looks like the feature with the most positive feedback is the year of birth — the younger a user is, the more likely they are respond to the online campaign (which, if you think about it, can be inferred intuitively too). Our prior assessment that number of catalog purchases was positively co-related with the response is also true: this was 6th most impactful positive feature.

Conversely, “Recency” had the greatest negative feedback — the longer ago a user visited the site, the more unlikely they were to purchase. This, again, holds true intuitively.

Here’s how the features are impacting the final outcome.

The feature impact indexes help us better comprehend the model, and along with the cluster network, it provides a clear picture of how the model will be processing inputs.

Our cluster has got 4 algorithms working together to bring us the final output.

After this analysis, its seems that model is viable for production but can always be improved upon through adding more data, and the occasional retrain.

Forecasting

Hazlo has already created an interactive form to fill out the data in case we want to do some manual predictions first; we can always connect the API for industrial use later. We can toggle the different inputs & see how the forecasts look for all of them.

The prediction form created by Hazlo. If you want to switch to manual data entry instead of toggling, you can press the “Switch to Raw Form” button in the bottom right corner of the card.

While we’re forecasting, we can also see the normalized feature importance during prediction, this helps us get a high level understanding of how the model is weighing each input in relation to others.

Here’s how a sample forecast looked like for me — a 96.64% chance of not responding, looks like we’ll have to up our marketing game for this user.

ML Ops

Hazlo also enables users to manage their model’s deployment. From recording forecasts to keeping tabs on inference times, you can do all this on your ML Ops page.

You can view production performance in the Inference Analysis card

and view recorded forecasts on the Live Inferences tab.

That’s about it. You’ve now prepped data, optimized your model & hosted it on the cloud for real-time use. The project performs in the top-percentile of Kaggle codes associated with the dataset — so you’re in good company.

You can connect APIs, do bulk forecasts with file uploads and a lot more. You can also share projects with a link: here’s the one we just built!

Forecast the Marketing Response | Hazlo.ai