Optimizing Personalized Brand Touchpoint Selection: Off-Policy Evaluation, Model Ranking, and Profit-Driven Targeting in Multi-Treatment Marketing Campaigns
Main Article Content
Abstract
Multi-treatment uplift modeling estimates the incremental impact of each brand action relative to a baseline, enabling scalable and data-driven personalized brand communication. This paper presents a reproducible benchmarking study of lightweight multi-treatment uplift learners across three marketing datasets covering different brand communication contexts: Hillstrom email creative selection, Bank contact channel allocation, and Starbucks promotional offer personalization. Semi-synthetic scenarios with known oracle policy values are also constructed. Learned policies are evaluated using matched-only estimation and three off-policy estimators, namely IPS, SNIPS, and doubly robust estimation. Evaluation is combined with overlap diagnostics, including clipping rate, effective sample size, and maximum importance weight, as well as sensitivity analysis over clipping and trimming choices. Across real datasets, the doubly robust estimator changes model selection in a non-trivial way. On Hillstrom, matched-only evaluation selects MMOALR, while doubly robust evaluation selects XLearner-LR, showing rank reversal. On Bank and Starbucks, doubly robust and matched-only evaluation agree on the winning model but differ materially in estimated profit levels. The study provides a practical evaluation framework for credible, cost-effective personalized touchpoint allocation and campaign budget optimization.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
S.A Neslin, D. Grewal, R. Leghorn,V. Shankar, M.L Teerling,S. Thomas, et al., “Challenges and Opportunities in Multichannel Customer Management,” Journal of Service Research, vol. 9, pp. 95-112, 2006, doi: 10.1177/1094670506293559.
K. N. Lemon and P. C. Verhoef, “Understanding Customer Experience Throughout the Customer Journey,” Journal of Marketing, vol. 80, pp. 69-96, 2016, doi: 10.1509/jm.15.0420.
M. Wedel and P. K. Kannan, “Marketing Analytics for Data-Rich Environments,” Journal of Marketing, vol. 80, pp. 97-121, 2016, doi: 10.1509/jm.15.0413.
E. Ascarza, “Retention Futility: Targeting High-Risk Customers Might Be Ineffective,” Journal of Marketing Research, vol. 55, pp. 80-98, 2018, doi: 10.1509/jmr.16.0163.
V. S. Y. Lo, “The True Lift Model: A Novel Data Mining Approach to Response Modeling in Database Marketing,” SIGKDD Explorations, vol. 4, pp. 78-86, 2002.
P. Rzepakowski and S. Jaroszewicz, “Decision Trees for Uplift Modeling with Single and Multiple Treatments,” Knowledge and Information Systems, vol. 32, no. 2, pp. 303-327, 2012, doi: 10.1007/s10115-011-0434-0.
S. Athey and W. I. Guido, “Recursive Partitioning for Heterogeneous Causal Effects,” Proceedings of the National Academy of Sciences, vol. 113, no. 27, pp. 7353-7360, 2016, doi: 10.1073/pnas.1510489113.
S. Wager and S. Athey, “Estimation and Inference of Heterogeneous Treatment Effects Using Random Forests,” Journal of the American Statistical Association, vol. 113, pp. 1228-1242, 2018, doi: 10.1080/01621459.2017.1319839.
S. R. Kunzel, J. S. Sekhon, P. J. Bickel, and B. Yu, “Metalearners for Estimating Heterogeneous Treatment Effects Using Machine Learning,” Proceedings of the National Academy of Sciences, vol. 116, no. 10, pp. 4156-4165, 2019, doi: 10.1073/pnas.1804597116.
Y. Zhao, X. Fang, and D. Simchi-Levi, “Uplift Modeling with Multiple Treatments and General Response Types,” In Proceedings of the 2017 SIAM International Conference on Data Mining (SDM). SIAM, 2017, pp. 588-596.
D. Olya, K. Coussement, and W. Verbeke, “A Survey and Benchmarking Study of Multi-Treatment Uplift Modeling,” Data Mining and Knowledge Discovery, vol. 34, pp. 273-308, 2020, doi: 10.1007/s10618-019-00670-y.
A. Swaminathan and T. Joachims, “The Self-Normalized Estimator for Counterfactual Learning,” In Proceedings of the Advances in Neural Information Processing Systems 28 (NeuriPS 2015), 2015.
M. Dudik, D. Erhan, J. Langford, and L. Li, “Doubly Robust Policy Evaluation and Optimization,” Statistical Science, vol. 29, pp. 485-511, 2014, doi: 10.1214/14-STS500.
P. R. Rosenbaum and D. B. Rubin, “The Central Role of the Propensity Score in Observational Studies for Causal Effects,” Biometrika, vol. 70, pp. 41-55, 1983, doi: 10.1093/biomet/70.1.41.
K. Hillstrom, “MineThatData E-Mail Analytics and Data Mining Challenge Dataset,” 2008. Dataset (CSV download).
S. Moro, P. Cortez, and P. Rita, “A Data-Driven Approach to Predict the Success of Bank Telemarketing,” Decision Support Systems, vol. 62, pp. 22-31, 2014, doi: 10.1016/j.dss.2014.03.001.
Udacity, “Starbucks Capstone Challenge Dataset,” 2018. Dataset (download/view via Kaggle).
K. L. Keller, “Conceptualizing, Measuring, and Managing Customer-Based Brand Equity,” Journal of Marketing, vol. 57, pp. 1-22, 1993, doi: 10.1177/002224299305700101.