Improving Real-Time Bidding in Online Advertising using Markov Decision Processes and Machine Learning Techniques
Article Sidebar
Main Article Content
Abstract
Real-time bidding has emerged as an effective online advertising technique. With real-time bidding, advertisers can position ads per impression, enabling them to optimise ad campaigns by targeting specific audiences in real-time. This paper proposes a novel method for real-time bidding that combines deep learning and reinforcement learning techniques to enhance the efficiency and precision of the bidding process. In particular, the proposed method employs a deep neural network to predict auction details and market prices and a reinforcement learning algorithm to determine the optimal bid price. The model is trained using historical data from the iPin You dataset and compared to cutting-edge real-time bidding algorithms. The outcomes demonstrate that the proposed method is preferable regarding cost-effectiveness and precision. In addition, the study investigates the influence of various model parameters on the performance of the proposed algorithm. It offers insights into the efficacy of the combined deep learning and reinforcement learning approach for real-time bidding. This study contributes to advancing techniques and offers a promising direction for future research.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
References
Cai, H., Ren, K., Zhang, W., Malialis, K., Wang, J., Yu, Y., & Guo, D. (2017, February). Real-time bidding by reinforcement learning in display advertising. In Proceedings of the tenth ACM international conference on web search and data mining (pp. 661-670).
Karlsson, N. (2020). Feedback control in programmatic advertising: The frontier of optimization in real-time bidding. IEEE Control Systems Magazine, 40(5), 40-77.
Liu, S., & Yu, Y. (2019). Bid-aware active learning in real-time bidding for display advertising. IEEE Access, 8, 26561-26572.
Liu-Thompkins, Y. (2019). A decade of online advertising research: What we learned and what we need to know. Journal of advertising, 48(1), 1-13.
Choi, H., Mela, C. F., Balseiro, S. R., & Leary, A. (2020). Online display advertising markets: A literature review and future directions. Information Systems Research, 31(2), 556-575.
Nuara, A., Sosio, N., TrovÃ, F., Zaccardi, M. C., Gatti, N., & Restelli, M. (2019, May). Dealing with interdependencies and uncertainty in multi-channel advertising campaigns optimization. In The World Wide Web Conference (pp. 1376-1386).
Geng, T., Sun, F., Wu, D., Zhou, W., Nair, H., & Lin, Z. (2021). Automated Bidding and Budget Optimization for Performance Advertising Campaigns. Available at SSRN 3913039.
Avadhanula, V., Colini Baldeschi, R., Leonardi, S., Sankararaman, K. A., & Schrijvers, O. (2021, April). Stochastic bandits for multi-platform budget optimization in online advertising. In Proceedings of the Web Conference 2021 (pp. 2805-2817).
Luzon, Y., Pinchover, R., & Khmelnitsky, E. (2022). Dynamic budget allocation for social media advertising campaigns: optimization and learning. European Journal of Operational Research, 299(1), 223-234.
Lin, C. C., Chuang, K. T., Wu, W. C. H., & Chen, M. S. (2020). Budget-constrained real-time bidding optimization: Multiple predictors make it better. ACM Transactions on Knowledge Discovery from Data (TKDD), 14(2), 1-27.
Zhang, Weinan, et al. "Real-time bidding benchmarking with ipinyou dataset." arXiv preprint arXiv:1407.7073 (2014).
Liao, H., Peng, L., Liu, Z., & Shen, X. (2014, August). iPinYou global rtb bidding algorithm competition dataset. In Proceedings of the Eighth International Workshop on Data Mining for Online Advertising (pp. 1-6).
Huang, G., Chen, Q., & Deng, C. (2020). A new click-through rates prediction model based on Deep&Cross network. Algorithms, 13(12), 342.
Ren, K., Qin, J., Zheng, L., Yang, Z., Zhang, W., & Yu, Y. (2019, July). Deep landscape forecasting for real-time bidding advertising. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 363-372).
Würfel, M., Han, Q., & Kaiser, M. (2021, December). Online advertising revenue forecasting: An interpretable deep learning approach. In 2021 IEEE International Conference on Big Data (Big Data) (pp. 1980-1989). IEEE.
Ghosh, A., Mitra, S., Sarkhel, S., Xie, J., Wu, G., & Swaminathan, V. (2020). Scalable bid landscape forecasting in real-time bidding. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part III (pp. 451-466). Springer International Publishing.
Du, M., Sassioui, R., Varisteas, G., State, R., Brorsson, M., & Cherkaoui, O. (2017). Improving real-time bidding using a constrained markov decision process. In Advanced Data Mining and Applications: 13th International Conference, ADMA 2017, Singapore, November 5–6, 2017, Proceedings 13 (pp. 711-726). Springer International Publishing.
Agrawal, N., Najafi-Asadolahi, S., & Smith, S. A. (2023). A Markov Decision Model for Managing Display-Advertising Campaigns. Manufacturing & Service Operations Management, 25(2), 489-507.
Shanahan, J., & den Poel, D. (2010). Determining optimal advertisement frequency capping policy via Markov decision processes to maximize click through rates. In Proceedings of NIPS Workshop: Machine Learning in Online Advertising (pp. 39-45).
Boutilier, C., & Lu, T. (2016). Budget allocation using weakly coupled, constrained Markov decision processes.