Finetuning Analytics Information Systems for a Better Understanding of Users : Evidence of Personification Bias on Multiple Digital Channels

vertaisarvioitu
© The Author(s) 2023. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Tiivistelmä

Although the effect of hyperparameters on algorithmic outputs is well known in machine learning, the effects of hyperparameters on information systems that produce user or customer segments are relatively unexplored. This research investigates the effect of varying the number of user segments on the personification of user engagement data in a real analytics information system, employing the concept of persona. We increment the number of personas from 5 to 15 for a total of 330 personas and 33 persona generations. We then examine the effect of changing the hyperparameter on the gender, age, nationality, and combined gender-age-nationality representation of the user population. The results show that despite using the same data and algorithm, varying the number of personas strongly biases the information system’s personification of the user population. The hyperparameter selection for the 990 total personas results in an average deviation of 54.5% for gender, 42.9% for age, 28.9% for nationality, and 40.5% for gender-age-nationality. A repeated analysis of two other organizations shows similar results for all attributes. The deviation occurred for all organizations on all platforms for all attributes, as high as 90.9% in some cases. The results imply that decision makers using analytics information systems should be aware of the effect of hyperparameters on the set of user or customer segments they are exposed to. Organizations looking to effectively use persona analytics systems must be wary that altering the number of personas could substantially change the results, leading to drastically different interpretations about the actual user base.

Kokoelmat

Artikkelit [3019]