Detecting AI-Generated Reviews for Corporate Reputation Management

In this paper, we provide a concrete case study to better steer corporate reputation management in the light of recent proliferation of AI-generated product reviews. Firstly, we provide a systematic methodology for generating high quality AI-generated text reviews using pre-existing human written reviews and present the GPTARD dataset for training AI-generated review detection systems. We also present a separate evaluation dataset called ARED that contains a sample of product reviews from Amazon along with their predicted authenticity from incumbent tools in the industry that enables comparative benchmarking of AI-generated review detection systems. Secondly, we provide a concise overview of current approaches in fake review detection and propose to apply an overall, integrated group of four predictive features in our machine learning systems. We demonstrate the efficacy of these features by providing a comparative study among four different types of classifiers in which our specific machine learning based AI-generated review detection system in the form of random forest prevails with an accuracy of 98.50% and a precision of 99.34% on ChatGPT generated reviews. Our highly performant system can in practice be used as a reliable tool in managing corporate reputation against AI-generated fake reviews. Finally, we provide an estimation of AI-generated reviews in a sample of products on Amazon.com that turns out to be almost 10%. To validate this estimation, we also provide a comparison with existing tools Fakespot and ReviewMeta. Such high prevalence of AI-generated reviews motivates future work in helping corporate reputation management by effectively fighting spam product reviews.