Alexander Bondarenko
ABSTRACT
Quality of search engine results returned to health-related questions is very critical, since a searcher may directly trust any suggestion in the top results. We analyze search questions that mention diseases / symptoms and remedies that are potential health-related misbeliefs. Using lists of medical and alternative medicine terms, we extract health-related search questions from 1.5~billion questions submitted to Yandex. As an initial study, we sample 30 frequent questions that contain a disease--remedy pair like "Can hepatitis be cured with milk thistle?". For each question, we carefully identify a ground truth answer in the medical literature and annotate the top-10 Yandex search result snippets as confirming the belief, rejecting it, or giving no answer. Our analysis shows that about 44%~of the snippets (that users may simply interpret as definitive answers!) confirm some untrue beliefs and are wrong, and only few include health risk warnings about using toxic plants.
- A. Armentia, S. Martín-Armentia, F. Pineda, B. Martín-Armentia, M. Castro, S. Fernández, A. Moro, and M. Castillo. 2020. Allergic Hypersensitivity to Garlic and Onion in Children and Adults. Allergologia et immunopathologia, Vol. 48, 3 (2020), 232--236.Google Scholar
- John W Ayers, Benjamin M Althouse, Jon-Patrick Allem, J Niels Rosenquist, and Daniel E Ford. 2013. Seasonality in Seeking Mental Health Information on Google. American Journal of Preventive Medicine, Vol. 44, 5 (2013), 520--525.Google ScholarCross Ref
- Leif Azzopardi. 2021. Cognitive Biases in Search: A Review and Reflection of Cognitive Biases in Information Retrieval. In Proceedings of the Conference on Human Information Interaction and Retrieval, CHIIR 2021. ACM, 27--37. Google ScholarDigital Library
- Rakesh Bal, Sayan Sinha, Swastika Dutta, Risabh Joshi, Sayan Ghosh, and Ritam Dutt. 2020. Analysing the Extent of Misinformation in Cancer Related Tweets. In Proceedings of the International AAAI Conference on Web and Social Media, ICWSM 2020, Vol. 14. 924--928.Google Scholar
- B. Barrett. 2003. Medicinal Properties of Echinacea: A Critical Review. Phytomedicine, Vol. 10, 1 (2003), 66--86.Google ScholarCross Ref
- Gaber El-Saber Batiha, Amany Magdy Beshbishy, Muhammad Ikram, Zohair S Mulla, Mohamed E Abd El-Hack, Ayman E Taha, Abdelazeem M Algammal, and Yaser Hosny Ali Elewa. 2020. The Pharmacological Activity, Biochemical Properties, and Pharmacokinetics of the Major Natural Polyphenolic Flavonoid: Quercetin. Foods (Basel, Switzerland), Vol. 9, 3 (2020), 347.Google Scholar
- Alexander Beloborodov, Pavel Braslavski, and Marina Driker. 2014. Towards Automatic Evaluation of Health-Related CQA Data. In Proceedings of the 5th International Conference of the CLEF Initiative, CLEF 2014 (Lecture Notes in Computer Science, Vol. 8685). Springer, 7--18.Google ScholarCross Ref
- Alexander Beloborodov, Artem Kuznetsov, and Pavel Braslavski. 2013. Characterizing Health-Related Community Question Answering. In Proceedings of the 35th European Conference on IR Research, ECIR 2013 (Lecture Notes in Computer Science, Vol. 7814). Springer, 680--683. Google ScholarDigital Library
- Sanmitra Bhattacharya, Hung Tran, and Padmini Srinivasan. 2012. Discovering Health Beliefs in Twitter. In Proceedings of the Information Retrieval and Knowledge Discovery in Biomedical Text, Papers from the 2012 AAAI Fall Symposium (AAAI Technical Report, Vol. FS-12-05 ). AAAI.Google Scholar
- Francesca Borrelli, Raffaele Capasso, and Angelo A. Izzo. 2007. Garlic (Allium sativum L.): Adverse Effects and Drug Interactions in Humans. Molecular Nutrition & Food Research, Vol. 51, 11 (2007), 1386--1397.Google ScholarCross Ref
- Sisson D. and Balmer C. 2014. A Chemical Burn from a Garlic Poultice Applied to the Face to Treat Toothache: A Case Report. Primary Dental Journal, Vol. 3, 1 (2014), 28--29.Google ScholarCross Ref
- Enyan Dai, Yiwei Sun, and Suhang Wang. 2020. Ginger Cannot Cure Cancer: Battling Fake Health News with a Comprehensive Data Repository. In Proceedings of the 14th International AAAI Conference on Web and Social Media, ICWSM 2020. AAAI Press, 853--862.Google Scholar
- Lila J. Finney Rutten, Kelly D. Blake, Alexandra J. Greenberg-Worisek, Summer V. Allen, Richard P. Moser, and Bradford W. Hesse. 2019. Online Health Information Seeking Among US Adults: Measuring Progress Toward a Healthy People 2020 Objective. Public Health Reports, Vol. 134, 6 (2019), 617--625.Google ScholarCross Ref
- Susannah Fox and Maeve Duggan. 2013. Health Online 2013. Health, Vol. 2013 (2013), 1--55.Google Scholar
- Pascal Geldsetzer. 2020. Knowledge and Perceptions of COVID-19 Among the General Public in the United States and the United Kingdom: A Cross-Sectional Online Survey. Annals of Internal Medicine, Vol. 173, 2 (2020), 157--160.Google ScholarCross Ref
- Jeremy Ginsberg, Matthew H. Mohebbi, Rajan S. Patel, Lynnette Brammer, Mark S. Smolinski, and Larry Brilliant. 2009. Detecting Influenza Epidemics Using Search Engine Query Data. Nature, Vol. 457, 7232 (2009), 1012--1014.Google Scholar
- Xiaoqing Guo and Nan Mei. 2016. Aloe Vera: A Review of Toxicity and Adverse Clinical Effects. Journal of Environmental Science and Health, Part C, Vol. 34, 2 (2016), 77--96.Google ScholarCross Ref
- Anat Hashavit, Hongning Wang, Raz Lin, Tamar Stern, and Sarit Kraus. 2021. Understanding and Mitigating Bias in Online Health Search. In Proceedings of the 44th International Conference on Research and Development in Information Retrieval, SIGIR 2021. ACM, 265--274. Google ScholarDigital Library
- Jiang Hu, Donna Webster, Joyce Cao, and Andrew Shao. 2018. The Safety of Green Tea and Green Tea Extract Consumption in Adults--Results of a Systematic Review. Regulatory Toxicology and Pharmacology, Vol. 95 (2018), 412--433.Google ScholarCross Ref
- Jimmy, Guido Zuccon, Bevan Koopman, and Gianluca Demartini. 2019. Health Cards for Consumer Health Search. In Proceedings of the 42nd International Conference on Research and Development in Information Retrieval, SIGIR 2019. ACM, 35--44. Google ScholarDigital Library
- Skyler B. Johnson, Henry S. Park, Cary P. Gross, and James B. Yu. 2018. Use of Alternative Medicine for Cancer and its Impact on Survival. JNCI: Journal of the National Cancer Institute, Vol. 110, 1 (2018), 121--124.Google ScholarCross Ref
- Bera K., Nosalova G., Sivova V., and Ray B. 2016. Structural Elements and Cough Suppressing Activity of Polysaccharides from Zingiber Officinale Rhizome. Phytotherapy Research, Vol. 30, 1 (2016), 105--111.Google ScholarCross Ref
- Mark A. Klebanoff and Abigail N. Turner. 2014. Bacterial Vaginosis and Season, a Proxy for Vitamin D Status. Sexually Transmitted Diseases, Vol. 41, 5 (2014), 295.Google ScholarCross Ref
- Syal Kumar, Gustav J. Dobos, and Thomas Rampp. 2013. Clinical Significance of Leech Therapy in Indian Medicine. Journal of Evidence-Based Complementary & Alternative Medicine, Vol. 18, 2 (2013), 152--158.Google ScholarCross Ref
- Annie YS Lau and Enrico W. Coiera. 2007. Research Paper: Do People Experience Cognitive Biases While Searching for Information? J. Am. Medical Informatics Assoc., Vol. 14, 5 (2007), 599--608.Google ScholarCross Ref
- Ji Li, Hong Zhao, Ju-Min Song, Jing Zhang, Yin-Lan Tang, and Chang-Mao Xin. 2015. A Meta-Analysis of Risk of Pregnancy Loss and Caffeine and Coffee Consumption During Pregnancy. International Journal of Gynecology & Obstetrics, Vol. 130, 2 (2015), 116--122.Google ScholarCross Ref
- Juliana Ester Martin Lopez. 2015. Candidiasis (Vulvovaginal). BMJ Clinical Evidence, Vol. 2015 (2015).Google Scholar
- Ossama Maadarani, Zouhair Bitar, and Mohammad Mohsen. 2019. Adding Herbal Products to Direct-Acting Oral Anticoagulants Can Be Fatal. European Journal of Case Reports in Internal Medicine, Vol. 6, 8 (2019).Google ScholarCross Ref
- B. Bülent Mentecs, Sezai Leventoug lu, .Igbal Osmanov, Dilek Kösehan, and Timucc in Erol. 2019. Anal Abscess due to Leech Therapy of Hemorrhoids: Mumbo Jumbo is Still in Vogue. Journal of Surgical Case Reports, Vol. 7 (2019).Google Scholar
- Peter Molan and Tanya Rhodes. 2015. Honey: A Biologic Wound Dressing. Wounds: A Compendium of Clinical Research and Practice, Vol. 27, 6 (2015), 141--151.Google Scholar
- C.A. Morris. 1971. Seasonal Variation of Streptococcal Vulvo-Vaginitis in an Urban Community. Journal of Clinical Pathology, Vol. 24, 9 (1971), 805--807.Google ScholarCross Ref
- C. Mulrow, V. Lawrence, B. Jacobs, C. Dennehy, J. Sapp, G. Ramirez, C. Aguilar, K. Montgomery, L Morbidoni, JM Arterburn, et al. 2000. Milk Thistle: Effects on Liver Disease and Cirrhosis and Clinical Adverse Effects: Summary. In AHRQ Evidence Report Summaries. Agency for Healthcare Research and Quality (US).Google Scholar
- Somayeh Nazari, Maryam Rameshrad, and Hossein Hosseinzadeh. 2017. Toxicological Effects of Glycyrrhiza glabra (Licorice): A Review. Phytotherapy Research, Vol. 31, 11 (2017), 1635--1650.Google ScholarCross Ref
- George Nitzburg, Ingmar Weber, and Elad Yom-Tov. 2019. Internet Searches for Medical Symptoms Before Seeking Information on 12-step Addiction Treatment Programs: A Web-Search Log Analysis. Journal of Medical Internet Research, Vol. 21, 5 (2019), e10946.Google ScholarCross Ref
- Olabisi Oduwole, Ekong E Udoh, Angela Oyo-Ita, and Martin M. Meremikwu. 2018. Honey for Acute Cough in Children. Cochrane Database of Systematic Reviews, Vol. 4 (2018).Google Scholar
- Shaun T O'Keeffe. 2017. Summertime Blues? A Re-Examination of the Seasonality of Web Searches for Restless Legs and Leg Cramps. Sleep Medicine, Vol. 37 (2017), 119--123.Google ScholarCross Ref
- F. Pantano, G. Mannocchi, E. Marinelli, S. Gentili, S. Graziano, FP Busardò, and NM Di Luca. 2017. Hepatotoxicity Induced by Greater Celandine (Chelidonium Majus L.): A Review of the Literature. Eur Rev Med Pharmacol Sci, Vol. 21, 1 Suppl. (2017), 46--52.Google Scholar
- Frances A. Pogacar, Amira Ghenai, Mark D. Smucker, and Charles L. A. Clarke. 2017. The Positive and Negative Influence of Search Results on People's Decisions about the Efficacy of Medical Treatments. In Proceedings of the International Conference on Theory of Information Retrieval, ICTIR 2017. ACM, 209--216. Google ScholarDigital Library
- Andrea Rambaldi, Bradly P. Jacobs, Gaetano Iaquinto, and Christian Gluud. 2005. Milk Thistle for Alcoholic and/or Hepatitis B or C Virus Liver Diseases. Cochrane Database of Systematic Reviews 2 (2005).Google Scholar
- N. Reider, N. Sepp, P. Fritsch, G. Weinlich, and E. Jensen-Jarolim. 2002. Anaphylaxis to Camomile: Clinical Features and Allergen Cross-Reactivity. Clinical and Experimental Allergy : Journal of the British Society for Allergy and Clinical Immunology, Vol. 30, 10 (2002), 1436--1443.Google ScholarCross Ref
- Navid Rekabsaz, Oleg Lesota, Markus Schedl, Jon Brassey, and Carsten Eickhoff. 2021. TripClick: The Log Files of a Large Health Web Search Engine .In Proceedings of the The 44th International Conference on Research and Development in Information Retrieval, SIGIR 2021. ACM, 2507--2513. Google ScholarDigital Library
- Razina Rouf, Shaikh Jamal Uddin, Dipto Kumer Sarker, Muhammad Torequl Islam, Eunus S Ali, Jamil A. Shilpi, Lutfun Nahar, Evelin Tiralongo, and Satyajit D. Sarker. 2020. Antiviral Potential of Garlic (Allium Sativum) and its Organosulfur Compounds: A Systematic Update of Pre-clinical and Clinical Data. Trends Food Sci Technol., Vol. 104 (2020), 219--234.Google ScholarCross Ref
- Saranrat Sadoyu, Chidchanok Rungruang, Thitima Wattanavijitkul, Ratree Sawangjit, Ammarin Thakkinstian, and Nathorn Chaiyakunapruk. 2021. Aloe Vera and Health Outcomes: An Umbrella Review of Systematic Reviews and Meta-Analyses. Phytotherapy Research, Vol. 35, 2 (2021), 555--576.Google ScholarCross Ref
- Carolien J.P. van den Bout-van den Beukel, Peter P. Koopmans, Andre J.A.M. van der Ven, Peter A.G.M. De Smet, and David M. Burger. 2006. Possible Drug-Metabolism Interactions of Medicinal Herbs with Antiretroviral Agents. Drug Metabolism Reviews, Vol. 38, 3 (2006), 477--514.Google ScholarCross Ref
- Michael Vö lske, Pavel Braslavski, Matthias Hagen, Galina Lezina, and Benno Stein. 2015. What Users Ask a Search Engine: Analyzing One Billion Russian Question Queries. In Proceedings of the 24th International Conference on Information and Knowledge Management, CIKM 2015. ACM, 1571--1580. Google ScholarDigital Library
- Alicja Warowicka, Łukasz Popenda, Gra.zyna Bartkowiak, Oskar Musidlak, Jagoda Litowczenko-Cybulska, Dorota Ku'zma, Robert Nawrot, Stefan Jurga, and Anna Go'zdzicka-Józefiak. 2019. Protoberberine Compounds Extracted from Chelidonium Majus L. as Novel Natural Photosensitizers for Cancer Therapy. Phytomedicine, Vol. 64 (2019), 152919.Google ScholarCross Ref
- Ryen White. 2013. Beliefs and Biases in Web Search. In Proceedings of the 36th International Conference on Research and Development in Information Retrieval, SIGIR 2013. ACM, 3--12. Google ScholarDigital Library
- Ryen W. White and Ahmed Hassan Awadallah. 2014. Content Bias in Online Health Search. ACM Trans. Web, Vol. 8, 4 (2014), 25:1--25:33. Google ScholarDigital Library
- Andreas Christian Windfeld and Florian Maximilian Meier. 2021. “Does Vinegar Kill Coronavirus?”--Using Search Log Analysis to Estimate the Extent of COVID-19-Related Misinformation Searching Behaviour in the United States. In Proceedings of iConference 2021.Google Scholar
- Richard A. Wright and Franklyn N. Judson. 1978. Relative and Seasonal Incidences of the Sexually Transmitted Diseases. A Two-Year Statistical Review. Sexually Transmitted Infections, Vol. 54, 6 (1978), 433--440.Google ScholarCross Ref
- Renfan Xu, Ke Yang, Jie Ding, and Guangzhi Chen. 2020. Effect of Green Tea Supplementation on Blood Pressure: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Medicine, Vol. 99, 6 (2020).Google Scholar
- Rui Yang, Bo-Chuan Yuan, Yong-Sheng Ma, Shan Zhou, and Ying Liu. 2017. The Anti-Inflammatory Activity of Licorice, a Widely used Chinese Herb. Pharmaceutical Biology, Vol. 55, 1 (2017), 5--18.Google ScholarCross Ref
- Sibel Yenidunya, Hacer Hatlas, and Reyhan Bayrak. 2012. To Determine of the Prevalence of Bacterial Vaginosis, Candida sp, Mixed Infections (Bacterial Vaginosis+Candida sp), Trichomonas Vaginalis, Actinomyces sp in Turkish Women from Ankara, Turkey. Ginekologia Polska, Vol. 83, 10 (2012).Google Scholar
Index Terms
- Misbeliefs and Biases in Health-Related Searches
Recommendations
Evaluating the performance and neutrality/bias of search engines
VALUETOOLS 2019: Proceedings of the 12th EAI International Conference on Performance Evaluation Methodologies and ToolsDifferent search engines provide different outputs for the same keyword. This may be due to different definitions of relevance, to different ranking aggregation methods, and/or to different knowledge/anticipation of users' preferences, but rankings are ...
Evaluation metrics for measuring bias in search engine results
AbstractSearch engines decide what we see for a given search query. Since many people are exposed to information through search engines, it is fair to expect that search engines are neutral. However, search engine results do not necessarily cover all the ...
Search bias quantification: investigating political bias in social media and web search
Users frequently use search systems on the Web as well as online social media to learn about ongoing events and public opinion on personalities. Prior studies have shown that the top-ranked results returned by these search engines can shape user opinion ...
Comments