Your LLM Has a Passport: Investigating Brand Origin Geographic Bias in E-Commerce Rankings

Abstract

Large Language Models (LLMs) have recently demonstrated strong zero-shot ranking capabilities in Information Retrieval (IR). However, their deployment in e-commerce raises concerns about fairness, particularly when geographic biases influence product visibility. This paper examines how brand origin and user location affect LLM-based rankings across three models developed by companies headquartered in different regions: Llama in the US, Mistral in Europe, and Qwen in China. Using the Amazon Shopping Queries Dataset, we evaluate the impact of brand information on rankings under three conditions: (i) no brand location context, (ii) explicit brand origin, and (iii) explicit user location. Our evaluation combines standard IR metrics (NDCG) with list similarity measures (Jaccard, FRBO) and geographic coverage of top-K results. Our analysis reveals systematic differences: Mistral favors European brands, Qwen exhibits strong North American bias, and Llama remains comparatively balanced. Providing brand origin information alters rankings, with sensitivity varying across models, while user location exerts the strongest effect, shifting continental representation in Qwen by up to 59%. Despite these biases, all models maintain competitive ranking performance. These findings highlight the risks of geographic skew in LLM-driven IR systems and underscore the need for fairness-aware ranking pipelines in e-commerce. By uncovering how LLMs encode and amplify geographic preferences, this study provides actionable insights for both researchers and practitioners working toward more transparent and equitable retrieval systems.

Tamara Czinczoll

Doctoral Researcher in NLP

I work on human-cognition-inspired pretraining strategies.