What is Relevance Score?
A relevance score is a numerical measure that quantifies how well a retrieved result matches a query or addresses an information need. These scores are used to rank search results, with higher scores indicating greater relevance, and to filter results by setting minimum thresholds. The scoring method depends on the retrieval approach: cosine similarity for vector search, BM25 scores for keyword search, neural network outputs for learned ranking models, or composite scores from hybrid methods.
Relevance scores serve multiple purposes in retrieval systems. They determine result ordering, with the most relevant items presented first. They enable filtering to remove low-quality matches that fall below acceptable thresholds. They provide interpretability by quantifying match quality. Some systems also use scores for weighted combination when merging results from multiple retrievers or when balancing retrieval results with other information sources.
Understanding relevance scores is important for building effective RAG systems and agent memory. Different scoring methods have different scales and interpretations: cosine similarity ranges from -1 to 1 (or 0 to 1 for non-negative vectors), Euclidean distance is unbounded and inverse (smaller is better), while neural models might output probabilities. When combining scores from different methods or setting thresholds, normalization and calibration become important. Monitoring score distributions helps identify retrieval quality issues and optimize system parameters.