SPELUNKER: Item Similarity Search Using Large Language Models and Custom K-Nearest Neighbors

Abstract

This paper presents a hybrid system for intuitive item similarity search that combines a Large Language Model (LLM) with a custom K-Nearest Neighbors (KNN) algorithm. Unlike black-box dense vector systems, this architecture provides superior interpretability by first using an LLM to convert natural language queries into structured, attribute-based searches. This structured query then serves as input to a custom KNN algorithm with a BallTree search strategy, which uses a heterogeneous distance metric to preserve distinct data types. Our evaluation, conducted on a dataset of 500 wine reviews, demonstrates the system's effectiveness. The LLM achieved an F1-score of 0.9779 in information extraction, while also demonstrating high fidelity with a Jaro string similarity of 0.9321. When we augmented the KNN algorithm with LLM-based re-ranking, we observed a statistically significant improvement in recall (p=0.013), indicating the LLM's ability to identify and promote relevant items that align with nuanced user intent. This approach effectively bridges the gap between human language and machine-understandable item representations, offering a transparent and nuanced search capability.