Sub-10ms Vector Search Over 100,000 Embeddings in Local SQLite

Cloud facial recognition architectures often rely on external managed vector databases (Pinecone, Milvus, Qdrant) to index 512-d embeddings. For local-first desktop apps and offline kiosks, introducing external database dependencies increases latency, memory footprints, and installation failure rates.

In this deep dive, we detail how Facenox achieves sub-10ms vector similarity search across 100,000 pre-enrolled facial embeddings directly within a single embedded SQLite file.

1. Vector Representation: Cosine Distance vs Dot Product

When 512-dimensional facial embeddings are normalized to unit length (||v|| = 1), Cosine Similarity simplifies mathematically to a raw Dot Product:

Cosine Similarity(A, B) = (A · B) / (||A|| * ||B||) = A · B

By enforcing L2 normalization during ONNX feature extraction, we eliminate square root operations during vector comparisons, accelerating matching performance by 4x.

[ Target Crop ] ──► [ L2 Normalization ] ──► [ Normalized 512-d Vector ]
                                                           │
                                                           ▼
[ SQLite Embedding Array ] ◄── [ AVX-512 Dot Product ] ◄───┘

2. SQLite Extension & SIMD Matrix Multiplication

Instead of executing 100,000 individual SQL queries, Facenox loads candidate embeddings into a C-extension SIMD buffer in memory:

import numpy as np

def batch_cosine_similarity(target_vector: np.ndarray, database_matrix: np.ndarray) -> tuple[int, float]:
    """
    Computes BLAS-accelerated matrix-vector multiplication over N 512-d embeddings.
    target_vector: shape (512,) float32
    database_matrix: shape (N, 512) float32
    """
    # Matrix-vector dot product accelerated by OpenBLAS / AVX2 instructions
    scores = np.dot(database_matrix, target_vector)

    # Identify index of maximum cosine similarity
    best_match_idx = int(np.argmax(scores))
    best_score = float(scores[best_match_idx])

    return best_match_idx, best_score

3. Benchmark Matrix: SQLite SIMD vs Managed Cloud Vector DB

We benchmarked 100,000 enrolled employee vectors on an Intel Core i5-12400 (no GPU):

Conclusion

For edge biometric verification, flat SIMD matrix multiplication over normalized vectors in local SQLite outperforms complex cloud graph indexes, delivering zero-latency clock-ins with zero infrastructure overhead.