Skip to content
OpenAgentSkill
Browsing is open · Catalog captured 2026-11-21 · Automatic updates and account services are paused.
Data & analytics

vector-database-configuration-validation

Validates a vector index's schema/dimension/distance-metric config and its actual recall/query-performance behavior before a production cutover — catching a mismatch that returns wrong-but-plausible results or degraded recall instead of an error. Use when a user asks to "validate my vector index config before we go live," "will this embedding dimension/metric mismatch break retrieval," "test recall before cutting over to the new index," "check this Pinecone/Weaviate/ Milvus schema for mistakes," or reports retrieval quality that silently dropped after a re-index or embedding-model change.

selvarajmurugesan90 · Static checks on file

Installation

Read the source and dependencies before running a command. Copying does not install or execute anything.

npx skills add selvarajmurugesan90/ops-engineering-skills --skill vector-database-configuration-validation

About this skill


name: vector-database-configuration-validation description: > Validates a vector index's schema/dimension/distance-metric config and its actual recall/query-performance behavior before a production cutover — catching a mismatch that returns wrong-but-plausible results or degraded recall instead of an error. Use when a user asks to "validate my vector index config before we go live," "will this embedding dimension/metric mismatch break retrieval," "test recall before cutting over to the new index," "check this Pinecone/Weaviate/ Milvus schema for mistakes," or reports retrieval quality that silently dropped after a re-index or embedding-model change. license: Apache-2.0 compatibility: "Claude Code, GitHub Copilot, OpenAI Codex, Cursor, Gemini CLI" metadata: domain: ai-agent maturity: stable

Vector Database Configuration Validation

Purpose

A vector index with the wrong dimension, distance metric, or HNSW parameter set for its embedding model doesn't fail loudly — it either rejects vectors with a dimension-mismatch error (the easy case) or, worse, silently accepts vectors and returns technically valid but degraded or nonsensical similarity results (the hard case, because nothing in the API response signals anything is wrong). This skill covers validating a vector index's configuration and actual query-performance behavior before it takes production traffic: confirming dimension and distance-metric agreement with the embedding model, running a labeled recall evaluation against real queries, and gating a cutover behind that evaluation rather than a visual "looks right" review. It assumes the index's operational tuning (sharding, replication, HNSW parameter selection) is handled in vector-database-operations-pinecone-weaviate-milvus and that vectors are arriving via vector-database-ingestion-pipeline-for-rag — this skill is specifically the pre-cutover validation gate sitting between those two.

When to use

  • Before cutting a RAG system's retrieval traffic over to a new or reconfigured vector index (new embedding model, new HNSW parameters, new sharding scheme, vendor migration).
  • Standing up a new index/collection and confirming its schema (dimension, distance metric, indexed metadata fields) actually matches what the embedding model and query patterns require.
  • Retrieval quality (recall, relevance) degraded after a re-index or an embedding-model change, and you need to confirm whether the index configuration itself is the cause.
  • Reviewing a Pinecone/Weaviate/Milvus index/collection definition in a PR before it's applied, to catch a dimension or metric mistake before it reaches production.
  • Migrating a corpus to a new vendor or a new index within the same vendor, and needing a go/no-go gate before the alias/pointer swap.

Prerequisites & environment

  • The embedding model's actual output dimension and the distance metric it was trained/tuned for (cosine vs. dot product vs. Euclidean) — get this from the model's own documentation, not assumed from a different model's defaults; a mismatch here is the single most common root cause this skill catches.
  • A labeled evaluation set of (query, expected relevant document ids) pairs representative of real production queries — at minimum 30-50 pairs covering the corpus's main topics; without this, "does the new index work" can only be answered by vibes, not a number.
  • Access to both the candidate (new/changed) index and, when validating a re-index or migration, the existing production index, so recall can be compared side by side rather than evaluated in isolation.
  • The vendor SDK/CLI for the index in question (Pinecone, Weaviate, or Milvus client) to run schema inspection and query calls directly, rather than only reading the config file that was intended to produce it.
  • Familiarity with your vendor's current alias/pointer-swap or blue-green index mechanism, since the recommended cutover pattern below depends on it (Pinecone: create-new-then-repoint; Weaviate/ Milvus: alias support varies by version — confirm current documentation).

Step-by-step guidance

  1. Confirm the index's configured dimension matches the embedding model's actual output dimension — check this against a real embedding call, not the model's marketing/spec-sheet number alone, since some models expose a configurable output dimension (e.g. Matryoshka-style truncation) that can silently differ from the nominal default:

    sample_vector = embedding_model.embed("sanity check")
    assert len(sample_vector) == index_config["dimension"], (
        f"embedding model outputs {len(sample_vector)} dims, "
        f"index configured for {index_config['dimension']}"
    )
    

    A dimension mismatch is usually caught immediately as a hard error on upsert — treat that as the good outcome; the dangerous case is a vendor/index configuration that silently pads or truncates instead of rejecting (verify current vendor behavior rather than assuming it errors).

  2. Confirm the distance metric configured on the index matches what the embedding model expects. This is the silent-failure case: a model tuned for cosine similarity queried with a dot-product or Euclidean index still returns a similarity-ranked result set — it's just the wrong ranking, with no error anywhere:

    # Weaviate collection (illustrative — verify current field names)
    vectorIndexConfig:
      distance: cosine     # must match the embedding model's trained metric
    
    # Milvus
    index_params = {"index_type": "HNSW", "metric_type": "COSINE", "params": {...}}
    

    If you cannot find explicit confirmation of which metric a given embedding model was tuned for, treat cosine as the safer default assumption bu

Read the complete source ↗