RAG vs Fine-Tuning: How Should AI Use Your Business Knowledge?

Executive summary

AWS announced general availability of native vector search in Amazon DynamoDB on August 5, 2026. DynamoDB now includes a vector index type and a SearchVectors API that stores embeddings as List(Number), supports up to 4096 dimensions, and returns up to Top-K=100 results.

One-sentence guidance: If your operational data already lives in DynamoDB, your queries are naturally scoped to a partition key (for example, per-tenant), and you prioritise low operational overhead, run a pilot. If you need global Top-K retrieval, complex range filters, or fine-grained index tuning, prefer a specialized vector DB or a hybrid design.

• Run a 2-week pilot measuring recall@K, p95/p99 latency, update freshness, and cost per million queries.
• Revisit tenant partitioning: DynamoDB vector search is partition-key scoped and alters multi-tenant designs.
• Treat AWS performance and recall statements as vendor claims that require independent benchmarking.

What AWS actually delivered (feature summary & limits)

According to the AWS announcement, DynamoDB adds a vector index type and SearchVectors API. Vectors are stored as the existing List(Number) attribute; indexes accept a dimension parameter (up to 4096) and distance functions Cosine, Euclidean, and Dot. SearchVectors returns up to 100 nearest results and supports inline exact-match filters.

AWS positions the feature as a serverless, horizontally scaling vector store co-located with operational data. The announcement does not publish the underlying index algorithm, detailed pricing per-query or per-GB for vectors, or the exact index tuning knobs available to customers.

• Supported dimensions: up to 4096.
• Distance functions: Cosine, Euclidean, Dot.
• Top-K limit: up to 100 results per SearchVectors call.
• Search scope: constrained to a single partition-key value per call.
• Filter semantics: inline filters support exact-match only, not range or prefix predicates.

Why this matters for RAG architectures

Native vector search in DynamoDB reduces the need to copy rows into a separate vector store, simplifying ingestion and lowering operational surface area. Teams can co-locate metadata and embeddings and perform similarity retrieval without an external sync pipeline.

However, architectural trade-offs—partition-scoped searches, limited inline filtering, and limited public tuning controls—change the calculus for multi-tenant apps and workloads that require global recall or advanced hybrid scoring.

• Operational benefit: simpler write path and unified access, billing, and governance inside a single AWS account.
• Architectural constraint: partition-key scoping complicates cross-partition top-K correctness and may require fan-out/merge strategies.
• Validation required: AWS performance/recall claims need real-world benchmarks under your workload.

Choosing a vector store: DynamoDB versus alternatives

Decide based on three core axes: query scope (per-partition vs global), required filtering capabilities, and the level of index control/tuning you need. AWS now offers multiple in-place vector options: DynamoDB native vectors, DynamoDB → OpenSearch zero-ETL, S3 Vectors, and external managed/specialized vector DBs (Pinecone, Weaviate, FAISS-based deployments).

• DynamoDB native vectors: best when you already store operational rows in DynamoDB and queries are usually partition-scoped (per-tenant).
• DynamoDB → OpenSearch (zero-ETL): useful when you need richer text search or complex filtering without a custom sync pipeline.
• S3 Vectors: cost-effective serverless option for large-scale, mostly-read workloads and analytical batch use cases.
• Specialized vector DBs: offer tuning knobs, global Top-K, and advanced filtering or hybrid scoring at the cost of extra sync/operational complexity.

Amazon DynamoDB (native vector search) — when to consider it

Consider DynamoDB vector search when embeddings and metadata are already operationally owned in DynamoDB, and the typical retrieval patterns are scoped by partition key (for example, per customer or per workspace). The co-location removes a synchronization headache and simplifies governance.

Avoid relying on DynamoDB vectors for use cases that require global top-K across many partitions without fan-out, complex range or prefix filters, or deep index tuning (quantization, efConstruction-like knobs).

• Pros: lower operational overhead, unified billing and IAM, simpler write path reducing sync drift.
• Cons: partition-scoped queries, exact-match inline filters only, Top-K capped at 100, and fewer documented tuning options.

Schema and partitioning patterns

Schema design must make the partitioning strategy explicit. If per-tenant isolation is desired, use tenantId as the partition key and store embedding as a List(Number) attribute on the same item. If you need global search, you will either fan-out queries to multiple partition keys and merge results or maintain a separate global index in S3 Vectors/OpenSearch/external vector DB.

Plan for update/delete semantics: co-located writes reduce sync drift, but measure how quickly SearchVectors reflects writes and deletions for your expected update rates.

• Example item: { PK: tenant#123, SK: item#456, embedding: [..], title: "...", metadata: {...}, updatedAt: "..." }.
• Match embedding dimension and distance function to your embedding model as part of index creation.
• If you expect high cardinality filters, prefer pre-partitioning or post-filtering instead of relying on inline filters.

Implementation checklist and pilot plan

A short pilot will answer most adoption questions. Prepare a representative dataset (1k–10k items per partition), fix an embedding model and preprocessing pipeline, and configure a DynamoDB vector index with the matching dimension and distance metric. Run the planned benchmarks and compare to a baseline vector DB or OpenSearch.

Measure both functional and operational signals: recall and precision metrics, tail latency, index update visibility, and predictable cost models under expected query volumes.

• Pre-deployment: choose embedding model, dimension, distance function; select sample partitions and data slices.
• Pilot tests: recall@10/20/100, p50/p95/p99 latency under concurrency, update/delete reflection time, and cost estimation for expected QPS.
• Migration steps: export embeddings if migrating out, use rollback plans, and monitor update consistency.

Benchmarks & measurement plan

Design reproducible benchmarks: vary dataset size, partition counts, and query fan-out patterns. Record recall@K, precision@K, p50/p95/p99 latency, cost per 1M queries, and index update latency under concurrent writes. Compare results against FAISS/Pinecone/OpenSearch or S3 Vectors baselines.

Make sure to hold embedding generation costs separate (Bedrock/OpenAI/Cohere) and report them as part of TCO calculations.

• Suggested dataset sizes: 1k, 10k, 100k per partition for scale steps; include a mixed partition distribution for multi-tenant testing.
• Essential metrics: recall@10/20/100, p99 latency, index update staleness, cost per million queries, and memory/storage footprint.
• Run A/B tests with same vectors and queries across DynamoDB vectors and a specialist vector DB to isolate index algorithm differences.

Security, governance, and compliance

Co-locating embeddings and operational data centralizes governance but increases blast radius. Validate encryption at rest/in transit, granular IAM for SearchVectors and write operations, audit logging, and deletion semantics for privacy compliance (GDPR right-to-be-forgotten).

Operational patterns include separating highly sensitive embeddings into dedicated tables/partitions, applying VPC endpoints, and thorough access orchestration for agents versus end users.

• Use least-privilege IAM roles for vector searches and write paths.
• Instrument audit logs for both search and write operations and correlate with requester context for investigations.
• Plan deletion workflows that guarantee embedded vectors are removed or invalidated within your compliance window.

Decision checklist (quick-read)

If you need per-tenant low-latency retrieval and your data already lives in DynamoDB → run a targeted pilot. If you require global Top-K, complex predicates, or index tuning, favour a specialized vector DB or hybrid architecture. Always baseline DynamoDB claims with your workload.

• Does your application store content in DynamoDB and is retrieval usually partition-scoped? If yes, pilot DynamoDB vectors.
• Do you require global Top-K without fan-out or advanced filtering and tuning? If yes, pick a specialized vector DB or S3 Vectors/OpenSearch hybrid.
• Include pilot acceptance thresholds for recall, tail latency, and cost before changing production architecture.

Frequently Asked Questions

Can DynamoDB replace Pinecone or FAISS for all RAG workloads? Not for all workloads. DynamoDB vectors are compelling when embeddings and metadata already live in DynamoDB and queries are partition-scoped. For global top-K retrievals, advanced filtering, or fine-grained index tuning, a specialized vector database or hybrid approach remains preferable.

What should I validate in a DynamoDB vectors pilot? Validate recall@K and precision@K against a baseline, measure p95/p99 latency under realistic load, measure index update visibility after writes/deletes, and model per-query costs at expected volume.

How do inline filters in DynamoDB vector search work? AWS documents inline filters as supporting exact-match conditions only (no range or prefix filters). The execution semantics (whether filters prune prior to nearest-neighbor search or post-filter the top candidates) are not fully documented and should be validated in pilot tests.

Conclusion

Native vector search in DynamoDB is an important addition to the set of architectural choices for Retrieval-Augmented Generation. It reduces operational friction for certain workloads but introduces partitioning and filtering trade-offs. Use a short, measurable pilot to confirm recall, latency, update freshness, and cost before adopting the feature widely.