RAG vs Fine-Tuning: How Should AI Use Your Business Knowledge?

Executive summary

AWS announced general availability of native vector search in Amazon DynamoDB on August 5, 2026. DynamoDB now includes a vector index type and a SearchVectors API that stores embeddings as List(Number), supports up to 4096 dimensions, and returns up to Top‑K=100 results.

One-sentence guidance: If your operational data already lives in DynamoDB, your queries are naturally scoped to a partition key (for example, per‑tenant), and you prioritise low operational overhead, run a pilot. If you need global Top‑K retrieval, complex range filters, or fine‑grained index tuning, prefer a specialized vector DB or a hybrid design.

• Run a 2‑week pilot measuring recall@K, p95/p99 latency, update freshness, and cost per million queries.
• Revisit tenant partitioning: DynamoDB vector search is partition‑key scoped and alters multi‑tenant designs.
• Treat AWS performance and recall statements as vendor claims that require independent benchmarking.

What AWS actually delivered (feature summary & limits)

According to the AWS announcement, DynamoDB adds a vector index type and SearchVectors API. Vectors are stored as the existing List(Number) attribute; indexes accept a dimension parameter (up to 4096) and distance functions Cosine, Euclidean, and Dot. SearchVectors returns up to 100 nearest results and supports inline exact‑match filters.

AWS positions the feature as a serverless, horizontally scaling vector store co-located with operational data. The announcement does not publish the underlying index algorithm, detailed per‑query pricing, or exact index tuning knobs available to customers.

• Supported dimensions: up to 4096.
• Distance functions: Cosine, Euclidean, Dot.
• Top‑K limit: up to 100 results per SearchVectors call.
• Search scope: constrained to a single partition‑key value per call.
• Filter semantics: inline filters support exact‑match only.

Why this matters for RAG architectures

Native vector search in DynamoDB reduces the need to copy rows into a separate vector store, simplifying ingestion and lowering operational surface area. Teams can co-locate metadata and embeddings and perform similarity retrieval without an external sync pipeline.

However, architectural trade‑offs—partition‑scoped searches, limited inline filtering, and limited public tuning controls—change the calculus for multi‑tenant apps and workloads that require global recall or advanced hybrid scoring.

• Operational benefit: simpler write path and unified access, billing, and governance inside a single AWS account.
• Architectural constraint: partition‑key scoping complicates cross‑partition Top‑K correctness and may require fan‑out/merge strategies.
• Validation required: AWS performance/recall claims need real‑world benchmarks under your workload.

Choosing a vector store: DynamoDB versus alternatives

Decide based on three core axes: query scope (per‑partition vs global), required filtering capabilities, and the level of index control/tuning you need. AWS now offers multiple in‑place vector options: DynamoDB native vectors, DynamoDB → OpenSearch zero‑ETL, S3 Vectors, and external managed/specialized vector DBs (Pinecone, Weaviate, FAISS‑based deployments).

• DynamoDB native vectors: best when you already store operational rows in DynamoDB and queries are usually partition‑scoped (per‑tenant).
• DynamoDB → OpenSearch (zero‑ETL): useful when you need richer text search or complex filtering without a custom sync pipeline.
• S3 Vectors: cost‑effective serverless option for large‑scale, mostly‑read workloads and analytical batch use cases.
• Specialized vector DBs: offer tuning knobs, global Top‑K, and advanced filtering or hybrid scoring at the cost of extra sync/operational complexity.

Amazon DynamoDB (native vector search) — when to consider it

Consider DynamoDB vector search when embeddings and metadata are already operationally owned in DynamoDB, and the typical retrieval patterns are scoped by partition key (for example, per customer or per workspace). The co‑location removes a synchronization headache and simplifies governance.

Avoid relying on DynamoDB vectors for use cases that require global top‑K across many partitions without fan‑out, complex range or prefix filters, or deep index tuning (quantization, efConstruction‑like knobs).

• Pros: lower operational overhead, unified billing and IAM, simpler write path reducing sync drift.
• Cons: partition‑scoped queries, exact‑match inline filters only, Top‑K capped at 100, and fewer documented tuning options.

Schema and partitioning patterns

Schema design must make the partitioning strategy explicit. If per‑tenant isolation is desired, use tenantId as the partition key and store embedding as a List(Number) attribute on the same item. If you need global search, you will either fan‑out queries to multiple partition keys and merge results or maintain a separate global index in S3 Vectors/OpenSearch/external vector DB.

Plan for update/delete semantics: co‑located writes reduce sync drift, but measure how quickly SearchVectors reflects writes and deletions for your expected update rates.

• Example item: { PK: tenant#123, SK: item#456, embedding: [..], title: '...', metadata: {...}, updatedAt: '...' }.
• Match embedding dimension and distance function to your embedding model as part of index creation.
• If you expect high cardinality filters, prefer pre‑partitioning or post‑filtering instead of relying on inline filters.

Implementation checklist and pilot plan

A short pilot will answer most adoption questions. Prepare a representative dataset (1k–10k items per partition), fix an embedding model and preprocessing pipeline, and configure a DynamoDB vector index with the matching dimension and distance metric. Run the planned benchmarks and compare to a baseline vector DB or OpenSearch.

• Pre‑deployment: choose embedding model, dimension, distance function; select sample partitions and data slices.
• Pilot tests: recall@10/20/100, p50/p95/p99 latency under concurrency, update/delete reflection time, and cost estimation for expected QPS.
• Migration steps: export embeddings if migrating out, use rollback plans, and monitor update consistency.

Benchmarks & measurement plan

Design reproducible benchmarks: vary dataset size, partition counts, and query fan‑out patterns. Record recall@K, precision@K, p50/p95/p99 latency, cost per 1M queries, and index update latency under concurrent writes. Compare results against FAISS/Pinecone/OpenSearch or S3 Vectors baselines.

Make sure to hold embedding generation costs separate (Bedrock/OpenAI/Cohere) and report them as part of TCO calculations.

• Suggested dataset sizes: 1k, 10k, 100k per partition for scale steps; include a mixed partition distribution for multi‑tenant testing.
• Essential metrics: recall@10/20/100, p99 latency, index update staleness, cost per million queries, and memory/storage footprint.
• Run A/B tests with same vectors and queries across DynamoDB vectors and a specialist vector DB to isolate index algorithm differences.

Security, governance, and compliance

Co‑locating embeddings and operational data centralizes governance but increases blast radius. Validate encryption at rest/in transit, granular IAM for SearchVectors and write operations, audit logging, and deletion semantics for privacy compliance (GDPR right‑to‑be‑forgotten).

Operational patterns include separating highly sensitive embeddings into dedicated tables/partitions, applying VPC endpoints, and thorough access orchestration for agents versus end users.

• Use least‑privilege IAM roles for vector searches and write paths.
• Instrument audit logs for both search and write operations and correlate with requester context for investigations.
• Plan deletion workflows that guarantee embedded vectors are removed or invalidated within your compliance window.

Operational appendix: Snowflake Cortex — CORTEX_MODELS_ALLOWLIST → Model RBAC (Aug 17–Sep 4, 2026)

Important operational note for teams running RAG and embedding pipelines on Snowflake: Snowflake published release note BCR‑2378 announcing deprecation of the account parameter CORTEX_MODELS_ALLOWLIST and a one‑time automated migration to model RBAC. Snowflake schedules the migration between August 17 and September 4, 2026, and indicates enforcement via the 2026_07 behavior‑change bundle will remove allowlist‑based access.

Practically, calls that create or use embeddings (AI_EMBED, EMBED_TEXT_*), Cortex Search Services, and Cortex Agents will be evaluated against model RBAC grants. Accounts that relied on the account allowlist must verify that the automated mapping created the intended application‑role assignments or apply manual remediation to avoid denied requests.

• The migration maps existing allowlist entries to application roles (CORTEX‑MODEL‑ROLE‑*) during Aug 17 — Sep 4, 2026; teams must verify mappings and correct them if necessary.
• Affected components include AI_EMBED, EMBED_TEXT_768/1024, Cortex Search Services, Cortex Agents, stored procedures, notebooks, and CI jobs that invoke embed functions.
• If not remediated, production embedding/RAG pipelines may see permission‑denied errors once enforcement completes.
• Snowflake recommends setting CORTEX_MODELS_ALLOWLIST = 'None' to opt into RBAC‑only behavior (test first).

1. 1Inventory embedding usage: query QUERY_HISTORY for AI_EMBED and EMBED_TEXT_* calls and search stored procedures, tasks, and CI pipelines for references.
2. 2Identify executor roles: determine which Snowflake ROLE executes each embedding call (service roles, scheduler roles, user roles, or PUBLIC).
3. 3Test RBAC in non‑prod: set CORTEX_MODELS_ALLOWLIST='None' in a test account and run representative embedding calls under each executor role.
4. 4During Aug 17—Sep 4, 2026, review the automated mapping that creates CORTEX‑MODEL‑ROLE‑* application roles and confirm each executor role was granted the correct application roles.
5. 5Manually grant or revoke application roles as needed using GRANT/REVOKE if the automated mapping does not match policy intent.
6. 6Update IaC/CI to manage model application role grants and add runtime error handling that surfaces DENIED permission errors as operational alerts.

Technical trade‑offs and risks

Moving model access from a single account allowlist to per‑role RBAC gives finer control and auditability but increases operational overhead: teams must manage grants per role and ensure CI/IaC reflect model‑to‑role bindings.

Risks include incorrectly mapped grants (too permissive), missed mappings causing service interruption, and third‑party integrations that implicitly relied on the account parameter.

• Automated mapping may not encode organizational intent; plan for manual remediation.
• Third‑party or legacy integrations may require mapping to specific Snowflake roles rather than relying on account settings.
• Error handling differences between allowlist denials and RBAC denials may require application changes to surface actionable alerts.

Post‑migration validation checklist

Validate the migration by running representative embedding calls for each executor role, checking Cortex Search Services and Agents, and confirming audit logs show model access events attributed to roles.

• Run AI_EMBED/EMBED_TEXT_* calls per role and ensure success.
• Confirm Cortex Search indexes and Agents return consistent results after the migration.
• Generate reports showing model usage by role for governance and cost reviews.

CI testing and deployment guidelines

CI owners should add a test matrix that runs embedding calls under the same roles used in production, include deny‑mode tests, and add monitoring alerts for permission failures.

• Pre‑migration matrix: staging tests for each executor role with embedding create/read calls.
• Deny‑mode tests: intentionally remove a grant to confirm systems alert and remediation steps are clear.
• Monitoring: alert on increased DENIED errors and add dashboards for embedding success rate by role.

Governance, audit, and cost control

Use model RBAC to enforce least privilege and to control access to high‑cost or proprietary models. Update governance documents to reflect per‑role grants and schedule periodic reviews.

• Classify models by cost and sensitivity, and only grant expensive models to approved roles.
• Require approval workflows for granting CORTEX‑MODEL‑ROLE‑* application roles in production.
• Schedule periodic audits and automated reports showing model usage and grants.

Uncertainties to verify with Snowflake

Snowflake's release note defines the migration window but does not publish every detail of the automated mapping rules, whether the migration can be re‑run, or exact per‑region enforcement timing. Teams should verify account‑specific behavior during the Aug 17—Sep 4, 2026 window and contact Snowflake support for account‑scoped questions.

• Mapping mechanics: naming conventions and edge cases for automated creation of CORTEX‑MODEL‑ROLE‑* are not fully documented.
• Custom/self‑hosted models behavior: confirm whether models registered via a Model Registry or Snowpark Containers require manual mapping.
• Rollback options: confirm with Snowflake whether and how the one‑time mapping can be undone.

Frequently Asked Questions

Will the migration change which users or jobs can call models? Possibly. The automated migration maps allowlist entries to model RBAC application roles; if the executor role was not granted the mapped application role, its calls will be denied. Validate grants for each executor role during the migration window.

What if my jobs run as ACCOUNTADMIN? Relying on ACCOUNTADMIN is brittle from a least‑privilege perspective. Create dedicated service roles with explicit model application role grants and avoid broad reliance on ACCOUNTADMIN where possible.

Where can I get official Snowflake documentation and support? Use the Snowflake release note BCR‑2378, the Snowflake Cortex AI (AI SQL) guide, and the Cortex privileges documentation. Open a Snowflake support ticket for account‑specific mapping questions.

Conclusion

Snowflake’s move from an account allowlist to model RBAC consolidates model access control into per‑role grants, improving auditability and finer control but requiring operational work: inventory, verify mappings during Aug 17—Sep 4, 2026, remediate grants, update IaC/CI, and add runtime alerts for permission denials to avoid production outages.