The RAG Superstack

AWS has made it remarkably easy to spin up an "AI-integrated" RAG application. S3 stores your knowledge base, Bedrock's ETL pipelines handle ingestion, and Bedrock Runtime gives you a menu of foundation models to choose from, priced to fit your budget. For documents, PDFs, wikis, policy text, this works beautifully.

Where it quietly falls apart is CSVs and Excel files. These files hold structured, tabular data, and the mechanics of semantic search were never built to reason about rows and columns, no matter how good the embedding model is.

How Semantic Search Actually Works

Before explaining why it breaks on spreadsheets, it helps to be precise about what semantic search is doing under the hood:

1. Vector embeddings: an embedding model converts raw text into high-dimensional vectors. Terms with similar meaning ("automobile" and "car") end up geometrically close together in that vector space.
2. Vector indexing: document text is chunked, embedded, and stored in a vector database or in-memory index.
3. Query encoding: a user's prompt ("budget vehicle") is converted into a vector using that same embedding model.
4. Similarity matching: the engine measures distance between the query vector and stored vectors, typically with cosine similarity or k-nearest-neighbors.
5. Ranking: results come back sorted by conceptual relevance, not keyword overlap.

Where This Breaks Down for Tabular Data

Semantic search is built for context retrieval, not deterministic calculation. It can tell you which rows are "about" a topic, but it cannot reliably compute a sum, an average, or a grouped trend, because vector similarity has no concept of arithmetic. Ask a vector index for "total revenue by region last quarter" and, at best, it retrieves rows that mention revenue; at worst, the model guesses at a number.

The core problem

Vector similarity answers “what is this about?” Analytics needs “what does this equal?”. And those are fundamentally different kinds of questions.

Enter Glue and Athena

Structured analytics needs structured storage and structured querying. Instead of embedding CSV rows and hoping semantic search can approximate arithmetic, convert them to Parquet and query them directly with Athena.

Data Ingestion

• User uploads a CSV.
• Its metadata is recorded in a local database for discovery.
• A Parquet file is generated from the source data.
• The Parquet file is stored in S3, registered in the Glue Data Catalog.

Query Flow

1. Intent extraction: the LLM analyzes the user's question and extracts the relevant metadata (dataset, entities, time range).
2. Dataset discovery: a discovery service matches that metadata against stored dataset descriptions to locate the exact S3 tables and schemas the question needs.
3. Schema injection: the system pulls the table DDL and column definitions from the AWS Glue Data Catalog and adds them to the Bedrock LLM's context alongside the prompt.
4. Text-to-SQL generation: Bedrock translates the natural-language question into a valid ANSI SQL query in Athena's Presto/Trino dialect.
5. Execution: the generated query runs in Amazon Athena, scanning the compressed Parquet files directly in S3.
6. Synthesis: Athena returns the exact numeric or tabular result, and Bedrock turns it into a clear natural-language answer.

Why This Beats Standard Vector Search for Tabular Data

• Columnar efficiency:Parquet + S3 gives you columnar storage and compression, so Athena scans only the columns a query actually needs, keeping cost low and queries fast.
• Centralized schema management:The Glue Data Catalog acts as the metastore. New CSVs update the schema automatically, with no risk of diluting or corrupting a vector index.
• Serverless, deterministic math:Aggregations, grouping, and multi-table joins are delegated to Athena's query engine instead of being guessed at by the model.

Semantic Search vs. SQL Analytics, at a Glance

The Hybrid RAG Architecture

The real payoff comes from combining both approaches into a single platform, each handling what it's actually good at:

• Bedrock Knowledge Bases: handles unstructured data (PDFs, docs, free text) via vector embeddings for semantic search and context retrieval.
• Glue + Athena: handle structured data (CSVs, financial reports, log dumps) via deterministic SQL queries.
• Bedrock Agents: act as the router, analyzing each incoming prompt and deciding whether to hit the vector store, run SQL through Athena, or merge results from both before answering.

Stop asking semantic search to do a database's job. Pair Bedrock with Glue and Athena, and your RAG app can handle unstructured context and deterministic math with equal confidence.