Data Agent Architecture: Intent, Tools, Data, and Semantics

40 min

What Is a Data Agent? Chatbot vs BI vs Workflow

Learning objective: define when an agent is justified; distinguish natural-language interface from autonomous analytical action.

A department head types "why did conversion drop in June" into a chat box and expects a trustworthy, sourced answer within a minute. A fixed BI dashboard cannot answer an open-ended follow-up question; a chatbot with no data access can only guess; a scheduled workflow can only run the report it was built for. A Data Agent sits between these: it understands the request in natural language, plans which data and tools it needs, executes a bounded set of read (and sometimes write) actions, and returns an answer with evidence. That extra capability is also extra risk, so the first design decision is not "how do we build the agent" but "is an agent even justified for this request, or is a dashboard or a fixed workflow safer and sufficient."

import pandas as pd

# clean: a read-only query log against the governed analytics sandbox this course uses
df = pd.read_csv("DS302_Analytics_Package_Clean.csv", parse_dates=["event_time", "ingested_at"])
df.shape
df.dtypes
df[["run_id", "source_system", "batch_id", "metric", "value", "schema_version"]].head()

# Ask first: is a dynamically planned agent even needed here, or would a fixed dashboard answer this?
df["source_system"].unique()

• The scope of the question changes almost daily, and a new question cannot be assembled from an existing dashboard's filters.
• The team needs to dynamically decide which of several data sources to check, rather than always querying the same fixed table.
• The answer must cite a traceable query and dataset version, not just a single number.
• The task has branching logic that requires human approval before it runs; a scheduled workflow cannot express that kind of conditional check.
• If a fixed dashboard, report, or workflow can already answer most questions in an acceptable amount of time, an agent is not needed; adding one only adds a layer of risk without a matching benefit.

Key takeaway: Completing the steps is not enough; the result must be verifiable, reproducible, and explainable.

45 min

Intent → Analysis Plan: Plan Before Execution

Learning objective: translate a request into entities, metrics, filters, time range, plan, evidence requirements, and approval needs.

A learner asks the agent "how are we doing this quarter" and the agent immediately fires off a query. Doing what, for whom, over what date range, compared against what baseline? A vague request executed immediately produces a vague, unverifiable answer. Before an agent calls any tool, it should turn the request into an explicit analysis plan: which entities and metrics are involved, what filters and time range apply, what steps will produce the answer, what evidence is required to trust the result, and whether any step needs human approval before it runs.

# Vague: no plan, straight to execution
# "how are we doing this quarter" -> agent guesses a query and returns a number

# Planned: the request is decomposed before anything executes
plan = {
    "entities": ["source_system"],
    "metrics": ["conversion_rate", "active_users"],
    "filters": {"schema_version": "v2"},
    "time_range": "2026-Q2",
    "steps": ["load current-quarter rows", "validate against contract", "aggregate by source_system"],
    "evidence_required": ["row count", "dataset_version", "definition of conversion_rate"],
    "approval_needed": False  # read-only aggregate; would be True for a write/publish action
}

1. 1Restate the request as a question
   Turn free text into a specific, checkable question before choosing any tool.
2. 2Resolve entities, metrics, filters, time range
   Fill in each element explicitly; if information is missing, ask instead of guessing.
3. 3List evidence requirements
   Decide up front what the final answer must cite to be considered trustworthy.
4. 4Flag approval needs
   Mark any step that writes, publishes, or has consequential impact as needing human approval.

• Interpreting "this quarter" as whatever 90-day window the agent picks, instead of the period the finance calendar actually defines.
• Reusing one department's definition of "users" when a different department asked the question with its own definition in mind.
• Presenting a single aggregated number as the final answer without ever requiring any evidence for it.
• Quietly filling in a "reasonable-sounding" assumption for an underspecified request instead of asking the requester to clarify.

Key takeaway: Completing the steps is not enough; the result must be verifiable, reproducible, and explainable.

45 min

Tool Layer: SQL, Python, Retrieval, and APIs

Learning objective: use allowlisted tools with narrow schemas; separate read from write/action tools.

A colleague gives the agent one general-purpose "run_sql" tool that accepts any query string, so it can also read it can also DROP a table, or read tables it was never meant to touch. A narrow tool layer is the boundary that keeps a Data Agent's capability matched to what was actually approved: instead of one all-powerful SQL tool, the agent gets several narrow tools, each with an explicit schema, a stated purpose, and a fixed read-only or write/action classification.

# Wrong: one tool that accepts arbitrary SQL, read or write, any table
def run_sql(query: str) -> pd.DataFrame:
    return db.execute(query)  # no allowlist, no read/write separation

# Right: narrow, purpose-built, read-only tools with explicit schemas
def query_metric_summary(source_system: str, metric: str, time_range: str) -> pd.DataFrame:
    """Read-only. Only queries the pre-approved analytics_sandbox view."""
    ALLOWED_SOURCE_SYSTEMS = {"crm", "billing", "support", "web_analytics", "ops"}
    if source_system not in ALLOWED_SOURCE_SYSTEMS:
        raise ValueError(f"source_system not allowlisted: {source_system}")
    return sandbox_view.query(source_system=source_system, metric=metric, time_range=time_range)

def publish_report(report_id: str, content: dict) -> None:
    """Write/action tool. Requires human approval before it runs (see Lesson 9)."""
    raise PermissionError("publish_report requires an approved run; see approval workflow")

• Each tool does exactly one clearly named thing; the name itself states its scope.
• Read tools and write/action tools are separate functions with their own permissions and their own allowlists.
• Parameters use explicit types and enums instead of free-form strings that let the agent assemble its own query conditions.
• Anything not explicitly approved, a source system, a table, a column, is denied by default rather than allowed by default.
• Every tool call is logged with the caller, the parameters, and the dataset version, so it can be audited afterward.

Key takeaway: Completing the steps is not enough; the result must be verifiable, reproducible, and explainable.

45 min

Semantic Layer and Trusted Metrics

Learning objective: ground metrics, dimensions, definitions, join paths, units, and policy in a trusted semantic layer.

Two departments ask the agent for "conversion rate" and get two different numbers, because one team's query includes refunded orders and the other's does not, and nobody wrote either definition down. Without a shared semantic layer, an agent free to invent its own SQL will silently invent its own definition of every metric, every time. A semantic layer fixes metric formulas, dimension definitions, join paths, units, and access policy in one place the agent must consult instead of guessing.

SEMANTIC_METRICS = {
    "conversion_rate": {
        "formula": "converted_sessions / total_sessions",
        "unit": "ratio (0-1)",
        "owner": "analytics-platform-team",
        "grain": "source_system x batch_id"
    },
    "active_users": {
        "formula": "count(distinct entity_id where last_event <= 30 days)",
        "unit": "count",
        "owner": "analytics-platform-team",
        "grain": "source_system x batch_id"
    }
}

def resolve_metric(name: str) -> dict:
    if name not in SEMANTIC_METRICS:
        raise ValueError(f"metric not in semantic layer, cannot resolve a trusted definition: {name}")
    return SEMANTIC_METRICS[name]

Key takeaway: Completing the steps is not enough; the result must be verifiable, reproducible, and explainable.

65 min

Guided Lab: Verifiable Read-Only Data Agent

Learning objective: with instructor guidance, build a read-only prototype that plans, queries, executes Python if needed, cites dataset versions, and verifies totals.

• Go to the DS 302 Practice page
• PreviewDownload the clean analytics sandbox (CSV, synthetic data)
• Download the Guided Lab starter notebook (read-only agent prototype)
• PreviewDownload the data dictionary (Traditional Chinese)
• PreviewDownload the data dictionary (English)

1. 1Plan
   Given a sample request, write out the entities, metrics, filters, time range, and evidence requirements before writing any query.
2. 2Query (read-only)
   Call only the narrow, allowlisted read tool against the sandbox; never a general-purpose query tool.
3. 3Execute Python if needed
   If aggregation is needed beyond the tool's own output, isolate it into a small, explicit function.
4. 4Cite dataset version
   Record schema_version and the row count actually used, alongside the answer.
5. 5Verify totals
   Cross-check at least one total (such as a row count or sum) against an independent expectation before presenting the answer.

• Presenting an aggregated number without recording the schema_version or row count behind it.
• Running the query once and assuming it is correct, without a second independent check on the total.
• Treating "the code ran without an error" as proof the logic is correct, without actually checking whether the output is plausible.
• Skipping the self-check of whether someone else could re-run this and get the same answer.

Key takeaway: Completing the steps is not enough; the result must be verifiable, reproducible, and explainable.