What Corporate Implementation Looks Like

Lunch and Learn — Session 3

Kerry Back, Rice University

What Your Graduates Will Face

In Sessions 1 and 2, we focused on what students can do and how faculty can use AI. Today, we step into the corporate world — the environment your graduates are entering.

AI Meets Enterprise Data

The Problem

• “Which customers buy from multiple divisions?”
• Touches three CRMs, an ERP, and an HR system
• Different naming conventions, date formats, data models
• Traditional answer: emails to five people over three weeks
• Enterprise systems were never designed to talk to each other

The Agent Answer

• Ask in plain English
• Agent queries each system sequentially
• Reconciles results in Python (not a single SQL join)
• Handles fuzzy matching, date formats, naming differences
• 30 seconds — but requires verification

The speed is real, but so are the risks. The 30-second answer may contain the same silent errors we discussed in Session 1. The new challenge: how do you verify a 30-second answer?

This is the core enterprise use case. Cross-system questions that take weeks to answer manually. An AI agent answers in 30 seconds — but the answer may contain silent errors. The challenge is building verification into the fast path.

XYZ Corp Custom Chatbot

A simulated enterprise for teaching

• $500M B2B industrial supplies distributor with three divisions: Industrial, Energy, Safety
• 9 enterprise systems: Salesforce (Industrial CRM), Legacy CRM (Energy), HubSpot (Safety), NetSuite (finance), SAP (supply chain), Oracle SCM, Workday (HR), Zendesk (support), QuickBooks
• 10 DuckDB databases pre-loaded as Parquet files — 41 tables, 26,000+ rows, 3 years of data
• Web interface: Ask questions in plain English → Claude queries databases, generates charts, creates documents
• Built for the Rice “From BI to AI” executive education program

XYZ Corp is a simulated company we built for the Rice executive education program. Nine enterprise systems, realistic data, realistic cross-system friction. Executives log in, ask questions in English, and watch the AI query multiple databases and produce answers.

Demo: Cross-System Query

LIVE DEMO

I’ll ask the XYZ Corp chatbot:

“Which customers buy from multiple divisions? Show combined revenue and flag name mismatches.”

Watch the agent:

1. Query Salesforce (Industrial division customers)
2. Query Legacy CRM (Energy division customers)
3. Query HubSpot (Safety division customers)
4. Fuzzy-match customer names across all three systems
5. Produce a result table with combined revenue

LIVE DEMO: Open the XYZ Corp chatbot and ask the cross-system customer question. Show the result table, point out the name mismatches, and discuss what verification would be needed before presenting this to a board.

The Result

Demo output

• 10 customers identified across 2+ divisions
• 4 buying from all 3 divisions: $56.5M combined revenue (11.4% of total)
• Only 3 of 10 had matching names across systems — 7 required fuzzy matching

Customer	Industrial	Energy	Safety	Combined
General Electric	GE Industrial	GE Energy Solutions	GE Safety Div	$18.4M
ExxonMobil	ExxonMobil Corp	Exxon Mobil	ExxonMobil LLC	$14.7M
Dow Chemical	Dow Inc	Dow Chemical Co	Dow Safety	$12.1M
Chevron	Chevron Corp	Chevron USA	Chevron Safety	$11.3M

The name mismatches are the story. “General Electric” appears as three different strings across three systems. Without fuzzy matching, these look like 12 separate customers, not 4.

This is the actual output from the XYZ Corp demo. Four companies buying from all three divisions. The name mismatches are the key insight — General Electric appears as three different strings. Without the agent’s fuzzy matching, you’d think these are separate customers.

Behind the Scenes

1️⃣ Query Salesforce Industrial division customers and revenue

2️⃣ Query Legacy CRM Energy division customers and revenue

3️⃣ Query HubSpot Safety division customers and revenue

4️⃣ Fuzzy Merge Python: match names, reconcile, deduplicate

Not a single SQL join. The agent runs sequential queries against each system, then merges in Python. This is critical because enterprise systems have different schemas, naming conventions, and date formats.

The agent doesn’t connect to a single database. It runs sequential queries — Salesforce first, then Legacy CRM, then HubSpot — and merges the results in Python. The fuzzy matching step is where it handles “General Electric” vs. “GE Industrial Solutions” vs. “GE Safety Division.”

From Data to Deliverable

📊 Query Agent queries multiple databases

🔗 Merge Reconcile across systems

🧮 Compute Calculate metrics and trends

📈 Chart Generate visualizations

📄 Narrate Assemble a finished report

Unlike a dashboard that answers yesterday’s questions, an agent answers any question you think of right now — constructing new queries for each one. The same pipeline produces a cross-system customer analysis, a quarterly executive summary, or a supply chain risk report.

The full pipeline: query, merge, compute, chart, narrate. This is what separates agents from dashboards. A dashboard answers pre-built questions. An agent constructs new queries for every question.

Demo: Quarterly Executive Summary

LIVE DEMO

I’ll ask the XYZ Corp chatbot to prepare a quarterly executive summary. The agent will:

• Query 6 systems (CRMs, Workday, Zendesk, finance)
• Compute KPIs: revenue by division, QoQ growth, headcount efficiency, customer concentration
• Generate charts and narrative

Then I’ll iterate on the same data:

• Draft 1: Comprehensive data summary (flat, everything included)
• Draft 2: Traffic-light format (red/yellow/green, flag >10% off plan)
• Draft 3: One-page executive brief (lead with risk, end with actions)

Same data, same agent — the prompt is the variable. Three different formats from three different instructions, demonstrating the iteration principle from Session 2 at enterprise scale.

LIVE DEMO: Show the quarterly summary, then iterate three times on the same data. Data dump, traffic light, executive brief. The key point: the prompt determines the output format. Same data, three different deliverables.

The Agent Loop

📋 System Prompt Schema descriptions, business rules, data gotchas

🔧 Tool Call LLM writes SQL and requests execution

⚡ Execute System runs SQL, returns results

✅ Test & Review LLM checks results and self-corrects

The test-and-review step is what separates agents from dashboards. A dashboard runs a pre-written query — if it’s wrong, it’s wrong forever. An agent reviews its results and self-corrects: “Row count seems low — let me check the WHERE clause.”

The agent loop. System prompt contains institutional knowledge. LLM writes SQL, system executes, LLM reviews. If something looks wrong — row count too low, unexpected nulls — it self-corrects and tries again.

The System Prompt

Institutional knowledge encoded as text

Available tables: - salesforce_opportunities: Industrial division deals - legacy_orders: Energy division (dates stored as MM/DD/YYYY text) - hubspot_deals: Safety division - workday_employees: HR data (headcount, department, hire date)

Business rules: - “Average deal size” means closed-won deals only - Customer names differ across systems — use fuzzy matching - Fiscal year starts February 1 - legacy_orders dates are text strings, not DATE type

The system prompt is what turns a generic LLM into your organization’s analyst. Without it, the agent makes the same mistakes a new hire would — averaging across all deals instead of closed-won, parsing date strings incorrectly, counting the same customer three times.

This is the system prompt from the XYZ Corp chatbot. It contains schema descriptions and data gotchas. Without this, the agent makes the exact silent errors we discussed in Session 1 — wrong filters, date misinterpretation, double-counting.

Enterprise Error Modes

Currency & Unit Errors

• Revenue in USD in one system, EUR in another
• Agent sums without converting
• Reported total is meaningless
• No error thrown — the numbers just add up wrong

Fiscal vs. Calendar Year

• Finance system uses fiscal year (Feb–Jan)
• CRM uses calendar year (Jan–Dec)
• Agent joins on “2025” — misaligns by one month
• Q4 revenue attributed to wrong period

Intercompany Elimination

• Division A sells to Division B
• Agent counts the transaction as external revenue
• Consolidated revenue overstated
• Standard accounting rule the AI doesn’t know

These are uniquely enterprise problems — they arise only when AI queries across multiple systems. The pattern is the same as Session 1: confident, well-formatted, wrong. The system prompt is the defense.

Enterprise-specific failures that go beyond Session 1. Currency mismatches, fiscal vs. calendar year, intercompany elimination. These only arise in cross-system queries — exactly the use case that makes enterprise AI valuable.

Deployment Architecture

Once you’ve proved value, how do you deploy? Real architectural decisions with real implications.

Where Does the LLM Run?

Cloud API

• Easiest setup, best model quality
• Data leaves your network
• Pay per token
• Zero-retention agreements available
• Morgan Stanley: GPT-4 to 16,000 advisors with zero data retention for SEC/FINRA compliance

On-Premise

• Full data control — nothing leaves the building
• Open-source models (Llama, Mistral)
• $50–200K GPU infrastructure
• Quality gap remains but narrowing
• Developer tooling gap is larger than model gap

Hybrid

• Route by sensitivity level
• Public data → cloud API (best quality)
• Sensitive data → on-premise model
• Becoming the mainstream enterprise approach
• Most sophisticated path

Three options. Cloud API: easiest, best quality, data leaves network. On-premise: full control, but GPU infrastructure and the developer tooling gap is larger than the model gap. Hybrid: route by sensitivity, becoming mainstream.

Build vs. Buy

Build Internally

• Maximum customization, full control
• Requires engineering team
• In our experience: prototype ~50 lines, production ~3,000 lines
• The gap: security, logging, error handling, access control, monitoring
• 6–12 month development cycle for production

Buy or Extend

• Faster time to value
• Vendor handles infrastructure
• Less customization, vendor lock-in risk
• Options: AI-native startups, incumbent platforms (Salesforce Agentforce), extend existing tools
• Evaluate with a framework (data integration, user experience, security, cost, extensibility)

A prototype is 50 lines — you can build one in an afternoon. Production is 3,000 lines: security, audit logging, error handling, access control, monitoring. The gap is real.

Governance

Financial Liability

• SOX compliance for public companies
• Audit trails for every AI-generated number
• Who signs off on AI-produced analysis?
• Board deck accuracy requirements

Privacy

• PII combination across systems
• GDPR right-to-erasure conflicts
• Data residency requirements
• Cross-border data transfers

Access Control

• Can the agent see data the user shouldn’t?
• Role-based filtering at the query level
• Prompt injection risks
• Audit who asked what

Operational Risk

• Inconsistent answers to same question
• Model version changes break workflows
• Audit logs and rollback procedures
• Human review gates for high-stakes output

Governance is the difference between a pilot and production. Four categories your graduates need to think about. These are the skills we should be teaching alongside the technical capabilities.

Building AI Systems

Let me pull back the curtain on how these systems are built. Two types: custom chatbots and custom agents.

Custom Chatbots

Architecture

• System prompt defines the chatbot’s behavior, domain knowledge, constraints
• Every user message sent to LLM along with system prompt + conversation history
• Web app hosted on corporate intranet
• Single API key → logging, cost tracking, access control

The System Prompt Is Everything

• Role: “You are XYZ Corp’s financial analyst”
• Knowledge: schema descriptions, business rules
• Constraints: “Never reveal raw salary data”
• Format: “Present financial data in tables with % change columns”
• The system prompt turns a generic LLM into a specialized tool

A custom chatbot is the simplest AI system to deploy. The XYZ Corp chatbot is essentially: system prompt (schema + rules) + Claude API + a web frontend + DuckDB for data. The intelligence comes from the system prompt and the model’s reasoning.

A chatbot is a system prompt plus an API. Every message goes to the LLM with the system prompt and conversation history. The system prompt is where institutional knowledge lives. The XYZ Corp chatbot is exactly this architecture.

Custom Agents

What Makes It an Agent

• Has tools — databases, Python, APIs, file systems
• Decides which tools to use based on the question
• Chains multiple tool calls before responding
• Different path for every question — not a script

The Claude Agent SDK

• Define tools as JSON Schema: name, description, input parameters
• Write a system prompt with domain expertise and strategy
• The agent loop: send message → LLM requests tool calls → execute tools → feed results back → repeat until done
• ~200 lines of application logic
• Complexity lives in the agent’s reasoning, not your code

An agent goes beyond a chatbot by having tools. Define tools as JSON Schema, write a system prompt with domain expertise, and implement the agent loop. About 200 lines of code. The complexity is in the agent’s reasoning, not the application logic.

Example: Portfolio Analyst Agent

Five tools, one agent, hundreds of different analyses

• get_holdings — retrieve portfolio positions with tax lot detail
• get_target_allocation — sector weight targets
• get_analyst_recommendations — Strong Buy stocks by sector
• run_sql — execute SQL against a live price database
• run_python — run Python for analytics (correlations, returns, statistics)

“Review the portfolio”

• Calls get_holdings
• Calls run_sql for current prices
• Calls run_python for market values
• Calls get_target_allocation
• Compares actual vs. target weights

“Harvest INTC losses and find a replacement”

• Identifies INTC lots with largest losses
• Calculates realized loss amount
• Checks if selling moves Tech below target
• Gets Strong Buy candidates for Technology
• Fetches prices for INTC + candidates
• Computes return correlations
• 7 tool calls, none predetermined

This is from the MGMT 675 course. Five tools, about 200 lines of code. The agent chooses its own path: “review the portfolio” triggers 4 tool calls, “harvest INTC losses” triggers 7 — across different tools, in different order, based on what the data shows.

AI+Code Lab

What It Is

• Shared Ubuntu VM at ai-lab.rice-business.org
• Each student gets an isolated workspace
• Left pane: file browser
• Right pane: terminal with Claude Code pre-launched
• Pre-provisioned API keys and skills
• Built for the Rice executive education program

What Students Build

• Exercise 1: 50-line data agent querying one database (~30 min)
• Exercise 2: Extend to query two systems and merge results (~30 min)
• Exercise 3: Wrap in a web interface (~45 min)
• Exercise 4: Red-team for security vulnerabilities (~30 min)
• All built from English instructions — Claude Code writes the code

Students build a working data agent from scratch using Claude Code — from a 50-line prototype to a multi-system agent with a web interface. The same progression that organizations follow: prototype → extend → deploy → secure.

The AI+Code Lab is where executive education students build agents hands-on. Each student gets an isolated workspace with Claude Code. They progress from a 50-line single-database agent to a multi-system agent with a web interface — the same progression organizations follow.

Docker: From Prototype to Production

Without Docker

• “It works on my machine”
• Install Python, pip, duckdb, anthropic…
• Manage conflicting library versions
• Secrets scattered across config files

With Docker

• Identical environment everywhere
• One command to build, one to run
• Dependencies frozen in the image
• Secrets injected at runtime, never baked in

A Docker container packages your agent, its dependencies, and its configuration into a single portable unit. The container provides an isolation boundary — agent code executes inside the container, not on the host. In production, companies deploy via orchestrators like Kubernetes or AWS ECS for scaling, restarts, and secret management.

Docker solves the “it works on my machine” problem. A container packages your agent with all dependencies into a portable unit. The container also provides security isolation. In production, Kubernetes handles scaling and restarts.

The Adoption Roadmap

🧪 Sandbox Prove value with a prototype on safe data

📋 Audit Add logging, access control, verification protocols

🚀 Deploy Production rollout with governance in place

Pre-Work (60–90 Days)

• Catalog enterprise systems and data sources
• Document schemas, business rules, data gotchas
• Secure API access and credentials
• Classify data sensitivity (PII, confidential, public)
• Identify 2–3 questions nobody can answer easily today

Signs You Should Wait

• No clear use case that justifies the investment
• Data quality issues that must be fixed first
• No executive sponsor with budget authority
• Regulatory uncertainty in your industry
• IT infrastructure not ready for API integration

Three phases. Sandbox: prototype on safe data. Audit: add enterprise infrastructure. Deploy: production with governance. Sometimes the right answer is to wait.

What This Means for Us

This is the world your graduates are entering. The skills they need: directing agents, verifying output, designing governance, and defending conclusions. Whether it’s a student checking a DCF model, a faculty member critiquing a paper draft, or a corporate team deploying an enterprise agent — the core skill is the same.

Session 1

What Students Can Do Now

• AI agents as analytical tools
• Silent errors as the real risk
• The checker’s toolkit

Session 2

How We Can Use AI

• Claude Code and Desktop
• Skills, MCP, Canvas
• Research workflows
• Assessment reform

Session 3

Corporate Implementation

• Enterprise data agents
• Custom chatbots and agents
• Deployment and governance

The thread connecting all three sessions: whether you’re a student, a faculty member, or a corporate team, the core skill is directing, verifying, and defending. The technology differs but the human skill is the same.

Discussion

What deployment challenges do you see in your students’ future industries? Where would AI agents add the most value to your research? What should we be teaching about governance and verification?