Project Guide | Nexus Hive

## <h1 align="center">Nexus-Hive</h1>

<p align="center"> <strong>Multi-agent NL-to-SQL copilot with governed analytics, audit trails, and multi-warehouse support</strong> </p>

<p align="center"> <a href="https://github.com/KIM3310/Nexus-Hive/actions/workflows/ci.yml"><img src="https://github.com/KIM3310/Nexus-Hive/actions/workflows/ci.yml/badge.svg" alt="CI"></a> <a href="https://codecov.io/gh/KIM3310/Nexus-Hive"><img src="https://codecov.io/gh/KIM3310/Nexus-Hive/branch/main/graph/badge.svg" alt="codecov"></a> <a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.11%2B-blue.svg" alt="Python 3.11+"></a> <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-green.svg" alt="License: MIT"></a> <a href="https://fastapi.tiangolo.com"><img src="https://img.shields.io/badge/FastAPI-0.115+-009688.svg" alt="FastAPI"></a> </p>

---

Nexus-Hive turns natural-language business questions into audited SQL, executes them safely against a warehouse, and returns chart-ready answers with a full agent trace. Every step -- translation, policy check, execution, visualization -- is independently inspectable, testable, and governed.

Why this matters: Most NL-to-SQL tools generate SQL and run it. Nexus-Hive inserts a policy engine, audit trail, and retry loop between generation and execution -- making it safe for environments where data access requires governance.

---

System Overview

Evaluation Path

• Start here: Ask one seeded business question, inspect the SQL policy decision, then open the agent trace.
• Local demo: Run make run, then open http://localhost:8000 or http://localhost:8000/docs.
• Checks: Run make verify; it covers lint, pytest, and smoke testing against the live local server.

---

Architecture Notes

• Architecture guide summarizes the system scope, first files to inspect, runtime commands, and known boundaries.
• Quality notes lists the local checks, CI surface, and release expectations for this repository.
• Enterprise readiness notes outlines security, data, operations, integration, and handoff expectations.

Architecture

flowchart TB
    User["Business User / Analyst"]
    User -->|"Natural language question"| API["FastAPI Runtime\n/api/ask"]

    subgraph AgentPipeline["LangGraph Multi-Agent Pipeline"]
        direction TB
        T["Agent 1: Translator\nNL --> SQL via Ollama/LLM\n+ heuristic fallback"]
        E["Agent 2: Executor\nPolicy engine + read-only SQL execution"]
        V["Agent 3: Visualizer\nChart.js config generation"]
        T --> E
        E -->|"error + retry budget > 0"| T
        E -->|"success"| V
    end

    API --> AgentPipeline

    subgraph Governance["Governance Layer"]
        direction LR
        PE["Policy Engine\nDeny / Review / Allow"]
        AT["Audit Trail\nJSONL + query tags"]
        GE["Gold Eval Suite\nSQL quality scoring"]
    end

    E --- PE
    E --- AT

    subgraph Warehouses["Warehouse Adapters"]
        direction LR
        SQLite["SQLite\n(demo, active by default)"]
        SF["Snowflake\n(live when configured)"]
        DB["Databricks\n(live when configured)"]
    end

    E --> Warehouses
    V --> UI["Frontend\nChart.js + agent trace viewer"]

The pipeline is built with LangGraph as a compiled state graph. Each agent is a node with typed state (AgentState), connected by edges with conditional routing. When the Executor rejects SQL (policy denial or execution error), the graph routes back to the Translator for correction -- up to 3 retries -- without re-running the Visualizer.

---

Local (3 commands)

git clone https://github.com/KIM3310/Nexus-Hive.git && cd Nexus-Hive
make install                 # creates venv, installs deps
make seed                    # seeds 10k enterprise sales records into SQLite
make run                     # starts uvicorn on http://localhost:8000


## If your default python3 is older than 3.11:
make BOOTSTRAP_PYTHON=/path/to/python3.11 install

Open http://localhost:8000 for the frontend, or http://localhost:8000/docs for Swagger UI.

Docker

docker compose up app

## With live LLM inference via Ollama:
docker compose --profile with-ollama up
docker exec nexus-hive-ollama ollama pull phi3

Verify Everything Works

make verify   # runs lint + pytest + smoke test against a live server

---

Tech Stack

---

Governance Features

---

Core API

## Ask a question (returns stream URL for agent trace)
curl -X POST http://localhost:8000/api/ask \
  -H "Content-Type: application/json" \
  -d '{"question": "Show total net revenue by region"}'


## Stream agent trace (SSE)
curl -N "http://localhost:8000/api/stream?q=Show+total+net+revenue+by+region&rid=REQUEST_ID"


## Policy check -- preview SQL before execution
curl -X POST http://localhost:8000/api/policy/check \
  -H "Content-Type: application/json" \
  -d '{"sql": "SELECT region_name, SUM(net_revenue) FROM sales GROUP BY region_name", "role": "analyst"}'


## Runtime metadata
curl http://localhost:8000/api/meta


## Run gold eval suite
curl http://localhost:8000/api/evals/nl2sql-gold/run

---

Test Results and Benchmarks

From pytest 8.3.5 on Python 3.11 -- make verify (lint + test + smoke)

Endpoint Response Times (local SQLite)

---

Snowflake

pip install -e ".[snowflake]"
export SNOWFLAKE_ACCOUNT=your_account.us-east-1
export SNOWFLAKE_USER=your_username
export SNOWFLAKE_PASSWORD=your_password
export SNOWFLAKE_DATABASE=ANALYTICS

Databricks

pip install -e ".[databricks]"
export DATABRICKS_HOST=https://your-workspace.cloud.databricks.com
export DATABRICKS_AUTH_TYPE=databricks-cli
export DATABRICKS_WAREHOUSE_ID=your_sql_warehouse_id
python scripts/bootstrap_databricks_demo.py

---

Deployment

Kubernetes

kubectl apply -f infra/k8s/namespace.yaml
kubectl apply -f infra/k8s/

GCP Cloud Run -- Terraform configs in infra/terraform/. See infra/terraform/README.md for variables and outputs.

Docker

docker compose up app

---

Enterprise Deployment

Production-grade overlays for regulated and high-scale environments.

Helm chart

A full Helm chart lives at helm/nexus-hive/ with three value overlays:

## Default / dev-staging
helm upgrade --install nexus-hive helm/nexus-hive \
  -n analytics --create-namespace


## Production: HPA, PDB, NetworkPolicy, ServiceMonitor, external secrets, HA spread
helm upgrade --install nexus-hive helm/nexus-hive \
  -n analytics --create-namespace \
  -f helm/nexus-hive/values.yaml \
  -f helm/nexus-hive/values-prod.yaml \
  --set image.tag=0.2.0


## Air-gapped: Ollama sidecar, no external LLM, internal registry only
helm upgrade --install nexus-hive helm/nexus-hive \
  -n analytics --create-namespace \
  -f helm/nexus-hive/values.yaml \
  -f helm/nexus-hive/values-airgap.yaml

See helm/nexus-hive/README.md for the full values reference.

Kubernetes production baseline

infra/k8s/production/ ships a hardened namespace baseline: Pod Security Standards (restricted), ResourceQuota, LimitRange, default-deny NetworkPolicy, and scoped allow policies for ingress and warehouse egress.

infra/k8s/observability/ ships a ServiceMonitor and PrometheusRule set with SLO-grade alerts (error rate, p99 latency, circuit breaker, audit-write stall).

Terraform for GCP Cloud Run

infra/terraform/ provisions Cloud Run v2 with Secret Manager, Workload Identity, VPC connector, uptime checks, and log-based alerting. See infra/terraform/README.md for the variable and output tables.

Runbooks

Five step-by-step runbooks live at docs/runbooks/:

---

Composite Discovery Narratives

Narrative discovery handouts for technical review conversations. They are fictional composites, not customer references, production evidence, or outcome claims.

Use these narratives to frame questions and implementation boundaries. Do not present them as live customer deployments, benchmarks, references, or guaranteed business results.

---

Benchmarks

benchmarks/ contains reproducible performance and accuracy tests.

question mix, thresholds gated on p95/p99 latency and checks pass rate.

and renders a per-case accuracy chart.

• benchmarks/load_test.js - k6 script, 50 VUs x 5 min with governed
• benchmarks/agent_accuracy_eval.py - runs the gold eval suite N times
• benchmarks/sample_results.json - realistic run output.

Sample run (from sample_results.json)

See benchmarks/README.md for the full usage and CI integration recipe.

---

Project Structure

Nexus-Hive/
  main.py                    # FastAPI entrypoint (thin, delegates to modules)
  config.py                  # Centralized config, constants, metric definitions
  graph/
    nodes.py                 # LangGraph agent nodes + graph builder
  policy/
    engine.py                # Policy evaluation, query tags, heuristic inference
    audit.py                 # Audit trail writer
    governance.py            # Governance surfaces (scorecard, warehouse brief)
  routes/
    ask.py                   # /api/ask + /api/stream (SSE agent trace)
    health_meta.py           # /health, /api/meta
    warehouse.py             # Warehouse adapter endpoints
    auth.py                  # Operator auth session endpoints
  services/
    build_helpers.py         # Runtime brief, meta builders
    streaming.py             # SSE streaming for agent trace
    openai_helpers.py        # Optional OpenAI integration
  warehouse_adapter.py       # Adapter base class + SQLite + registry
  snowflake_adapter.py       # Live Snowflake adapter
  databricks_adapter.py      # Live Databricks adapter
  circuit_breaker.py         # Ollama circuit breaker
  security.py                # HMAC sessions, token auth, RBAC
  seed_db.py                 # 10k-row synthetic dataset generator
  tests/                     # 8 test modules, 80+ test cases
  infra/
    k8s/                     # Kubernetes manifests
    terraform/               # GCP Cloud Run Terraform configs
  frontend/                  # Chart.js + agent trace viewer
  docs/
    adr/                     # Architecture Decision Records
    solution-architecture.md
    discovery-guide.md

---

Related Projects

---

Cloud + AI Architecture

• Cloud + AI architecture blueprint
• Machine-readable architecture manifest
• Validation command: python3 scripts/validate_architecture_blueprint.py

Enterprise Productization

• Product operating model defines the product scope, trust boundary, operating checks, and service path for this repository.

System Architecture

• System architecture maps the runtime boundary, data/control flow, cloud or local deployment surface, and operating assumptions for this repository.

Service Architecture

• Service architecture defines the cloud resources, account information, cost controls, and production guardrails needed to turn this repo into a scoped service without publishing public financial assumptions.