Distributed API Gateway

NovaHub: Curated Data for Every Application

Your CRM, HIS, EMR, and AI agents all need the same thing -- curated, resolved, intelligence-ready patient data in real time. NovaHub delivers it. Register a dataset, get an API. No custom code, no integration projects. Real-time data from DataCloud to any application.

Real-Time
Curated data delivered to CRM, HIS, EMR, and AI agents
Zero Code
Register a dataset, get an API -- no custom integrations
Zero Downtime
Atomic updates -- old data serves until new data is ready
Any App
Standard HTTP and gRPC -- works with any consumer

Publisher-Subscriber Data Flow

DataLake publishes datasets via Arrow Flight gRPC. NovaHub subscribes, builds optimized shards, and atomically swaps them in. The old data serves requests until the new data is ready. Zero downtime.

Publisher
DataLake
Golden datasets & schedules
Arrow Flight
gRPC streaming transport
Scheduler
Job Scheduler
Cron & interval triggers
Columnar Builder
High-throughput processing
Shard Builder
Partition pruning (skip irrelevant data segments)
Atomic Swap Layer
Atomic Swap
Old data served during build
Crash Recovery
Old data intact on failure
Instant Cutover
Zero-downtime promotion
API Server (Horizontally Scaled)
Async API Server
Python
Storage Reader
Purpose-built for fast reads
Identity Resolution
Surrogate -> MDM IDs
Inline Filters
Multi-type, multi-operator
Consumers
CRM Frontend
Patient 360 views
PEP + CRM
Engagement & operations
Analytics
OLAP engine
AI Agents
LLM tool calls

Core Capabilities

We optimized for three things: throughput, correctness, and operational simplicity. In that order.

GP

Generic Path-Based API

One route pattern: GET /api/v1/{dataset}/{query_key}/{value}. It dynamically routes to any configured dataset. Register a new dataset in DataLake and the endpoint appears. No code change.

FK

Identity Resolution Engine

Hospital sends a UHID, we resolve it to the golden MDM ID on the fly. Multi-level identity resolution means one identity dataset powers every endpoint.

FL

Inline Filtering Engine

Rich operators across Boolean, Number, String, Date, and Timestamp types. Supports transforms like lower, upper, floor, ceil, year, month, and more -- all type-safe.

AI

Columnar Serialization

Purpose-built columnar format optimized for clinical datasets with nested structures. Dramatically faster writes and smaller storage footprint compared to JSON -- benchmarked on production data, not synthetic tests.

AF

Arrow Flight Streaming

Incremental batch streaming for datasets exceeding 2 GB. Configurable batch sizes keep memory bounded while maximizing throughput over gRPC.

AU

Zero-Downtime Atomic Updates

During a data rebuild, the old version stays live. If the build crashes, old data is intact. If it succeeds, the swap is instant. The API never serves partial or inconsistent data.

TP

Two-Phase Architecture

API Server runs as multiple replicas for horizontal scale. Scheduler runs as a single instance for coordination. Clean separation of concerns with independent scaling.

CF

Config-Driven Extensibility

Register a new dataset in DataLake. NovaHub auto-syncs it, generates the route, starts serving. No code changes, no deployments. This is how we ship new data products in hours.

Engineering Deep Dive

For technical teams evaluating NovaHub. Measured on production workloads with real clinical data.

Columnar vs JSON Serialization

Measured on a 115,906-record clinical dataset with nested structures.

Write Speed
147s2s
70.4x faster
Storage Size
1,340 MB656 MB
51% smaller
Throughput
788 rec/s55,490 rec/s
70.4x

Shard Build Performance

8-shard dataset with partition pruning (skip irrelevant data segments) enabled.

Full Build Time
19.7 min16.7 sec
70.8x faster
Shard Filtering
Full scanPartition prune
5-10x faster
Memory Efficiency
BaselineOptimized
8x lower

One Route, Every Dataset

We wrote hundreds of hand-coded routes before we figured this out. One generic endpoint pattern handles them all.

HTTP Route Pattern
GET /api/v1/{dataset}/{query_key}/{value}

The path parameters are fully dynamic. When a new dataset is registered in DataLake, NovaHub automatically creates the data store, generates the FK resolution mappings, and starts serving the route -- no code change, no redeployment.

/api/v1/patient_360/uhid/P-10042

Full patient profile by hospital UHID

/api/v1/lab_results/mrn/MRN-88210

Lab history by medical record number

/api/v1/care_gaps/mdm_id/MDM-5519

Open care gaps by golden MDM identity

Inline Filtering -- 24+ Operators
GET /api/v1/lab_results/uhid/P-10042?filter=test_date.gte.2024-01-01&filter=result_value.between.80,120&filter=test_name.ilike.%glucose%
Boolean
eq, neq
Number
eq, gt, gte, lt, lte, between
String
eq, like, ilike, in, startswith
Date
eq, gt, gte, lt, lte, between
Timestamp
eq, gt, gte, lt, lte, between

Identity Resolution Engine

A patient has a UHID at one hospital, an MRN at another, a phone number in the CRM. NovaHub resolves all of them to one golden identity in a single hop.

1
Inbound Key
Consumer sends hospital-specific UHID: P-10042
2
Surrogate Lookup
Storage engine resolves UHID -> internal surrogate key SK-7831
3
MDM Resolution
Surrogate key maps to golden MDM ID: MDM-5519
4
Data Fetch
Storage engine retrieves all records linked to MDM-5519 instantly
5
Response
Unified patient record returned in real time

Technology Stack

Boring choices that work. We picked for latency and operational simplicity, not resume-driven development.

PY
Python / Async Framework
Async API server with auto-generated OpenAPI docs
SE
Storage Engine
Purpose-built data store -- optimized for fast reads
CS
Columnar Serialization
Dramatically faster writes and smaller storage than JSON
AF
Arrow Flight
gRPC client/server for high-speed batch streaming
OL
OLAP Engine
In-process analytical engine for ad hoc queries and shard filtering
JS
Job Scheduler
Cron and interval scheduling for data sync jobs
DK
Docker
Container-per-replica deployment with resource limits
K8
Kubernetes
Horizontal pod autoscaling for API server replicas

Also available as a product

NovaHub Powers Data Intelligence

Use NovaHub's real-time API layer to serve curated clinical data to your existing CRM, HIS, or EMR -- without adopting the full THB platform.

Explore Data Intelligence

Curated Data for Any Application. Zero Custom Code.

Register a dataset. Get an API endpoint. Deliver intelligence-ready patient data to your CRM, HIS, or AI agents. Come see it running.