ES
← Back
University courseworkData Architecture — Final project (URL, June 2026)Group project

ACME EV — real-time telemetry for an electric-vehicle fleet

Kafka (Confluent) with Schema Registry and DLQ, a live dashboard on Vercel and a Gold layer in Databricks

Problem

Ingest and exploit telemetry from a fleet of up to 10,000 electric vehicles across 10 Latin American branches (GPS every 30 s and status every 60 s), guaranteeing availability, integrity and confidentiality, with 20% annual growth.

Data

Telemetry simulated in Python (GPS position, battery, on/off, fault code 000–999, km) over real street routes (Google Routes / OSRM), a dimensional catalog of branches, vehicles and customers (synthetic randomuser.me identities) and a Gold layer in Databricks.

Methods

  • Python generator with vehicle physics (haversine, bearing, variable speed), battery and offline/reconnect logic with buffered frames released in a burst
  • Publishing to Kafka with JSON Schema validated in Schema Registry; invalid messages go to a dead-letter topic (DLQ)
  • Serverless function on Vercel as a Kafka → browser bridge (kafkajs) with an incremental cursor
  • Web dashboard (HTML/JS, vector WebGL Google Maps) with marker clustering and per-branch routes loaded on demand
  • Gold layer in Databricks: Delta tables + an enriched view joining Bronze telemetry with the catalog
  • Scaled from 300 to 10,000 vehicles by optimising only the client, without rewriting the state logic
  • 33 pytest tests for the generator; a 10-page management/technical report and a cloud efficiency and cost annex

Results

  • A 10,000-vehicle fleet ≈ 500 sustained events/s and 43.2 M events per day (arithmetic from the brief, documented as a «hard figure» in the annex; RU/s and cost figures are declared as estimates, not measurements).
  • Dashboard deployed and running in production (Vercel), scaled to 10,000 vehicles: routes split per branch (~10 MB each) loaded on demand, plus marker clustering.
  • Verified fix: 0 vehicles drawn over the sea after regenerating real street routes and discarding water-crossing ones.
  • 33 generator tests passing (pytest).

Gallery

Tools

PythonConfluent Cloud (Kafka)Schema RegistryDatabricks (Delta)VercelJavaScriptGoogle Maps PlatformOSRMpytest

Links

Demo

University work; no claim of sole individual authorship.