Lubili
Engineering work

Deployment platform for web apps

Point it at a GitHub repository and it builds the app in an isolated container, stores the logs and serves the site on its own subdomain.

Designed and built by Arya Shrimali, founder of Lubili
Type
Deployment and build platform
Role
Designed and built end to end
Stack
TypeScript, Fastify, AWS ECS Fargate, Docker, Apache Kafka (KafkaJS), ClickHouse, PostgreSQL with Prisma, Amazon S3, http-proxy, Socket.IO, Zod
Deployment platform overviewA GitHub repository is built in its own container. The output goes to S3, and a reverse proxy serves it to visitors by subdomain. Build logs branch off through Kafka into ClickHouse.GitHub repoSourceBuild containerECS FargateAmazon S3OutputReverse proxyBy subdomainVisitorBrowserKafkaLog linesClickHouseLog storelogs
The flow in one picture. The full architecture is further down the page.Scroll sideways or pinch to zoom

Context

What the deployment platform does

You give the platform a GitHub repository. It builds the frontend app in an isolated container, stores the build logs, uploads the output to S3 and serves the site on its own subdomain.

It follows the same core idea as Vercel or Netlify.

Why it matters for a business

It shows the skills behind any system that runs untrusted or long-running work: isolated build environments, event streaming, log storage that can be queried, and routing many customer sites through one entry point.

What Arya built

Inside the deployment platform

  • An API server (Fastify)

    POST /api/project validates the GitHub URL with Zod and assigns a random subdomain slug. POST /api/deploy refuses a second deployment while one is queued or in progress (HTTP 409), records the deployment as QUEUED, launches a build task and returns HTTP 202.

  • Isolated builds on AWS ECS Fargate

    Each deployment launches its own container task, with per-deployment settings passed as environment overrides.

  • A build container (Docker, Node 22)

    It clones the repository, runs npm install and npm run build, uploads every file in the build output folder to S3 with the correct content type, and publishes the build output to Kafka as it runs.

  • A log pipeline

    Log messages go to a Kafka topic over SASL and TLS. A batch consumer in the API server writes them to ClickHouse, committing each offset only after the insert and sending heartbeats during long batches.

  • Log history

    GET /api/logs/:deployment_id reads a deployment's logs from ClickHouse with a parameterised query.

  • A reverse proxy (Fastify and http-proxy)

    It reads the subdomain from the request host, looks up the project in PostgreSQL and proxies the request to that project's files in S3, serving index.html for the root path.

  • Data

    PostgreSQL through Prisma for projects and deployments, with a deployment status model.

  • Socket.IO

    Runs alongside the API server, with a channel per deployment. Channel subscriptions are prepared for live log streaming.

Engineering

Key decisions

  1. One container per build.

    A broken or hostile build cannot affect other projects or the API.

  2. Kafka between the builder and the database.

    Build containers never write to the log store directly, and logs wait in Kafka if the consumer restarts.

  3. ClickHouse for logs.

    A columnar store suited to large, append-only event data and time-range queries.

  4. Manual offset commits.

    A log message is only marked as done after it is stored.

  5. A concurrency guard on deploy.

    Double-clicking deploy cannot start two builds.

Results

What it demonstrates

  • Building and running customer code safely in isolated environments.
  • Streaming events between services without losing them.
  • Storing large volumes of logs or events and querying them later.
  • Serving many tenants from one entry point by subdomain.

Architecture

The full system

For engineering readers. Everything in the drawing is described in the text above.

Deployment platform architectureA developer submits a GitHub repository to a Fastify API server. POST /api/project validates the URL with Zod and assigns a subdomain slug. POST /api/deploy returns 409 if a deployment is already active, records it as QUEUED, returns 202 and launches an ECS Fargate task. The build container clones the repository, runs npm install and npm run build, uploads the build output to S3 and publishes its log lines to a Kafka topic over SASL and TLS. A KafkaJS batch consumer in the API server inserts them into ClickHouse with manual offset commits. GET /api/logs/:deployment_id reads them back. Projects and deployments live in PostgreSQL through Prisma. Socket.IO accepts per-deployment channel subscriptions, prepared for live log streaming. Visitors open a project's subdomain; a Fastify and http-proxy reverse proxy looks the project up in PostgreSQL and serves its files from S3, with the root path serving index.html.DeveloperBrowserSubmits a GitHub repoSocket.IOIn the API serverChannel per deploymentSubscriptions preparedfor live log streamingPostgreSQLvia PrismaProjects, deploymentsStatus modelVisitorBrowserOpens <slug> subdomainsubscriptionsAPI SERVERFastifyPOST /api/projectCreate projectZod-validated GitHub URLAssigns a subdomain slugPOST /api/deployQueue a build409 if a deploy is activeRecords QUEUED, returns 202Launches a build taskLog consumerKafkaJS batchManual offset commitsHeartbeatsPrismaKafkaSASL over TLSTopic: container-logsBuffered log messagesClickHouseLog storeTable: log_eventsParameterised queriesRead by GET /api/logsconsumeinsertAWSECS RunTaskFargatePer-deploy environmentoverridesBuild containernode:22-slimgit clonenpm install, npm run buildUpload dist with content typesStreams build outputAmazon S3Build outputOne prefix per projectRunTasklog linesREVERSE PROXYFastify + http-proxyHost subdomainread from HostProject lookupPostgreSQLS3 prefixproject filesRoot path/ serves index.htmlsubdomainlookupserve files
Builds run in their own container, logs move through Kafka into ClickHouse, and one proxy serves every project by subdomain.Scroll sideways or pinch to zoom

The API server queues a build on ECS Fargate, the build container streams its output through Kafka into ClickHouse, and a reverse proxy serves each project from S3 by subdomain. The Socket.IO link is dashed because live streaming to the browser is prepared, not yet wired.

Stack

Technology

  • TypeScript
  • Fastify
  • AWS ECS Fargate
  • Docker
  • Apache Kafka (KafkaJS)
  • ClickHouse
  • PostgreSQL with Prisma
  • Amazon S3
  • http-proxy
  • Socket.IO
  • Zod

For your business

Similar problems Lubili can solve

CI and build pipelines, background job systems, event and log pipelines, multi-tenant platforms, and internal developer tools.

Need background jobs, pipelines or infrastructure that does not lose data?

Tell us what has to run reliably and what happens today when it fails. Every message gets a personal reply.

Book a free consultation (opens in a new tab)Tell us about it

Free. No obligation. You get a one-page written summary.