Phylo Runner

Architecture & workflow

How the pieces fit together, and what happens between uploading a dataset and viewing a result. Both diagrams are generated from an exact node/edge spec that mirrors the real services, ports, buckets and API routes in the repository.

Architecture

The services, their ports, and the trust boundary between them.

Architecture diagram. The browser (Next.js web UI on port 3000) talks only to the FastAPI API on port 8000. Inside the server's private network sit the API, the API worker, PostgreSQL on 5432, MinIO object storage on 9000/9001 with the phylo-datasets and phylo-artifacts buckets, and four method services: Dowser 8001, GCtree 8002, BrepPhylo 8003, ClonalTree 8004. The API reads run and file metadata from PostgreSQL and streams dataset and artifact bytes from MinIO to the browser. The worker claims queued runs from PostgreSQL, stages datasets from MinIO, and calls the method services over HTTP. Method services upload artifacts directly to MinIO with scoped credentials. MinIO is never exposed to the browser.
In scope: the runtime services, their ports, the two MinIO buckets, and the request/data edges between them — including that the browser reaches object storage only by streaming through the API. Omitted for clarity: auth providers, email/password-reset, the one-time MinIO bootstrap job, and the duplicated production-profile services (the web-prod/api-prod variants are the same shapes). See the repository README.md and docs/DEPLOYMENT.md for ops detail.

Workflow

From your side it is three steps: upload a dataset, choose a method, and collect the trees. This is what happens in between. The steps inside the dashed box run asynchronously in the worker (decoupled from your browser via a database-backed queue), so you can leave the page and the run continues.

Workflow flowchart, top to bottom. The browser calls POST /uploads/init, then PUT /uploads/{id}/content which streams the file to the MinIO phylo-datasets bucket, then POST /runs?method={id} which creates a run row with status QUEUED. Asynchronously, the worker claims the queued run via a PostgreSQL lease, stages the dataset from MinIO into a run workspace, calls POST {METHOD_URL}/jobs, waits while the method computes and uploads its artifacts to the MinIO phylo-artifacts bucket with a manifest, polls GET /jobs/{id}/status then GET /jobs/{id}/result, and ingests the manifest into run_artifacts rows in PostgreSQL, setting status to DONE. The browser then polls GET /runs/{id}/status, /result and /artifacts, and fetches each artifact via GET /artifacts/{id}/preview or /download, which the API streams from MinIO.
In scope: the successful upload → run → method job → artifact → fetch path, with the real routes and buckets. Omitted: error, cancellation and retry paths; resumable-upload session bookkeeping; and per-method internal stages. The upload and run creation are synchronous; everything in the dashed box is asynchronous and survives a page reload.