Colombo

01 / Context

The problem behind the project

A photographer should be able to send an image from a camera in the field without losing it when a network, storage service, or downstream CMS becomes unavailable.

Photographs often originate from equipment and networks that cannot participate in a modern browser upload flow. Cameras still use FTP, connections drop, and the newsroom CMS or storage can be temporarily unavailable. I wanted a gateway that accepts the equipment photographers already use while giving clients an honest, recoverable delivery contract.

An upload acknowledgement is therefore a durability boundary. It cannot mean the gateway merely received some bytes in memory, and it cannot imply that storage delivery and newsroom confirmation have already finished.

02 / Implementation

Design and implementation

I designed and implemented the upload gateway, including its Rust rewrite, durable acceptance boundary, and independent delivery retries.

I rewrote the gateway as one Rust process supporting FTP, explicit FTPS, and HTTP, with PostgreSQL tenant lookup and CMS-issued assignment/storage authority. Session credentials stay isolated even when multiple camera connections use the same username.

Before returning success, the service hashes and fsyncs the file and its private operation context into an atomically installed spool entry. Fixed workers recover accepted and interrupted entries after restart. Stable storage keys and persisted delivery state separate S3 work from CMS notification retries.

I also built durable receipts for mobile reconciliation, credential refresh within the upload budget, bounded retention, passive-port handling for camera clients, structured metrics, and local CMS/storage fixtures. The prior Java service remains available as a historical implementation.

03 / Architecture

Architecture and workflow

1. 01

   Resolve the assignment

   Identify the tenant and validate the supplied credential with its CMS, obtaining an assignment and short-lived storage authority.

2. 02

   Accept durably

   Hash and persist the file and operation context. Only then return FTP success or HTTP acceptance with an operation ID.

3. 03

   Deliver independently

   Upload to a stable S3 object and persist that result before attempting the newsroom callback.

4. 04

   Reconcile the receipt

   Expose accepted, uploading, delivered, callback-confirmed, failed, and expired states without returning storage secrets or internal paths.

5. 05

   Recover and retain

   Restart-safe workers resume eligible work; terminal receipts remain available for reconciliation and are later retained or expired explicitly.

04 / Approach

Engineering decisions

05 / Evidence

Evidence and outcomes

The repository contains protocol and container checks around durable acceptance, tenant isolation, callback retries, credential failures, receipt semantics, and real FTP/FTPS behavior. One compatibility fix retires an old passive endpoint before issuing a replacement, matching persistent camera behavior.

An operational investigation found that scanning 6,630 retained spool records every second consumed roughly 9.8–13% of one CPU core in the observed setup. The scheduler was changed to track pending work in memory and scan retained records once per minute. The regression checks the scheduling cadence; that investigation did not by itself prove the eventual production CPU reduction.

A stalled FTP transfer does not hold a shared session lock while waiting for its data socket. Disconnect cleanup for one camera therefore does not prevent another connection from receiving a greeting.

Source and technical notes