Event production companies have an awkward file problem. Several videographers shoot the same event, each ending up with files of 50–80 GB. They upload from venue wi-fi that drops out, or from a hotel room the next morning. Then the production team has to deliver everything to a client, who wants one link and, often, one zip.
We built StageStore for this. So far it has carried about 1,600 files, just over 1 TB, across 26 events. This post explains how it's put together and why, including the parts that didn't work the first time.
The design rule: files never pass through our servers
The most important decision is a simple one. The app never touches the file bytes. Our server decides who may upload or download; the browser then talks directly to object storage using short-lived presigned URLs.
That removes whole categories of problems:
- No request-size or duration limit on a 80 GB upload.
- No server bandwidth to pay for or saturate.
- Uploads can be split into parts and resumed, because the storage layer supports multipart uploads natively.
The stack
Everything runs on one serverless platform, under one account:
| Piece | Technology | Role |
|---|---|---|
| App | Next.js on serverless edge compute | Pages, API routes, access checks |
| Database | Serverless SQLite | Organizations, events, links, file records |
| Files | S3-compatible object storage | Uploads and built zips |
| Background jobs | Durable workflows | Building zips |
| Schedule | Cron triggers | Nightly storage measurement |
Two choices are worth explaining.
Storage with no download fees. In this business the same finished 100 GB gets downloaded repeatedly by clients, so per-gigabyte download charges would otherwise be the biggest line on the bill.
Raw SQL, no ORM. We started with Prisma and could not get its query engine to load reliably in the serverless runtime. The schema is small, so plain SQL through the native binding is simpler and more explicit anyway.
Uploads that survive bad wi-fi
Videographers open a link, with no account, and drop their files in. Underneath:
- Each file is an independent multipart upload, so parallel files don't block or break each other.
- The part size scales with the file size, so even very large files stay under the 10,000-part limit on a single multipart upload.
- The browser keeps the in-progress state locally (IndexedDB), so a refresh or crash restores the queue.
- On resume, the client asks storage which parts have already landed and uploads only the missing ones.
A dropped connection or closed laptop never restarts a file from zero. We think that's the single feature that matters most to the people shooting.
Zip downloads: the hard part
Clients want a single zip. A zip of 100 GB of video can't be built in a single request, and it can't be built on one machine in a reasonable time. Here is the approach.
Pre-compute the layout
Video and photos don't compress, so entries are stored, not deflated. That means every byte position in the final archive is known before any data is read. We use ZIP64 throughout, because files and archives well over 4 GB are normal here.
The only value not known up front is each file's CRC-32 checksum.
Split the work, combine the checksums
The archive is written as a multipart upload of 40 MiB parts. A coordinator splits those parts into blocks, and up to 64 parallel workers each build a block, reading source ranges from storage, computing CRC slices, and uploading parts.
A file may be split across workers, so no single worker sees all of its bytes. Slice checksums are combined with the standard crc32_combine algorithm from zlib, and the result is one correct CRC for the file. The few parts that depend on final checksums, such as the data descriptors and the central directory, are written last by separate "fixer" steps once every worker is done.
We wrote the ZIP64 writer and CRC code by hand, with no library. The result was checked against unzip, Python's zipfile and macOS Archive Utility, and it's byte-identical whether built by 1, 8 or 64 workers. A roughly 93 GB event builds in about 3–10 minutes.
Reliability, learned in production
Building archives this large exposed things you only see at scale:
- Reads that hang halfway through an object. Fix: read in bounded windows, shrink the window after a stall, and retry until there's no progress.
- Hung writes. Fix: a stall watchdog on every part.
- Instances that die silently. Fix: the coordinator checks the status of its own workers instead of trusting them.
- A deploy in the middle of a build. Fix: the plan is stored with the job, so a running build never changes under it.
- A damaged source file. After repeated failures, it's flagged, left out, and the zip is rebuilt without it. The client sees which file was skipped, and the team knows to ask for a re-upload.
The principle behind all of it: failures must be visible, never silent. A client should never receive a zip that quietly contains less than they think.
Access without accounts
Videographers and clients use token links, with no sign-up. The team signs in with a WhatsApp code, which suits how teams in the UAE already work, with email and password as an option. Every organization's data is scoped by tenant, and each organization's own logo appears on its upload and gallery pages.
Public signup is protected by a bot check and WhatsApp code verification.
What we'd tell another team
- Keep bytes off your app server. Presigned URLs and multipart uploads solve most of the large-file problem on their own.
- Make everything resumable and idempotent. Networks fail, and so do workers. Design for retries from the start.
- Don't trust a single read. At hundreds of gigabytes, rare storage hiccups stop being rare.
- Prefer visible failure. Flag the bad file, say so, and carry on.
StageStore is built by SolidStack in the UAE for event production teams. If you'd like to try it, see files.solidstack.ae. If you have a similar problem, such as large files, long-running jobs or cloud architecture, get in touch.