A bucket, a CDN, and 37 gigabytes of regret

DateTime.AfterPost - DateTime.Now = 6 minutes

A while back I built a Raspberry Pi digital frame that pulls from Google Photos. It broke as rsync stopped working with Google Images but

what mostly stopped working was my patience with paying Google every month to store pictures I look at twice a year.

So I decided to build my own gallery on AWS. Drop files in a folder, run one command, get a proper gallery behind a login.

It costs me about a dollar sixty a month. It also cost me 37 gigabytes of family videos, for about six hours, which I’ll get to.

The idea

The whole thing is four moving parts:

– S3 holds the photos and videos

– CloudFront serves them, and checks you’re allowed in

– Cognito handles the login (with passkeys — Face ID, no password)

– Two Lambdas: one makes thumbnails when you upload, one handles clicks in the gallery

No server. Nothing running when nobody’s looking at it. The bill is basically storage and that’s why it’s so cheap.

I did this “pair-programming” (or vibe coding as the young call it) with Claude Code, which is relevant to the story, so I’ll be honest about which parts it got right and which parts it got very wrong.

Things I learned that I didn't expect

Your file names collide more than you think

My first instinct was to dump everything flat in the bucket: media/IMG_4898.MOV, done.

Then we counted. Across 5,439 recovered files, **1,342 names appeared more than once**. Worse, **677 of those were genuinely different files that happened to share a name** — two different `IMG_4898.MOV`, from two different cameras, years apart.

A flat layout would have silently overwritten them. No errors. You’d just find out in five years when the wrong video plays.

So the key became the file’s own SHA-256:

media/3f9a2b8c1d4e5f60.mp4

Two nice side effects. Identical copies collapse automatically – 1,595 duplicates in my library uploaded exactly once. And because the key never changes, CloudFront can cache it forever.

The real filename lives in a little JSON file next to it.

CloudFront can check your login without running any code

This was the bit I liked most.

You sign in, and a Lambda hands your browser a signed cookie. After that, CloudFront validates that signature *itself*, at the edge, for every photo you load. No Lambda runs. No compute cost. Browsing 3,000 photos costs nothing but bandwidth.

Even better: I put the write API behind the *same* check. Which means the Lambda that handles “archive this” and “move to that category” contains zero authentication code. It can’t be reached without a valid session, so it doesn’t need to check for one.

Tags are not a database

I wanted to store categories as S3 object tags. Felt clean.

S3 says no, in three ways: maximum 10 tags per object, no structured values, and list_objects_v2 doesn’t return tags – so reading them in bulk costs one API call per object anyway.

What tags *are* good for is telling S3 itself to do something. A lifecycle rule can say “anything tagged archived=true moves to Glacier”. It cannot read a JSON file. So archiving writes both: a tag S3 acts on, and a JSON field the gallery reads.

8.5 hours of MPEG-2

Half my library was digitised camcorder footage — old .mpg files, 768×576 PAL.

Sadly, no browser plays MPEG-2. There’s no plugin, no workaround, no “enable this flag”. The file is just not web content.

So a local script transcodes anything unplayable to H.264 before upload, using the hardware encoder on my Mac:

Movie 1.mpg 9142 MB → Movie 1.mp4 4444 MB in 207s

Half the size, and it actually plays. 8.5 hours of footage took about twenty minutes.

Now the part where I lost everything

About three hours in, the gallery was working and I asked Claude to deploy the frontend. It wrote a deploy script containing this line:

aws s3 sync ./site s3://my-bucket/ --delete

--delete means “make the destination match the source.” The source was a folder with three files. The destination was the bucket with all my media in it.

S3 did exactly what it was told. 37.5 gigabytes, gone in about forty seconds.

Versioning wasn’t enabled. There’s no trash can in S3 – a delete on a non-versioned bucket is immediate and permanent, and no support ticket brings it back. I checked every angle: no backup vault, no replication, no cached copies at the CDN edge. Nothing.

I got lucky. I still had the originals on a drive, and spent the evening recovering 95 GB and re-uploading. But for about six hours I genuinely thought I’d deleted my grandmother’s voice recordings.

Three things came out of that:

Versioning is on now. It should have been on before there was a single byte in that bucket.
The deploy script no longer has a delete path at all. It copies three named files. There is no pruning step and there never will be.
The API’s IAM role has no s3:DeleteObject permission. Not “we’re careful not to call delete” – it literally cannot. The thumbnail Lambda can only delete thumbnails, and only in the thumbnails folder.

That last one is the lesson I’d actually hand to someone else. Don’t rely on your code being careful. Make the dangerous thing *impossible* at the permissions layer, because code changes and permissions are checked every single call.

I always saw posts in Reddit (example1, example2, example3) about people complaining “it happened to them” and I, carelessly using auto-mode in Claude Code, was sure it was gonna happen to me some day as well. I thought it was gonna be the repository or some temporary folder with nonsense, but it was actually the bucket with my images and video…

Bugs that don't announce themselves

A few of these cost me more time than the whole architecture did.

CloudFront checks signatures before it runs your functions. I had a tiny edge function meant to redirect signed-out visitors to the login page. It never ran. CloudFront validated the (missing) signature first and returned raw XML. I’d assumed the opposite order and had to test it to find out.

Every write returned 403 and it wasn’t an auth problem. CloudFront signs requests to a Lambda URL, and Lambda refuses unsigned payloads – so the *browser* has to compute the SHA-256 of its own request body and send it in a header. Buried in the docs. Obvious in hindsight. POST with a body failed, POST without one worked, which is exactly the clue I ignored for a while.

My thumbnail pipeline was quadratic and I didn’t notice. Each upload triggered a Lambda, and each Lambda rebuilt the gallery index by reading *every* file’s metadata. Fine for ten photos. For 3,500 it’s about six million API calls. I’d “fixed” the resulting slowness by capping concurrency at 2, which made it dramatically worse – 15,000 throttled events in two hours, and a third of my photos silently never got thumbnails.

The fix was to stop rebuilding the index on every upload and do it once at the end. Obvious once you see it. Invisible while it’s happening, because nothing errors – the work just quietly doesn’t finish.

Was the AI worth it?

Yes.

I wrote a whole post about working with AI and this project was a good test of it.

What it was genuinely great at: writing the Terraform, remembering that S3 tag values reject parentheses but accept Hebrew, spotting that 677 of my filenames collided before I hit the problem, and transcoding decisions I’d have had to look up.

What it got wrong: it wrote the line that deleted my bucket. It also put the Terraform state file *inside the bucket the CDN was serving, unauthenticated* – which meant my infrastructure state was publicly downloadable by anyone who guessed the URL. I only caught that because I ran a second, separate security review over the whole thing.

That’s the honest summary. It moved about ten times faster than I would have, and it made two mistakes that a careful human probably wouldn’t. Both were recoverable, one barely.

Double honesty – It’s not “ten times faster”, it’s “to be or not to be”. I wouldn’t even begin this project and just throw things into the bucket and that’s it. Now I have a beautiful gallery with my images and videos and I don’t have to choose if I should upgrade or not my Google subscription.

It also caught its own mistakes when asked to look, which is the part I’d emphasise. The second value isn’t “it writes code.” The value is that reviewing is cheap now, so you can afford to review everything, twice, from different angles.

The cost

For about 66 GB of photos and video:

What	Price
S3 storage (66 GB)	~$1.53
Glacier (cold backups)	~$0.08
CloudFront, Lambda, Cognito	$0 (free tier)
Total	~$1.60/month

Cognito is free up to 10,000 monthly users, which my family is unlikely to trouble. CloudFront gives you 1 TB of transfer a month free, forever. The Lambdas run for a few seconds per upload.

The thumbnails deserve a mention: 3,600 of them take up 82 MB total against 66 GB of originals. Opening the gallery downloads the 82 MB, not the 66 GB. That single decision is the difference between “works on mobile data” and “don’t”.

Get it

It’s all on GitHub, MIT licensed: github.com/javitolin/aws-gallery

Everything specific to my setup is in one gitignored config file, so you can point it at your own bucket and domain. There’s a CLAUDE.md in there with the architecture and every gotcha above written down properly, which is useful whether or not you use AI.