unrelenting.technology

New image upload/optimization for sweetroll2

Website update: imgroll image optimization has been deployed. Now I can finally properly share pics! :D

Meme: I CAN HAS IMAGE PROCESSING?
Download original

How it works: the micropub media endpoint in sweetroll2 uploads to S3 (with a callback URL in the metadata), returns an S3 URL. The imgroll Lambda notices the upload, extracts metadata, does processing, uploads resized versions to S3, POSTs to the callback a rich object with metadata and links to the sizes. But from there, there’s three ways of getting the object into the post instead of the URL:

  • if everything goes right, it’s processed quickly: the callback is forwarded to the post editor via Server-Sent Events and the URL gets replaced with the object right in the browser;
  • if the post is saved with the S3 URL before the processing is done: the callback handler modifies all posts with that URL in any field;
  • same but after the processing is done: the micropub endpoint replaces all URLs for which these callbacks have happened.

Also, the images are served from CloudFront now, on a CNAME subdomain (with a certificate issued by AWS ACM). Which has required.. switching DNS providers: the 1984 FreeDNS was being buggy and wouldn’t apply my changes. Now I’m on desec.io which is currently API-only and has no web UI, but that’s actually cool because I now have all the DNS records in a script that deploys them using curl.

I’ve been rewriting the engine that runs this website in the past few months..

and now, finally, unrelenting.technology runs on sweetroll2! Longer writeup coming, this is more of a test post.

Building a reader on your website is not too hard when you already have webmention processing (so you have code to parse entries and whatnot). So I kinda have one now. There’s even some Microsub support, but that’s not complete yet.

There’s a funny bug in my feed fetching though: OAuth for the open web is always on top of the feed (its published date gets set to feed fetch time every time) :D

I was wondering why replies sent with the Omnibear Micropub browser extension ended up with the URL /replies/ instead of the auto generated slug. Turns out Omnibear sends mp-slug=”” and my server happily accepted the empty slug :D

I rewrote micro-panel (the “admin panel” for this site) from scratch with LitElement and no material design components. It’s really tiny now! The minified bundle is 57kb (and that still includes a code editor with syntax highlighting). The previous version was nearly 1mb.

Also, the new version is a bit simplified: no iframe mode, only cookie auth. And it doesn’t wrap the whole page in an element, it’s now more of a set of elements.

Check out this piece of code, by the way:

async close () {
  if ('animate' in this && 'finished' in Animation.prototype) {
    await this.animate({transform: ['none', 'translateY(100vh)']},
      {duration: 300, easing: 'ease-out'}).finished
  }
  this.hidden = true
}

Finally made some updates to this website! On the surface, slightly refreshed design (single column). Under the hood, the Haskell part is now build with GHC 8.2 and the database is upgraded to Postgres 10.

Also, a new section: KnowledgeBase. It’s kinda like a “personal wiki” thing. Alphabetically sorted list of pages, WikiStyleTitles that are put into URLs without lowercasing, and these pages don’t end up on the front page.

So Amazon Lambda has a 6 MB limit on request (and response) size. Binary files have to be Base64 encoded (LOL) which makes the limit even SMALLER! So my micropub media endpoint chokes on full DSLR resolution photos. Yeah the "right way" is to have the API Gateway endpoint upload to S3, and the upload event trigger the Lambda processing which would download from S3, and use a separate Lambda for authentication on that endpoint… but I need the processed URLs in the response body. I need everything to happen in one request! How did AWS engineers not see that use case coming?!

Since 2017-02-01 I've been working on a big change to Sweetroll, the engine that powers this website, and today it's finally live, right here! (Also it's now on my newer VPS, hosted at prgmr and running HardenedBSD 11. The old one was FreeBSD 10 at DigitalOcean.)

tl;dr about the change: from one Haskell app powered by a Git+JSON store and embedded Duktape templates (lol) to two services (Haskell + Node.js) backed by Postgres. It's really cool. I'll be writing more documentation for it soon.

Added Pushover notifications to Sweetroll. It’s a JavaScript plugin that calls an HTTP request function provided from Haskell, because why not :-)

This day in “computers are terrible”: Sweetroll crashes after a few requests to the HTTP proxy… on my laptop with FreeBSD 11-CURRENT… when compiled as a static binary. I don’t think the same static binary ever crashed on my server with 10-RELEASE though. Anyway, removed the -static flag.