Proxying Vapor 3 Using nginx and Docker

Vapor is a framework for running server side code, all built in Swift. There’s a great post by bygri about how to build a Vapor app using Docker. If any of what I’ve just said is confusing, I highly recommend reading that post before getting started here.

During my building of a couple of Vapor apps, I’ve found getting the sites running while hitting localhost:8080 to be really easy. The hard part is putting that site behind a domain locally and even more, getting it deployed to a server. So I’m going to put a huge disclaimer here: I’m no expert in nginx or Docker. I’ve been able to piece things together using web searches and a lot of conversation in the Vapor Discord room. There’s a great channel for Docker specifically, where I’ve been hugely helped by @bygri. He’s good people.

This post’s goal is purely to show how I got my Vapor app - which worked in development - deployed to production and successfully proxied by nginx. There will be a few extra bits I learned along the way as well (just to sweeten the deal for you).

My initial thought process was this:

  1. Everything for my web app lives inside of the git repository.
  2. I have a script to run for local development
  3. When it’s time to deploy, I clone the repo to my server, and run a script to boot it all up there.

Turns out I was a bit mistaken. If you read through the post I linked at the top, you’ll notice that he has separate Dockerfiles for development and production. I skipped over that part, much to my initial peril (so don’t do that).

What I did was create a repository on the Docker Hub which would hold a named, built image. That image is built with the following Dockerfile:

# Build image
FROM swift:4.1 as builder
RUN apt-get -qq update && apt-get -q -y install \
  && rm -r /var/lib/apt/lists/*
WORKDIR /app
COPY . .
RUN mkdir -p /build/lib && cp -R /usr/lib/swift/linux/*.so /build/lib
RUN swift build -c release && mv `swift build -c release --show-bin-path` /build/bin

# Production image
FROM ubuntu:16.04
RUN apt-get -qq update && apt-get install -y \
  libicu55 libxml2 libbsd0 libcurl3 libatomic1 \
  tzdata \
  && rm -r /var/lib/apt/lists/*
WORKDIR /app
COPY --from=builder /build/bin/Run .
COPY --from=builder /build/lib/* /usr/lib/
COPY Resources/ ./Resources/
EXPOSE 8080
ENTRYPOINT ./Run serve -e prod -b 0.0.0.0

If you have discerning eyes you’ll notice that this Dockerfile is almost exactly the same as the one in the post  I referenced above (go read it and come back if you didn’t earlier). That’s because I copied it from him 😊. The one addition I had to make was COPY Resources/ ./Resources. What this does is copies the Resources directory from my local drive and into the Docker image.

I was initially skeptical that I needed the EXPOSE directive since nginx would proxy over there by default, but I’ve confirmed that it’s not exposed on my server and this will make for good documentation, so 🤷‍♂️.

I built that image using docker build -t jsorge/taphouse.io. Then I can push the image using docker push jsorge/taphouse.io. These 2 commands will build the image and tag it locally, then push it up to Docker Hub. I highly suggest scripting this stuff (I’m becoming partial to Makefiles to get it all done).

Now let’s flip our environment to the production server. I don’t need much to bring in the Vapor app since it’s already built and hosted. I just need to reference it and mount the Public folder.

But then comes the fun part. I’ll catch you on the flipside of the compose file:

version: "3.3"
services:
  web:
	image: jsorge/taphouse.io
	volumes:
	  - ./Public:/app/Public
  nginx:
	image: nginx:alpine
	restart: always
	ports:
	  - 80:80
	  - 443:443
	volumes:
	  - ./Public:/home/taphouse/Public
	  - ./nginx/server.conf:/etc/nginx/conf.d/default.conf
	  - ./nginx/logs:/var/log/nginx
	depends_on:
	  - web

Ok, so what’s going on here? We’re grabbing the nginx image from alpine (a trusted provider of nginx). I’m honestly not sure what the restart command does but it showed up on my DuckDuckGo results of examples. But the rest I understand.

ports: I’m exposing ports 80 and 443 to get http and https traffic listened to (the syntax for ports is <external>:<container-internal>). I could pick other internal ports to listen to and update my nginx configs appropriately but didn’t feel like rocking the boat too much. I plan on adding TLS support via letsencrypt but that’s a problem for another day.

volumes: I’m sure people familiar with Docker won’t need this explained (or any of this compose file really) but this part blows my mind a bit. The volumes directive basically puts a link from the local machine to inside the container. The link could be a file or a folder; that part doesn’t matter. So to get the root of my site working in nginx, I mount the Public folder that exists locally on my machine into the container at the specified path. The syntax for this is <local-path>:<container-path>.

I also created the path of nginx/logs on my server, and mounted it as the log directory in the container. This allows me to have the logs from the container persisted to my server volume and easily read them as they come in. This is super cool!

depends_on: This one is pretty cool. It establishes the dependency chain between your containers and starts them up in the proper order to satisfy them. In this case, nginx depends on my Vapor app (called web).

The nginx configuration file is the last link in the chain to get us going. This is what mine looks like:

server {
	server_name taphouse.io;
	listen 80 default_server;

	root /home/taphouse/Public/;

	try_files $uri @proxy;

	location @proxy {
		proxy_pass http://web:8080;
		proxy_pass_header Server;
		proxy_set_header Host $host;
		proxy_set_header X-Real-IP $remote_addr;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_pass_header Server;
		proxy_connect_timeout 3s;
		proxy_read_timeout 10s;
	}
}

I got this by finding the suggested configuration for Vapor 2. Curiously this page hasn’t made its way to the Vapor 3 docs but I’m guessing that has something to do with them wanting you to deploy on Vapor Cloud. I’m removing my tinfoil hat now.

As far as nginx configuration files go, this one looks pretty standard. I don’t know the exact ins and outs of what’s going on but the point of note is here: proxy_pass http://web:8080;. Remember how I said earlier that Docker Compose provides its own networking? Well it turns out I can use my Vapor container name as the host and it will resolve everything inside the container network. Super cool!

From here I got these files on the server and ran my make server command (which aliases to docker-compose -f docker-compose-prod.yml up --build -d) and tried hitting http://taphouse.io. It worked! 🤯

The next thing on my list is to get acme.sh working so that TLS is up and running, and I can disable port 80. That’s my current holdup, but when I get that done I’ll be sure to write that up as well.

If I made any grievous errors or you have general feedback, I’d love to hear it. I’m still very green when it comes to Docker, nginx, and server admin stuff in general.

Building a Blog Engine

For the last few months I’ve been thinking about what to do with my blog setup. I experimented a bit with Jekyll but that ended up not really appealing to me. In the consideration of switching platforms I realized I wanted something that would be easy to move (and not be beholden to should the need to move again arise). Markdown text files check that box; but what about images?

How would I move the images I’ve hosted on my own server? Where do they land in the new location? I had no good idea until I stumbled up on textbundles. Textbundles basically wrap up a markdown file and assets folder inside of a directory. Bundles are a very familiar concept for anyone on a Mac; we’ve had them since OS X became a thing.

I first decided to figure out how to make this format work in Jekyll. Would it be a hook I need to build? Where would that go in the pipeline and how does it work? Also, how do I even Ruby?

All of this started to make my new site feel like a distant goal. Learning a new language and framework was kind of daunting. Not that I’m opposed to such things, but I don’t have a ton of extra time available plus it could easily distract me from my goal.

In the meantime I decided to give Vapor a look again. I’d played with v2 last year when trying to rebuild my company website. It was… ok. But Vapor 3 really clicked for me and I could see while writing it how a blog engine could work. So I figured I’d go for it.

Meet Maverick.

As of this writing I’m probably 80% done with the site. In local testing it does exactly what I need it to do on a basic level. Blog post and static page support. Everything served from a textbundle. That’s the easy part.

The hard part now is how to deploy. I’ve got a lot to learn about Docker and TLS. I tried adding TLS to the new Taphouse site and couldn’t get it figured out. But also if I want to make this an easy system for someone else to adapt, how do I do that?

I know that posts should be separated from the engine itself. So the bare repo for new sites should have some script to grab the Maverick app and run it in some container. Since it’s just a self-contained Swift app that should be straightforward enough, right?

All of this consideration kind of has me paralyzed at the moment. I was hoping to punt on deployment a bit until I got Ulysses and Micro.blog support wrapped up but they seem to depend on TLS connections, and that brings me back to getting things deployed.

So that’s where I’m at. I’m super excited about getting Maverick onto my server and running it full time. But getting there means jumping a few more hurdles first.

Rebuilding taphouse.io

A few months ago, when I decided I needed to update JavaScript on my server that ran both my blog and company website, I broke the company site. I wrote it a few years ago using a very early iteration of the Sails.js framework. Fast forward to today and I want to bring it back. I also have been wanting to dabble in server-side Swift. Enter Vapor.

I got the site up and running in Vapor 3 in a few days (last year I had started doing this in Vapor 2, but 3 was just released so the timing was good). I’m a big fan of what the Vapor team is doing and may yet take on a separate project based on Vapor. But that’s for another day.

Today is about Docker. I want my new company site to be housed in a single git repo (✅), and deployable using Docker. I want to have the vapor app in a container, and nginx on the front of it in a separate container. I also want to have SSL on the site, managed by letsencrypt. I’m also hoping to have a dev environment and a production environment (all self contained inside this repo).

This is where my holdup is. I’ve never used Docker, and have been inundating myself trying to figure it all out. I want to build this in the open, so I’ll be microblogging and regular blogging about my progress – however slow it may be.

My First Swift Package

I’m embarking on phase 1 of my blog rethink and have decided that whatever I do next will use the textbundle spec for file transport and storage. It will likely be some git repo or perhaps a Dropbox synced folder but all of the options that I’ve seen don’t strongly link a post together with assets. Textbundle solves that.

I think initially I might wrap Jekyll in something, so that the thing making the static site is really an implementation detail. The idea being that my server has an endpoint that would accept XML-RPC as well as micropub inputs, transform those to a textbundle and then build the static site from there.

So my first step was to download my posts stored on Ghost as well as my assets folder. I wanted a way to build textbundles out of these for easy migration. My first inclination was a Ruby script. I don’t know Ruby, so that became problematic from the start. So I turned to a more familiar language.

I made a Swift package called textbundleify, and the code is up on GitHub if you’re interested. It’s not polished in the slightest, but was a fun chance to try something new. It gets run from a directory containing Markdown files and if you give it a directory that contains pictures (or a directory of directories of pictures), it will embed linked images to the referenced textbundle.

I had to learn to use Process as well as NSRegularExpression (though I still don’t know how regular expressions really work). It was quite weird getting an NSRange for use on a Swift string, which only works on String.Range ranges. It was fun to put together over a few hours, and should do the migration trick I’m wanting.

The next step is to figure out the proper order of operations in converting textbundles (which Jekyll doesn’t support) to the structure that Jekyll does. I’ve still got Ruby learning on my horizon but at least there are a couple of wonderful souls who have offered some help to me if I need it.

Onward!

View Controllers Make Crummy Navigators

My fellow iOS developers, we have a problem. We’ve been led astray by Apple. Don’t get me wrong, iOS is great. I love writing code in Swift (and Objective-C!). I’m a fan of Xcode too. And UIKit really makes a lot of really powerful things quite easy. But allow me this hot take: View controllers should never have to deal with navigation.

There’s a long running joke about MVC actually standing for massive view controllers and navigation is a key reason. Even a relatively simple view controller with a couple of possible destinations, using storyboards and segues, needs to implement methods that could grow to be dozens of lines long and end up knowing about the implementation details of view controllers that come after them.

What are we to do? There must be some way that can keep our view controllers clean of navigation and doesn’t fight the UIKit frameworks completely, right?

There is!

If we treat view controllers like functional pieces, we can expose a small interface to interact with them and use delegation to get stuff out. Here’s a trivial example:

protocol PeopleViewControllerDelegate: AnyObject {
    func personSelected(_ person: Person, from viewController: PeopleViewController)
}

final class PeopleViewController: UIViewController, UITableViewDelegate {
    var delegate: PeopleViewControllerDelegate?
    private var people: [People]?

    func configureWithPeople(_ people: [Person]) {
        self.people = people
        // fill a table with people
    }

    func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
        // find the person
        self.delegate?.personSelected(person, from: self)
    }
}

This view controller (while grossly incomplete) gets configured with an array of people that will fill a table view. Tap on a row and the view controller’s job is done. It hands off the person to its delegate. What happens next is out of its hands. Perhaps we’ll show a detail screen, but maybe it’s just an API call and this view controller gets dismissed. Our PeopleViewController doesn’t know, and doesn’t care. Kind of great, right?

This begs the question of what exactly is the delegate? I’ve come to really like the coordinator pattern myself. I like having objects whose specific responsibility it is to manage the flow of my app (or just a given object’s slice of the app).

There are some things that you’ll have to give up, of course. Storyboard segues are a non-starter in this pattern. This means that there are large swaths of sample code (Apple’s and others’) that you’ll need to learn to either ignore or translate into this pattern.

So let’s trim down the size of our view controllers. Let’s narrow their scopes of responsibility, and in turn we’ll get code that is more maintainable and easier to reason about. I think we will all be better off.

Childlike Faith

And calling to him a child, he put him in the midst of them and said, "Truly, I say to you, unless you turn and become like children, you will never enter the kingdom of heaven. Whoever humbles himself like this child is the greatest in the kingdom of heaven. (Matthew 18:2-4, ESV)

I was on a walk with Finnian (my 21 month old) and as we were on the sidewalk he wanted to walk on the curb that separates the sidewalk from the road. You might have heard that kids his age don’t have the greatest of balance and I can confirm those reports. So he would reach up to grab my hand to keep steady as he walked on the curb.

But the thing is, he didn’t even look at me. He just stuck his hand up in the air and went along the curb. He was confident that I would help him stay balanced by taking his hand.

It struck me the third or fourth time he did this that he had immense faith that I would help him to not fall. And then God reminded me that this is how I’m supposed to be with him. Jesus says that I need to become like a child to enter the kingdom of heaven. What does that mean?

I should stick out my hand and ask for help more often. But I don’t. During everyday life I try to go it on my own. It doesn’t even occur to me to pray while I’m figuring out some way to make something animate just right or parse a feed. But why not? Honestly, I don’t know. But I want to.

How do I do this? I come before God and ask him to help me remember him throughout the day.

Rejoice always! Pray constantly. Give thanks in everything, for this is God's will for you in Christ Jesus. (1 Thessalonians 5:16-18, HCSB)

do not be anxious about anything, but in everything by prayer and supplication with thanksgiving let your requests be made known to God. And the peace of God, which surpasses all understanding, will guard your hearts and your minds in Christ Jesus. (Philippians 4:6-7, ESV)

Amen!

At a previous job I used to keep a post-it note on my monitor that just said “Pray”. Maybe I should bring that back and put it on the monitors I use on a day-to-day basis. That constant reminder that God is close and I’m beckoned to draw near to the throne of grace (Hebrews 4:16) could be the kickstart I need.

Reconsidering My Blogging Setup

I have a very on-again, off-again habit of writing. Sometimes I’ll have a lot to write about, and often I’ll go long stretches without writing much. I backed Micro.blog on its Kicskatarter the day it came out (happy birthday to me!), and it’s gotten me thinking about what blogging even is. Whoa.

I’ve usually considered blogging to be posts like this one. Longer things that I host on my website. But now I’ve got my microblog. I usually think of these posts as “Tweets that I own”. We all know that when you post content to a place you don’t control –  like Twitter, Facebook, or Medium – that you don’t own the post. Not the URL, the appearance, none of it. You gave them the things. But when you own the domain, you can control it all.

So my mental model of replacing tweeting on Twitter with Micro.blog was a good start. But now I see that microblogging is still blogging. Whoa.

Now I’ve got the itch for my website to feature both longer posts and microblog posts. I’ll have feeds for long content, micro content, and a mix of both. Posts should get to micro.blog through one of those feeds. And I’ll own the links to all of my content.

Now for the real question: how can I get this done?

I’m currently using Ghost to power my blog. But Ghost doesn’t seem interested in supporting microblogging (which is fine) so I’m going to look elsewhere. I don’t really want to use micro.blog itself because I don’t get enough control over things there.

What about Wordpress? I don’t like PHP, nor have I had good experience with them in the past. So that’s out.

I’ve been looking at Jekyll. That may end up being the winner but it will take some experimenting. I need to make sure existing links to ghost content will continue working and Jekyll ends entries in .html, but Ghost doesn’t. I’ll have to learn some level of nginx rules to get this done, is my guess.

But I think what I really want is something like Brent Simmons’s Wildcat system. I just wish it was written in Swift so I could understand how it’s put together. I’ve never worked in Ruby before, and while that’s not a dealbreaker it’s definitely additional friction.

I also would love to find something that can work with my text editor (Ulysses), can support image uploading automatically, and has Mac & iOS variants for me to write and publish posts easily.

This is going to take some time, and hopefully I’ll get it right.

Hawaii Trip

We spent the last week on the Oahu island of Hawaii, getting home late last night. It was an amazing trip and couldn’t have gone any better. We rented an AirBnB house in Kaneohe (which is the windward side of the island). The weather was quite good most of the time; even when it was raining, the rain was warm. Quite the change from Seattle.

View from Diamond Head

Waikiki Beach

Going into the trip I was most nervous about how the boys would do. We took them on a plane last year to Phoenix but Finnian is 20 months now and can’t be still for more than 3 seconds. We didn’t buy him a seat on the plane so he was going as a lap baby. Atticus is 4 now and we figured we could distract him with shows & movies on “his” iPad. They both did wonderfully well. A couple of screaming fits by Finnian but that was manageable by getting him to sleep.

Dole Pineapple

Tree on the Dole Train

Of course, I took along my camera. I’ve got the Sony A7II body and while I normally have a 50mm f1.8 prime on there, I wanted something that could capture the landscapes we’d be seeing and get good portrait shots of us on the trip. So I rented the Sony 24-105mm f4 zoom lens from Hawaii Camera. In short, the lens is amazing and now I want one.

Emily had done tons of research and put together a packed itinerary that got us all around the island. We hit up the Dole plantation, the North Shore, hiked Diamond Head and hung out on Waikiki beach, visited Sea Life, and went to a luau at Disney’s Aulani resort. We had a lot of fun and came home really tired.

Finnian before the Luau

Atticus on the trip home

Xcoders Talk: Practical Coordinators

Last November I gave a talk at the Seattle Xcoders meetup about the coordinator pattern. There’s been a lot of virtual ink spilled on the topic of design patterns and the coordinator pattern specifically. I’ve been using it in a refactor of a side project app as well as using it pretty heavily at zulily in some of the new projects I had been doing.

I got inspired to give the talk after last year’s Swift by Northwest conference, where Dave DeLong gave a really great talk about improving MVC. He has a different take on coordinators than I do and his solution is really interesting. But I like where I’ve landed with coordinators for the time being personally.

I decided to put a little different spin on this talk than I’ve seen in other coordinator talks, and prepped a sample app that I would refactor. We go from storyboard segues to a coordinator handling the transition between two screens. There was a lot of live coding and it worked out pretty well. I hope it’s entertaining if not useful for you and I’d love to hear your thoughts on how it all turned out.

Video: https://vimeo.com/258453512 Slides: https://d.pr/NfRMIt

Making Proper Apps From Websites

While at zulily, I used to have JIRA and GitLab inside of wrappers provided by Fluid instances. Now that I’m at Lyft I’ve done the same thing with JIRA, but we use GitHub instead. For the longest time I would take links sent to me for those pages and paste them into the apps by hand since clicking on the links would open my browser instead of the app silos that I hand-crafted for these sites.

Well today @jackbrewster taught me about something great that takes this to another level. The app Choosy will let me route URL clicks to whatever app I want. I set up rules in Choosy to look at the web address that I clicked and now have it routing my JIRA and GitHub links to their proper Fluid apps. It’s wonderful!

My default browser is Safari, I keep Chrome around as my Flash ghetto (and now only it can be used for Meet as well). I can use Choosy to send the Hangouts/Meet links straight to Chrome as well. This makes everything feel completely seamless.

One other tip I got from Jack was about 1Password. Fluid instances don’t support it directly but you can invoke the global 1Password mini, find the site you need, and press ⌘⇧C to copy to the clipboard. Focus will then switch back to your Fluid app and you can paste the value in there. I was using the mouse to do this previously. This way sure seems a lot better.