Keter

The Yesod scaffolding comes with some built-in support for the Keter deployment engine, which is also written in Haskell and uses many of the same underlying technologies like WAI and http-client. Keter works as a reverse proxy to your applications, as well as a system for starting, monitoring, and redeploying running apps. If you’d like to deploy with Keter, follow these steps:

1. Edit the config/keter.yaml file in your scaffolded application as necessary.

2. Set up some kind of server for hosting your apps. I recommend trying Ubuntu on Amazon EC2.

3. Install Keter on that machine. Please follow the instructions on the Keter website, which will be the most up to date.

4. Run yesod keter to generate a Keter bundle, e.g., myapp.keter.

5. Copy myapp.keter to the /opt/keter/incoming directory on your server.

If you’ve got things configured correctly, you should now be able to view your website, running in a production environment! In the future, upgrades can be handled by simply rerunning yesod keter and recopying the myapp.keter bundle to the server. Note that Keter will automatically detect the presence of the new file and reload your application.

The rest of this chapter will go into some more details about various steps, and provide some alternatives for people looking to either not use the scaffolding or not use Keter.

Compiling

The biggest advice I can give is: don’t compile on your server. It’s tempting to do so, as you have to just transfer source code around, and you avoid confusing dependency issues. However, compiling a Yesod application takes significant memory and CPU, which means:

• While you’re recompiling your app, your existing application will suffer performance-wise.

• You will need to get a much larger machine to handle compilation, and that capacity will likely sit idle most of the time, since Yesod applications tend to require far less CPU and memory than GHC itself.

Once you’re ready to compile, you should always make sure to stack clean before a new production build, to make sure no old files are lying around. Then, you can run stack build to get an executable, which will be located at dist/build/myapp/myapp.

Files to deploy

With a Yesod scaffolded application, there are essentially three sets of files that need to be deployed:

1. Your executable.

2. The config folder.

3. The static folder.

Everything else, such as Shakespearean templates, gets compiled into the executable itself.

There is one caveat, however: the config/client_session_key.aes file. This file controls the server side encryption used for securing client side session cookies. Yesod will automatically generate a new one of these keys if none is present. In practice, this means that, if you do not include this file in deployment, all of your users will have to log in again when you redeploy. If you follow the advice above and include the config folder, this issue will be partially resolved. Another approach is to put your session key in an environment variable.

The other half of the resolution is to ensure that once you generate a config/client_session_key.aes file, you keep the same one for all future deployments. The simplest way to ensure this is to keep that file in your version control. However, if your version control is open source, this will be dangerous: anyone with access to your repository will be able to spoof login credentials!

The problem described here is essentially one of system administration, not programming. Yesod does not provide any built-in approach for securely storing client session keys. If you have an open source repository, or do not trust everyone who has access to your source code repository, it’s vital to figure out a safe storage solution for the client session key.

SSL and static files

There are two commonly used features in the Yesod world: serving your site over HTTPS, and placing your static files on a separate domain name. While both of these are good practices, when combined they can lead to problems if you’re not careful. In particular, most web browsers will not load up Javascript files from a non-HTTPS domain name if your HTML is served from an HTTPS domain name. In this situation, you’ll need to do one of two things:

• Serve your static files over HTTPS as well.

• Serve your static files from the same domain name as your main site.

Note that if you go for the first option (which is the better one), you’ll either need two separate SSL certificates, or a wildcard certificate.

Warp

As we have mentioned before, Yesod is built on the Web Application Interface (WAI), allowing it to run on any WAI backend. At the time of writing, the following backends are available:

• Warp

• FastCGI

• SCGI

• CGI

• Webkit

• Development server

The last two are not intended for production deployments. Of the remaining four, all can be used for production deployment in theory. In practice, a CGI backend will likely be horribly inefficient, since a new process must be spawned for each connection. And SCGI is not nearly as well supported by frontend web servers as Warp (via reverse proxying) or FastCGI.

So between the two remaining choices, Warp gets a very strong recommendation because:

• It is significantly faster.

• Like FastCGI, it can run behind a frontend server like Nginx, using reverse HTTP proxy.

• In addition, it is a fully capable server of its own accord, and can therefore be used without any frontend server.

So that leaves one last question: should Warp run on its own, or via reverse proxy behind a frontend server? For most use cases, I recommend the latter, because:

• Having a reverse proxy in front of your app makes it easier to deploy new versions.

• Also, if you have a bug in your application, a reverse proxy can give slightly nicer error messages to users.

• You can host multiple applications from a single host via virtual hosting.

• Your reverse proxy can function as both a load balancer or SSL proxy as well, simplifying your application.

As discussed above, Keter is a great way to get started. If you have an existing web server running like Nginx, Yesod will work just fine sitting behind it instead.

daemon off; # Don't run nginx in the background, good for monitoring apps
events {
    worker_connections 4096;
}

http {
    server {
        listen 80; # Incoming port for Nginx
        server_name www.myserver.com;
        location / {
            proxy_pass http://127.0.0.1:4321; # Reverse proxy to your Yesod app
        }
    }
}

server {
    listen 80;
    server_name myserver.com;
    rewrite ^/(.*) http://www.myserver.com/$1 permanent;
}

server {
    listen 80;
    server_name static.myserver.com;
    root /home/michael/sites/mysite/static;
    # Since yesod-static appends a content hash in the query string,
    # we are free to set expiration dates far in the future without
    # concerns of stale content.
    expires max;
}

server {
    listen 80; # Incoming port for Nginx
    server_name www.myserver.com;
    location / {
        proxy_pass http://127.0.0.1:4321; # Reverse proxy to your Yesod app
    }
    location /static {
        root /home/michael/sites/mysite; # Notice that we do *not* include /static
        expires max;
    }
}

description "My awesome Yesod application"
start on runlevel [2345];
stop on runlevel [!2345];
respawn
chdir /home/michael/sites/mysite
exec /home/michael/sites/mysite/dist/build/mysite/mysite

[Service]
ExecStart=/home/sibi/.local/bin/my-yesod-executable
Restart=always
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=yesod-sample

[Install]
WantedBy=multi-user.target

systemctl enable yesod-sample
systemctl start yesod-sample

Nginx + FastCGI

Some people may prefer using FastCGI for deployment. In this case, you’ll need to add an extra tool to the mix. FastCGI works by receiving new connection from a file descriptor. The C library assumes that this file descriptor will be 0 (standard input), so you need to use the spawn-fcgi program to bind your application’s standard input to the correct socket.

It can be very convenient to use Unix named sockets for this instead of binding to a port, especially when hosting multiple applications on a single host. A possible script to load up your app could be:

spawn-fcgi \
    -d /home/michael/sites/mysite \
    -s /tmp/mysite.socket \
    -n \
    -M 511 \
    -u michael \
    -- /home/michael/sites/mysite/dist/build/mysite-fastcgi/mysite-fastcgi

You will also need to configure your frontend server to speak to your app over FastCGI. This is relatively painless in Nginx:

server {
    listen 80;
    server_name www.myserver.com;
    location / {
        fastcgi_pass unix:/tmp/mysite.socket;
    }
}

That should look pretty familiar from above. The only last trick is that, with Nginx, you need to manually specify all of the FastCGI variables. It is recommended to store these in a separate file (say, fastcgi.conf) and then add include fastcgi.conf; to the end of your http block. The contents of the file, to work with WAI, should be:

fastcgi_param  QUERY_STRING       $query_string;
fastcgi_param  REQUEST_METHOD     $request_method;
fastcgi_param  CONTENT_TYPE       $content_type;
fastcgi_param  CONTENT_LENGTH     $content_length;
fastcgi_param  PATH_INFO          $fastcgi_script_name;
fastcgi_param  SERVER_PROTOCOL    $server_protocol;
fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;
fastcgi_param  REMOTE_ADDR        $remote_addr;
fastcgi_param  SERVER_ADDR        $server_addr;
fastcgi_param  SERVER_PORT        $server_port;
fastcgi_param  SERVER_NAME        $server_name;

Desktop

Another nifty backend is wai-handler-webkit. This backend combines Warp and QtWebkit to create an executable that a user simply double-clicks. This can be a convenient way to provide an offline version of your application.

One of the very nice conveniences of Yesod for this is that your templates are all compiled into the executable, and thus do not need to be distributed with your application. Static files do, however.

A similar approach, without requiring the QtWebkit library, is wai-handler-launch, which launches a Warp server and then opens up the user’s default web browser. There’s a little trickery involved here: in order to know that the user is still using the site, wai-handler-launch inserts a "ping" Javascript snippet to every HTML page it serves. It wai-handler-launch doesn’t receive a ping for two minutes, it shuts down.

CGI on Apache

CGI and FastCGI work almost identically on Apache, so it should be fairly straight-forward to port this configuration. You essentially need to accomplish two goals:

1. Get the server to serve your file as (Fast)CGI.

2. Rewrite all requests to your site to go through the (Fast)CGI executable.

Here is a configuration file for serving a blog application, with an executable named "bloggy.cgi", living in a subfolder named "blog" of the document root. This example was taken from an application living in the path /f5/snoyman/public/blog.

Options +ExecCGI
AddHandler cgi-script .cgi
Options +FollowSymlinks

RewriteEngine On
RewriteRule ^/f5/snoyman/public/blog$ /blog/ [R=301,S=1]
RewriteCond $1 !^bloggy.cgi
RewriteCond $1 !^static/
RewriteRule ^(.*) bloggy.cgi/$1 [L]

The first RewriteRule is to deal with subfolders. In particular, it redirects a request for /blog to /blog/. The first RewriteCond prevents directly requesting the executable, the second allows Apache to serve the static files, and the last line does the actual rewriting.

FastCGI on lighttpd

For this example, I’ve left off some of the basic FastCGI settings like mime-types. I also have a more complex file in production that prepends "www." when absent and serves static files from a separate domain. However, this should serve to show the basics.

Here, "/home/michael/fastcgi" is the fastcgi application. The idea is to rewrite all requests to start with "/app", and then serve everything beginning with "/app" via the FastCGI executable.

server.port = 3000
server.document-root = "/home/michael"
server.modules = ("mod_fastcgi", "mod_rewrite")

url.rewrite-once = (
  "(.*)" => "/app/$1"
)

fastcgi.server = (
    "/app" => ((
        "socket" => "/tmp/test.fastcgi.socket",
        "check-local" => "disable",
        "bin-path" => "/home/michael/fastcgi", # full path to executable
        "min-procs" => 1,
        "max-procs" => 30,
        "idle-timeout" => 30
    ))
)

CGI on lighttpd

This is basically the same as the FastCGI version, but tells lighttpd to run a file ending in ".cgi" as a CGI executable. In this case, the file lives at "/home/michael/myapp.cgi".

server.port = 3000
server.document-root = "/home/michael"
server.modules = ("mod_cgi", "mod_rewrite")

url.rewrite-once = (
    "(.*)" => "/myapp.cgi/$1"
)

cgi.assign = (".cgi" => "")