Code that writes code and conversation about conversations

In a previous post I talked about the importance of frameworks and also made some very vague criticism of framework critics. This comment was made of the post:

Concerning Yesod, the criticism that I harbor is that it seems (seems!) to rely too much on Template Haskell and Quasi Quoting where a plain old combinator library would do.

I am very sorry to be picking on an individual. I appreciate the statement being made directly to me instead of just passing it along to someone else, and I hope he forgives me for feeling the need to use this as a specific example.

The idea that Template Haskell (TH) and Quasi Quoting (QQ) is bad and combinators should be used instead betrays a lack of knowledge about the purpose of TH/QQ in its use in Yesod: the same level of type-safety and conciseness simply cannot be achieved with combinators. So now we have a criticism that arises from a lack of knowledge about the subject at hand. To the comment's credit, it is qualified with "it seems". It might seem that way because the criticism was repeated from or reinforced by other really smart Haskellers, but none of the people making the statement (including the commentator in this case) have ever used Yesod.

For some Haskellers there is such a dogma of aversion to Template Haskell and Quasi Quoting and love of combinators that smart people are making uninformed statements. I am hoping we can all acknowledge that frameworks require code generation to reach high levels of type-safety, abstraction, and productivity. I am not asking for a free pass, but instead of harboring criticism, ask the important question: "Why is it that Yesod uses TH/QQ for routing?". If the answer is not satisfactory, then please criticize, but a first step in a healthy dialogue is missing here, after which we can have useful informed criticism.

I know this paragraph is going to sound really pedantic, but it bears repeating, at least to remind myself to do it more. When we express an opinion or make a statement we are not going to want to later admit that we are wrong. In the case of criticism it is much easier on everyone to ask questions when possible and otherwise keep things well qualified. The one being critiqued won't get as defensive. If the idea behind the questioning is shown to be invalid, the critic does not have to eat humble pie, but can instead be thankful for a good answer.

Those that contribute to and use Yesod are well aware that TH/QQ is a trade-off - it does give up the familiarity, ease of modification, and composability of normal Haskell code. Lets see what benefits they provide that cannot be achieved with combinators alone.

Explanation of Template Haskell & Quasi-Quoting use

Routing

Our goal which requires Template Haskell is type-safe URLs. They make a site much more robust, especially in the face of changes. If you change your URL scheme, just recompile, and GHC will tell you every single place in your application that needs to be fixed.

Our simple example application just looks up (blog) posts.

newtype PostId = PostId Int -- some kind of database ID

If you use Yesod, you use its routing:

/             HomeR GET
/post/#PostId PostR GET

And we are done routing! We just need to define our handler functions

getHomeR :: Handler RepHtml
getHomeR = ...
getPostR :: PostId -> Handler RepHtml
getPostR postId = ...

Lets talk over what went on here. A data declaration was created for each URL:

data PostRoute = HomeR | PostR PostId

If you want to link to a specific post, you don't need to start splicing together strings, dealing with how to display a PostId, or anything else. You just type in "PostR postId". If you accidently use a String instead of a PostId, your code won't compile. So you can't accidently link to "/post/my-blog-title". So we now have type-safe urls. We can pattern match on them, as is done in Yesod to authorize a user for certain routes. We can also stick urls in our templates and know that they are valid.

So Yesod is generating code similar to this:

render HomeR = "/"
render (PostR postId) = "/post/" ++ (toString postId)

But Yesod is also parsing and dispatching. Route parameters are automatically parsed - in this case an Int. If a user enters "abc" as a post id, the parsing fails and an appropriate error response is automatically returned. The String from the route is automatically converted to an Int. So there is some parsing/dispatching code like this:

dispatch ("/":[]) = return getHomeR
dispatch ("/post/":postId:[]) = parseInt postId >>= return . getPostR

In order to make this system work, you need several components: a data type that dispatches to a function, a parse function, and a render function. Writing all of this code manually is both tedious and error-prone: it's very easy to accidently let your parse and render functions get out-of-sync. This is why Yesod uses TH: by having a single, well-tested piece of code do the whole thing, we avoid a lot of boilerplate and sidestep bugs.

It is actually possible to create type-safe urls with routing combinators by using Jeremy Shaw's recently released web-routes-boomerang package. While this package does not use Quasi Quoting it still requires separately declaring route data types and then using Template Haskell to avoid some of the boilerplate. That template Haskell creates references for the separate route parser. You still have to connect the route data types to functions to dispatch to in a dispatch function. It is a very neat use of combinators and the boomerang parsing concept. However, at the end of the day there is still more boilerplate and mental re-combining - it is not any simpler to use than Yesod's routing.

Templates

Technically you don't have to use the Shakespeare family of compile-time templates with Yesod, but users love them and we see little desire to try to use any alternatives.

<html>
<head>
<linkrel="stylesheet"href=@{StaticRnormalize_css}>
<title>If you know html, you know Hamlet
<body>
<ahref=@{HomeR}> Go Home
<ul>
<li> Insert your Haskell #{variable} with the ease of use of dynamic languages
<li> But with the guarantee that it exists at compile time
<li> and it will be XSS escaped if needed.

This is a compile-time template. Notice the link that refers to HomeR. This is from our route declaration, and compiling this template will fail unless the route exists. By default we also generate routes for Static files (the StaticR above). This means you don't have to load up the browse and check the debugger tool to see if it failed to load a stylesheet - if the template compiles it works.

Any variables interpolated exist as normal Haskell code. This avoids a needless error-prone step of creating a name-value mapping for a run-time template. The variable insertion expands to something like:

(Builder "<li> Insert your Haskell") `mappend` (toBuilder . toHtml variable) `mappend` (Builder "with the ease of use of dynamic languages")

Under the hood, the very efficient blaze-builder is being used, but that is an encapsulated implementation detail. toHtml is called on every insertions. This lets us guarantee safety against XSS attacks. If your value is already XSS escaped, you simply use an Html type that has a ToHtml instance defined that does not escape it.

The alternative systems that Haskell has offered have not been as compelling. blaze-html is a combinator library to define your entire HTML file as Haskell code. But html is an xml-like standard to be read by a browser. It is the combinator library that is adding complexity by taking something which is designed as text and turning it into code. Combinators compose into functions, which is very good, but Hamlet composes into Widgets, which work just as well. Moreover, Hamlet is not just designed for Haskellers - it is designed for web designers. Designers love using Hamlet because it is just HTML with simple ways to insert Haskell values.

Persistent

One can create a Yesod site without using the Persistent library. We provide a 'tiny' scaffolding option for this. However, we recommend using the Persistent library for data storage. The goal of Persistent is to provide type-safe database queries and automatically marshal data. This means that unlike other Haskell database libraries, mistyping the name of a column is a compilation error, and you get to work directly with regular Haskell records. This is all facilitated by declaring a quasi-quoted schema.

Person
    name String
    age Int Maybe
BlogPost
    title String
    authorId PersonId

The original version of Persistent had a lot more Template Haskell generation than it does now. We now use combinators for Persistent queries, but we used to use generated data constructors, simply because we didn't realize there was an alternative way to implement Persistent using existential types. So on the one hand, those that are broadly critical of the use of Template Haskell would be correct in that case, but on the other hand there wasn't any meaningful dialogue. We would have loved for a Template Haskell hater to tell us: "but you don't need TH, you can implement this with existential types this way". But insteaad we had to wait a long time for a similar library with existential types to show up, and then we switched to that style of implementation. I should also note that we have considered not using Quasi Quoting now that the implementation has changed - it makes QQ less necessary, but at a minimum Template Haskell is still required to maintain conciseness.

Template Haskell and Quasi-Quoting can be easy for the end user

Easy to use and type-safe templates and urls require Template Haskell and Quasi Quoting, and are among the most important features in Yesod, and they are what set it apart from all other web frameworks that we know of. But new features means limited horizons: those that haven't used them are very unlikely to recognize the difference, and it is nearly impossible to appreciate the benefits without using it yourself.

Template Haskell and Quasi-Quoting provide type-safety and features not otherwise possible without a lot of boilerplate. If you would like, you can get the exact same result by manually typing out the boilerplate. However, this is a meaningless exercise most of the time that risks creating bugs.

The 3 Yesod Quasi-Quoting cases are ridiculously easy to learn. There is a fake Haskell user that is sometimes used for the sake of argument. This user is not intelligent enough to figure out how to use a dead simple routing syntax, but is somehow comfortable learning routing combinator libraries. In fact, learning a routing syntax consisting of a total of 5 different tokens specialized for the task at hand is likely easier than learning a combinator library. The combinator library can compose functions, but it is hard to translate that into a meaningful advantage.

As simple as the routing DSL is, the majority of real Haskell programmers are probably uncomfortable modifying the code behind it. That is a tradeoff we are ok with - routing is a limited domain that the framework can readily handle - the programmer should be spending time on custom application code or enhancing other areas of the framework that have much greater variation.

Conclusion

I hope this explains some of the benefits of Yesod to those who haven't used it yet, and the utility behind Yesod's uses of Template Haskell and Quasi Quoting. I also hope we can move from dogma and uninformed criticism into meaningful dialogue. Intelligent conversation is one the best reasons to be a part of the Haskell community.