The RqData
module is used to extract key/value pairs from the QUERY_STRING, cookies, and the request body of a POST
or PUT
request.
Let's start with a simple hello, world! example that uses request parameters in the URL.
> module Main where > > import Happstack.Server (ServerPart, look, nullConf, simpleHTTP, ok) > > helloPart :: ServerPart String > helloPart = > do greeting <- look "greeting" > noun <- look "noun" > ok $ greeting ++ ", " ++ noun > > main :: IO () > main = simpleHTTP nullConf $ helloPart
[Source code for the app is here.]
Now if we visit http://localhost:8000/?greeting=hello&noun=rqdata, we will get the message hello, rqdata
we use the look
function to look up some keys by name. The look
function has the type:
> look :: (Functor m, Monad m, HasRqData m) => String -> m String
Since we are using look
in the ServerPart
monad it has the simplified type:
> look :: String -> ServerPart String
The look
function looks up a key and decodes the associated value as a String
. It assumes the underlying ByteString
was utf-8 encoded. If you are using some other encoding, then you can use lookBS
to construct your own lookup function.
If the key is not found, then look
will fail. In ServerPart
that means it will call mzero
.
In the previous example we only looked at parameters in the
URL. Looking up values from a form submission (a POST or PUT request)
is almost the same. The only difference is we need to first decode the
request body using decodeBody
:
> {-# LANGUAGE OverloadedStrings #-} > import Control.Monad (msum) > import Happstack.Server ( Response, ServerPart, Method(POST) > , BodyPolicy(..), decodeBody, defaultBodyPolicy > , dir, look, nullConf, ok, simpleHTTP > , toResponse, methodM > ) > import Text.Blaze as B > import Text.Blaze.Html4.Strict as B hiding (map) > import Text.Blaze.Html4.Strict.Attributes as B hiding (dir, label, title) > > main :: IO () > main = simpleHTTP nullConf $ handlers > > myPolicy :: BodyPolicy > myPolicy = (defaultBodyPolicy "/tmp/" 0 1000 1000) > > handlers :: ServerPart Response > handlers = > do decodeBody myPolicy > msum [ dir "hello" $ helloPart > , helloForm > ] > > helloForm :: ServerPart Response > helloForm = ok $ toResponse $ > html $ do > B.head $ do > title "Hello Form" > B.body $ do > form ! enctype "multipart/form-data" ! B.method "POST" ! action "/hello" $ do > B.label "greeting: " >> input ! type_ "text" ! name "greeting" ! size "10" > B.label "noun: " >> input ! type_ "text" ! name "noun" ! size "10" > input ! type_ "submit" ! name "upload" > > helloPart :: ServerPart Response > helloPart = > do methodM POST > greeting <- look "greeting" > noun <- look "noun" > ok $ toResponse (greeting ++ ", " ++ noun)
[Source code for the app is here.]
decodeBody
even needed?The body of the HTTP request is ignored unless we call
decodeBody
. The obvious question is,
Why isn't the request body automatically decoded?.
If servers had unlimited RAM, disk, CPU and bandwidth available, then automatically decoding the body would be a great idea. But, since that is generally not the case, we need a way to limit or ignore form submission data that is considered excessive.
A simple solution would be to impose a static quota an all form
data submission server-wide. But, in practice, you might want finer
granularity of control. By explicitly calling decodeBody
you can easily configure a site-wide static quota. But you can also
easily adapt the quotas depending on the user, particular form, or
other criteria.
In this example, we keep things simple and just call
decodeBody
for all incoming requests. If the incoming
request is not a PUT
or POST
request with
multipart/form-data
then calling decodeBody
has no side-effects.
BodyPolicy
and defaultBodyPolicy
to impose quotasThe only argument to decodeBody
is a BodyPolicy
. The easiest way to define a BodyPolicy
is by using the defaultBodyPolicy
function:
> defaultBodyPolicy :: FilePath -- ^ directory to *temporarily* store uploaded files in > -> Int64 -- ^ max bytes to save to disk (files) > -> Int64 -- ^ max bytes to hold in RAM (normal form values, etc) > -> Int64 -- ^ max header size (this only affects headers > -- in the multipart/form-data) > -> BodyPolicy
In the example, we define this simple policy:
> myPolicy :: BodyPolicy > myPolicy = (defaultBodyPolicy "/tmp/" 0 1000 1000)
Since the form does not do file uploads, we set the file quota to 0. We allow 1000 bytes for the two form fields and 1000 bytes for overhead in the multipart/form-data encoding.
decodeBody
Using decodeBody
is pretty straight-forward. You simple call it with a BodyPolicy
. The key things to know are:
look
and friends
decodeBody
only works once per request. The first time you call it the body will be decoded. The second time you call it, nothing will happen, even if you call it with a different policy.
<form>
When using the <form>
element there are two important recommendations you should follow:
enctype
to multipart/form-data
. This is especially important for forms which contain file uploads.
method
to POST
or the form values will show up in the URL as query parameters.
The lookFile
function is used to extract an uploaded file:
> lookFile :: String -> RqData (FilePath, FilePath, ContentType)
It returns three values:
The temporary file will be automatically deleted after the Response
is sent. Therefore, it is essential that you move the file from the temporary location.
In order for file uploads to work correctly, it is also essential that your <form> element contains the attributes enctype="multipart/form-data"
and method="POST"
The following example has a form which allows a user to upload a file. We then show the temporary file name, the uploaded file name, and the content-type of the file. In a real application, the code should use System.Directory.renameFile
(or similar) to move the temporary file to a permanent location. This example looks a bit long, but most of the code is just HTML generation using BlazeHtml. The only really new part is the use of the lookFile
function. Everything else should already have been covered in previous sections. So if you don't understand something, try looking in earlier material.
> {-# LANGUAGE OverloadedStrings #-} > import Control.Monad (msum) > import Happstack.Server ( Response, ServerPart, defaultBodyPolicy > , decodeBody, dir, lookFile, nullConf, ok > , simpleHTTP, toResponse ) > import Text.Blaze as B > import Text.Blaze.Html4.Strict as B hiding (map) > import Text.Blaze.Html4.Strict.Attributes as B hiding (dir, title) > > main :: IO () > main = simpleHTTP nullConf $ upload > > upload :: ServerPart Response > upload = > do decodeBody (defaultBodyPolicy "/tmp/" (10*10^6) 1000 1000) > msum [ dir "post" $ post > , uploadForm > ] > > uploadForm :: ServerPart Response > uploadForm = ok $ toResponse $ > html $ do > B.head $ do > title "Upload Form" > B.body $ do > form ! enctype "multipart/form-data" ! B.method "POST" ! action "/post" $ do > input ! type_ "file" ! name "file_upload" ! size "40" > input ! type_ "submit" ! value "upload" > > post :: ServerPart Response > post = > do r <- lookFile "file_upload" > ok $ toResponse $ > html $ do > B.head $ do > title "Post Data" > B.body $ mkBody r > where > mkBody (tmpFile, uploadName, contentType) = do > p (toHtml $ "temporary file: " ++ tmpFile) > p (toHtml $ "uploaded name: " ++ uploadName) > p (toHtml $ "content-type: " ++ show contentType)
[Source code for the app is here.]
Remember that you must move the temporary file to a new location or it will be garbage collected after the Response is sent. In the example code we do not move the file, so it is automatically deleted.
By default, look
and friends will search both the
QUERY_STRING the request body (aka, POST/PUT data) for a key. But
sometimes we want to specify that only the QUERY_STRING or request
body should be searched. This can be done by using the body
and queryString
filters:
> body :: (HasRqData m) => m a -> m a > queryString :: (HasRqData m) => m a -> m a
Using these filters we can modify helloPart
so that the greeting
must come from the QUERY_STRING and the noun
must come from the request body:
> helloPart :: ServerPart String > helloPart = > do greeting <- queryString $ look "greeting" > noun <- body $ look "noun" > ok $ greeting ++ ", " ++ noun
queryString
and body
act as filters which
only pass a certain subset of the data through. If you were to
write:
> greetingRq :: ServerPart String > greetingRq = > body (queryString $ look "greeting")
This code would never match anything because the
body
filter would hide all the QUERY_STRING values, and
the queryString
filter would hide all the request body
values, and hence, there would be nothing left to search.
RqData
Monad for better error reportingSo far we have been using the look
function in the
ServerPart
monad. This means that if any
look
fails, that handler fails. Unfortunately, we are not
told what parameter was missing -- which can be very frustrating when
you are debugging your code. It can be even more annoying if you are
providing a web service, and whenever a developer forgets a parameter,
they get a 404 with no information about what went wrong.
So, if we want better error reporting, we can use functions like
look
in the RqData Applicative Functor
.
We can use getDataFn
to run the RqData
:
> getDataFn :: (HasRqData m, ServerMonad m, MonadIO m) => > RqData a > -> m (Either [String] a)
> module Main where > > import Control.Applicative ((<$>), (<*>)) > import Happstack.Server (ServerPart, badRequest, nullConf, ok, simpleHTTP) > import Happstack.Server.RqData (RqData, look, getDataFn) > > helloRq :: RqData (String, String) > helloRq = > (,) <$> look "greeting" <*> look "noun" > > helloPart :: ServerPart String > helloPart = > do r <- getDataFn helloRq > case r of > (Left e) -> > badRequest $ unlines e > (Right (greet, noun)) -> > ok $ greet ++ ", " ++ noun > > main :: IO () > main = simpleHTTP nullConf $ helloPart
[Source code for the app is here.]
If we visit http://localhost:8000/?greeting=hello&noun=world, we will get our familiar greeting hello, world.
But if we leave off the query parameters http://localhost:8000/, we will get a list of errors:
Parameter not found: greeting Parameter not found: noun
We could use the Monad
instance RqData
to build the request. However, the monadic version will only show us the first error that is encountered. So would have only seen that the greeting
was missing. Then when we added a greeting
we would have gotten a new error message saying that noun
was missing.
In general, improved error messages are not going to help people visiting your website. If the parameters are missing it is because a form or link they followed is invalid. There are two places where there error messages are useful:
If you are providing a REST API for developers to use, they are going to be a lot happier if they get a detailed error messages instead of a plain old 404.
So far we have only tried to look up String
values
using look
. But the RqData
module provides a
variety of ways to work with values besides Strings
.
For some types, it is sufficient to use Read
to parse
the String
into a value. RqData
provides
functions such as lookRead
to assist with this. The advantage of using lookRead
instead of calling look
and applying read
yourself is that lookRead
ties into the RqData
error handling system neatly.
> lookRead :: (Functor m, Monad m, HasRqData m, Read a) => String -> m a
Here is a trivial example where we create a lookInt
function which looks for an Int
parameter named int.
> module Main where > > import Control.Applicative ((<$>), (<*>)) > import Happstack.Server (ServerPart, badRequest, nullConf, ok, simpleHTTP) > import Happstack.Server.RqData (RqData, lookRead, getDataFn) > > lookInt :: RqData Int > lookInt = lookRead "int" > > intPart :: ServerPart String > intPart = > do r <- getDataFn lookInt > case r of > (Left e) -> > badRequest $ unlines e > (Right i) -> > ok $ "Read the int: " ++ show i > > main :: IO () > main = simpleHTTP nullConf $ intPart
[Source code for the app is here.]
Now if we visit http://localhost:8000/?int=1, we will get the message:
Read the int: 1
If we visit http://localhost:8000/?int=apple, we will get the error:
Read failed while parsing: apple
checkRq
Sometimes the representation of a value as a request parameter will be different from the representation required by Read
. We can use checkRq
to lift a custom parsing function into RqData
.
> checkRq :: (Monad m, HasRqData m) => m a -> (a -> Either String b) -> m b
In this example we create a type Vote
with a custom parsing function:
> module Main where > > import Control.Applicative ((<$>), (<*>)) > import Happstack.Server (ServerPart, badRequest, nullConf, ok, simpleHTTP) > import Happstack.Server.RqData (RqData, checkRq, getDataFn, look, lookRead) > > data Vote = Yay | Nay deriving (Eq, Ord, Read, Show, Enum, Bounded) > > parseVote :: String -> Either String Vote > parseVote "yay" = Right Yay > parseVote "nay" = Right Nay > parseVote str = Left $ "Expecting 'yay' or 'nay' but got: " ++ str > > votePart :: ServerPart String > votePart = > do r <- getDataFn (look "vote" `checkRq` parseVote) > case r of > (Left e) -> > badRequest $ unlines e > (Right i) -> > ok $ "You voted: " ++ show i > > main :: IO () > main = simpleHTTP nullConf $ votePart
[Source code for the app is here.]
Now if we visit http://localhost:8000/?vote=yay, we will get the message:
You voted: Yay
If we visit http://localhost:8000/?vote=yes, we will get the error:
Expecting 'yay' or 'nay' but got: yes
checkRq
Looking again at the type for checkRq
we see that
function argument is fairly general -- it is not restricted to just
string input:
> checkRq :: RqData a -> (a -> Either String b) -> RqData b
So, checkRq
is not limited to just parsing a String
into a value. We could use it, for example, to validate an existing value. In the following example we use lookRead "i"
to convert the value i
to an Int
, and then we use checkRq
to ensure that the value is within range:
> module Main where > > import Control.Applicative ((<$>), (<*>)) > import Happstack.Server (ServerPart, badRequest, nullConf, ok, simpleHTTP) > import Happstack.Server.RqData (RqData, checkRq, getDataFn, look, lookRead) > > data Vote = Yay | Nay deriving (Eq, Ord, Read, Show, Enum, Bounded) > > inRange :: (Show a, Ord a) => a -> a -> a -> Either String a > inRange lower upper a > | lower <= a && a <= upper = Right a > | otherwise = > Left (show a ++ " is not between " ++ show lower ++ " and " ++ show upper) > > oneToTenPart :: ServerPart String > oneToTenPart = > do r <- getDataFn (lookRead "i" `checkRq` (inRange 1 10)) > case r of > (Left e) -> > badRequest $ unlines e > (Right i) -> > ok $ "You picked: " ++ show i > > main :: IO () > main = simpleHTTP nullConf $ oneToTenPart
[Source code for the app is here.]
Now if we visit http://localhost:8000/?i=10, we will get the message:
$ curl http://localhost:8000/?i=10 You picked: 10
But if we pick an out of range value http://localhost:8000/?i=113, we will get the message:
$ curl http://localhost:8000/?i=113 113 is not between 1 and 10
Sometimes query parameters are optional. You may have noticed that the
RqData
module does not seem to provide any functions for
dealing with optional values. That is because we can just use the Alternative
class from Control.Applicative
which provides the function optional
for us:
> optional :: Alternative f => f a -> f (Maybe a)
Here is a simple example where the greeting
parameter is optional:
> module Main where > > import Control.Applicative ((<$>), (<*>), optional) > import Happstack.Server (ServerPart, look, nullConf, ok, simpleHTTP) > > helloPart :: ServerPart String > helloPart = > do greet <- optional $ look "greeting" > ok $ (show greet) > > main :: IO () > main = simpleHTTP nullConf $ helloPart
[Source code for the app is here.]
If we visit http://localhost:8000/?greeting=hello, we will get Just "hello"
.
if we leave off the query parameters we get http://localhost:8000/, we will get Nothing
.
HTTP is a stateless protocol. Each incoming Request
is processed
with out any memory of any previous communication with the
client. Though, from using the web, you know that it certainly doesn't
feel that way. A website can remember that you logged in, items in
your shopping cart, etc. That functionality is implemented by using
Cookies
.
When the server sends a Response
to the client, it can include a special Response
header named Set-Cookie
, which tells the client to remember a certain Cookie
. A Cookie
has a name, a string value, and some extra control data, such as a lifetime for the cookie.
The next time the client talks to the server, it will include a copy of the Cookie
value in its Request
headers. One possible use of cookies is to store a session id. When the client submits the cookie, the server can use the session id to look up information about the client and remember who they are. Sessions and session ids are not built-in to the HTTP specification. They are merely a common idiom which is provided by many web frameworks.
The cookie interface is pretty small. There are two parts to the interface: setting a cookie and looking up a cookie.
To create a Cookie
value, we use the mkCookie
function:
> -- | create a 'Cookie' > mkCookie :: String -- ^ cookie name > -> String -- ^ cookie value > -> Cookie >
Then we use the addCookie
function to send the cookie to the user. This adds the Set-Cookie
header to the Response
. So the cookie will not actually be set until the Response
is sent.
> -- | add the 'Cookie' to the current 'Response' > addCookie :: (MonadIO m, FilterMonad Response m) => CookieLife -> Cookie -> m () >
The first argument of addCookie
specifies how long the browser should keep the cookie around. See the cookie lifetime section for more information on CookieLife
.
To lookup a cookie, we use some HasRqData
functions. There are only three cookie related functions:
> -- | lookup a 'Cookie' > lookCookie :: (Monad m, HasRqData m) => > String -- ^ cookie name > -> m Cookie > > -- | lookup a 'Cookie' and return its value > lookCookieValue :: (Functor m, Monad m, HasRqData m) => > String -- ^ cookie name > -> m String > > -- | look up a 'Cookie' value and try to convert it using 'read' > readCookieValue :: (Functor m, Monad m, HasRqData m, Read a) => > String -- ^ cookie name > -> m a
The cookie functions work just like the other HasRqData
functions. That means you can use checkRq
, etc.
The following example puts all the pieces together. It uses the cookie to store a simple counter specifying how many requests have been made:
> module Main where > import Control.Monad > import Control.Monad.Trans > import Happstack.Server > import Control.Monad ( msum ) > import Happstack.Server ( CookieLife(Session), ServerPart, addCookie > , look, mkCookie, nullConf, ok, readCookieValue > , simpleHTTP ) > > homePage :: ServerPart String > homePage = > msum [ do rq <- askRq > liftIO $ print (rqPaths rq) > mzero > , do requests <- readCookieValue "requests" > addCookie Session (mkCookie "requests" (show (requests + 1))) > ok $ "You have made " ++ show requests ++ " requests to this site." > , do addCookie Session (mkCookie "requests" (show 2)) > ok $ "This is your first request to this site." > ] > > main :: IO () > main = simpleHTTP nullConf $ homePage
[Source code for the app is here.]
Now if you visit http://localhost:8000/ you will get a message like:
This is your first request to this site.
If you hit reload you will get:
You have made 3 requests to this site.
Now wait a second! How did we go from 1 to 3, what happened to 2? The browser will send the cookie with every request it makes to the server. In this example, we ignore the request path and send a standard response to every request that is made. The browser first requests the page, but it also requests the favicon.ico
for the site. So, we are really getting two requests everytime we load the page. Hence the counting by twos. It is important to note that the browser does not just send the cookie when it is expecting an html page -- it will send it when it is expecting a jpeg, a css file, a js, or anything else.
There is also a race-condition bug in this example. See the cookie issues section for more information.
When you set a cookie, you also specify the lifetime of that cookie. Cookies are referred to as session cookies
or permanent cookies
depending on how their lifetime is set.
The lifetime of a Cookie
is specified using the CookieLife
type:
> -- | the lifetime of the cookie > data CookieLife > = Session -- ^ expire when the browser is closed > | MaxAge Seconds -- ^ expire after the specified number of seconds > | Expires UTCTime -- ^ expire at a specific date and time > | Expired -- ^ expire immediately
If you are intimately familiar with cookies, you may know that cookies have both an expires
directive and a max-age
directive, and wonder how they related to the constructors in CookieLife
. Internet Explorer only supports the obsolete expires
directive, instead of newer max-age
directive. Most other browser will honor the max-age
directive over expires
if both are present. To make everyone happy, we always set both.
So, when setting CookieLife
you can use MaxAge
or Expires
-- which ever is easiest, and the other directive will be calculated automatically.
There is no explicit Response
header to delete a cookie you have already sent to the client. But, you can convince the client to delete a cookie by sending a new version of the cookie with an expiration date that as already come and gone. You can do that by using the Expired
constructor. Or, you can use the more convenient, expireCookie
function.
> -- | Expire the cookie immediately and set the cookie value to "" > expireCookie :: (MonadIO m, FilterMonad Response m) => > String -- ^ cookie name > -> m ()
Despite their apparently simplicity, Cookies
are the source of many bugs and security issues in web applications. Here are just a few of the things you need to keep in mind.
To get an understanding of cookie security issues you should search for cookie security issues and cookie XSS
One important thing to remember is that the user can modify the cookie. So it would be a bad idea to do, addCookie Session (mkCookie "userId" "1234")
because the user could modify the cookie and change the userId at will to access other people's accounts.
Also, if you are not using https
the cookie will be sent unencrypted.
When you call addCookie
the Cookie
will not be available until after that Response
has been sent and a new Request
has been received. So the following code will not work:
> do addCookie Session (mkCookie "newCookie" "newCookieValue") > v <- look "newCookie" > ... >
The first time it runs, look
will fail because the cookie was not set in the current Request
. Subsequent times look
will return the old cookie value, not the new value.
Browsers impose limits on how many cookies each site can issue, and how big those cookies can be. The RFC recommends browsers accept a minimum of 20 cookies per site, and that cookies can be at least 4096 bytes in size. But, implementations may vary. Additionally, the cookies will be sent with every request to the domain. If your page has dozens of images, the cookies will be sent with every request. That can add a lot of overhead and slow down site loading times.
A common alternative is to store a small session id in the cookie, and store the remaining information on the server, indexed by the session id. Though that brings about its own set of issues.
One way to avoid having cookies sent with every image request is to host the images on a different sub-domain. You might issues the cookies to www.example.org, but host images from images.example.org. Note that you do not actually have to run two servers in order to do that. Both domains can point to the same IP address and be handled by the same application. The app itself may not even distinguish if the requests were sent to images
or www
.
In order to calculate the expires
date from the max-age
or the max-age
from the expires
date, the server uses getCurrentTime
. This means your system clock should be reasonably accurate. If your server is not synchronized using NTP
or something similar it should be.
Cookie updates are not performed in any sort of atomic manner. As a result, the simple cookie demo contains a race condition. We get the Cookie
value that was included in the Request
and use it to create an updated Cookie
value in the Response
. But remember that the server can be processing many requests in parallel and the browser can make multiple requests in parallel. If the browser, for example, requested 10 images at once, they would all have the same initial cookie value. So, even though they all updated the counter by 1, they all started from the same value and ended with the same value. The count could even go backwards depending on the order Requests
are received and Responses
are processed.
The mkCookie
function uses some default values for the Cookie
. The Cookie
type itself includes extra parameters you might want to control such as the cookie path, the secure cookie option, etc.