russiantequila

I went to Google IO this past week, and they had a whole long spiel about the wonders of html5. Well... i've gotta admit - some of the stuff looked kinda cool, so i decided to play with it. I wrote a web serivice in bistro that returned a list of fibonacci numbers, and then a corresponding page that requested that list through jquery, and generated a tiling and then drew a fibonacci spiral over it:

The code for both was remarkably simple. On the service side, all i had to do was define a recursive function for generating fibonacci numbers (yes, it's not tail-recursive, i know, but i was lazy), and a simple controller for binding to the appropriate REST method:

Prest-o change-o, we're done.

Now for the javascript. The majority of the code here is a rather weak attempt to figure out the tiling. My math skillZ are kinda rusty, and i know i should have defined objects for the squares, and dealt with alternating corners that way, but this did the trick:

Pretty straightforward, really. Notice the lack of work on getting the F# service up and running - just 4 lines of code and you're good.

I'm stoked.

I threw the whole "project" up under bistro samples - you can find the info here under the Examples tree.

As i'm building out more components of fs bistro, i'm finding more and more interesting tidbits of info. Figure i'll throw some here and hopefully they help someone.

The specific problem i was trying to solve was, given a Type that represents a F# module, pull out all functions marked as ReflectedDefinition, get at their return type (and structure) and be able to invoke them dynamically.

Let's take this step-by-step. Part 1 - Pull out all functions marked as reflected definition. Well, specifically, that means all functions that have their quoted representation stored in the assembly as well. A little-documented method on the Expr class - Expr.TryGetReflectedDefinition will let you do that:

 
let get_module_info (t:System.Type) =
    if not <| FSharpType.IsModule t then []
    else
        t.GetMethods(BindingFlags.Static ||| BindingFlags.Public) |>
        Seq.fold (fun state (m: MethodInfo) ->
                    match Expr.TryGetReflectedDefinition m with
                    | Some ex -> state @ [m, get_parms ex, try_get_return_sig ex]
                    | None -> state) []
 

Specifically, we're taking a class we're assuming to be a Module (side note - modules end up being static classes in the namespace they're marked with, none if not marked. Nested modules are... you guessed it - nested static classes), getting all of its public static methods (that's how functions get represented) and then effectively iterating over all of them, doing some computation when Expr.TryGetReflectedDefinition resolves is Some. TryGetReflectedDefinition will give you an entry point (if there's one to be had) into the expression tree.

The expression tree is actually a very interesting glimpse into how even imperative constructs get realized as functional compositions. Given a function like this

 
[<ReflectedDefinition>]
let public sample3 (ctx: IExecutionContext) (p2: string option) =
    let now_ts = System.DateTime.Now
    let p =
        match p2 with
        | None -> "empty"
        | Some v -> v
 
    ctx.RenderWith ("Test.template")
    (now_ts, p)
 

the expression tree looks like this

Lambda (ctx,
        Lambda (p2,
                Let (now_ts, PropGet (None, System.DateTime Now, []),
                     Let (p,
                          Let (matchValue, p2,
                               IfThenElse (UnionCaseTest (matchValue,Option`1.Some),
                                           Let (v,PropGet (Some (matchValue),
                                                         System.String Item, []),v), Value ("empty"))),
                          Sequential (Call (Some (ctx),
                                            Void RenderWith(System.String),
                                            [Value ("Test.template")]),
                                      NewTuple (now_ts, p))))))

If you take a look at the arguments of Let, you'll see that it's bind site, value and next statement. That type of chaining glues your functions together. But I digress...

So, right now, we have our expression tree and we have a MethodInfo object. That method info is the same type of method handle we would expect from any other static method. That means that we can invoke it much the same way we would -

 
// mi being our method info object, and parm_list being... well... the parameter list
let ret_val = mi.Invoke(null, parm_list)
 

So that's 2 of three. We got our expression tree, we know how to invoke our method, but what are we actually getting back?

Chances are, that's not a question you'll need to answer often, but it's still a useful piece of info. The Quotations.Patterns (at time of writing, f# ctp 1.9.6.2) namespace contains a list of active patterns that match up to the expression "function calls" that we saw in our expression tree. That means that we can recursively pattern match over our expression tree to get at its structure. We can either use specific cases to look for certain patterns

 
/// returns the first non-None result of invoking func with v1 or v2
let choose func v1 v2 =
    match func v1 with
    | Some _ as ex -> ex
    | None -> func v2
 
/// attempts to locate the return type of the given expression tree
let rec try_get_return_sig = function
| Lambda (_,ex) -> try_get_return_sig ex
| Let (_,_,cont) -> try_get_return_sig cont
| IfThenElse (_,t,e) -> choose try_get_return_sig t e
| NewTuple e -> Some <| List.map (fun (ex: Expr) -> ex.ToString(), ex.Type) e
| Sequential (f,s) -> choose try_get_return_sig f s
| TryFinally (tr,fn) -> choose try_get_return_sig tr fn
| TryWith (tr,_,wt,_,fn) -> choose (choose try_get_return_sig fn) wt tr
| Var var as ex -> Some [(ex.ToString(), ex.Type)]
| _ -> None
 

or we can use the ExprShape patterns to traverse the tree without concerning ourselves with the actual components we're navigating. What the piece of code up above is doing, is it's looking for return values that are explicit Tuple creations. I had a need to analyze functions returning tuples and peek inside their definitions to know the names of the components they're returning. This way, the user can return an arbitrary data structure, but i still get my hands on the names of the individual components i'm getting back. The pattern matches that i have listed out are the tree branches that i care about. Lambda, Let, etc can result in the production of a Tuple, and NewTuple is the actual production. Everything else (there's a significant number of other components) does things that i don't care about (like call out to other functions which may not be marked for quotation, etc), so i just ignore it.

that's it.

Alright, so i threw that blurb out there a few days back, with little more than zOMG! THIS IS COOL!!!ONE!11!. I figured a bit more detail wouldn't hurt.

One of the things that has made F# a bit more difficult to learn for me is that outside of the more basic concepts, everything is explained on a rather theoretical level. It makes sense, really - that's where functional came from, but it does make it more difficult to visualize the feature and understand their applicability. Let's see if i can do any better.

So - f# quotations are simply the full representation of the syntax tree of a fragment of code, available run-time. That means that you have access to anything within <@ @> or marked with a ReflectedDefinition attribute. The common use case that you read everywhere is that you can write code in f# and then execute it elsewhere. And that's certainly cool, but it's only one aspect. One that i've found incredibly useful is the ability to gather more information from dynamically loaded code than previously possible.

With standard reflection you get to see the shell. You get member names, method signatures (as an ordered list of types) and inheritance info (there's more, but these are the core elements). However, with quotations, i can see much more. I can get parameter names, i can get local variables i can get the full source of the block i'm evaluating!

My use for this? In bistro, I used to make the developer decorate his class with member attributes that gave me information about how that member was used. Now, I can pull that information straight from a function signature, and not require any additional markup. Here's how:

Sample target function:

 
#light
open Microsoft.FSharp.Reflection
open System.Reflection
 
module NestedProgram =
    module Program =
        [<ReflectedDefinition>]
        let public sample f_name p2 =
            printf "sample {%A}\r\n" f_name
 

Code:

 
#light
 
open System
open System.Reflection
open DefaultControllers.NestedProgram.Program
 
open Microsoft.FSharp.Quotations.Patterns
open Microsoft.FSharp.Quotations.DerivedPatterns
open Microsoft.FSharp.Quotations
 
open Microsoft.FSharp.Reflection
 
let strip (text: string) = match text.IndexOf "@" with | -1 -> text | _ as i -> text.[..i-1]
 
let rec get_parms exp =
    match exp with
    | Lambda (var,body) ->
        [strip var.Name] @ get_parms body
    | _ -> []
 
let get_info (t:System.Type) =
    if not <| FSharpType.IsModule t then failwith <| sprintf "%A is not a module" t
    else
        let funcs = t.GetMethods(BindingFlags.Static ||| BindingFlags.Public)
 
        Array.iter (fun (m: MethodInfo) ->
            match Expr.TryGetReflectedDefinition m with
            | Some ex ->
                printf "\t%s.%s %s\r\n" t.FullName m.Name (String.concat " " (get_parms ex))
            | None -> ()) funcs
 
let rec searchTypes t =
    if FSharpType.IsModule t then get_info t
 
AppDomain.CurrentDomain.GetAssemblies() |>
Array.iter (fun (asm: Assembly) ->
                printf "asm: %s\r\n" <| asm.GetName().Name
                Array.iter searchTypes (asm.GetTypes())
                )
 
ignore <| System.Console.Read()
 

This code will go through the entire loaded assembly list, find all types and give me the function signatures. How i consume that is now up to me, but it's there for the taking.

As a side note - I'm going to try and blog more about the real-world applicability of f#. I think that the notion of f# or functional programming not being applicable to web development and the like is a misconception, and hopefully i'll be able to illustrate that.

I've been spending some time on this bistro stuff, and it's ending up being rather interesting. Let's not go into details of the MVC stuff. That's all fairly standard - render with a Velocity port (NVelocity), controllers do your dirty work, blah, blah, blah. Let's get to the fun stuff. I started by saying that this whole thing makes sense because of REST and AOP. And that's definitely the case:

 
namespace NoRecruiters3.Controllers.Content
{
    [Bind("/content/resume/{resumeId}")]
    public class Resume: AbstractController
    {
        protected string resumeId;
 
        public override void ProcessRequest(HttpContext context, IContext requestContext)
        {
            Posting resume = PostingHelper.Instance.LoadPosting(null, resumeId);
            requestContext.Add("resume", resume);
            requestContext.Add("contentType",
                        ContentType.Resume.Equals(
                                    ContentTypeHelper.Instance.GetLastDefault()) ? "resume" : "ad");
            requestContext.RenderWith = "/Templates/Resume/view.bistro";
        }
    }
}

You can see the explicit bind statement (those are optional, btw. if you strip away the namespace prefix of NoRecruiters3.Controllers you get Content.Resume as your class name, so it would be invoked that way). So when we perform a GET on resource /content/resume/some_resume_id, we invoke this controller with (resumeId = some_resume_id) .

On the interception side I also have my security stuff:

 
namespace NoRecruiters3.Controllers.Content
{
    [Bind("/content/resume")]
    public class ResumeAccessValidator: AbstractController
    {
        public override void ProcessRequest(HttpContext context, IContext requestContext)
        {
            if (!User.Authenticated)
                requestContext.Transfer("/action/login");
        }
    }
}

You can see that we're intercepting all method calls (let's talk in those terms) and redirecting elsewhere when some auth condition isn't met. Cool. We've got our classic AOP example.

Here's the tangent. As i started porting my app to it, i saw an interesting opportunity. Let's take the search example. I've got a search controller that's invoked by /action/search/{contentType}:

 
[Bind("/action/search/{contentType}")]
[DependsOn("currentTags")]
public class Search : AbstractController
{
    protected string contentType;
 
    public override void ProcessRequest(HttpContext context, IContext requestContext)
    {
        prepareSearch(context, requestContext);
        prepareTags(context, requestContext);
 
        requestContext.RenderWith = "/Templates/Search/search.bistro";
    }
}

Okay. Now, let's factor in that I also want to be able to search by tags, in addition to the query. Well, there's lots of ways of doing that, but what about... overloading our url? Let's try this: /action/search/{contentType}/with-tag/{tag}:

 
namespace NoRecruiters3.Controllers.Actions
{
    [Bind("/action/search/{contentType}/with-tag/{tagList}")]
    [Provides("currentTags")]
    public class Tag : AbstractController
    {
        protected string tagList;
 
        public override void ProcessRequest(HttpContext context, IContext requestContext)
        {
            List currentTags = (List)context.Session["currentTags"];
 
            if (currentTags == null)
            {
                currentTags = new List();
                context.Session.Add("currentTags", currentTags);
            }
 
            foreach (string tag in tagList.Split(','))
                if (!currentTags.Contains(tag))
                    currentTags.Add(tag);
        }
    }
}

Check that out. We now have a separate controller that deals with managing selected tags on the session. So what about search? Well the way our url matching rules are going to work, is that we'll actually match both - the search controller will be invoked in addition to the tag controller. So with that one URL we kill both birds.

It's not really AOP, but ... then it kind of is. We can argue that tag management is a cross-cutting concern. The interesting thing here is that we can organize our controllers into very concise, almost atomic operations.  The only remaining question is this - we have a rule of most-generic-first, i.e. the most generic match gets executed the first. In our example, the search controller is more generic than the tag. But the tag modifies data that the search needs to use. That's when the requires/dependsOn/provides attributes come in. They tell the system that the controller has input/output dependencies, and the system makes sure that controllers are executed in an order that will satisfy those dependencies.

Yet more later...

I mentioned it in passing a few times, but I've been playing around with a twist on some of the main-stream technologies in the web world. REST, AOP and template-driven MVC. I've been playing around with this on the no-recruiters platform, which I'm using as a development guinnea pig. It's a live site, the traffic is (for now) low, and so it both makes me drive it to compeltion and gives me a chance to fiddle and figure out how a framework is from the usability side.

I've been calling it Bistro (that's what happens when developers try their hands at marketing). Bistro is actually the russian word for "quick", and so the idea is that this another extension for workflow server that lets you quickly build applications.

Back to the idea. The concept is straight-forward enough.

• Build applications where URLs are effectively method invocations.
• Allow an AOP-style syntax for defining cross-cutting controllers that service these method invocations.
• Move away from MSFT's client-server abstraction model aka WebForms, and use a templating engine. Right now it's the NVelocity port (the original, not the Castle fork). If this gains traction, there are a few features I'd like to implement (I like the template inheritance model in Django, etc), so I'd like the freedom of an independent fork.
• Build this on top of workflow to take advantge of the VS.NET integration, the ORM pieces, and the logging facilities.

Example (and I'll use some of the no-recruiters (aka nr) nomenclature throughout).

I have a function called search. My search method is going to be /content/search/resume. This method needs a few things to happen (yay, classic AOP examples). First, we need to log this method call. Second, we need to make sure that you have access to perform this method call. Third, we need to prepare the search results, capitalizing on the fact that the difference between a resume search and an ad search is a parameter, and finally we need to render the results.

High level, this breaks down as follows

• Define a controller called "AccessValidator" and bind it to the / context.
• Define another controller, "RequestLogger" and also bind it to the / context.
• Define a controller "Search", and bind it to the /content/search/{searchType} context (I'll touch upon parameterized contexts in a second)
• Create a template file /content/search/resume (this assumes that we've addressed the IIS no-extension URL rewrite issue)

A few explanations. Binding - just like it sounds. We define that a given controller is executed when a url matches its bind point. Bind points can have wild-cards, but if they don't, are root-relative URL fragments. Arbitrary elements can be parameterized, and parameterized elements are passed into matching controllers as parameters. Also, the binding definition lets you control the order in which things are run (think of point-cut definitions - before, insteadof, etc).

So in the example, here's how the processing would go.

1. Request comes in for /content/search/resume
2. AccessValdiator is called
3. RequestLogger is called
4. Search controller is called, given a parameter of searchType = 'resume'. This controller performs the search, and places the results onto a context variable.
5. /content/search/resume.vm (or .bistro) template is pulled up and rendered, using the search results present in the context variable.

I'll be the first to admit that a lot of this is a game of semantics, but in larger systems semantics mean quite a bit. Additionally, much of the advantages of AOP apply, without the major disadvantage of breaching OO Encapsulation.

The utility of REST here is to make AOP useful. By definining meaningful, method-like URLs, resultinng applications become much more susceptible to AOP. Point-cuts become a lot more general and the definitions are more concise. Additionally, we solve the problem of "i don't want to encode template names in my code". The common problem of how do I transfer control without being dependent on what the target is. In this nomenclature, you don't deal with specific templates, you are just requesting that an action be performed. What that action means, and how it's represented to the end user is up to how the action is implemented.

 More to come...