russiantequila

It's been a while (seems like my writing is slightly manic - either all or nothing), but i actually have something to write about. I finally got a chance to clean up the FSharpExtensions project for bistro (you can find the new release here). While the underlying concepts are the same, a recent stint with Django (which is a strong influence in the Bistro framework) reminded me of how little work you have to do in python to get going with a controller, and that bistro and the fs extensions still have some catching up to do on the brevity front.

So here's what's new (from the download page)

Return values can now contain expressions, using the 'named' aliasing function: given function foo(), "foo() |> named 'bar'" will place the result of calling 'foo' onto the request context under the name 'bar'

Translation: before you had to

but now, you can

Session values do not need aliasing - the statement 'SessionValue foo' will place 'foo' onto the session context directly

Which means this will work without the 'named' function shown above

The session and request context namespaces have now been merged. This means that it is no longer necessary to qualify function input parameters with the session discriminator, as data will be pulled from wherever it is available. Return values should still use the SessionValue discriminator to specify that something should go onto the session

This one is a bit trickier and has some ramifications (good and bad).

You can see that I didn't need to use the old-style syntax of having to specify the type of something in my second controller (it would have been (something: string session) before). The drawback of this is that you now can have a collision between the two namespaces, which is bad. There are two considerations here - one, the order of processing is deterministic. Request context always wins, as bistro apps tend to be light on the session. Two, my work with NDjango (which has a similar limitation) has shown me how little I run into that, so it's a limitation i'm willing to live with, seeing as how it drops the need for funky types and wrapping/unwrapping.

A new syntax for form fields is now available. If a controller function takes a record type as a parameter, and that record type is marked with the 'FormData' attribute, the fields of that record are populated from the form data.

This one is pretty cool. Say I have a form (with ndjango syntax):

and a corresponding controller. The old way of handling this form would have been

the new way, however, is a bit neater:

All you do is declare a record that defines the fields and data types you care about, and work with that record. What's more, is that by default (this is an overridable parameter to the FormData attribute) the fields are automatically passed through to the request context without your intervention, so you don't have to repeat the field list twice for no good reason.

Neat, huh? All of these changes has taken it to the point where you almost never need type declarations, and the compiler can typically infer the necessary data types.

On the über-geeky side of this, the most complex part of all of this was actually recognizing and parsing out calls to the 'named' function. It took the inference code from this:

to this:

I'm sure there's room for improvement, but a lot of work comes in recognizing all the different ways the quotation tree can be built. The unit tests show all the different invocations. Let's just say that part took some digging...

I went to Google IO this past week, and they had a whole long spiel about the wonders of html5. Well... i've gotta admit - some of the stuff looked kinda cool, so i decided to play with it. I wrote a web serivice in bistro that returned a list of fibonacci numbers, and then a corresponding page that requested that list through jquery, and generated a tiling and then drew a fibonacci spiral over it:

The code for both was remarkably simple. On the service side, all i had to do was define a recursive function for generating fibonacci numbers (yes, it's not tail-recursive, i know, but i was lazy), and a simple controller for binding to the appropriate REST method:

Prest-o change-o, we're done.

Now for the javascript. The majority of the code here is a rather weak attempt to figure out the tiling. My math skillZ are kinda rusty, and i know i should have defined objects for the squares, and dealt with alternating corners that way, but this did the trick:

Pretty straightforward, really. Notice the lack of work on getting the F# service up and running - just 4 lines of code and you're good.

I'm stoked.

I threw the whole "project" up under bistro samples - you can find the info here under the Examples tree.

so the second i post up the ndjango port, the new f# ctp hits the web... nice timing on my part . well, i can take a hint. i spent some time yesterday and branched off a new fork in the svn for a version that works against the new ctp. i still have to spend some time playing with it, but at first glance, the old code worked. There's quite a bit of library name change (moving away from ocaml naming conventions it seems like), but once the renaming was done, all of our unit tests ran just fine. I have to admit that i've grown used to the underscore notation for f# code, and seeing camel case throughout will take some getting used to. Also - I'm starting to see some functions renamed in a way that's less clear than before... hopefully the f# team won't fall into the trap of c# v java where some classes and methods were named differently just for the sake of being different (*cough stringbuilder/stringbuffer cough*). And why do away with the String module?

On another note, we updated the ndjango integrations projects to include a hook for asp.mvc. You should be able to use ndjango with it now, by just declaring a variable in your Global.asax and then instantiating it on app start. That'll load up the right dlls and hook in where necessary.

i'm going to be speaking about f# and applicability outside the lab in a few days (see here), so i figured i'd put this up here as part of my prep for the meeting. so - 10 cool things about f# as a .net/oo language

1. #light syntax. Indentation based syntax, while i'm sure viewed as the bane of our existence, is amazingly easy to deal with, given a proper editor and assistance. Yes, sometimes it's quirky, but not having to look at 40 pages of curly braces is worth it. You simply end up writing less code, seeing more code per screenful, and writing code that's easier to read.
2. type inference. F# is the ultimate combination - it's a statically typed language, that reads like a dynamic one. That's because the F# compiler infers value and parameter types from their usage. There are mechanisms to specify data types, but unless you're doing funky reflective programming, you don't need them often. Most of the time, the compiler will hit it spot on. That means that this works

    
   let f x y = x + y
   // val f : int -> int -> int
    

   but so does this

    
   let f2 (x: float) y = x + y
   // val f2 : float -> float -> float
    

3. record types. Or rather how you consume them. Records are compound data structures (kind of like structs). With a record definition of

    
   type SomeRecord = {
       first_name: string
       last_name: string
       dob: DateTime
   }
    

   you can get an instance by just writing

    
   let alex = { first_name = "alex"; last_name = "p"; dob = (System.DateTime.Parse("01/01/1900")) }
    

   Cool, huh? Notice i didn't have to specify what the type of alex is. It's actually inferred from the record labels i supplied in the expression. What's more, is that i can now create a clone of that record:

    
   let alexs_twin = { alex with first_name = "twin" }
    

4. object expressions. Ever have a piddly little object that you needed to return? Ever wonder why you have to go through the hoopla of defining a class for something that'll never be used outside of where it's declared? Well, in f#, you don't. This example comes from a parser implementation for a markup language. It was broken up into a tag class, and a node class. Tag class did the parsing, node was the AST node representation, and typically did very little work, with most of the functionality being defined in the base clas Node. This specific tag translates words into template syntax:

    
   type TemplateTag() =
       interface ITag with
           member this.Perform token parser tokens =
               let args = List.tl <| smart_split token
               let buf =
                   match args with
                       | "openblock"::[] -> "{%"
                       | _ -> raise (TemplateSyntaxError ("invalid format for 'template' tag"))
               [{
                   new Node(Block "templatetag")
                   with
                       override this.walk walker =
                           {walker with buffer=Some buf}
               }], tokens
    

   What this little code snippet does is rather than explicitly defining a new class TemplateNode that just overrides the "walk" method on Node, it combines the instantiation and definition steps, and returns an anonymous subclass of node, with the "walk" method redefined to do something else. Now, you can take this even further, and use object expressions to define anonymous interface implementations. Say you wanted to make sure that when an operation was done, some resource was released, but that's all you cared about. Simple - use an object expression to implement the IDisposable interface, and you're done:

    
   let disp = { new Object() interface IDisposable with member x.Dispose() = (*do stuff*) }
    

   Side note here - object expressions make a lot of sense when coupled with the fact that F# has the concept of closures, but that's more functional so just a quick blurb on this. A closure means that as you define your instance/type combination, you have access to local values of all of your parent scopes. That's why, in the TemplateTag example, my Node class implementation can access the local value "buf" from within it's method definition.

5. tuples. Ever need to return 2 values from a function? Okay, you do a return value and an output parameter. Annoying to consume, but doable. But what about 3? or 4? At that point it gets to be just unmanageable. In F# this is simple. A tuple is a composition of multiple other types. If i have a function (or method, to stay with our theme) that took my "SomeRecord" from before, and returned the first and last names only, it would look like this

    
   member x.get_names record = record.first_name, record.last_name
    

   That's it! Now, when i consume it, i can either treat it as a single value of type tuple, and then use special accessor functions to read from it:

    
   let names = someclass.get_names alex
   printf "first name %s\r\n" <| fst names
   printf "last name %s\r\n" <| snd names
    

   or using f# syntactical sugar to unfold the value explicitly:

    
   let first, last = someclass.get_names alex
   printf "first name %s\r\n" <| first
   printf "last name %s\r\n" <| last
    

   or ignore one of the values entirely:

    
   let _, last = someclass.get_names alex
   printf "last name %s\r\n" <| last
    

6. discriminated unions. When the set of values you care about depend on what you're representing, discriminated unions are awesome. Example here is - if you wanted to have a single data type that represented a vehicle, but wanted to store different attributes for different vehicle types, you could define a discriminated union that stores passenger capacity and Color for passenger cars, and maximum load and number of wheels for trucks:

    
   type Vehicle =
   | Passenger of int * Color
   | Truck of float * int
    

7. null value handling. F# doesn't really have the concept of a null. I mean it does, but it mostly comes up when interfacing with external libraries that'll slip one in here or there. Outside of that, if you say that method someclass.get_names takes a record of type SomeRecord, you can be sure that you'll never get a null there. Now, if you want to allow for that, you can use the "Option" type. Option is a discriminated union of

    
   type Option<'a> =
   | None
   | Some of 'a
    

   which means that it either contains "None", or Some value of type 'a (the whole ' thing is to identify a generic type parameter, so the Option type is a generic type). That means that whenever you're dealing with values that may not be supplied, you have to explicitly handle that scenario. That also means that the pesky NullReferenceException goes bye-bye.

8. lists, list literals and other fun built-in structures. We're starting to get closer to f#'s functional nature, but this is still very much oo-applicable. Lists. F# has awesome support for lists. For starters, you can do stuff like this:

    
   // new list, containing 0 through 5
   let l = [0..5]
   // new list, concatenating l with the list containing 6
   let l2 = l @ [6]
   // h is the head element in l, t is the remainder
   let h::t = l
   // pull the first three elements out into h1,2,3 respectively
   let h1::h2::h3::t = l2
    

   then you can move on to generating expressions

    
   // 10 squares
   let l3 = [for i in 0..10 -> i * i]
    

   The other thing is that default lists in F# are immutable structures, so when you modify them, you're actually creating new instances that contain the modified values. But fear not - there's no copying involved. Since each instance is immutable, f# can share elements among instances, allowing the operation

   some_giant_list @ [0]

   to not have to copy the contents of some_giant_list. Oh and with arrays, you can also do slicing notation:

    
   // generate array of 0 through 5
   let a = [|0..5|]
   // get the first 4 elements
   let a2 = a.[..3]

   .

9. pattern matching. We're getting closer and closer to functional territory, but with f# that line is somewhat blurred. Pattern matching is effectively a very concise way to write a long if-then-elseif. You can

    
   // match over literals
   match alex.first_name with
   | "alex" -> printf "woo"
   | _ as name -> printf "don't know %s" name
    
   // and over lists
   match l with
   | h::t -> printf "the first element is %d, and there are %d in the tail" h <| List.length t
   | [] -> printf "the list is empty"
    
   // and over Types (there shouldn't be a space between the : and the ?)
   match disp with
   | : ? System.ICloneable as cloneable -> printf "%A is clonable" cloneable
   | : ? System.IDisposable as disposable -> printf "%A is disposable" disposable
   | _ as unk -> printf "dunno what %A is" unk
    

   you can also pattern match over discriminated unions (that's actually the main way of working with those), and various other data structures.

10. access to .net libraries. Okay, i'm cheating a little here, because the remarkable part isn't that there's access to the .net library, but that a functional, statically-typed language has access to the .net library. The fact that I can reach out to components written in c#, with no penalty, is what will make this language more mainstream than most other functional implementations (scala and the like, of course, notwithstanding).

It's been a few days longer than I'd like, but now I have something worth sharing. I bounce back and forth between more framework-oriented work and more application-oriented work, but they do end up being two sides of the same coin. Here I want to talk about framework stuff. Oh and a quick disclaimer - this is more thinking out loud, and less a final solution. Input, as always, is most welcome.

In the oo world, your common unit of function is an object. An object may do more than one thing, but it typically does at least one thing, has at least one "function" (not in the programming sense, but in the "i do something" sense). As such, the problem of storing meta information on your object has long been solved - attributes. If I have an object with data elements on it (fields or properties), then I can encode information for a consuming runtime by applying attributes to it. It is a fairly common design pattern for an object that functions as a component in a larger framework (think any ORM framework as an example) to encode information on its data elements, have the runtime read that information, and populate/read from those elements, and then invoke some method on the object.

In f#, on the other hand, your common unit of function is ... a function. Unlike an object, a function typically does one thing (though a function with side effects most certainly may do more than one). Now, I do still have the concept of attributes in f#, and i can decorate my function with multiple attributes, but there's a slight problem. To implement the design pattern I mentioned before (fields marked by attributes, populated by the runtime), I have to have two things - locally-scoped data elements that are mutable externally. The only way to do that is to implement a class, with mutable properties just like i would in c#. This means that I lose my functional advantage, and now am using f# as a different dialect of an oo language. Feasible, but not pleasant. Or necessary.

Let's take a different approach. I'm going to bring back a piece of functionality i talked about in earlier articles - quotations, and using quotations to determine function signatures. I won't go into those details again, but suppose i have a function like this

 
[<ReflectedDefinition>]
let public sample2 (l_name: string) =
    let f_name = "alex"
    (f_name, l_name)
 

I've shown before that i can use quotations to get at the function signature, and extract the type, name, and position of every parameter to this function. This means that my fictitious runtime, upon learning of this function, can find out that function "sample2" take a parameter "l_name" of type "string". Well that certainly a start. But what happens when I want to pass along more information? For example, what happens when I want to tell the runtime that value l_name will never be the empty string (yay for more contrived examples). Before we answer that question, let's answer a slightly simpler one - how do you handle nulls in f#? Well - we use the Option type, which is effectively a discriminated union, with discriminators None (denoting ... well ... no value), and Some v, denoting that it does contain a value, and the value is v. Option is a generic type, so we could write it as "l_name Option<string>". Or, better yet, we can write it as "l_name string option". Cool. It's succinct, and provides information. Now, since Option is a type, our runtime will see that function sample2 has a single parameter l_name, of type Option<string>, which tells it that it's a string value that may be null.

Okay, back to the original question - how do i inform the runtime of something else - like the string parameter will never be empty? Well, we can use the same thing - we can define our own discriminated union, say... NonEmpty, and change our function signature to be "l_name string NonEmpty". What's even more interesting is that you can compose these values - i can say that the string is optional, but if present, then non empty - "l_name string option NonEmpty" or "l_name string NonEmpty option". So now we have a way of attaching meta information (of sorts) to our parameter values, and it mostly doesn't get in the way.

There are, of course, a few drawbacks and limitations. Two jump to mind - first, these markers aren't parametrizable - meaning that while an actual attribute will let you define parameters that carry the meta information (an attribute for an orm that will denote a field mapping will let you specify the database field name as a string parameter), this approach only allows for marker attributes. Second, while not horribly obtrusive, you do introduce the need to "unwrap" your value. If you do have the example above of l_name string option NonEmpty, you have to do something like l_name.Value.Value to get at the actual l_name.

I don't think either of these are showstoppers. The first one, you can supplement the marker attributes with regular attributes if you must have arbitrary parameters. The second one, you can write a generic decomposition function, something like

 
/// annotates a session-scoped value
type session<'a> =
    | SessionValue of 'a
 
    member x.Value = match x with SessionValue a -> a
 
/// annotates a form-scoped value
type form<'a> =
    | FormValue of 'a
 
    member x.Value = match x with FormValue a -> a
 
/// recursively retrieves values from an arbitrarily-annonated value
let rec get_value (v: obj) =
    match v with
    | : ? session<_> as ssn -> get_value ssn.Value
    | : ? form<_> as frm -> get_value frm.Value
    | : ? Option<_> as opt ->
        match opt with
        | Some v -> get_value v
        | None -> null
    | _ -> v
 

Edit: this actually ends up not working. The type parameters in the type tests get constrained to obj, meaning that you're testing an arbitrary data type against (as an example) session, which will only pass for session, and nothing else. I'm working on ways around this and will update when i found something.

Do note that one drawback of get_value is that it is forced to restrict v to be of type obj, and as a result, restricts the return value to be of type obj as well, forcing casts when consuming. I'm still playing with this, and will post an update if i find a better way.

Finally, on the framework side, reading this information from a function signature is actually not horribly difficult either. (Caveat - this implementation is a simple proof-of-concept, and as such isn't overly extensible. You'll notice that the list of possible parameters is hard-coded a tuple of fixed order and length. That can be replaced with a different data structure, but works for now). This sample takes a function signature and uses discriminated union markers to infer scope and directionality of parameters to a given function.

 
/// active pattern that is matched when input is a generic type whose definition is pattern_type
let (|MatchGenericType|_|) (pattern_type: Type) (input: Type) =
    if input.IsGenericType && input.GetGenericTypeDefinition() = pattern_type then
        Some <| input.GetGenericTypeDefinition()
    else
        None
 
/// active pattern that is matched when input is an option type
let (|MatchOptionType|_|) (input: Type) =
    if input.IsGenericType && input.GetGenericTypeDefinition() = optiont then
        Some <| input.GetGenericTypeDefinition()
    else
        None
 
/// gets the generic type parameter of this type. note that this function
/// assumes that the supplied type is a specific definition of a generic
/// type, and that it has at least one type parameter
let get_generic (t: Type) = t.GetGenericTypeDefinition().GetGenericArguments().[0]
 
/// recursively peels off outer markers of scope/directionality
let rec decompose state (a: string * Type) =
    let name, parm_type = a
    let form_list, depends, requires, session, request = state
 
    if parm_type.IsGenericType then
        (name, get_generic parm_type) |>
            decompose
                (match parm_type with
                | MatchOptionType opt_type ->
                    form_list, depends @ [name], requires, session, request
                | MatchGenericType sessiont ssn_type ->
                    form_list, depends, requires, session @ [name], request
                | MatchGenericType formt form_type ->
                    form_list, depends, requires, session @ [name], request
                | _ -> raise (new ApplicationException(sprintf "%A is not a known resource marker" parm_type)))
    else
        // if this value has been marked by any of these, then we do nothing
        // otherwise, we assume it to be a requires request value
        if any_have name [session; form_list] then state
        elif any_have name [depends] then
            form_list, depends, requires, session, request @ [name]
        else state
 
/// builds a list of form, depends, requires, session and request fields
/// based on a function signature. signature is a list of string * Type
/// describing the input and output parameters of the function we're dealing with.
let load signature =
    List.fold_left (decompose parameter_fields) ([],[],[],[],[]) signature
 

Note - i hacked this out of a live project, and I'm sure there are some dangling references. I can provide the full (compiling) code to this for those interested.

That's it for now.

As i'm building out more components of fs bistro, i'm finding more and more interesting tidbits of info. Figure i'll throw some here and hopefully they help someone.

The specific problem i was trying to solve was, given a Type that represents a F# module, pull out all functions marked as ReflectedDefinition, get at their return type (and structure) and be able to invoke them dynamically.

Let's take this step-by-step. Part 1 - Pull out all functions marked as reflected definition. Well, specifically, that means all functions that have their quoted representation stored in the assembly as well. A little-documented method on the Expr class - Expr.TryGetReflectedDefinition will let you do that:

 
let get_module_info (t:System.Type) =
    if not <| FSharpType.IsModule t then []
    else
        t.GetMethods(BindingFlags.Static ||| BindingFlags.Public) |>
        Seq.fold (fun state (m: MethodInfo) ->
                    match Expr.TryGetReflectedDefinition m with
                    | Some ex -> state @ [m, get_parms ex, try_get_return_sig ex]
                    | None -> state) []
 

Specifically, we're taking a class we're assuming to be a Module (side note - modules end up being static classes in the namespace they're marked with, none if not marked. Nested modules are... you guessed it - nested static classes), getting all of its public static methods (that's how functions get represented) and then effectively iterating over all of them, doing some computation when Expr.TryGetReflectedDefinition resolves is Some. TryGetReflectedDefinition will give you an entry point (if there's one to be had) into the expression tree.

The expression tree is actually a very interesting glimpse into how even imperative constructs get realized as functional compositions. Given a function like this

 
[<ReflectedDefinition>]
let public sample3 (ctx: IExecutionContext) (p2: string option) =
    let now_ts = System.DateTime.Now
    let p =
        match p2 with
        | None -> "empty"
        | Some v -> v
 
    ctx.RenderWith ("Test.template")
    (now_ts, p)
 

the expression tree looks like this

Lambda (ctx,
        Lambda (p2,
                Let (now_ts, PropGet (None, System.DateTime Now, []),
                     Let (p,
                          Let (matchValue, p2,
                               IfThenElse (UnionCaseTest (matchValue,Option`1.Some),
                                           Let (v,PropGet (Some (matchValue),
                                                         System.String Item, []),v), Value ("empty"))),
                          Sequential (Call (Some (ctx),
                                            Void RenderWith(System.String),
                                            [Value ("Test.template")]),
                                      NewTuple (now_ts, p))))))

If you take a look at the arguments of Let, you'll see that it's bind site, value and next statement. That type of chaining glues your functions together. But I digress...

So, right now, we have our expression tree and we have a MethodInfo object. That method info is the same type of method handle we would expect from any other static method. That means that we can invoke it much the same way we would -

 
// mi being our method info object, and parm_list being... well... the parameter list
let ret_val = mi.Invoke(null, parm_list)
 

So that's 2 of three. We got our expression tree, we know how to invoke our method, but what are we actually getting back?

Chances are, that's not a question you'll need to answer often, but it's still a useful piece of info. The Quotations.Patterns (at time of writing, f# ctp 1.9.6.2) namespace contains a list of active patterns that match up to the expression "function calls" that we saw in our expression tree. That means that we can recursively pattern match over our expression tree to get at its structure. We can either use specific cases to look for certain patterns

 
/// returns the first non-None result of invoking func with v1 or v2
let choose func v1 v2 =
    match func v1 with
    | Some _ as ex -> ex
    | None -> func v2
 
/// attempts to locate the return type of the given expression tree
let rec try_get_return_sig = function
| Lambda (_,ex) -> try_get_return_sig ex
| Let (_,_,cont) -> try_get_return_sig cont
| IfThenElse (_,t,e) -> choose try_get_return_sig t e
| NewTuple e -> Some <| List.map (fun (ex: Expr) -> ex.ToString(), ex.Type) e
| Sequential (f,s) -> choose try_get_return_sig f s
| TryFinally (tr,fn) -> choose try_get_return_sig tr fn
| TryWith (tr,_,wt,_,fn) -> choose (choose try_get_return_sig fn) wt tr
| Var var as ex -> Some [(ex.ToString(), ex.Type)]
| _ -> None
 

or we can use the ExprShape patterns to traverse the tree without concerning ourselves with the actual components we're navigating. What the piece of code up above is doing, is it's looking for return values that are explicit Tuple creations. I had a need to analyze functions returning tuples and peek inside their definitions to know the names of the components they're returning. This way, the user can return an arbitrary data structure, but i still get my hands on the names of the individual components i'm getting back. The pattern matches that i have listed out are the tree branches that i care about. Lambda, Let, etc can result in the production of a Tuple, and NewTuple is the actual production. Everything else (there's a significant number of other components) does things that i don't care about (like call out to other functions which may not be marked for quotation, etc), so i just ignore it.

that's it.

Alright, so i threw that blurb out there a few days back, with little more than zOMG! THIS IS COOL!!!ONE!11!. I figured a bit more detail wouldn't hurt.

One of the things that has made F# a bit more difficult to learn for me is that outside of the more basic concepts, everything is explained on a rather theoretical level. It makes sense, really - that's where functional came from, but it does make it more difficult to visualize the feature and understand their applicability. Let's see if i can do any better.

So - f# quotations are simply the full representation of the syntax tree of a fragment of code, available run-time. That means that you have access to anything within <@ @> or marked with a ReflectedDefinition attribute. The common use case that you read everywhere is that you can write code in f# and then execute it elsewhere. And that's certainly cool, but it's only one aspect. One that i've found incredibly useful is the ability to gather more information from dynamically loaded code than previously possible.

With standard reflection you get to see the shell. You get member names, method signatures (as an ordered list of types) and inheritance info (there's more, but these are the core elements). However, with quotations, i can see much more. I can get parameter names, i can get local variables i can get the full source of the block i'm evaluating!

My use for this? In bistro, I used to make the developer decorate his class with member attributes that gave me information about how that member was used. Now, I can pull that information straight from a function signature, and not require any additional markup. Here's how:

Sample target function:

 
#light
open Microsoft.FSharp.Reflection
open System.Reflection
 
module NestedProgram =
    module Program =
        [<ReflectedDefinition>]
        let public sample f_name p2 =
            printf "sample {%A}\r\n" f_name
 

Code:

 
#light
 
open System
open System.Reflection
open DefaultControllers.NestedProgram.Program
 
open Microsoft.FSharp.Quotations.Patterns
open Microsoft.FSharp.Quotations.DerivedPatterns
open Microsoft.FSharp.Quotations
 
open Microsoft.FSharp.Reflection
 
let strip (text: string) = match text.IndexOf "@" with | -1 -> text | _ as i -> text.[..i-1]
 
let rec get_parms exp =
    match exp with
    | Lambda (var,body) ->
        [strip var.Name] @ get_parms body
    | _ -> []
 
let get_info (t:System.Type) =
    if not <| FSharpType.IsModule t then failwith <| sprintf "%A is not a module" t
    else
        let funcs = t.GetMethods(BindingFlags.Static ||| BindingFlags.Public)
 
        Array.iter (fun (m: MethodInfo) ->
            match Expr.TryGetReflectedDefinition m with
            | Some ex ->
                printf "\t%s.%s %s\r\n" t.FullName m.Name (String.concat " " (get_parms ex))
            | None -> ()) funcs
 
let rec searchTypes t =
    if FSharpType.IsModule t then get_info t
 
AppDomain.CurrentDomain.GetAssemblies() |>
Array.iter (fun (asm: Assembly) ->
                printf "asm: %s\r\n" <| asm.GetName().Name
                Array.iter searchTypes (asm.GetTypes())
                )
 
ignore <| System.Console.Read()
 

This code will go through the entire loaded assembly list, find all types and give me the function signatures. How i consume that is now up to me, but it's there for the taking.

As a side note - I'm going to try and blog more about the real-world applicability of f#. I think that the notion of f# or functional programming not being applicable to web development and the like is a misconception, and hopefully i'll be able to illustrate that.

While playing around with the bistro stuff, i've been slowly investigating other concepts that have been floating around over the last few years. One of them is F# (start here and go from there). I've spent a few weeks wrapping my brain around the rather orthogonal concept of functional programming, and followed someone's advice in trying to learn about Haskell first (the exact quote that sold me on the idea was "trying to learn F# without knowing Haskell is like trying to learn VB without knowing C#"). That approach has been immensely helpful, as it's introduced me to the purer concepts of functional programming without mixing things with the all-too-accessible .NET framework.

So. The first real experiment for me has been the most mathematical application I could easily wrap my head around - the rolling checksum in RSync. I've managed to cobble together an implementation that... at the very least behaves as the algorithm expects. I'll be filing this in as I go, hopefully making a full-fledged file synchronization app in F# that uses WCF. I am not planning on supporting the original C# port, or making this implementation protocol-compatible with the original RSync, as the protocol itself is not documented, and reverse-engineering it out of the port that we have would be a nightmare. I'll move that piece off to SourceForge in the next few weeks and make it open there.

Anywho, here's what i've got so far:

 
#light
 
open System
open System.IO
 
/// the rsync constant
let M = 2 |> pown <| 16
 
/// the a(k,l) in rsync, with k = 0. defined as a(k,l) = (sum (i=k:l) X[i]) mod M
let a_initial l (data: int[]) = (data |> Array.fold_left (+) 0) % M
 
/// the b(k,l) in rsync, with k = 0. defined as b(k,l) = (sum (i=k:l) (l-i+1) * X[i]) mod M
/// uses the stateful fold function to carry through two states - one is the to-date sum, the other
/// is a simple position counter, since our sum is a function of position. so in \l (v,i) l is l
/// from b_initial, v is the computed to-date sum, and i is the index
let b_initial l (data: int[]) = ((data |> Array.fold_left ((fun l (v, i) a -> ((l-i+1) * a + v, i+1)) l) (0,0)) |> fst) % M
 
/// the rolling a(k,l) defined as a(k+1,l+1) = (a(k,l) - X[k] + X[l+1]) mod M
/// note that the implementation of this function is actually done as a((k-1)+1,
/// (l-1)+1) so that it can work in a scanning, as opposed to a recursive fashion
let a1 a xk1 xl = (a - xk1 + xl) % M
 
/// the rolling b(k,l) defined as b(k+1,l+1) = (b(k,l) - (l-k+1) * X[k] + a(k+1,l+1)) mod M
/// note that the implementation of this function is actually done as b((k-1)+1, (l-1)+1) so
/// that it can work in a scanning, as opposed to a recursive fashion
let b1 a b xk1 k l = (b - (l - k + 1) * xk1 + a) % M
 
/// the final summation of a(k,l),b(k,l). defined as s(k,l) = a(k,l) + M * b(k,l)
let s1 a b = a + M * b
 
/// computes the hash sums with the given file
let computeSums (stream: Stream) (s_out: int -> int -> unit) =
    let block_size = int (Math.Round(Math.Sqrt(float stream.Length)))
    let (_buffer: byte[]) = Array.create block_size (byte 0)
 
    let mutable bytesRead = 0
    while bytesRead < _buffer.Length do
        bytesRead <- stream.Read(_buffer, bytesRead, _buffer.Length )
 
    let buffer = Array.map (fun a -> int a) _buffer
    let mutable prev_a = a_initial (block_size - 1) buffer
    let mutable prev_b = b_initial (block_size - 1) buffer
 
    let mutable k = 0
    let mutable l = buffer.Length-1
 
    // scanWindow should always be k:(l+1) or (k-1):l bytes, as we will peak ahead/look behind at one offset value
    let mutable scanWindow = (Array.to_list buffer) @ [stream.ReadByte()]
 
    while k + l < (int stream.Length) do
        s_out k (s1 prev_a prev_b)
        prev_a <- a1 prev_a (List.hd scanWindow) scanWindow.[scanWindow.Length-1]
        prev_b <- b1 prev_a prev_b (List.hd scanWindow) k l
        k <- k+1
        l <- l+1
        scanWindow <- (List.tl scanWindow) @ [stream.ReadByte()]
 
let outStream = new StreamWriter(new FileStream(@"c:\test.out", FileMode.Create))
 
let stream1 = new FileStream(@"C:\ph.jpg", FileMode.Open)
let sums1 = Array.create ((int stream1.Length) + 1) 0
computeSums stream1 (fun a b -> sums1.[a] <- b)
 
let stream2 = new FileStream(@"C:\ph1.jpg", FileMode.Open)
let sums2 = Array.create ((int stream2.Length) + 1) 0
computeSums stream2 (fun a b -> sums2.[a] <- b)
 
Array.iter2 (fun a b -> outStream.WriteLine((a,b, if a=b then "" else "neq"))) sums1 sums2
outStream.Close()
 

This should work and compile with the latest CTP of F#. As you can see, the last block invokes the computeSums function, with a stream pointed to a local file and a function that's used to then spit sum values back out. It also saves off two consequtive passes comparing the sum values. The idea there is that the two files are identical except for a few bytes in the middle, so this way i can track easily where the sums differ and where they don't. I welcome any "you're doing it wrong!" comments, as the whole functional thing is new to me.