But Will It Play In Peoria? Part 2 (Running F# as a Plain Vanilla CGI app)

Programming should be fun. All you need is a good set of values, some skill, and the right attitude. F# is the most fun I’ve had in a programming language in years. This essay series is about that: having fun. Most books and essays are about writing awesome code. There’s not a lot of material about writing code aweseomely. The code you’ll find here has bugs! Just like your code. Can you find them? If you’re looking for the final version of the code, you can find it in the project’s GitHub page. You can find the story of how all of this started on the series index page.

So we’ve got a parameter we pass to the command-line, “F”, that determines output format. So far all it does is output “Hey there”. Let’s make it a little more useful.

First we put on our architectural hat and ask a question: are we asking the same program to do two different things? Now that we’re adding some kind of rudimentary html output, is this too much?

I don’t think so, mainly because we’re really not doing anything as much as we are deciding that types should be able to tell us what they are in an html-friendly way.

So why does this belong in the code? Why not just use a templating library to parse it any way we’d like and never have the output tweaking we’d like become part of the codebase?

That’s a much better question that I don’t have an answer to — and it worries me. But the fundamental truth remains: we’re not adding any processing and the point here is to demo F# code, so let’s make a decision that any of our types should also be able to display themselves as some sort of plain-jane html code.

Being good TDD’ers, first we write a test:

[<Test>]
let ``NameNumberPairType: ToHtml``() = 
    let input=OptionExampleFileLines.FromStrings initialTestFile |> Seq.toArray
    let ret=groupAndSum input |> Seq.toArray
    Assert.AreEqual
        ("<div class='NameNumberPairTypeItem'><span class='NameNumberPairTypeItemName;>\
        a</span>=<span class='NameNumberPairTypeItemNumber'>22</span></div>"
        , ret.[0].ToHtml())

Then we write some code:

type NameNumberPairType = { Name:string;Number:int} with
        member self.ToHtml() =
            "<div class='NameNumberPairTypeItem'><span class='NameNumberPairTypeItemName'>"
            + self.Name + "</span>="
            + "<span class='NameNumberPairTypeItemNumber'>"
            + self.Number.ToString() + "</span></div>"

Tests run okay. Now we write a test for the collection class:

Wait one. This is really going to be a mess of html, and it’s impossible for it to be wrong, so let’s write the code first. Then we’ll use the code to write the test. Promise not to tell the other consultants!

And….we run into problems. We can’t write this code.

type OptionExampleFileLines = private OptionExampleFileLines of OptionExampleFileLine[] with
    member self.ToHtml() =
        let makeItemIntoHtmlLIItem (item:NameNumberPairType) =
            "<li class='OptionExampleFileLinesItem'>"
            + item.ToHtml()
            + "</li>\l\n"
        let makeItemsIntoHtmlItemList (items:NameNumberPairType[]) = 
            items |> Array.map(fun x->
                "<li class='NameNumberPairTypeListItem'>"
                + (makeItemIntoHtmlLIItem x) +
                "</li>\l\n"
                )                
        "<div class='OptionExampleFileLines'><ul class='OptionExampleFileLinesList'>\l\n"
        + "Option List List goes here"
        // Next line doesn't work
        //+ (makeItemsIntoHtmlItemList self.OptionExampleFileLines)
        + "</ul></div>\l\n"

The reason? An essay or two ago, I noted that we had two types, NameNumberPairType and OptionExampleFileLine, that basically do the same thing. Sometimes I used one, sometimes the other. I said that we’d probably delete one.

That day has now arrived. Our tests are making our design better. (I tell people that TDD is really Test-Driven Design, not Test-Driven Development, because the purpose is to create your design, not do unit tests. Testing is about a lot more than TDD, and TDD is about a lot more than testing.)

I like using the name OptionExampleFileLine instead of the more generic NameNumberPairType, so I’m going to keep that one.

There’s a whole bunch of shuffling around that goes on now. But since I have tests that are passing, it’s no big deal. I’m basically going to take everything from NameNumberPairType and put it in OptionExampleFileLine, all the while making sure my tests continue to pass. I check in what I have, then get out the hard hat, flamethrower, axe, and demolitions.

(Later) That took about an hour, and involved a lot of refactoring code. I understand that folks looking at the early essays may have freaked out about all the bugs, but remember that we’re only talking about a day or two of actual work — and the purpose is to go on a journey through creating and deploying an app. Now that we’ve added testing, the code may still be crap. Ha! But it’s my crap, and I’m free to refactor and cleanup all I’d like without fear that I might be breaking something. I own it. Now it’s beginning to be a “real” application. My little program is growing up! I am so proud! (blushes)

We now have two types: OptionExampleFileLinesType and OptionExampleFileLineType. I like putting “Type” at the end of all of my types to disambiguate them in the code. The collection class manages the item class. Pretty standard stuff. We’ve kept our smart constructors, with options where necessary. I’ve also organized the methods so that it’s basically constructors, then converters.

Dang! We’ve got a lot of methods on these records! Starting to look like OO, Markham!

Fair enough, but we’ve added these as-needed. Could we factor everything out to functions that took lightly-types generics? Sure thing! But why? I’ve got a small number of types that do a small number of things. Don’t you think it might be a bit early to be generalizing the heck out of everything?

I do.

Just like TDD, DevOps, project structure, frameworks, and a dozen other things, that day will come. It’s just not today.

So our program types look like this. (In AppTypes.fs)

module AppTypes
open SystemTypes
open SystemUtils
 
type OptionExampleFileLineType = 
    private { Name:string;Number:int} with
    static member FromKVPairString (kv:System.Collections.Generic.KeyValuePair<string,int>) =
        {Name=kv.Key;Number=kv.Value}
    static member FromNameAndNumber (name:string) (number:int) =
        {Name=name; Number=number}
    static member FromKVPair 
        (pair:System.Collections.Generic.KeyValuePair<string,int>) =
        {Name=pair.Key;Number=pair.Value}
    static member FromString (s:string) = 
        let split=splitLineIfPossibleIntoTwoPieces '=' s
        if split.IsNone then None
        else
            let tryParseInt = tryParseGeneric<int> (snd split.Value)
            if tryParseInt.IsNone then None
            else Some {Name=fst split.Value; Number=tryParseInt.Value}
    override self.ToString() =
    self.Name + "=" + self.Number.ToString()
    member self.ToHtml() =
        "<div class='NameNumberPairTypeItem'><span class='NameNumberPairTypeItemName'>"
        + self.Name + "</span>="
        + "<span class='NameNumberPairTypeItemNumber'>"
        + self.Number.ToString() + "</span></div>"
    member self.ToKVPair = 
        System.Collections.Generic.KeyValuePair<string,int>(self.Name, self.Number)
type OptionExampleFileLinesType = 
    private {OptionExampleFileLines:OptionExampleFileLineType[]} with
    static member FromSeq (lines:seq<OptionExampleFileLineType>)=
        {OptionExampleFileLines=lines|>Seq.toArray}
    static member FromTypedCollection 
        (keyValueCollection:System.Collections.Generic.KeyValuePair<string,int> seq) =
            keyValueCollection 
            |> Seq.map(fun x->OptionExampleFileLineType.FromKVPair x)
            |> Seq.toArray
            |> OptionExampleFileLinesType.FromSeq
    static member FromStringKVCollection 
        (keyValueCollection:System.Collections.Generic.KeyValuePair<string,string> seq) =
        /// Process anything with alpha=number, ignore the rest
        let tryParsingKeyIntoAnInteger 
            (pair:System.Collections.Generic.KeyValuePair<string,string>)
                =System.Int32.TryParse pair.Key
        let tryParsingValueIntoAnInteger 
            (pair:System.Collections.Generic.KeyValuePair<string,string>)
                =System.Int32.TryParse pair.Value
        let doesKVWorkForUs (pair:System.Collections.Generic.KeyValuePair<string,string>) =
            (fst (tryParsingKeyIntoAnInteger pair) = false)
            && (fst (tryParsingValueIntoAnInteger pair) = true)
        let optionLines = keyValueCollection 
                        |> Seq.filter(fun x->doesKVWorkForUs x)
                        |> Seq.map(fun x->OptionExampleFileLineType.FromNameAndNumber
                                            x.Key (snd (tryParsingValueIntoAnInteger x)))
        optionLines
    static member FromStrings (strings:seq<string>) =
        strings |> Seq.map(fun x-> OptionExampleFileLineType.FromString x) |> Seq.choose id
    member self.ToHtml() =
        let makeItemIntoHtmlLIItem (item:OptionExampleFileLineType) =
            "<li class='OptionExampleFileLinesItem'>" + OSNewLine
            + item.ToHtml()
            + "</li>" + OSNewLine
        let makeItemsIntoHtmlItemList (items:OptionExampleFileLineType[]) = 
            items |> Array.map(fun x->
                "<li class='NameNumberPairTypeListItem'>" + OSNewLine
                + (makeItemIntoHtmlLIItem x) + OSNewLine +
                "</li>" + OSNewLine
                ) |> String.concat ""         
        "<div class='OptionExampleFileLines'><ul class='OptionExampleFileLinesList'>" + OSNewLine
        + (makeItemsIntoHtmlItemList self.OptionExampleFileLines)
        + "</ul></div>" + OSNewLine

Our App tests look like this:

let initialTestFile = 
    seq [
        "a=9"
        ;"b=8"
        ;"c=10"
        ;"a=4"
        ;"b=4"
        ;"c=11"
        ;"#$%^"
        ;""
        ;"a=9"
        ;"   "
        ;"   asdf"
        ]
[<Test>]
let ``OptionExampleFileLines: Initial test file``() = 
    let ret=OptionExampleFileLinesType.FromStrings initialTestFile |> Seq.toArray
    Assert.AreEqual(7, ret.Length)
    Assert.AreEqual("a=9", ret.[0].ToString())
    Assert.AreEqual("a=9", ret.[6].ToString())
[<Test>]
let ``groupAndSum: Initial test file``() = 
    let input=OptionExampleFileLinesType.FromStrings initialTestFile |> Seq.toArray
    let ret=groupAndSum input |> Seq.toArray
    Assert.AreEqual(3, ret.Length)
    Assert.AreEqual("a=22", ret.[0].ToString())
    Assert.AreEqual("b=12", ret.[1].ToString())
    Assert.AreEqual("c=21", ret.[2].ToString())
[<Test>]
let ``NameNumberPairType: ToHtml``() = 
    let input=OptionExampleFileLinesType.FromStrings initialTestFile |> Seq.toArray
    let ret=groupAndSum input |> Seq.toArray
    let retHtml = ret.[0].ToHtml()
    let expectedString=
        match MajorOSType with
            |Windows->"<div class='NameNumberPairTypeItem'><span class='NameNumberPairTypeItemName'>a</span>=<span class='NameNumberPairTypeItemNumber'>22</span></div>"
            |Linux->"<div class='NameNumberPairTypeItem'><span class='NameNumberPairTypeItemName'>a</span>=<span class='NameNumberPairTypeItemNumber'>22</span></div>"
    Assert.AreEqual(expectedString, retHtml)
[<Test>]
let ``OptionExampleFileLines: ToHtml``() = 
    let input=OptionExampleFileLinesType.FromStrings initialTestFile |> Seq.toArray
    let ret=groupAndSum input |>OptionExampleFileLinesType.FromSeq
    let retHtml = ret.ToHtml()
    let expectedString=
        match MajorOSType with
            |Windows->"<div class='OptionExampleFileLines'><ul class='OptionExampleFileLinesList'>\r\n<li class='NameNumberPairTypeListItem'>\r\n<li class='OptionExampleFileLinesItem'>\r\n<div class='NameNumberPairTypeItem'><span class='NameNumberPairTypeItemName'>a</span>=<span class='NameNumberPairTypeItemNumber'>22</span></div></li>\r\n\r\n</li>\r\n<li class='NameNumberPairTypeListItem'>\r\n<li class='OptionExampleFileLinesItem'>\r\n<div class='NameNumberPairTypeItem'><span class='NameNumberPairTypeItemName'>b</span>=<span class='NameNumberPairTypeItemNumber'>12</span></div></li>\r\n\r\n</li>\r\n<li class='NameNumberPairTypeListItem'>\r\n<li class='OptionExampleFileLinesItem'>\r\n<div class='NameNumberPairTypeItem'><span class='NameNumberPairTypeItemName'>c</span>=<span class='NameNumberPairTypeItemNumber'>21</span></div></li>\r\n\r\n</li>\r\n</ul></div>\r\n"
            |Linux->"<div class='OptionExampleFileLines'><ul class='OptionExampleFileLinesList'>\n<li class='NameNumberPairTypeListItem'>\n<li class='OptionExampleFileLinesItem'>\n<div class='NameNumberPairTypeItem'><span class='NameNumberPairTypeItemName'>a</span>=<span class='NameNumberPairTypeItemNumber'>22</span></div></li>\n</li>\n<li class='NameNumberPairTypeListItem'>\n<li class='OptionExampleFileLinesItem'>\n<div class='NameNumberPairTypeItem'><span class='NameNumberPairTypeItemName'>b</span>=<span class='NameNumberPairTypeItemNumber'>12</span></div></li>\n</li>\n<li class='NameNumberPairTypeListItem'>\n<li class='OptionExampleFileLinesItem'>\n<div class='NameNumberPairTypeItem'><span class='NameNumberPairTypeItemName'>c</span>=<span class='NameNumberPairTypeItemNumber'>21</span></div></li>\n</li>\n</ul></div>\n"
    Assert.AreEqual(expectedString, retHtml)

Since our string format has one than one line, we have to make sure we think about OS Type. Once again, I grabbed some reusable code from another project and put in the SystemUtils file. It tells me the major OS type. (It probably needs refactoring, but it works and that’s not important right now)

/// Are we running on linux?
let isLinuxFileSystem =
    let os = System.Environment.OSVersion
    let platformId = os.Platform
    match platformId with
        | PlatformID.Win32NT | PlatformID.Win32S | PlatformID.Win32Windows | PlatformID.WinCE | PlatformID.Xbox -> false
        | PlatformID.MacOSX | PlatformID.Unix -> true
        | _ ->false
type MajorTypeOfOS = Windows|Linux 
let MajorOSType = if isLinuxFileSystem then Linux else Windows

One thing to notice is that I’ve wrapped my html tags in css classes. This isn’t an html-writing app, but I’ve learned from experience that anytime I output html I might as well make it so I can change formatting later on. It’s easy now — tough later. Plus there have been more than one project where I had to demo the scaffolding instead of the “real” app, usually due to Product Owners and UX folks working through look-and-feel issues ad infinitum. (Would you like the piano here or over there, Mr. Smith? Hell if I care, let’s make sure the piano works, then you guys can spend the rest of the week shuffling it around.)

We’ve got seven tests and seven green lights. I’ll check the code back in and actually add the html output.

The bottom of “doStuff”, our spot for adding new stuff and factoring it out, looks like this now:

let optionLines = OptionExampleFileLinesType.FromStringKVCollection keyValuesFromFile
    match opts.outputFormat with
        |Html->printfn "%s" (optionLines.ToHtml())
        |_->printfn "%s" (optionLines.ToString())

This caused some problems with our old friend groupAndSum, which always looked like the heart of the program. It’s signature is all wrong.

let groupAndSum (optionLines:seq<OptionExampleFileLineType>) =
    optionLines 
        |> Seq.map(fun x->x.ToKVPair) 
        |> groupAndSumKV
        |> Seq.map(fun x->
            OptionExampleFileLineType.FromKVPairString 
                (System.Collections.Generic.KeyValuePair<string,int>(fst x, snd x))
            )

We’re managing that collection type now, so we shouldn’t be passing around sequences. But we can’t pass in the managed collection because then the function can’t get to the members! For now let’s take the method and drop it in the managed class.

When we do that, the groupAndSumKV method, which works on any Key-Value vector, has to go somewhere before our AppTypes. That makes sense, since it’s generic. So it goes in the SystemUtils class.

That made some of our tests fail to compile so we’ll fix that. Once we’ve got all green lights again we take a look at the output from the command line.

We have both text and html output working

But does this html stuff actually work as html?

Pipe the output to a test html file and run it

Well heck, that doesn’t look right

Ah! We are not closing our list items

Easy fix. Go back and close the list items, then update the tests.

As it turns out, we had a bug in our collection ToString method. Fixed like so:

member self.ToHtml() =
        let makeItemIntoHtmlLIItem (item:OptionExampleFileLineType) =
            "<li class='OptionExampleFileLinesItem'>" + OSNewLine
            + item.ToHtml()
            + "</li>" + OSNewLine
        let makeItemsIntoHtmlItemList (items:OptionExampleFileLineType[]) = 
            items |> Array.map(fun x->
                (makeItemIntoHtmlLIItem x) 
                ) |> String.concat OSNewLine   
        "<div class='OptionExampleFileLines'><ul class='OptionExampleFileLinesList'>" + OSNewLine
        + (makeItemsIntoHtmlItemList self.OptionExampleFileLines)
            + "</ul></div>" + OSNewLine

(We initially had two li tags). So now I’ll update the tests and inspect to see that it’s working fine.

Our goal here is to run as a plain CGI app. We’ve got it returning html. I might be okay with just that. But I’ll write one more function just because it bugs my html sensibilities not to even have a document — just return fragments. If we’re going to show a page when people click a button, we can write the 10 lines or so it takes to make it a page. Plus I know from experience that this won’t work unless we do one thing more.

Do I write this code? No, I do not. I already have a function that takes this minimum amount of information and makes a web page. It’s a common shim. So I re-use it. The output code in doStuff now looks like this. Starting to look a little complicated. Might be time for some refactoring coming up.

match opts.outputFormat with
        |Html->printfn "%s" (processedLines.ToHtml())
        |WebPage->
            let webserverReturnPage = 
                wrapFragmentIntoAnHtmlPageWebServerReturnString 
                    "MyPage"
                    "main.css"
                    "main.js"
                    (processedLines.ToHtml())
            printfn "%s" webserverReturnPage
        |Text->printfn "%s" (processedLines.ToString())

Note that I don’t care if there’s no css or javascript file. Web pages aren’t programs. Browsers have to put up with all kinds of missing stuff. Glad I’m not coding a browser! (By the way, there’s a lot more stuff about headers and such we could talk about. Perhaps another day.)

Now that we have an app that returns data in the right format, can we get it to run on linux in the directory we want?

I pop over to one of the servers I manage and sftp the files over. At this point, there’s some flailing around while mono kicks me around a bit. I’m used to compiling everything all into one executable, dependencies and all. Memory’s cheap. Dependency hell sucks. But mono refuses to do this, complaining that it can’t load NUnit.Framwork, even though it’s right there in the directory (With the correct version).

Bastards.

Being the obstinate ornery old cuss that I am, hell if I’m going to let mono kick be around. So I spend a couple of hours after supper watching TV and pounding away at the keyboard trying to get it to work.

I finally give up. It works okay without all the dependencies compiled in, so I bundle it like so:

mkbundle --static -o stupid1 stupid1.exe

Works on my machine

Also Putty stopped working, so that required an upgrade (after some poking around to make sure it wasn’t a network config issue). So far in this battle, the tools are winning out over productivity in trying to get stuff done. Or in other words, situation normal.

First thing in the morning, it was yet another hour getting apache configured correctly.

A word in support of framework thrashing: Even though I’ve talked about how thrashing sucks, everybody thrashes. It’s the natural state of the developer. If you’re not thrashing from time-to-time, you’re doing something wrong. The trick is to wisely invest your time in what you thrash in. Ten years from now people will still be setting up Apache webservers and configuring them. I can even guarantee you that as ancient and uncool as CGI is today? Some poor schmucks will still be coding in it years from now. Linux will still be around. HTML and CSS will continue to work with code you write today. Thrashing in stuff that’s foundational to everything else nad long-lasting isn’t thrashing. It’s learning. thrashing in something that four years from now will no longer be cool is a freaking good way to waste you and your customer’s time while trying to do stuff like all the cool kids Focus on value. Also get off my lawn already! (shakes fist at sky)

I had to wrap the file in a bash script and then call that.

#!/bin/bash

mono stupid1.exe -F:w

Once the permissions are set, the form returns the default values.

Yay! We’re getting default values back

Of course, it’s not taking the input parameters yet, but we’ve closed our second feedback loop, at least manually: the build/deploy loop. Over time we’d expect to tweak, optimize, and automate it all. Just like everything else we’re doing.

Next I had to actually take whatever parameters the form was sending. This took me down yet another rabbit hole, this time based around Google’s new insistence that everything be https now instead of just http. Of course, I didn’t realize that was the problem — the symptom was that something was eating my form variables. You can spend a lot of time being very sad googling the wrong problem. I was POSTing the form, but I was receiving a GET on the server did I realize that my problems had to do with certificates and automatic redirects. Who knew? In an SSL environment, requests to the server can get bounced around a lot. This can cause other problems.

When you’re returning potentially a lot of data, POST is the way to go, and CGI sends you the post data on a string sent into stdin. Looking around, I found a function to reuse that takes everything from stdin that I found on the web years ago. It’s not tested, but it goes in SystemUtils and it’ll be tested over time.

let rec readStdIn lines= 
    match System.Console.ReadLine() with
    | null -> List.rev lines
    | "" -> readStdIn lines
    | s -> readStdIn (s::lines)

I ran the code from the client, and I got nothing. Then I realized what was going on. By default the app was reading everything from a file and ignoring what didn’t fit. CGI parameters don’t fit. Instead of this cool-looking recursive code above, earlier I had used some equally cool-looking Seq.Infinite code to read stuff in. But that code was looking for a user manually-entering stuff, not streams.

Good grief. Nothing to get worked up over, just more noise. Programming when done right should teach one humility. Instead, I’m going to change the F file parameter, which controls output format, to OF. Then I’ll add a IF for input format. This is all easy “wire up the pieces” work that I don’t care for but it’s literally five minutes of work. (It’s the same stuff we did for the inputFile parameter when we added it.)

Something important: Many times, when you see somebody demonstrating some new tech, they’ll show you how quick and easy it is to do something. (Usually that’s because they’re trying to sell you something or get you excited.) I’m not trying to do that here with my little command-line parameter library. The magic of doing incremental type and function development with F# is that you only create tokens that you actually use. Yay for busy, forgetful, and distracted people! No matter how complex your project, if you manage your files well, I’ve found there’s only a small number of tokens you have to learn and master to complete it — and you’re the person creating those tokens! So instead of buying/downloading a framework and having to learn it, instead you’re creating your own minimalist framework as you go along.

You become your own Yoda.

That’s amazingly productive. The more you do it, the more productive it gets.

Added a new method, “inputStuff”. I have three main functions I’m working on: “inputStuff”, “doStuff”, and “outputStuff”. From there everything eventually gets factored out. inputStuff currently looks like this:

/// Outside of the onion coming in
let inputStuff (opts:OptionExampleProgramConfig):OptionExampleFileLinesType = 
    let fileIsThere =System.IO.File.Exists(fst opts.inputFile.parameterValue)
    let inputStrings:string seq =
        match opts.inputFormat with
            |Stream->Seq.initInfinite (fun _ -> System.Console.ReadLine())
            |File-> System.IO.File.ReadAllLines(fst opts.inputFile.parameterValue)
                    |> Array.toSeq
            |CGI->
                // This is specific to the test html harness form and should go away
                let cgiVariables=processCGISteamIntoVariables() 
                                |> Seq.toArray
                if cgiVariables |> Array.exists(fun x->x.Key="myInput")
                    then
                        let var=(cgiVariables 
                            |> Array.find(fun x->x.Key="myInput")).Value
                        var.Split([|OSNewLine|], System.StringSplitOptions.None) 
                            |> Array.toSeq
                    else Seq.empty
    OptionExampleFileLinesType.FromStrings inputStrings

The big work on this last little piece was writing up a function to parse incoming CGI variables on a POST. I think I already had this somewhere, but I can’t locate it. Of course I wrote the test first, using whatever junk I had typed into my dummy form (which now has a second input field just for kicks)

[<Test>]
let ``Parse Sample CGI Input``()=
    let sampleStream=["myInput=a%3D9%0D%0Ab%3D4%0D%0Ac%3D10%0D%0Aa%3D3&blub=dfg+asdf"]
    let procInput=processCGIStream sampleStream |>Seq.toArray
    Assert.AreEqual(2, (procInput |> Seq.length))
    Assert.AreEqual("myInput", procInput.[0].Key)
    Assert.AreEqual("a=9\r\nb=4\r\nc=10\r\na=3", procInput.[0].Value)
    Assert.AreEqual("blub", procInput.[1].Key)
    Assert.AreEqual("dfg asdf", procInput.[1].Value)

Then I wrote the method. Feeling the call of F#, I really wanted to do something with a lot of piping and tuples. It’s early yet for this function, however, so I made it as stupid as I could. I might be looking at this a lot! Later on I’ll have something cool to show all of my functional friends. But not yet.

let processCGIStream incomingStream =
    let findVariables = incomingStream |> List.map(fun x->
        let goodLine = (lineContainsADelimiter '=' x) &&  (x.Split([|'='|]).Length>1)
        let multipleVariablesOnLine = lineContainsADelimiter '&' x
        if goodLine
            then
                let lineSplitByVariable = 
                    if multipleVariablesOnLine
                        then x.Split([|'&'|])
                        else [|x|]
                let variableValueMatch = lineSplitByVariable |> Array.map(fun x->
                    let lineSplitByEquals=x.Split([|'='|])
                    let newKey=lineSplitByEquals.[0]
                    let newVal=System.Net.WebUtility.UrlDecode(lineSplitByEquals.[1])
                    System.Collections.Generic.KeyValuePair<string,string>(newKey,newVal)
                    )
                Some variableValueMatch
            else None
        )
    findVariables |> List.choose id |> Seq.concat

I had to add one thing I didn’t expect: if there’s an Input File parameter, the input type should always be “File”.

The bottom of loadConfigFromCommandLine changes:

let inputFileParmOnLine = args |> Array.exists(fun x->x.IndexOf("-I:")<>(-1))
    if inputFileParmOnLine
        then
            {configBase = newConfigBase; inputFile=newInputFile; inputFormat=File; outputFormat=newOutputFormat.parameterValue}
        else
            {configBase = newConfigBase; inputFile=newInputFile; inputFormat=newInputFormat.parameterValue; outputFormat=newOutputFormat.parameterValue}

If things feel like they’re moving a lot faster, that’s because they are. As we exercise the type system, as more and more code gets written, tested, and factored out, development speed continues to increase. In a production environment, you always want to be able to develop about 40 times faster than people can give you stuff to do. That’s because the actual programming is a very small part of making stuff people want. The hard part is the people.

I did a couple of command-line tests, then changed the BASH file on the server.

#!/bin/bash

mono stupid1.exe -IF:C -OF:w

Ran some tests on the server — and the CGI parsing crashed. It was a problem with System.Net.WebUtility.UrlDecode. Good grief! A .NET library? Seriously? It exists on Windows but not on linux.

sigh

Fixed that by hacking up a C# Dll that does URI decoding(!) I found on a webpage and it works okay.

This is a nice example of where the Windows CGI parsing test passed fine — but the code was broken in linux. I wish that “write once, run anywhere” was a reality, but we programmers have been promised that for 30 years or more. This is why we really need tests on the outside of our code — called and inspected by the OS the code runs under. Just like nuking the site from orbit, it’s really the only way to be sure.

Works From CGI

Note there’s still an SSL warning. At some point I may fix that. Or not. But it works!

So now what am I worried about? I’m not worried about the crappy code, as you guys can probably tell. (Please no comments from the peanut gallery) My major job now, at only three days in, is setting up and beginning to optimize work patterns: testing, deployment, feedback loops, and so on. From the coding standpoint, I’m worried about crappy tests. The tests are the real specifications for any program, and we’ve got way too few of them. Having said that, there’s not a lot that it does. We really haven’t added much core functionality. Most everything that happened in these two essays were OS-specific, not app-specific. (That’s actually good news, because anything we do to fix those problems will be reusable over and over again.)

Remember our philosophy that the app is the function, so if this essay series continues, we’re going to need to talk about testing apps from the OS. I’d also do something fun like a SPA. Maybe talk a bit about web frameworks.

This work represents about three days of a few hours here or there. In that time we’ve started to stand up an app that does what we want no matter where it’s deployed. And we’re doing that with tests as a safety net. Within a week or so we should have a reusable core that we can build out as needed. Within two weeks we should start seeing a build pipeline and some DevOps starting to happen.

Most of what I horsed around with this time was Operations: certificates, CGI, Apache, tooling, difference in runtimes, and so on. That’s great! Developers should constantly be knee-deep in operations. Perhaps not in production, but at the very least in an exact mirror-image of production. (Long story here about the Blue-Green Deployment Model)

I know this essay series is probably driving some folks nuts. Everything’s half-assed!

I hear you. But that’s the point. Starting a new project is always a struggle. As developers, far too often we want to sit in our ivory tower and just “do coding”. But that ain’t the job. It never was the job. And the more you think of that as being the job the worse programmer you’ll be. You have to relax enough to let a lot of things slide and start standing stuff up you can fix later. So much of this job is a combination of passion and relaxed competence. In the first week or three, the perfect is the enemy of the good enough. We incrementally make things rock solid, sometimes in a messy way. Pure code doesn’t spring forth from our head like demigods from the head of Zeus. We have to struggle. And the code is the easy part! The people and workflow thing is where the real worry is.

This has been a hoot. I always love hanging out in the IDE, poking around with some code, and talking to friends. Thanks for dropping by!


Follow the author on Twitter

July 25, 2018

Leave a Reply

Your email address will not be published. Required fields are marked *