Tuesday, July 14, 2009

Asteroids 3d strikes back

I got back to working on my asteroids clone, written in F# using the XNA framework.

Wednesday, July 8, 2009

Interfaces made convenient by type constraints

If my previous entry convinced you to make extensive use of interfaces, you may have encountered a problem, namely that calling methods implementing interfaces requires casting:

let link = new Link("link", config, ...)
// We want to write:
link.Send(data)
// But that does not compile. Instead, we must write:
(link :> ILinkOutput).Send(data)


The bad news are that there is no work around. The good news is that this situation does not happen too often in practice, as the creator of a link is in general not the user. New instances are often forwarded to other objects or functions, which will see them as ILinkOutput, thus not requiring a cast:

let register (p : ISender) (l : ILinkOutput) =
p.AddOutputLink(l) // OK: p is an ISender, no need for explicit cast
l.ConnectInputTo(p) // OK: l is an ILinkOutput, no need for explici cast

let link = new Link(...)
let process = new Process(...)
register process link // OK: process and link automatically upcasted


Now, what happens if we have a function taking a process and which needs to use two of its interfaces? There are two obvious solutions, which are not so pretty.

Ugly-ish:
let register (p1 : ISender) (l : Link) (p2 : IReceiver) =
p1.AddOutputLink(l) // Nice
p2.AddInputLink(l) // Nice
(l :> ILinkOutput).ConnectInputTo(p1) // Not so nice
(l :> ILinkInput).ConnectOutputTo(p2) // Not so nice


Uglier:
let register (p1 : ISender) (l1 : ILinkOutput) (l2 : ILinkOutput) (p2 : IReceiver) =
p1.AddOutputLink(l) // Nice
p2.AddInputLink(l) // Nice
l1.ConnectInputTo(p1) // Nice (is it?)
l2.ConnectOutputTo(p2) // Nice (is it?)

// Set up two processes communicating with each other.
// As links are uni-directional, use two links.
let p1 = new Process(...)
let p2 = new Process(...)
let link_p1_p2 = new Link(...)
let link_p2_p1 = new Link(...)

// p1 -> p2
register p1 link_p1_p2 link_p2_p1 p2 // Oops!!!
// p2 -> p1
register p2 link_p2_p1 link_p1_p2 p1 // Oops!!!


So many 1s and 2s... We really meant to write:

// p1 -> p2
register p1 link_p1_p2 link_p1_p2 p2
// p2 -> p1
register p2 link_p2_p1 link_p2_p1 p1


The correct solution uses generics and type constraints:

let register
(p1 : ISender)
(l : 'L
when 'L :> ILinkInput
and 'L :> ILinkOutput)
(p2 : IReceiver) =

p1.AddOutputLink(l)
p2.AddInputLink(l)
l.ConnectInputTo(p1)
l.ConnectOutputTo(p2)

// Set up two processes communicating with each other.
// As links are uni-directional, use two links.
let p1 = new Process(...)
let p2 = new Process(...)
let link_p1_p2 = new Link(...)
let link_p2_p1 = new Link(...)

// p1 -> p2
register p1 link_p1_p2 p2
// p2 -> p1
register p2 link_p2_p1 p1


Another use of type constraints is for cases when you want to pass lists of ISenders to a function.

Problem:
let sendAll (ps : ISender list) data =
ps
|> List.iter (fun p -> p.Send(data))

let p1 = new Process(...)
let p2 = new Process(...)

sendAll [ (p1 :> ISender) ; (p2 :> ISender) ] data // Blah!


Solution:
let sendAll (ps : 'S list when 'S :> ISender) data =
ps
|> List.iter (fun p -> p.Send(data))

let p1 = new Process(...)
let p2 = new Process(...)

sendAll [ p1; p2 ] data


Which can also be written:
let sendAll (ps : #ISender list) data =
ps
|> List.iter (fun p -> p.Send(data))

let p1 = new Process(...)
let p2 = new Process(...)

sendAll [ p1; p2 ] data

Breaking up classes using interfaces

When I started writing F# programs whose code span over multiple files, I was surprised to see that the order of compilation of these files matters. If the code in file B.fs refers to types or functions in A.fs, then A.fs must be compiled before B.fs.

Now, what happens if A.fs and B.fs depend on each other? In this case, you have a problem. The easy fix is to merge the two files, but this solution does not scale well. Another solution is to break code into smaller pieces, building small "islands" of circular dependencies, which can each be moved into their own file. Easily said, but not so easily said.

Consider an application involving processes and links. Processes can be executed, and can communicate with each other using unidirectional links.

The code might look like this:

type Process(name, config, ...) =
// Construction
...

// Methods
// Connection to links
member x.AddInputLink(link : Link) = ...
member x.AddOutputLink(link : Link) = ...

// Control execution
member x.ExecuteInSomeWay(args) = ...
member x.ExecuteInSomeOtherWay(args) = ...
member x.CheckIsReady() = ...
member x.MarkAsReady() = ...

// Debugging, and other stuff...
member x.ToggleDebugging(b) = ...
member x.SetDebugOutput(o) = ...
...

and type Link(name, config, ...) =
// Construction
...

// Methods
// Connection to processes
member x.ConnectOutputTo(p : Process) = ...
member x.ConnectInputTo(p : Process) = ...

// Read/Write data
member x.Send(data) = ...
member x.Receive() = ...

// Debugging ...
member x.ToggleDebugging(b) = ...
member x.SetDebugOutput(o) = ...
...



As Process and Link are mutually dependent, their code must be all in one file. That may seem OK at first, but things will become harder to manage as more code is added over time.

A first solution is to introduce interfaces for Process and Link, simply duplicating all their method signatures, and replace all occurrences of classes Process and Link by their interfaces. The interfaces can be moved to a separate file, which should be shorter, as it contains no code. Classes Process and Link can now each live in separate files.

Interfaces.fs:
type IProcess =
// Connection to links
abstract AddInputLink : ILink -> ...
abstract AddOutputLink : ILink -> ...

// Control execution
abstract ExecuteInSomeWay : Args -> ...
...

// Debugging, and other stuff...
member x.ToggleDebugging : bool -> unit
...

and type ILink =
// Connection to processes
abstract ConnectOutputTo Process -> unit
...



Process.fs:
open Interfaces

type Process(name, config, ...) =
// Construction
...

// Interface implementations
interface IProcess with
member x.ExecuteInSomeWay(args) = ...
...


Link.fs:
open Interfaces

type Link(name, config, ...) =
// Construction
...

// Interface implementation
interface ILink with
member x.ConnectOutputTo(p : IProcess) = ...
...


Still, Interfaces.fs may be excessively long, and we may end up in a situation where all interfaces are defined in one large file. Not very pleasant.

The next step consists of breaking up the interfaces. The debugging-related methods obviously belong to a IDebugable interface, which need not be concerned with the details of processes and links.

A process which is connected to the input side of a link has no need to have access to the methods taking care and transferring data from the link to the process, which indicates that ILink could be split in e.g. IInputLink and IOutputLink.

There is another case of excessive method exposure. Links do not control the execution of processes, that's the task of the scheduler. The final decomposition may look as shown below.

IActivable.fs:
type IActivable =
abstract CheckIsReady : unit -> bool
abstract MarkAsReady : bool -> unit


IExecutable.fs:
type IExecutable =
abstract ExecuteInSomeWay : Args -> ...
abstract ExecuteInSomeOtherWay : Args -> ...


InputInterfaces.fs:
type IReceiver =
abstract AddInputLink : ILinkInput -> unit

and type ILinkInput =
abstract Receive : unit -> Data
abstract ConnectOutputTo : IReceiver -> unit


OutputInterfaces.fs:
type ISender =
abstract AddOutputLink : ILinkOutput

and type ILinkOutput =
abstract Send : Data -> unit
abstract ConnectInputTo : ISender -> unit


The techniques shown here are by no means specific to F#, and are part of the "proper programming" techniques every object-oriented programmer should know and use. Before using F#, I almost exclusively used interfaces for polymorphism, on "as-needed" basis. The fact is, interfaces are also useful for modular programming.

Decomposing programs into interfaces and implementation helps writing reusable, readable code. It does have a cost, though, as it requires more typing. As long as keyboards remain cheaper than brains, the benefits should outweigh the costs.

Concurrency on a single thread

When I wrote my article about concurrency, I did a bit of searching to find out how to use multi-threading in F#. One of the hits was a blog entry on HubFS by gneverov titled "Hell Is Other Languages : Concurrency on a single thread". At that time, I was a bit confused by the article. Why would one want single-threaded concurrency when we already had multi-threaded concurrency? Part of the answer lies in reactive programming.

Recently, I read an article by Shawn Hargreaves, "Pumping the Guide", where he describes some design issues with using the Guide on the Xbox. The Guide is the part of the Xbox GUI which is used for performing common operations, such as entering text, selecting a device where game data is saved, seeing which of your friends are online and what they are doing... The problem is to have the game still running in the background while the Guide is opened.

The solution chosen by the XNA team was to make calls to the Guide non-blocking. This however makes it harder for game developers to use it, as it forces them to keep track of the application state explicitly, including both the state of the game and the state of the Guide.

Kevin Gadd's shows a elegant solution to this problem in C# using custom enumerators in an article aptly named "Asynchronous programming for xna games with Squared.Task".

Kevin's implementation in C# is as pretty as it can get, but it would probably look even better in F# using asynchronous computations. Although the async { ... } notations is exactly what we want, the Async functions do not seem to support the kind of single-threaded concurrency that Kevin's Square.Task does.

The code by gneverov however should be easy to modify to execute one single step in one of the active tasks, if it does not already support that.

Thursday, July 2, 2009

WiX again.

In my previous post I expressed my disappointment with WiX. However, after an additional day of struggling with it, I may change my mind. I managed to find a pretty straightforward solution to (almost) all my problems. Getting there wasn't straightforward at all, though.

Here are the lessons I learned in the process:

Do not overwrite existing files using <File> elements.
When your software gets uninstalled, it will remove your file, but the old file won't be restored. The solution that first popped up in my mind, namely to backup old files, isn't as easy as it seems. The problem is that the backup copy will be left behind after the software gets uninstalled.

<CopyFile> is not your friend.
...at least as far as backing up existing files goes. I tried to back up the old file by copying it into my application's folder using CopyFile, and failed miserably. Maybe I missed something, but it seems CopyFile is designed to make copies of the files you install, not of existing files.

<CustomAction> is way too complex for its own good (and yours).
CustomAction can be used in many different ways, using slightly different syntaxes. I attempted to use it to backup files, and finally succeeded. It wasn't easy. I tried to use it to run a command (COPY), but it failed. Same problem with other commands such as REN and DEL. No success running batch files either. I tried running a jscript, and that failed too. Since the creator pf WiX advises against using scripts in custom actions, I did not insist. I ended up using "xcopy.exe", which worked (with a catch: the target must be an existing directory, otherwise xcopy demands to interact with the user, and we don't want that).

I found ONE use of CustomAction, and I am not planning on trying out other uses. At least not for a while.

<RemoveFile> is your friend.
Unlike CopyFile, it's easy to use to clear up stuff in your application's folder. I used it to remove the backup after restoring it during uninstallation.

In case you wonder how to handle back-ups in WiX, that's the way I did it:


<Property Id='XCOPY'>xcopy.exe</Property>
...
<CustomAction Id='Backup'
Property='XCOPY'
ExeCommand='"[ExistingFolder]PlugIn.dll" "[OldFiles]" /Y'
Result='check'
Execute='deferred' />

<CustomAction Id='Install'
Property='XCOPY'
ExeCommand='"[NewFiles]PlugIn.dll" "[ExistingFolder]" /Y'
Result='check'
Execute='deferred' />

<CustomAction Id='Restore'
Property='XCOPY'
ExeCommand='"[OldFiles]PlugIn.dll" "[ExistingFolder]" /Y'
Result='ignore'
Execute='deferred' />

<InstallExecuteSequence>
<Custom Action='BackupPlugIn' Before='InstallPlugIn'>Not Installed</Custom>
<Custom Action='InstallPlugIn' After='InstallFiles'>Not Installed</Custom>
<Custom Action='RestorePlugIn' Before='RemoveFiles'>Installed</Custom>
</InstallExecuteSequence>

...
<Directory Id='NewFiles' Name='New Files'>
<Component Id='NewFiles' Guid='...'>
<File Id='NewFile' Name='Plugin.dll' Source='build/PlugIn.dll' />
</Component>
</Directory>

<Directory Id='OldFiles' Name='Old Files'>
<Component Id='OldFiles' Guid='...'>
<CreateFolder />
<RemoveFile Id='CleanUpBackup' Name='PlugIn.dll' On='uninstall' />
</Component>
</Directory>


Note how I avoided overwriting files in the "Component" elements. I do that using CustomAction instead, both when installing my plug-in and when restoring the old one.

Something that's missing are rollback actions. There should be one for each deferred action, namely an action removing the xcopy-ed file. I don't know of any standard program to remove files on Windows, and I'd rather not write one myself. Where is "rm"?

UPDATE:


WiX experts are probably shaking their heads when they read this code:


<Custom Action='RestorePlugIn' Before='RemoveFiles'>Installed</Custom>
</InstallExecuteSequence>


While it does do what I intended when the user removes the software, it is also executed when users perform other actions than install or uninstall, such as repair an installation. As a result, an attempt to repair an installation will restore the old plug-in, which of course is more likely to destroy the installed software...

Wednesday, July 1, 2009

WiX, because we can.

For today's post, I thought I might write on something that's not directly related to F#. Instead, this article deals with WiX, a piece software intended to help creating installers. WiX and F# do share their approach to writing programs, though, namely the declarative way.

First, I must say I am no expert in installers. I have built a couple rpm packages on Linux, and found the experience OK, but not particularly enjoyable. I have also used NullSoft Scriptable Install System (NSIS for short) for top10, and gave up with it. My main gripe is that NSIS is really more a scripting language, and not a particularly good one at that. I don't think it's worse than the other install systems out there, though I haven't tried them all. I did try WISE, for instance, but failed to install it. Now, that's irony...

The problem is that these tools try to solve the problem using a procedural approach. The programmer has to specify what to do during an installation, which typically includes copying files on the hard disk, setting up a few shortcuts, messing up with the registry. Specifying all this is a tedious task, but it's not even half the job. One must also specify what to do when the user decides to uninstall the software. Typically, this means clearing up the mess in the registry, removing the shortcuts, deleting files.

Obviously, it should be possible to deduce the uninstallation process from the installation process. It's not just a matter of convenience, but of correctness.

WiX attracted a bit of attention when it came out, being Open Source software produced by Microsoft. Another side of it worth of interest is that it uses a declarative approach to specifying installation procedures. Instead of writing the sequence of operations needed to (un)install software, one specifies what to install, and where. Which is the way it should be.

As attractive as it seems to be, I must say I am disappointed. The three full days of experimenting I have spent on WiX have been rather painful.

The first issue is that WiX is based on Windows Installer, a.k.a MSI, whose documentation is pretty hard to grasp for a newcomer. Part of the problem resides in the core design choices, which seem genuinely weird to me. An MSI installer is not a program, it's a database, which contains a description of a workflow, where the actions make up the installation procedure. Now, installations a probably some of the most linear processes one can think of, meaning workflows are complete overkill. Why one would use a database to store a workflow is another mystery.

WiX attempts to hide these layers of insanity. All you need to do to copy a file on the system is specify which file you want to copy, and where. WiX takes care of generating the code to undo the copy if it fails, as well as the code to remove the file when uninstalling.

Sadly, WiX only goes halfway. If your installation process happens to write over an existing file, the uninstallation process won't restore the original file. This must be handled manually, and requires a decent understanding of MSI, its workflows, custom actions and databases.

What's worse, WiX adds another layer of wickedness of its own, namely XML. That was probably the only technology missing in the stack...

In case you urgently need a headache, I suggest you go have a look at Suggested InstallExecuteSequence, which is the sequence of actions one should take to install software. That's 60 of them, no less. If that sounds like a lot, well it is. The catch is that the same sequence is used for uninstallation. Approximately half of these actions are active when installing, the other half being active while uninstalling. The question that immediately pops up is "why not use two separate sequences"? But then, one would not need conditions, such a shame...