Home of Alex Thissen

WCF Extensibility part 5: Make your endpoints behave

Alex Thissen — Tue, 25 Aug 2009 06:36:00 GMT

In the previous post we looked at an operation behavior that applied the operation formatter to a particular operation description. Because this is default behavior for all operations in the interface defined, we will make life a little easier on us to have the operation behaviors applied automatically. A WCF endpoint consists of address, binding and contract. The latter is represented by the [ServiceContract] annotated interface, in our case IRcon and the target of our exercise.

The extensibility point that the WCF runtime provides is an endpoint behavior. Endpoint behaviors allow you to manipulate an endpoint’s definition. To create an endpoint behavior you need to implement the System.ServiceModel.Description.IEndpointBehavior interface:

public interface IEndpointBehavior
{
    void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters);
    void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime);
    void ApplyDispatchBehavior(ServiceEndpoint endpoint, EndpointDispatcher endpointDispatcher);
    void Validate(ServiceEndpoint endpoint);
}

This should look pretty familiar to you if you have read the previous part in this series. Again, you can apply your behavior both on the client and service side. There’s another validation moment in the Validate method and you can add binding parameters to the binding context for later use.

The implementation of ApplyClientBehavior is the most interesting one. Here we will apply the operation behavior to each of the endpoint’s operations.

public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
{
    if (endpoint.Binding is SourceRconTcpBinding)
    {
        foreach (OperationDescription description in endpoint.Contract.Operations)
        {
            ApplyFormatterBehavior(description, endpoint);
        }
    }
}

private void ApplyFormatterBehavior(OperationDescription operationDescription, ServiceEndpoint endpoint)
{
    if (operationDescription.Behaviors.Find<SourceRconOperationFormatterBehavior>() == null)
    {
        SourceRconOperationFormatterBehavior formatterBehavior = new SourceRconOperationFormatterBehavior();
        operationDescription.Behaviors.Add(formatterBehavior);
    }
}

Inside the implementation a simple loop over the operation descriptions in the contract is enough to get things going. The helper method ApplyFormatterBehavior makes sure that the SourceRconOperationFormatterBehavior behavior is applied only once.

An endpoint behavior needs to be added to the Behaviors collection of an endpoint to make it do its work. Given the client-side interface definition IRconProxy (which derives from both IRcon and IClientChannel) you can create a channel factory. The factory holds the the definition of the endpoint, provided by the implicit ABC in line 1 (address and binding are obvious, the generic <T> of the ChannelFactory represents the contract).

ChannelFactory<IRconProxy> factory = new ChannelFactory<IRconProxy>(binding, new EndpointAddress(uri));
factory.Endpoint.Behaviors.Add(new SourceRconEndpointBehavior());
factory.Open();

Line 2 adds the behavior to the Behaviors collection of the endpoint. When the factory opens, the behavior gets applied. The operation formatters get applied at that moment as well. The operation formatters are used when the actual calls through the channel are made.

Side note: if you were to use a ClientBase<T> derived proxy, you would need to drill down into the proxy.ChannelFactory.Endpoint.Behaviors to get at the Behaviors collection, or use the proxy.Endpoint property for easier access.

To summarize, an endpoint behavior allows you to manipulate your endpoint. Behaviors are an opt-in, so it is your choice whether to apply them or not.

WCF Extensibility part 4: Applying formatting to operations

Alex Thissen — Fri, 21 Aug 2009 20:11:00 GMT

In the previous part we inspected the way you can influence the transition from a service operation to a WCF Message object. This was accomplished with an implementation of the IClientMessageFormatter interface, in our case the SourceRconOperationFormatter class. Now it’s time to look at how to apply this message formatter to the client channel stack.

The message formatting happens based on the formatter that is specified as part of the description of an operation. This means that in theory you could change the formatter from operation to operation. In order to get the formatter in place you need to change the Formatter property of the System.ServiceModel.Description.OperationDescription for the operation of your choice. Obviously this has to be done before the first method call is made. The extensibility point that allows you define which formatters are used for client and service operations is an operation behavior. When applied an operation behavior will influence the way an operation behaves when called. Hence the name.

Operation behaviors must implement the System.ServiceModel.Decription.IOperationBehavior interface. First of all look at that interface definition:

public interface IOperationBehavior
{
    void AddBindingParameters(OperationDescription operationDescription, BindingParameterCollection bindingParameters);
    void ApplyClientBehavior(OperationDescription operationDescription, ClientOperation clientOperation);
    void ApplyDispatchBehavior(OperationDescription operationDescription, DispatchOperation dispatchOperation);
    void Validate(OperationDescription operationDescription);
}

As you can see it has two methods to apply the specifics of your behavior to an operation description: one for client-side and one for service-side operations. We need to fill in the client-side proxy part of it only. Furthermore, the behavior can validate the operation to which it is applied. More on this in a moment. And finally, should the behavior need to add specific parameters to the entire binding (which can be accessed later on from the binding context) it is free to do so in the AddBindingParameters method.

The SourceRconOperationFormatterBehavior class explicitly implements the IOperationBehavior interface. There is no point to have public method as it is unlikely that the implementation will be called other than via the interface.

public class SourceRconOperationFormatterBehavior: IOperationBehavior
{
    #region IOperationBehavior Members

    void IOperationBehavior.AddBindingParameters(OperationDescription operationDescription, BindingParameterCollection bindingParameters)
    {
        // Nothing for us to do
    }

    void IOperationBehavior.ApplyClientBehavior(OperationDescription operationDescription, ClientOperation clientOperation)
    {
        // Apply our own formatter
        clientOperation.Formatter = new SourceRconOperationFormatter(operationDescription);

        // This behavior takes care of both request and replies at client
        clientOperation.SerializeRequest = true;
        clientOperation.DeserializeReply = true;
    }

    void IOperationBehavior.ApplyDispatchBehavior(OperationDescription operationDescription, System.ServiceModel.Dispatcher.DispatchOperation dispatchOperation)
    {
        // Client-side only
        throw new NotImplementedException();
    }

    void IOperationBehavior.Validate(OperationDescription operationDescription)
    {
        // Always valid?
        // In our case we cannot have ref or out parameters
        // We could check that parameters are primitive types
    }

    #endregion
}

The ApplyClientBehavior method set the Formatter property of the client operation to which the behavior is applied. It also expresses that the client operation will serialize and deserialize both request and reply messages.

Inside the Validate method we could check whether the operation is compatible with the behavior. For example, we should check that there are no ref or out parameters. We could check that the parameters are either representable as a string, and whether the return value is either of type string (the reply messages always contain string data) or that it is deserializable from a string. I have not been able to get the validate method to be triggered and all implementations of IOperationBehavior inside the framework have an empty method body. With Reflector you can see that the client validations on behaviors should go off when the ChannelFactory is opening. I’m not sure why it does not fire.

You can go the extra mile and have your behavior derive from Attribute:

[AttributeUsage(AttributeTargets.Method)]
public sealed class SourceRconOperationFormatterBehaviorAttribute : Attribute, IOperationBehavior

so it can be used as an attribute above a method:

[ServiceContract]
public interface IRcon
{
    [OperationContract(Action = "status")]
    [SourceRconOperationFormatterBehavior]
    string GetStatus();

You would probably want to change the name of the behavior/attribute class to something like SourceRconOperationAttribute so it looks like [SourceRconOperation]. This makes more sense. In our case the behavior is not really optional as we need to apply the operation formatter for every operation in a service contract to make things work. Having an interface that is only partially usable for remote control seems weird.

Having made the remarks above on how behaviors are non-optional, the next topic will be about a streamlined way to accomplish that all operations in a service contract will have the operation formatter behavior applied.

WCF Extensibility part 3: Formatting operations

Alex Thissen — Thu, 20 Aug 2009 22:00:00 GMT

Time to get serious and drill down into the first area of the big picture. We are going to take a look at “operation formatting”, which is roughly located in the highlighted area.

One of the crucial parts of the WCF channel layer is the way that method invocations get transformed into WCF messages. This capability allows the end-programmer to work with the object-oriented paradigm. This means you call methods on proxy objects, instead of having to fiddle with the messages going back and forth between a client and service yourself.

Let’s say we have a proxy that was created like so:

[ServiceContract]
public interface IRcon
{
    [OperationContract(Action = "kick")]
    string Kick(string steamID);
}

public interface IRconProxy : IRcon, IClientChannel { }

class Program
{
    static void Main(string[] args)
    {
        SourceRconTcpBinding binding = new SourceRconTcpBinding("abc123");
        string uri = "raw.tcp://192.168.100.101:27015";
        IRconProxy proxy = null;
        ChannelFactory<IRconProxy> factory = new ChannelFactory<IRconProxy>(binding, new EndpointAddress(uri));
        factory.Endpoint.Behaviors.Add(new SourceRconEndpointBehavior());
        factory.Open();

        proxy = factory.CreateChannel();

        string reply = proxy.Kick("1:0:1337");

Take a look at the last line. There you can see a method call being made on the proxy that was created with a call to ChannelFactory<IRconProxy>.CreateChannel. We want to influence the way this method call gets transformed into a specialized System.ServiceModel.Messaging.Message class, called SourceRconMessage. The regular transformation would change a method call into normal Message objects. Our message object holds the spcifics for the Source rcon protocol. Here is the shape of the class, corresponding to the elements of the rcon protocol:

public class SourceRconMessage : Message
{
    public int RequestID { get { ... } }
    public ServerData ServerData { ... } }
    public string String1 { get { ... } }
    public string String2 { get { ... } }
}

We will get back to the specifics of the message in a later post.

The process of changing a method invocation to a message and vice versa is called operation formatting. The key interfaces for all classes that performs this task are System.ServiceModel.Dispatcher.IClientMessageFormatter and IDispatchMessageFormatter. They are essentially the same but mirrored, because they represent the formatting that will happen at the client and service respectively.

public interface IClientMessageFormatter
{
	object DeserializeReply(Message message, object[] parameters);
	Message SerializeRequest(MessageVersion messageVersion, object[] parameters);
}

We will only look at the client message formatter, since we are building a client-side only implementation of the channel stack. The service’s side would have DeserializeRequest and SerializeReply, mirroring the direction requests and replies are sent and received.

Let’s look at each of the implementations of the two methods of the IClientMessageFormatter interface:

// Changes a list of parameters and operation description into a SourceRconMessage
public Message SerializeRequest(MessageVersion messageVersion, object[] parameters)
{
    SourceRconMessage message = new SourceRconMessage(operationDescription.Messages[0].Action, parameters);
    message.Headers.MessageId = new UniqueId(message.RequestID.ToString());

    return message;
}

No rocket science here. The process of changing the action and parameters of the method call into a message is performed by a custom-built constructor on the SourceRconMessage class. The one thing to remember for later though is the mysterious property of MessageId in the Headers of the message that we set. This will be revisited at a later stage.

// Goes from SourceRconMessage to return value and parameter list
public object DeserializeReply(Message rawMessage, object[] parameters)
{
    // Deserialize message and return value
    object returnValue = null;
    SourceRconMessage message = (SourceRconMessage)rawMessage;

    // TODO: Use returnParameter.ParameterType to get the type of return parameter
	// and deserialize to that type from string
    returnValue = message.String1;

    return returnValue;
}

The DeserializeReply method will get a deserialized reply message that was handed from the channel stack. This should be a SourceRconMessage object (which we should probably check for to be sure it really is). Right now we only extract the single valuable piece of information from the message: the String1 property. It is passed back as the return value, which will work as long as the return value is of type string. Notice how there is no type information available for the return value and parameters.

The two methods above go into the SourceRconOperationFormatter class.

public class SourceRconOperationFormatter: IClientMessageFormatter
{
    // Goes from SteamRconMessage to return value and parameter list
    public object DeserializeReply(Message rawMessage, object[] parameters) { ... }

    // Changes a list of parameters and operation description into a SteamRconMessage
    public Message SerializeRequest(MessageVersion messageVersion, object[] parameters) { ... }
}

The class can contain much more. A later and improved version will have a special constructor to hold the operation description details to be able to have type information on the parameters and return value for the sake of DeserializeReply.

WCF Extensibility part 2: The big picture

Alex Thissen — Thu, 20 Aug 2009 21:20:36 GMT

Before I drill down into each of the smaller parts of the client-side channel stack implementation, I want to sketch out an overview of some of the moving parts of the channel layer. This picture is by no means complete, and I will probably revisit it several times over the next couple of posts. It should however provide you with a better understanding of what we are trying to accomplish in what part of the channel layer. And remember, this picture is a sketch and may not be 100% exact or correct.

What you see in the picture is the client-side of the WCF channel layer. I am trying to convey here that from the method call to the actual transport several things happen along the way.

First of all, when you call a service (or something else at the other end of the transport) you make a method invocation on a proxy. This might be

a raw transparent proxy like we created in the first part of the series with a call to ChannelFactory<T>.CreateChannel.
a ClientBase derived proxy, like the one you get when you use Add Service Reference (ASR). This proxy uses the same transparent proxy on the inside.

This method call is translated to a message object representing the method call. This message is dropped into a stack of channels that will massage the message every which way. The different channels in the stack all have their specific purpose, such as security, reliable sessions, or transport. The last channel in the stack is always the transport channel. The final transport channel will encode the message using an encoder and will serialize the bytes across the transport, whatever that may be (HTTP, MSMQ, TCP, UDP, email, …).

For Request/Reply message exchange patterns there will come a stream of bytes in response. The stream will be decoded by the message encoder, which gives another message object. The same holds true for duplex messaging, but for duplex the incoming request stream might be initiated by the service, without a previous message from us, the client. We will not encounter duplex/callback messages. Finally, the response message will be turned into the completion of the method call by yielding the return value and ref or out parameters. The method call returns to the caller. You may have guessed that this is for synchronous method calls on the proxy and you are absolutely correct. For now, we will leave asynchronous operations out of scope.

For each of the following posts I will highlight the area that we focus upon. Any feedback is welcome.

WCF Extensibility part 1: background on game server protocols

Alex Thissen — Mon, 17 Aug 2009 20:38:00 GMT

Talking to game servers

There are various kinds of multiplayer games that allow you to battle against other gamers. I’ll focus on the First Person Shooters (FPS) and Real Time Strategy (RTS) games and skip the (Massive Multiplayer Online) Role Playing Games (MMORPG and RPG). Usually the FPS and RTS games will connect to a central server that is located on your LAN or out in the Internet. The game players will interact by means of the server using a special protocol. The administrators of the server are regular game players for most of the time, but have elevated privileges to control the game server. Typical things that an admin will do is change the currently played map, restart game rounds, and kick or ban misbehaving players from the server.

This type of control can be done “locally” when the admin is one of the players connected to the server, or remotely. The latter is called remote control and it uses a special protocol particular to the type of game engine. Luckily most of these protocols are reasonably well documented. There are not that many game engines out there and hence a small number of protocols. The most famous ones are:

Unreal (1, 2 and 3)
Quake (1, 2 and 3)
Half-Life (aka Goldsource, based on Quake engine)
Source
Doom 3

Right now, we’ll zoom in on the Source engine with its protocol for remote control of game servers based on this engine.

Issuing commands

If you have a Source-based game like Half-Life 2 or Team Fortress 2 it is pretty easy to use control from the game interface. You can simply connect to the server to control by joining a multiplayer game. Once on the server you can access a console window to send commands. A comprehensive list of commands is available. Here are some examples of common commands:

rcon_password abc123

changelevel de_dust2
kick 1:0:12345
mp_restartround 1

The first command rcon_password is needed to prevent unwanted control of a game server.

You can even use remote control by specifying the server address and password first, so you do not have to connect to the game at all.

rcon_address 192.168.100.101:27015
rcon_password abc123

The game itself is needed to control the server, because the game knows the protocol specific to the engine. But other than that it is inconvenient that you would need to own and start the game just to control the server. There are a fair number of code implementations out there that capture the protocol and allow a programmer to use the API to control a server remotely. I wrote several myself, which is a nice programming exercise with an interesting theme of gaming. Well, at least it beats the common customer/order/product samples.

Understanding the protocol

The Source engine is created and maintained by Valve and their developer wiki holds a pretty good explanation of the Source remote control (rcon) protocol.

Here’s my take on the protocol:

Source uses a TCP connection from the client to the game server. After connecting the client should perform an authentication handshake. If the authentication was successful, you can start issuing commands. The protocol uses multiple message exchanges between client and server to accomplish both. Every message has the same structure:

packet size (int): entire message excluding packet size integer. It must be at least 10.

request id (int): random number used for correlating responses to requests

server data (int): depending on purpose of message the server data has one of the following values

SERVERDATA_AUTH = 3: initial authentication handshake message
SERVERDATA_EXECCOMMAND = 2: command message
SERVERDATA_RESPONSE_VALUE = 0: response from command message
SERVERDATA_AUTH_RESPONSE = 2: acknowledgement of authentication message

string1 (null-terminated string): either command to run, password or result

string2 (null-terminated string): reserverd, must be empty string ("");

A typical rcon session will look like this:

Let’s take a look at some of the individual packets. Let’s assume that the password is “abc123” and that the command that was issued is “changelevel de_dust2”

	Authentication	Junk	Acknowledgment	Exec command	command reply
packet size	4 + 4 + 7 + 1 == 16	4 + 4 + 1 + 1 == 10	4 + 4 + 1 + 1 == 10	4 + 4 + 21 + 1 == 30	4 + 4 + (string1.length + 1) + 1 == 10
request id	0x2e0e0000	0x2e0e0000	0x2e0e0000	0xd2010000	0xd2010000
server data	3	0	2	2	0
string 1	“abc123\0”	“\0”	“\0”	“changelevel de_dust2\0”	“\0” (command specific, varies depending of success or failure of method)
string 2	“\0”	“\0”	“\0”	“\0”	“\0”

After the exec command and corresponding reply packets you can continue issuing exec commands.

The sequence of message above is the ideal picture. In reality things might be a little more difficult, because:

a reply may consist of multiple reply packets.
authentication might not be successful because of a wrong password.

Eventually we will take everything into account, but for now we will pretend that multi-packet responses do not exist and that authentication is always successful.

If that works, the exchange of raw bytes across the network should look something like this (for the 5 messages listed above):

0000   10 00 00 00 2e 0e 00 00 03 00 00 00 61 62 63 31  ............abc1
0010   32 33 00 00                                      23..

0000   0a 00 00 00 2e 0e 00 00 00 00 00 00 00 00        ..............

0000   0a 00 00 00 2e 0e 00 00 02 00 00 00 00 00        ..............

0000   1e 00 00 00 d2 01 00 00 02 00 00 00 63 68 61 6e  ............chan
0010   67 65 6c 65 76 65 6c 20 64 65 5f 64 75 73 74 32  gelevel de_dust2
0020   00 00                                            ..

0000   0a 00 00 00 d2 01 00 00 00 00 00 00 00 00        ..............

Next steps

Our task at hand is to learn WCF to speak this protocol, while maintaining the channel layer concepts.

Windows Communication Foundation extensibility to the max

Alex Thissen — Mon, 17 Aug 2009 19:27:00 GMT

Last week I visited Microsoft at the Redmond campus and spent the larger part of the week with the Windows Communication Foundation and Workflow Foundation team. I spoke with a lot of the team members and managed to get good insights into the things to come in WCF/WF 4.0.

I got a chance to talk to Nicolas Allen and Kenny Wolf. Besides the work-related issues that I wanted to investigate I also discussed my long-running attempt to create a very odd WCF client implementation. They gave me a couple of hints on how to take the last hurdles to get everything working. While still shrugging off the last bits of jetlag I managed to get the first implementation up and running. Although WCF was never designed with such weird things in mind, this proves that WCF is extensible beyond your imagination. I’ll walk you through it all.

Part 1: Background on game server protocols
Part 2: The big picture
Part 3: Formatting operations
Part 4: Applying formatting to operations
Part 5: Make your endpoints behave

But, just as a teaser preview, here are some code snippets of what this will all amount to:

[ServiceContract]
public interface IRcon
{
	[OperationContract(Action = "status")]
	string GetStatus();
	[OperationContract(Action = "changelevel")]
	string ChangeLevel(string mapName);
	[OperationContract(Action = "mp_restartround")]
	void Restart(int seconds);
}

public interface IRconProxy : IRcon, IClientChannel { } 

class Program
{
	static void Main(string[] args)
	{
		SourceRconTcpBinding binding = new SourceRconTcpBinding("abc123");
		string uri = "raw.tcp://192.168.100.101:27015";
		IRconProxy proxy = null;

		try
		{
			ChannelFactory<IRconProxy> factory = new ChannelFactory<IRconProxy>(binding, new EndpointAddress(uri));
			factory.Endpoint.Behaviors.Add(new SourceRconEndpointBehavior());
			factory.Open();

			proxy = factory.CreateChannel();
			proxy.Open(TimeSpan.FromSeconds(10));

			string reply = proxy.GetStatus();
			proxy.Restart(3);
		}
		catch (Exception ex)
		{
			Console.WriteLine(ex);
		}
		finally
		{
			proxy.Close();
		}
	}
}

Here, the most interesting lines of code are 31 and 32: a WCF client channel talking a proprietary, non-SOAP protocol over a TCP connection.

StackOverflow DevDays 2009

Alex Thissen — Thu, 06 Aug 2009 07:13:20 GMT

Most of you must know the community site StackOverflow.com. But did you know that StackOverflow is also organizing the StackOverflow DevDays? “A full day of inspiration and learning for developers brought to you by the team behind StackOverflow.com and FogBugz”. The day will feature speakers like Joel Spolsky, Jeff Atwood, Scott Hanselman, Jon Skeet and many others. The DevDays tours several cities in the United States, but they are also coming to Europe! Two editions of the DevDays will take place in England (London and Cambridge) and the last one is in Amsterdam, The Netherlands. Not all speakers will make an appearance everywhere, so make sure you book the right edition to meet your favorite speaker.

Some details are on the Amsterdam edition still have to be determined, such as venue. I will be presenting a code-only (or very low PowerPoint) presentation on ASP.NET MVC. This presentation will be in English and I guess the others are in English as well given the expected international audience.

As soon as more details are available, I will update them here. I do know that you will have to hurry, as the seats are running out. Almost every edition in the United States is sold out. In Europe, London is sold out, while Cambridge has 45 and Amsterdam 35 seats left as the time of this blogpost.

Go book your seat now!

Catching the .NET Service Bus part 1: Exposing a service on the bus

Alex Thissen — Tue, 21 Jul 2009 19:58:00 GMT

The last couple of evenings I spent some time getting familiar with the .NET Services and the Service Bus in particular. Here is a story of how I got my sample service to be available on the .NET Service Bus.

I assume that you are familiar with the .NET Service Bus already; otherwise you can read some good articles on MSDN (documentation and two articles) and Clemens Vasters’ blog.

My goals were easy: what does it take to get a pretty standard WCF service up and running using the .NET Service Bus. For completeness sake, this is the service contract (sticking with my ever lasting theme of gaming):

namespace GamingServiceLibrary
{
	[ServiceContract]
	public interface IGameServerManagement
	{
		[OperationContract]
		GameServerStatus RetrieveServerStatus(IPAddress address, int port);
	}
}

I wanted to have the request/reply styled service implementation available with message security enabled. Like myself you will need a .NET Services account to be able to use the .NET Services (Service Bus and Access Control service) and have a so-called solution (mine is named “KillerApps”). You will find that solution names must be globally unique, so don’t be late and find out your intended name is already taken!

Hosting and exposing your service

First of all you need to have the service host register your service endpoint on the service bus. A lot of the plumbing is taking care of by the NetTcpRelayBinding, which does all the negotiation with the bus. All we have to do is choose an endpoint on the service bus and authenticate ourselves to the bus. For simplicity I chose the following name “GameServerService” as the significant part of the endpoint address.

To create the complete endpoint address you can use a helper method:

Uri address = ServiceBusEnvironment.CreateServiceUri("sb", solutionName, "GameServerService");

which will generate an URI like so sb://KillerApps.servicebus.windows.net/GameServerService. The italic/bolded parts are the pieces that get filled in for you. The scheme corresponds to the type of relay binding. For TCP it will be sb, for basic, WS, federation and web HTTP bindings it will be http or https.

The service host needs some preparations that can be done through code or config. I did parts in both.

string solutionName = "KillerApps";
string solutionPassword = "test1234"; // Nope, not my real password and you should replace it with your own

// Create the credentials object for the endpoint.
TransportClientEndpointBehavior userNamePasswordServiceBusCredential = new TransportClientEndpointBehavior();
userNamePasswordServiceBusCredential.CredentialType = TransportClientCredentialType.UserNamePassword;
userNamePasswordServiceBusCredential.Credentials.UserName.UserName = solutionName;
userNamePasswordServiceBusCredential.Credentials.UserName.Password = solutionPassword;

// Create the service host reading the configuration.
ServiceHost host = new ServiceHost(typeof(GameServerService), address);

// Create the ServiceRegistrySettings behavior for the endpoint.
IEndpointBehavior serviceRegistrySettings = new ServiceRegistrySettings(DiscoveryType.Public);

// Add the Service Bus credentials to all endpoints specified in configuration.
foreach (ServiceEndpoint endpoint in host.Description.Endpoints)
{
  endpoint.Behaviors.Add(userNamePasswordServiceBusCredential);
  endpoint.Behaviors.Add(serviceRegistrySettings);
}

// Open the service.
host.Open();

The approach above is from the Getting Started sample in the .NET Services SDK for July 2009. Nothing fancy, just with some fixes:

The sample duplicates the line for setting the username instead of one for the password.
Also, the sample uses the old URIs for the service bus endpoints. You can now use the helper method).

A quick disclaimer: do not store your password in your code or config like this. Keep it safe and encrypted.

A service that wants to expose an endpoint on the Service Bus always need to authenticate, so not any odd service can claim an endpoint with your solution name in it. Likewise, the client needs to authenticate, so the service bus can check whether it is allowed to access a particular service endpoint. The endpoint behavior for password based authentication is prepared and added to all endpoints. The behavior takes care of passing a username (always the name of the solution) and password to the bus.

There are other authentication types, such as CardSpace and certificates. The place to configure each of these authentication types is in another part of the .NET Services: the Access Control Service. That’s a topic for another post, but when you are looking for some guidance on the combination of CardSpace and the bus I can recommend the Hands-on Lab “.NET Services: Introduction to the Access Control Service” from the Geneva Training Kit. The host of your service chooses the authentication type it wants to use and the Access Control Service will perform the authentication.

Also, the service will be added to the registry discoverable for everyone through the ServiceRegistrySetting endpoint behavior.

The other part needed for successful hosting resides in the config file. The service’s configuration holds two endpoint definitions:

<services>
  <service behaviorConfiguration="GamingServiceBehavior" name="GamingServiceLibrary.GameServerService">
            <endpoint binding="netTcpRelayBinding" bindingConfiguration="default" contract="GamingServiceLibrary.IGameServerManagement">
                <identity>
                    <dns value="netbook01"/>
                </identity>
            </endpoint>
    <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
    <host>
      <baseAddresses>
        <add baseAddress="http://netbook01:8732/Design_Time_
Addresses/GamingServiceLibrary/GameServerService" />
      </baseAddresses>
    </host>
  </service>
</services>

Looking at this config section you can see two endpoints: one for the actual service contract implementation and one for metadata exchange. The base address in the host element does not really matter for the first endpoint, because we are providing the sb:// specific adress in the code. Also, both the binding and the service have configurations that are being referred to.

The binding has this configuration:

<bindings>
    <netTcpRelayBinding>
        <binding name="default" connectionMode="Relayed">
            <security mode="Message">
                <message clientCredentialType="UserName" algorithmSuite="Default" />
            </security>
        </binding>
    </netTcpRelayBinding>
</bindings>

It specifies the connection mode of the service to the service bus and the client later on. Relayed means a client will be relayed through the service bus. Direct and Hybrid are the other options. The former allows a direct connection between client and service after service bus negotiation (which might not turn out to be possible). The latter is a mode where direct is attempted with a fallback to Relayed when direct connections fail.

The <security> element configures the security of the messages. This is completely orthogonal to the way authentication has occurred. Security is optional to protect the communication between client and service, whereas authentication does not protect but is mandatory for exposing a service on the bus.

Summary

Hosting and exposing your service on the .NET Service Bus requires you to use one of the special Relay binding that are new in the Microsoft.ServiceBus assembly. Upon connection, the binding will negotiate the details of the endpoint with the host. In the process the binding will attempt to authenticate the host to the bus.

Next time we will take a look at how to enable message security and I will also explain the security configuration, the endpoint behavior and <identity> element that is skipped over for now.

Another year of MVP

Alex Thissen — Fri, 10 Jul 2009 11:44:38 GMT

Thanks to the community for appreciating my efforts enough to have Microsoft let me be Most Valuable Professional for another year.

A bit late in reporting this. I guess my new job was keeping me occupied. Hopefully lots of articles and conference talks will come from all the new stuff I will encounter at Achmea.

Bye Class-A, hello Achmea

Alex Thissen — Tue, 30 Jun 2009 17:32:12 GMT

Today was my last day as an employee of Class-A. Some two months ago I started talking with Achmea about a position there. We were both convinced that there would be real nice opportunities. I have decided to take them up on the offer and will be joining their ranks as an architect.

This means that I will be moving away a little from teaching and coaching of programmers. Surely I will continue teach others, but with more of a focus on the Achmea developers. This does not mean that I will disappear completely. No way. You will be seeing me at the various conferences, events and the community at large. At Achmea I will probably see a lot of new things that should give me a lot to talk and blog about (who knows I might just get a more frequent rhythm for blogging).

I have worked for exactly 4 years at Class-A and enjoyed my time there immensely. The primary reason for me to move along was a focus shift to being an architect. I have acted in that role on my last two larger projects at customers and had a lot of fun in doing that.

I will certainly miss all of my colleagues at Class-A, their wit and the discussions at our “Kennis-sessies”. Our ways will not part completely and I am convinced I will meet them ever so often. Thanks to Anko and Astrid for having me on board the past years.

In a while I hope to give an update on what I am doing at Achmea and the thing that I learned and can share with you.

Developer Days 2009

Alex Thissen — Tue, 02 Jun 2009 21:08:25 GMT

Last week Microsoft The Netherlands held another great edition of the Developer Days. With a fantastic line-up of speakers and a smooth organization I guess everyone must have enjoyed it. I know I had a good time and hope you did as well.

Thanks to those of you who attended any of my sessions. I have uploaded my slides and demo code to the Media section of my site.

Production debugging ASP.NET applications (slides and demos)
”Hoe vaak heb je al meegemaakt dat de ASP.NET applicatie op jouw ontwikkelmachine perfect draait, maar dat er van alles mis gaat op de test- of zelfs productie-omgeving? De problemen die optreden kunnen varieren van bugs, onverklaarbare excepties, memory leaks, hangende pagina’s tot zelfs crashes van de webserver. Als je geen Visual Studio hebt is het lastig te achterhalen wat er mis zou kunnen zijn en zelfs Visual Studio kan je niet altijd helpen in sommige van deze situaties. Er is hoop: in deze sessie zul je een aantal praktische tools en technieken leren om veel productie-problemen te achterhalen en op te lossen. Verplichte kost voor iedere ASP.NET ontwikkelaar.”
An overview of Oslo (slides and demos)
”Microsoft werkt met Oslo aan de volgende generatie van een ontwikkelplatform voor applicaties. Oslo gebruikt domein-specifieke modellen, de taal M en nieuwe tooling om je productiever te laten worden bij het ontwikkelen van applicaties. Modelleren van applicaties speelt hierbij een centrale rol. Maar, wat bedoelt Microsoft met modelleren en welke rol gaat Oslo spelen in je .NET applicatie? Deze sessie geeft een overzicht van Oslo en laat je zien wat de visie van Microsoft is ten aanzien van executable modellen, runtimes en het ontwikkelen van applicaties met behulp van Oslo in het algemeen. De vervolgsessie "The M Language of Oslo" zal dieper ingaan op de taal M. Deze sessie is daarvoor een goede voorbereiding.”
The M language of Oslo (slides and demos)
”Het Oslo modelleer platform heeft een eigen taal genaamd M. In deze sessie zul je leren hoe je met behulp van M de schema's van een model definieert en nieuwe modellen aanmaakt. Daarnaast krijg je te zien dat je met M ook een textuele Domain Specific Language (DSL) kunt maken. Verder gaan we in op de tooling van M, zoals de M compiler voor het compileren en uitrollen van modellen naar de Oslo Repository. Het is aan te raden de eerder geplande Oslo Overview sessie bij te wonen, als je een goed overzicht van het Oslo modelleer platform wilt hebben.”

I do apologize for some of the demos in the production debugging talk not running as smoothly as I had intended them to. It turned out that having too little disc space is not very good for your demos. Although I had prepared several backup demos and strategies, they could not save me from that situation all the time. I hope it wasn’t too disturbing and that the message of how to debug and which tools to use still came across.

Let me leave you with two pictures from slidedecks:

on how and when to take a dump.

And one slide that I removed from the Oslo Overview slidedeck (which is a enhanced version of an earlier presentation “Oslo explained” I did for the Stichting DotNed. I had no idea whether any people at the DevDays would be offended, hence I took it out just to be safe. I still think it is a good question with an answer that really describes what Oslo as a modeling platform is all about. (Nope, although the woman called Pernille Holmboe, is a model from Oslo, it is not the correct answer to the popquiz). Drop me a line if you want some more details on the machine.

Do you have an ASP.NET production problem?

Alex Thissen — Thu, 07 May 2009 20:43:49 GMT

At the end of the month I will present at the Developer Days. One of my sessions is about ASP.NET debugging in production(-like) situations. To demonstrate several tools and techniques I have prepared some cases that I encountered at various companies where I troubleshooted applications. I thought it might also be nice to present some other situations where people are or have been struggling to find a bug or solve a production problem.

Do you have a production problem or solved one? This could manifesting itself like so:

weird and slow response times (not performance related)
“hanging” pages that seem to have unexplainable wait times
memory leaks
crashing application pools or unexpected restarts

I know the following might be a long shot, but who knows. In case you’re interested, send me your details and see if we can work something out to have me investigate your problem. As is nearly always the case there are some rules and limitations to this offer:

No guarantees that I will find your bug or solve the problem
You should be comfortable with me investigating your applications. I will probably not need your source code, but chances are you need to give me your dump files. These will also contains a lot of information about your application.
If I can solve the case, I might want to show it at the Developer Days. Obviously in a simplified situation to isolate the problem or bug and without any ties to your source or dump.

So, to summarize: if you have an ASP.NET production problem (works on your dev machine, but not on the test or production server) and weren’t able to fix it yourself and found no one who could, why not drop me a mail (athissen AT killer-apps DOT nl) and find out if I can make the difference. I will make sure you get details on retrieving information that helps on tracking down potential problems. If you’ve got any situations you were able to solve that give a nice demo, I would love to hear about it as well.

Using Unity to do poor man’s tracing

Alex Thissen — Wed, 25 Mar 2009 17:02:00 GMT

For this post I will assume you are familiar with Dependency Injection, Inversion of Control and Unity. There are a couple of tutorials out there that show you the basics of all of these.

OK, say you have got a class Widget that others depend upon, and that you have an interface IWidget to loosen the coupling.

public interface IWidget
{
	string Tagerize(bool leetified, ref string nickName, out int skillFactor);
	void Fizz();
}

Never mind the silly names, I was on some kind of high when I wrote it. Also, the ref and out are there for a purpose. To resolve the Widget object through an IWidget reference, this is what you would do.

bool leetified = false;
int skillFactor;
string nickName = "IncredibleElmo";

IWidget tracedViaInterface = container.Resolve<IWidget>();

// Direct invocation of method
string result = tracedViaInterface.Tagerize(leetified, ref nickName, out skillFactor);

When dependencies of type IWidget are resolved in your application like above, or via a “cascaded” resolution (as part of a dependency of a dependency, and so on), you are ready to do some fancy stuff. Unity puts you in control of the creation of the depended-upon object, meaning that you can tell Unity to do cool things and impress all of your friends.

For example, say your colleagues (or whomever you want to put the blame on) haven’t done much about tracing/instrumentation facilities in the application’s code. With Unity you can do some quick instrumentation. It can give you tracing for calls to Unity-created objects. Here’s how that would go.

First, you’ll need Unity 1.2 (or higher) which is part of Enterprise Library 4.1. Since version 1.2 there is a marriage between the Policy Injection Application Block (PIAB) and Unity, essentially giving you PIAB with the ease of Unity. The marriage can be found in a Unity container extension called “Interception” inside the assembly Microsoft.Patterns.Unity.Interception.

container
	.AddNewExtension<Interception>();
	.Configure<Interception>()
	.SetInterceptorFor<IWidget>(new InterfaceInterceptor());

This additional configuration of the Unity container uses the interception mechanism to give you a man-in-the-middle object through your interface (call it a interceptor or proxy) to the actual Widget object. The interceptor can do additional work for you on all or some calls to the Widget object. You would like to trace these calls.

For the tracing to work you need a call handler that knows what to do during the interception of the call. Here’s the code for my TraceCallHandler:

public class TraceCallHandler : ICallHandler
{
	public TraceCallHandler() { }

	#region ICallHandler Members

	public IMethodReturn Invoke(IMethodInvocation input, GetNextHandlerDelegate getNext)
	{
		IMethodReturn result = getNext().Invoke(input, getNext);

		// Do interception tracing
		TraceCall(input, result);

		return result;
	}

	private void TraceCall(IMethodInvocation input, IMethodReturn result)
	{
		StringBuilder builder = new StringBuilder();
		builder.AppendFormat("{0}::{1}(", input.MethodBase.ReflectedType.Name, input.MethodBase.Name);

		List<string> arguments = new List<string>();
		bool first = true;
		for (int index = 0; index < input.Arguments.Count; index++)
		{
			if (!first) builder.Append(", ");

			string paramName = input.Arguments.ParameterName(index);
			ParameterInfo info = input.Arguments.GetParameterInfo(paramName);
			builder.AppendFormat("{0} = {1}", info, input.Arguments[paramName]);

			if (info.ParameterType.IsByRef || info.IsOut)
			{
				for (int counter = 0; counter < result.Outputs.Count; counter++)
				{
					if (result.Outputs.ParameterName(counter) == info.Name)
						builder.AppendFormat("=> {0}", result.Outputs[counter]);
				}
			}

			first = false;
		}

		builder.Append(")");
		if (result.ReturnValue != null) builder.AppendFormat(" => {0}", result.ReturnValue);

		// Log string
		Log(builder.ToString());
	}

	private void Log(string text)
	{
		Debug.WriteLine(text);
	}

	public int Order { get; set; }

	#endregion
}

If you want to have a look at what this does, focus on the Invoke first. That is the action that the call handler will perform, should the interceptor object choose to call this handler. Invoke will do the actual call on our Widget target object (but it doesn’t even need to) and then trace the call using the contextual information. The TraceCall method is some fancy outputting to the Debug window (or to DbgView if you’re not debugging).

You will need to instruct the interceptor to consider calling your handler. You will need an attribute on whatever you want to handle: your interface method or the entire interface e.g.

public class TraceableAttribute : HandlerAttribute
{
	public override ICallHandler CreateHandler(IUnityContainer container)
	{
		return new TraceCallHandler();
	}
}

By applying this attribute to the traced class, the interface, one or more of its methods you should be good to go. For completeness sake I give you my Widget implementation. Feel free to use the ToBrEzAhCasing extension method in your business logic somewhere.

[Traceable]
public class Widget : MarshalByRefObject, IWidget
{
	#region IWidget Members

	public string Tagerize(bool leetified, ref string nickName, out int skillFactor)
	{
		nickName = nickName.ToBrEzAhCasing();
		skillFactor = leetified ? 1337 : 42;
		return String.Format("{0} pwns @{1}", nickName, skillFactor);
	}

	public void Fizz() { }

	#endregion
}

public static class StringExtensions
{
	public static string ToBrEzAhCasing(this string text)
	{
		int index = 0;
		return new String(text.ToCharArray().Select(c => index++ % 2 == 0 ? Char.ToUpper(c) : c).ToArray());
	}
}

The output is as follows:

IWidget::Tagerize(Boolean leetified = False, System.String& nickName = InCrEdIbLeElMo, Int32& skillFactor = 42=> 42) => InCrEdIbLeElMo pwns @42
IWidget::Fizz()

You can go all the way and include much more in this tracing. This serves just to show you how to do the interception. The rest is up to you.

In conclusion: what this boils down to is tracing through interception of calls to attributed methods (directly attributed or via an interface or their class) provided the objects have been created by the Unity container and pass through the proxy. I.e. no internal calls will be traced, nor will calls to objects that you created without the use of the container. Hey, I said it was poor man’s tracing. All you need to do is attribute your stuff.

Notes and various other kinds of disclaimers/CMA clauses:

in this article I described the scenario of using an interface to do interception. Wait, there are more options. It is also possible to intercept based on a MarshalByRefObject derived class or a class with virtual methods. These need different interceptor classes (TransparentProxyInterceptor and VirtualMethodInterceptor respectively). There’s more to be found here.
The wiring of the interception extension can be done through configuration as well.
You can apply matching rules in a Unity policy that will get evaluated before using the call handlers. If a match can be made (e.g. based on type, parameter of a method, some Tag attribute, …) then the call handlers of the policy will execute (or not for a mismatch).
You really do not need to build your own log/trace handler. The PIAB ships with a shipload of call handlers, including a LogCallHandler. This handler uses the Logging Application Block to do its logging output. The circle is complete, or whatever. I might just show how to do this in a later post.
Yes, there are some tiny bugs in the TraceCall method. I should not output values of ‘out’ parameters and it would be better to capture the ToString of an reference type before the call instead of only after the call has happened.

SDN Event 30th March 2009 in Driebergen

Alex Thissen — Mon, 23 Mar 2009 21:13:00 GMT

Next Monday, 30th of March there will be an SDN event in Driebergen, The Netherlands. The central theme of the event is Testing in software. I will be there talking about two test-related topics (descriptions in Dutch):

Make your software more testable with Unity:
”Software schijnt beter testbaar te worden wanneer je allerlei design patterns inzet. Termen als Dependency Injection (DI) en Inversion of Control (IoC) worden daarbij regelmatig genoemd. Maar hoe zit dat dan?
In deze sessie zul je niet alleen uitleg krijgen over de design patterns, maar ook het Unity framework voor IoC. Aan de hand van een concrete case wordt duidelijk gemaakt hoe hiermee betere unit tests en beter testbare code geschreven kunnen worden.”
Testing with Visual Studio 2008:
”Visual Studio 2008 biedt een scala aan mogelijkheden voor het testen van de software die je ermee ontwikkelt. In deze sessie zul je zien dat VS2008 niet alleen unit testing ondersteunt, maar ook web en load testen. Deze sessie bevat weinig slides en heel veel demos om je alle voorzieningen voor testen in Visual Studio te tonen.”

Afterwards I will make the slides and demos available as usual.

Hope to see you there.

Disappearing testrunconfig dialog for Code Coverage

Alex Thissen — Wed, 18 Mar 2009 10:00:37 GMT

Ever had this happen: Whenever you need to make changes to the code coverage settings of a Visual Studio 2008 solution, the “testrunconfig” dialog simply disappears before you get a chance to edit any setting. I’m talking about this dialog:

an clicking the item highlighted with the yellow marker.

Martin Platt mentions this is because of certain custom project templates) and mentions the BizTalk and Reporting Services project templates.

The Database projects from the Visual Studio 2008 Database Edition (aka DataDude) are also culprits of closing the dialog. This is for the VS2008 SKU of Database with the GDR release.

The workaround is to right-click the database projects and choosing Unload Project from the context menu, you c/should be able to visit the Code Coverage section again. If other project types are in the way, you may have to unload those as well. Once you have made the changes you can reload the projects if necessary.

The bug has been filed at Microsoft Connect and appears to be related to another issue regarding empty unit tests. That issue has been fixed in KB962866, but also fixes the disappearing dialog problem. You can get the hotfix from the Code Gallery.

Mocking objects that change arguments

Alex Thissen — Mon, 16 Feb 2009 21:01:20 GMT

Most mocking framework know how to mock an interface or (abstract) class and fake the method calls that are performed on these mocks. However, hardly any of them can do anything other than provide a return value when the mock method is called.

This poses a serious problem when ref or out parameters are provided or when reference types are used that need to be manipulated as part of the mock method. Here's a scenario in which I needed to do exactly that.

The System.Net.Sockets.Socket class can send and receive raw data through a byte[] buffer. Take a look at some of the signatures for the Socket::Send method:

public int Receive(byte[] buffer)
public int Receive(IList<ArraySegment<byte>> buffers, SocketFlags socketFlags);

When the Receive method is called, a byte[] buffer or list of ArraySegments of byte buffers is passed. The return value of Receive will indicate the actual number of bytes that were received. But, ... more importantly, the buffer now contains the bytes read from the socket. How would you go about mocking this?

At a first impression you could think that you would create a wrapper or abstraction of the Socket, give it an interface and mock it from there. True, but this still leaves you with a new class to test. If you strive for a high code coverage, or want to do an integration test up until the Socket level, then this offers no way out.

Enter stage for TypeMock. Using this mocking framework you can handle method calls on mocked objects and have dynamic return values. The key to this all is the DynamicReturnValue class. I wrote this extension method for the Socket class. This extension method prepares a mocked Socket object to fake reception of network data. This data is provided in a packet (an ArraySegment of byte). From this packet the data is read and transferred to the buffer that is passed to the Receive method when it is called later on.

public static void FakeReceive(this Socket socket, ArraySegment<byte> packet)
{
  Mock<Socket> mock = MockManager.GetMockOf<Socket>(socket);
  Assert.IsNotNull(mock, "Socket is not mocked. Supply a mocked socket object");
 
  mock
    .ExpectAndReturn("Receive", new DynamicReturnValue(
        delegate(object[] parameters, object context)
        {
          byte[] buffer = (byte[])parameters[0];
 
          int offset;
          int length;
          if (parameters.Length == 1)
          {
            offset = packet.Offset;
            length = packet.Count;
          }
          else
          {
            offset = (int)parameters[1];
            length = (int)parameters[2];
          }
 
          // Check whether packet has enough data
          Assert.IsTrue(packet.Count <= length, "Size of packet is too small for expected data received from socket.");
          Array.Copy(packet.Array, packet.Offset, buffer, offset, length);
 
          return length;
        }));
}

In this method I expect that the extended Socket object is a mocked object. I arrange the mock to expect and return a DynamicReturnValue for the Receive method. The constructor for the DynamicReturnValue takes a delegate that is called instead of the Receive method on the mocked object. The constructor takes two parameters. The most important one is the object[] parameters, that holds the parameters/arguments of the method called (Receive in our case).

The ExpectAndReturn method does not know which overload of Receive is called. You should have enough control over the situation to know which one it is. The FakeReceive method assumes that the overloads are either of these:

public int Receive(byte[] buffer);
public int Receive(byte[] buffer, int offset, int size, SocketFlags socketFlags);

For the first overload the number of parameters is 1. The entire packet is transferred to the buffer in that case. Otherwise the requested number of bytes is read into the buffer. The provided packet should contain enough bytes. In both cases the number of bytes read is returned.

The next code fragment shows how to use the FakeReceive method in a test fixture.

MockObject<Socket> mock = MockManager.MockObject<Socket>();
Socket socket = mock.Object;
 
byte[] packet = DeploymentItemHelper.LoadBinaryItem(@"GameSpyMaster2MultiPacketResponse\GameSpyMasterResponseHeader.bin");
socket.FakeReceive(packet, 1);
socket.FakeReceive(new ArraySegment<byte>(packet, 1, 16));
socket.FakeReceive(new ArraySegment<byte>(packet, 17, 9));
 
GameSpyMaster2Response response = new GameSpyMaster2Response(key, validation);
response.Receive(socket);

First, the mock Socket is created. Next, data is loaded from a binary test file. The socket is prepared for three consecutive Receive calls that read more and more from the binary data. Lastly, the GameSpyMaster2Response object is tested by calling the Receive method (not the one from Socket, but of the object under test) passing in the mocked and prepared socket.

Works like a charm. There's room for improvement, though. You could allow for faking more overloads of Receive. I just called YAGNI and went on.

Quick tip on Visual Studio 2008 and UAC

Alex Thissen — Mon, 16 Feb 2009 09:43:41 GMT

On a Windows Vista (and presumably Windows Server 2008) that has User Account Control (UAC) enabled you are no longer able to drag and drop files from Windows Explorer into your VS solution. I always used this to quickly add files to a project. What a bummer.

There are 2 workarounds:

Copy or move the files to the correct location in your Visual Studio project folder. Then go into Visual Studio and click the Show All Files button for that project.

You might need to click Refresh (not F5) on the project for the new file(s) to show up. Select them all and right-click. Choose Include In Project from the context menu.

-or-
Instead of dragging the files simply copy and paste them into your project.

Sounds too easy for this to work, but it does.

Option 2 certainly makes it easier and faster for me.

Improvements and changes to Oslo SDK and Repository in January CTP

Alex Thissen — Sat, 31 Jan 2009 17:21:19 GMT

Since yesterday the Oslo SDK January 2009 CTP has been released. I took a quick look to find out what is different from the PDC alpha bits that where released in October 2008.

First off, read the Read Me file found at the download site. It lists a number of changes in detail. To summarize the important changes mentioned:

Token Actions for MGrammar:
"Any production in a token can now have a code action or a graph action (formerly known as term construction)! You can now specify a return type for a token definition in the case of code actions, similar to a syntax definition."
I have to check what this means exactly.

Don Box mentions that he likes the ability to specify a right-hand side in token rules, just like in syntax rules (see above). I assume he means this one.
MGraph syntax supported in MGrammar right-hand-side:
You can now use the following:

syntax EndPoint = uri:Listen =>
ServiceEndPoint { ListenUri = uri, ... }

where previously you were only able to do

syntax EndPoint = uri:Listen =>
ServiceEndPoint { ListenUri { "http://..." }, ... }
Additional M Collection members have been implemented: Count, Choose and Distinct for all collections and All and Exists for Logical*.

Besides changing a typo in the Models\System\Mvc\Activities\ValidatingViewActivity .m file and removal of ViewAcitvity.m in the same folder nothing has changed in the models source code.

In my previous post I explained that the Oslo Repository setup did not install correctly for non-default instances. It contains a better setup procedure, that let's you choose your SQL instance during setup:

It will also let you choose whether to install a repository at all. Not everyone might want to have a repository on their local machine. There's a CreateRepository.htm readme file explaining all of the command-line switches for the CreateRepository.exe utility. Nothing much seems to have changed in these switches

The Oslo SDK setup that follows let's you pick individual items to install.

The Visual Studio integration has some small steps forward as well. If I recall correctly, you could not add references before, which is possible now.

Still a long way to go towards becoming a full-featured project type, but getting closer.

Installing Oslo SDK and creating an Oslo repository from scratch

Alex Thissen — Wed, 28 Jan 2009 20:44:43 GMT

Because my machine doesn't perform well enough when running a Virtual PC image I ventured out to install the Oslo SDK onto my Windows Vista. Turns out to be a worthwhile experience.

Some requirements while trying to install it caught me off-guard:

Windows Installer 4.5 to be able to install the next one (SQL Server 2008).
SQL Server 2008. I installed the Express version which is a free download.

After preparing your machine, you are ready to install.

Install Oslo SDK
Download and install the Oslo SDK October 2008 CTP from the Oslo Development Center. Pretty simple. Next, next, Finish. You will notice that the repository could not be installed if you have an Express edition or non-default instance (cheapskate-me installed the Express edition). Not to worry, just read on. Even if you have got a default instance, you might want to create the repository by hand for reason obvious in the next step.
Create the Oslo repository
Go to the C:\Program Files\Microsoft Repository folder and run the following command:

CreateRepository.exe /RepositoryAuditing+
/SchemaPolicy+ /RepositoryName:TestRepository
/DataFileInitialSize:20MB /DataFileGrowth:10MB
/LogFileInitialSize:20MB /LogFileGrowth:10MB
/Verbose+ /SqlServer:.\SQLExpress

Let's elaborate on some of the command-line options I have specified:

Repository auditing is not enabled by default. Consider turning it on straight away. I have no real arguments for doing so, and I am not sure if it can be turned on afterwards (I guess it should be possible). Same goes for Schema policies. AFAIK policies and auditing are only available for full SQL Server installations (as in: not for Express editions).
The default initial size of the Repository is a whopping 1GB. I have plenty of hard disk space on my machine, but not that much. How about you?
Make sure you also change the log file initial size and the growth sizes for both the data and log file. Otherwise you'll have some 2GB+ less disk space. You have been warned.
The SQL server name is critical if you have a named instance or want to install on a remote server. For a local SQL Express it needs to be '.\SQLExpress'.
Verbose: it's always nice to see what is going on
Specify a different name for the repository if you want to have multiple instances running side-by-side or don't like the default one of 'Repository'.

Backup the repository
Perform a backup of the repository database either at this point or later. This will make it lightning fast to restore a repository database in case of a total mess-up on your behalf or the pre-alpha software (I bet it will be the former). There is no good tooling support just yet for removing models from the repository, so a restore from backup is the recommended workaround. You might consider doing the backup after installing the Microsoft models (see next step).
Install the Microsoft models into the repository
The Oslo SDK has a precompiled image for their Oslo models in it. Check out the "C:\Program Files\Microsoft Repository" folder again and find a model image file called Models.mx. To install this into the repository type:

mx.exe -i:models.mx -s:.\SQLExpress -db:Repository

I found that this image and one that you compile by hand from the model source code (at "C:\Program Files\Microsoft Oslo SDK 1.0\Models") are not equivalent.
All done
Yep, you're ready to go.

Early thoughts on MEntity

Alex Thissen — Sat, 15 Nov 2008 08:06:00 GMT

Even though no details on MEntity have been revealed yet, here are my thoughts on MEntity and its place and purpose in the world of "Oslo". This current thinking has been fed by the discussions Paul Gielens and I had with Don Box. Here goes.

A model and a DSL for the Entity Framework

Don mentioned that CSDL/MSL/SSDL (or the EDMX file format for that matter) is essentially the Entity Framework's model. The "metadata" in these files can be perceived as the model driving any application using the Entity Framework. The EF runtime consume and directly execute the model data, making the EF Oslo model an executable model.

The Entity Framework team will presumably start working on a model for the conceptual, mapping and storage layers. They will use MSchema to formulate the model and have the schema for storing the model generated using the m.exe and mx.exe compilers. Using MGraph one can define the EF model data for a particular application using the EF. The schema and model data end up in the Oslo repository powered by Microsoft SQL Server.

To make defining the model data easier and more intuitive the EF team will almost certainly define a new Domain Specific Language (DSL) for the Entity Framework: MEntity. They will use MSchema MGrammar (thanks Dennis for spotting this mistake) to specify the MEntity language. The model for EF will be defined in MSchema. I have no clue just yet as to what it will look like. It can be expected though that MEntity will be much easier than the EDMX format if you want to forsake the EF designer in Visual Studio and go for textual modeling. Moreover, you can use Quadrant to inspect and manipulate the model data in the Oslo repository.

EF having a model might not be much on its own, but look at the larger picture. Your application will probably use more models for other technology stacks (WF, WCF, WPF, ASP.NET and so on) and might have an application-model. To me it seems valuable that EF will have the same model engine that the others use as well. You get to leverage your existing (well, by that time at least) Oslo knowledge and modeling techniques, and work with EF in a more declaratively fashion.

Adapting the EF runtime to Oslo

So, using the MEntity DSL any EF developer gets to define your EDM and its mappings using Oslo model data. The Entity Framework runtime will need to be able to read the Oslo model data from the SQL Server repository and use it to build up the Entity Data Model (EDM) and the mappings to the storage layer. When you first think of this it may seem a bit silly to have EF bootstrap of off a database. Give it some more thought and it is not so weird that the runtime should read data from a repository with a well-known schema before it can read other (entity) data from databases.

As an intermediate step to use Oslo without having to go through too much troubles implementing/changing the current runtime, one could use the native support that Oslo will have for a XAML representation of the model data. The WF and WCF runtimes will most likely do the same thing to avoid having to perform big changes in the beginning. Simply "read" the model data in XAML format from the repository and use that just as it is currently done in the runtimes. That would have been a really nice option for EF as well, ... if they had used the XAML format for CSDL, MSDL and SSDL. (read my other (forthcoming post on this question to the EF team). I guess these are just implementation details that get worked out.

Oslo, EF and other databases

However, this does beg the question of how you can justify the Microsoft SQL Server repository for Oslo in situations where you would want to use EF to target non-Microsoft entity providers and databases. Is Oslo and consequently MEntity only suited for EF in a homogenous situation of SQL Server only?

Well, I'll give it some more thought the next couple of weeks. Any comments, ideas or suggestions you might have are more than welcome.

Lap around Cloud Services

Alex Thissen — Mon, 27 Oct 2008 18:48:20 GMT

During the keynote Microsoft has unveiled Microsoft Azure as their "Operating System for the cloud". At Microsoft the cloud is seen as a set of connected servers/services.

With Azure an organization does not have to worry about running hardware, operating systems on servers, take care of data storage for a full 100%. Instead, some software can be run in the cloud (using Azure as the "OS"), and spend more time on your business logic as opposed to getting everything to run. Of course you can still use or combine this with on-premise (the old-fashioned) applications and services, as in Software + Services.

Azure gives support for utility computing, meaning that it is always there and for a "pay as you use" pricing model. The business model is consumption based and has SLA levels with financial guarantees should anything go wrong.

The key features of Azure highlighted:

Automated service management
The platform takes care of deployment, monitoring and manages your services given the rules and roles you have defined.
The goal is to keep a service responsive and healthy during failures and upgrades. It is even possible to go into "raw" mode and build your own virtual machine and start managing your services yourself.
Powerful service hosting environment
Based on Hyper-V virtualization.
Scalable available cloud storage
You will be given simple storage abstractions. There are blobs (later on also file streams for videos and such stuff) for large data items, service state using tables (and caches) and queues.
There is an emphasis on massive scale, availability, durability. Important to say that it is not a database in the cloud.
Develop as you are used to develop
You get to run a simulated cloud environment on your developer desktop. ASP.NET Furthermore there is a ecosystem of tools and support, ranging from integration with VS. To help you debug a live service (not very wise to really debug such things) you can use logging and alerts (later on tracing as well). Plus we should get the usual documentation and community sources.

The Technical Preview will start this noon with only limited resources for the first period. The features that matter most are important in this preview are: VMs with dedicated resources, a simple service architectures and a single large datacenter on the west coast of the United States. Initially you can use managed code, but native code will be services later on as well.

In the end there will be a commercial release. This means you will have to pay for these services.

Some notes on routing in classic ASP.NET web applications

Alex Thissen — Mon, 20 Oct 2008 21:27:00 GMT

Routing in .NET 3.5 SP1 differs from regular request handling in many ways. Here's a quick outline how routing works:

Routes define some URL matching pattern and are configured/held in an ordered list accessible through RouteTable.Routes
Incoming requests pass through the UrlRoutingModule that will walk the route table and will try to find the first match
If a match was made, the route can extract values from the URL and store these in a RouteData object under the Values properties (there is more to store in a RouteData, but that's beyond the basics)
Assuming a match was made, the route determines the HTTP handler class
The HTTP handler is passed the RouteData and is called to handle the request.

When you use System.Web.Routing in a "classic" web application, you will need a special kind of http handler class to leverage the new RouteData. (Quick tip: you might not need the Request.QueryString in your handler any more, as you can extract that information when determining the route data.). For Web Forms you can find the WebFormRouteHandler implementation in the "Using Routing with WebForms" article by Phil Haack. The route handler by Phil expects the interface IRoutablePage on the web page so it can stick the RequestContext in it (step 4-5).

But, as soon as you have a custom http handler implementation, you are left empty handed again.

Phil's implementation assumes two things:

A System.Web.UI.Page derived class.
A registered http handler endpoint (either through an .ashx file or via web.config's <httpHandlers> section)

There is a quick fix for Phil's code for the first bullet: simply change line 46 to become:

var page = BuildManager.CreateInstanceFromVirtualPath(virtualPath, typeof(IRoutablePage)) as IHttpHandler;

I would also recommend changing some names. The IRoutablePage interface could be IRoutableHttpHandler and the WebFormRouteHandler could be ClassicRouteHandler, or whatever tickles your fancy.

Bullet 2 is hard to bypass. You need to make peace with that assumption. It should be OK for everyday scenarios.

To complement this, I quickly wrote up a generic base class and corresponding interface to allow you to write a custom handler using System.Web.Routing without the need to have this registered anywhere. This will give you tremendous freedom for hooking up a custom HTTP handler. (You will have to come up with an example, because I can't think of one, except you or me being too lazy to do the registration in the web.config. Well, I can, but that involves some crazy on-the-fly generation of handlers that no-one wants. Or do they?)

The interface IRoutableHttpHandler should be implemented by the HTTP handler instead of IHttpHandler.

public interface IRoutableHttpHandler : IHttpHandler
{
  RequestContext RequestContext { get; set; }
}

Very similar to the IRoutablePage of Phil (it is exactly similar if you followed my suggested renaming of his code).

public class MySuperbadCustomHandler : IRoutableHttpHandler
{
  public void ProcessRequest(HttpContext context)
  {
    context.Response.ContentType = "text/plain";
    context.Response.Write("I'm superbad");
  }
 
  public bool IsReusable
  {
    get { return false; }
  }
 
  #region IRoutableHttpHandler Members
 
  public RequestContext RequestContext { get; set; }
 
  #endregion
}

Above is a rather silly custom HTTP handler, but it should be clear where to put your stuff. (No, I'm not telling)

The generic route handler looks like this:

public class GenericHttpRouteHandler<T> : IRouteHandler
  where T : IRoutableHttpHandler, new()
{
  #region IRouteHandler Members
 
  public IHttpHandler GetHttpHandler(RequestContext requestContext)
  {
    T handler = new T();
    handler.RequestContext = requestContext;
    return handler;
  }
 
  #endregion
}

Your route should use this handler and specify the custom HTTP handler as the generic T type. Here's a sample:

routes.Add(new Route("sweet", new GenericHttpRouteHandler<MySuperbadCustomHandler>()));

Small and simple, so there must be a catch. Yep, there is one major one: without a known endpoint it is hard to do the authorization (file or url) based access checks. So, in essence anyone can access the HTTP handler. Not to be used for secured resources. You have been warned.

SDN Conference 2008

Alex Thissen — Sun, 12 Oct 2008 20:28:29 GMT

This year's edition was great, maybe even the nicest ever. Noordwijkerhout and its hotel annex conference venue are a great place to stay. The previous years at Papendal, Arnhem were very nice, but I guess this one tops it.

Anyway, I met loads of people, from famous to acquaintances from the community, the ones I met on projects, courses or coaching gigs. To name just a few:

Julie Lerman: had loads of very long and good conversations on the Entity Framework (what else?) and ways to tell people the vision and overview of EF.
Dennis Doomen: yes, I am still alive.
Taco Ditiecher
Jean-Pierre Broeders
Donald Hessing
Mirjam van Olst

Many, many more, mostly the regulars. Hopefully you/these don't mind that you're not in the list. It's just that I expected to meet you there.

I've uploaded the slides and demos to my sessions:

Introduction to ASP.NET (demos, for slides see the nearly identical DevDays slides)
Technology Deatmatch (slides only, don't expect too much in these)
Introduction to Unity 1.1 (slides and demos)

Everyone, thanks very much for spending the time to come see my sessions and/or talk to me. I appreciate it.

Golden Oldie: Dynamically generated thumbnails article

Alex Thissen — Sat, 30 Aug 2008 09:20:49 GMT

Related to my previous post on dynamic images, a long way back I wrote a two-part article for the now deceased (paid) website ASPToday. It was about a HttpHandler implementation that would generate a thumbnail image from an existing image. The handler was accompanied by a server control that would allow you to specify the original image as the source. The thumbnail is then created on-the-fly during run-time automatically. Since ASPToday is no longer online, I have posted the article "Developing a custom ASP.NET Web control for thumbnails" in one piece under my blog downloads. There's source code to go with it as well. It's an old article as I finished writing it in November 2003. Still a good read I hope (and in English).

Also, when you start generating your thumbnail images in an http handler using the new ASP.NET version for dynamic images (or my simpler implementation for that matter), give Matt's article "Creating great thumbnails in ASP.NET" a read. You'll find it worthwhile.

Trivia fact: ASP.NET dynamic images retro

Alex Thissen — Fri, 29 Aug 2008 20:56:06 GMT

You can find a nice release of the ASP.NET team for dynamically generated images for web applications on the Codeplex website. Others have already blogged about this, such as Scott Hanselman and David Hayden, so I'm not going to repeat that. I do want to add some information to this though.

It's a "Did you know that..." trivia fact. This will get the attention of everybody on the parties that I happen to visit. It might do the same for you. Here goes: did you know that up until .NET Framework version 2.0.40301 (Alpha release of March 2004) there was an entire System.Web.UI.Imaging namespace dedicated to generating images?

In fact, there was even more. The System.Web.Handlers namespace contained a handler DynamicContentHandler and there was a DynamicImage control to go with the handler that had the following options:

Specify the image as an System.Drawing.Image, Byte[] or filename.
Separate scaling for both desktop and mobile browsers.
Store the generated image in cache or on disk.
Retrieve parameters from the querystring or the storage format (from the bullet above)

Look at the content of the now released Microsoft.Web.GeneratedImage.dll assembly and tell me if you find similar things. Mind you, the new implementation is definitely what should/could have been built in ASP.NET 2.0: it's all 2.0-ish now, databinding and generics and stuff. You know, the good things.