NullReference

Två hypade teknologier, SOA och molntjänster (på engelska Cloud Computing) . SOA har många förklarad som dött. Delvis fel enligt mig. Sedan finns det de som fyllt begreppet med så många associationer att man tycker begreppet i sig är dött. SOA har inget buzz längre, men innehåller viktiga principer för att kunna bygga hållbara distribuerade system. Molntjänster, å andra sidan, är på toppen av sin hype, och där väntar vi fortfarande på en accepterande majoritet.

För att lyckas i sin molnsatsning är SOA en nödvändig pusselbit. Jag ska förklara min syn på detta.

Vadå SOA?

SOA handlar om att skapa tydligare koppling mellan verksamhet och IT. Det handlar mer om informationsarkitektur än att skapa web services eller slänga in en broker (t ex BizTalk) och skapa nya orkestreringar ovanpå sin arkitektur med tjänsteändpunkter. Det handlar om att skapa autonoma och löst kopplade informationsdomäner (eller allmänt kallat services).

Udi Dahan har som alltid en förnuftig syn på det här, exempelvis. Han blandar in event-driven arkitektur med artefakter som commands och events (distribution med pub/sub och service bus). För att reda ut begreppsförvirring mellan SOA och DDD har han dessutom myntat begreppet ABC (Autonomous Business Component = service (SOA) = bounded context (DDD)). Inom en ABC kan vi i vissa passande fall välja att använda CQRS.

En tidig ikon inom SOA är Pat Helland, metaforernas mästare. Mest känd för Metropolis. Helland har skrivit en annan artikel som diskuterar distribuerade system och SOA. Utgångspunkten är löst kopplade (meddelande-baserade) system med minskade krav på konsistens (eller om jag för göra en egen tolkning, det vi kallar eventual consistency) till skillnad från att lägga en distribuerad transaktion och få synkronicitet och konsistens.

Vad har det här med molnet att göra?

Först ett konstaterande. De flesta företag kommer inte att gå all-in vad gäller molnet. Nej. De kommer inte flytta alla sina system till molnet. Möjligtvis med undantag av start-up-företag, som inte har något arv med legacysystem osv. Om ett företag inte har en bra SOA, hur kan man då börja partitionera ut delar till molnet? Det går att bryta isär system med ”forcerad integration”. Men det optimala är att kunna välja vad som kan flyttas ut i molnet utifrån verksamhetens behov. Fundera på det en stund. Det borde inte vara IT som hindrar verksamheten när nya behov uppstår. Och tvärtom. Om vi har alignment mellan verksamhet och IT borde inga hinder uppstå. I den bästa av världar i alla fall.

En annan aspekt på detta är molnets karaktär. Molnet är instabilt. Molnet kostar pengar utifrån förbrukning. För att vårt system ska klara av denna miljö behöver vi ett system som klarar att maskiner går ner oväntat (lösa kopplingar, asynkrona meddelanden, durabilitet). Vi behöver kunna skala upp och ut och ner och in vartefter behoven varierar vad gäller resurser. Olika delar belastas olika, vi vill kunna styra detta utifrån verksamhetens behov, dvs styra det på ABC-nivå.

Ytterligare en aspekt är att ett företag har möjlighet att köpa vissa funktioner som en software-as-a-service. Kanske för att nytt behov uppstår. Kanske för att minska kostnad. Kanske för att klara av mer last.

Hur menar du egentligen?

Jag tror ett konkret exempel kan belysa hur jag menar. Kraftigt förenklat, men det är nödvändigt för att inte trassla in sig i det oväsentliga.

Tänk dig ett ordersystem som håller information om ett företags ordrar. Systemet har utvecklats evolutionärt så att det även innehåller information om företagets kunder. Då det uppkom krav att meddela kunder via e-post byggdes även en notifikationsfunktion in i ordersystemet. Figur 1 visar systemet.

Figur 1. Ordersystem med mycket ansvar (inte SOA)

Tänk dig istället följande arkitektur av ordersystemet. Företaget har en duktig arkitekt som byggt upp en löst kopplad eventdriven asynkron arkitektur. Kundhanteringen sköts i en informationsdomän och kommunicerar med omgivande system enbart med hjälp av events. Pub/sub av events kan ske via en service bus (troligen flera instanser, kanske NServiceBus). Se figur 2.

Figur 2. Ordersystem med uppdelat ansvar

Tillbaka till företagets situation. Då företaget växer märker man att olika avdelningar har olika behov. Marknadsavdelningen vill ha tillgång till kundinformation och planera marknadsåtgärder och bearbeta prospekts. Ekonomiavdelningen är mer intresserad av färdiga ordrar och det belopp som ska faktureras.  Försäljningsavdelning (säljarna) vill kunna lägga in prospekts och när de träffar kunder på fältet vill de direkt kunna fylla i den här informationen.

I detta scenario tänker vi oss att företaget skulle vilja prioritera kundhanteringen och börja använda ett CRM-system i molnet. På så sätt kan säljarna komma åt CRM-systemet var de än är och marknadsavdelningen får stöd i sin process att skapa efterbearbetning av kunderna och göra riktade marknadsåtgärder mot vissa segment.  Företaget behöver dessutom inte köpa in ny hårdvara för att drifta CRM-systemet och skapa infrastruktur för att skapa åtkomst för säljarna på fältet.

Vilken arkitektur (figur 1 eller 2) passar bäst om företaget vill genomföra detta?

Slutlösningen med en hybrid av on-premise och molnet kan se ut enligt bilden nedan.

Figur 3. Ordersystem med användning av molntjänst

Vad har vi uppnått?

Det finns många fördelar med vår SOA-style hybrid arkitektur.

• Flexibel arkitektur. Vi kan när som helst välja att flytta tillbaka CRM-hanteringen on-premise. Vi kan dessutom välja att flytta ut andra delar, t ex Notifikationsfunktionen, till molnet.
• Säkerhet. Vi slipper öppna upp åtkomst till det interna kundsystemet mot internet. Vi kan använda t ex Windows Azure Appfabric Service Bus för att skapa koppling mellan molnet och on-premise.
• Skalbarhet. Molnet har ”oändlig” skalbarhet. Den asynkrona hanteringen tillåter en elastisitet och att vi kan ha kontroll på var flaskhalsar uppstår i systemet och då vidta lämpliga åtgärder (skala upp och ut).
• Tillgänglighet.  Det asynkrona lösningsmönstret tillåter att maskiner startas om när som helst utan att andra system påverkas. Molnet är oförutsägbart på det sättet att vi inte har kontroll över infrastrukturen, uppgraderingar på plattformsnivå gör att instanser placeras om i datacentret, nätverksteknik (switchar, mm) går sönder, osv.

Slutsatsen

Molnet är en möjlighet; t ex för att skapa skalbarhet, åtkomstbarhet, kostnadsminskning, flexibilitet.

Företag har olika förutsättningar att lyckas med molnet. En viktig förutsättning är att de har en arkitektur som baseras på lösa kopplingar; helst event-drivet, helst asynkront, helst i alignment med verksamheten. En bra SOA helt enkelt.

Att bygga system enligt ovan principer är viktigt om man har molnet i åtanke. Det finns ramverk och lösningsmönster som hjälper oss, några redan nämna. Ett spännande ramverk är Lokad-cqrs, som bygger på dessa principer och dessutom gör det möjligt att sömlöst flytta sitt system mellan molnet och on-premise.

//Peter

In the previous post I showed how to set up the SQL Server Service Broker and in this post I will use a simple NServiceBus sample to show how to get the messages from the Service Broker, using ServiceBroker.net.

You can find the NServiceBus transport for ServiceBroker.net in the official NServiceBus-contrib at github.

If you want to look at the code while reading you can get it at my github repository for this sample application.

Configuring NServiceBus and ServiceBroker.net

Begin by creating a new NServiceBus sample application. Look in the official samples included with NServiceBus if you need help with this. Beyond the references to NServiceBus you need to add references to the following dll’s:

• ServiceBroker.Net.dll
• NServiceBus.Unicast.Transport.ServiceBroker.dll

The first thing we want to do is to create our own Transport Factory. We do this to create a “bridge” between the SQL Server Service Broker and our MSMQ transport which we will use later to publish other events. You don’t have to use MSMQ at all, and only stick to the SSSB transport, but in our scenario we will use MSMQ as our main transport.

Create a new interfaced (I called mine IChinookTransportFactory) which has one method, GetTransport.

public interface IChinookTransportFactory
{
    ITransport GetTransport();
}

It’s the GetTransport method that will return the ServiceBrokerTransport for NServiceBus to use. Create a class implementing your newly created interface:

public class ChinookTransportFactory : IChinookTransportFactory
{
    private readonly string _connectionString;
    private readonly IMessageSerializer _messageSerializer;

    public ChinookTransportFactory(string connectionString, IMessageSerializer messageSerializer)
    {
        _connectionString = connectionString;
        _messageSerializer = messageSerializer;
    }

    public ITransport GetTransport()
    {
        return new ServiceBrokerTransport
                    {
                        InputQueue = "ChinookEventServiceQueue",
                        ErrorService = "ErrorService",
                        ReturnService = "ChinookEventService",
                        NumberOfWorkerThreads = 1,
                        MessageSerializer = _messageSerializer,
                        MaxRetries = 2,
                        ConnectionString = _connectionString
                    };
    }
}

In the GetTransport method you specify what Service Broker Services to use (the ones we created in the previous post) along with some other settings.

Notice that the Transport Factory takes a connection string as a parameter. We will have to tell our NServiceBus service which connection string to use. We do this in our StructureMap bootstrapper:

public class Bootstrapper : IWantCustomInitialization
{
    public void Init()
    {
        ObjectFactory.Configure(x =>
                                    {
                                        x.For<IChinookTransportFactory>().Use<ChinookTransportFactory>()
                                            .Ctor<string>()
                                            .EqualToAppSetting("connectionString");
                                    });
    }
}

The connection string itself is set in our app.config:

<appSettings>
  <add key="connectionString" value="Server=localhost\SQLEXPRESS;Database=Chinook;Integrated Security=SSPI;" />
</appSettings>

We now have most of the “back bone” done and the next thing to do is to create our event message in our application and then create something that actually picks up the messages from the Service Broker and hands them over to NServiceBus.

When creating our event message we need to make sure we match the namespace we defined in our store procedure in the last post. We used the following:

<Messages xmlns="http://tempuri.net/ServiceBrokerNetSample.Events">

As my sample application is called ServiceBrokerNetSample, what we’ll do is creating a new folder called “Events” in our application root. In that folder, create a new class called CustomerEmailChangedEvent that implements IMessage:

namespace ServiceBrokerNetSample.Events
{
    public class CustomerEmailChangedEvent : IMessage
    {
        public int CustomerId { get; set; }
        public string PreviousEmailAddress { get; set; }
        public string NewEmailAddress { get; set; }
    }
}

First, make sure your event is called the same thing as what we named the MessageName variable in our trigger we created in the previous post. The properties must also be named the same thing as we named the xml elements in the trigger:

SET @CustomerIdXml = (select @CustomerId for xml path('CustomerId'))
SET @PreviousEmailAddressXml = (select @OldEmail for xml path ('PreviousEmailAddress'))
SET @NewEmailAddressXml = (select @NewEmail for xml path ('NewEmailAddress'))

This will make sure that the NServiceBus XmlSerializer correctly serialize the xml to our CustomerEmailChangedEvent.

Now that we have our message event let’s create a class called ChinookNotificationForwarder that implements IWantToRunAtStartup:

public class ChinookNotificationForwarder : IWantToRunAtStartup
{
    private readonly IBus _bus;
    private readonly ITransport _transport;

    public ChinookNotificationForwarder(IBus bus, IChinookTransportFactory transportFactory)
    {
        _bus = bus;
        _transport = transportFactory.GetTransport();

        _transport.TransportMessageReceived += TransportMessageReceived;
    }

    void TransportMessageReceived(object sender, TransportMessageReceivedEventArgs e)
    {
        var message = e.Message.Body[0];
        _bus.SendLocal(message);
    }

    public void Run()
    {
        _transport.Start();
    }

    public void Stop()
    {
        _transport.Dispose();
    }
}

This class has a dependency on the bus and on our Transport Factory we created earlier. In the constructor we use this Transport Factory to create our ITransport which in this case will be the ServiceBrokerTransport. Then we wire up an event handler for when a transport message is received on that transport and in that event handler we get our event message from the event args.

How do we know that our message is at position 0 in e.Message.Body? Well, in the previous post, where we created the store procedure, we defined our Transport Message as:

<TransportMessage><Body><![CDATA[<Messages xmlns="http://tempuri.net/ServiceBrokerNetSample.Events"><'+@MessageName+'>...

And then in our trigger, we passed in the MessageName to the store procedure. This gives us the following in our event handler:

Note! If you named your event message in your application to something else than what it’s named in the trigger you most probably will get the following exception:

Could not extract message data. System.TypeLoadException: Could not handle type…

Your message will then end up in the ErrorServiceQueue in the SQL Server Service Broker.

The last thing we do in the event handler is telling the bus to do a SendLocal on our message. From here it’s just regular NServiceBus stuff going on, but for the record (and for the sake of fulfilling our scenario mentioned in the previous post), I’ll show the event handler that handles the CustomerEmailChangedEvent:

namespace ServiceBrokerNetSample.EventHandlers
{
    public class CustomerEmailChangedHandler : IHandleMessages<CustomerEmailChangedEvent>
    {
        private readonly IBus _bus;

        public CustomerEmailChangedHandler(IBus bus)
        {
            _bus = bus;
        }

        public void Handle(CustomerEmailChangedEvent message)
        {
            //Do what you want to do... We will just publish an event for anyone interested.

            _bus.Publish<ChinookCustomerEmailChangedEvent>(x =>
            {
                x.CustomerId = message.CustomerId;
                x.NewEmailAddress = message.NewEmailAddress;
            });
        }
    }
}

All this event handler does is publish an event for the rest of the world to use in whatever way they want.

That’s it! We now have a working sample application that uses ServiceBroker.net to fetch messages from the SQL Server Service Broker and hands it over to NServiceBus.

To try it out, run your application and set a debug point in your notification forwarder and in your handler, and then do an update on an email address in the Customer table.

There are quite a lot of “magic strings” floating around in our Store Procedure and Triggers that have to match our class names and properties, but once you’ve set this up it’s pretty easy to add more functionality.

I’ve added the sample application to a github repository. It also includes all the SQL scripts needed to set up the SQL Server Service Broker.

If you have any questions, feel free to post a comment here or contact me.

Twitter: @jens_pettersson

//J

This will be (at least) a two part series on how to use NServiceBus and ServiceBroker.net, where I walk you through how we (Frontwalker) used those techniques in a recent project. The first part will be how to set up the SQL Server Service Broker and the next part I’ll use the ServiceBroker.net and NServiceBus.

ServiceBroker.net was created by Joseph Daigle and it’s a simple wrapper API for the SQL Service Broker. It also has a transport for NServiceBus that uses the SSSB. This transport is now part of the official NServiceBus-contrib over at github.

Our scenario was:

The client had a rather old application and they wanted that some of the changes made to that applications underlying data would be processed and sent to another application. In other words, pretty simple, one-way, integration. The old application didn’t publish any events when the client did something, it just inserted/updated/deleted rows in a SQL Server 2005 database. Just for the record, the application itself was a-not-so-task-based MS Access application…

To solve this we had to listen to changes in the columns we were interested in and then notify our integration service to take action. We are using NServiceBus in our projects and decided to use the ServiceBroker.net to solve this scenario. In this series I will be using the Chinook database and a simple NServiceBus sample to illustrate how we did this.

Setting up the Service Broker in SQL Server

First, make sure that the Service Broker is enabled on your database. Right click your database in Microsoft SQL Server Management Studio and select Properties –> Options. Set “Broker Enabled” to True.

The next thing we need to do is to define a message type and a message contract that the Service Broker will use. We do that by running the following SQL:

CREATE MESSAGE TYPE NServiceBusTransportMessage
    VALIDATION = NONE ;
GO

CREATE CONTRACT NServiceBusTransportMessageContract
    ( NServiceBusTransportMessage SENT BY ANY);
GO

After that, we create our service queue, our service and also an error service queue and an error service.

CREATE QUEUE [dbo].[ChinookEventServiceQueue];
GO

CREATE SERVICE ChinookEventService
    ON QUEUE [dbo].[ChinookEventServiceQueue]
    (NServiceBusTransportMessageContract);
GO

-- Error service
CREATE QUEUE [dbo].[ErrorServiceQueue];
GO

CREATE SERVICE ErrorService
    ON QUEUE [dbo].[ErrorServiceQueue]
    (NServiceBusTransportMessageContract);
GO

Run these scripts and refresh your database and you should see the following:

It’s in the ChinookEventServiceQueue all your messages from your SQL server will end up before being picked up by ServiceBroker.net and NServiceBus.

Now we need something in our database to actually send messages to our newly created service queue. For this, we will use a Store Procedure. You can copy and paste this from below but make sure the @MessageContract and the @MessageType variables have the same name as the message type and message contract you created earlier.

Another very important part is that the @TransportMessage variable get the correct xmlns set (the <Messages xmlns=”http://tempuri.net/Namespace.of_your.Events”>). The xmlns need to be set to tempuri.net/ and then the correct namespace of where your events resides. (If you don’t know what your namespace will be, just alter this store procedure later).

Note! The http://tempuri.net/Namespace.of_your.Events is the default namespace of NServiceBus XmlSerializer. You can change this when configuring NServiceBus.

CREATE PROCEDURE [dbo].[SendNServiceBusMessage]
 @TargetService NVARCHAR(200),
 @MessageName NVARCHAR(200),
 @MessageContent NVARCHAR(4000)
AS

BEGIN
-- Sending a Service Broker Message
DECLARE @InitDlgHandle UNIQUEIDENTIFIER;
DECLARE @MessageContract NVARCHAR(200);
DECLARE @MessageType NVARCHAR(200);
DECLARE @TransportMessage NVARCHAR(4000);

SET NOCOUNT ON

SET  @MessageContract = 'NServiceBusTransportMessageContract';
SET  @MessageType = 'NServiceBusTransportMessage';

BEGIN TRANSACTION;

 BEGIN DIALOG @InitDlgHandle
   FROM SERVICE @TargetService
   TO SERVICE @TargetService
   ON CONTRACT @MessageContract
   WITH ENCRYPTION = OFF;

SET @TransportMessage ='<TransportMessage><Body><![CDATA[<Messages xmlns="http://tempuri.net/ServiceBrokerNetSample.Events"><'+@MessageName+'>' +
    @MessageContent +'</'+@MessageName+'></Messages>]]></Body></TransportMessage>';

 SEND ON CONVERSATION @InitDlgHandle
   MESSAGE TYPE @MessageType
    (@TransportMessage);

COMMIT TRANSACTION;

END

GO

Now we need something to actually use this Store Procedure to create messages for our service queue. This is the scary part, we will need to use a trigger on the table we’re interested in!

Now you have a choice. You can either create one trigger per change you’re interested in or create one “global” trigger that listens to all changes to a given table and then use an event handler in NServiceBus to later decide what was changed. We used the latter in our project, but in this sample I’m just going to listen for changes in one column.

Let’s say we have the following scenario for this example:

When a customer changes his email address, we want to notify some other service…

The Chinook database have a table named “Customer” and it contains a column named “Email”. Let’s create the trigger that listens for changes to that column.

Here’s the SQL for creating the trigger:

CREATE TRIGGER [dbo].[TRG_EmailChanged]
   ON  [dbo].[Customer]
   FOR UPDATE
AS
IF UPDATE(Email)
BEGIN
	SET NOCOUNT ON;

	DECLARE @MESSAGENAME NVARCHAR(255)

	-- This is what event your application will use later
	SET @MESSAGENAME = 'CustomerEmailChangedEvent'

	DECLARE @CustomerId int
	DECLARE @OldEmail NVARCHAR(60)
	DECLARE @NewEmail NVARCHAR(60)

	SELECT @OldEmail = Email FROM deleted
	SELECT @NewEmail = Email, @CustomerId = CustomerId FROM inserted

	IF @OldEmail <> @NewEmail
	BEGIN

		DECLARE @CustomerIdXml XML
		DECLARE @PreviousEmailAddressXml XML
		DECLARE @NewEmailAddressXml XML
		DECLARE @XmlText NVARCHAR(MAX)

		-- I'm using select for xml because otherwise special characters might cause problems
		SET @CustomerIdXml = (select @CustomerId for xml path('CustomerId'))
		SET @PreviousEmailAddressXml = (select @OldEmail for xml path ('PreviousEmailAddress'))
		SET @NewEmailAddressXml = (select @NewEmail for xml path ('NewEmailAddress'))

		-- This is the text that we will use in our message
		SET @XmlText = cast(@CustomerIdXml as NVARCHAR(MAX))
			+ cast(@PreviousEmailAddressXml as NVARCHAR(MAX))
			+ cast(@NewEmailAddressXml as NVARCHAR(MAX))

		exec SendNServiceBusMessage
			@TargetService = N'ChinookEventService',
			@MessageName = @MESSAGENAME,
			@MessageContent = @XmlText;
	END
END
GO

Updated the trigger and added CustomerId too. Kind of useless to send an event like this without saying wich customer it affects…

I’m by no means a SQL expert, so I’ve kept the trigger as simple as possible. Basically, what’s going on is that I check if the old value differs from the new value and if so I create a message body in XML and pass that and the name of the message to the store procedure we created earlier.

Now we have a trigger that, if a customers email address is changed, creates a message and hands that to the Service Broker. You can try it out by running a simple update on a customer:

update Customer set Email = 'jens.pettersson@local.loc' where CustomerId = 60

Now, under your Service Broker > Queues, right click the ChinookEventServiceQueue and select “Select top 1000 rows” and you should see one row. Look in the last column, called “casted_message_body”. There you should see the XML your trigger passed on to your store procedure and more that the sp added for this message to work with NServiceBus.

That’s it for the first part! It’s quite a lot of scary SQL stuff going on, but when you’ve set this up you really only need to worry about new triggers later on.

In the next part, I will build a simple NServiceBus sample that actually uses the ServiceBroker.net. This post didn’t really say anything about that or NServiceBus, but this stuff is needed to get something up and running.

//J

Here at Frontwalker we do alot of NServiceBus development for our customers. We think all successful open source projects needs an active community, NServiceBus got that.

But one area that needs improvement is an active contrib where the commity can gather useful add-ons and extensions.

Therefore we have decided to devote some our time to manage the contrib project for NServiceBus. For this to be successful the community needs to be behind this and contribute with both code and documentation. We’re just trying to facilitate that. So, please fork away!

The official NServiceBus-Contrib project is located over at github. We’re planning to move all the content from the old contrib here.

The discussion group is found at http://groups.google.com/group/nservicebus-contrib

We’re using teamcity.codebetter.com as build server. From here you can download the latest binaries.

More details will follow in upcoming posts.

Keep those pull requests coming!

In this blog entry, I’m gonna discuss how how big part of the rest of the system/application you need to integrate in your test, before checking in your code.

We have all heard how much more expensive it gets when a fault is found at a later stage of the test chain (or even in a production system). It is a fact that software companies of any notable size does use several test stages. What these stages are called and how many they are, varies from company to company. The first test stage however is the test the developer is supposed to perform before checking in the code. This might be called “unit test” or “basic test” for example. Then there is typically an integration test, where it is tested that several modules delivered independently are able to talk to each other. At the next test stage, maybe called “function test”, it could be tested that the system’s functionality works. Another test stage could test non-functional requirements, like stability and performance.

The further down the test chain, the closer to the real production environment we get. Real database management systems are used (I call these “databases” from now on), expensive hardware, and so on.

And again, if a bug is found late in the test chain, it is usually more expensive to fix, since you might want to pass all the test stages again. This could make you feel guilty as a developer, when a bug is found, and maybe lead to pretty strange behaviour to avoid finding the bugs “late”.

There are many examples of companies where the developers are supposed to test in production like environments, with real databases, expensive hardware and so on before checking in the code. It is easy to understand that this kind of testing takes a lot of time, and is typically performed manually.

All these test stages and the company culture revolving around the test stages can be very confusing. What is a developer actually supposed to do before checking in the code?

The developer needs to test the changed code towards it’s interfaces until the developer is confident the change works.

This statement holds a lot in it, so let me elaborate.

First we consider that the code must be tested towards its interfaces (for example other classes, protocols, etc). Here, it helps a lot if the interfaces are clearly understood. If you know (by documentation or experience) what the interfaces used by the code return, you can much faster get confident that the change works, by somehow getting the interface to return the values you need for your test. Now, you should preferrably mock the interface in an automated test case, which will let you get confident of your code change very quickly.

Understand that as long as you get the interface to return the values you like, it does not matter where the values originate from. A String is a String, regardless if you mock it into your stubs or if you dig it up from a database in some layer far below yours.

That is, it does not add value to the test if you dig up the string from the database, it just takes more time and money.

A recurring statement from people prefering to test (manually) using real databases is that “they want to be sure that it works”. This is in a sense a good ambition. They want to do a good job for their employer, and not get blamed later. However, time and money are critical resources for the employer, and time and money must be spent right. If you can test quickly and cheaply, getting you confident that the change works, you should do it. At odd times, a bug you could have found using (manual) database-based test might slip to later test stages, but this means 1) you did not understand the interfaces and/or 2) you were sloppy while testing.

Testing faster and cheaper frees up time and money for the employer, making the developers more productive.

Sometimes however, you might not be able to get confident of the test using just stubs. This is when the interface is not well understood, since it is not well documented or for some reason can not be trusted. Also, if you are new to the code base, it is harder (and should be harder) to get confident. In this case it might be fine to test using real databases (if not anything else, it will teach you more about how the real system functions), but it could be smarter try understanding the interface instead and then test the code change using stubs.

To sum it up, it is a waste of time and resources to integrate a lot of “real” software (like databases) into your test, and you should not do it routinely.

(The topic of manual versus automated testing is maybe for some other time )

When you run someone else’s software, there are sometimes error messages which are hard to understand. Most likely, you will always understand error messages from your own software, not necessarily since the error message is good, but since you wrote the code for the software, you will know much about the conditions when it is reported.

It is very annoying to stumble upon a bad error message, for two reasons: 1) the error message does not help you analyse the problem, 2) you realise that phrasing a better error message would probably been very easy for the developer, but she ignored it.

The hard and fast rule for error message is: the message needs to be specific, and using the language of the reader.

Examples of implications:

You need to tell which file you could not open. Stating something like “failed to open file” or including a stack trace with FileNotFoundException is bad.

Stack traces are in general bad, confusing and, all too often, lengthy. If an exception is thrown all the way up to the highest architectural level and you at this point don”t have more information than the instance of the exception and the strack trace, something is wrong, for example a bug in your or in a library’s code. At this point, you have to log what you know, and then finding a fix for your bug or your unhandled configuration problem.

When integrating with legacy databases you’re often required detect changes by polling. This method have some quirks that can be problematic at times. The first issue is that you need to figure out when rows has been inserted or updated. If your legacy db has timestamp columns for inserts and updates this is quite straight forward, just remember when the time you lasts checked for changes and get all changes that has occurred since then. If no such columns exists you’re in trouble and the pain of writing manual code for keeping track of all that awaits you. The other problem is that it’s hard to figure out which fields that was updated. This might not be a big issue but if you need to make decisions based on that info you need maintain a copy of the data in order to figure out what has changed. This is almost always the case if you want to move a way from CRUD style events like CustomerUpdated and the likes to a more business aligned model with events like CustomerActived, CustomerMoved etc etc.

We faced all the problems above and decided to tackle the problem from another angle. As a nice coincidence I had noticed the Joseph Daigle had created a ITransport implementation for NServiceBus that work with SQL Server Service Broker (SSSB). I had to patch the transport, to support xml serialization in order to be able to send messages direct from within SQL Server. With this in place we could add triggers to the relevant tables to add messages to the SSSB queue.

We added the following stored procedure to help with creating xml in the correct format for the NServiceBus xml parser and finally submitting the message to the queue.

For instructions on how to setup SSSB and create the necessary Services and queues please read more here.

Receiving messages from SSSB

The next step was to enable our regular NServiceBus endpoint to receive messages from SSSB. Our main transport is Msmq so we have to start up the SSSB Transport and forwards all incoming messages to our regular transport using SendLocal. This is easily done by implementing IWantToRunAtStartup causing the NServicebus Host to kick off a separate thread to handle this. Next we subscribe to the TransportMessageReceived event and send the incoming messages to our Msmq transport for regular message processing. This combined with the fact that SSSB supports distributed transactions gives us a nice and robust bridge between SSSB and Msmq.

The code is displayed below.

The image below outlines the moving parts of this solution

In closing

This is an example of using the SSSB transport to do event based interop with SQL Server. You can of course use the transport for pure SSSB backed solutions where you use SSSB queues as your main transport for messages between your regular NServiceBus endpoint. This removes the dependency on MSMQ but will make your SQL Server a single point of failure and a potential bottleneck.

Pick your poison!

This summer I borrowed the book “Clean Code” from the office (author: Robert C. Martin).

I write this from memory, without cheating and opening the book again.

As the title suggests, it’s about how to write “clean code”, and what this is, is discussed in the beginning of the book. The rule of thumb that I liked, is that clean code looks “pretty much as you expect it”. That is, when you look at someone else’s code and it doesn’t surprise you with ugliness, it is actually clean! I have myself experienced this when I looked (just a little) an the Java SDK source code. I was like “yeah good code, nothing special to it”. But this means that it is clean.

All in all, I really liked this book, much for its great ability to inspire. An experienced developer will have heard about quite a lot of the contents, but probably not all, and together it is a very inspiring  read.

My favorite part is the discussion of 2 of the parts of the SOLID design principles, namely Single Responsibility and Open Closed Principle.

As I feel it, Single Responsibility Principle will take the design of your application very far if you are able to apply it. The rule basically says “A class (or other program entity) shall have just one responsibility, i.e. one reason to change.” The advantages of this are quite obvious: such a class will have a greater chance of being reused since it has a more clean purpose, and it will be easier to maintain, since this class has (by definition) just one reason to change, so you can’t mess anything else up, while you modify it!

The SOLID design principle I think is second best, is “Open/Closed Principle”, meaning the system shall be “open for extension but closed for modification”. If you are able to apply this, you will find that you can add classes and stuff when implementing new functionality, rather than modifying existing code. Again, there is a smaller risk you mess up the existing functionality.

The book contains numerous thoughts and tips and tricks about how to make your code cleaner, for example:

• A function shall not do stuff on different levels of abstraction.
• Various patters you can return to if you find your code being ugly, for example if you think exception handling takes to much focus from the reader.

A great read!

We had a discussion today on how a test can tell you that something is off with your design. If a feature is hard to test you should listen carefully because your tests might be telling you something. The piece of code we where discussing is setting up return values for queries against our persistent store.

Stub(x=>x.Query(Arg<Func<Customer,bool>>.Is.Anything)) .Return(new[] { customer });

As you can see we have to specify the input to our query method using some hairy nested generics. Our test was just trying to return a specific customer when the repository was queried so having go through all the generic mumbo jumbo just felt wrong. This lead us into changing our old Query method

IEnumerable<T> Query<T>(Func<T, bool> criteria);

To use the query object pattern

IQueryResult<T> Query<T>(IQuery<T> query);

With this is place we can encapsulate our queries in separate classes implementing IQuery and there by adhering to the Single Responsibility Principle. As an added bonus our queries can now be reused instead of copy and pasted everywhere.

And the test?

This change in design allowed our test to focus on the right thing. That is:  “given the query GetCustomerByCrmIdOrOrgNr return this active customer”.

Stub<IRepository>(x => x.Query(Arg<GetCustomerByCrmIdOrOrgNr>.Is.Anything))
.Return(new List<Customer>{ activeCustomer }.ToLinqResult());

This did also allow us to refactor the test a bit further:

QueryFor<GetCustomerByCrmIdOrOrgNr>()
.Return(activeCustomer.AsLinqResult());

Ah, that’s more like it!

Final note:

We’re packaging our IRepository implementation into a public project that soon will be published on our github account.

I stumbled on a very cute design pattern the other day, and I’d like to share it.

It’s supposed to be used when a Factory doesn’t provide enough flexibility, since the creator of the object wants to control also how many properties to set on the created object.

One of the reasons I liked this pattern is since it can provide so much readability to the code. I spent a few minutes trying to find the example I read the other week, but I couldn’t find it again. Those examples I found on the web this time did not use the readability enhancements, but they were still of course examples of the Builder pattern.

A Builder is a class able to construct another object, much like a Factory.

The feature that I think provided the readability, is that methods are called “withX” and return “this”, which means that it’s possible to repeatedly invoke “withX” methods.

This is how cute it gets:

Car.Builder carBuilder = new Car.Builder();
Car car = carBuilder.withTires(4).
                                           withColor(Car.Red)).
                                           withCylinders(8).
                                           withHorsePower(1000).
                                           build();

In this example, Car.Builder is the Builder class, able to construct Car objects.  How explicit this code is! It’s easy to see that “8″ means “number of cylinders” and so on.

One idea behind the Builder pattern is also that it is able to provide convenience methods for recurring groups of attributes. Let’s assume a Car with 8 cylinders and horse power 1000, is a “sports car engine”. Then the Builder class can provide a method for just this, and the code would loook like:

Car.Builder carBuilder = new Car.Builder();
Car car = carBuilder.withTires(4).
                                           withColor(Car.Red)).
                                           withSportCarEngine().
                                           build();

The build() method returns the Car that was constructed, and may also contain consistency checks, i.e. that the set of properties set was appropriate.