Thursday, 08 November 2007
Typically, Genome is used to map tables to their data domain objects. But what to do when you have to use a database that is not made for mapping objects and therefore is not in any normalization form etc.?
Thursday, 08 November 2007 16:23:06 (W. Europe Standard Time, UTC+01:00)  #    Disclaimer  |  Comments [0]  | 
 Tuesday, 18 September 2007

While documenting/testing Genome 3.3 I stumbled about this strange behaviour, which seems to be a bug of the C# 3.0 beta 2 compiler.

I was trying to compile the following GROUP BY example with Genome:

var ordersPerCountryPerYear1 = from o in Helper.DB.Extent()
                               group o by new { o.Customer.Address.Country, o.OrderDate.Value.Year } into g
                               select new
                                 Country = g.Key.Country,
                                 Year = g.Key.Year,
                                 OrderCount = g.Count()

And received the following error from the compiler:

error CS1061: 'System.Linq.IGrouping' does not contain a definition for 'Count' and no extension method 'Count' accepting a first argument of type 'System.Linq.IGrouping' could be found (are you missing a using directive or an assembly reference?)

However, my team insisted that the extension method Count() is provided by Genome. To find out, why the compiler does not find it, they asked me to call it directly in Main():


After inserting this call in my code, the program suddenly compiled (including the statement, the C# compiler complained about previously).

We think this is a bug of the compiler. As a workaround I now have the following method on one class in my project to satisfy the compiler :-) :

static void ThisIsNeverCalled()

Posted by Chris

Technorati Tags:

Tuesday, 18 September 2007 16:24:01 (W. Europe Standard Time, UTC+01:00)  #    Disclaimer  |  Comments [5]  | 
 Sunday, 16 September 2007

There are a lot of discussions how to properly mock an O/RM to achieve unit testing.

In this article, I will describe how an application implemented with Genome can be tested by means of unit testing. By stealing (and also modifying) the sample domain from Fowler, I’ll demonstrate this with a simple WebShop application. The use case that we are going to investigate is the ordering process itself.

The sample code used is provided for download at the end of the article. You need Genome 3.3 (beta 1) to execute the sample.

Sunday, 16 September 2007 14:25:25 (W. Europe Standard Time, UTC+01:00)  #    Disclaimer  |  Comments [0]  | 
 Saturday, 15 September 2007
TheServerSide.Net has just published our case study about DataReturn's experiences of using Genome to rebuild website for BMW USA. The site has quite critical performance requiremens of up to 1 million sessions per day.

Posted by Chris

Technorati Tags:

Saturday, 15 September 2007 09:19:10 (W. Europe Standard Time, UTC+01:00)  #    Disclaimer  |  Comments [0]  | 
 Tuesday, 10 July 2007

Recently we received a question from a customer about how to express a sort criteria based on a condition. As the sort expression in Genome is nothing else than an implicit function (lambda) executed on the candidate element to return the value to sort by, this can be quite easily achieved.

As a simple example, let’s use the Northwind database again. A customer there has a Country and a Region property. Imagine you want to order all customers in the USA by Region and all the other customers by Country. Additionally you want to have USA customers first, then the others.

A Genome set can be sorted using the OrderBy() method. It takes an implicit function as a parameter, which is denoted using [] in OQL. The condition can be expressed using the ? operator, which is translated to a CASE WHEN in SQL. Hence, the order function can be expressed like this:

When the customer is located in the USA, then order by region; otherwise, order by country:

Country=="USA" ? Region : Country

To make sure customers located in the USA are listed first, I am prefixing the Region with a space (I know this is a hack, but it will do for this example). Hence, the final OQL looks like this:

extentof(Customer).OrderBy( [ Country == "USA" ? " " + Region : Country])

which translates to the following SQL:

SELECT … FROM Customers
    CASE WHEN (Country="USA")
       THEN (" " + Region)
       ELSE (Country)

Posted by Chris

Technorati Tags: ,

Genome | OQL
Tuesday, 10 July 2007 20:00:41 (W. Europe Standard Time, UTC+01:00)  #    Disclaimer  |  Comments [0]  | 
 Wednesday, 04 July 2007
The Genome OQL Query Analyzer (QA) is a very helpful tool when you are developing with Genome: it allows you to load a mapped business layer to execute any kind of OQL queries. In this article, I demonstrate how the latest mapped business layer, along with its mapping, can be loaded by starting the Genome QA with a single click in Visual Studio’s solution explorer.
Genome | OQL
Wednesday, 04 July 2007 12:47:14 (W. Europe Standard Time, UTC+01:00)  #    Disclaimer  |  Comments [0]  | 
 Friday, 29 June 2007

In my last post, I described how to use the Genome Web Application starter kit to rapidly create a web application that uses Genome.  To prove that Genome is not just for web applications, I will now show how to create a windows forms application with Genome.  There's no wizard that will create forms automatically (yet?), so there are a couple more manual steps.  As you will see, however, it's not too difficult.

I open visual studio, and reuse the datadomain and mapping projects I created in the first post in this series.  Then I add a new Windows Application project, call it WinForms and designate it as Start Project. I reference the datadomain and mapping projects, and the System.Configuration assembly.  I also reference the TechTalk.Genome assembly, which is located in C:\Program Files\TechTalk\Genome 3.2 Professional Edition for .NET 2.0\Bin\TechTalk.Genome.dll on my pc. I rename the Form1 files to MainForm. I add an App.config file, and paste the connection string from the web project that I created in my last post.  I then add a bindingsource to the form, and call it customersBindingSource.  I configure the bindingsource to use the Genome.DataDomain.Customer class from the DataDomain assembly (1). I add a DataGridView control to the form, dock it to fill the form, and have it use the customer bindingsource.  You can choose which columns you want to display in the grid; not all of them are that interesting to watch ;-)

So far, we used standard Windows Forms development techniques, but we still need to fill the datasource of our customer bindingsource.  This is where it gets more interesting.  We need a bit of plumbing code, but fortunately, we can use that most venerable of code reuse techniques: copy and paste.  I Open up the directory where Genome is installed (C:\Program Files\TechTalk\Genome 3.2 Professional Edition for .NET 2.0 on my pc).  I drill down to Samples -> CSharp -> QuickStart -> Step3-> WinFormsClient.  I Copy the files ContextBoundForm.cs, ContextFactory.cs  and Helper.cs to the directory of my WinForms project, and include them in the project.  I open the files and change the namespaces to WinForms. In Helper.cs, I change the name of the _schemaAssembly field to "01ReverseEngineering.Mapping" (which is the name of the mapping assembly compiled by the mappings project from my first blog post in this series.). I change the name of the connection string to "MyRole" (or whatever you defined as the name of the connection string in your app.config file). Now I open MainForm.cs in code view and make the class inherit from ContextBoundForm.

This code won't compile yet. The reason being that there's a using directive TechTalk.Genome.Extensions.Windows.Forms in ContextFactory.cs.  If you are familiar with the theory behind windows databinding, you will know that  the IBindingList interface is essential when binding lists of data to a datagrid or a listbox.  However, Genome's Set<T> (which is what is returned by any Genome query) does not implement that interface.  The Genome team provided a SetBindingList<T>, which is a wrapper around Set<T> that implements IBindingList.  You can find the code for this class in C:\Program Files\TechTalk\Genome 3.2 Professional Edition for .NET 2.0\Extensions\Windows.Forms\SetBindingList.cs. I add this file to the WinForms project (you can add it either as a copy or as a link). I compile to make sure that I have copied all code I need.

What is all this plumbing code about? One of the central concepts in Genome is the Context.  A idea of a Context in Genome is comparable to a connection to a database.  There can be many connections to the same database, but you need at least one connection in order to do something usefull on your database. The same is true for a Context.  The Genome equivalent of a database is called DataDomain, by the way.  ContextBoundForm inherits from System.Windows.Forms.Form and adds a propery and a couple of methods for using a Context.  ContextFactory does exactly what its name suggests.

Now at last we can populate the custerm binding source. I open MainForm.cs in design view, and double-click on the Form's title bar to create a handler for the load event. Thanks to all our plumbing code, I only need to write customersBindingSource.DataSource = new SetBindingList<Customer>(Helper.DB.Extent<Customer>(Context)); - after adding using directives for Genome.DataDomain and TechTalk.Genome.Extensions.Windows.Forms, that is.  I hit F5 to debug and run the application.  If all goes well, the main form loads and you will see the list of customers in the data grid.  In my case, not all went well at first and I had to make sure that my windows user account had appropriate permissions on the Northwind database.  The result looks something like this:

That's it for today.  This post turned out to be way longer than I anticipated, so I will save up the Orders and OrderDetails part and the editing part for the next post.

(1): If you have never worked with bindingsources in a windows forms application (and even if you have), I can highly recommend Brian Noyes's book Data Binding with Windows Forms 2.0 - Programming Smart Client Data Applications with .NET: it's full of information about working with data binding in Windows Forms, and is an enjoyable read.  If you are interested in the inner workings of data binding, be sure to check out chapter 7, Understanding Data-Binding Interfaces.

Posted by Dirk

Technorati Tags: object relational, getting started

Friday, 29 June 2007 09:59:29 (W. Europe Standard Time, UTC+01:00)  #    Disclaimer  |  Comments [0]  | 
 Thursday, 28 June 2007

Mats Helander, whom I have already had the pleasure to meet personally several times, wrote about an O/RM challenge in his blog.

While it is always fun to participate in challenges, I want to criticize the problem Mats describes first, before I show how you can solve it with Genome.

The challenge only concerns how efficiently an O/RM can read up a set of whole tables from the database. This does not make sense for two reasons:

  1. Usually, you don’t want to load up all the data from a database into memory (that’s one of the reasons why you use a database).
  2. If you have special cases, where you cache whole tables from a database (e.g. some lookup data), caching takes place very seldom (e.g. once a day) and thus the efficiency of loading up the data is not of such a big importance.

Mats expresses the challenge in such a way that he demands that the O/RM may not join the related objects when loading from the database to find out about their relationships. Instead, the O/RM should load up all objects at once, and “discover” the relations between the objects afterwards on its own (without using the JOIN from the database). This results in three SELECT statements (SELECT * FROM Customers; SELECT * FROM Orders; SELECT * FROM OrderLines).

An O/RM usually maintains only an identity map to cache object lookup queries. This helps object references mapped through foreign key fields in the database to be followed without extra database roundtrips (given that the related objects are already loaded into memory). This means that following an Order to its related Customer works in memory, if all data is loaded up. To discover the Orders belonging to the Customer, however, the O/RM needs to perform a lookup query.

Some O/RMs, including Genome, allow collections to be preloaded in order to avoid unnecessary roundtrips when traversing object graphs deeply. So you can tell the O/RM to preload all the Orders of the Customers retrieved, and to preload all the OrderLines of the Orders retrieved. In this case, the O/RM builds a map for relating the objects in memory while loading up the data.

Usually you only want to load up the related children of the parent table. It doesn’t make sense to load up all orders from the database only to fulfill the orders of three specific customers. To ensure this, an O/RM typically JOINs the related data to the filtered parent table.

Not filtering the parent table is a very special case. Introducing an optimisation for this case is possible, but would make no sense (for the initial reasoning above). Besides that, I wonder how large the loaded table has to be in order for that additional JOIN to make a significant difference, giving the whole performance optimisation sense at all. I guess in those cases, it is out of the question to cache the results in memory anyway, which is the premise of the scenario. Another drawback of this optimisation I want to point out is that it can turn out to be less efficient very quickly when the parent reference is nullable, as unnecessary data is loaded up again.

Still, this is a challenge and a lot of people interested in O/RM read it; so, let’s solve it with Genome.

Genome provides two infrastructures for retrieving and caching relations: collections and indexing.

The collection infrastructure provides rich support for handling specialised relation types such as 1:n and n:m relations. Usually, I would recommend using Genome’s collection mapping feature to support Mats’ scenario, except that Genome uses a JOIN to limit the related objects loaded up from the database.

Indexing is a Genome infrastructure that automatically detects even complex relationships, based on the loaded data. It is more complex to configure, use and maintain, but it can support Mats’ exotic scenario. Having mapped the business layer with Genome, the following three lines of code will do the trick:

using (Context.Push(LocalContext.Create()))
    IndexManager.FillIndex(Context.Current, dd.Extent<OrderDetail>(), 
                           IndexManager.GetIndex(dd.Schema, typeof(OrderDetail), "IdxOrder"));

    IndexManager.FillIndex(Context.Current, dd.Extent<Order>(),
                           IndexManager.GetIndex(dd.Schema, typeof(Order), "IdxCustomer"));

    Set<Customer> customers = dd.Extent<Customer>().ToArray();



Inside the using block, the first two lines of code load up all OrderDetails and all Orders. Additionally, they saturate the indexes for the relationships Order->OrderDetail and Customer->Order. The third line of code loads up all customers. When Dump(customers) traverses through the object graph, all relationships are served from memory.

Note that this feature is not limited to simple 1:n and n:m relationships. It works for more complex relationships as well, such as retrieving pending orders of a customer etc.

Posted by Chris

Technorati Tags: object relational, challenge

Thursday, 28 June 2007 14:17:52 (W. Europe Standard Time, UTC+01:00)  #    Disclaimer  |  Comments [0]  | 
 Tuesday, 26 June 2007

In Genome, you can use ToObject() to return a single object from a query that has zero or one result element. The Genome documentation gives the following explanation about the restriction on the number of result elements:

The method should not be called for sets that may contain more than one element. Calling the method for these kinds of sets results in different behaviours based on the database platform and the calling context. If you need to retrieve the first element of a set, the combination of Set.GetRange and Set.ToObject has to be used, as in the following example.

But when can you be sure that a query returns only zero or one element? And what happens if you do not follow that advice?

A common case that would require using ToObject() is when you need to map an inverse object reference of a 1:1 relationship in the domain model.

Imagine the following example, where a company car can be assigned to one or no employee: In this case, the company car has an object reference to the employee it can be assigned to, represented by a foreign key in the database. Vice-versa, the employee has an object reference to the assigned company car, implemented through a lookup query that returns the car that is being assigned to this employee:

public abstract class CompanyCar
	public abstract Employee AssignedTo { get; set; }

<Type name="CompanyCar">
	<Member name="AssignedTo"><NearObjectReference/></Member>

public abstract class Employee
	public abstract CompanyCar AssignedCar { get; }

<Type name="Employee">
	<Member name="AssignedCar" Oql="extentof(CompanyCar)[ccar: ccar.AssignedTo==this].ToObject()" />

Note the following details:

  1. Employee.AssignedCar is readonly, while CompanyCar.AssignedTo is read/writeable. This is logical, since Employee.AssignedCar is only mapped to a query, where you cannot “set” the result. Of course, you can implement a more sophisticated property on Employee which would allow the car to be set directly for an employee, but I leave this out for simplicity’s sake.
  2. The lookup query mapped to Employee.AssignedCar retrieves a single car instance by using ToObject(). This assumes that the lookup query returns only zero or one result element, which is the point I wanted to discuss in this article.

Having mapped this, you can freely navigate from CompanyCar to Employee and vice-versa, as shown below.

Navigating from CompanyCar to Employee executes a lookup query for the foreign key against the database:

SELECT ... FROM Employee WHERE Id = {CompanyCar.AssignedTo}

Navigating from Employee to CompanyCar executes a lookup query for the primary key of the Employee instance in the AssignedTo fields of the CompanyCar table:

SELECT TOP 1 ... FROM CompanyCar WHERE AssignedTo = {Employee.Id}

The beauty of this mapping is that the domain model’s user does not need to be aware in which direction the relationship is mapped in the database. You can even build more complex queries using the property . For example, finding all Employees that have a CompanyCar assigned to them is easy in OQL:

extentof(Employee)[AssignedCar != null]

This translates to the following SQL:

SELECT ... FROM Employee
  LEFT OUTER JOIN CompanyCar ON CompanyCar.AssignedToId=Employee.Id

If you change the database schema to point the foreign key in the other direction, the same OQL is translated to the following SQL:

SELECT ... FROM Employee WHERE NOT AssignedCar.Id IS NULL

The important point that I want to make is that ToObject() works fine as long as you can be sure it will return only zero or one result. In my example, if there were more than one car assigned to an employee, then the SQL query would return duplicate employee entries for those employees with more than one car assigned:

SELECT ... FROM Employee 
  LEFT OUTER JOIN CompanyCar ON CompanyCar.AssignedToId=Employee.Id

A wrong approach to fixing this problem is to use a distinct projection, eliminating the duplicate employee entries from the result:

 [this distinct]extentof(Employee)[AssignedCar != null]

This translates to:

  LEFT OUTER JOIN CompanyCar ON CompanyCar.AssignedToId=Employee.Id

There are several reasons why using a distinct projection is not a good solution to the problem in this case. First of all, you do not want to change the semantics of your query in OQL, just to work around this problem. There might be many other places where you run into similar problems which you would have to fix with additional projections or other hacks one by one. Second, DISTINCT limits the query in some cases, e.g. you cannot sort by a field not contained in the selector anymore when using DISTINCT.

The right approach to solving this problem reflects the fact that there can be more than one CompanyCar assigned to an employee in the mapping of the relationship. There are two ways of doing this.

The simplest is to tell Genome to expect more than one result in the lookup, and return only the first. This can make sense if you want to return “any” of the assigned cars. If you combine it with an order criterion that defines which cars to return first, this can be even more meaningful. The following mapping would return the most expensive car assigned to an employee:

<Type name="Employee">
	<Member name="AssignedCar" Oql="extentof(CompanyCar)[ccar: ccar.AssignedTo==this].OrderBy([Price descending]).GetRange(0,1).ToObject()" />

.OrderBy([Price descending]) ensures that the most expensive car is returned first.

GetRange(0,1) tells Genome to make sure only one result is returned. Depending on how AssignedCar is used in another OQL, Genome shapes the resulting query accordingly.

After mapping AssignedCar as above, the original query

extentof(Employee)[AssignedCar != null]

is now translated to the following SQL:

SELECT ... FROM Employee
    SELECT TOP 1 FROM CompanyCar WHERE CompanyCar.AssignedToId = Employee.Id
    ORDER BY CompanyCar.Price DESC

Note that, because GetRange(0,1) is used in the mapping of Employee.AssignedCar, Genome implements the same OQL to search for employees without a car, using a sub-query instead of using LEFT OUTER JOIN.

Depending on your business use case, you might choose to do more complex refactoring of your business model to reflect that more than one car can be assigned to an employee. For example, you could introduce an additional property to Employee that returns Set with all assigned cars. You would still need to define how queries that retrieve only a single instance of a car should work, no matter how many cars are assigned to an employee.


When using ToObject() alone to retrieve a single element from a set, make sure that the set can only contain zero or one element. Otherwise, you will end up with unwanted side effects when building more complex queries based on this query.

When you know a query can return more than one element, but you just want to fetch the top element, use GetRange(0,1) in combination with ToObject(). For example, if you want to retrieve the most expensive car from the car pool, use

extentof(CompanyCar).OrderBy([Price descending]).GetRange(0,1).ToObject()

It is also possible that your data is inconsistent; more than one CompanyCar may be assigned to an Employee, although business rules forbid this. In this case, this is a bug in your software which you should resolve otherwise (e.g. through proper business rule checks or database constraints). Using GetRange(0,1) does not really help, as it does not represent the business intent you want to implement.

It may also be tempting to use GetRange(0,1) “just to be sure”, but note that this adds unnecessary performance overhead. For example, joining in a TOP query is far more complex and slow than joining in the same without GetRange(0,1). In fact, this is the reason why we have not included GetRange(0,1) in ToObject().

Posted by Chris

Technorati Tags: object relational, getting started

Genome | OQL
Tuesday, 26 June 2007 09:46:19 (W. Europe Standard Time, UTC+01:00)  #    Disclaimer  |  Comments [0]  | 
 Friday, 22 June 2007

In my previous installment, I wrote about using the Database Reverse Engineering Wizard to create classes and mappings for an existing database.  In today's post, I talk about using the Genome Web Application Wizard to create a simple, yet fully functional ASP.NET Site to view and edit entries in your database.

I'm basically continuing where I left off last time.  I have a Visual Studio Solution with two projects: one with the DataDomain classes, and one with the mapping files.  Now I add a new project using File -> Add -> New Project ...  I select Visual C# (the Wizard seems to be a C#-only affair), and then Genome Web Application.

I gve the project a name, and click OK.  I click Next on the Wizard's Welcome page, and come to the page where I select the DataDomain Schema (or Mapping) project, and the Business Project.  The Wizard is smart enough to automatically detect the appropriate projects in the solution.

Next once more, I enter the connection string again, click Finish and Finish again.  I now have a third project in my solution, with appropriate references, CSS adapaters, a default page, a master page and a couple of helper classes.

At this point, I made the mistake of thinking that the wizard had malfunctioned because there are no pages for my business objects.  There's only a default.aspx page, and that one is empty.  One of my colleagues kindly explained to me that I need to add those pages manually.  That way, I can specify exactly what such a page should contain.

So I right-click the web application project, point to Add and select New Item.  There are two Genome-related items to choose from, the Genome Details Page and the Genome List Page. I select Genome List Page and call it Orders.aspx.

I click on Add, and another part of the wizard appears.  I click Next to get past the welcome screen, I choose my business object class (Order) and click next again. A screen with several settings appears.  I leave the default values, except that I enable in-place editing.  I click Finish twice and the wizards adds Orders.aspx to the web application.

I set Orders.aspx as the project's start page and hit F5.  A browser window opens and I see a page with filter options and the results of the search.  Go ahead and play around with the filter options.  The Edit and Delete links are fully functional as well.  Play around with these as well.  Show Details leads to an HTML 404 error however, since we haven't yet defined a details page for the Order business entity.  I close the browser and return to Visual Studio.

I add another item to the web application, but this time I select a Genome Details Page.  I click through the wizard, select Order for my business class and basically accept all the default values.  The wizard adds an OrderDetails.aspx page to the project. I hit F5 again, and the Orders.aspx page opens again.  I click on Show Detail for one of the items, and this time OderDetails.aspx opens and shows the details of the Order object.

You can repeat this process for all business entities that you want to view and edit in the web application.  Using the Genome Web Application wizard, you can quickly generate a small but fully functional web site for editing your business data.

In the next installment, I will show how to create a Windows Forms application that uses Genome.

Posted by Dirk

Technorati Tags: object relational, getting started

Friday, 22 June 2007 09:50:56 (W. Europe Standard Time, UTC+01:00)  #    Disclaimer  |  Comments [0]  |