Archive

Posts Tagged ‘Lambda Expression’

Custom Paging with Linq and ObjectDataSource

May 17th, 2011 1 comment

Implementing custom paging with Linq and ObjectDatasource allows us to build data access layers which deliver good performance on large data while still keeping a modular and easy to maintain business object structure.
Since the data request pattern for custom paging is different from query patterns which deliver a ‘complete’ data set, additional functionality has to be implemented.

The following examples assume an ObjectDataSource in connection with an ASP.NET GridView control. When using custom paging, every DataBind() on the GridView results in two calls:

  1. First, the call to the data retrieval method with selection and sort parameters plus a ‘Skip’ cound and a ‘Limit’ count to restrict the number of records returned.
  2. Second, a call to a method which delivers the total amount of records available (so the control knows how many pages there are. This method also needs to receive the same selection parameters as the first method.

Basically the same filtering has to be performed in both methods, whereas the first method returns an IEnumerable, and the second returns a single integer. With an increasing number of selection parameters, a lot of redundant functionality needs to be added to both methods. So the question is, if the filtering could be performed in a single method which is then accessed by method 1 and 2.

This is easily possible if the result of the query is an existing type. Linq defered execution will enable reusing of the filtering expression for different purposes. However if the expression returns an anonymous type as an IQueryable, further processing of that type becomes more difficult.

Basically the problem is that a non-generic IQueryable is not supported by the Linq extension methods such as Count(), Skip(), Take(), OrderBy(), OrderByDescending(). Also the extension method ToArray() is not supported for a non-generic IEnumerable.

So by finding a way to implement those missing extension methods, we could save ourself implementation work and eliminate redundancies at the same time.

The following extension methods provide the missing functionality that will also work with non-generic IQueryable and IEnumerable and thus supports anonymous types.

/// This helper class will provide several Linq extension methods for sorting and custom paging.
public static class LinqExtensions
{
    /// Method used for sorting according to the pseudo SQL syntax emmitted by ObjectDataSource.
    public static IQueryable OrderBySQLSyntax( this IQueryable source, string sSQLOrder )
    {
        string[] asOrder = sSQLOrder.Split( new char[] { ' ' }, 2 );
        bool descending = ( asOrder.Length == 2 && String.Equals( asOrder[ 1 ], "DESC" ) );
        return OrderByExtension( source, asOrder[ 0 ], descending );
    }
 
    /// Used for sorting in ascending/descending order according to the property provided.
    public static IQueryable OrderByExtension( this IQueryable source, string propertyName, bool descending )
    {
        ParameterExpression x = Expression.Parameter( source.ElementType, "x" );
        LambdaExpression selector = Expression.Lambda( Expression.PropertyOrField( x, propertyName ), x );
        MethodCallExpression mce = Expression.Call( typeof( Queryable ), descending ? "OrderByDescending" : "OrderBy",
            new Type[] { source.ElementType, selector.Body.Type }, source.Expression, selector );
        return source.Provider.CreateQuery( mce );
    }
 
    /// Adds a count expression to the query.
    public static int CountExtension( this IQueryable source )
    {
        MethodCallExpression mce = Expression.Call( typeof( Queryable ), "Count",
            new Type[] { source.ElementType }, source.Expression );
        return source.Provider.Execute<int>( mce );
    }
 
    /// Converts the IEnumerable to an array. The main purpose is to stop the defered execution chain.
    public static IEnumerable ToArrayExtension( this IQueryable source )
    {
        MethodCallExpression mce = Expression.Call( typeof( Enumerable ), "ToArray",
            new Type[] { source.ElementType }, source.Expression );
        return source.Provider.Execute<IEnumerable>( mce );
    }
 
    /// Adds custom paging support to an IQueryable.
    public static IQueryable SkipExtension( this IQueryable source, int iSkip )
    {
        MethodCallExpression mce = Expression.Call( typeof( Queryable ), "Skip",
            new Type[] { source.ElementType }, new Expression[] { source.Expression, Expression.Constant( iSkip ) } );
        return source.Provider.CreateQuery( mce );
    }
 
    /// Adds custom paging support to an IQueryable.
    public static IQueryable TakeExtension( this IQueryable source, int iTake )
    {
        MethodCallExpression mce = Expression.Call( typeof( Queryable ), "Take",
            new Type[] { source.ElementType }, new Expression[] { source.Expression, Expression.Constant( iTake ) } );
        return source.Provider.CreateQuery( mce );
    }
}

If the above appears to be too complicated, there is an alternative approach, which will be based on the fact that ObjectDataSource will call the data retrieval method first, and then call the count method. Therefore we could actually cache the value for the Count() method when the select method is called. This would also have a positive impact on performance. I am not sure though, if this order of calling can be relied on. Caching cannot be done in an instance variable, since ObjectDataSource apparently creates a new instance upon every call. The place where it makes most sense is withing the HttpContext.Current.Items location inside web applications. In case you are using several ObjectDataSources on one page you have to make sure the parameter name for the cached value is unique, so there are no collisions. See my next post for details.

Categories: C#, Software Development Tags: , , , , , , , , , , , , , , ,

Sorting with custom lambda expressions

October 12th, 2010 No comments

When working with dynamic sorting e.g. when using a GridView/ObjectDataSource combination which will sort according to the column header clicked, the most intuitive way of realizing sorting behavoiur in the ObjectDataSource is by using a large if/switch construct.

A smarter way is to use custom lambda expressions with Linq, which will automatically construct the correct Linq query according to the column name and sort order. No matter how many columns, whether sorted ascending or descending, the following code will handle it all:

/// <summary>This Linq helper class constructs a custom lambda expression used for sorting.</summary>
public static class LinqExtensions
{
    /// <summary>Generic method used for sorting according to the pseudo SQL syntax emmitted by ObjectDataSource.</summary>
    public static IQueryable<T> OrderBySQLSyntax<T>( this IQueryable<T> source, string sPropertyParameter )
    {
        string[] asParams = sPropertyParameter.Split( new char[] { ' ' }, 2 );
        bool descending = ( asParams.Length == 2 && String.Equals( asParams[ 1 ], "DESC" ) );
        return (IQueryable<T>)OrderByExtension( source, asParams[ 0 ], descending );
    }
 
    /// <summary>Used for sorting in ascending/descending order according to the property provided.</summary>
    public static IQueryable OrderByExtension( this IQueryable source, string propertyName, bool descending )
    {
        var x = Expression.Parameter( source.ElementType, "x" );
        var selector = Expression.Lambda( Expression.PropertyOrField( x, propertyName ), x );
        MethodCallExpression mce = Expression.Call( typeof( Queryable ), descending ? "OrderByDescending" : "OrderBy",
            new Type[] { source.ElementType, selector.Body.Type }, source.Expression, selector );
        return source.Provider.CreateQuery( mce );
    }
}
Categories: C#, SQL Tags: , , , , , , ,

Using Linq Distinct() without IEqualityComparer

January 18th, 2010 No comments

Given an IEnumerable class, such as a generic list or array, it is only possible to use the Distinct() method when working with simple data types. As soon as we are operating on a list of objects though, we are forced to write your own class implementing IEqualityComparer, which is a bit bothersome in many cases. At tehe first glance, it seems that Microsoft has simply forgotten to implement Lambda Expressions for Distinct() and similar functions. Another reason might be that these functions immensely benefit from the use of a hash based comparison algorithm, and basically that is what the IEqualityComparer is all about. See my other blog post about this subject.

For those who are just looking for a simple solution, the following one-liner might be useful:

SomeObject[] array_1 = new SomeObject[] { ... }
SomeObject[] array_2 = array_1.GroupBy( x => x.SomePropertyOrMethod ).Select( x => x.First() ).ToArray();

A standard implementation using IEqualityComparer could look like this:

byte[][] hash_distinct = hash_duplicate.Distinct( new ByteArrayComparer() ).ToArray();
/* .... */
public class ByteArrayComparer : IEqualityComparer<byte[]>
{
    public bool Equals( byte[] a, byte[] b )
    {
        if ( a == null || b == null )
            return a == b;
        return a.SequenceEqual( b );
    }
    public int GetHashCode( byte[] x )
    {
        if ( x == null )
            throw new ArgumentNullException();
        int iHash = 0;
        for ( int i = 0; i < x.Length; ++i )
            iHash ^= ( x[ i ] << ( ( 0x03 & i ) << 3 ) );
        return iHash;
    }
}

In my tests the performance gain by using an IEqualityComparer implementation instead of the above solution is about 100% when working on an array of 18000 elements.

Categories: C#, Software Development Tags: , , , ,