LINQ Queries - Deferred Queries

The heart of the technology introduced with LINQ is called query expressions. I wrote the article LINQ Queries that introduced these queries using an array of words from a famous quote from American history, Thomas Paine's The Crisis):

 "These are the times that try men's souls!" 

The final example in that article shows how the words can be easily sorted. (code below) And the result is almost an even more profound question:

are men's souls! that the These times try

If not for that extra 'the' in there to make it ungrammatical, this would be an interesting question itself. This article will illustrate the concept of "deferred queries" that is an inherent part of most LINQ queries by showing a 'before and after' example with 'the' removed in the 'after' example.

Here's the complete 'before' code for the query. To code this yourself, drop this into the Click event for a button.

 ' This just sets up the array
 Dim Words As String = "These are the times that try men's souls!"
 Dim WordArray() As String = Split(Words)
 ' This defines the sequence of words returned by the query
 Dim Sequence As IEnumerable(Of String) = _
    WordArray.OrderBy(Function(n) n)
 ' This enumerates the query
 For Each Name As String In Sequence
    Console.Write(Name & " ")
 Next 

One might think that the Sequence collection is created in the statement:

 Dim Sequence As IEnumerable(Of String) = _
    WordArray.OrderBy(Function(n) n) 

It's not, however. Technically, only the query is stored in Sequence. The query is actually performed - that is, the sort against the original collection, WordArray - when it is enumerated in the For-Each loop. Misunderstanding this critical bit of processing can be the source of devilish bugs. To demonstrate this, let's get rid of that troublesome 'the' in WordArray and simply execute the same loop again. (Add this code in the same Click event after the code above. I added a question mark at the end just to make the result look better.)

 ' Get rid of 'the' in the array
 WordArray(2) = ""
 ' Enumerate the array again
 For Each Name As String In Sequence
    Console.Write(Name & " ")
 Next
 Console.Write("?") 

The Sequence collection is not changed. Only the source, WordArray is changed. But the result is different anyway.

are men's souls! that These times try ?

That's what a "deferred LINQ query" is. The query is only executed when it's enumerated, not when it's defined.

To make matters more confusing, some LINQ queries are deferred and some are not. All of the Aggregate queries, for example, are not deferred. These return a single value so they're not enumerated. This example shows that when the original WordArray is changed, the answer stays the same this time.

 ' This just sets up the array
 Dim Words As String = "These are the times that try men's souls!"
 Dim WordArray() As String = Split(Words)
 ' This defines the an aggregate query on WordArray
 Dim ArrayCount As Integer = WordArray.Count(Function(n) True)
 Console.WriteLine(ArrayCount)
 Console.WriteLine(vbNewLine & "------------")
 ' Add another member of the array
 ReDim Preserve WordArray(UBound(WordArray) + 1)
 WordArray(UBound(WordArray)) = "Whatever"
 ' The Array is not counted again, the orginal count is still used
 Console.WriteLine(ArrayCount) 

Microsoft's documentation on this point is a bit skimpy. I recommend Chapter 3 of Pro LINQ - Language Integrated Query in VB 2008 instead.