Wooley's LINQ Wonderings

Screen scraping and creating Word documents with LINQ to XML

jwooley — Thu, 12 Jun 2008 18:22:00 GMT

At TechEd Developers 2008 in Orlando, I had the pleasure of competing in Speaker Idol. In that competition, we had the opportunity to present a topic in 5 minutes. Unfortunately, the topic I choose really needed 10 minutes to cover at the level of detail it needed. Instead of limiting the topic, I decided to go ahead and present it a bit too fast.

If you want to see the video, or see how to use VB 9's XML Literals and LINQ to XML to fetch data from a web page (that must be XHtml compilant), manipulate it and insert it into a Word 2007 file, it is now available on the Developer Landing page, and the Library page. of the TechEd site. If you prefer, you can jump right to the video in either WMV or MP4 file formats. If you're not familiar with LINQ to XML, go ahead and download the video and just watch it at half speed ;-)

LINQ to SQL's support for POCO

jwooley — Thu, 12 Jun 2008 02:25:00 GMT

One of the strengths that LINQ to SQL has over the upcoming Entity Framework is its support for POCO, or Plain Old Class Objects. With LINQ to SQL, the framework doesn't require any particular base classes, interfaces or even reliance on the 3.5 framework for the resulting objects. I demonstrated this in the talk I did at the Teched Tweener weekend. Download the demo project to see this in action.

In this sample, I created two separate projects. The first class library project, I created only targeting the 2.0 framework. As a result the project can not use any LINQ specific techniques. This will also allow us to consume the resulting objects in projects that do not have access to the newer framework, or to all of the namespaces. This is particularly important in cases like Silverlight. To call attention to the differences in the projects, I declared the 2.0 project in C# and the LINQ enabled project in VB.

The 2.0 class library project consists of a single class file. This represents the Subject entity from the Linq In Action database.

namespace UnmappedClasses

{

public class Subject

{

public Guid ID { get; set; }

public string Name { get; set; }

public string Description { get; set; }

}

Notice here, there are no interfaces, base classes or custom attributes. Excluding the attributes is critical here because the standard <Table> and <Column> attributes reside in the System.Data.Linq.Mapping namespace which would not be supported in the 2.0 framework.

Admittedly, it consists of three auto-implemented properties. Auto-implemented properties are used for brevity here and are consumable by the .Net 2.0 Framework because it relies on compiler features rather than runtime features.

Because we can't allow the class structure to include the attributes, we can't use the LINQ to SQL designer classes or SQL Metal to generate our classes. We do need to have a way to indicate the mapping to our data store. Here is where the XML Mapping file comes in handy.

When instantiating the DataContext, we can either rely on the inline attributes, or an external mapping file. Luckily, the XML mapping file's structure is concise and very similar to the attributes that would have been applied to the class otherwise. The main difference we need to do is indicate the Type that is used for a given table since we are not directly annotating the class itself. The other difference you may notice is that I don't include the Storage attribute. While there is nothing to stop me from using that in a Mapping source, we can't identify the backing field when using auto-implemented properties.

<?xml version="1.0" encoding="utf-8"?>

</Type>

</Table>

</Database>

Now, with that out of the way, we can get to the LINQ portion of the work. Actually, that is quite easy. In our 3.5 enabled project, we will create a XmlMappingSource, pass it into the constructor of the DataContext and then fetch the object from this context as we would any other LINQ enabled class.

Dim map = XmlMappingSource.FromXml(XDocument.Load("C:\Projects\LINQ\AdvancedLinqToSql\WinformDemo\lia.map").ToString)

Using dc As New DataContext(My.Settings.liaConnectionString, map)

Me.SubjectBindingSource.DataSource = dc.GetTable(Of UnmappedClasses.Subject)()

End Using

This example happens to bind the results to a Winform object binding source, but you could expose it to ASP directly, through an encapsulation layer, like a repository pattern, or a service interface.

Crossposted from www.ThinqLinq.com

LINQ in Action at TechEd Developer 2008 in Orlando

jwooley — Tue, 03 Jun 2008 15:41:00 GMT

If you're at Tech Ed Developer in Orlando, make sure to find me. If you miss me on the convention floor, I'll also be participating in Speaker Idol on Wednesday 6/4 at noon. In addition, I will be doing a book signing in the store at 1:00 on Thursday, 6/5. As you can see from this picture, they have a couple copies of the book that you can buy if you didn't bring your copy. I look forward to meeting you.

Danny Simmons compares the Entity Framework to similar technologies

jwooley — Wed, 28 May 2008 02:37:00 GMT

It seems that everyone else is chiming in on Danny Simmons' recent comparisons of the Entity Framework with other similar technologies. There are several items I wanted to address from his observations.

Regarding the EF vs. LINQ to SQL, he makes two basic points: 1) That there isn't a provider model for other data sources and 2) That LINQ to SQL requires a 1-1 table to object mapping. On the second item, there is no denying the limitation. While you can work around the issue with LINQ to SQL's support for views, stored procs, and functions, this is the key differentiator between LINQ to SQL and the Entity Framework. Danny's statements regarding inheritance limitations and entity splitting both stem back to the TPH (table per hierarchy) limitation of LINQ to SQL.

Regarding the issue with the provider model, the original intent was to have a full provider model which vendors would be able to consume. When the ADO vNext initiatives came out, it was fairly clear that the provider model would be supported in the Entity Framework. As a result, the provider model for LINQ to SQL was essentially shelved. That was the genesis of Matt Warren's series on building an IQueryable provider which basically outlines how one would go about building a provider for LINQ. Since Matt has basically handed the LINQ to SQL code base over to Danny's team, we'll see where this progresses.

In Danny's discussions of other technologies, he seemingly makes two contradictory statements. In the beginning he states,

"... the truth is that different problems merit different solutions."

He then goes on to the statement:

"This makes it easier to create a conceptual model which is how you want to think about your data and then reuse that conceptual model for a number of other services besides just building objects. Long-term we are working to build EDM awareness into a variety of other Microsoft products so that if you have an Entity Data Model, you should be able to automatically create REST-oriented web services over that model (ADO.Net Data Services aka Astoria), write reports against that model (Reporting Services), synchronize data between a server and an offline client store where the data is moved atomically as entities even if those entities draw from multiple database tables on the server, create workflows from entity-aware building blocks, etc. etc."

By this, he is indicating that the EF should be the defacto mechanism for building the entity structures around which the various components of your system will be built. Thus one tool to answer the various solutions. Is the EF really the best way to expose relational data from a reporting or analysis perspective, time and performance analyses will tell. He does go on in the comments to his post indicating that he is not necessarily advocating re-using the same entities for all of these various application components, but rather to use a similar tooling to create the potentially separate entities that each subsystem will consume. While it makes the programmatic portion easier, the jury is still out on the other areas.

I'm still getting up to speed on the EF, but do have concerns regarding the POCO (Plain Old Class Objects) story and some of the other requirements that the EF puts on your entity and database models.

Crossposted from http://www.thinqlinq.com/Default/Danny-Simmons-compares-the-Entity-Framework-to-similar-technologies.aspx

Projecting an unmapped property into an anonymous type with LINQ to SQL

jwooley — Thu, 08 May 2008 23:40:00 GMT

On page 216 of LINQ in Action, I made a comment that unmapped properties in a mapped class cannot be used in a LINQ to SQL projection. This was true with the beta bits, but only partially true with the release bits. To begin, let's consider the Author table we have in the book samples.

The Author class has separate fields for the first and last name. Each of these is mapped to the corresponding fields in the Author table. In the book, we show how you can create a read only property in a partial class (so that it won't get clobbered when we regenerate our classes in the designer). The new property is trivial:

   Partial Public Class Author

        Public ReadOnly Property FormattedName() As String

Get

                Return Me.FirstName & " " & Me.LastName

            End Get

        End Property

    End Class

Notice here that there are no mapping attributes to this property. In part, that is because there is no corresponding field in the table. As we show in the book, you are free to query the author table and return Author objects. From there, you can display the FormattedName as follows:

           Dim authors = From a In context.Authors _

                          Select a

            For Each a In authors

                Console.WriteLine(a.FormattedName & "; " & a.WebSite)

            Next

This works fine because we are projecting the complete Author type. However, in early builds, we couldn't project the unmapped properties into an anonymous type like this:

            Dim authors = From a In context.Authors _

                          Select a.FormattedName, a.WebSite

If you tried to use this projection, you would get a runtime exception. In the RTM bits, the behavior was modified. Now, if you try to run the above query (sample 8.25 in the book samples for anyone following along). You will see that the query succeeds and the anonymous type is used. So how can they know how to populate the FormattedName when it is not mapped and doesn't exist in the table itself? No, the provider doesn't look inside the property, determine the mapped properties that are used, and fetch them. While that could work in our simple example, many unmapped properties would use significantly more resources, many of which may not be members of our class, or methods without direct translation in TSQL. If you look at the generated SQL that is issued when the query is consumed, you might be able to figure out what is happening in this case.

SELECT [t0].[ID], [t0].[LastName], [t0].[FirstName], [t0].[WebSite], [t0].[TimeStamp]
FROM [dbo].[Author] AS [t0]

Notice here, our select clause to the database is not optimized to only return the fields we requested. Instead, all of the fields are returned. So what's going on? They discovered in evaluating the Select clause that there were unmapped properties. At that point, they just turned around and populated a full author object. Using this object, the provider turns around and generates the anonymous type from the object rather than directly from the underlying data store directly. It's a bit of smoke and mirrors at this point.

So the question that came up asks if the next printing of the book needs to be adjusted to remove the statement that you can't project an unmapped property. While you can indeed project these properties, you can't use them elsewhere in the query. Thus if you wanted to sort the data based on the unmapped property, the exception would be thrown. Consider the following query./p>

            Dim authors = From a In context.Authors _

                          Order By a.FormattedName _

                          Select  a.FormattedName, a.WebSite

In this case when we try to run it, we get the following error:

"System.NotSupportedException: The member 'LinqInAction.LinqBooks.Common.VB.SampleClasses.Ch8.Author.FormattedName' has no supported translation to SQL."

Because of this, I plan to leave the note in the chapter warning you of using the unmapped property in your query. Unfortunately, I don't have enough space in the book to insert this complete explanation at this time. I hope this explanation helps some of you who are confused at this point.

Crossposted from ThinqLinq.com.

Geek Speak resources for the LINQ Migration Strategies talk

jwooley — Thu, 03 Apr 2008 01:44:00 GMT

I had a blast on the Geek Speak today. If you missed it, they will have it available on demand from their blog at http://blogs.msdn.com/geekspeak/. You can even subscribe to the audio feed using your the Zune podcasting functionality from their rss feed. One of the things I love about these events is the variety of the questions that attendees bring.

As I mentioned, the slide deck I used for the session is available in the Files section here as Linq Migration Strategies.

Below are some of the questions that came up and some links to back up my answer for your reading edification.

Q: How do you import schema into VB at the top of the file, if schema is in project it will give you intellisense?

Beth Massi and Avner Aharoni demonstrate this in the webcast available at http://blogs.msdn.com/bethmassi/archive/2008/01/18/channel-9-interview-xml-properties-and-enabling-intellisense.aspx. They also demonstrate generating the schema from a sample XML document.

Q: How would you replace an existing data layer with LINQ to SQL?

I started showing how to take the Personal Web Starter Kit and begin to LINQ enable it. The completed project is available at http://code.msdn.microsoft.com/LinqPersonalWeb.

Q: How does LINQ perform as compared to the other alternatives?

Any time you add a level of indirection there will be some performance penalty. With LINQ to Objects, they basically leverage the iterator pattern the same way the C# 2.0 Yield method works. For LINQ to SQL, the best series on performance is from Rico Mariani starting with http://blogs.msdn.com/ricom/archive/2007/06/22/dlinq-linq-to-sql-performance-part-1.aspx

Q: Where can we find patterns and practices guidance on LINQ

This post is more along the lines of the Framework Design Guidelines book, but it is good information anyway: http://blogs.msdn.com/mirceat/archive/2008/03/13/linq-framework-design-guidelines.aspx

There is also a Live from Redmond VB9 webcast discussing best practices at http://msevents.microsoft.com/CUI/WebCastEventDetails.aspx?EventID=1032337466

Additional "How do I" videos are available at http://msdn2.microsoft.com/en-us/vbasic/bb466226.aspx

Q: What advice do you have for passing data across tiers? If we can’t pass context across tiers, can we pass resultant objects?

There isn't really a single difinitive post on this one. Searching for "DataContext" and "Short lived" or "Unit of work" should supply a number of examples, including the following:

http://msdn2.microsoft.com/en-us/library/system.data.linq.datacontext.aspx

This forum post includes responses by Keith Farmer and Matt Warren who were both intimately involved with creating LINQ to SQL: http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2485909&SiteID=1. Matt states, "You'll most often want a new context for every interesting unit of work. DataContext's should be short lived if they can be. The only scenario where you'd keep a DataContext around for a long time would be if you were bulding the equivalent of a 2-tier UI based data-entry/editting appliction."

Dino Esposito notes: http://weblogs.asp.net/despos/archive/2008/03/19/more-on-datacontext-in-hopefully-a-realistic-world.aspx

Some additional resources mentioned:

Rick Strahl: http://www.west-wind.com/weblog/default.aspx

Beth Massi: http://blogs.msdn.com/bethmassi/

MSDN LINQ forum: http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=123&SiteID=1

Q: Regarding Linq to Entities, I hear that the entities will not use the same change tracking model to permit emitting across tiers, by making the EntitySets serializable and have the changes tracked on a set by set basis, can you confirm deny? Info on this?

I don't know enough to comment on this. I would recommend asking the question on the ADO.NET Entity Framework forum at http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=533&SiteID=1.

Crossposted from ThinqLinq.com

GeekSpeak discusses LINQ Migration Strategies

jwooley — Tue, 01 Apr 2008 11:45:00 GMT

Tomorrow, Wednesday 4/2/2008, I will be the guest speaker on the Geek Speak webcast. We will be discussing strategies for beginning to incorporate LINQ into your existing application infrastructure. In many cases, that does not mean replacing your entire data stack, but rather using pieces of LINQ to add functionality and in new components. Please join us. The Geek Speak webcasts are often driven by attendee questions. The more questions, the better the event.

When: Wednesday, April 02, 2008 12:00 -1:00 PM (GMT-08:00) Pacific Time (US & Canada) or 3:00 - 4:00 PM Eastern Time.

Where: Live Meeting

Registration URL:

http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032374380&Culture=en-US

Crossposted from www.ThinqLinq.com.

LINQ enabled Personal Web Starter Kit

jwooley — Wed, 12 Mar 2008 11:44:00 GMT

I love it when projects take a life of their own. A while back, I posted my LINQ enabled Personal Web Starter Kit in VB and received several requests to provide a C# port. Thankfully, one brave soul stepped up and did the port for me. Thanks go to Stephen Murray for undertaking the challenge. As is often the case, one of the best ways to learn a technology is to use it.

If you're interested in this sample, you can check out the project at the MSDN code center. Specifically, you can access the original VB version or Stephen's C# Port. You can read more about this project on the Thinq Linq web site. As always, let us know what you Thinq.

LINQ in Action going to press

jwooley — Mon, 21 Jan 2008 08:16:00 GMT

At long last, the process of doing my first book is coming to a close. I started this project last March. Through the process we had to revisit our work numerous times, including each time a new CTP or Beta drop came. For me, 10 months, and Fabrice 2 years later, we found out this week that the book is going to press.

What does this mean for you, if you purchased the eBook, the final version is available now. Additionally, the samples are available online in both C# and VB. We are also making three chapters available for free if you are considering the book, but not sure yet.

If you purchase the hard copy from Manning, we understand that it should be in around the first of February. This should mean that it will be shipping from the online outlets, like Amazon by the middle of February.

I hope you find the book as rewarding to read as it did for us to write.

crossposted from www.thinqlinq.com

Crossposted from http://devauthority.com/blogs/jwooley/default.aspx

Annoucing ThinqLinq.com

jwooley — Thu, 10 Jan 2008 07:46:00 GMT

You may have seen me present it at a speaking engagement. You may have watched the podcasts. You may have even downloaded the sample application. Now you can see it in action.

ThinqLinq.com is now live.

The site was designed completely in VB with LINQ as the data access mechanism. The base application was built in 2 hours from not knowing RSS to being able to import a RSS feed, displaying it on a form and producing a new feed from the imported data. The site is a testimate to the power of LINQ and the RAD capabilities that it brings. Head on over to the site and check it out.

Crossposted from http://devauthority.com/blogs/jwooley/default.aspx

More on VS 2008 breaking changes

jwooley — Mon, 03 Dec 2007 00:49:00 GMT

In addition to the items I mentioned in my previous Beta 2 - RTM breaking change list, I found a link on the VS 2008 samples page that a whitepaper has been issued on this.

In addition, there is a whitepaper specific to breaking changes between VB 2005 and VB 2008 available here.

Technorati tags: VB Visual Studio

Crossposted from http://devauthority.com/blogs/jwooley/default.aspx

Projecting XML from LINQ to SQL

jwooley — Tue, 20 Nov 2007 07:08:00 GMT

Among the new cool features in Visual Studio 2008, one of the best may be the XML Literal support with VB 9 and LINQ. In my last post, I mentioned some changing features from the Beta to RTM. One that could easily be overlooked is a change to the way LINQ to SQL can now directly project into XML literals.

Through the Beta cycle, there was an issue with projecting XML elements directly from a LINQ to SQL query. If you haven't seen LINQ to SQL with XML, here's a code sample that explains what I'm referring to:

Dim dc As New LinqBlogDataContext
'Formulate the Query to get the last 10 blog posts
Dim query = (From p In dc.PostItems _
             Order By p.PublicationDate Descending _
             Take 10 _
             Select p).ToArray

'Create a root Site node with 10 child "item" nodes.
'Each node will be filled in through a LINQ query
Dim fooShort = <site>
                 <%= From p In query _
                     Select <item>
                          <title><%= p.Title %></title>
                              <link>http://ThinqLinq.com/Default.aspx?Postid=<%= p.Id %></link>
                              <pubDate><%= p.PublicationDate %></pubDate>
                              <guid isPermaLink="false">42f563c8-34ea-4d01-bfe1-2047c2222a74:<%
p.Id %></guid>
                              <description><%= p.Description %></description>
                            </item> %>
               </site>

In this code, I'm performing two queries. The first one sets up the LINQ to SQL query and pre-fetches the results into an Array. In the beta builds, if we didn't include the pre-fetching ToArray, the second query which projects the results of the first into individual <item> nodes. What is the difference between these queries? The first query uses LINQ to SQL and projects results directly from the database. Because we pre-fetch the results into an array of objects, the resulting query only uses LINQ to Objects rather than the direct LINQ to SQL implementation.

With the final RTM of Visual Studio, we no longer need to pre-fetch the results from the query. Instead, we can directly project our desired XML from the select statement without needing the intermediary step. Here is the revised code. Notice, we can now perform the same result with a single LINQ query rather than two.

Dim fooNew = <site>
               <%= From p In dc.PostItems _
                   Order By p.PublicationDate Descending _
                   Take 10 _
                   Select <item>
                            <title><%= p.Title %></title>
                            <link>http://ThinqLinq.com/Default.aspx?Postid=<%= p.Id %></link>
                            <pubDate><%= p.PublicationDate %></pubDate>
                            <guid isPermaLink="false">42f563c8-34ea-4d01-bfe1-2047c2222a74:<%= p.Id %></guid>
                            <description><%= p.Description %></description>
                          </item> %>
             </site>

The result is more concise. You may find you want to continue separating your query definition from your XML creation in order to improve maintainability. If this is the case, simply keep the first code sample and remove the call to .ToArray. Because LINQ to SQL is composable, you can separate the queries into two code sets. When the query is evaluated, the two expressions will be combined into a single query to the database and the projection will continue to work.

Enjoy working with VB 9 and XML. In my opinion it is one of the killer features of Visual Studio 2008. If you give it a try, I think you might find the same.

Technorati tags: linq, VB Visual Studio XML

Crossposted from http://devauthority.com/blogs/jwooley/default.aspx

Visual Studio 2008 (Orcas) Changes from Beta 2 to RTM

jwooley — Tue, 20 Nov 2007 06:53:00 GMT

I've just updated the ThinqLinq proof of concept site for the Visual Studio 2008 release that came out today. If you're following the sample application, or are looking for a sample VB 9 implementation of LINQ in a web site, check out the download at http://devauthority.com/files/13/jwooley/entry101097.aspx. In case you are interested, here are the changes that were necessary to move from Beta 2 to the RTM. (The first two items are repeats from my post earlier today).

Open the DBML file as XML and change the UTF encoding from 16 to 8. Otherwise the compiler will complain about the DBML file's format.

Replace the Add methods with InsertOnSubmit. Likewise, change Remove to DeleteOnSubmit. You may be able to do a global search and replace on this, but be careful not to make changes to any IList object implementations, only LINQ to SQL ones.

SyndicationFeed.Load removed the option to pass in a URI. Instead, I used SyndicationFeed.Load(New System.Xml.XmlTextReader(UrlString)).

The SyndicationFeed's Item PublishDate property is changed to the new DateTimeOffset type rather than the simpler DateTime. As a result, get the date from the PublishDate.Date property.

When projecting XML elements as part of a LINQ to SQL query, you no longer need to pre-fetch the results into an array. I'll make a separate post on this item.

That's all I've found so far. I've already updated both the ThinqLinq site and the samples for the upcoming LINQ in Action book. I'm sure I've missed something, but so far, the upgrade is relatively easy this time around. The IDE does appear to be running faster as well.

Technorati tags: linq, orcas Visual Studio

Crossposted from http://devauthority.com/blogs/jwooley/default.aspx

VS 2008 RTM and the .NET Framework 3.5 Shipped

jwooley — Mon, 19 Nov 2007 19:09:00 GMT

It's offical, Soma annouced on his blog that the 3.5 .Net Framework along with Visual Studio 2008 have shipped. I have been excited by the things coming with this release since PDC 2005 and am glad to finally work the the official bits. I am aware of a couple breaking changes between the Beta 2 and RTM, and will try to keep you updated as I find more items.

The biggest changes to be aware of in regards to LINQ to SQL are the following:

Open the DBML file as XML and change the UTF encoding from 16 to 8. Otherwise the compiler will complain about the DBML file's format.
Replace the Add methods with InsertOnSubmit. Likewise, change Remove to DeleteOnSubmit. You may be able to do a global search and replace on this, but be careful not to make changes to any IList object implementations, only LINQ to SQL ones.

There are a couple other minor updates that may catch you off-guard. I'll update you once I've had a chance to play with the final bits.

Crossposted from http://devauthority.com/blogs/jwooley/default.aspx

Feedburner access to Wooley's various Wonderings

jwooley — Fri, 09 Nov 2007 23:24:00 GMT

I have set-up several feedburner syndication options for my postings for your enjoyment. Feel free to move your aggregator over to those versions of the feeds and show me how much you digg what I have to say. Here's the links for you:

Wooley's Wonderings: My main feed
Jim's Samples and Presentations: My downloads and sample applications from my presentations. Subscribe here to get updates whenever I add new samples.

I have a bunch of things I want to discuss from my DevConnections trip, so stay tuned.

Crossposted from http://devauthority.com/blogs/jwooley/default.aspx