Dan Swearingen's Tech Stuff Blog

“In addition to our summer and winter estate, he owned a valuable piece of land. True, it was a small piece, but he carried it with him wherever he went.”

From Woody Allen’s Love and Death.

So, what HAVE I been spending my time on? My little piece of the Windows Vista operating system.

For the last 20 months I’ve been building the digital locker assistant (DLA), a dedicated download client that works with Microsoft’s online digital locker, which is in turn part of Microsoft’s Windows Marketplace. Windows Marketplace is where that mysterious “Windows Catalog” link on your Start/Programs menu goes to.

Windows Marketplace supports direct browser based downloading. However, when the download is greater than 1-2 Gigabytes using the DLA is a much better way to go. The most popular use of the DLA so far has been buying and downloading entire copies of Windows Vista and Office 2007.

We were rather skeptical that users would want to download Vista or Office since they are really big downloads. However, earlier in the year the success selling and downloading super large games from Windows Marketplace convinced everyone that downloading Vista would be attractive to consumers. And indeed it has!

You can get the digital locker assistant two ways: If you have Windows XP, go to the Windows Marketplace website, create an account and download and install the MSI. It’s only a 1 meg download.

Or, it’s built into every copy of Windows Vista (except Server versions).

Actually building a part of Windows Vista was a huge effort but it’s really neat to install Vista and see my little piece in there. When I say “my,” it’s more like I’m using the Pluralis Majestatis, the Royal We. I was part of a team and We had LOTS of help.

I was the dev lead for the DLA for XP and Vista. Two very senior Windows developers with me at Vertigo, Chris Idzerda and Ralph Arvesen, rounded out the dev team (that is, they actually did most of the work). Initially, I was dev lead and PM but soon we needed more help with the process and got a full-time program manager, Anne Warren, who was also PM for the Windows Marketplace (WMP) website. The website dev team was some 15 developers and we had a build team of one (that should have been three). Our test team was in India so the dev/test cycle was almost 24/7, something like 24/6 – we’d hand off work in the afternoon and it would be tested all (our) night with a nice bug list waiting for us in the morning.

And then there’s the rest of the Vista team at Microsoft: really a cast of thousands. I think they ALL emailed me at least once. The High DPI functionality team. The Localization team (“do you know your UI looks really bad in Arabic?”), the Group Policy team, Remote Desktop team… you get the picture.

Let’s look at the app

In Vista there are two ways the digital locker assistant (DLA) may be invoked. The primary way is when you buy something at Windows Marketplace and it’s in your digital locker and you click “download.”

You may also browse to the DLA by finding it under the Vista Control Panel.

Then look under Programs or Programs and Features (using Classic view) and find “Manage programs you buy on line.” If you open this link you will invoke the DLA and if you have never sync’d up with your online digital locker you will see this:

If you have software in your online digital locker you can see it listed here by clicking on “Sign in if you already have a digital locker account.” Digital locker accounts are Windows LiveID (a.k.a. Passport) accounts mapped to a Windows Marketplace account. You’ll get a login prompt:

and after synchronizing with your online digital locker you’ll see all your purchased, free, and trial items listed. In my case (below) I clearly have a bunch of games in my locker. These were just to test downloading large items. Right.

Technology under the covers

The DLA is built in Win32/C++ as an ATL Windows application but we get some goodies from WTL as well. For those going “huh?” look at my post ATL and WTL resources.

At this point most people are asking me: why C++? Why not .NET and/or WPF? Or, if you’re using C++, why not MFC?

The DLA started (and is still available) as a downloadable application for XP. Our target users are what Alan Cooper would call “permanent beginners” (like that relative that always calls you for tech support…) — with a modem.

This means making the download as small as possible. Vertigo is a premier .NET shop but we could not use .NET because the 22 MB .NET runtime install kills us (that .NET never made it into the XP Service Packs… argh). Fortunately, we happen to have a few developers around (i.e., old geezers) who can do C++. We used ATL again to keep the size of the executable small.

In hindsight, it was just as well that the XP effort started in C++. Once we expanded the project to include being built into Vista we found that, in Windows System programming and the Vista source tree, C++ is expected and still king (See my post Has Microsoft flipped the Bozo bit on .NET? for a full discussion).

This meant that we could develop one source code base and, with some care, make it build in the Windows OS build system for Vista and VS2005 for XP.

Single source is nice but why not make a single binary that runs on Vista and XP? Sigh. We do — sort of, but it’s complicated.  From a programmer’s perspective, Vista makes one dramatic change from traditional Win32 applications and that’s in how localized resources are loaded.

To handle localization traditional Windows practice is to create an RC file for all resources (dialogs, images, sounds, strings, keyboard shortcuts, etc.) which are compiled into the resource DLL. Localization teams produce localized RC files based on your master RC file and these are all built into a suite of resource DLLs. At run time the application loads the appropriate resource DLL based on logic you have to write which looks at the calling thread’s locale settings.

Internal to the application is a language-neutral block of resources (typically English-US based) and if an appropriate external resource DLL cannot be found for the current locale settings, this internal block is used instead. This is known as “fallback” behavior.

Here’s the new twist in Vista: in Vista the OS loader (not the app) picks the resource DLL and locates it in memory where the app thinks its internal fallback resources are. This is expected behavior and currently only appears to work for a native Vista-built application so our “legacy” resource loading technique as used in XP was not acceptable to those who guard the Windows Source tree. Did I mention all the code reviews? Making Vista-style resource loading work in XP, while theoretically possible, was a task we did not choose to take on. So we ended up with one set of source code feeding two build processes; one for XP and one for Vista. Through careful coding there are remarkably few “if Vista do this, if XP do that” points in the code. 

While we currently block running the XP installer on Vista (in theory blocking installation of the XP DLA on Vista), it turns out that the XP DLA runs fine on Vista. I should not be suprised by this becuase we did quite a bit of casual sanity testing on this but it was not initially part of the test matrix. We found out by somewhat by accident as users were upgrading their XP machines (where the user had added the XP DLA) to Vista and then running the XP DLA.

For our downloading mechanism we hand off all download jobs to Microsoft BITS (Background Intelligent Transfer Service). While BITS works well for us I still think Micorsoft is tempting the gods by including “Intelligent” in their product name. BITS is the guts behind hwo Microsoft Updates are downloaded. I’ve also discovered that Google Updater uses BITS as well. What we gained by using BITS was automatic download management including background downloads, downloads that persist when our application is not running, downloads that seamlessly restart when the machine is rebooted, and lots of error handling algorthms that we did not have to write or maintain. I’d use BITS again if needed. We did have to build a simple HTTP download as well because some modem-based accelerators do not play nice with BITS.

Overall it was a great experience. While it was sometimes chaotic and exhausting, it was a lot of fun too. 

I’d do it again.

Really.

After I’ve had a couple years to rest.

Jeff Atwood (Coding Horror) graciously invited me to share in his recent Five Things You Didn’t Know About [Jeff Atwood] (and [his] office) post.

I too have been humbled and impressed by the other people’s stories in the Five Things meme so here are mine:

1.  I am baffled by other people’s fascination with professional sports.

I can identify with playing a sport – I just cannot understand the motivation behind the activity and entire economies driven by watching, talking, and writing about sports. That a city cares whether or not it has a football team and whether they have a nice stadium when at the same time its schools suck makes no sense to me. That a soccer fan needs to throw a beer bottle at someone who does not like their team makes no sense to me. 

2.  My family set me up to meet a girl they thought I’d like – and it worked.

I met my wife Janet at a family Thanksgiving dinner after months of my family telling me “oh, we want you to meet this girl we think you’ll like.” We’ve been together since that day in 1987.

3.  I am an Astrophysicist by education.

My career in software is a profitable side-effect. In school I was interested in everything and focusing on one thing to get a degree in was a challenge. Science seemed to be a recurring interest and my employer at the time (Hewlett-Packard) would pay tuition in that area so I chose to major in physics. After my BS I changed schools to work on a Masters. I was at Cal. State Northridge and they had a world class solar observatory and the astronomers were nice to work with so my M.S. in Physics was based on research into the solar magnetic field structure’s relationship with gas brightness and velocity. For my Ph.D. I wanted to work on something other than solar astronomy so amongst the multitude of choices I had (I think it was two) I joined the astronomy department at Indiana University, Bloomington. I was there five and a half years and finished my coursework, my Physics and Astronomy Ph.D. qualifying exams, and made good progress on my dissertation research. However, family and financial issues prevented my staying the extra one or two years it would have needed to complete my Ph.D so I left IU with a “thank you for playing” Masters in Astronomy. In the mean time I had picked up good programming skills that people were very interested in paying me for.

4.  I am an artist by inclination.

My family is populated mainly by performing artists and musicians. Throughout my life I have built things with my hands and the design and construction of structures of fantasy remains my hobby. I’ve focused on model railroading but I also draw and build other 3D art forms. The artist in me determines whether I’m interested in a project: if there’s room for creativity I’m interested. If you just need me to turn the crank, I’d rather be digging ditches.

5.  I’m a recovering Aspie (link)

This might be something you already know. In which case my not realizing you know is a demonstration that I have Asperger’s Syndrome. That’s an Aspie joke.

I was not diagnosed until I was in my 30s. I had always known I had to work especially hard at things that appear easy to people around me and all my life I have been known as “really smart – but weird.” It has been really helpful to have a diagnosis. It hasn’t changed anything I do but it has helped me feel vastly better about the coping mechanisms I employ. I still have most of the traditional problems Asperger’s deal with. Let’s face it, most good software developers are what we call “on the spectrum” so I’ve always fit in really well in good development organizations. However, unenlightened employers almost invariably feel I’m stubborn and arrogant when I’m actually very shy and I strive very hard to be a nice person. As far as being stubborn, I like to think I mitigate it by usually being right (although I now know it usually doesn’t matter that you’re right, I’m still naïve enough to be surprised). Fortunately, I’ve had enough really good employment experiences over the years to be comfortable with what part is me and what part is them.

In my fourth grade class we had a two day workshop put on by representatives from NASA. The Space Race was big in American life at that point. The activity was a simulated mission to Mars with the class divided into mission control and crews of two ships traveling together. On the morning of the first day we were informed that one of the ships had crashed on the surface of Mars and was unable to make the return flight. I was in charge of life support on one of the ships. In a few minutes I provided a solution with charts and resource burn-down graphs. By doing that I effectively ruined the next day and a half of the activity for the whole class.

Soon, after days of tests and interviews I spent less and less time in my regular class and starting in 5th grade was moved to a special class with a only six students (2 each in 4th, 5th, and 6th grades) and told I could study whatever I wanted. I studied the history of transportation, astronomy, puppet theater, building geometric shapes out of manila folders, whatever my interests wandered to. As a class we regularly left school and took a bus downtown to get books out of the San Francisco main branch library as needed since we exhausted the resources of the school library after a couple weeks.

For the next four years I was in the San Francisco schools I never sat in a regular classroom and had the most fun I ever had in school (until graduate school). Every now and then someone would stop by with a textbook and say something like “Dan, you really need to learn some math. Work these chapters, please?” A few days later I’d hand in the work and could go back to following my interests.

When I was middle-school age I moved away from San Francisco. High school and college were much harder. Many teachers mistook my shyness (silence) and inability to flow with the program as stupidity. They would put me in remedial reading one week and only to return me to “A track” (college prep) the next after demonstrating reading over 1200 words per minute (as high as their machine could go). I graduated high school with a D-plus GPA but nearly maxed out my SATs. Only in graduate school and when working on a Ph.D. in astrophysics was I having fun in school again.

I’m a former United States Marine. That probably should make six things you didn’t know about me. I put it here in the Asperger’s portion because many of my most effective coping mechanisms were learned during training in the Corps and I’ll always be grateful for that. I also learned how to get shoes REALLY shiny.

One of the most common gotchas to getting any data driven application working is the database connection string. How many times in your development life have you heard “I can’t connect to the database” and the problem ends up a minor detail in the connection string?

When I was working on a project with IdeaBlade one of the developers showed me a neat trick: keep a UDL file on your desktop. 

While there are whole websites devoted to connection string details, a simple UDL file on a Windows system gives you a really easy way to configure and test a connection string directly against the database you want to use.

In Windows the extension “.udl” is registered as a “Microsoft Data Link” and the default program is OLE DB Core Services. Not very intuitive so let me walk through the basics.

A UDL file is actually a text file so start by creating a text file on your desktop. Right-click on the desktop and select New and Text Document.  Note: I did this on Windows Vista but this works the same way on any modern version of Windows (I’ve checked as far back as NT 4.0).

Name your new file Connection.udl – or, whatever but you need the “.udl” extension. Ignore the “If you change a filename extension…” warning . The file will take on an icon appearance that is not a text file. Here’s how it looks on Vista

And XP

Now double click on (or right-click Open) the file.  You should see a familiar database connection configuration dialog.

To connect to your local SQLExpress database go to the Provider tab and select Microsoft OLE DB Provider for SQL Server.

Now go to the Connection tab and select the database to connect to. You may just browse through the list but SQLExpress instances do not always show up. You may also type the server name into the listbox. For SQLExpress it will usually be [machine name]\SQLEXPRESS if you have a full SQL 2005 instance installed it will just be the machine name. In this case I’m connecting to my local SQLExpress database on my machine HAVOCVISTA.

Select the authentication you want to use. If using a username and password choose whether you would like to embed the password in the connection string.

Now select the particular database. If the dropdown is populated when you click on it you already have a good connection. If not, then the problem is going to be the name and/or the authentication provided. Assuming all’s well, select the database and click the Test Connection button.

Now close by clicking the OK button.

Here’s the “Ah Ha” step.

Now open the file in notepad – remember, the .udl file is simply a text file. You should see something like this:

In the file is the connection string you just tested. Copy and paste where needed in your application and you should be good to go.

The story of ATL and WTL (fit for a cocktail napkin):

In the beginning, (at least in *this* beginning) there was COM. As developers embraced COM programming for Windows applications Microsoft created the Active Template Library (ATL) as a framework to simplify and envelop the routine tasks in the creation of COM components.

Using ATL developers were happy with the ease with which small fast components could be created. They were unhappy with all the code wrappers they had to write to use any Windows controls. And, of course, everyone wrote their wrappers differently so this became an entropy generator on projects. While ATL does provide “Window” classes, they really don’t help much outside of COM. They’re mainly intended for COM control property pages.

In response, developers within Microsoft developed (in an unsupported way) the Windows Template Library (WTL). WTL extends ATL and provides a framework of light wrappers to use with Windows controls. Developers competent with ATL find WTL a great framework to quickly build small (in terms of KB EXE size) applications. Versions 7.0-7.5 are available from Microsoft but WTL has been released in the public domain and is maintained at SourceForge (see links below).

On a recent project I found these resources useful in catching up on how to work with ATL/WTL:

ATL resources:

Code:

ATL is part of the Windows SDK. 

Web:

New MFC and ATL Features - MSDN
ATL Samples - MSDN
ATL Server Samples - MSDN 

Code Project: Active Template Library (ATL)

Books:

ATL Internals: Working with ATL 8, 2nd Edition (New! July, 2006) (Amazon), by Christopher Tavares, Kirk Fertitta, Brent Rector, Chris Sells.  Very good but does not have any material on ways to build ATL Window apps without WTL (Note: I stand corrected, see Chris’s post below). Chris Sells’ web page for this book 

Beginning ATL 3 COM Programming (1999)(Amazon), by Julian Templeman, Richard Grimes, Alex Stockton, Karli Watson, George V. Reilly. This is “beginning” as in “beginning cliff diving” – has an ATL Windows app section.

Professional ATL Com Programming (1998) (Amazon), by Richard Grimes.

Developer’s Workshop to COM and ATL 3.0 (2000) (Amazon), by Andrew Troelsen.
 

WTL resources:

Code:

Windows Template Library (WTL) 7.0 4/2/2002
Windows Template Library (WTL) 7.1 12/9/2003
Windows Template Library (WTL) 7.5 6/13/2006
WTL open source at SourceForge. V 7.5 is released, 8.0 in development.

Web:

Chris Sells: Whitepapers, sample code and walkthroughs. Based on an old version of WTL but still the best starting point. WTL Makes UI Programming a Joy, part 1 and part 2, 6/2000 

Yahoo! Tech Group: WTL – online group discussion and help.

Code Project: Windows Template Library (WTL)
Code Project: Michael Dunn’s series on WTL for MFC Programmers is outstanding. Something like ten parts. Starts with WTL for MFC Programmers, Part I – ATL GUI Classes 

Books:

None as far as I know.

Sci Fi literature is happening again. Ok, I’m sure some of you are saying “it’s been happening, where were you?” All I know is that in the 90′s my wife was a librarian and we had good visibility on the fact that Science Fiction was being completely overwhelmed by Fantasy Fiction (you know, sword and sorcery stuff).

My favorite theme is “what happens when life becomes virtual?” Simon Fell recommended Charles Stross’s Acclerando on his blog. It started kind of tough but I really enjoyed it.

If we can take on any shape and live in any kind of world, at what point do we stop being human?

One of our PCs died last week. What a chore. The machine was about three years old and there were no warning signs. The machine is at a desk in the bedroom and while I was watching TV one evening I heard it stop. I tried restarting; it whirred for a bit and then shut down. Uh-oh. Pulling the machine out I’m thinking “oh, it’s just a power supply.”

It’s never the power supply 

I have a bunch of really nice power supplies in their boxes from all the previous occasions I thought it was the power supply. My experience in over 20 years of “modern” PC ownership: it’s never the power supply. Not to say power supply failures don’t happen, it’s just that they don’t happen to me (yet). My experience has been: if the machine dies, it’s dead. There’s no fixing it. Pull the hard drives out and move on.

Ok, I heard a bunch of people say to themselves “wait, you could troubleshoot it and have it working in a few weeks after a few dozen trips to Fry’s and Radio Shack for some simple parts. An oscilloscope would show you …” Right. Look, $699 buys a lot of machine these days. $699 is much cheaper than several weeks of my “spare” time lost to chasing a problem (that the CPU really did fry because the heat sink was too full of dust).

What really fails? 

For me the devil has been miscellaneous “motherboard/CPU issues” (three times, counting this failure), and disk failure (once). I typically have 4-5 machines in use in the house at any one time. Over 20 years x 4.5 machines = 90 machine-years of PC use so with only four failures I think I’ve had very few problems. The one disk issue gave me lots of warning: it was an NT 4.0 system and the system log started showing disk errors. I was able to get everything off onto a new drive so I’ve never lost data (so far). I still have emails stored from 1990 (I’m not sure that’s a good thing).

What works?

Running machines all the time 24×7 is definitely better than turning them on and off once a day. I have a domain controller that is a Pentium Celeron 300 MHz built ten years ago. It’s had three drive upgrades but still runs fine.

Distribute critical data. I almost never make backups. There. I said it. Yeah yeah yeah, I know you’re supposed to make backups. EVERYONE knows you’re supposed to make backups.

At least I’m honest enough to admit I generally do not make backups. Anyone else who says they are continuously backed up is simply a liar.

Who has time to row fifty (or one hundred) CDs through the CD burner? It’s not that I have not tried. I backed up a 20-meg hard drive onto 3.5″ floppies once (remember 3.5 floppies?). I bought a tape drive once. Two hundred fifty whole Megabytes a tape. Modern hard drive capacity growth devours any backup strategy I can think of. Modern hard drives are also your fastest cheapest back up medium (see Raid 1 below).

Source Code Control: I use Source Safe for a lot of my data backup. I copy stuff to my laptop. I copy stuff to my work computer.

Raid 1 your disks: I buy SATA Raid 1 on all my new desktops. An extra $80 for a 200 Gig backup drive? That’s a no-brainer. Why bother with Raid 1 if I seemingly do not believe in backups? While I have never lost any critical data I DO mind how long it takes to rebuild a machine and get all my critical applications and tools re-installed.

Vacuum once in awhile: I now buy those cool cases with windows in the side and lights inside so it’s visually really offensive when it’s all full of dust inside.