We have (had) over 500 channels on our current service and over 100 HD channels and you know what: we never watch anything beyond 30 minutes of local news each morning. We’re paying over $170 a month and there’s nothing we bother watching.

We have a 46″ screen in the living-room that has never seen a cable connection. Kids are way ahead of old folks like me. Hulu, Boxee, YouTube, Blu Ray, DVD: these are how we watch things now. When I asked our son if it was ok to get rid of cable TV he actually asked “what’s that?”

That killed my last doubt: it’s a waste of money, we’re pulling it all out.

The money saved in 10 months by getting rid of cable TV will buy two really nice media PCs so we’re keeping our cable-provider based internet connectivity. We’ve had 24/7 high-speed Internet in our house since 1995.

The land-line phone is probably next. Six months of saved phone bills will pay for a nice signal amplifier so our mobile phones will work great indoors.

Can you hear me now?

Update 2/21/2010 — I’m weak. I ended up cutting down to $16.95 “basic cable.”

ARPANET in 1977

I’m reblogging ’cause I’m busy but I’ll have my media PC build and description up soon.

Here are the choices I made.

Start with C++

If you want to be a professional programmer for the rest of your days, you should be an expert in C++. It’s all there: pointers, Object Oriented Programming, complex build issues: all the experience you’ll need.

Get a book to structure the “class”

All books suck to some extent. Just pick one and go. I chose Sams Teach Yourself C++ in One Hour a Day (6th Edition). It is big on C++ syntax, light on Computer Science topics.

Assign outside projects

I’m assigning tasks like a console Hangman game, The Game Of Life, things like that. Nothing makes concepts concrete like actually building a program that does something.

Teach Computer Science as taught at a great university

To be a professional programmer you need to be literate in computer science. Like any other self respecting astrophysicist (and therefore professional know-it-all) I tried for years to avoid it but finally gave in and learned CS.

After my student makes a pass through the Sams book above, I’ll start using the curriculum available free online at MIT’s Open Courseware. We’ll be starting with 6.00 Introduction to Computer Science and Programming

Could I ever! I don’t know. Could I??

I’m old. I’m not so sure I’m a good example. The generations of my programming learning:

1. 1970′s — Poking around using BASIC on pre-PC microcomputers.

2. 1980′s — Writing a fair bit of software on my Timex ZX-18 using their BASIC.

3. 1980′s — Fortran 77 on PC and SunOS.

4. 1980′s — Writing a fair bit of software on PCs running DOS with Turbo Pascal.

5. late 1980′s — FTP, lots of ftp. Starting to use email. Writing Fortran apps on mini computers and using my PC as a VT-100 terminal with Kermit and a 1200 baud modem. Learned vi. Turbo C 1.0 (pre-ANSI standard).

6. 1990′s — Lots of Fortran 90 written under HP-UX and VMS. Starting to use C to interface with hardware.

7. 1990′s — CDROM based “Multimedia” projects. Silly aquarium games. Very painful under Windows 3.11.

8. 1990′s — WWW, the internet, HTML, cgi-bin scripts in Perl. Spending lots of time configuring web servers.

9. late 1990′s — The dot-com boom. All web, all the time. Microsoft ASP (classic: VBScript). Learned COM, ATL to enhance big MS technology web servers.

10. 2000′s — Big non-Microsoft web technologies. Linux, Java, Tomcat, other application servers.

11. 2000′s — Post dot-com: back to Microsoft with .NET! Love it. Back to client applications.

12. 2000′s — back to the web: ASP.NET. Love it.

13. 2000′s — set your way back machine: A huge project pulls me back to ATL/MFC/C++. Coding like it’s 1990 (Vista systems).

14. 2000′s — Computer game technology: classic C++, wicked algorithms, cool deep technology stacks like rendering pipelines, AI, terrain.

Looking back, what have I learned? A lot and especially that not all learning pain is gain.

I have growing respect for technologies that have evolved and stand the test of time. Looking down the list the Internet, client applications, and web-connected applications ebb and flow but feel like they will be around for awhile.

C++ is awesome. C#/.NET is way up there too. C# feels like C++ but someone has been in and tidied everything up. HTML has got to be one of the most ubiquitous formats on the planet.

Oh: it had to be quiet!

I ordered everything from Amazon and the prices are what I paid in January 2009. The basic system cost $1,062 and the KVM cost $564.

The Keyboard, Video, and Mouse:

I’ve been really happy with this system. Quiet, stable, fast, and I love the case!

We need to find a good, easy to use software program for generating the [website name here]. PC based. Any suggestions?

My snappy answer is

WordPress

WordPress.org       WordPress at Wikipedia

Why?

I’ve used notepad, vi, VisualStudio, Netscape Gold, DreamWeaver, … All kinds of different tools. On the server side: Netscape Server, cgi-bin with perl, IIS (since NT 3.51), all sorts of pre-IIS app servers I can’t even remember the names of, Java technologies and LAMP.

I’ve used those bastard website-in-a-box things almost every ISP offers.

Way too many times.

Looking back I have to ask myself: which techniques and technologies stood up to the test of time? Which do I look back at with no regrets?

1. WordPress on LAMP or IIS
2. IIS/ASP and IIS  with ASP.NET

On the desktop editing/managing side? VisualStudio and Visual Source Safe

That’s it. I assert that you can build ANYTHING with these technologies. I routinely build sites that use BOTH 1. and 2. — like polyweb.com.

If you are a beginner: go pure WordPress. if you need more site functionality there is almost anything you could want available as a wad of PHP you can tack on.

If you need a real web application — and you think it is going to get large: ASP.NET is astoundingly powerful and lots of standard functionality is available out of the box. Especially identification/authentication and data driven UI functions.

The idea is that only an actual human can read this image and see the word within. The human is then asked to type this word correctly before they are allowed to enter a comment or create an account, etc.

There were three waves of failure for the Captcha technique.

1. Initially, webmasters did not even distort the work in the picture. Spammers simply used OCR technology to “read” the picture.
2. Next, webmasters used scrambled words like above. Spammers used better OCR.
3. Webmasters improved the scrambling (see below). Spammers enlisted humans to do the OCR (!)

Enlisting humans to do OCR? Basically the image is relayed to users who are either paid money or paid by access to porn to solve the captcha and return the solution which can then be used to gain machine or script access to the target system.

Wikipedia does its usual great job explaining the whole Captcha thing.

Jeff Atwood does it best at Coding Horror: March 04, 2008 — CAPTCHA is Dead, Long Live CAPTCHA! 

Jeff also has a great thread on it in his discussion of his ongoing efforts to manage bad behaviour at his excellent StackOverflow site.

This time I think I have to actually stop and fix it — because I won’t have a new machine for another week or so.

Dang. Forced learning.

Using perfmon it is clear the disk is thrashing. Note the top two counters,  pages/sec (yellow) and Average disk access queue (blue)  are pegged but CPU (green) is mild.

This is with the machine basically idle. I have the usual suite of things like Google Desktop and Tortiose SVN running so I’m not shocked things are accessing the disk but the machine is almost unusable and it was running fine about a month ago. No, I checked, I don’t have a virus. In my life the problem has never been “it’s a virus.” It has always been “I did something stupid.” I guess viruses also fall into that I-did-something-stupid bin …

I’m sure some of you are ahead of the story and saying “has he checked his disk space?” Turns out I have almost 21% free space on a 320 GB drive. Is that a problem?

Yes.

This is a Windows XP system with NTFS on the C: drive. I actually have couple more 320 GB drives on this machine but they’re basically empty. Why are they empty? Dumb reasons. I have a bunch of alpha and beta quality projects going, each of which has all kinds of massive data sets and each of which the developers insist “install it on the C: drive — it doesn’t work quite right in other locations.” Sigh. They are mostly not my developers so I can’t explain (or yell) to them HOW STUPID IT IS TO WELD YOUR APP TO THE C: DRIVE.

But why is an 80% full NTFS partition a problem? When I started this I actually did not know why but I’ve known for years that NTFS disk performance goes to crap once you get north of 70% full or so — but why? I found some of the best information on this topic on this page by Mitch Tulloch at O’Reilly Windows Devcenter.

Based on Mitch’s descriptions, my best theory is that it’s a combination of the Master File Table (MFT) getting fragmented as well as space needed for the pagefile. This hints at two fixes: move the paging file and clear some disk space.

Getting into the disk management applet,

I saw that Dell helpfully left me a 3.11 GB partition that was unused so I formatted it FAT32, declared a 3067 MB page file there, and removed the one on the C: drive.  Note: Ideally the pagefile partition would be on a physically separate disk but I’m working with what I have.

After a reboot I have:

Performance is much better — the machine now thrashes for about 3 minutes after boot and login as opposed to 15+ (!) minutes before moving the pagefile location.

However, I’m at 79% full (down from 82%) on the C: drive so I’m still in serious risk of MFT fragmentation so let’s clean up the disk.

Gina Trapani wrote this very helpful post about using WinDirStat to see what’s using space on your disk.

WinDirStat is a really cool tool!  I’ve learned about (and subsequently forgotten) this tool several times. I have on the order of 1,000,000 files on this system so the graphical tool to help me home in on the disk hogs is really helpful.

After some quality time marvelling at all the cruft I had accumulated on my machine (why did I have two cygwin installations? Why did I have one? ) I moved or deleted  about 60 GB of stuff and got to around 38% free and the machine is running much better now.

The scary thing is that I absolutely “need” the 180 GB in use now. It was only a few years ago that 30 GB drives were ok…

I started learning about “done” when my parents started making me do chores around the house.

Doing dishes: Walk to sink, wash dishes, put in drainer. Done. Am I right? I though I was right.

“Did you empty out the dishpan?” my mom asked.

“No.”

“Well, you’re not done yet.”

Later, I say “now I’m done.”

“Did you wipe down the sink, counters, and stove?”

“That’s part of ‘washing the dishes?’” Even a ten year-old understands the concept of scope creep. The answer was yes but through my push-back I won a change in the name of the task. From then on “Doing the dishes” had to be called “Doing the dishes and cleaning up the kitchen.”

The bottom line: in a team environment the implementer can influence but does not own the determination of when a task is done. Like project scope, doneness is a value negotiated with the customer of the work.

I was recently asked to define “done” in my organization. This is a clear sign that something might be broken. Maybe it just means that there have been recent disagreements about done?

What is Not Done?

I often find that a helpful way to define a positive value is by defining the corresponding negative value.

Not Done is also “it doesn’t work.” To me this is an obvious corollary but I find that this does not occur to some people. The reverse is certainly true.

Not Done is when code “Works on My Machine.” I cringe when I hear “it’s working on my machine” especially since this is typically a response to someone saying “I can’t get it to work” This is ESPECIALLY bad when heard in response to “I can’t get it to build.” When a developer says “it works on my machine” they are really saying:

I do not know what this software depends on.

I do not know what’s involved in moving this software to a clean machine.

I have not checked in all the required code into the source code control system (and I probably don’t know what’s missing).

I do not really care that it does not work on your machine. I’m hoping that’s your problem…

Not Done is when code has not been tested. Untested code does not really work. This is because code that has not been tested invariably has some issues when actually exercised. I consider unit testing the absolute bare minimum level of testing before code can be considered done. An alternative to unit testing? Sure, a full blown test team.

Not done is when code cannot be reliably deployed. This is related to the “it works on my machine” issue. Reliable deployment means the developer has thought about all the steps, interactions and dependencies their code has and has documented them in detail sufficient to allow installation. Maybe the developer even went wild and wrote some install scripts or set up a full featured installer. It could happen.

Documentation

Is documentation required before code can be considered Done? If the code cannot be replicated, tested, or deployed without some written assistance then yes, it cannot be considered done until enough documentation has been written.

I find explaining to a developer that this piece of code is theirs to work on forever and ever until they write enough documentation to hand it off works as a powerful motivator to getting documentation written.

So What does Done mean?

Based on the discussion above I think I’ve outlined my definition of Done:

It has been tested.
It is reproducible and deployable on all supported environments.
It is documented sufficiently.
Most importantly (recalling my experience with the dishes), the customer agrees that it does what they want.

“In addition to our summer and winter estate, he owned a valuable piece of land. True, it was a small piece, but he carried it with him wherever he went.”

From Woody Allen’s Love and Death.

So, what HAVE I been spending my time on? My little piece of the Windows Vista operating system.

For the last 20 months I’ve been building the digital locker assistant (DLA), a dedicated download client that works with Microsoft’s online digital locker, which is in turn part of Microsoft’s Windows Marketplace. Windows Marketplace is where that mysterious “Windows Catalog” link on your Start/Programs menu goes to.

Windows Marketplace supports direct browser based downloading. However, when the download is greater than 1-2 Gigabytes using the DLA is a much better way to go. The most popular use of the DLA so far has been buying and downloading entire copies of Windows Vista and Office 2007.

We were rather skeptical that users would want to download Vista or Office since they are really big downloads. However, earlier in the year the success selling and downloading super large games from Windows Marketplace convinced everyone that downloading Vista would be attractive to consumers. And indeed it has!

You can get the digital locker assistant two ways: If you have Windows XP, go to the Windows Marketplace website, create an account and download and install the MSI. It’s only a 1 meg download.

Or, it’s built into every copy of Windows Vista (except Server versions).

Actually building a part of Windows Vista was a huge effort but it’s really neat to install Vista and see my little piece in there. When I say “my,” it’s more like I’m using the Pluralis Majestatis, the Royal We. I was part of a team and We had LOTS of help.

I was the dev lead for the DLA for XP and Vista. Two very senior Windows developers with me at Vertigo, Chris Idzerda and Ralph Arvesen, rounded out the dev team (that is, they actually did most of the work). Initially, I was dev lead and PM but soon we needed more help with the process and got a full-time program manager, Anne Warren, who was also PM for the Windows Marketplace (WMP) website. The website dev team was some 15 developers and we had a build team of one (that should have been three). Our test team was in India so the dev/test cycle was almost 24/7, something like 24/6 – we’d hand off work in the afternoon and it would be tested all (our) night with a nice bug list waiting for us in the morning.

And then there’s the rest of the Vista team at Microsoft: really a cast of thousands. I think they ALL emailed me at least once. The High DPI functionality team. The Localization team (“do you know your UI looks really bad in Arabic?”), the Group Policy team, Remote Desktop team… you get the picture.

Let’s look at the app

In Vista there are two ways the digital locker assistant (DLA) may be invoked. The primary way is when you buy something at Windows Marketplace and it’s in your digital locker and you click “download.”

You may also browse to the DLA by finding it under the Vista Control Panel.

Then look under Programs or Programs and Features (using Classic view) and find “Manage programs you buy on line.” If you open this link you will invoke the DLA and if you have never sync’d up with your online digital locker you will see this:

If you have software in your online digital locker you can see it listed here by clicking on “Sign in if you already have a digital locker account.” Digital locker accounts are Windows LiveID (a.k.a. Passport) accounts mapped to a Windows Marketplace account. You’ll get a login prompt:

and after synchronizing with your online digital locker you’ll see all your purchased, free, and trial items listed. In my case (below) I clearly have a bunch of games in my locker. These were just to test downloading large items. Right.

Technology under the covers

The DLA is built in Win32/C++ as an ATL Windows application but we get some goodies from WTL as well. For those going “huh?” look at my post ATL and WTL resources.

At this point most people are asking me: why C++? Why not .NET and/or WPF? Or, if you’re using C++, why not MFC?

The DLA started (and is still available) as a downloadable application for XP. Our target users are what Alan Cooper would call “permanent beginners” (like that relative that always calls you for tech support…) — with a modem.

This means making the download as small as possible. Vertigo is a premier .NET shop but we could not use .NET because the 22 MB .NET runtime install kills us (that .NET never made it into the XP Service Packs… argh). Fortunately, we happen to have a few developers around (i.e., old geezers) who can do C++. We used ATL again to keep the size of the executable small.

In hindsight, it was just as well that the XP effort started in C++. Once we expanded the project to include being built into Vista we found that, in Windows System programming and the Vista source tree, C++ is expected and still king (See my post Has Microsoft flipped the Bozo bit on .NET? for a full discussion).

This meant that we could develop one source code base and, with some care, make it build in the Windows OS build system for Vista and VS2005 for XP.

Single source is nice but why not make a single binary that runs on Vista and XP? Sigh. We do — sort of, but it’s complicated.  From a programmer’s perspective, Vista makes one dramatic change from traditional Win32 applications and that’s in how localized resources are loaded.

To handle localization traditional Windows practice is to create an RC file for all resources (dialogs, images, sounds, strings, keyboard shortcuts, etc.) which are compiled into the resource DLL. Localization teams produce localized RC files based on your master RC file and these are all built into a suite of resource DLLs. At run time the application loads the appropriate resource DLL based on logic you have to write which looks at the calling thread’s locale settings.

Internal to the application is a language-neutral block of resources (typically English-US based) and if an appropriate external resource DLL cannot be found for the current locale settings, this internal block is used instead. This is known as “fallback” behavior.

Here’s the new twist in Vista: in Vista the OS loader (not the app) picks the resource DLL and locates it in memory where the app thinks its internal fallback resources are. This is expected behavior and currently only appears to work for a native Vista-built application so our “legacy” resource loading technique as used in XP was not acceptable to those who guard the Windows Source tree. Did I mention all the code reviews? Making Vista-style resource loading work in XP, while theoretically possible, was a task we did not choose to take on. So we ended up with one set of source code feeding two build processes; one for XP and one for Vista. Through careful coding there are remarkably few “if Vista do this, if XP do that” points in the code. 

While we currently block running the XP installer on Vista (in theory blocking installation of the XP DLA on Vista), it turns out that the XP DLA runs fine on Vista. I should not be suprised by this becuase we did quite a bit of casual sanity testing on this but it was not initially part of the test matrix. We found out by somewhat by accident as users were upgrading their XP machines (where the user had added the XP DLA) to Vista and then running the XP DLA.

For our downloading mechanism we hand off all download jobs to Microsoft BITS (Background Intelligent Transfer Service). While BITS works well for us I still think Micorsoft is tempting the gods by including “Intelligent” in their product name. BITS is the guts behind hwo Microsoft Updates are downloaded. I’ve also discovered that Google Updater uses BITS as well. What we gained by using BITS was automatic download management including background downloads, downloads that persist when our application is not running, downloads that seamlessly restart when the machine is rebooted, and lots of error handling algorthms that we did not have to write or maintain. I’d use BITS again if needed. We did have to build a simple HTTP download as well because some modem-based accelerators do not play nice with BITS.

Overall it was a great experience. While it was sometimes chaotic and exhausting, it was a lot of fun too. 

I’d do it again.

Really.

After I’ve had a couple years to rest.