NTFS tricks and the full c: disk

Posted in Campfire Stories, Windows Details on March 9th, 2009

In the last week or so I’ve hit a point on my workstation where booting and running everyday apps has just slowed to an unbearable crawl.  This happened to me before but I switch machines enough that I usually avoid the issue by, well, switching machines.

This time I think I have to actually stop and fix it — because I won’t have a new machine for another week or so.

Dang. Forced learning.

DiskThrash

Using perfmon it is clear the disk is thrashing. Note the top two counters,  pages/sec (yellow) and Average disk access queue (blue)  are pegged but CPU (green) is mild.

This is with the machine basically idle. I have the usual suite of things like Google Desktop and Tortiose SVN running so I’m not shocked things are accessing the disk but the machine is almost unusable and it was running fine about a month ago. No, I checked, I don’t have a virus. In my life the problem has never been “it’s a virus.” It has always been “I did something stupid.” I guess viruses also fall into that I-did-something-stupid bin …

I’m sure some of you are ahead of the story and saying “has he checked his disk space?” Turns out I have almost 21% free space on a 320 GB drive. Is that a problem?

Yes.

This is a Windows XP system with NTFS on the C: drive. I actually have couple more 320 GB drives on this machine but they’re basically empty. Why are they empty? Dumb reasons. I have a bunch of alpha and beta quality projects going, each of which has all kinds of massive data sets and each of which the developers insist “install it on the C: drive — it doesn’t work quite right in other locations.” Sigh. They are mostly not my developers so I can’t explain (or yell) to them HOW STUPID IT IS TO WELD YOUR APP TO THE C: DRIVE.

But why is an 80% full NTFS partition a problem? When I started this I actually did not know why but I’ve known for years that NTFS disk performance goes to crap once you get north of 70% full or so — but why? I found some of the best information on this topic on this page by Mitch Tulloch at O’Reilly Windows Devcenter.

Based on Mitch’s descriptions, my best theory is that it’s a combination of the Master File Table (MFT) getting fragmented as well as space needed for the pagefile. This hints at two fixes: move the paging file and clear some disk space.

Getting into the disk management applet,

Diskmanager

I saw that Dell helpfully left me a 3.11 GB partition that was unused so I formatted it FAT32, declared a 3067 MB page file there, and removed the one on the C: drive.  Note: Ideally the pagefile partition would be on a physically separate disk but I’m working with what I have.

After a reboot I have:

Disks

Performance is much better — the machine now thrashes for about 3 minutes after boot and login as opposed to 15+ (!) minutes before moving the pagefile location.

However, I’m at 79% full (down from 82%) on the C: drive so I’m still in serious risk of MFT fragmentation so let’s clean up the disk.

Gina Trapani wrote this very helpful post about using WinDirStat to see what’s using space on your disk.

WinDirStat is a really cool tool!  I’ve learned about (and subsequently forgotten) this tool several times. I have on the order of 1,000,000 files on this system so the graphical tool to help me home in on the disk hogs is really helpful.

windirstat

After some quality time marvelling at all the cruft I had accumulated on my machine (why did I have two cygwin installations? Why did I have one? ) I moved or deleted  about 60 GB of stuff and got to around 38% free and the machine is running much better now.

The scary thing is that I absolutely “need” the 180 GB in use now. It was only a few years ago that 30 GB drives were ok…

Hoocoodanode? What’s that mean?

Posted in Blogs on blogs, Uncategorized on December 26th, 2008

I’ve been reading a lot of blogs about the current financial meltdown and keep seeing “Hoocoodanode?” all over the place.  I finally had a “D’oh!” moment.

Define: Hoocoodanode? = “hoo-cood-ah-node?”

= “Who Could ‘a knowed?” = Who could have known?

Ok, I’m an idiot so I had to write this.

Oh yeah, I’m a Nerd!

Posted in Blogs on blogs on June 27th, 2007


I am nerdier than 99% of all people. Are you a nerd? Click here to find out!

Am I a Nerd?

Posted in Blogs on blogs on June 27th, 2007

Mingle2 Free Online Dating - Science Quiz

It’s Done™ (a.k.a. it Works™)

Posted in Campfire Stories, The Art of Programming on June 10th, 2007

Have you ever had a disagreement about when something is actually done (as opposed to “not done yet”)?

I started learning about “done” when my parents started making me do chores around the house.

Doing dishes: Walk to sink, wash dishes, put in drainer. Done. Am I right? I though I was right.

“Did you empty out the dishpan?” my mom asked.

“No.”

“Well, you’re not done yet.”

Later, I say “now I’m done.”

“Did you wipe down the sink, counters, and stove?”

“That’s part of ‘washing the dishes?’” Even a ten year-old understands the concept of scope creep. The answer was yes but through my push-back I won a change in the name of the task. From then on “Doing the dishes” had to be called “Doing the dishes and cleaning up the kitchen.”

The bottom line: in a team environment the implementer can influence but does not own the determination of when a task is done. Like project scope, doneness is a value negotiated with the customer of the work.

I was recently asked to define “done” in my organization. This is a clear sign that something might be broken. Maybe it just means that there have been recent disagreements about done?

What is Not Done?

I often find that a helpful way to define a positive value is by defining the corresponding negative value.

Not Done is also “it doesn’t work.” To me this is an obvious corollary but I find that this does not occur to some people. The reverse is certainly true.

Not Done is when code “Works on My Machine.” I cringe when I hear “it’s working on my machine” especially since this is typically a response to someone saying “I can’t get it to work” This is ESPECIALLY bad when heard in response to “I can’t get it to build.” When a developer says “it works on my machine” they are really saying:

I do not know what this software depends on.

I do not know what’s involved in moving this software to a clean machine.

I have not checked in all the required code into the source code control system (and I probably don’t know what’s missing).

I do not really care that it does not work on your machine. I’m hoping that’s your problem…

Not Done is when code has not been tested. Untested code does not really work. This is because code that has not been tested invariably has some issues when actually exercised. I consider unit testing the absolute bare minimum level of testing before code can be considered done. An alternative to unit testing? Sure, a full blown test team.

Not done is when code cannot be reliably deployed. This is related to the “it works on my machine” issue. Reliable deployment means the developer has thought about all the steps, interactions and dependencies their code has and has documented them in detail sufficient to allow installation. Maybe the developer even went wild and wrote some install scripts or set up a full featured installer. It could happen.

Documentation

Is documentation required before code can be considered Done? If the code cannot be replicated, tested, or deployed without some written assistance then yes, it cannot be considered done until enough documentation has been written.

I find explaining to a developer that this piece of code is theirs to work on forever and ever until they write enough documentation to hand it off works as a powerful motivator to getting documentation written.

So What does Done mean?

Based on the discussion above I think I’ve outlined my definition of Done:

It has been tested.
It is reproducible and deployable on all supported environments.
It is documented sufficiently.
Most importantly (recalling my experience with the dishes), the customer agrees that it does what they want.

A simple WPF demo and architecture discussion

Posted in Windows Details on April 24th, 2007

This post is based on a short demo/talk I gave at Vertigo recently.

I have two main goals in this discussion:

  • How VS2005 handles WPF projects/apps (showing some Blend / VS2005 gotchas)
  • A way to look at the role of XAML in WPF architecture

Petzold books

Charles Petzold’s books have always been a huge help to me in getting all the nitty gritty details in Windows UI technology. I started my Chuck P. habit way back with his Programming Windows book. The Book for Win32 Windows programming. Ok, I do have other books too but this is the one I’d want if I was lost on a desert island alone with a c-compiler…

More recently, Petzold has written authoritative .NET WinForms books and I started to notice a trend. Especially in Programming Microsoft Windows Forms, his .NET 2.0 WinForms book, Petzold really pushes a code-everything-yourself approach as opposed to using any of Visual Studio’s “helpful” tools. His point: if you do not understand the code the various visual designer/code generators in VS create, you really do not know what is going on.

I see this as just another statement of the well-known Weasley’s Law:

Never trust anything that can think for itself

if you can’t see where it keeps its brain!

Arthur Weasley — Harry Potter and The Chamber of Secrets

I’m going to focus this very small discussion on how XAML fits in to the WPF dev process and WPF architecture and point out a few gotchas in the current WPF dev tool set.

The WPF dev process

I’m going to step through building a baby application modeled closely on the sample application Microsoft provides at MSDN in Get Started Using Windows Presentation Foundation but I’m going to walk through it highlighting aspects Petzold points out in his new WPF book Applications = Code + Markup.

Prerequisites

If you want to build and execute the demo code you will need a working WPF dev environment which requires the following: the Microsoft .NET Framework version 3.0 and the Windows Software Development Kit (SDK), Visual Studio 2005 (C# Express ok), Visual Studio 2005 extensions for .NET Framework 3.0 (WCF & WPF, November 2006 CTP or newer), and Microsoft Expression Blend (RC Version or greater).

First let’s play with a trivially simple WPF project. Source code here.

Step 1

We start Start with an empty WPF project in VS2005 which I’ve named WPFDemo.

In Solution Explorer, click the Show All Files button.

Select the files: (all the code files in the Step 1 folder)

App.xaml

ExpenseReportPage.xaml

ExpenseReportPage.xaml.cs

HomePage.xaml

HomePage.xaml.cs

Right-click on them and Include In Project.

Look at App.xaml:

    1 <Application

    2   xmlns=http://schemas.microsoft.com/winfx/2006/xaml/presentation

    3   StartupUri=HomePage.xaml>

    4 </Application>

App.xaml declares the application starting point. So let’s try to build. Assuming you followed the steps exactly you will see the following build error:

WPFDemo.exe does not contain a static ‘Main’ method suitable for an entry point

This is because by default when App.xaml was included by Visual Studio its Build Action was set to Page.

Select App.xaml and change its Build Action to Application Definition. Confirm that the application now builds and runs correctly.

Note: App.xaml is the most common name for this file but there’s nothing special about the name App.xaml. What’s important is its content and that it is configured as the Application Definition. Since the error message we got above implies this configuration has something to do with the Main() method it should also make sense that an application can only have one file configured as the application Definition.

In the steps above I’ve enacted discussion that may be found on pp. 479-481 of Petzold’s Applications = Code + Markup.

Where’s Main()?

After a quick examination of all the files we included in the project so far you should be asking about Main() – like, where is it? App.xaml defines the entry point to the application and if you dig into the generated files for the application you will find an App.g.cs file in the obj\Debug folder:

Examining this file we find a Main():

   33     /// <summary>

   34     /// GeneratedApplication

   35     /// </summary>

   36     public partial class GeneratedApplication : System.Windows.Application {

   37 

   38         /// <summary>

   39         /// InitializeComponent

   40         /// </summary>

   41         [System.Diagnostics.DebuggerNonUserCodeAttribute()]

   42         public void InitializeComponent() {

   43 

   44             #line 3 “..\..\App.xaml”

   45             this.StartupUri = new System.Uri(“HomePage.xaml”, System.UriKind.Relative);

   46 

   47             #line default

   48             #line hidden

   49         }

   50 

   51         /// <summary>

   52         /// Application Entry Point.

   53         /// </summary>

   54         [System.STAThreadAttribute()]

   55         [System.Diagnostics.DebuggerNonUserCodeAttribute()]

   56         public static void Main() {

   57             XamlGeneratedNamespace.GeneratedApplication app = new XamlGeneratedNamespace.GeneratedApplication();

   58             app.InitializeComponent();

   59             app.Run();

   60         }

   61     }

The #line 3 directive causes any compile error occurring here to be directed to the file indicated: back to the XAML file. The #line default, #line hidden directives restore error indication back to normal. These directives also affect debug step-through.

The key takeaways from this example are:

  • VS2005 creates all kinds of generated files you should be aware of.
  • Even a “pure XAML” app involves additional generated code and files.
  • VS2005 patched for WTF is not (yet) fully XAML aware.

Obviously it would be great if we could wait for the next release of Visual Studio codenamed “Orcas” (Here’s a great site by Scott Guthrie all about Orcas). But here at Vertigo we’re delivering WTF applications today so we need to be aware of the issues – and build anyway.

I’ll jump ahead by simply pasting in more UI content to the application and demonstrate a typical but very simple Blend/VS2005 development scenario. Source code here.

Step 2

Without making any changes to the structure of the app but adding UI code we get:

The idea is that you select a name on the list and click the button to view that person’s expense report. If you’re curious: the data displayed is hard coded as XML structured data in the HomePage.xaml file.

The first thing to do is to name the Button element in the HomePage.xaml file. To me it’s interesting that we already built and ran this app and yet nothing squawked about the button being an un-named element.

Being a Visual Studio-centric kind of guy I naively click on the button in the Design view and look at the control properties.

First I look for the “Name” property. Huh, not there.

Ok, next I look for the “ID” property. Double-Huh, that’s not there either?

To name the button you need to go to the XAML code and in the tag <Button add Name=btnView. Note that there is Intellisense support within the XAML code an the Button tag.

Now I want to wire the Click event.

Can I access control events in the Properties panel? No.

Can I double-click on the button in the design view to add a click handler? No.

However, I can add a reference to the “Click” property in the Button tag but this does not automagically wire up an event handler stub.

What I can do is this: rebuild the app (not just Build but Rebuild) so that the Name I gave the button is in-scope for Intellisense and then add the handler in the XAML file’s associated .cs file.

In HomePage.xaml.cs, I add the event handler by starting to type “btn…” whereupon Intellisense shows me the btnView and I add the event handler and let VS2005 create an event handler stub.

All looks ok – and utterly familiar. I then edit the handler to call goToExpenseReportPage() which I have already prepared and the application is finished.

Now let’s add a twist. Go to the event handler and select the btnView_Click name and select Refactor / Rename and rename the event handler any other random name. I used ShowExpenseReport. Rebuild the code and confirm the application still works correctly.

Mixing it up with Blend

Now open the same solution in Blend.

Select the button in the design view and select the events list in the properties panel.

Note: from Blend’s perspective the button’s click event is not wired up. This is because Blend manipulates the application by only looking at a project’s XAML files, not its .cs files. Therefore, it is excruciatingly easy at this point to go ahead and wire the click event up AGAIN.

Just to make a point, let’s do just that. let’s wire up a second event handler to the button.

In Blend double-clicking in the click event box shown above wires up the event and creates the event handler stub in HomePage.xaml.cs. This sequence should open and/or foreground VS2005 to the created stub. We can add a call to goToExpenseReportPage() and the code builds fine but may, or may not, run fine. On some machines the second event handler fails when called because navigation has already moved to the second page, disposing of objects needed by the handler code so you get null reference errors.

If we examine HomePage.xaml.cs we note that it still only shows the first event handler wired that we created with VS and does not show the connection to the second Blend created handler.

If we examine the XAML file we find the button tag has a reference to the Blend created handler (Click=btnView_Click/>) but no reference to the VS created handler. T add more confusion, the VS takes the Blend created event handler reference and generates the expected hookup in the generated file obj\Debug\HomePage.g.cs:

#line 81 “..\..\HomePage.xaml”

this.btnView.Click += new System.Windows.RoutedEventHandler(this.btnView_Click);

Again the #line directive redirects debug and error output.

Given that we’re building our apps with a combination of Visual Studio and Blend what should we do?

Blend/VS2005 recommendations:

  1. Use Blend to wire the events and create stubs. This means the event will be seen in the element’s tag in the XAML.
  2. Use the default name for the event handler.

Number 1 implies that a typical WPF app will have events and handlers referenced to controls in the XAML file, the handlers in the code-behind file, and actual code wire-ups of the events in the generated .cs files (the code-behind-behind files).

If you look purely at the code behind file (XAML.cs above) you are going to see the handlers with nothing more than their names to hint which control they bear on.

This is why I say (Number 2) renaming event handlers away from the default [control name]_[event name] pattern is a bad idea.

Why emphasize and favor using Blend? Currently Blend behaves in design mode the way we wish Visual Studio would. I’m sure this will all have to be revisited once Orcas ships but for now this is the approach I recommend.

XAML in WPF

At the beginning I made some claims that I would discuss XAML’s role from a more architectural point of view.

In architecture docs at MSDN I found two statements that seem to bear on this:

If you work the first half of Petzold’s Applications = Code + Markup, that is, the non-XAML part, you find that making a WPF is properties, properties, properties. Set the 85 bagillion properties of a Window, of a button, etc.

But look at it this way: properties are data. Data can be easily described by XML. Enter XAML. Once you have all the properties in your applications held in XAML it becomes really easy for any number of tools to traverse and modify this data.

We just demonstrated this by using Blend and Visual Studio to act on the same XAML files. The process is not perfect yet but it is a powerful technique I think we will see more and more often.