Having been a newbie software developer (and I’m still fairly “new”), I did not understood why software developers in our company are quitting. Having been a fresh graduate, I was, then, still amazed by the fact that there are people out there in the real world that will pay me real world money just to write code – a hobby that I will do for free. I was puzzled. We are working in an environment that fosters creativity and innovation, a company that strictly subscribes to the “programmers’ bill of rights”, and a company that scores high in the Joel Test (11/12).  Those, plus the fact that we can surf Facebook anytime and drink all the brewed coffee in the pantry, made me believe that this is the best company to work for locally; it made me believe that anyone leaving our company is insane.

Just a few months after starting my job (I had my internship there for 6 months), one of the bestest programmer I know (he’s actually a “software architect”, but hates being called one), my mentor when I was still an intern, filed his letter of resignation. This guy is one of the most passionate programmer I know. He gets it. Paul Graham classifies him as a hacker and he actually worked for Microsoft (in Redmond, not the local whatever-they-are-doing-there branch).

For some, asking people why they quit their job is taboo; I actually enjoy having those type of conversation. Gives me insight. So I asked the fallen hacker, “Why did you quit?”. He pointed to the picture of his new born baby (his desktop wallpaper) and said with an enthusiastic smile “He is my new priority”. He then explained to me, in a fashion that I would understand, why working overseas means “better opportunity” for his family. Being a bit unconvinced with his explanation, I asked him “Is that all? Better opportunity?” and he replied back with a playful smile, “In my new job, I will be able to code again. I will no longer be a ‘Word Programmer’”. He has always joked about being a Word Programmer – programming in Microsoft Word using narrative texts (use cases, scenarios, etc) and UML diagrams; programming by story telling. He was indeed a hacker.

Having spent a year and a half on this job, I can say that I now have a vague idea on why programmers (or I.T people, in general) quit. Even though I’m only a year-and-half year old in corporate servitude, I had been handled by four managers and had been transferred to four teams and that gave me great insights on the crucial dynamics that affects employee turnovers. In my year-and-half stay, I have seen three managers, a project manager, two architects, four senior developers and a handful of intermediate developers go. Just a few months ago, four of my teammates left. The statistics above is a little off; it is more than 20%.

So why do programmers quit?

There are a lot of models that scientifically explains turnover, but most of them does not really cover the programmers psyche because most of them are written three decades ago. These models are not exactly wrong; they just don’t capture how programmers think and react.

So let me present to you my model. Let’s call it the Programmer Turnover Model. There are only two controlling variables on this model: Opportunities and Grievances. When these two factors achieve “equilibrium” (fancy word eh?), the employee will finally pull the trigger and call it quits. Seldom do I see or hear that a programmer quits just by one of these controlling variables; it is usually a combination of both. But it is not unheard of that an employee quits just because of opportunities or grievances: If you’re boss is verbally abusing (Grievances) you everyday, that is enough reason to quit.

Opportunities

Opportunities are the voice inside us that always yells “The grass is greener on the other side”. It is the greedy side of the human emotion. It is the human emotion that seeks larger paychecks and better benefits.

Programmers have all the incentives in the world to quit. In fact, if you want to suddenly increase your salary, all you have to do is quit your job and find another one. A 25% increase in salary between job hops is typical. In fact, most local programmers do it. You don’t even need to be highly competent to do this; you just need to impress the interviewers and pass the interview. In an industry that caters to outsourced work (typically boring business systems) and software maintenance jobs, the bar that is being set is not that high (not “Google high”). Fact of the matter is, most grown-ups (including my mom) advice I.T. workers to just “get a few years of experience and switch jobs”.

Local industry culture believes that the sooner you get overseas, the better it is for you. Those programmers that work or have worked overseas are perceived to be “successful”. And what better way to increase your chance working abroad than to beef-up your resume with “experiences”? Not only that you get larger paychecks and better benefits when hopping jobs, you also get a set of fresh credentials to put into your resume. When you work for a single company, you’re stuck with their technology. And in an industry where your skills are obsolete within five years (max), being stuck in a technology is a really really bad thing for you.

It seems that the game is rigged for the employers. Employers are nearly powerless in stopping the greedy side of human emotion. Aside for paying large sums of cash to these spoiled programmers, they incur the risk of losing expensive human capital. All hope is not lost, the human emotion provides some form of protection against job hoppers. The loving side of the human emotion.

A few moments ago, I made it sound that it is so easy to job hop. Well, it is for soulless programmers (and about half of programmers are). But programmers are human too and they possess some of that soft, caring, vulnerable human emotion. People just don’t quit their job out of greediness. They are attached to their officemates, they been used to the smell of that particular brand of brewed coffee in the morning, they love how their $5 dollar computer chair squeaks when they lean back while waiting for their programs to compile, they love those once-a-month company parties, they know their way into their programs and they’re quite competent in it, they love to see their crush on the opposite cubicle, they are considered the domain expert inside the department. Humans resist change because attachments are hard to break.

It is painful to leave a company and that is what stops most people to quit their job even if they know that there are better opportunities out there.

Grievances

Grievances are the devil’s advocate that whispers “Stop taking this sh*t and quit”. Opportunities provide all the incentives in the world to switch jobs and although the touchy human emotion tells us to stay because it is painful to leave, grievances makes it justifiable and easy to quit.

Grievances builds up over time that’s why it’s hard to avoid it. It stacks up high. Each person have a certain threshold on how high they can keep their grievances stack before it falls down and crumble (translation: Each person can only take so much bullshit).

There are many forms of grievances, but all of them contributes to the metaphorical grievances stack:

• My company is still coding in Cobol when there are better technologies available.
• We don’t have source control.
• My boss is a back-seat driver. His hobbies are micromanagement and micromanagement.
• The elevator takes forever.
• The air conditioner is too cold.
• The code base is so poorly written that it breaks in 27 different places when you add a single line of code. The previous developer maintaining the code base is a literal arts major.
• We are responsible for testing our own code.
• My teammates are bozos.
• I hate my project. It’s not fun and exciting. It’s grunt work.
• I have written just 150 lines of code in the last 2 years.

Grievances have many forms but grievances are simply things that gets listed down in the “I do not want” column of your employees checklist.

The good thing about grievances is that managers and employers can manage and control them. Unlike Opportunities where employers are powerless, Grievances can be managed and controlled. It’s not easy but it’s doable. And the first thing that needs to happen is communication.

If you’re an employer/manager working with programmers, understand that your programmers have all the incentives in the world to leave you for better opportunities. Do not make it easy for them by giving them reasons to quit.

The Price of Turnover in Software Development

In his influential book, “The Mythical Man-Month”, Fred Brooks emphasized the importance of Conceptual Integrity in software design and architecture:

Most European cathedrals show differences in plan or architectural style between parts built in different generations by different builders. The later builders were tempted to “improve” upon the designs of the earlier ones, to reflect both changes in fashion and differences in individual taste. So the peaceful Norman transept abuts and contradicts the soaring Gothic nave, and the result proclaims the pridefulness of the builders as much as the glory of God.

Against these, the architectural unity of Reims stands in glorious contrast. The joy that stirs the beholder comes as much from the integrity of the design as from any particular excellences. As the guidebook tells, this integrity was achieved by the self-abnegation of eight generations of builders, each of whom sacrificed some of his ideas so that the whole might be of pure design. The result proclaims not only the glory of God, but also His power to salvage fallen men from their pride.

Even though they have not taken centuries to build, most programming systems reflect conceptual disunity far worse than that of cathedrals. Usually this arises not from a serial succession of master designers, but from the separation of design into many tasks done by many men.

I will contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas.

The conceptual integrity of software is a measure of how well software conforms to a single, simple set of design principles. When done properly, it provides the most functionality using the simplest idioms. It makes software easier to use by making it simple to create and learn.

Conceptual integrity is achieved when the software’s design proceeds from a small number of agreeing individuals. For software to maintain conceptual integrity, the design must be controlled by a single, small group.

When turnover is high, the conceptual integrity of your product is being sacrificed. And because conceptual integrity is the most important consideration in system design, having bad conceptual integrity will make your product hard to use. This in turn will affect your sales and eventually your bottom line.

Not only will the loss of conceptual integrity of your product negatively affects your sale, it will also affect the morale of your best programmers. High turnover rates induces the Software Peter Principle:

The software Peter principle is used in software engineering to describe a dying project which has little by little become too complex to be understood even by its own developers.

It is well known in the industry as a silent killer of projects, and by the time the symptoms arise it is often too late to do anything about it. Good managers can avoid this disaster by establishing clear coding practices where unnecessarily complicated code and design is avoided.

When your project becomes too complex to be understood by your best programmers, they will be naturally be forced to rewrite it. They will go into their manager’s office and ask for a re-write, the manager’s will then disagree because there are no “business reasons” to do so (and maybe they are right because “rewriting is probably the ‘single worst strategic mistake that any software company can make’”), so you’re best programmers go back to their cubicle grumpy and dives into the unmaintainable piece-of-crap that is your codebase. Soon, they quit (because there are better opportunities and you gave them enough reason to leave you).

The managers then instruct their H.R. people to hire a new software developer to maintain the unmaintainable codebase. But since the programmer is new, and inexperienced,  he makes implementation choices that work but have unintended negative consequences. Further degrading the conceptual integrity of your product; making it increasingly difficult to understand. It’s a death spiral.

If there is one thing that the best programmers hate other than maintaining crappy software, it is working with incompetent programmers. And because you have a crappy codebase and you need software “maintainers” to maintain your codebase, you hire cheap, incompetent programmers and that will send you down further in the death spiral. Paul Graham puts it nicely in his essay “What Happened to Yahoo!”:

Microsoft (back in the day), Google, and Facebook have all been obsessed with hiring the best programmers. Yahoo wasn’t. They preferred good programmers to bad ones, but they didn’t have the kind of single-minded, almost obnoxiously elitist focus on hiring the smartest people that the big winners have had.

Why would great programmers want to work for a company that didn’t have a hacker-centric culture, as long as there were others that did? I can imagine two reasons: if they were paid a huge amount, or if the domain was interesting and none of the companies in it were hacker-centric. Otherwise you can’t attract good programmers to work in a suit-centric culture. And without good programmers you won’t get good software, no matter how many people you put on a task, or how many procedures you establish to ensure “quality.”

In technology, once you have bad programmers, you’re doomed. I can’t think of an instance where a company has sunk into technical mediocrity and recovered. Good programmers want to work with other good programmers. So once the quality of programmers at your company starts to drop, you enter a death spiral from which there is no recovery.

Turnover reduces the conceptual integrity of your product. When there is no conceptual integrity, your product becomes hard to use and hard to maintain for programmers. The programmers maintaining the products hate the code so much that they quit. You hire new inexperienced programmers that further reduces the conceptual integrity of your product. You enter into a death spiral and you can no longer attract the best programmers because your code stinks. You enter another death spiral from which there is no recovery. That is the price of high turnover in software development: You lose your best talents then you lose your product and eventually your company.

How NOT to Retain your Best Software Developers and Programmers

One of the most popular form of “employee retention” is Training Bonds. Although it is not legally enforceable (or if it is but the company won’t really run after you because it is expensive to sue), most bright programmers will never sign a training bond. For most hackers, a training bond is a sign that a company is being ran by “suits”. Joel Spolsky attributes training bonds to “blind loyalty”:

I think that companies need to keep their employees loyal by treating them well, not by enforcing blind loyalty though a contract. I am not going to make the mistake of signing this clause again. Fortunately, with today’s shortage of qualified programmers, the balance of power today is sharply in your favor when you start a new job, and I think you have a very high chance of getting a job without signing restrictive employment clauses.

Another popular form of employee retention mechanism is counter-offer. Counter-offer is when your current employer gives you an offer (counter-offer) that is larger than the offer you received from the company you intend to jump to. Counter-offer is a big no-no because it sends the following message to your employee:

We have not really paid you the amount you deserve. But since we really need you to stay, we are willing to pay you more. We don’t think that you deserve this, but our hands are tied.

It is much worse when the news of a counter-offer leaks to your other employees. They will also feel that they are not being paid the amount they deserve, they will feel cheated and eventually quit. More turnovers, yay! Always remember the bribery never promotes loyalty.

If you think bribing your programmers when they accomplish something extraordinary will increase their loyalty: you are wrong. A lot of books have been written on the subject of incentive pay and why it is considered harmful:

… “incentives (or bribes) simply can’t work in the workplace”. DeMarco and Lister go further, stating unequivocally that any kind of workplace competition, any scheme of rewards and punishments, and even the old fashion trick of “catching people doing something right and rewarding them,” all do more harm than good. Giving somebody positive reinforcement (such as stupid company ceremonies where people get plaques) implies that they only did it for the lucite plaque; it implies that they are not independent enough to work unless they are going to get a cookie; and it’s insulting and demeaning.

I know that the title says “How to Retain your Best Software Developers and Programmers”, but I was only able to give the “How NOT to do it” guide. It took me a year-and-a-half to understand what motivates programmers to quit their job. It may take me another year-and-a-half to understand how to motivate them to NOT quit their jobs. You may wait for it, or you can just read Spolsky’s blog for a complete guide on how to manage and retain the best programmers that you will ever find.

Larry Wall haz dem

Larry Wall, the creator of the infamous Perl programming language, has once said that the greatest virtues of a good programmer are Laziness, Impatience and Hubris. To not mislead the lazy “computer science” students, allow me to define each virtues:

1. Laziness — The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful, and document what you wrote so you don’t have to answer so many questions about it. Hence, the first great virtue of a programmer. See also impatience and hubris.
2. Impatience – The anger you feel when the computer is being lazy. This makes you write programs that don’t just react to your needs, but actually anticipate them. Or at least pretend to. Hence, the second great virtue of a programmer. See also laziness and hubris.
3. Hubris – Excessive pride. Also the quality that makes you write (and maintain) programs that other people won’t want to say bad things about. Hence, the third great virtue of a programmer. See also laziness and impatience.

A few days ago, a tester at our company filled a track that contained multiple attachments (Screenshots, Error Logs, etc) to assist the developers on their investigation. As we are using Microsoft’s Team Foundation System for source control and bug tracking, the most obvious way to get these attachments is through TFS Web Access; Frankly speaking, this is the only “official” way to access the track (bug report) on our company.

The TFS Web Access homepage UI

I don’t know if its a lack of feature or what, but there is no way in TFS Web Access to download multiple attachments at once. Additionally, when downloading attachments that contains a space on their filename, TFS Web Access will automatically concatenate the filename and that will remove the file’s extension (EG: Screenshot of the bug.jpg will be renamed to Screenshot)… You still need to manually rename the file with the proper extension to open it. And because I’m lazy, I don’t want to log-in to TFS Web Access, type in the track ID, click on the attachments tab, click on the file(s) I want to download, browse for the location where to save the file, create a new folder for the track, click save, minimize my browser, open the download location and (unzip the attachment, it its zipped, which normally is) to just view a single screenshot.

Since I am a big believer of Wall’s three virtues, I created a tool that will automate those boring and repetitive tasks for me (and allow me to download *all* attachments from a work item + workaround the “spacing” bug). The tool is called WIF or Work Item Fetcher and it is currently in Closed-Limited Beta.

At first, I thought about parsing the web pages to download the attachments using HTML Sanitizers and Regular Expressions but then I remembered that only Chuck Norris can parse HTML using regular expressions! So I searched the web and found out about the TFS SDK! Go .NET! Go Laziness! Okay, enough chit-chat, the rest of the post will be dedicated to discussing how to use the TFS API to download attachments from TFS.

Step 1: Acquiring the needed libraries

To be honest, I don’t really know where to download the DLLs. I’ve searched and found nothing. What I did was I referenced the DLLs located at C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies\. The DLLs should be available after you download and install Team Explorer for Visual Studio 2005. For our purposes, you should reference the following DLLs:

• Microsoft.TeamFoundation.DLL
• Microsoft.TeamFoundation.Client.DLL
• Microsoft.TeamFoundation.WorkItemTracking.Client.DLL

You must then add the needed declarations like so:

using Microsoft.TeamFoundation;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.WorkItemTracking.Client;

Compile your project to check. Remember to always compile your project after each step.

Step 2: Connecting to your TFS server

If you want to do anything with your TFS SDK, the first thing that you should do is to connect to your TFS server. Connecting is straight-forward, but a little tricky if you need to provide your network credentials. First, lets assume that the program will run on a machine that is connected to the domain with the proper credentials (AKA straight-forward way):

const string TFSServer = "http://mytfsserver:8090";
 
using (TeamFoundationServer tfsConn = new TeamFoundationServer(TFSServer))
{
	try
	{
		tfsConn.EnsureAuthenticated();
	}
	catch (Exception ex)
	{
		MessageBox.Show(ex.Message, "Cannot connect to TFS Server!", MessageBoxButtons.OK, MessageBoxIcon.Error);
	}
}

A call to EnsureAuthenticated(); is needed to check if your connection is properly authenticated. If not, the method will throw an exception.

How about if you need to manually provide login credentials or your machine is not joined to a domain? Just use the NetCredential object from System.Net to pass your credentials:

const string TFSServer = "http://mytfsserver:8090";
 
NetworkCredential netCred = new NetworkCredential("Username", "Password", "Domain");
 
using (TeamFoundationServer tfsConn = new TeamFoundationServer(TFSServer, netCred))
{
	try
	{
		tfsConn.EnsureAuthenticated();
	}
	catch (Exception ex)
	{
		MessageBox.Show(ex.Message, "Cannot connect to TFS Server!", MessageBoxButtons.OK, MessageBoxIcon.Error);
	}
}

Step 3: Acquire the Work Item

Now that we’re good and connected to TFS, our next task is to acquire the Work Item from TFS. First, we’ll need to acquire the WorkItemStore service from TFS by calling the GetService method:

WorkItemStore wis = (WorkItemStore)tfsServer.GetService(typeof(WorkItemStore));

Then, we’ll get the Work Item by invoking the GetWorkItem method:

WorkItem wi = wis.GetWorkItem(1000); //Where 1000 is the Work Item ID

Now, your code should look something like this:

const string TFSServer = "http://mytfsserver:8090";
 
NetworkCredential netCred = new NetworkCredential("Username", "Password", "Domain");
 
using (TeamFoundationServer tfsConn = new TeamFoundationServer(TFSServer, netCred))
{
	try
	{
		tfsConn.EnsureAuthenticated();
                WorkItemStore wis = (WorkItemStore)tfsServer.GetService(typeof(WorkItemStore));
                WorkItem wi = wis.GetWorkItem(1000); //Where 1000 is the Work Item ID
	}
	catch (Exception ex)
	{
		MessageBox.Show(ex.Message, "Something has gone wrong!", MessageBoxButtons.OK, MessageBoxIcon.Error);
	}
}

Step 4: Get the attachments’ URI

After acquiring the Work Item, we can now directly iterate through the attachment collection to access its attachment’s URI:

foreach(Attachment attachment in wi.Attachments)
{
	string FileName = attachment.Name;
	string attachmentURI = attachment.Uri;
}

Cool huh? Just like that! Your entire code should look like this:

const string TFSServer = "http://mytfsserver:8090";
 
NetworkCredential netCred = new NetworkCredential("Username", "Password", "Domain");
 
using (TeamFoundationServer tfsConn = new TeamFoundationServer(TFSServer, netCred))
{
	try
	{
		tfsConn.EnsureAuthenticated();
                WorkItemStore wis = (WorkItemStore)tfsServer.GetService(typeof(WorkItemStore));
                WorkItem wi = wis.GetWorkItem(1000); //Where 1000 is the Work Item ID
 
		foreach(Attachment attachment in wi.Attachments)
		{
			string FileName = attachment.Name;
			string attachmentURI = attachment.Uri;
		}
 
	}
	catch (Exception ex)
	{
		MessageBox.Show(ex.Message, "Something has gone wrong!", MessageBoxButtons.OK, MessageBoxIcon.Error);
	}
}

Step 5: Downloading the attachments

And now that we have the attachment’s URI, all there is left to do is to download it. Thankfully, .NET provided us with the WebClient class that simplifies the downloading:

using (WebClient wClient = new WebClient())
{
	wClient.Credentials = netCred; //Provide your network credentials.
	wClient.DownloadFile(attachment.Uri, Path.Combine(@"C:", attachment.Name));
}

That’s it! You’re done. The completed code should look like this:

using (TeamFoundationServer tfsConn = new TeamFoundationServer(TFSServer, netCred))
{
	try
	{
		tfsConn.EnsureAuthenticated();
                WorkItemStore wis = (WorkItemStore)tfsServer.GetService(typeof(WorkItemStore));
                WorkItem wi = wis.GetWorkItem(1000); //Where 1000 is the Work Item ID
 
		foreach(Attachment attachment in wi.Attachments)
		{
			string FileName = attachment.Name;
			string attachmentURI = attachment.Uri;
 
			using (WebClient wClient = new WebClient())
			{
				wClient.Credentials = netCred;
				wClient.DownloadFile(attachmentURI, Path.Combine(@"C:", FileName));
			}
		}
 
	}
	catch (Exception ex)
	{
		MessageBox.Show(ex.Message, "Something has gone wrong!", MessageBoxButtons.OK, MessageBoxIcon.Error);
	}
}

Step 6: Optional Improvements

The code above is the minimum required to have a fully functional attachment downloader for TFS. There are a couple of things that you can do to improve your program:

• Automatically create a directory for each Work Item. This way, you will have an organized local storage of the attachments.
• Instead of using the DownloadFile method to download your file, you can use the DownloadFileAsync method to download the file asynchronously. DownloadFileAsync is non-thread blocking. This call will also allow you to listen to EventHandlers for the download progress.
• After the download, you can check if the file is a zip file (by using Path.GetExtension), and if it is, you can use SharpZipLib to automatically unzip the package.
• Sometimes, instead of providing the Incident number, your users will provide the Incident Fix number. Its good to anticipate that and provide a way to resolve the Incident Number from the Incident Fix. To do that, first check if your user provided an Incident Fix by calling wi.Fields.GetById(25).Value.ToString() == “Incident Fix” then if yes, you can get the Incident Number by calling int ParentID = (int)wi.Fields.GetById(10120).Value. Thank you Impatience!

I can’t really share my program with you guys (you know all of that “corporate” legal mumbo-jumbo). But here’s the screenshot:

Work Item Fetcher @ TFS

Thank you Laziness for making me go to great effort to reduce the total effort and time needed to download attachments from TFS. Thank you for making me write labor-saving programs that other people will hopefully find useful.

Thank you Impatience for pissing me off when I am manually downloading and renaming attachments one by one. Thank you too for making me angry when my initial prototype failed when I gave it an Incident Fix number rather than an Incident number.

And, as always, thank you Hubris! (People around me knows exactly why!)

Paul Graham (Author of Hackers and Painters)

I love Paul Graham. The guy speaks my mind. Although I don’t agree with all of his ideas (E.G. Java programmers are stupid. PERIOD.), most of his ideas are golden. For those of weak hearts (and sensitive ego), please proceed with care.

The essay that will follow, entitled Great Hackers, basically compliments the main idea of my previous post which is “To do something well you have to love it.”

Some quotable quotes from this essay:

“Ordinary programmers write code to pay the bills. Great hackers think of it as something they do for fun, and which they’re delighted to find people will pay them for.”

“Hackers like to work for people with high standards. But it’s not enough just to be exacting. You have to insist on the right things. Which usually means that you have to be a hacker yourself. I’ve seen occasional articles about how to manage programmers. Really there should be two articles: one about what to do if you are yourself a programmer, and one about what to do if you’re not. And the second could probably be condensed into two words: give up.”

“The distinguishing feature of nasty little problems is that you don’t learn anything from them. Working on nasty little problems makes you stupid. Good hackers avoid it for the same reason models avoid cheeseburgers.”

“I’ve found that people who are great at something are not so much convinced of their own greatness as mystified at why everyone else seems so incompetent.”

“The key to being a good hacker may be to work on what you like. When I think about the great hackers I know, one thing they have in common is the extreme difficulty of making them work on anything they don’t want to. I don’t know if this is cause or effect; it may be both.”

“One difference I’ve noticed between great hackers and smart people in general is that hackers are more politically incorrect. And I can see why political incorrectness would be a useful quality in programming. Programs are very complex and, at least in the hands of good programmers, very fluid. In such situations it’s helpful to have a habit of questioning assumptions.”

AND NOW, THE ESSAY

Great Hackers

Edisons

There’s no controversy about which idea is most controversial: the suggestion that variation in wealth might not be as big a problem as we think.

I didn’t say in the book that variation in wealth was in itself a good thing. I said in some situations it might be a sign of good things. A throbbing headache is not a good thing, but it can be a sign of a good thing– for example, that you’re recovering consciousness after being hit on the head.

Variation in wealth can be a sign of variation in productivity. (In a society of one, they’re identical.) And that is almost certainly a good thing: if your society has no variation in productivity, it’s probably not because everyone is Thomas Edison. It’s probably because you have no Thomas Edisons.

In a low-tech society you don’t see much variation in productivity. If you have a tribe of nomads collecting sticks for a fire, how much more productive is the best stick gatherer going to be than the worst? A factor of two? Whereas when you hand people a complex tool like a computer, the variation in what they can do with it is enormous.

That’s not a new idea. Fred Brooks wrote about it in 1974, and the study he quoted was published in 1968. But I think he underestimated the variation between programmers. He wrote about productivity in lines of code: the best programmers can solve a given problem in a tenth the time. But what if the problem isn’t given? In programming, as in many fields, the hard part isn’t solving problems, but deciding what problems to solve. Imagination is hard to measure, but in practice it dominates the kind of productivity that’s measured in lines of code.

Productivity varies in any field, but there are few in which it varies so much. The variation between programmers is so great that it becomes a difference in kind. I don’t think this is something intrinsic to programming, though. In every field, technology magnifies differences in productivity. I think what’s happening in programming is just that we have a lot of technological leverage. But in every field the lever is getting longer, so the variation we see is something that more and more fields will see as time goes on. And the success of companies, and countries, will depend increasingly on how they deal with it.

If variation in productivity increases with technology, then the contribution of the most productive individuals will not only be disproportionately large, but will actually grow with time. When you reach the point where 90% of a group’s output is created by 1% of its members, you lose big if something (whether Viking raids, or central planning) drags their productivity down to the average.

If we want to get the most out of them, we need to understand these especially productive people. What motivates them? What do they need to do their jobs? How do you recognize them? How do you get them to come and work for you? And then of course there’s the question, how do you become one?

More than Money

I know a handful of super-hackers, so I sat down and thought about what they have in common. Their defining quality is probably that they really love to program. Ordinary programmers write code to pay the bills. Great hackers think of it as something they do for fun, and which they’re delighted to find people will pay them for.

Great programmers are sometimes said to be indifferent to money. This isn’t quite true. It is true that all they really care about is doing interesting work. But if you make enough money, you get to work on whatever you want, and for that reason hackers are attracted by the idea of making really large amounts of money. But as long as they still have to show up for work every day, they care more about what they do there than how much they get paid for it.

Economically, this is a fact of the greatest importance, because it means you don’t have to pay great hackers anything like what they’re worth. A great programmer might be ten or a hundred times as productive as an ordinary one, but he’ll consider himself lucky to get paid three times as much. As I’ll explain later, this is partly because great hackers don’t know how good they are. But it’s also because money is not the main thing they want.

What do hackers want? Like all craftsmen, hackers like good tools. In fact, that’s an understatement. Good hackers find it unbearable to use bad tools. They’ll simply refuse to work on projects with the wrong infrastructure.

At a startup I once worked for, one of the things pinned up on our bulletin board was an ad from IBM. It was a picture of an AS400, and the headline read, I think, “hackers despise it.” [1]

When you decide what infrastructure to use for a project, you’re not just making a technical decision. You’re also making a social decision, and this may be the more important of the two. For example, if your company wants to write some software, it might seem a prudent choice to write it in Java. But when you choose a language, you’re also choosing a community. The programmers you’ll be able to hire to work on a Java project won’t be as smart as the ones you could get to work on a project written in Python. And the quality of your hackers probably matters more than the language you choose. Though, frankly, the fact that good hackers prefer Python to Java should tell you something about the relative merits of those languages.

Business types prefer the most popular languages because they view languages as standards. They don’t want to bet the company on Betamax. The thing about languages, though, is that they’re not just standards. If you have to move bits over a network, by all means use TCP/IP. But a programming language isn’t just a format. A programming language is a medium of expression.

I’ve read that Java has just overtaken Cobol as the most popular language. As a standard, you couldn’t wish for more. But as a medium of expression, you could do a lot better. Of all the great programmers I can think of, I know of only one who would voluntarily program in Java. And of all the great programmers I can think of who don’t work for Sun, on Java, I know of zero.

Great hackers also generally insist on using open source software. Not just because it’s better, but because it gives them more control. Good hackers insist on control. This is part of what makes them good hackers: when something’s broken, they need to fix it. You want them to feel this way about the software they’re writing for you. You shouldn’t be surprised when they feel the same way about the operating system.

A couple years ago a venture capitalist friend told me about a new startup he was involved with. It sounded promising. But the next time I talked to him, he said they’d decided to build their software on Windows NT, and had just hired a very experienced NT developer to be their chief technical officer. When I heard this, I thought, these guys are doomed. One, the CTO couldn’t be a first rate hacker, because to become an eminent NT developer he would have had to use NT voluntarily, multiple times, and I couldn’t imagine a great hacker doing that; and two, even if he was good, he’d have a hard time hiring anyone good to work for him if the project had to be built on NT. [2]

The Final Frontier

After software, the most important tool to a hacker is probably his office. Big companies think the function of office space is to express rank. But hackers use their offices for more than that: they use their office as a place to think in. And if you’re a technology company, their thoughts are your product. So making hackers work in a noisy, distracting environment is like having a paint factory where the air is full of soot.

The cartoon strip Dilbert has a lot to say about cubicles, and with good reason. All the hackers I know despise them. The mere prospect of being interrupted is enough to prevent hackers from working on hard problems. If you want to get real work done in an office with cubicles, you have two options: work at home, or come in early or late or on a weekend, when no one else is there. Don’t companies realize this is a sign that something is broken? An office environment is supposed to be something thathelps you work, not something you work despite.

Companies like Cisco are proud that everyone there has a cubicle, even the CEO. But they’re not so advanced as they think; obviously they still view office space as a badge of rank. Note too that Cisco is famous for doing very little product development in house. They get new technology by buying the startups that created it– where presumably the hackers did have somewhere quiet to work.

One big company that understands what hackers need is Microsoft. I once saw a recruiting ad for Microsoft with a big picture of a door. Work for us, the premise was, and we’ll give you a place to work where you can actually get work done. And you know, Microsoft is remarkable among big companies in that they are able to develop software in house. Not well, perhaps, but well enough.

If companies want hackers to be productive, they should look at what they do at home. At home, hackers can arrange things themselves so they can get the most done. And when they work at home, hackers don’t work in noisy, open spaces; they work in rooms with doors. They work in cosy, neighborhoody places with people around and somewhere to walk when they need to mull something over, instead of in glass boxes set in acres of parking lots. They have a sofa they can take a nap on when they feel tired, instead of sitting in a coma at their desk, pretending to work. There’s no crew of people with vacuum cleaners that roars through every evening during the prime hacking hours. There are no meetings or, God forbid, corporate retreats or team-building exercises. And when you look at what they’re doing on that computer, you’ll find it reinforces what I said earlier about tools. They may have to use Java and Windows at work, but at home, where they can choose for themselves, you’re more likely to find them using Perl and Linux.

Indeed, these statistics about Cobol or Java being the most popular language can be misleading. What we ought to look at, if we want to know what tools are best, is what hackers choose when they can choose freely– that is, in projects of their own. When you ask that question, you find that open source operating systems already have a dominant market share, and the number one language is probably Perl.

Interesting

Along with good tools, hackers want interesting projects. What makes a project interesting? Well, obviously overtly sexy applications like stealth planes or special effects software would be interesting to work on. But any application can be interesting if it poses novel technical challenges. So it’s hard to predict which problems hackers will like, because some become interesting only when the people working on them discover a new kind of solution. Before ITA (who wrote the software inside Orbitz), the people working on airline fare searches probably thought it was one of the most boring applications imaginable. But ITA made it interesting by redefining the problem in a more ambitious way.

I think the same thing happened at Google. When Google was founded, the conventional wisdom among the so-called portals was that search was boring and unimportant. But the guys at Google didn’t think search was boring, and that’s why they do it so well.

This is an area where managers can make a difference. Like a parent saying to a child, I bet you can’t clean up your whole room in ten minutes, a good manager can sometimes redefine a problem as a more interesting one. Steve Jobs seems to be particularly good at this, in part simply by having high standards. There were a lot of small, inexpensive computers before the Mac. He redefined the problem as: make one that’s beautiful. And that probably drove the developers harder than any carrot or stick could.

They certainly delivered. When the Mac first appeared, you didn’t even have to turn it on to know it would be good; you could tell from the case. A few weeks ago I was walking along the street in Cambridge, and in someone’s trash I saw what appeared to be a Mac carrying case. I looked inside, and there was a Mac SE. I carried it home and plugged it in, and it booted. The happy Macintosh face, and then the finder. My God, it was so simple. It was just like … Google.

Hackers like to work for people with high standards. But it’s not enough just to be exacting. You have to insist on the right things. Which usually means that you have to be a hacker yourself. I’ve seen occasional articles about how to manage programmers. Really there should be two articles: one about what to do if you are yourself a programmer, and one about what to do if you’re not. And the second could probably be condensed into two words: give up.

The problem is not so much the day to day management. Really good hackers are practically self-managing. The problem is, if you’re not a hacker, you can’t tell who the good hackers are. A similar problem explains why American cars are so ugly. I call it the design paradox. You might think that you could make your products beautiful just by hiring a great designer to design them. But if you yourself don’t have good taste, how are you going to recognize a good designer? By definition you can’t tell from his portfolio. And you can’t go by the awards he’s won or the jobs he’s had, because in design, as in most fields, those tend to be driven by fashion and schmoozing, with actual ability a distant third. There’s no way around it: you can’t manage a process intended to produce beautiful things without knowing what beautiful is. American cars are ugly because American car companies are run by people with bad taste.

Many people in this country think of taste as something elusive, or even frivolous. It is neither. To drive design, a manager must be the most demanding user of a company’s products. And if you have really good taste, you can, as Steve Jobs does, make satisfying you the kind of problem that good people like to work on.

Nasty Little Problems

It’s pretty easy to say what kinds of problems are not interesting: those where instead of solving a few big, clear, problems, you have to solve a lot of nasty little ones. One of the worst kinds of projects is writing an interface to a piece of software that’s full of bugs. Another is when you have to customize something for an individual client’s complex and ill-defined needs. To hackers these kinds of projects are the death of a thousand cuts.

The distinguishing feature of nasty little problems is that you don’t learn anything from them. Writing a compiler is interesting because it teaches you what a compiler is. But writing an interface to a buggy piece of software doesn’t teach you anything, because the bugs are random. [3] So it’s not just fastidiousness that makes good hackers avoid nasty little problems. It’s more a question of self-preservation. Working on nasty little problems makes you stupid. Good hackers avoid it for the same reason models avoid cheeseburgers.

Of course some problems inherently have this character. And because of supply and demand, they pay especially well. So a company that found a way to get great hackers to work on tedious problems would be very successful. How would you do it?

One place this happens is in startups. At our startup we had Robert Morris working as a system administrator. That’s like having the Rolling Stones play at a bar mitzvah. You can’t hire that kind of talent. But people will do any amount of drudgery for companies of which they’re the founders. [4]

Bigger companies solve the problem by partitioning the company. They get smart people to work for them by establishing a separate R&D department where employees don’t have to work directly on customers’ nasty little problems. [5] In this model, the research department functions like a mine. They produce new ideas; maybe the rest of the company will be able to use them.

You may not have to go to this extreme. Bottom-up programming suggests another way to partition the company: have the smart people work as toolmakers. If your company makes software to do x, have one group that builds tools for writing software of that type, and another that uses these tools to write the applications. This way you might be able to get smart people to write 99% of your code, but still keep them almost as insulated from users as they would be in a traditional research department. The toolmakers would have users, but they’d only be the company’s own developers. [6]

If Microsoft used this approach, their software wouldn’t be so full of security holes, because the less smart people writing the actual applications wouldn’t be doing low-level stuff like allocating memory. Instead of writing Word directly in C, they’d be plugging together big Lego blocks of Word-language. (Duplo, I believe, is the technical term.)

Clumping

Along with interesting problems, what good hackers like is other good hackers. Great hackers tend to clump together– sometimes spectacularly so, as at Xerox Parc. So you won’t attract good hackers in linear proportion to how good an environment you create for them. The tendency to clump means it’s more like the square of the environment. So it’s winner take all. At any given time, there are only about ten or twenty places where hackers most want to work, and if you aren’t one of them, you won’t just have fewer great hackers, you’ll have zero.

Having great hackers is not, by itself, enough to make a company successful. It works well for Google and ITA, which are two of the hot spots right now, but it didn’t help Thinking Machines or Xerox. Sun had a good run for a while, but their business model is a down elevator. In that situation, even the best hackers can’t save you.

I think, though, that all other things being equal, a company that can attract great hackers will have a huge advantage. There are people who would disagree with this. When we were making the rounds of venture capital firms in the 1990s, several told us that software companies didn’t win by writing great software, but through brand, and dominating channels, and doing the right deals.

They really seemed to believe this, and I think I know why. I think what a lot of VCs are looking for, at least unconsciously, is the next Microsoft. And of course if Microsoft is your model, you shouldn’t be looking for companies that hope to win by writing great software. But VCs are mistaken to look for the next Microsoft, because no startup can be the next Microsoft unless some other company is prepared to bend over at just the right moment and be the next IBM.

It’s a mistake to use Microsoft as a model, because their whole culture derives from that one lucky break. Microsoft is a bad data point. If you throw them out, you find that good products do tend to win in the market. What VCs should be looking for is the next Apple, or the next Google.

I think Bill Gates knows this. What worries him about Google is not the power of their brand, but the fact that they have better hackers. [7]

Recognition

So who are the great hackers? How do you know when you meet one? That turns out to be very hard. Even hackers can’t tell. I’m pretty sure now that my friend Trevor Blackwell is a great hacker. You may have read on Slashdot how he made his own Segway. The remarkable thing about this project was that he wrote all the software in one day (in Python, incidentally).

For Trevor, that’s par for the course. But when I first met him, I thought he was a complete idiot. He was standing in Robert Morris’s office babbling at him about something or other, and I remember standing behind him making frantic gestures at Robert to shoo this nut out of his office so we could go to lunch. Robert says he misjudged Trevor at first too. Apparently when Robert first met him, Trevor had just begun a new scheme that involved writing down everything about every aspect of his life on a stack of index cards, which he carried with him everywhere. He’d also just arrived from Canada, and had a strong Canadian accent and a mullet.

The problem is compounded by the fact that hackers, despite their reputation for social obliviousness, sometimes put a good deal of effort into seeming smart. When I was in grad school I used to hang around the MIT AI Lab occasionally. It was kind of intimidating at first. Everyone there spoke so fast. But after a while I learned the trick of speaking fast. You don’t have to think any faster; just use twice as many words to say everything.

With this amount of noise in the signal, it’s hard to tell good hackers when you meet them. I can’t tell, even now. You also can’t tell from their resumes. It seems like the only way to judge a hacker is to work with him on something.

And this is the reason that high-tech areas only happen around universities. The active ingredient here is not so much the professors as the students. Startups grow up around universities because universities bring together promising young people and make them work on the same projects. The smart ones learn who the other smart ones are, and together they cook up new projects of their own.

Because you can’t tell a great hacker except by working with him, hackers themselves can’t tell how good they are. This is true to a degree in most fields. I’ve found that people who are great at something are not so much convinced of their own greatness as mystified at why everyone else seems so incompetent.

But it’s particularly hard for hackers to know how good they are, because it’s hard to compare their work. This is easier in most other fields. In the hundred meters, you know in 10 seconds who’s fastest. Even in math there seems to be a general consensus about which problems are hard to solve, and what constitutes a good solution. But hacking is like writing. Who can say which of two novels is better? Certainly not the authors.

With hackers, at least, other hackers can tell. That’s because, unlike novelists, hackers collaborate on projects. When you get to hit a few difficult problems over the net at someone, you learn pretty quickly how hard they hit them back. But hackers can’t watch themselves at work. So if you ask a great hacker how good he is, he’s almost certain to reply, I don’t know. He’s not just being modest. He really doesn’t know.

And none of us know, except about people we’ve actually worked with. Which puts us in a weird situation: we don’t know who our heroes should be. The hackers who become famous tend to become famous by random accidents of PR. Occasionally I need to give an example of a great hacker, and I never know who to use. The first names that come to mind always tend to be people I know personally, but it seems lame to use them. So, I think, maybe I should say Richard Stallman, or Linus Torvalds, or Alan Kay, or someone famous like that. But I have no idea if these guys are great hackers. I’ve never worked with them on anything.

If there is a Michael Jordan of hacking, no one knows, including him.

Cultivation

Finally, the question the hackers have all been wondering about: how do you become a great hacker? I don’t know if it’s possible to make yourself into one. But it’s certainly possible to do things that make you stupid, and if you can make yourself stupid, you can probably make yourself smart too.

The key to being a good hacker may be to work on what you like. When I think about the great hackers I know, one thing they have in common is the extreme difficulty of making them work on anything they don’t want to. I don’t know if this is cause or effect; it may be both.

To do something well you have to love it. So to the extent you can preserve hacking as something you love, you’re likely to do it well. Try to keep the sense of wonder you had about programming at age 14. If you’re worried that your current job is rotting your brain, it probably is.

The best hackers tend to be smart, of course, but that’s true in a lot of fields. Is there some quality that’s unique to hackers? I asked some friends, and the number one thing they mentioned was curiosity. I’d always supposed that all smart people were curious– that curiosity was simply the first derivative of knowledge. But apparently hackers are particularly curious, especially about how things work. That makes sense, because programs are in effect giant descriptions of how things work.

Several friends mentioned hackers’ ability to concentrate– their ability, as one put it, to “tune out everything outside their own heads.” I’ve certainly noticed this. And I’ve heard several hackers say that after drinking even half a beer they can’t program at all. So maybe hacking does require some special ability to focus. Perhaps great hackers can load a large amount of context into their head, so that when they look at a line of code, they see not just that line but the whole program around it. John McPhee wrote that Bill Bradley’s success as a basketball player was due partly to his extraordinary peripheral vision. “Perfect” eyesight means about 47 degrees of vertical peripheral vision. Bill Bradley had 70; he could see the basket when he was looking at the floor. Maybe great hackers have some similar inborn ability. (I cheat by using a very denselanguage, which shrinks the court.)

This could explain the disconnect over cubicles. Maybe the people in charge of facilities, not having any concentration to shatter, have no idea that working in a cubicle feels to a hacker like having one’s brain in a blender. (Whereas Bill, if the rumors of autism are true, knows all too well.)

One difference I’ve noticed between great hackers and smart people in general is that hackers are more politically incorrect. To the extent there is a secret handshake among good hackers, it’s when they know one another well enough to express opinions that would get them stoned to death by the general public. And I can see why political incorrectness would be a useful quality in programming. Programs are very complex and, at least in the hands of good programmers, very fluid. In such situations it’s helpful to have a habit of questioning assumptions.

Can you cultivate these qualities? I don’t know. But you can at least not repress them. So here is my best shot at a recipe. If it is possible to make yourself into a great hacker, the way to do it may be to make the following deal with yourself: you never have to work on boring projects (unless your family will starve otherwise), and in return, you’ll never allow yourself to do a half-assed job. All the great hackers I know seem to have made that deal, though perhaps none of them had any choice in the matter.

Notes

[1] In fairness, I have to say that IBM makes decent hardware. I wrote this on an IBM laptop.

[2] They did turn out to be doomed. They shut down a few months later.

[3] I think this is what people mean when they talk about the “meaning of life.” On the face of it, this seems an odd idea. Life isn’t an expression; how could it have meaning? But it can have a quality that feels a lot like meaning. In a project like a compiler, you have to solve a lot of problems, but the problems all fall into a pattern, as in a signal. Whereas when the problems you have to solve are random, they seem like noise.

[4] Einstein at one point worked designing refrigerators. (He had equity.)

[5] It’s hard to say exactly what constitutes research in the computer world, but as a first approximation, it’s software that doesn’t have users.

I don’t think it’s publication that makes the best hackers want to work in research departments. I think it’s mainly not having to have a three hour meeting with a product manager about problems integrating the Korean version of Word 13.27 with the talking paperclip.

[6] Something similar has been happening for a long time in the construction industry. When you had a house built a couple hundred years ago, the local builders built everything in it. But increasingly what builders do is assemble components designed and manufactured by someone else. This has, like the arrival of desktop publishing, given people the freedom to experiment in disastrous ways, but it is certainly more efficient.

[7] Google is much more dangerous to Microsoft than Netscape was. Probably more dangerous than any other company has ever been. Not least because they’re determined to fight. On their job listing page, they say that one of their “core values” is “Don’t be evil.” From a company selling soybean oil or mining equipment, such a statement would merely be eccentric. But I think all of us in the computer world recognize who that is a declaration of war on.

From: http://www.paulgraham.com/gh.html

ASP.NET Migration Problem

If you’ve read my previous blog post, I talked about some of the problems encountered while migrating some of our web applications from .NET 1.1 to .NET 2.0. Well, those products are now handed off to the Test Department for their routine torture.

Just a couple of days after the hand-off (Hand-off to Test or HOTT), the tester who is testing the product filed two (2) Tracks (Incident/Bug/Problem). Since I am the only developer working on the product, all of the tracks will automatically be assigned to me. The said tracks that were filed have the same “type” of problem.

When browsing for a file (Backup File / File Upload), the value of the Textbox would change back to its default value when the page re-loads.

The web application that I am supporting was originally written in .NET 1.1 using Visual Studio 2003. To migrate it to .NET 2.0, we converted the solutions to VS2005. Aside from some minor problems like RESX incompatibility and broken Calendar Controls, the Web Application worked — I thought it did.

We have a Backup Restoration Page to allow the user to restore the web application in-case of Armageddon-like circumstances. The said page basically contain 3 components:

1. A Read-Only Textbox that holds the path of the Backup File . This Textbox has a default value of C:\Backups\Backup.bak
2. A Browse button that will launch a custom page that will allow the user to browse for the Backup File. The value selected there will be posted-back to the Backup Restoration Page.
3. A “Restore” button to start the restoration process.

When I tried to repro (reproduce) the bug on my Virtual Machine setup, I was able to verify that the path of the Backup File is losing its value on postback. To put it more precisely, the Textbox is not retaining the correct path since it is changing back to its default value on postback.

After hours of debugging, I was no where close in determining the cause of the problem. So I did a quick search on Google and found out that this is a common problem during migration from ASP.NET 1.1 to ASP.NET 2.0.

The problem is that Microsoft changed the behavior of read-only textboxes on ASP.NET 2.0.

ASP.NET 2.0 had a design change by which a control if marked with its ReadOnly property as true, would ignore client side changes and would lose the same across postback. So if you tried modifying the text box value or add a value to the text box using Javascript you wouldnt be able to retireve the value in the code behind or simply the value will be lost across postback.

This behaviour is by design in ASP.NET 2.0 and it has been designed with the idea that a ReadOnly TextBox shouldnt be modified in the client side by a malicious code.

Microsoft has a point. What’s the point of using a Textbox if you’re going to set it to Read-Only? Why not use a Label instead?

Fortunately, we have three simple workarounds for this problem.

1. Use a Label instead of a Read-Only Textbox. (Heard of Borders?)
2. Set the Read-Only property of the Textbox to FALSE (If you don’t really need the Read-Only Property)
3. But if you are really insisting of using a Read-Only Textbox, then we need to “fool” ASP.NET that the Textbox is not Read-Only even if it is.

To “fool” ASP.NET, you need to set the Read-Only property of the Textbox to FALSE during design time (Using the Property Pane) then set it to Read-Only on Page Load during runtime. You CANNOT use Textbox1.ReadOnly = true. You need to set the read-only attribute using the Attribute.Add() method.

// This is part of the C# code-behind file.
protected void Page_Load(object sender, EventArgs e)
{
     TextBox1.Attributes.Add("readonly", "readonly");
}

That will solve your postback problems.

References:

• http://msdn.microsoft.com/en-us/library/system.web.ui.webcontrols.textbox.readonly.aspx
• http://www.dotnetspider.com/resources/3120-ASP-NET–TextBox-Ready-Only-losing-client-side-changes-values-across-postback.aspx
• http://stackoverflow.com/questions/856838/migration-asp-net-1-1-to-asp-net-2-0-broken-postback

Do you know how much Programmer's Blood it cost to put this tiny sticker in a Software Box?

In my previous blog post, I’ve mentioned that I’m working on a project to migrate an entire product line to Vista. What I meant by that is that I am responsible for making sure that our current product will run properly on Windows Vista. To guarantee that it is indeed Running Properly it should behave *exactly* like it behaves in Windows XP.  In simple terms, my entire job security is riding on making our products compatible with Microsoft’s failed attempt at a MAC-ish Operating System and with Microsoft’s successful attempt (a first in their history) to make their operating system NOT backward compatible with older applications. So much for Raymond Chen’s camp undying obsession with backward compatibility.

This products that I’m talking about are a set of Web Applications written in ASP.NET version 1.1. This web applications are as old as Windows XP. I checked its File Modification History in our TFS repository and the last time that anyone have touched its code base is in 2005 — and that’s just for a minor code modification to fix a bug. I believe this apps were written way way back in 2004 — when I was in Third Year High School!

To make this products compatible with Vista, there are a couple of steps that had to be done. I’ll enumerate each steps and what it takes to accomplish them.

.NET Upgrade (.NET 1.1 to .NET 2.0)

Windows Vista comes pre-installed with .NET 2.0 (and .NET 3.0). Our products are running on .NET 1.1. One approach that most people will suggest is to just install .NET 1.1 on Windows Vista and everything will be perfect. Right? WRONG. Firstly, .NET 1.1 has already reached its end-of-life. Microsoft pulled support for .NET 1.1 last October 14, 2008. Although it was extended, its a sign that this technology is old enough to be replaced. Secondly, installing .NET 1.1 on a Vista machine is not as easy as you think. No, we are not lazy programmers, we just don’t want to introduce unnecessary complications to the system. We are, by the way, Software Developers.

Those Lazy Programmers

So the approach we took is to upgrade our Visual Studio 2003 solutions to Visual Studio 2005 solutions. This, in turn, will “force” the web application to use .NET 2.0. You might say, “How hard could it be? Visual Studio 2005 can convert old solutions automatically using its Upgrade Wizard. All you have to do is click next!, You lazy programmer!”. That’s correct. Upgrading the solution is no problem. Converting every file to .NET 2.0 compliant code is.

We encountered two conversion problems. The first one is relatively simple to fix. It’s a problem where the old .NET 1.1 Resource files (*.resx) were not converted to the .NET 2.0 resource files. The two resource file formats are different (the .NET 2.0 have a *.designer.cs component) and in very rare cases, the .NET 1.1 *resx files won’t work in .NET 2.0. This problem is very hard to catch since the incompatible *.resx files won’t throw a compile error. The strings just won’t be detected. But once found it was relatively easy to fix. After a day of investigation, we found out that the XML format for the resource files changed in .NET 2.0. As a simple hack (fix), we manually changed the <string> parameters of each .NET 1.1 strings to match the .NET 2.0 <string> parameters.

The second problem is harder to find and much much harder to fix. The problem is related to ASP.NET’s datetime picker control.

Our Calendar Control before upgrading to .NET 2.0

On our previous version running .NET 1.1, there is no problem with the datetime picker. But when we installed the .NET 2.0 version, the calendar objects are all over the place.

The Migrated .NET 2.0 Calendar Control

Two days of investigation later, we found out that the previous developer made this custom Calendar control by inheriting from .NET 1.1’s System.Web.Controls. During pre-render, he reads the rendered Calendar Control’s HTML and then does something funny with it. What he wanted to achieve is to replace the “Month” from the Calendar header (gray part) with a drop down box that contains a list of the months. So what he did is he “catch” the HTML stream from the Calendar Control and perform a String.Replace on it!

His Design Goal

The actual String.Replace command is something like this:

//szHTML = Get current HTML stream
//szCurrentMonth = Get current month from HTML
szHTML.Replace(szCurrentMonth, "
<select><option value="January">January</option>
       <option value="February">February</option>
       <option value="March">March</option>
       <option value="April">April</option>
       <option value="May">May</option>
       <option value="June">June</option>
       <option value="July">July</option>
       <option value="August">August</option>
       <option value="September">September</option>
       <option value="October">October</option>
       <option value="November">November</option>
       <option value="December">December</option>
</select>
 
");

It worked fine before the migration, but somehow we broke it during migration to .NET 2.0. A further investigation of the “emitted” HTML stream from the .NET 2.0 control shows this:

<table style="border-style: solid; border-width: 1px; border-collapse: collapse;" title="Calendar" border="0" cellspacing="0" cellpadding="2">
<tbody>
<tr>
<td style="background-color: Silver;" colspan="7">
<table style="width: 100%; border-collapse: collapse;" border="0" cellspacing="0">
<tbody>
<tr>
<td style="width: 15%;"><a style="color: Black;" title="Go to the previous month" href="javascript:__doPostBack('ctl01','V3378')">&lt;</a></td>
<!-- THIS WILL BE THE PART THAT WILL BE REPLACED BY A DROP DOWN BOX -->
<td style="width: 70%;" align="center">May 2009</td>
<td style="width: 15%;" align="right"><a style="color: Black;" title="Go to the next month" href="javascript:__doPostBack('ctl01','V3439')">&gt;</a></td>
</tr>
</tbody></table>
</td>
</tr>
<tr>
<th align="center" scope="col">Sun</th>
<th align="center" scope="col">Mon</th>
<th align="center" scope="col">Tue</th>
<th align="center" scope="col">Wed</th>
<th align="center" scope="col">Thu</th>
<th align="center" scope="col">Fri</th>
<th align="center" scope="col">Sat</th>
</tr>
<tr>
<td style="width: 14%;" align="center"><a style="color: black;" title="April 26" href="javascript:__doPostBack('ctl01','3403')">26</a></td>
<td style="width: 14%;" align="center"><a style="color: Black;" title="April 27" href="javascript:__doPostBack('ctl01','3404')">27</a></td>
<td style="width: 14%;" align="center"><a style="color: Black;" title="April 28" href="javascript:__doPostBack('ctl01','3405')">28</a></td>
<td style="width: 14%;" align="center"><a style="color: Black;" title="April 29" href="javascript:__doPostBack('ctl01','3406')">29</a></td>
<td style="width: 14%;" align="center"><a style="color: Black;" title="April 30" href="javascript:__doPostBack('ctl01','3407')">30</a></td>
<!-- THE TITLE ATTRIBUTE WAS ALSO REPLACED ACCIDENTALLY -->
<td style="width: 14%;" align="center"><a style="color: Black;" title="May 01" href="javascript:__doPostBack('ctl01','3408')">1</a></td>
<td style="width: 14%;" align="center"><a style="color: Black;" title="May 02" href="javascript:__doPostBack('ctl01','3409')">2</a></td>
</tr>
.....</tbody></table>

A comparison with the “emitted” .NET 1.1 control shows this:

<table style="border-style: solid; border-width: 1px; border-collapse: collapse;" title="Calendar" border="0" cellspacing="0" cellpadding="2">
<tbody>
<tr>
<td style="background-color: Silver;" colspan="7">
<table style="width: 100%; border-collapse: collapse;" border="0" cellspacing="0">
<tbody>
<tr>
<td style="width: 15%;"><a style="color: Black;" title="Go to the previous month" href="javascript:__doPostBack('ctl01','V3378')">&lt;</a></td>
<!-- THIS WILL BE THE PART THAT WILL BE REPLACED BY A DROP DOWN BOX -->
<td style="width: 70%;" align="center">May 2009</td>
<td style="width: 15%;" align="right"><a style="color: Black;" title="Go to the next month" href="javascript:__doPostBack('ctl01','V3439')">&gt;</a></td>
</tr>
</tbody></table>
</td>
</tr>
<tr>
<th align="center" scope="col">Sun</th>
<th align="center" scope="col">Mon</th>
<th align="center" scope="col">Tue</th>
<th align="center" scope="col">Wed</th>
<th align="center" scope="col">Thu</th>
<th align="center" scope="col">Fri</th>
<th align="center" scope="col">Sat</th>
</tr>
<tr>
<td style="width: 14%;" align="center"><a style="color: black;" href="javascript:__doPostBack('ctl01','3403')">26</a></td>
<td style="width: 14%;" align="center"><a style="color: Black;" href="javascript:__doPostBack('ctl01','3404')">27</a></td>
<td style="width: 14%;" align="center"><a style="color: Black;" href="javascript:__doPostBack('ctl01','3405')">28</a></td>
<td style="width: 14%;" align="center"><a style="color: Black;" href="javascript:__doPostBack('ctl01','3406')">29</a></td>
<td style="width: 14%;" align="center"><a style="color: Black;" href="javascript:__doPostBack('ctl01','3407')">30</a></td>
<td style="width: 14%;" align="center"><a style="color: Black;" href="javascript:__doPostBack('ctl01','3408')">1</a></td>
<td style="width: 14%;" align="center"><a style="color: Black;" href="javascript:__doPostBack('ctl01','3409')">2</a></td>
</tr>
.....</tbody></table>

Executing the String.Replace command in the .NET 1.1 code will produce the proper output. However, Microsoft decided to upgrade the Calendar control in .NET 2.0 by adding a tooltip for each date!

Tool Tip in .NET 2.0

Since the String.Replace has no checking (this is not REGEX afterall), it also replaced the “Month” in the tooltip with the drop down box (<select> command), thus generating the broken control shown above. This is not, in anyway, the fault of the original developer. Can you anticipate what will Microsoft do?

This is my first encounter on how Microsoft can break your products.

MSDE to Microsoft SQL Server 2005 Express

Our web applications were written before the release of SQL Server 2005 Express. The only *free* choice back then is Microsoft SQL Server Desktop Engine, a free scaled-down version of Microsoft SQL Server 2000. Since the support for MSDE officially ended last April 8, 2008, we are then forced to upgrade to Microsoft SQL Server 2005 express.

Our previous MSDE setup runs on default instance. In migrating to SQL Server 2005, we run the risk of “conflicting” with our other products that runs on default instance of SQL 2005. So the decision was made to migrate from default instance to named instance.

Migrating to SQL Server 2005 Express is straightforward at first look. The first and the most obvious thing to do is to modify the installer (InstallShield) so that it will install SQL 2005 instead of MSDE.  To compensate with the instance problem, the second thing to do is to modify each and every connection string in the entire app to point to the proper named instance. Making sure that each and every connection string in your entire app has been changed is a monumental task… especially if different version of connection strings were used. After that, you also need to modify the Service Query String so that it will point to the proper SQL Service ($MSSQLSERVER to $Instance_Name). There are also minor modifications here and there, but its not worth the mention.

After three days of manual modification, we got to the point where we were able to install the product. We tested it and everything seems to work fine. We then got to the part where are going to test for the database backup. Lo and behold, it failed. But no error message was given. We looked for the bug for an entire day but our search proved futile. We called it a day and went home. The next day, my brilliant manager excitedly told us the problem.

Microsoft SQL Server 2005 Express does not have SQL Server Agent.

Our backup routine uses SQL Server Agent. When a user request for a backup, the SQL Server will “schedule” that backup request and the SQL Server Agent will “execute” the request when the schedule parameter is met.

Simply put, No SQL Server Agent, No Backup.

We have three solutions to the problem:

1. Ship the product with Microsoft SQL Server 2005 standard or enterprise edition (Instead of the Express edition)
2. Remove the “scheduled” backup. Instead, just provide a “one shot” backup
3. Create our own custom “Backup Scheduler” using Windows Task Scheduler

We talked to the marketing folks and they eliminated option 1 because, according to them, we can’t make our customer pay for something that was free before. Option 2 was eliminated due to the simple fact that we can’t ship an upgraded version of our product with downgraded features (But this was actually done to one of the product that had missed its deadline). So Option 3 was our only choice.

The plan in creating Option 3 is simple. Create a “scheduled task” using schtasks.exe that will invoke sqlcmd.exe that will invoke the sql command to invoke the stored procedure that was doing the backups. The schtasks will then be invoked by the ASP.NET page using Process.Start();. So I created a wrapper class that will simplify this process. The class was created in 2 hours and I had it running and compiled to the ASP.NET page in just 3 hours. BUT IT DIDN’T WORK.

Further investigation revealed that IIS 6.0 has a different security model than IIS 5.1 (which XP uses). Spawning a process from within IIS 6.0 will have a credential of NETWORK SERVICE. Yes, we can impersonate since we are using Windows Authentication, but it won’t work unless we force the user to type-in the credentials again. Which is bad in a usability point of view.

The workaround was to put the scheduler on a Windows Service that will then be run with proper credentials. The problem then was the communication between the ASP.NET page and the Windows Service. How will the Service know when to invoke the backup? Our solution is a little crude but simple. We did it old-skul UNIX style! The ASP.NET page will create a “marker” file in a certain, defined location. The Windows Service will then monitor that location… waiting for the “marker” file. If the marker file is found, the service will then invoke the backup procedure. Fortunately for us, all the data needed for the backup is in the database, so we don’t need to pass anything between the ASP.NET page and the Service. We could have used sockets or mutex, but its overkill for our purpose.

This are just the major steps that had to be done in order to make our web applications compatible with Windows Vista. There are a couple of minor steps that also needs attention like the IIS 5.1 to IIS 7.0 migration and the new Vista Security Model.

The next time you see the “Vista Compatible” sticker on a Software Box, think of the programmers who nearly died in making your software run on a “cuter” operating system.

Windows 7, we are ready!

While having dinner with my teammates at the Shangri-La Plaza, we had a nostalgic conversation about our adventures in technology, particularly in programming. Just a bunch of geeks talking about geeky stuffs on a Friday night.

According to our Project Lead, he started his programming adventure by writing programs in BASIC on an Intel 486 computer a Sharp MZ80 Series (2 Mhz and 24KB of RAM). He also mentioned that he had once used a cassette tape to store his programs. He is that old (80+ years old in Programmer Years). Another teammate told me that his programming adventure started when he first discovered Visual Basic. For me, my programming adventure began when my elementary professor taught us Web Development (as in HTML) way way back in 1999. 1999 was the time when having an E-Mail account can get you laid . *I know, HTML is not programming! STOP BOTHERING ME*

After sometime reminiscing the days of Web 1.0, Visual Basic 4, Dial-Up Internet, mIRC, Hacked Unix Shell Accounts, Hacked AOL accounts and the Borland Turbo C Compilers, I suddenly brought up the topic of programming milestones — Those important events in your programming life that marks an important transformation. Those paradigm-shifting events that changed your geeky life forever. What’s interesting with this conversation is that everyone in the table can relate to each others milestones, its like everyone have the same programming milestones. Maybe all programmers have the same set of programming milestones? Like in life (as in real life), where we consider both College Graduation and Marriage as milestones.

Anyway, here are my programming milestones so far (Sorted by Date):

1. Arrays (1999)

I’ve previously mentioned that my first ever functional program is a tiny tool that can be used to help determine the winner(s) of a Millionaire’s Game (A Monopoly-like game). I made that tool because, most often than not, the players participating in the game will just agree that noone should win because they are just too lazy to “compute” for the winner.

I made that program in C using Borland’s Turbo C Compiler (TC.EXE) back in 1999. Back then, all I know are Console I/O (printf,scanf), IF Statements and FOR Loops! So its safe to say that I don’t know that Arrays even exists. Here’s how my program in C looked like:

#include <stdio.h>
 
int main()
{
	char Player1[100], Player2[100], Player3[100], Player4[100], Player5[100];
	int P1Score, P2Score, P3Score, P4Score, P5Score;
 
	printf("Enter the name of Player 1: ");
	scanf("%s", &Player1);
 
	printf("\nEnter the name of Player 2: ");
	scanf("%s", &Player2);
 
	printf("\nEnter the name of Player 3: ");
	scanf("%s", &Player3);
 
	printf("\nEnter the name of Player 4: ");
	scanf("%s", &Player4);
 
	printf("\nEnter the name of Player 5: ");
	scanf("%s", &Player5);
 
	//... You Get The Point!
}

I don’t even have an inkling that char Player1[100] is an array. I just know that Player1[100] is a variable that can store 100 characters. That’s it! But anyway, I was limited to copy-pasting similar block of codes to accomodate for each player. Even the computation logic was copy-pasted.

You can tell how happy I was when I learned of arrays.

2. Graphical User Interfaces – Visual Basic 6.0 (2001)

From 1999-2001, I have probably made five programs in C. Not all of them worked but some of them did – like my Hindu Arabic to Roman Numeral Converter and my poor attempt to simulate SM’s POS System. My first POS System have a pseudo-GUI. The “graphical objects” are made up of ASCII characters like all DOS-based programs and like most DOS-based programs, its barely usable.

When my High School Computer Professor showed me Visual Basic 6.0 in early 2001, all I can think about is this:

“Oh Shoot! Now I can make my program look like the *real* thing and sell them on the Internet and then I’ll be as rich as Bill Gates!”

Kidding aside, going from Terminal-based (Console) program to Windows Application is one of the greatest milestones in my programming life. Having your mom use your user-friendly POS system for your little business is very very rewarding.

3. Filestream – Writing and Reading Files (2002)

When my professor in High School taught me Visual Basic 6.0 and Graphical User Interfaces, I thought then that making your program look like the *real* thing is just the same as making your program *real*. Real as in functional. But it isn’t the same. Let’s face it, if your program can’t persist data, it is close to useless. So my dreams of becoming as rich as Bill Gates were suddenly crushed, at least temporarily.

Year 2002 was that magnificent year when I was able to force my mom into upgrading our old and broken home computer. I can still remember the *shiny* new specs of that computer:

• AMD K6-2 500 MHZ
• Seagate 20 GB HDD (IDE, 5400 RPM)
• ECX Motherboard (Can’t remember the exact model)
• 128MB SDRAM PC100/133
• Connectix 56K Dial-Up PCI Win Modem

That was the biggest and most important investment my mom made: A DIAL-UP MODEM! It was the very first time that I had unlimited (Unlimited Dial-Up Prepaid Card) access to the internet, at home. That was also the first time that  I had access to almost unlimited amounts of low-resolution P*rn images! Also, and most imporantly, that was the first time that I had access to the vast quantity of knowledge available in the Internet.

I can still remember that fateful summer day. I was making a simple tic-tac-toe game in Visual Basic 6.0, the game logic are all done and the game is already 100% fully functional. I gave the game to my brother to try it out and he asked me if there is a Top-Score list in my program. It made me think. How can I persist data even after the program was closed? Fortunately, the internet was there to help me out.

After a couple of hours searching and asking in the Forums, someone told me that I should write the data in a File and read it everytime the program is launched.

For the very first time in my entire programming life, I was able to persist data in a file! After 4 years of just working inside the RAM, I can now write stuffs on my Harddisk! A whole new universe of possibility was unlocked. I was so excited! Hundreds of idea came rushing into me. Suddenly I want to make my own Integrated Library System (OPAC), Sales and Inventory System, Dictionary and hundreds of different programs!

Now I really believe that I can make my program look and feel like the *real* thing. My dream of becoming as rich as Bill Gates is back on track!

Those are the three most important programming milestones in my programming life. There are also others but they are not as enlightening as the those three. The other milestones in my programming life happened when I discovered the following:

• Pointers
• Relational Databases (Microsoft SQL Server)
• Serialization (Marshalling)
• Socket Programming
• Cloud Computing (Web Services)
• .NET Framework

Also, another milestone. Today was the very first day that I celebrated our National Labor Day (May 1). It is funny that I celebrated Labor Day by coming to work.

Happy Labors Day Everyone!

My First Official "Coding Horror" Encounter

Back in College, I have a friend who’s programming prowess is so godly that it can surpass or match mine . He is a C++ Junkie and belongs to the Java Cult. His mental agility is so great that he can beat me in writing complex algorithms anytime of the day. In fact, I chose him to be the “Architect” for our project in Compiler Theory Class (Kompayler Studio 2008: The First Tagalog Kompayler, Programming Language and IDE). He is THAT great of a programmer… BUT

Being a Great Programmer is not the same as being a Great Software Developer

“Aren’t programming and software development the same?” Uhhmm, NO. Programming is just a single part of Software Development. Software Development, on the other hand, is an entirely different song and dance.

In one of our converstations, I told him that instead of reading books about programming he should try to read books about software development. I gave him that advice not because I want him to suck in programming (I’m not that envious). I gave him that advice because I want him to realize that programming alone won’t make a great software product and that programming prowess alone won’t give him the best programming job around… at least here in the Philippines.

One of our main differences is reflected on how we write code (which eventually shows in our software). For example, say that we have a project that requires us to deliver a Simple Number Sorting Application that will ask the user to input 50 numbers and sort it. This project is resource constrained in such a way that we need to deliver it in 2 days.

What I would do is use a pre-existing bubble sort algorithm available in the class library (in .NET, I suppose), then I would add additional functionality, then create a test harness (unit test) to test my functions and then I’m done with the “coding” part. Then I would use the rest of my time to design an intuitive user-interface, provide a user manual and work on the packaging.

My Simple Sorting Application

On the other hand, my friend would create a complex multi-threaded sorting algorithm that will have a runtime efficiency of  O(Log N). Not only that, it will be written in Haskel (or any other hardcore programming language) and it will also be invoked via command-line! So hackerish.

His Complex Sorting "Tool"

My point is simple… In Software Development, there is a thing called Tradeoffs. What do I mean by Tradeoff? Take the above example. I traded-off the uber-cool complex multi-threaded sorting algorithm to USABILITY AND FUNCTIONALITY which, by the way, has more practical and positive impact to my users. You could argue that I suck for using bubble sort and not the complex multi-threaded sorting algorithm and you could also argue that my bubble sort has a runtime complexity of O(N^2). But being a Software Developer, I know that using that “complex multi-threaded sorting algorithm” would only be faster by 0.000000000000001312 seconds (remember, we are only sorting 50 elements) and that “improvement” will not be of any significant use to my end-users and I might as well use my time in creating extra  features, such as Print and Copy to Clipboard, that will surely make my product more functional and easy to use.

At work, I am working on a project to “migrate” our products to Vista. To Migrate meaning “to make it compatible with Vista”. One of our product has an Autorun executable which will decide which “components” (SQL Server, VC++ 8.0 Redistributable, .NET Framework) to install. During a routine test run at the Test Deparment, they found out that this certain product is not working on Vista and filled a Track against it. The said Track was assigned to me and I’m expected to fix it. So I pulled the Visual Basic 6.0 source code of the Autorun executable from our TFS and performed an investigation. While investigating, I saw a very weird block of code…

Private dim VCREDIST as long
Private dim NETFRAMEWORK as long
Private dim SQLSERVER as long
Private dim COMPONENT1 as long
Private dim COMPONENT2 as long
Private dim COMPONENT3 as long
Private dim COMPONENT16 as long
 
VCREDIST = 1
NETFRAMEWORK = 2
SQLSERVER = 4
COMPONENT1 = 8
COMPONENT2 = 16
COMPONENT3 = 32
COMPONENT16 = 65536

If you see a 1 or a 0 in the code, its pretty normal because you know that it is a flag for something . But if you see a 7 or 16 or 32, you know something is wrong. So I ignored that and went straight to the actual function that does the shell execution to install the component and I saw something like this…

While ComponentInstalled != 13
 
	installNum = GetNextComponent	
 
	if installNum AND 1 then
		InstallComponent(installNum)
	else
		installNum = installNum * 2
	end if
 
End While

I was dumbfounded and stupified, why is there an “AND 1″ in that code? and what is the purpose of the * 2? So I asked my manager for his input. And after staring at the code for 30 minutes, my brilliant manager shouted “That code is using BIT PATTERNS!“. Instantly, I understood what the code does.

VCREDIST = 1                 '0000 0000 0001
NETFRAMEWORK = 2        '0000 0000 0011
SQLSERVER = 4              '0000 0000 0111
COMPONENT1 = 8           '0000 0000 1111
COMPONENT2 = 16          '0000 0001 1111
COMPONENT3 = 32          '0000 0011 1111
COMPONENT16 = 65536    '1111 1111 1111

That block of code is actually a binary pattern used for flagging the components. Each placeholder in that binary pattern is a FLAG (1 or 0) which actually tells the function GetNextComponent if that component should be installed or not. The *2 is used for shifthing the binary pattern so that the active placeholder for a certain flag will be at the rightmost (last bit) of the pattern! Brilliant!

To solve the issue, I just needed to force the Autorun to Install VCREDIST in Vista by adding this line of code in the Vista Condition of the GetNextComponent function:

VCREDIST = VCREDIST OR 1

So it will always install the Visual C++ 8.0 Redistributable in Vista. The actual problem is that the Autorun Executable thinks that Vista does not need the redistributable and will not install it.  That’s why our product that needs the redistributable will not work.

I have great admiration for the brilliance of the programmer who wrote this. The method is so efficient that instead of declaring 12 Integers for Flag, he would only need to use a single long! My manager told me that the one who wrote this application is actually a “CONTROLLER” programmer. A type of programmer in our organization that writes the firmwares for our hardware controllers.

When writing firmwares that needs to be tight and needs to run in realtime, time complexity is your biggest enemy and you need to code against that. But in an Autorun application for installation, you DONT! In this example, Readability and Maintability was traded-off to efficiency that was not even needed! In turn, a simple issue that can be fixed in a matter of seconds was fixed in half a day.

As Software Developers, we should have the right mindset to decide on what trade offs are necessary. We should not only be Great Programmers who can write great codes but we should transcend to being Great Software Developers who can “take a concept, build a team, set up state of the art development processes, design a software product, the right software product, and produce it. Not just any software product: a high quality software product that solves a problem and delights your users. With documentation. A web page. A setup program. Test cases. Norwegian versions. Bokmål and Nynorsk. Appetizers, dessert, and twenty seven eight-by-ten color glossy photographs with circles and arrows and a paragraph on the back of each one explaining what each one was.”

As for my Friend, he is now working at Oracle. My only wish is that he would be able to work on their database management system where efficiency has higher value than code-readability.

The training is about Windows Communication Framework or WCF — Microsoft’s answer to building Service-Oriented Architectures (SOA). Gone were the days of Web Services, more so, socket programming! It’s all abstracted now. Yay for lame programmers!

But anyway, the training was GREAT! I was able to learn the ins and outs of WCF. But what’s interesting is that I learned a new “way” to do rollbacks automatically! It’s called…

TRANSACTIONS!

But what’s more interesting is that Transactions is not really part of the WCF nor the.NET 3.5. It’s an existing feature from .NET 1.1! )So what’s Transactions? Well, its really hard to explain so I’ll just present a scenario on how and where you can use Transactions.

Let’s say that you have a Point of Sales (POS) system that have 2 tables namely Inventory table and Sales table. And you have a method called CustomerBuysStuff(int ItemID). The said method will be called if a customer buys new stuff (obviously)… so the implementation would be something like:

1. Deduct 1 unit from the Inventory table using the ItemID
2. Add 1 unit from the Sales table using the ItemID

This means that this particular method must execute two (2) SQL statements and both of them must SUCCEED or you will have a decrepancy on your data. If you can remember from your Database Management Course, you have studied a concept called ACID. This scenerio represents the A in ACID — Atomic Operations. In our case, this means that “All statements in a group must execute, or no statement in a group must execute. “

Frequently in a database application you come across a situation where you need to execute two or more SQL commands in such a way that if any one of the statements fails, then no other statement will be able to change the database.

I’ll try to demonstrate Transactions using REAL code (Click the image to enlarge):

Let’s call the code snippet as “No Rollback”. This assumes that you are 101% sure that both operation will succeed that you did not even bother to write a rollback method if ever one of the two fails. This is a very bad practice in the real world because it will create a discrepancy in your data that cannot be traced — and worst, debugged.

The above code snippet is called “With Rollback”, because in this code snippet, when the second command fails, you cared enough to rollback the changes made by the first command. This is how I did it in the college days when I’m too lazy to use Stored Procedures (and TRANS/COMMIT) and I’m too stupid to not have researched about Transactions! Oh the dark ages. Mind you, this method is not 100% fool-proof. What will happen if your rollback command also failed? Discrepancy!

Now, let’s call this code snippet “Almighty Transaction”. In the above code, there is no extra method or SQL command to invoke to do the rollback when the pre-requisite part of your code fails. And oh, there’s also no extra code to detect an exception or an error. It just magically knows that something awful had happened and it will just magically rollback ALL the changes that happened before the error. Like, MAGICALLY! Isn’t it freakin great?!

Also, this will work on ANYTHING (Not only on Database Related Stuffs). During my thesis days, I have to write a method to perform rollbacks when my encryption/decyption algorithm fails. It took me a whole week and lots of messy code to make it work perfectly. If only I have the knowledge of Transactions then, it could have just taken me 5 minutes. DAMN!
So yeah… I love these TRAININGS (Have 6 more to go for this year)! You can learn lots of useful stuffs in a very short span of time (Like 3 days). They will even give you a certificate that is signed by Billy-G himself! Haha! Another additional line in my Resume.

What’s great about the training company that my company chose is that they are official Microsoft Partners… This means that I can take the official WCF Online Test and if I pass, I will then be a “Certified” MCP (Microsoft Certified Professional)!

Remember…

TransactionScope AtomicScope = new TransactionScope()
using(AtomicScope)
{
//ATOMIC OPERATIONS
AtomicScope.Complete();
}