Monday, April 18, 2005

Managed Code Controversy

Normally, I don't get involved in these types of debates. I have my opinion about the .NET Framework, and about which languages to use during development, but I tend to keep those opinions to myself.

I read a couple of articles today that I felt the burning desire to comment on.

Here is the first article, written by a guy named Mark Russinovich: http://www.sysinternals.com/blog/

Here is a response, written by Ken Henderson: http://blogs.msdn.com/khen1234

I don't know who either of those guys are, but I believe that Ken Henderson has to be an MS employee to have a blog at blogs.msdn.com(but I'm not sure).

I think Ken pretty accurately describes the fallacies inherent in Mark's arguments, but I think he missed a REALLY BIG point.

Notepad.exe was written by Microsoft. The program has been through at least 4 versions(win95/NT/2000/XP). Microsoft has a bunch of really, really talented developers, who work on their projects full time, and have to pass rigorous standards for quality(despite what the media would have you believe).

Notepad has been updated and optimized by the best(arguably) developers in the world for at least TEN years. I would expect that an app that has been around for 10 years would be incredibly streamlined and as efficient as possible. Every developer that touches an app should remove some running time, and add some efficiency into the code.

Compare that to the .NET version of Notepad that Mark was evaluating. It was written by one guy, Muhammad M.M Soliman, as an example app, to demonstrate how easy it is in .NET to create an application that does something.

It is ludicrous to compare a 10+ year old production app produced by the most successful software development company in the world to an app written in a day or two last year by an unheard-of single developer. I don't know how good of a developer Mr. Soliman is, and I don't mean to disparage his abilities, but I can almost guarantee that he is less skilled than the team(s) of developers at MS working on Notepad.

For Mr. Russinovich, if you want to spout off about how .NET is the end of the development world, at least compare apps that are comparable. You cannot compare a string to a number and get meaningful results, and you cannot compare MS's Notepad to Mr. Soliman's Notepad-like application. The poor choice of comparison invalidates your entire article, and removes any credibility for the points you were trying to make.

To truly describe the difference between .NET and native code, you would have to write a program in a native language(prob C++), and you would have to write the same app in .NET(prob C#). Then you would have eliminated variables such as programmer ability and application maturity. Only than can you accurately compare .NET and native applications.
Listening to: You Won - Keith Urban - Golden Road (05:21)

18 comments:

Anonymous said...

Your assumptions of what happens to an application over time (that it becomes more efficient, rather than less) isn't true in any project I've ever seen.

I have never worked on, or been near, any project during which developers consciously focused on code efficiency. Once implemented and shipped, code is usually ignored. It's a fact of life in almost any project that has a deadline. Sure, it may be tweaked when bugs arise, but never rewritten. That's the reason you see bugs ship in concurrent releases in every product on the planet. Exceptions surely exist - but 9 times out of 10, other more critical objectives (adding new code for new features, followed by fixing bugs in new code) overrule spending any significant amount of time focused on app footprint or performance.

FWIW, Notepad has actually grown in size, not shrunk. Compare NT 4's RTM version (45,328 bytes) to Windows XP SP2's (69,120 bytes) the difference isn't huge, mind you - but 10+ years of code churn isn't really visible in any net positive manner.

Cullen Waters said...

The fact that notepad has become larger over time supports my theory. The changes for notepad over the past 10 years have been the addition of more features, and updating the UI. Those code changes are going to require more lines of code, and therefore a larger binary.

I think that Microsoft has shown that their development style increases a product's stability and efficiency over time.

Windows is the prime example of this. In general, the fixes that MS releases for their OS cause the OS to be less buggy. This same process will continue to improve the apps that are provided with their OS.

Your statement that you have never been involved in or near a project that got better as time went on is a sad statement for our industry.

As an example, look at SourceGear's Vault program. Each version, from 1.0 to 3.02 has gotten more responsive and less buggy. Another widely known project this is true in is RSSBandit. The previous version was incredibly slow, but the current version is significantly faster to open and during execution.

Bugs ship in concurrent releases because they are rated as lower priorities than other bugs or feature additions.

Good software should improve with every version, in all facets. In order to add features, you have to rewrite some code, and when you rewrite code, you should be looking at ways to improve the code, not just copying and pasting.

Anonymous said...

"Your statement that you have never been involved in or near a project that got better as time went on is a sad statement for our industry."

Agreed. But note I didn't say they didn't get better - rather, I said that the opportunity to focus on performance never arose. Thems the breaks, the larger a project gets. Too many projects suffer from an all you can eat mentality for too long, resulting in feature creep that has to be carefully scaled back to be rational again. And while code is improved, rarely is an opportunity available to rewrite an entire section of an application (or an operating system) when teams are focused on major milestone after major milestone, not left for a time to earnestly think about performance uber alles.

It isn't copying and pasting, either. Code is derivative. You start with n, then n+1, then n+2. Unless you rewrite the whole enchilada with every version, you have massive code reuse with insertion and rewriting where you must make the changes to evolve.

The UI for notepad (on NT) has remained relatively unchanged since NT 4... I can't recall the last feature added to notepad. Thus it doesn't actually seem to support your theory.

"Bugs ship in concurrent releases because they are rated as lower priorities than other bugs or feature additions."

Sure, for the known bug count, but the number of unknown bugs that actually ship will grow dramatically as the girth of the codebase grows.

Anonymous said...

Ok I'm going to be blunt: if you don't know who Mark Russinovich is then you should probably not comment on his article. Lets just say that he knows a couple of things about windows and programming that you obviously don't...

Sorry for the harsh words.
/P

Cullen Waters said...

Regardless of whether Notepad is more efficient now than it was 10 years ago, my basic statement still stands.

You cannot compare a production application written by a team at Microsoft to an application written for a tutorial article by a single developer of unknown skill level.

I would be willing to bet that Mr. Soliman spent Zero time checking his code for memory usage, or optimizing his code(since the code was clearly written to be a teaching tool, and not a production app).

I am a developer, working on production code, and have worked both in Win32 and .Net, so I think I definitely have the right to comment on blog entries made by any developer.

If Mr. Russinovich didn't want to have people's comments, he wouldn't enable post commenting. The whole point of blogs and commenting is to spark intelligent discussions about topics, which is only possible when two or more people exchange ideas.

Anonymous said...

Well, Mr. Waters, I guess you already learnt in your time on Internet that you should not pay attention to the "small and direct" messages from anonymous users.

It's like that: express your opinion and show a base to it. Like you did. I've liked your blog entry a lot, and at the end of this, this time I guess most of us agree that Mr. Russinovich wasn't exactly happy.

Which, contrary to what the "anonymous" above thinkgs, doesn't spoil in any way his great works. It just seemed a opinion written in a hurry from him, and I'm sure he is smart enough to understand the message from most of the community.

You and Ken has put in words what most of us surely have thought. Great work and congrats for this blog entry. ;-)

Cheers,
Charles Canato.

Anonymous said...

Just for reference, try querying Amazon's or Microsoft's web site for "Russinovich". That should provide some background on Mark. You can also visit his company's web sites at http://www.winternals.com/ and http://www.sysinternals.com/

Happy Hunting!

elrond11 said...

I think the point here is that adding overhead to an operating system like the JVM or the CLR only hurts the performance overall. Sure it makes it easier to develop applications by speeding development time or by not requiring the developer to be as versed in the computer architecture. In the end though .NET apps will require more because it is another program running on top of the operating system. Sure MSOFT will fine tune that as best as possible over the years. When I heard about .NET it only reminded me of interupted BASIC. It seemed to me a backward direction. Sure not everyone can program in Assembly to make the most efficient programs. Fortunately the hardware technology has progessed fast enough to allow for architectural changes like this. But as a purist and someone who has some experience with real-time programming there is nothing like small and compact. Adding the CLR is not making the OS any more compact.

Anonymous said...

The whole "a lot of programmers worked on speeding up the notepad" is simply rediculous. "Defence" is even more rediculous.
There were not a single feature added. between releases of notepad. Code size grew bigger, most likely as result of changing the compiler and runtime libraries, and possibly using worst compilation settings (any programmer knows what I mean - you can compile same sourcecode into different executable sizes)
I'm is pretty sure that aside of getting bigger, it also got slower.

Of course difference is small, and of course i'm is not blaming developers for that - it is absolutely normal situation. Unlike fantasy of jedi-programmers sharpening their notepadsaber over 10 years.

Anonymous said...

p.s. my point is that .NET really consumes more ram, and will consume more ram (and you need to face that unless you choose live in fantasy world where programmers optimize notepad over years :\ ).
But said RAM is very cheap comparing to development costs, so it is more or less okay that .NET apps eat more RAM.

Anonymous said...

"FWIW, Notepad has actually grown in size, not shrunk. Compare NT 4's RTM version (45,328 bytes) to Windows XP SP2's (69,120 bytes)"

The main difference is the icon. The XP version contains a 32-bit icon which takes up significantly more space.

Anonymous said...

Just because it was written by MS doesn't mean a darn thing. They got a gazillion other projects, and I'm sure they didn't devote more resources to the notepad app than any other developer would.

Fact is that at my work, we're translating an app from VB6 to VB.NET. VB6 version uses 10MB of RAM, and starts up instantly. .NET version takes several SECONDS to load and takes up 60MB of RAM.

I'm not even going to comment on all the time it takes to dig through .NET framework to find the "new" way of doing the same old thing.

Anonymous said...

erm, I think many many people have missed something here.

Why Notepad? Quite simply because the sourcecode to this very popular (and small) application is freely available. This means that you can compare the 2 versions of notepad accurately. If someone rewrote Notepad (as is implied in the article) then the comparison would be pretty much invalid, but as it is *as much the same code as is possible to be*, the differences are down to the development tools used to create it.

The same can be said for The Quake 2 conversion someone did (and linked to on MSDN). The net result was an app that was 10% slower for the .net version with as little changes (fully documented if you go look) made as to get it compiled and running.

Anonymous said...

I am surprised to see you claim notepad is fast because microsoft had some "great programmers" working on it and its been around for a long time hence probably optimized and thus will be faster than a .net based version of the same program written by some programmer

My comments:

1. I don't know if you have even programmed in c or c++ for windows api. In this case a program written by a decent programmer will easily equal
Microsoft's version in startup speed or memory requirements

2. I hope MS has better work to do than to optimize a miniscule notepad app that's already fast :)

Anonymous said...

"but I can almost guarantee that he is less skilled than the team(s) of developers at MS working on Notepad"

Team(s) [sic!] of developers at MS, working 10 years on improvements on Notepad (which never arrived).

Did anyone see a less powerful editor ever?

(My grandma is still laughing...)

Anonymous said...

Notepad is stupid editor so you can't say it has been developed in 10 years since all features it contains can be programmed by those "excellent programmers" in one day, if they are so good.

The only difference through years is new about box and new icon.

Quote
In this case a program written by a decent programmer will easily equal Microsoft's version in startup speed or memory requirements
Not true. Microsoft knows about so called Native API, and everybody else, even those using exposed Win API are doing it through layer called Windows Subsistem. Thi s layer in turn calls Native API function which are not documented (that is, they are, but the documentation is very very rare and urcane). So you see, when MS do it, it doesn't use public API but hidden, faster...

Quote
Adding the CLR is not making the OS any more compact.
Of course not - at first.
At first, every tech is some year ahed its time so this is true for .NET. In the future NET framework will be the system not another layer. So you can expect NET programs to work as good as todays native code.

The main pooint here is how much in the future lays NET fully optimized and trasnparent.
You see, looking 1 application no matter how complex or simple it is has no meaning on my own computer: if I write 10K application that does the same job as 500K application which is better ? Which to use ? I mean who needs as small as 10K? 500KB is small and 10K is exageration :) NOT TRUE!

You don't run single application. I have 20 apps sitting in my tray right now and another 20 hidden somewhere in the system. If I have hundreed 10K applications and hundreed 500K applications, which is better? Comparing those apps isolated from environement is one thing and comapring them together is another. On top of those 100 apps I have to usualy laumch something more expensive, like Delphi or Half Life 2. You got my point.

To say this in short words: you can't say NET is bad, everybody how say that knows nothing. The only bad thing about NET is it is few years earlier (which is good if you ask me). Creator of Delphi didn't dissapointed again.

What is really BAD about NET is MS propaganda. You never heard Sun screaming Java this Java that....... that is apparent from Ken Henderson's article and I turned sikc as I was reading it since among few valid arguments only thing I could see was MS did this MS did that, MS is greatest, NET is greatest.... (not litteraly)

... and that is bools*it and the main reason why so many people don't like anything that comes from Microsoft.

Anonymous said...

Mark may be a brilliant developer, but he is also a human, and so has mixed opinions, feelings and ideologies, which may or may not contribute professionally to his persona as a developer.

What I am trying to say, is that I totally agree that he missed the point with .NET - which I also struggled to point out because of opinions like his - that managed code will win in the end anyway, because as the code pattern remains the same, it is up to the running VM to execute it and thus, get optimised over the years. Who knows, maybe we will see hardware chips running managed code, just as today we have a chip running x86 code. All the talk about more resources, memory consumption etc, will only survive until dawn.

Anonymous said...

--------------------------
Not true. Microsoft knows about so called Native API, and everybody else, even those using exposed Win API are doing it through layer called Windows Subsistem. Thi s layer in turn calls Native API function which are not documented (that is, they are, but the documentation is very very rare and urcane). So you see, when MS do it, it doesn't use public API but hidden, faster...
------------------------

So, Why is it that when I use an API monitoring application I can see notepad calling standard API functions like any other app? Unless of course you consider the unicode (W) version of functions the "Native" API.

Windows 3.1 notepad and Windows Vista notepad are exactly the same in almost every respect. the only difference is since the Core Editbox now supports up to 2GB of text, so does notepad.

I imagine they added a few resources in that time frame to support Luna and Aero themes, but aside from minor changes like that, very little changed. And when it did, chances are the dev that went in there didn't care about performance of the app as a whole, but rather adding whatever feature they were assigned to and getting out of there.


regardless of what is said- it was a fair comparison. It's a simple application. a Textbox, a few windows. some saving/loading and menus that use sendmessage to change the flags on the textbox- there, application completed.

There isn't ROOM for optimization, on the count of either version.

Look at the UTF conversion bug. It sat in notepad for over 8 years. a easy fix! So I guess they were too focused on performance?