So this is one of my pet topics: I have been in the IT industry for near to two decades.  Started out working with some assembler, wrote some low level transport layers on networks, worked with a variety of operating systems and at some point did some open source stuff on Linux , then worked with database systems etc.

At some point I entered the Microsoft world: I became and MCSD and MCSE around 1996, and at that point I actually knew all applications/ software offered by Microsoft at an expert level.   The combination off all of this knowledge and experience enabled me to architect enterprise level, high performance, high scalability applications.

Today (13 years later) it is not possible to be an expert on all Microsoft software anymore (and my guess is that goes for other vendors as well – in fact, in my opinion it is not even possible to be an expert on a single tool such as SQL Server anymore.  You need to choose a field within SQL Server, and become an expert on that.  SQL Server is just an example, this is true for most areas of the total IT industry.  An exponential increase in technology, complexity, options etc. is making it near impossible to keep up. 

A concern I have is: How do new guys/graduates catch up?  Is it still possible for a graduate to start as a junior developer and work his way up to become a good systems architect?  It is pretty much impossible to keep up as well as catch up detailed knowledge about all the layers of abstractions that senior resources have aggregated over years and years.  The only workable solution seems to be:

• You need to have a base knowledge about the abstraction layers you are working with
• You need to trust these abstractions
• This will enable you to focus on keeping up

This sounds workable, but again my critical mind is not happy.  Sure, I will entrust my “Hello World” application to someone who has this level of knowledge (And I will probably get a much better user experience than anything I can ever develop or even image ;-) ).  Will I trust this guy when it comes to mission critical, high performance, highly scalable applications?  I don’t think so!

The interesting thing about this is: I attended a session presented by Rockford Lhotka at Tech-Ed, Barcelona recently, where Rocky was speaking about “How to Manage Technology And Not have it Manage You”.  The interesting thing about the audience was that we were all mostly “seasoned IT staff”  ;-).  The above concern seemed to be a shared by the whole group. I take this as a sign that I’m not paranoid, and that the above really is an issue.

Are their any solutions/remedies to this situation?  No silver bullet available I’m afraid.  But I suggest the following:

• Junior resources should be encouraged to pick up a broad level of experience before starting to specialise in a certain technology/ architecture/ type of application.
• Junior guys that want to move on to become system architects must make it their mission to understand the mind of the guys who designed the systems they work with.  In all applications there are numerous trade-offs and design decisions – Make an effort to understand the thought processes and motivations behind these decisions. Work with stuff which turned out great, as well as things which turned out to be not so great
• Senior guys – take care when creating your API set/ abstraction/reusable module  - note that someone will at some point trust that whatever you created is doing the best job possible.