In my last post on this topic, I looked at some of the commonalities between learning to play a musical instrument and writing software. In each case, there are “rudiments” to be learned, boring though they may be, that help one become great at either craft. In this second installment, we’re going to look at what happens when you’ve finally learned to play and decide you’re ready to take the next step: Composition.

Finding Your Own Voice
There us a huge difference between playing the notes written on a page and making music. When we find a piece of music we really like, more often than not it’s the feel of it that draws us. More than the simple, linear arrangement of notes or chords, it’s the interpretation of those notes through the act of performing that makes a simple sheet of musical notation actual music.

For many musicians, this is more than enough. But for some, the satisfaction of simply playing music written by others wanes quickly and they seek to find their own voice, to express their own ideas, their own reflecting glass for their experiences. So they turn their energy to musical composition.

To be a successful songwriter or composer, you need:

• Experience: First and foremost, you must have mastered the art of playing music before you can master composing music. I’m certain that this is an arguable point (I’ve never seen a point that wasn’t), but in my experience, those who can’t play generally can’t write very well either. They tend to have unrealistic expectations of the musicians that will need to take what they’ve written and perform it. They also tend not to have a good feel for what musical structures fit well together and sound most natural to the human ear. It’s that sense of recognition and comfort, after all, that differentiates what people consider music from background noise.
• Creativity: This seems like kind of an obvious one, but you’d be surprised how many people I run into that try to write music based on an astoundingly detailed knowledge of musical theory. Certainly, theory provides you with the tools to understand the what of music, but the how, and more importantly, the why must come from the mind and imagination of the composer. Without creativity in composition, we’d still be listening to waltzes and chamber music, the lute would still be the string instrument of choice and percussion would be optional (though some may argue it is anyway). Human creativity pushes the envelope and fosters an environment where the art can evolve beyond the status quo.
• Knowledge: This is related to experience, but is slightly different. Imagine you’re composing a piece for a major symphony orchestra. Think of all the individual instruments for which you’d need to write parts: Violins, violas, cellos, basses, winds, brass, percussion, etc. And within each instrument group, you need to write the first part, second part, etc. If the only instrument you’ve ever studied is the clarinet, then you’ll likely write a fantastic clarinet part, but will likely find it very difficult to successfully write all the other parts in a way that leverages the strengths of each instrument and creates one unified piece of music. Understanding and knowing the composite parts of the larger ensemble of instruments will certainly lead to a more cohesive and sonorous piece of music.

The Composition of Architecture 
Musical composition clearly has parallels to the software development process when you consider the role of the architect, vague though the definition of that role can often be.

In my experience, I’ve found it very unusual to see successful architects that didn’t start out, in some capacity, as software developers. What I find to be the case more often than not is that the best developers on a particular team gravitate into the role of the architect, either de facto or de jure. I can’t speak for all architects that followed this particular career progression, but I can speak for myself.

I was drawn to software architecture for a couple of reasons:

1. Writing software handed to me and designed by someone else started to hold less interest day-to-day for me
2. I felt I had good ideas for designing systems and wanted to be in the position to express them

In other words, I wanted to stop just being a “performer” and wanted to become a “composer” because I saw it, primarily, as a venue to better express my technical ideas. That being said, I still code as often as I can (which isn’t very often these days, but I do find time now and again). I’ve never met a composer who didn’t continue to play on a regular basis, even though it wasn’t their primary vocation. We do this because it keeps us sharp and in touch with the ultimate results of our work. If we can’t play it, it will be difficult for others to play it. Some of the worst architectural designs I’ve seen have come from architects who never paid their dues in the trenches writing code and making software work on a real project. I once had a friend who told me, “True wisdom comes from great pain and suffering.” Over the years, that dictum has been proven true time and time again.

I think that creativity is one of the most overlooked characteristics in successful software developers and architects. When we think of people who work in software, it’s easy for us to picture the long-faced nerd with an overbite, thick black glasses and a picket protector tucked into the front of wrinkled white dress shirt, purely analytical and socially inept. While the industry certainly has its share of those archetypal individuals, it’s also filled with mainstream, highly creative people who simply love the malleability of software and the ability to make it into just about anything you want or need it to be. Like music, software architecture has “theory” (we tend to call it “best practices”) that, if followed, get you moving in the right direction, and ultimately it’s the creative use of those guidelines that allow new ideas, techniques, and yes, best practices to be tried and vetted. It is creativity, I think, that makes a lot of musicians and other folks with artistic backgrounds who work in the software industry so successful. Because the tenets of theory are more guidelines as opposed to hard rules, musicians tend to be more comfortable experimenting with things that haven’t been tried before. Musicians don’t have a corner on the creative market, of course, but for the most part they have active, creative minds that can drive the “next big idea” to make developers, architects and the products they produce better. When you stop to consider something like test-driven development (TDD) and the progenitors of that movement, it’s easy to see the important role creativity can play in building better software. Many people would call this “Thinking outside the box.” A creative approach to developing software precludes the idea of a box in the first place.

Certainly, the role of knowledge in software architecture should be self-evident. In this case, it also warrants a discussion of its own and will be the topic of the next and last post in this series: Teaching.

The first time I saw Ron Jacobs talk about service-oriented architecture was at Microsoft TechEd 2004. He did a talk with John Devados and it was the first time I can remember someone talking about services in a way that made sense to me as a developer as well as an architect. It seemed that every time I went to see a talk on SOA, I was looking at a lot diagrams of clouds with lines connecting them to other clouds. Any questions about the practical application of services was met with the answer, “Well, it depends.” Ron was the first person I ever saw who offered real, prescriptive guidance on how to design and implement properly factored services, and it had a big impact on how I’ve approached SOA ever since.

Beyond being an in-demand conference speaker, Ron is also the creator of ARCast.tv, the show that I and several other architects at Microsoft got the chance to co-host just a little over a year ago when Ron moved-on to become a technical evangelist for the Windows Communication Foundation and Workflow Foundation product teams. Heck, I even had the opportunity to do an interview with Ron back in 2006 at the Microsoft National Architecture Forum in Vale, CO.

In our newest episode of ARCast.tv, Ron gets to sit in the interviewee seat and talk about some of the key differences and benefits of SOAP-based and RESTful services, and how Windows Communication Foundation supports the implementation of both types of services.

User experience.

For so long, many of us in the software industry felt that it just didn’t matter. The attitude of “build it and they will come,” especially in the enterprise, was a little too common. Software users today are becoming more sophisticated as they use their computers more often to do more things, and their expectations of the experiences they should have their software and the websites they visit are rising. As developers and architects, the status quo for so long will no longer be good enough, and we ignore this fact at our own peril.

This past summer at Microsoft TechEd Developer 2008, my buddy and colleague, Bob Familiar, sat down with author, speaker and “Supreme and Exalted Dictator for Life” of Rolling Thunder,  David Platt to discuss, well, why software sucks. In this interview, David talks about the growing divide between software developers and their customers and provides some practical ways to make software suck less.

David is an extremely funny guy and this interview is very entertaining while also providing some great insights about usability issues with software today.

Check out the interview and be sure to visit David’s site, SuckBusters, and report any particularly sucky applications or web sites you’ve come across!

Are you free this coming Monday night? If so, I’d like to ask you out…to attend the St. Louis .NET User Group meeting. This month’s speaker is Jeff Fattic, a Technical Team Lead at Quilogy in St. Charles. Jeff’s talk is entitled Application Lifecycle Management (ALM) Exposed. Here is the abstract:

Most statistical organizations estimate the historical rate of software development project success at 33%! What factors contribute to this? Poor requirements, lack of formal process, insufficient testing, insufficient training, and much more! Application Lifecycle Management (ALM) is the combination of formal process and specialized tools used to make software development projects more successful. While it requires some additional effort and investment, it can help your development department get to market quicker, meet governance criteria, improve quality, meet and beat budgets and schedules. In this discussion, I hope to show you how to use Visual Studio Team System to cover many of the aspects of Visual Studio Team System.

ALM is the most important topic that not nearly enough people discuss, and Jeff brings considerable experience to the table in this discussion. This should be a great talk and I hope to see a full house. Here are the meeting details:

Date:
Monday, January 26, 2009

Location:
Microsoft Offices
Three City Place Drive, Suite 1100
St. Louis, MO  63141

Schedule:
5:30 PM – 6:00 PM: Welcome
6:00 PM – 7:30 PM: Program

This month’s meeting is sponsored by Talentporte. I’ll see you there.

au⋅top⋅sy  [aw-top-see] noun

1. inspection and dissection of a body after death, as for determination of the cause of death; postmortem examination.
2. an analysis of something after it has been done or made.

Source: Dictionary.com Unabridged (v1.1)

On Monday, January 5, 2009, Anne Thomas Manes, a well respected analyst at the Burton Group, wrote a post on her blog pronouncing SOA to be DOA. My colleague, Hanu Kommalapati, brought the post to my attention and I was getting ready to dig in my heals and put up a fight. Once I read Anne’s post, I found that not only was a fight unnecessary, but that I agreed with most of her points.

Seeing that SOA has been one of the most visible and widely talked about technology initiatives of the past decade, the fact that it has been pronounced “dead” would seem to require an autopsy to determine the cause of death and discover any foul play that might have been involved in its untimely demise.

Positive Identification
Before we begin, we need to make sure we know the identity of the corpse. For the context of this discussion, we’re talking about “SOA,” not “service-oriented architecture.” “Aren’t those the same thing?” you might ask. Again, for our purposes here, I’m going to make the following differentiation.

Service-oriented architecture is a technical philosophy combined with a practical approach to building distributed, loosely-coupled systems, focused on business capabilities that embody (no pun intended) the following four tenets:

1. Boundaries are explicit
2. Services are autonomous
3. Services share schema and contract, not class
4. Service compatibility is based on policy

In other words, service-oriented architecture represents the best practices for building services and the systems that consume them. Fortunately, service-oriented architecture is alive, well and thriving in new ways thanks to cloud computing and software + services, but more on that later.

SOA is the ubiquitous panacea presented and re-presented over the course of the last ten or so years, a business cure-all that has generated a lot of hype and attention, but few tangible, measureable results. At some point in the middle part of this decade, SOA extended far beyond simply building a solid service-oriented infrastructure. It began to involve things like governance councils, enterprise service busses (ESBs) and a myriad other things. It was a simple, eloquent concept made huge and untenable.

“SOA” is the body on the table, so let’s begin.

Cause of Death #1: Morbid Obesity
It’s not only becoming the primary cause of death for people in the United States, but it’s also the number one reason SOA projects in the enterprise fail. Over the years, the term SOA has gained a reputation as being a kind of magical elixir, capable of curing bleeding budgets and making IT organizations young and virile again. It has been pitched by good intentioned people to be the ultimate solution for solving system integration challenges and as a holy grail of enterprise reuse. Additionally, many of the top industry analysts and journalist were saying very much the same kinds of things. These factors created a lot of top-down interest in SOA within large corporations, which means that there was a lot of money being put behind trying to implement SOA.

How do I know this? Well, I was one such “good intentioned” person.

A few years back, prior to my joining Microsoft, a group of fellow architects and I set out with the intention of trying to make things better for the IT organization in our company. We started out with a simple, self-imposed charter:

• Create a set of best practices for designing and developing services for software project teams
• Develop a means of making deployed services visible to project teams
• Evangelize service-oriented architecture to project teams, creating a common understanding of the principles and a specific context for our business

A simple, tactical approach, nes pas?

After awhile, management got wind of what we were working on and wanted to officially mandate the effort to ensure we were putting something together that would truly serve the entire organization. Once that happened, we found it impossible to keep our effort lean, tight and focused. The small group of five people that started the effort soon “blossomed” to eighteen people, with extended optional members, most in upper management, that would chime in when they felt it necessary.

Suddenly, we were trying to solve huge strategic problems with SOA. We were tasked with standardizing  processes across several departments and business units. We were talking to dozens of people about services that would standardize key business capabilities in their part of the company. We were developing taxonomies, dictionaries and registries to manage an eventual service environment. We were talking about governance standards and protocols and trying to answer questions like, “How do you get services approved for enterprise use?” and “How do you submit your service for deployment into the enterprise infrastructure?” and “What is your SLA for the service and who owns it?” and “If that group owns the service, who owns the SLA for the underlying data?” And while we were doing all of this, we were not doing one very important thing: Designing and developing services.

Fifteen thousand empty calories a day, and suddenly our initiative was obese and hypertensive.

While this specific situation comes from my personal experience, I by no means think it’s unique. I’ve seen and heard this same story time and time again. Activities that would be absolutely ludicrous in a typical software development project become commonplace as soon as we start talking about SOA. The result is good intentioned people get mired down in politics and alternative agendas until the effort as a whole finally collapses under its own gargantuan weight.

Cause of Death #2: Cultural Shift Syndrome 
The SOA effort of which I was a part was similar to other failed efforts for another key reason: The focus tends to be on the technology, not on the people and culture of an organization.

In general, services are not complex or difficult to build, deploy and manage. However, implementing SOA in the enterprise requires a different way of thinking about enterprise software development. Developers, architects and project managers have to begin thinking about the development process in a more holistic way. They need to include existing services in their system design while looking inward to determine if there are new services their project can produce to benefit the organization.

In general, this is fundamentally different from how just about every enterprise IT shop in the world operates, where projects are funded by customers, either internal or external, and project managers work a project to that level of funding. Being responsible for using that customer’s money responsibly causes project managers to focus on the requirements of their project exclusively and incents them to minimize risk at all costs. Relying on services outside the team’s control introduces risk and could, therefore, be discouraged or even banned by project managers. This, in turn, encourages the creation of multiple siloed applications sprinkled throughout the company. Ultimately, this might work very well for individual customers, but makes a cohesive and cost efficient enterprise architecture impossible.

The bottom line is that technology is pretty simple to change, but changing people is extremely hard, especially when the funding model of an organization provides incentives against sharing and collaboration. Trying to approach the classic view of SOA in the enterprise presents the non-trivial challenge of trying to modify an organization’s culture and long-held opinions and policies about how it does it’s day-to-day business. If this fundamental change is a requirement for SOA success, then most initiatives would be over before they began.

Cause of Death #3: Bad Medicine
I remember once sitting in a conference room with my SOA team listening to a major software vendor (whose name I will not mention because I have shame, even though they didn’t) explain that they were there in the hopes of “selling us SOA.” After I finished my fit of laughing and asked for clarification, they stated that, by buying their suite of SOA software, we would have everything we needed to implement an SOA. This wasn’t unique to that vendor either. Many big name companies were actively engaging customers purporting that implementing a SOA was a matter of buying software licenses and/or hardware from them.

This, of course, is ridiculous, but was demonstrative of how crazy things had gotten in the SOA world. Certainly, third party software had a role to play in helping an organization build and manage a service-oriented infrastructure, but the idea that it was all that was needed was disingenuous at best, fraudulent at worst. The truly unfortunate part of this was that many companies looking to streamline their path to SOA goodness purchased these products and services only to find that, in the end, they still had most of the same problems they had at the outset, now they just had better visibility and reporting for them.

The end result was that companies often spent millions and took months (or even years) to evaluate, purchase and implement these SOA solutions. In today’s economic climate, it’s little wonder that many of these companies are drastically reducing their SOA initiatives or simply throwing them out, chalking it up to experience and moving on to other, more pressing things.

SOA: Dead, or In a Coma?
So, now that we have identified the leading causes of SOA’s demise, we need to ask the question, “Is SOA really dead or simply in a coma due to blunt force trauma to the head?”

Honestly, I have concerns about making the proclamation, “SOA is dead,” because what it stood for at its inception was not only good, but still very relevant today. Seeking to build a loosely-coupled enterprise service layer deriving from key business capabilities as part of a larger IT infrastructure is a good idea and at the heart of what SOA once stood for. By pronouncing SOA dead, I do not want to give people the impression that this vision and perhaps even services themselves are dead as well. The service-oriented approach to software development is not only still alive, but gaining momentum and expanding in the form of cloud computing.

Services are also vital to perhaps one of the most important trends we’ve seen in technology in quite awhile: Software + Services. Again, one of the major misperceptions about SOA was that services in and of themselves would solve all of IT’s problems. The S+S approach to architecture provides a model that uses services to augment client applications running on potentially multiple platforms and using a plethora of technologies. This model has the potential to provide huge benefits to companies, leveraging a services layer to provide commonality of key business capabilities while fully utilizing the power and versatility to deliver those capabilities on clients throughout the enterprise.

Instead of pulling the sheet over SOA’s head and sending it to the morgue, I propose that we look at SOA like it has been in a deep coma, at the mercy of forces operating all around it, and recently woke up with amnesia and is starting anew, getting back to basics. As SOA leaves the hospital and starts to reintegrate with society, it should stay focused on its core principles:

• Stay Tactical: It’s far too easy to start out trying to make a bowl of soup and end up attempting to boil the ocean. Focus on activities that deliver real solutions to development teams. The time to take a larger, more strategic view will be obvious when it arrives.
• It’s About the Services, Stupid: Related to the first item, keep your focus on designing, building an deploying good services in your environment. As an organization, seek to find a way to catalog and expose the availability of different services to reduce functional redundancy and increase reuse and consistency.
• Focus on Architecture: Building reliable and extensible system architectures is hard, and no silver bullet exists from any software vendor that will supplant the need to think hard about the services in your environment and the role they should an can play to software development in your enterprise. Always remember that good architecture is platform independent.
• Evangelize…Gently: You will never have 100% success in getting people to think in terms of services. Become an advocate and help people understand the benefits of service-orientation without setting mandates or imposing policy. Success with tactical level services over time will eventually lead to policies that make sense and everyone can live with.

Ms. Mannes summarizes these suggestions nicely in her post:

But perhaps that’s the challenge: The acronym got in the way. People forgot what SOA stands for. They were too wrapped up in silly technology debates (e.g., “what’s the best ESB?” or “WS-* vs. REST”), and they missed the important stuff: architecture and services.

So I say: “The old SOA is dead; long live the new SOA!”

My friend and colleague Clint Edmonson is working with community leaders to put on a great learning event called SharePoint Saturday on February 7th at Centriq in Leawood, Kansas, just outside of Kansas City. This event, targeted primarily at SharePoint developers and architects, will cover a wide variety of topics and will be presented by a great assembly of SharePoint luminaries, professionals and Microsoft MVPs.

Here is the current lineup for the day:

As you can see, there will be something for everyone who works with SharePoint. To learn more about SharePoint Saturday, check out the event’s web site. If you’re ready to register for this free event, you can do so here.

So, what do software architects really do anyway? It sounds like a question that should have an easy answer, but put five of us in a room and ask us about the types of work we do on a regular basis, I can almost guarantee that you'll get five different answers:

Q: Do we design software applications and systems?
A: Yes.

Q: Are we thought leaders in our respective organizations?
A: Definitely (or at least we try to be).

Q: Do we mentor and help other developers and architects get better at what they do?
A: Sure.

Q: Do we advise senior management in areas of corporate IT strategy, technology adoption, software standards and governance?
A: More than I like, but certainly.

Few other roles in the professional world have such a wide and varied set of potential responsibilities. In fact, it can be argued that software architects are what their organizations need them to be, truly geeky embodiments of the phrase, "Improvise, adapt, overcome."

Last June, I had the pleasure of participating in a panel discussion at TechEd Developer 2008 in Orlando, Florida with four of my colleagues: Joe Shirey (moderator), Michael Stiefel, Miha Kralj, and Patrick Weikle. The topic of discussion for that session was the role of the architect and, as you can clearly see from this panel of "experts" on the subject, there was some consensus, but quite a bit of disagreement as well.

I've had some people tell me that they think this conversation is frivolous and a distraction from doing the "real work" that needs to be done in the world of software. To a certain degree, I understand where this point of view comes from. However, the role of the architect in software is very new, tracing its roots back only thirty years or so. When you stop and consider the amount of time the role of the architect in the structural engineering and construction world has been around, it's no surprise that there is considerably more clarity around what they do. I believe that only through having this discussion with a wide variety of architects from all manners of corporations, educational institutions and government organizations from around the world will we ever be able to get closer to understanding the real value that the architect can bring to IT.

This panel discussion was posted on ARCast.tv yesterday on Channel 9. Please take a few minutes to watch the video and join in on the conversation, either here on my blog or on Channel 9.

First, I'd like to thank everyone who came to my talk last night at the St. Louis .NET User Group meeting. I was nice to see so many people from the community come out and support this group. I hope that my presentation helped clarify what software plus services really means from and architectural and development standpoint, as well as providing you with some ideas on how you might apply some of these principles to your work as well.

As promised, I've uploaded my deck from last night.

There was also the matter of the Day In the Life... video that I showed, but could not get the sound system to work. So, since we spent some time last night talking about the Silverlight Streaming Service, I went ahead and uploaded the video to the service and embedded it below.

By all means, feel free to comment or send me an email if you'd like to talk more about S+S.

Unfortunately, most people still look at SharePoint as an "intranet in a box" solution for quickly putting up team home pages and managing documents. The truth is that SharePoint Server 2007 has evolved into a rich development platform that can be extended and utilized in your custom software projects. If your project has requirements for search, content management and/or membership, MOSS can save time and money on the project by providing that functionality which can then me implemented as part of your solution.

A couple of months ago, I had the opportunity to sit down with Sundar Swaminathan, a Solutions Architect at Quilogy in St. Charles, Missouri, to talk with him about his experience as lead architect on the project to redesign Energizer.com (the web site that goes on and on and on...sorry, I couldn't resist!) to run on top of MOSS. Fortunately, I had my camera gear with me and we were able to turn it into an ARCast.tv segment.

ARCast.TV - Sundar Swaminathan on MOSS for Public Facing Web Sites

Please feel free to join in the discussion. Leave a comment either on the ARCast,tv site or here on my blog. Enjoy the conversation and thanks for watching!

For those of you who didn't get the memo, Ron Jacobs is no longer hosting the very popular Channel 9 show, ARCast. After spending a couple of years traveling the world talking to hundreds of practicing architects, Ron decided he needed a change and is now a Technical Evangelist in Redmond working with the WCF and WF teams.

Because of the popularity of ARCast worldwide, I and a small team of fellow architects from across the country have been working over the past several months to bring into being what we're calling, "ARCast 2.0." Instead of having one regular host on the show, any architect evangelist in the world that has a story to tell and is willing to sit down and record it can be the host of an ARCast episode.

We published the first official episode of ARCast 2.0 last week. My colleague and good friend Bob Familiar sat down with Simon Guest, Senior Director of the Platform Architecture Team at Microsoft, and had a great conversation about what his team does and what types of content and activities his team will have for the community in the future. If you haven't had a chance to watch the interview yet, you can watch it here.

And this week it's my turn.

A few weeks ago at TechEd Developer 2008, I had an opportunity to sit down in the TechEd Online "fishbowl" and have a conversation with Juval Lowy about his ideas concerning interface-based design in software. Any conversation that starts with, "Object-orientation has completely failed us," is going to be a good one.

You can watch the full ARCast video here:

ARCast.tv - Juval Lowy on Interface Based Design

If you never want to miss an episode of ARCast, then be sure to subscribe to the show's feed here. Please let me know what you think of the interview.