Tieu Luu's Blog

Vendors need to realize that in this day and age service interfaces for their products should be a core feature. Unfortunately many vendors view them as secondary requirements and thus don’t invest the necessary resources into their development. Often they are built in as an afterthought and usually by some junior developer on the team. The results are poorly designed service interfaces into the products with inadequate levels of functionality and poor usability. If I’m going to buy a piece of enterprise software, I’m not looking only at what functionality it provides, but also how well it will integrate into my existing environment. Vendors that get this will win the sales, vendors that don’t will fall to the wayside.

There are a variety message exchange patterns that can be used to implement an SOA. Know them and when to apply them. It can mean the difference between an SOA that scales and performs vs. one that doesn’t. The traditional synchronous request/response may not always be the most appropriate in all scenarios. Here are some examples of other types of MEPs:

• Asynchronous request/response using a callback–the consumer sends a request and provides a reference to a callback service as part of the request; instead of responding synchronously to the request, the service responds asynchronously by sending the response to the callback service
• Publish/subscribe–consumers register their interest in receiving some data that the service provides by subscribing to the service; whenever the service gets or creates that data, it publishes it to all subscribed consumers. This is somewhat of an extension of the asynch req/resp with callback MEP. The difference is that the pub/sub MEP is used in more of a one-to-many type of scenario, i.e. the service has some data that may be applicable/consumed by multiple consumers; whereas the asynch req/resp with callback is used in more of a one-to-one scenario, i.e. the consumer is requesting some data or functionality that is specific to it (not applicable to other consumers) and doesn’t want/need to wait for the response to return immediately
• Request and poll–the consumer sends a request to the service but doesn’t want/need to wait for the response so the service returns some type of identifier that the consumer can use later to poll the service for the response. This MEP is useful for when you’re trying to achieve the behavior of the asynch req/resp with callback but can’t provide a callback service perhaps because firewall settings don’t allow inbound messages.
• One-way push, aka “fire and forget”–the consumer sends the service a request and doesn’t need a response so it just fires it off and forgets about it; perhaps it would be more accurate to call it a message instead of a request since the consumer’s not expecting a response. Also it doesn’t have to be the consumer firing off the message, it can be the service doing that as well. This is in fact the publish half of the pub/sub MEP for scenarios that don’t have any type of guaranteed delivery requirements.

A lot of folks talk describe the main benefit of SOA is the increased agility that it brings to the organization; mainly describing the architectural and technical approaches and mechanisms that bring about increased agility. The problem with that is that the barriers to agility at many different organizations are not technical in nature. Take the government for example, the reason it takes them such a long time to roll out a change in their IT systems is due to the processes or bureacracy that exists not because of technical challenges. Technical challenges exist but they are not the long poles in the tent. Many of these organizations can gain increased agility by first streamlining their bureacratic processes–for example, security certification and accreditation processes that are required before a system can go live–these often require a lot of paper pushing but don’t really do much to ensure that the system is really more secure.

I came across this article in SOA World today:

Bridging the Gap between Business & IT with BPMN & BPEL
— Because the role of IT organizations is to enable business managers to run their businesses better, there has been a constant need for aligning IT closer to business. We often hear business managers complain that a software solution isn’t what the business needed. We have personal experience with this problem – in a previous life one of us worked for a consulting company that built turnkey applications for large enterprises.

It basically talks about the approach and tooling from Oracle to allow business users and IT to jointly create business process models to, as the title says “bridge the gap between business and IT”. While I have no major objections to the approach they’re describing in the article–I think it’s actually a pretty good solution to address the standards divide, i.e. BPMN vs. BPEL, in the business process space, I do object to how they envision this being used. They describe essentially a process where a business user uses the graphical BPMN-based modeling tool to design the business process and IT takes this as a blueprint to create the BPEL-based executable process.

I don’t know what kind of business users they have come across (or are envisioning), but I’ve never met any business user who’s able or willing to use any type of modeling tool to layout their business process for you. They’re too busy actually trying to run the business. The best you can expect to get from them is some type of document that roughly describes the process and it’s up to us IT shmucks to try create some type of process model out of that.

Oracle has one of the better suites for BPM out there since their acquisition of Collaxa several years ago, but let’s get real guys, it’s not the business users that are using that, no matter how graphical and friendly the tool is–it’s still us IT guys that are the ones solely using that stuff. These guys need to know who their customers are and how they’re using the product and not make up some nirvana-like scenario of business users modeling their business process and IT taking that to create the executable for alignment between the business and IT.

Those of you who’ve worked on very large scale enterprise-wide architectures know that it’s always the people and politics that are the biggest hurdles. I came across the term “politecture” in Nick Malik’s blog here. You can follow the link for the definition, but it’s essentially an architecture that’s been designed with the influence of political factors. Some may say this isn’t right, but I think it’s perfectly fine, even necessary when you’re designing a large scale architecture that has to be adopted by many people and organizations.

So what’s this have to do with the SOAP/WS-* vs. REST debate? I tend to agree that there are very many scenarios in large scale distributed systems in which a simpler RESTful approach is a technically superior solution. However, IBM, Microsoft, OASIS et al. have done such a good job touting the power of standards in the WS-* world that most folks within large enterprises are sold on the WS-*-based architecture. Whether or not those standards actually help you is a topic for another discussion, but the point is that within most large enterprises you’re going to have a much better chance of getting people to adopt your WS-* “standards-based” architecture than you will with a “maverick” REST-based architecture. Is this a problem? Maybe, but from what I’ve seen usually not. So what if you end up with a technically more complex solution? I can solve the technical problems that result from the added complexity of WS-*. Like I said at the beginning of this post, it’s the people and political issues that are the biggest hurdles. From my experience, it’s usually the people and politics that cause projects to fail or be canceled. If I can’t get people to adopt my architecture because I’m going with a “non-standards-based RESTful architecture”, I’m not even going to have a project to worry about it failing because WS-* is more complex, less interoperable, etc.

As an architect, you’re not implementing, but you need to get everybody to agree to what you’ve defined so that they will implement it. If this means making some compromises that perhaps result in a less technically superior solution, but will allow you to achieve consensus then go ahead and do it. As I’ve painfully realized in my transition from developer to architect over the past few years, things are not so black and white with architecture, i.e. you can’t always just go with the solution that makes the most technical sense. If you fail to see this or cannot adapt to this, then you will not be a successful architect.

Two of the most common mistakes I’ve seen by inexperienced people trying to implement SOA are trying to make everything a service and building a huge data model and trying to force everyone to use it.

First of all, not every capability needs to or should be exposed as a service. There are overhead and security risks with exposing something as a service. If the capability is an internal capability that isn’t used by anything outside your system, then don’t make it a service. Another way to look at it is if it’s a capability that isn’t used or shared beyond the boundaries of your organization, then again it probably doesn’t make much sense to create a service out of it. Of course there can always be exceptions, but you’re better off by starting with those capabilities that are used beyond your system and organizational boundaries.

The second mistake is committed by those who spend months (or years) building huge, really detailed data models and accompanying XML schemas for their domains and then require anybody building services in that domain to use those schemas. Don’t get me wrong, having common logical data models is a good thing and having some common schemas based on those models is also a good thing. The problem is that these guys are building these things like they’re building relational database models, with little flexibility for customization and extension. Building data models and schemas for services is not the same as building a relational model for a system, even a very large system. It takes a very different approach guys. Take a look at NIEM and ebXML’s Core Components, they were designed from the get go to support extension and customization. Their models and schemas are designed to be modular so that you can compose them into structures that match the specific needs of your service, yet they are all still derived from the same “core” such that you get the common understanding across all the services that use them. There’s a lot more to talk about with this particular issue. I’ll discuss more in a follow-up post.

In a previous post, I talked about how EA can be used to influence SOA to provide the strategic perspective. Here are some more thoughts along those lines, with some guidance on service-oriented analysis using DODAF views.

I mentioned that SOA should be aligned to the organization’s strategic objectives and that EA can help drive this. So how?

The DODAF defines a view called the OV-1 (high level operational concept) that describes/depicts the key entities (i.e. operational nodes), their activities, and interactions in support of the organization’s mission(s). So once the organization’s strategic objectives are determined, how the organization will execute on those objectives can be modeled using the OV-1. The OV-1 is a simple high level view, but the creation of it forces you to think about how the different groups/sub-organizations will interact to execute the strategic objectives. If you can’t figure this out, then you have some unachievable objectives!

With the OV-1 created, you now have the organization’s strategic objectives codified in an architectural artifact. Scoping your SOA using the OV-1 and using it as the starting point for service-oriented analysis ensures that your SOA is aligned to the organization’s strategic objectives.

As I mentioned earlier, the OV-1 depicts the operational nodes (groups, departments, etc.), their activities, and interactions in support of the execution of the organization’s objectives. Since SOA’s all about creating and sharing capabilities across different ownership domains, the key things to focus on first from the OV-1 are the operational nodes and their interactions (the activities within each operational node can be useful later for further decomposition). With these operational nodes and their interactions, you have your candidates for the service providers, service consumers, and services. From that point on, it’s just further analysis and decomposition to identify the services that comprise your SOA and who should provide them.

So there you have it–step-by-step guidance on how to ensure your SOA is aligned to the strategic objectives of your organization. Of course, this is all easier said than done.

Stay tuned, in a future installment I will discuss how other DODAF views can be used in that further analysis and decomposition.

Those of you who work with the DoD are probably very familiar with DODAF and lately have probably heard a lot about how DODAF is incompatible/inadequate for representing Service Oriented Architectures. While there may be some truth to that, I consider it to be somewhat of a red herring. DODAF is primarily used to create Enterprise Architectures (EA). The real problem (at least based on what I’ve come across) is that those working on EA’s don’t really understand SOA and vice versa. Those working on EAs are usually more strategy and management focused and don’t understand (or aren’t interested) in the principles of service orientation. On the flip side, those working on SOAs are usually very technically-focused and don’t understand many of the management and strategic objectives of EA (or find them boring and don’t want to be bothered by them). As a result the two efforts are usually separate when they need to be interdependent. So how are they interdependent? There are a couple ways to look at it.

One, SOA is an architectural style and this style can be applied in the creation of the EA. An EA models how an enterprise works using multiple layers/views, e.g. organizational, operational, technical. Using a service-oriented style, the enterprise can be modeled as a set of service providers/consumers and the services through which they interact. This approach can be taken through all the layers of the EA. This is how SOA can be used to influence the EA.

On the other hand, the EA effort can and should also influence the SOA effort. As mentioned earlier, those working on the SOA effort are very technical and implementation-focused, they look at it as “let’s build an SOA”. The SOA needs to be aligned with the strategic objectives or problems that the enterprise is trying to address. Those that are very technical and implementation-focused will usually not be able to identify those strategic objectives/problems. For example, which parts of the enterprise are very dynamic such that they would benefit from the agility that can be achieved through SOA (and can justify the cost and effort)? An EA provides a holistic view of the enterprise such that the strategic objectives/problems can be identified and prioritized which in turn should be used to scope and define a roadmap for the SOA.

So stop arguing about whether or not DODAF can be used to depict SOAs and start figuring out how to reconcile the EA and SOA efforts!

With the growing popularity of enterprise mashups, more and more organizations are starting to service-enable their data sources, i.e. building web services on top of their data sources to enable access to their data. Unfortunately, most of them are not addressing their underlying data problems first before doing so. I’ve written about this before, talking about the need for an SOA data strategy. It still amazes me to see how many people out there still think that by slapping web services on top of their databases, amazing things will happen and their problems will be solved. All that will happen is that this will bring all their crappy data up front and center on some flashy AJAX-based mashup application. We’ve seen this before in the past with BI and reporting applications that used some of the more traditional data integration technologies. After the BI app is built, the end user realizes that she still can’t do any real analysis because the data is crap.

soa, enterprise mashups, data strategy

The RESTful approach is very attractive for large scale integration scenarios that cross many organizational boundaries. This is because the constraints imposed by the REST principles emphasize interoperability and scalability. The constraint of uniform interfaces supports those scenarios in which the consumer base for the services is so broad that it makes it difficult to create and maintain a large set of custom interfaces. In such cases, it makes more sense to apply a design in which a single interface can support all the required interactions. Unfortunately, modeling everything as a set of resources that are all exposed through a uniform interface is not always easy. Developers are accustomed to designing a specific interface for each piece of functionality or data that they wish to expose; forcing them to always use a uniform interface is antithetical to this. However, scenarios in which services are just providing access to data can easily support the uniform interface constraint. This is because any kind of data can be manipulated through the same set of create, read, update, and delete operations. Thus, scenarios in which it is primarily data that needs to be shared through web services make REST an easy choice.

soa, web services, REST, SOAP