Tieu Luu's Blog

Current gig allows me the opportunity to get back and do some coding. Currently working on building some continuous monitoring capabilities for a client–new buzzword in the federal cybersecurity space. Meanwhile building a little prototype that will store device configuration data (e.g. software inventory, versions, patches) into HBase and then use MapReduce to calculate counts of devices for each particular configuration. Volume of data can get pretty large if you think about number of apps installed on machine plus patches, times the number of machines in a large enterprise, times how often these checks need to be performed. Good fit for processing using the BigData technologies, plus the structure of the data fits well into the nested key-value-pair structure of HBase.

Just came across a very good article by Marc de Graauw over at InfoQ on why WS-RM is useless.  His main point being that usually when you have reliable messaging requirements, they should be handled at the business logic layer rather than at the “transport layer”.  I encountered some similar issues earlier this year when designing a REST-based federated incident ticketing architecture.  A couple of the use cases that had to be supported were to allow one ticketing system to create/assign a ticket to another ticketing system as well for one ticketing system to make updates to a ticket in another ticketing system.  It was a requirement that these assignments and updates were guaranteed to make it from one system to another, e.g. otherwise the assignee may not get the ticket and would not work to resolve the incident.  One of the biggest arguments about REST is the lack of support for reliable messaging but this did not prove to be a problem because the design addressed it at the business logic level.  The ticket creation/assignment was implemented by POSTing a representation of the ticket to the target system while the ticket update was implemented by PUTing an updated representation of the ticket to the target system.  Ensuring that the update always made it through to the target system was easy because the PUT operation in HTTP is idempotent so we could keep resending if we didn’t get a successful response code.  Ensuring that the creation/assignment always made it through was not as simple but still was not difficult.  Because POST is not an idempotent operation in HTTP we couldn’t simply just resend if we didn’t get a successful response code, otherwise multiple duplicate tickets would get created in the target system.  To address this, we just included the source system’s ticket id in the representation that’s sent in the POST request and added a bit of logic in the target system’s implementation to first check and see if it already has a ticket with that source system id before creating.  Now we could do the same thing that we were doing with the update ticket operation to ensure that the creation/assignment always made it through, i.e. resend until we got a successful response code without worrying that duplicate tickets would be created in the target system.  So there you go–who needs some stinkin WS-RM!

HATEOAS or “Hypermedia As The Engine Of Application State” is one of the core principles of REST and it essentially boils down to having links in the representations of your resources.  How to apply this principle in the design of a service and the benefits that it provides depend on the nature of the service you are building, i.e. whether your service is more process-centric or data-centric.  What I mean by a process-centric service is a service that’s more focused on  exposing or sharing some business process or capability with its clients.  A data-centric service on the other hand is more focused on exposing or sharing some piece of data.  Of course, services can be both, but most of the services I’ve seen tend to be one or the other.  So how is HATEOAS applied to these two types of services?  With process-centric services, as the client interacts with the service, it progresses along the steps of the process until finally the desired goal is achieved.  With these types of services HATEOAS is used to enable the clients to progress to subsequent steps or states of the process, i.e. the links in the resource representations are transitions that take the client to the next state in the process.  With data-centric services, as clients interact with the service, they are accessing (reading or writing) the data provided by that service. Typically the client will also explore and access other related data as I wrote about in a previous post.  In this case, HATEOAS is used to express the relationships with the other related data, i.e. the links in the resource representations allow the client to easily traverse the rich relationships and access related data.

Over the past few years I’ve been helping a lot of government clients design and implement architectures for information-sharing.  When it comes to information-sharing one of the most important aspects is the relationships in the data being shared because it’s those relationships that provide the context to help you understand and utilize the information better.  Within an enterprise, key data types are often dispersed across multiple systems so a lot of the things we did included implementing data services (i.e. SOAP/WSDL-based Web Services) on top of each of these systems to provide shared access to the underlying data.  While this approach does help share the systems’ underlying data with the rest of the enterprise, you still end up with these web services providing “islands of information”.  I.e. the relationships across the data being shared were not exposed and made easily traversable.  This is one of the reasons I prefer a REST-based approach for these more data-centric types of web services.  By following the REST principle of HATEOAS (Hypermedia As The Engine Of Application State), we explicitly expose these relationships across the shared data as links in the representations of the data that are returned from the data services and the links also make  it easy to traverse those relationships to access the related data.  This is in fact why information-sharing on the Web is so successful.  So by using a REST-based approach to create these data services, you can in effect re-create this architecture of the Web within the enterprise to enable effective information-sharing within the enterprise.

The popularity of widgets these days has brought to attention the need for interoperability, i.e. for widgets developed for one site or platform to be able to run in other sites and widgets developed by different people to be able to work with each other. So much so that I know of at least 3 somewhat competing specifications for widgets. There’s the gadget portion of the OpenSocial specs which was adopted from the Google Gadgets work. Then there’s OpenAjax which is more broadly focused on Ajax interoperability but has a lot of pieces geared towards widget interoperability. And finally there’s the W3C’s Widgets 1.0 Family of Specifications. Based on a preliminary analysis, the OpenAjax specs appear to be the most comprehensive for widget interoperability issues but OpenSocial seems to have gained more adoption. The W3C work appears to be moving very slowly (no surprise) and it’s unclear to me how much adoption they have. Right now my preference is the OpenAjax specs based on how comprehensive it is but admittedly I have not done a thorough analysis and comparison of the three. Plus, a spec that is too comprehensive may not always be a good thing, e.g. tends to be more complex and may be trying to standardize more than what’s necessary. Stay tuned, I’ll post a more thorough analysis once I’ve had more time to parse through each of them in more detail.

I’ve recently asked one of my developers to research some integration and middleware technologies for a project we’re working on. After spending a couple days on this, he said to me “these things are all part of ESBs now”. I.e. all the integration and middleware vendors have pretty much taken these capabilities and bundled them into their ESB platforms. “Oh you need a message bus? That’s part of our ESB now.” “Oh you need this adapter? That’s included in our ESB.” In most cases when you’re implementing an SOA, you will need some piece of middleware or integration technology. With just about every vendor making these things part of their ESB suites, you’re probably gonna end up getting an ESB even if you don’t need a whole freakin’ ESB. That’s why all SOAs need ESBs (regardless of whether or not you really need them)!

The more I think about the phrase “service reuse” or any other variation of it, the more I don’t like it. I feel like it’s making me think about services from a perspective that is too technical and too low-level. Reuse is something you want to achieve with code but you shouldn’t be thinking about services as just code. When you’re designing and building your services, you need to think about it from a much higher and more business-oriented perspective. You shouldn’t be building a service because you think it’s a great piece of code that other applications may be able to reuse. You build a service because it represents some significant capability of your organization that needs to be offered up to other units within the organization or external partners so that the business can operate more effectively. It’s about sharing business capabilities and information, not reusing software. When you keep this in mind, you’ll be more likely to build services that end up getting “more reuse” because you’re providing valuable business capabilities or information that others need to use.

While services make it easier to share your data and functionality, some developer still has to write the code to consume your service and present the results to the end user. In many cases, that code to consume and present a visualization of the service can be reused by other applications. This has already exploded on the Web with Facebook widgets, embeddable YouTube videos, blog traffic counters, etc., so what I’m talking about here is nothing new.

However, within the enterprise the use of shared UI widgets is still not very prevalent. This is problematic because it still leaves the creation of useful applications out of those shared enterprise services within the hands of IT. This really does not make the enterprise that much more agile at delivering useful capabilities to the end users. Real agility is achieved when end users can create those capabilities themselves by building lightweight micro-apps that are composed of various shared widgets for consuming and visualizing those enterprise services. These micro-apps can themselves be packaged up as widgets so that they can be reused within other apps.

So the next thing to do after you’re done building those enterprise services is to start building some UI widgets for those services so that you end up with a portfolio of widgets that can be shared alongside your portfolio of services. This will extend the ability to share (or reuse) those services all the way to the end user, i.e. the last mile. You can think of this as sort of a “widgets-based architecture” that is built on top of your services-based architecture.

Vendors need to realize that in this day and age service interfaces for their products should be a core feature. Unfortunately many vendors view them as secondary requirements and thus don’t invest the necessary resources into their development. Often they are built in as an afterthought and usually by some junior developer on the team. The results are poorly designed service interfaces into the products with inadequate levels of functionality and poor usability. If I’m going to buy a piece of enterprise software, I’m not looking only at what functionality it provides, but also how well it will integrate into my existing environment. Vendors that get this will win the sales, vendors that don’t will fall to the wayside.