Rate this page del.icio.us  Digg slashdot StumbleUpon

What is middleware? In plain English, please.

by

I listened to a webcast from JBoss World today with a group of people. After hearing several speakers announce new middleware products and initiatives (as JBoss is the leading force in open source middleware), one of them turned to me and asked, “Just what is middleware?” When I started to describe transaction servers and database connection pool sharing, she held up a hand and said, “No. I want to know what it is in real world terms, and why it’s a big deal.”

That got me thinking and sent me to Google to look for a short definition of middleware. I found a lot of them, but they mostly were either too vague or too dependent on the reader already having some knowledge about middleware. Then, I found this one:

middleware: The kind of word that software industry insiders love to spew. Vague enough to mean just about any software program that functions as a link between two other programs, such as a Web server and a database program. Middleware also has a more specific meaning as a program that exists between a “network” and an “application” and carries out such tasks as authentication. But middleware functionality is often incorporated in application or network software, so precise definitions can get all messy. Avoid using at all costs…

And that really got me thinking about how to describe middleware and why it matters. What I was searching for was a real-world analogy that would make sense to people with varying levels of computer and software experience. And then, it hit me.

Middleware is plumbing.

There are four ways that this is true.

First, it’s mostly invisible.

You don’t generally see much of the plumbing in your house. What you see is the water. As a consumer, you don’t see middleware. You see the web sites and the information flow that middleware make possible. This is part of why middleware is hard to define. If you live and work with software, and if you’re reading this you probably do, then you’re very aware of software packages at the top level in a logical view, such as e-commerce web applications, and packages that exist at the bottom level, such as databases and the operating system. The middle part, that plumbing that ties everything together, can seem less concrete and identifiable.

Second, it provides a standard way of doing things.

If you wanted to build your own plumbing from scratch, you could. But it’s much easier to just buy plumbing fixtures. You, as a software developer, could design and build your own application servers, database connection drivers, authentication handlers, messaging systems, etc. But these wouldn’t be easy to build and maintain. It’s much easier to make use of middleware components that are built according to established (and especially open!) standards. In middleware, these standards take the form of libraries of functions that your programs call through well-defined application programming interfaces (APIs). You call these functions instead of having to invent your own.

Third, it ties together parts of complex systems.

In your house, you have kitchens, heating systems, bathrooms, washing machines, garden faucets, etc. Each plays an important part in making your house livable. You almost never have to worry about not having running water because the plumbing is robust and reliable. It just keeps the water moving through the pipes. Middleware keeps information moving through complex web-based applications. One of its primary tasks is to connect systems, applications, and databases together in a secure and reliable way. For example, let’s say you bought an over-priced sweater at a store web site last night. What happened? You looked through various sweaters’ images, selected color and size, entered a charge card number, and that was it, right? Well, behind the scenes, middleware made sure that the store’s inventory database showed that sweater in stock, connected to the charge card company’s database to make sure that your card wasn’t maxed out, and connected to the shipping company database to verify a delivery date. And, it made sure that hundreds or thousands of people could all shop that site at the same time. Also, while it looked to you like you were looking at one web site, middleware tied together many different computers, each in a different location, all running the store’s e-commerce application, into a cluster. Why is this important? To make sure that you can always get to the store online, even if some of these computers are down due to maintenance or power failures.

There’s another similarity about your household plumbing and middleware tying systems together. They both enable you to tie together systems that were built by different people, at different times, without your having to reconstruct everything from scratch. Think about your house for a minute. If your house is older, you probably have several generations of plumbing all working together. You didn’t have to upgrade your washing machine with multiple service packs when you installed a new hot water heater. In middleware, one the most powerful approaches is service-oriented architecture (SOA) based on an enterprise system bus (ESB). As its name implies, an ESB provides a server, messaging, and APIs that function like a hardware bus. In order to integrate enterprise software applications developed at different times, by different organizations, and even communicating via different protocols, you don’t have to rewrite them to speak one consistent language. The ESB enables you to “plug” these applications as services into the bus. The ESB takes care of transforming messages between the protocols and routing the messages to the correct services. In February, Red Hat announced the JBoss SOA Enterprise Platform. This platform provides a flexible, open source solution for businesses to integrate applications, SOA services, and business events, as well as to automate business processes.

Fourth and finally, it lets you worry about other things.

When you put an addition onto a house, what do you worry about? Bathroom fixtures, kitchen appliances, flooring, colors, and how to pay for it all. It’s a very stressful process. The last thing you want to worry about is whether you want 3/4-inch or half-inch pipe, copper or PVC connectors, #9 or #17 solder, etc. With middleware taking care of all the invisible functions, you (as a software developer or a business owner) can concentrate on building software to solve your business problems and fulfill your customers’ needs.

My father once told me, “when you move to a new town, don’t look for a doctor–first thing, you find a good plumber so you can sleep nights.” It’s like that with middleware. It may be mostly invisible, but it keeps things running so a lot of developers and managers and customers can sleep at night.

30 responses to “What is middleware? In plain English, please.”

  1. Zaine Ridling says:

    Nice post, Len, thanks for clarifying this vaporous term.

  2. Anay Kamat says:

    I always wanted a good explanation on middleware systems. As per what you have mentioned, middleware systems are definitly pretty useful. However, what I’ve seen is most of the time, developers tend to use high-end middleware tools without there being an actual need for it. This results in a very complicated software architecture.

  3. Jack Smith says:

    “The last thing you want to worry about is whether you want 3/4-inch or half-inch pipe, copper or PVC connectors…”

    Except that when you do build your addition, you will need to worry about those exact isues. You’ll need adapters for attaching new metric pipes to old Imperial sizes; you’ll probably have couplers that are designed to connect plastic to copper; and you’ll also need to know the current regulations about when to use copper, plastic, or (heaven forbid) steel. In this sense, your analogy that middleware is like plumbing is surprisingly accurate. Just when you aquire all the materials and expertise in working with one standard, a new standard replaces it, and you start all over again, with the added headache of making the two work together during the transition period.

    At least that has been my experience over the past twenty years as a developer.

  4. techmatt says:

    In response to Jack Smith

    The only way a home owner would need to worry about the pipes and adapters is when there is a lack of a plummer doing it for them.
    When it comes to middleware you seem are the plummer. Therefore, it is your job to worry about it, and fix it. The end user is not worried about it. I think the explanation and comparison is perfect. One the average end user (or Boss) can understand and fits in every way.

  5. Karl O. Pinc says:

    So if middleware is the plumbing that neither the end-user nor the boss need to worry their heads about, if it’s just the stuff that connects the other bits and pieces, then how is it different from the DNS server or the DHCP server or the LDAP server or the SAN controller or the VM hypervisor or all the other components that also connect the bits and pieces?

  6. Len DiMaggio says:

    Responding to the question about why aren’t other servers (DNS, LDAP, etc.) or, for that matter, any other thing, server, system, etc. that is not part of your application considered middleware?

    One thing that I think sets middleware apart from other types of software is the way it helps you to tie together different programs/apps/services by providing a common denominator. For example, the enterprise system bus (ESB) that is referred to in the article. The ESB can take care of transforming messages between various protocols and routing the messages to the correct services. The services communicate through messaging over the ESB. Each service does not have to be programmed to be able to communicate directly to every other service as they all communicate over the ESB. Hibernate is another middleware example in that it sits in the middle, between you and the specific DBs.

    So, the water in the pipes would equate to messaging over the ESB. ;-)

    Thanks for commenting!

  7. Jack Smith says:

    “Each service does not have to be programmed to be able to communicate directly to every other service as they all communicate over the ESB”

    Correct, BUT each service which needs to communicate – directly or not – with another service necessitates the development of a corresponding adapter to translate.

    Middleware as a ‘product’ doesn’t exist. Middleware as a design construct is a fallacy. Middleware is not Messaging, although most people confuse the two.

  8. Beth Benoit says:

    Nicely done Len! I like the analogy a great deal. It hits the spot.

    It’s not necessary for the definition to be exact, or the analogy to be 100% perfect. This is a great definition for the audience for which it was intended…people who have indeed been wondering “what is middleware?”

  9. Suresh Abimanyu says:

    Good work Len!

    You made it very clear what I had been searching. Thanks.

  10. PabloEndres.com says:

    What is X in English please!

    Reading through the Red Hat Magazine I found an article with something of a Dejavu: user awareness and “education”. I’ve read it, heard it and said it a billion times; if we can’t make the users aware of the risks that they are exposed to on a dai…

  11. Scott says:

    Hmm.. The analogy makes sense but does not help understand what companies are offering. I often get confused as to what is “middle” about a company’s product. The confusion seems to arise because the product is presented in marketing speak – What the product will do for me in vague non-committal terms. I rarely hear what it is and what it does clearly described. If I did I would instinctively pick up the definition of middleware.

  12. Jessica J Erskine says:

    I totally agree with your analogy. I might have said it was like a UN translator, allowing interactions between two things speaking two different languages or, more precisely, allowing communication between two things with very different agendas. Just another thought. For the record, I’m a techie, from way back, who believes in jargonless documentation and end-user manuals and training. If real users don’t truly understand the use of technology, then we’re all screwed…there’s more of them than there is of us (the technically proficient user/programmer/developer). We (techies) need to stop enjoying the power of end-user confusion. We are not high priests and priestesses, and this is not a religion, or at least it shouldn’t be. Just a few thoughts to mull over as we try to decide what new stuffy will protect our new system.

  13. Bob Lunney says:

    And here I thought middleware was a geeky April Fools joke!

  14. Michael Toth says:

    Great Description!
    Now how can I get paid as much as a plumber makes? :)

  15. Krish says:

    Good Explanation !!

  16. TC says:

    Doesn’t anybody learn the OSI model anymore? Middleware is not new. Anything that fits into layers 4,5 and even 6 could be considered middleware.

    All this makes DNS, DHCP and definitely LDAP middleware products. Which just proves that the term middleware is just a new term to describe an old classification of products. Though there is a whole new catalog of what is considered middleware. I know this because I have been supporting middleware in one form or another for the past 20 years.

    I will say this is one of the best definitions of middleware I have ever come across and will add it to my repertoire of definitions of those things that my customers ask about.

  17. Gilbert Healton says:

    “ ‘middleware: The kind of word that software industry insiders love to spew. Vague enough to mean just about… so precise definitions can get all messy. Avoid using at all costs…’ ”

    Thank for the pin! Once “Middleware” started being thrown in the air I’ve been trying to catch it, exactly… But it’s a target that never stood still long enough to pin down. The most true and most practical definition are in your article! Unfortunately , while I love the definition you quoted, the truest I’ve seen, marketing/spewing fever will keep the term alive despite sage advice closing the quote.

    Decades of dealing with house plumbing and software plumbing made my really appreciate your analogy. Your analogy can even cover the “in the computer” leaky and frozen pipes, dripping valves, and pipes rotted so thin they break when you grasp them, I’ve spent so much time on.

    Decades of trying to “understand” made me appreciate Jessica’s post rebelling against the religion I first encountered in the alters of the glass walled rooms. The plumbing analogy is very much in line with my belief that people who understand a technology can explain it in words regular people can understand while people with lesser knowledge often hide behind walls of jargon.

  18. Paul R says:

    As a network engineer that makes his living dealing with the type of people that throw the middleware term around, I have to say I don’t agree with this definition. Middleware always seems to be represented as a message bus of sorts, not an abstraction layer. I don’t neccasarily agree with all the points of this definition, but it definitely describes what middleware should be. The unfortunate realty is that most deployments are not confined to being used as just plumbing, and people without real infeastructure knowledge try to use it to provide things like high availability and redundancy. These designs over complicate things, and should be avoided at all costs.

    Just my two cents, take it at that if you want.

    Paul R.

  19. MiddleWare says:

    Thanks for the definition. I work in middleware and I still can’t explain what I do to people. I feel like the guy in Office Space trying to explain his job to the consultants. “I make it so the engineers don’t have to talk to the sales people”.

  20. Gopi Krishna says:

    That is a nice explaination. I undertand what is a middle ware now.

  21. Adedoyin Afolabi says:

    Thanks for the explanation,i understand perfectly what middleware is now.

  22. Giri Guevara says:

    Middleware and Plumbing thats a good analogy.
    Well you could extend them to Lighting fixtures next time around

  23. Ugochi says:

    Thanks a billion, u are a mind reader!

  24. Pablo Endres’s Blog » Blog Archive » What is X in English please! says:

    […] Reading through the Red Hat Magazine I found an article with something of a Dejavu: user awareness and “education”. I’ve read it, heard it and said it a billion times; if we can’t make the users aware of the risks that they are exposed to on a daily basis, of some basic concepts, we are all screwed!. I bet all of you have lived at least once to be sitting in an class room, auditorium, web cafe or any other place and the guy or gal next to you just smacks you with the most basic of questions: What is X? in English please! That’s the moment when one of two things happen: […]

  25. Ольга says:

    Я тоже временами такое замечаю, но как-то ранее не придавала этому никакого значения.

  26. K.S.OMNATH says:

    A good and easy description.
    Thank 4 u to let me knew about the meaning of middleware.
    And i feel that the middleware is just like “AIR”.u may consider about this one..

  27. Noah says:

    This is an excellent article

  28. Margaret says:

    Thanks to Len for the valuable insight, and to all for the comments, especially TC’s reference to OSI. This article was helpful to me in the effort to scope how-to detect/log/reconcile changes made to applications in our production environment.

  29. dhyka says:

    Thank you for the down to earth explanation, it gives me a clear idea about middle ware.

  30. Bob Machin says:

    I’m very fond of analogies and this is a really good one. No chance that you might find time to do the same with the OSI model?

    all the best, Bob