Business Processes for Developers
September 26, 2008 | matthieu
Any good blog introduction has to start with a Wikipedia quote. Call that Matthieu’s Rule #4 of Blog Writing (I’ll tell you about the 3 first ones another day). Quoting the big W is a good way to not have to forge a good definition yourself, so let’s proceed:
A business process or business method is a collection of interrelated tasks, which accomplish a particular goal.
A business process begins with a customer’s need and ends with a customer’s need fulfillment. Process oriented organizations break down the barriers of structural departments and try to avoid functional silos.
Business Processes are designed to add value for the customer and should not include unnecessary activities. The outcome of a well designed business process is increased effectiveness (value for the customer) and increased efficiency (less costs for the company).
Clear, no? Not so much actually. A long assemblage of semi buzzwords perfectly forged for direct publication to CIO magazine. It certainly sounds nice, all this customer’s need fulfillment and value added, could you get some for me too, please? So far, it’s always been the way business processes have been described. For people who actually have to implement a business process, developers like you and me, it’s not very helpful. A little too, how to say… light on the details side.
A business process is whatever happens after you’ve hit that “order” button on Amazon, until you receive the vacuum cleaner robot you’ve been dreaming of. Including when the item isn’t available anymore and the production line has to be notified. Or when a new supplier needs to be selected because the last one ran out of business (selling vacuum cleaner robots can be tough sometimes). Or when they sent you a pool cleaning robot instead and you have to send it back.
Another business process takes place when you buy a house. It could be a good time now, except that you might actually have more cash than your bank. So you have to visit places, select one that you like, make offers, sign a contract, select a loan, secure the loan, end your escrow, all within a precise timing. Painful little process.
In many organizations those processes are ad-hoc, disseminated between various sources, from pieces of paper to people’s brain and computer backends. Or it’s just hard-coded within your own application. And often, it’s just fine this way. But sometimes, it bites.
You know those little pieces of code that you’ve assembled so carefully and that users keep on changing their mind about? Turns out they change their mind for a reason. And modeling this logic in its own layer using specific semantics that allow easy modification can make a lot of sense. Separation of concerns apply for processes too and unfortunately for quite a few applications, is often ignored.
So what if you wanted to implement that service, helping home buyers, from selecting an agent to moving in? How would you explicitly implement the process that drives your application? If you look for standards, there’s pretty much only WS-BPEL nowadays. A few years ago you would have all those standards that would compete with each other and the market was fragmented. I hear BPMN? Definitely nice to have but I’m talking about implementation here, BPMN is not in that field. So your first step would probably be to select a WS-BPEL engine, an open source one preferably.
The thing is, even if BPEL defines the right level of semantics (I’ll come back to that point in a later post), as a language it really sucks. It’s XML all over the place, tied to WSDL, verbosity at its best with many levels of unnecessary indirection that forces you to triple check most references. Check for yourself if you want. And it somehwat makes sense: BPEL has been designed for tools to generate and other tools to consume, not for people to write. It’s a bit like Java bytecode if you want, the compiler generates it, the VM reads it but no sensible human (except the implementers) should have to look at it.
So the toolset for programmers is actually fairly poor at the moment. Don’t get me wrong, design tools are actually good but when you’re used to code, the expressiveness is limiting. Which is why we’ve started thinking about changing that. And the solution we’re currently designing will probably be organized around those main pieces:
- A Simple Process Execution Language (dubbed SimPEL) that feels like a modern programming language but with domain specific constructs.
- A versatile runtime for SimPEL that can be easily embedded in most environments. One important goal is to make it easy to use, whatever your language: Javascript, Ruby, Python, Java, PHP, … The runtime should also be able to execute BPEL, being part of Apache ODE.
- A task manager called Singleshot, integrated with the runtime and easy to invoke from SimPEL programs, to give a human-friendly face to your processes.
- An IDE with all the features you’re used to: syntax highlighting, debugging, refactoring. …
All of this with a strong emphasis on web standards: RESTful interfaces and feeds where appropriate. And open source under a liberal license.
So expect more posts detailing different parts of that solution and some of them going deep into the guts of SimPEL, which is what I’m working on right now. I’ll end this too long description with a couple of teasers. First the screenshot above comes from Singleshot, our task manager. Second and last, below is a small sample program written in SimPEL which currently executes. I also have a longer one, it doesn’t execute yet but I’m getting there.
process InvokeTwoWays { receive(itwPl, itwOp) { |initial| operands = <operands></operands>; operands.op1 = initial; operands.op2 = 3; invoke(calculator, add, operands) { |result| response = result; } reply(response); } }
Mathieu. This is very interesting. Collaxa -pre BPEL- worked on a concept called ScenarioBean which was a Java-based scripting language with parallel (flow and flowN), event-driven (listen/handle) and async continuations (call, receive).
Again this is very interesting. If I had to redo it again today. I would include 3 changes.
1) REST+JSON - REST meaning do not allow people to create verbs…and extend REST to have correlation and async callback. Use JSON instead of XML.
2) Activities are not enough. Most workflow models fail because they do not include a solid entity model. The result is that the data is in jail in the workflow engine. So it would be nice if the model had both support variales of type message and entities (a simple JSON to hibernate mapping would go a long way).
3) orchestration and state. If you look at IBM, they use BPEL for orchestration and have a separate engine for state transition. In the real world, you need both. State transitions and business events are really key.
I have been thinking about something like SimPEL for feedly: as the complexity of our application increases, we end up doing a lot of mashups which need to aggregate data and update multiple JSON services.
Good luck with this initiative and please ping me if I can be of any help.
[...] his post, Matthieu explains why we need processes, why writing BPEL by hand is not such a great idea, and [...]
Edwin,
Thanks a lot for your interest, a few remarks based on your feedback:
1) A RESTful interface is going to be the default remote channel to communicate with the engine. And we’ll probably have a few language specific bindings when you want to use the engine in-process. We’ve thought about JSON as well but there are a few translation issues to iron out, XML and JSON don’t map as well as it may seem to each other. I wouldn’t drop XML though.
2) We already have something like this in ODE, where you can map “external variables”. Instead of being store in the engine datamodel, those are mapped to whichever table you configure. What I actually like about it is that it’s an opt-in feature. I think it’s a sound default as there are only a few entities used by your process that you really want to access externally. External variables haven’t been rolled in SimPEL yet but they will be.
3) Interesting. I can definitely agree for task-driven state transitions that usually back a workflow. But do you think there’s also a need for a state machine outside of that use case?
Regarding your help offer, it’s much appreciated. I’ll contact you. I’d also like to repeat that SimPEL is being developed at Apache, as part of an open community where we welcome everybody.
Mathieu,
Let me try to explain what I mean by 3) a little more. My experience is that if you are building a large scale application, it is always best to first model your data model/schema first. That is going to be the foundation. For example, if you are building an expense processing application, you need a schema to capture the information in the expense report. With entities like an expense report which have a lifecycle (”created”, “submitted”, “approved”, “rejected”) it is usually very valuable to be able to capture that lifecycle information in your schema and be able to expose the lifecycle transitions as business event to the external world.
Given that context, I would like to be able in SimPLE create a entity variable for my expense report based on my data model (hence requirement #2) and I would like to be able to simple to be able to listen to business events like myExpenseReport.lifecycle[ "created" -> "submitted" ]
Regarding 1), I understand why you would not want to drop XML in the very short term but if you want to create something really simple and for the next 2-3 years out, I would recommend you try to reconsider: Most new services offer both plain old XML and JSON. Over time, JSON will win for 80% of the real world cases.
I will take a look at the Apache project. Also try to see if you can reach out to Keith Swenson at Fujitzu: he is a true visionary in this space and an amazing person to work with.
It’s great to see this getting more momentum and attention, since I think the XML syntax has been the primary obstacle to orchestration achieving broader adoption as a generic software tool.
The IBM technology that Edwin’s referring to is their “business state machine” (article on developerWorks), and my understanding was that it was similar to BPMN in that it was a notation that it compiled down to BPEL for execution with back-references to the original diagram.
Guys,
in case you missed it, I wrote last year a concept programming language as part of WSPER (http://www.wsper.org) that has the concept of state at its core.
I am glad to see these kind of ideas catching up at Oracle too.
Jean-Jacques
[...] * Import/Export of designed processes from Intalio|Designer to a set of different format * SimpEL * … The Q&A session was met with enthusiasm, and users could ask whatever question they [...]
[...] through multiple protocols, WSDL to be consumed by BPEL processes, and REST to be consumed by SimPEL ones. In a perfect world, your development tool would let you design these data objects using UML [...]
Subscribe to the comments via RSS Feed
Leave a Comment