In a recent Blog Post I wrote about a concept we call “External Tasks” where services were not actively called from the Workflow Engine (PUSH) but where “Workers” retrieve their tasks from the Workflow Engine (PULL). We discussed this a lot and got so much feedback that we decided to support this pattern out-of-the-box in Camunda BPM 7.4.
Let’s have a look at an example (yes – we do have customers implementing video processing :-)):
In this case transcoding and uploading a video file can take ages. It is done by special software which often does not have a SOAP or REST API to call. So in this case it is much easier to make an External Task out of it:
You have to set a topic name. Compare this to messaging where you use named queues in order to send messages to a specific recipient. This is the same as the topic name. If a worker wants to retrieve work he specifies the topic name he is interested in. Retrieving work can be done by Java API or by REST API. The following example queries for transcoding tasks and completes them via REST API:
var pollPayload = {
"workerId":"worker01", // some unique name for the current worker instance
"maxTasks":5,
"topics": [
{
"topicName": "transcodeVideo",
"lockDuration": 10000, // How much time the worker thinks he needs to process the task
"variables": ["video"] // Which variables should be returned in the response (to avoid additional REST calls to read data)
}
]
};
$.ajax('https://localhost:8080/engine-rest/engine/default/external-task/fetchAndLock/', {
data: JSON.stringify(pollPayload),
contentType : 'application/json',
type : 'POST',
success: function (result) {
for (var index = 0; index < result.length; ++index) {
var externalTask = result[index];
// Here the actual work would be done
$.ajax(REST_BASE_URL + '/external-task/' + externalTask.id + '/complete', {
data: JSON.stringify( {"workerId":"worker01"} ),
contentType : 'application/json',
type : 'POST'
});
}
});
};You can see the full example in this HTML/JavaScript page simulating the two external workers from the process above.
More Information
You find more information:
- In the complete executable source code implementing the example
- In the User Guide
- In the REST API Reference
Use Cases
I see actually a lot of use cases for External Tasks:
- Temporal Decoupling (without the need for messaging): If you look at the pattern it can replace a JMS queue between the Service Task (Consumer) and the Service Implementation (Provider), but still provide the decoupling messaging would provide. As we saw JMS being a pretty painful experience in a lot of projects, this is good news: You can get rid of JMS in your architecture!
- Polyglot Architectures / Non-Java-Environments: Assume you build a .NET based architecture it might not be that easy to write Java Delegates to call your services. It is already easy to use Camunda from any technology by using REST interfaces, this is now also true for Service Implementations!
- Better Scaling Patterns: You can start/stop workers as you like – and run as much as you want of them. By doing so you can scale each Service Task (or to be precise: Each topic) individually.
- Connect Cloud BPM to On Premise Services: Imagine you run Camunda BPM somewhere in the cloud (as customers of us do), you can still have Services on-premise, as they can now query their work via REST over SSL, which is quite Firewall-friendly.
- Avoid Timeouts: In the transcoding example the processing will take hours. So you could not synchronously call a service that blocks for hours (as you will get transaction and connection timeouts). So you would need some asynchronous callback anyway. The External Task pattern makes this easier to handle.
So great times ahead – like always 🙂
The External Task will be available with the upcoming Camunda BPM 7.4 release. Try it out and let us know what you think of it!
Camunda Developer Community
Join Camunda’s global community of developers sharing code, advice, and meaningful experiences
