Search this Blog

Search this Blog


Follow our Blog by Email

Friday, May 29, 2015

Camunda BPM 7.3.0 Final Released

I’m pleased to announce the availability of Camunda 7.3.0 Final. The release focuses on improving Camunda BPM for large production deployments. The highlights are:
  • Process instance repair and modification: go back or skip ahead in a process.
  • Authorizations: restrict access to processes, tasks and administrative actions.
  • Search for process instances and incidents in Cockpit
  • Starting new process instance at any activity / set of activities
  • Tasklist extensibility and customization: extend the Tasklist Web Application with Html and Javascript based plugins.
A detailed feature overview can be found at the end of this the post.

Thursday, May 7, 2015

Long running processes and service authentication - SAML tokens to the rescue?

When executing long running business processes which include service invocations you often have to solve a problem: What credentials should you use when calling the service?

Let's have a look at a simple example:

A user might start the process manually. Then service 1 might be executed with the credentials of this user. But how about service 2? Should it be executed with credentials from the first user or the user who finished the user task? How about service 3?

Wednesday, May 6, 2015

Camunda + Liferay Community Meeting

Yesterday evening we met at our Partner Ancud IT in Nurremberg (Germany). Ancud presented their experiences using Portals (namely the Open Source Portal Liferay) in combination with Process Engines (namely camunda :-)). That was already a topic in a recent webinar, you can check out the recording online: English or German. Afterwards we had interessting discussions about DMN use cases and a "social collaboration hub" research project. But let's start from the beginning.

Friday, April 24, 2015

Camunda BPM 7.3.0-alpha3 released

A new alpha release of our open source BPM and Workflow platform is available for download. The highlights of this release are:
  • Process Engine Java and Rest Api
    • Process-level authorizations: define ACL-like authorizations for restricting possible interactions with process-related resources (Process Definitions, Process Instances, Tasks ...), in both Java and REST Api.
    • Deep tissue refactoring of implementation of Multi Instance constructs in process engine.
    • Many, many bug fixes related to multi instance, event subprocesses, boundary events and compensation.
    • Api for process instance repair: Modify a process instance's execution structure through the ability to start and cancel activities in a flexible way.
    • Job Executor Threadpool can be configured for shared process engine in Apache Tomcat
    • Improved Error Handling: call activities now also map variables in case an error is thrown, access error code as process variable in error handler
    • Improved Task Query capabilities: processInstanceBusinessKeyIn(...), processDefinitionKeyIn(...), .taskDefinitionKeyIn(...), .parentTaskId(...), ,
    • Adds Support for Databases: Mysql 5.5 / 5.6, Maria DB 10.0, Microsoft SQL Server 2014, PostgreSQL 9.4, IBM DB2 10.1 /10.5
    • Adds Support for Application Servers: Wildfly 8.2, JBoss EAP 6.4
    • Adds Support for Java Runtimes: IBM® J9 virtual machine (JVM) 8, OpenJDK 6 / 7
    • Rest API now supports Jackson 2.3.x
  • Cockpit
    • Better auditing through improved User Operation Log now supporting all process instance related interactions
    • Restrict visibility of process definitions, instances and tasks and the possible interactions on these based on authorizations
    • Enterprise Edition-only feature: UI for process instance repair: Modify a process instance's execution structure through the ability to start and cancel activities in a flexible way.
    • UI improvements
    • is now used as diagram renderer
    • Activity instance tree now correctly differentiates the multi instance body scope from the inner activity instance
  • Tasklist
    • Start process dialog: Restrict visibility of processes which a user is not authorized to start.
    • Improved Filter Capabilities
  • Admin
    • Configure process-related authorizations
The full release notes can retrieved from Jira.

Tuesday, March 17, 2015

New Camunda Workbench Release - Adds Stepping in Debugger

After a one day hack session, Nico and myself are proud to release a new version of Camunda Workbench.

The Highlights of this Release are:

  • Upgrade to v0.9.0
  • Create a new process
  • Debugger now has stepping support
  • Breakpoints after activities
  • Display the type of variables
  • Enhanced Script Editing

Download alpha9 for free!

Wednesday, March 11, 2015

Camunda meets Cassandra @Zalando: Community Meeting Write-up

Jorn Horstman, André Hartmann and Lukas Niemeier from Zalando Tech visited us yesterday evening to present their prototype for running Camunda engine on Apache Cassandra.
They published their slides.

Zalando is a "multinational e-commerce company that specialises in selling shoes, clothing and other fashion and lifestyle products online". 2014 they had a revenue of €2.3 billion and currently have 8,500 employees (Source: Wikipedia). AND: they are Camunda enterprise edition subscribers and use Camunda process engine for processing their orders. Whenever you buy something in their online shop, a process instance is kicked off in Camunda process engine.

Zalando's current Architecture

Zalando's system needs to scale horizontally.

Currently Zalando's order processing runs on PostgreSQL database. They partition their order and process engine data over 8 Shards. Each shard is an independent instance of PostgreSQL. Such an "instance" is a small cluster with replication for performance and failover.

At the application server level they run on Apache Tomcat and use the Spring Framework. For each shard, they have a datasource for which they create an instance of camunda process engine. This is replicated over 16 nodes. When a new order comes in, it is assigned to one of the shards and an entry is made in a global mapping table, mapping orderIds to shards. Then the corresponding process instance is stated in that shard.

Zalando's current Architecture

When messages come in, they are first correlated to an order. When the order is resolved, they can deduce the responsible shard for the order and resolve the corresponding process engine.

They say that this works quite well but has some drawbacks:
  • They need to implement the sharding themselves,
  • The mapping table needs to be maintained,
  • Queries must be done against all of the shards and data must be aggregated manually.

In the context of their "Hackweek", they looked into Apache Cassandra as an alternave.

The Cassandra Prototype

Over the course of a week, they built a protoype where they exchanged the relational DB persistence layer in Camunda with an alternative implementation based on Cassandra.
They say that they wanted to
  • Learn more about Cassandra
  • Get a better understanding of the Camunda DB structure

Their Goal was not to run this in production (yet).

The source code of their prototype is on GitHub.

They provide an alternative implementation of Camunda's PersistenceSession interface.

In their prototype, they replicated Camunda's relational model in Cassandra. They had a table for executions in which each execution became a row, they had tables for variables, tasks etc. They did this on purpose since they wanted to start with a naive implementation and then learn from that.

During development they used a setup where they multiplexed the persistence session and executed all statements both on an SQL database and cassandra in order to be able to enhance support progressively while always having a working system.

As a result, they can execute simple processes.

Lessons Learned

After they presented their prototype, we discussed the lessons learned. We focused on the core process execution usecase, not on complex monitoring or task queries and things like that, assuming those usecases could be implemented on top of a dedicated search database such as elasticsearch into which data in fed from the execution cluster in near real time.

  • Copying the relational data model is problematic. A single process instance is composed of many entities, if they are stored as multiple rows in different tables,
    • Data related to a single process instance is distributed accross the cluster. This does not match the process engine's access pattern: often all data related to a process instance must be read or updated. If data is distributed, the network overhead is considerable.
    • Without transactional guarantees we cannot make changed to multiple rows in different tables atomically.
  • Eventual consistency: Cassandra keeps multiple copies of a data item. When making changes, you can configure whether you want to wait for all copies of a particular data item to be updated or whether one of the copies is enough or whether you need something in between (quorum, etc...). After some discussion, we concluded that the process engine would require to update all copies with most write operations.

It would be better to keep data related to a single process instance together in one row inside a single table. Then we could update it atomically, reading and updating it would entail minimal network overhead. In addition, we could require write operations to wait for all copies of the data they change to be updated.

From this the following model results:
  • Data of a single process instance is kept in one row.
  • If the process engine does an update to the row, it must first lock it. Conceptually, this could be a logical lock based on a retry mechanism.
  • If the row is updated / deleted, the operation must be performed on all updates. (Forfeiting the A in the context of P).

Consequences: No intra-process instance concurrency (concurrency inside a single process instance) involving different process engine instances (a single process engine can still lock the row, do things in multiple threads, join the operations and do one atomic update, releasing the lock).

Discussing all of this was a lot of fun! More Meetings are scheduled

Monday, March 9, 2015