Search this Blog

Search this Blog


Follow our Blog by Email

Tuesday, March 17, 2015

New Camunda Workbench Release - Adds Stepping in Debugger

After a one day hack session, Nico and myself are proud to release a new version of Camunda Workbench.

The Highlights of this Release are:

  • Upgrade to v0.9.0
  • Create a new process
  • Debugger now has stepping support
  • Breakpoints after activities
  • Display the type of variables
  • Enhanced Script Editing

Download alpha9 for free!

Wednesday, March 11, 2015

Camunda meets Cassandra @Zalando: Community Meeting Write-up

Jorn Horstman, André Hartmann and Lukas Niemeier from Zalando Tech visited us yesterday evening to present their prototype for running Camunda engine on Apache Cassandra.
They published their slides.

Zalando is a "multinational e-commerce company that specialises in selling shoes, clothing and other fashion and lifestyle products online". 2014 they had a revenue of €2.3 billion and currently have 8,500 employees (Source: Wikipedia). AND: they are Camunda enterprise edition subscribers and use Camunda process engine for processing their orders. Whenever you buy something in their online shop, a process instance is kicked off in Camunda process engine.

Zalando's current Architecture

Zalando's system needs to scale horizontally.

Currently Zalando's order processing runs on PostgreSQL database. They partition their order and process engine data over 8 Shards. Each shard is an independent instance of PostgreSQL. Such an "instance" is a small cluster with replication for performance and failover.

At the application server level they run on Apache Tomcat and use the Spring Framework. For each shard, they have a datasource for which they create an instance of camunda process engine. This is replicated over 16 nodes. When a new order comes in, it is assigned to one of the shards and an entry is made in a global mapping table, mapping orderIds to shards. Then the corresponding process instance is stated in that shard.

Zalando's current Architecture

When messages come in, they are first correlated to an order. When the order is resolved, they can deduce the responsible shard for the order and resolve the corresponding process engine.

They say that this works quite well but has some drawbacks:
  • They need to implement the sharding themselves,
  • The mapping table needs to be maintained,
  • Queries must be done against all of the shards and data must be aggregated manually.

In the context of their "Hackweek", they looked into Apache Cassandra as an alternave.

The Cassandra Prototype

Over the course of a week, they built a protoype where they exchanged the relational DB persistence layer in Camunda with an alternative implementation based on Cassandra.
They say that they wanted to
  • Learn more about Cassandra
  • Get a better understanding of the Camunda DB structure

Their Goal was not to run this in production (yet).

The source code of their prototype is on GitHub.

They provide an alternative implementation of Camunda's PersistenceSession interface.

In their prototype, they replicated Camunda's relational model in Cassandra. They had a table for executions in which each execution became a row, they had tables for variables, tasks etc. They did this on purpose since they wanted to start with a naive implementation and then learn from that.

During development they used a setup where they multiplexed the persistence session and executed all statements both on an SQL database and cassandra in order to be able to enhance support progressively while always having a working system.

As a result, they can execute simple processes.

Lessons Learned

After they presented their prototype, we discussed the lessons learned. We focused on the core process execution usecase, not on complex monitoring or task queries and things like that, assuming those usecases could be implemented on top of a dedicated search database such as elasticsearch into which data in fed from the execution cluster in near real time.

  • Copying the relational data model is problematic. A single process instance is composed of many entities, if they are stored as multiple rows in different tables,
    • Data related to a single process instance is distributed accross the cluster. This does not match the process engine's access pattern: often all data related to a process instance must be read or updated. If data is distributed, the network overhead is considerable.
    • Without transactional guarantees we cannot make changed to multiple rows in different tables atomically.
  • Eventual consistency: Cassandra keeps multiple copies of a data item. When making changes, you can configure whether you want to wait for all copies of a particular data item to be updated or whether one of the copies is enough or whether you need something in between (quorum, etc...). After some discussion, we concluded that the process engine would require to update all copies with most write operations.

It would be better to keep data related to a single process instance together in one row inside a single table. Then we could update it atomically, reading and updating it would entail minimal network overhead. In addition, we could require write operations to wait for all copies of the data they change to be updated.

From this the following model results:
  • Data of a single process instance is kept in one row.
  • If the process engine does an update to the row, it must first lock it. Conceptually, this could be a logical lock based on a retry mechanism.
  • If the row is updated / deleted, the operation must be performed on all updates. (Forfeiting the A in the context of P).

Consequences: No intra-process instance concurrency (concurrency inside a single process instance) involving different process engine instances (a single process engine can still lock the row, do things in multiple threads, join the operations and do one atomic update, releasing the lock).

Discussing all of this was a lot of fun! More Meetings are scheduled

Monday, March 9, 2015

Thursday, March 5, 2015

Camunda BPM 7.3.0-alpha2 released

A new alpha release of our open source BPM and Workflow platform is available for download. The highlights of this release are:
  • Process Engine and Rest Api
    • Native JSON and XML Variable Value Types
    • Improved Sorting: specify multiple sort parameters in rest api, sort task by variable value
    • Historic Job Log
    • Support for databases: Oracle 12c
    • Partially Order Historic Activity Instances by occurrence
  • Cockpit
    • UI improvements
    • Advanced Process Instance Search (enterprise feature)
  • Tasklist
    • Sort by multiple parameters
    • Sort by variable value
  • Admin
    • Browse Users by Group
  • JavaScript SDK
    • Add group resource
    • Support local variables
The full release notes can retrieved from Jira. The known issues as well.

Tuesday, February 17, 2015

Feedback wanted: Camunda BPM and Business Rules

Have you ever used the business rules task in BPMN?
Are your company's business rules closely related to your processes?

Many of our users will answer these questions with a 'yes'.

With the new DMN standard for decision management and business rules being close to the final release at OMG, the whole topic has recently been getting increased attention. The DMN (Decision Model and Notation) standard has been built in a way so that it can easily interact and integrate with BPMN.

Camunda BPM is widely considered a great BPMN engine. We have recently added the CMMN standard for case management to this engine, which was a great success for our users. Now we are thinking about adding a solution for executing business rules based on DMN.

We want to give you the chance to influence our decision, so that we build what you need most.

This is why we are asking you to participate in our survey. It's just 10 questions, won't take longer than 10 minutes and doesn't require any personal data. Please participate, even if you do not know DMN or if you do not care about business rules.

English version:

German version:

Thanks for taking part and for helping us decide. 

Monday, February 9, 2015

{ "json" : "everywhere" } - How to use json in a process

Since Camunda BPM platform 7.2 it is very easy to read, write and manipulate json objects by using Camunda Spin. Among other things, this feature is used to serialize Java objects inside a process instance in the process engine.

But what if you want to use Json Variables without mapping to Java objects? With the next alpha release we will introduce a new ValueType "json" (and "xml" but that is an other story).

In this post I demonstrate how you can use the json value type to

  • implement HTML Forms which work with a Json Variable,
  • implement BPMN Sequence Flow conditions based on the properties of the Json Variable.

Thursday, February 5, 2015

Data mapping and transformation with Camunda Spin and Java 8

Working with text based data formats such as Xml and Json is a common requirement when implementing BPMN processes.

Since version 7.2 Camunda provides an optional library which is called Camunda Spin. Spin is a lightweight wrapper library which provides a easy to use API when working with text based data formats such as XML and Json.

In this post I show how the Camunda Spin can be used for implementing data transformations and mapping in combination with the Java 8 Stream processing API and contrast this to the classical Java 6 / 7 way of doing it.