Data Stores to share data in between processes / pools


#1

Hi all,

I just played with the Data Stores feature in the stage area. I stumbled across two things:

  1. It is not possible to place data stores in between pools. (In my experience it is crucial for modeling shared persistent data that it is possible to do exactly that.)

  2. When connecting a Data Store with an element that sits in a different pool, the data association is not visible, but sits on some visual layer “behind” that other pool. I assume this observation is somehow related to observation 1 - hence one post here.

Many greetings,
Martin.


#2

+1!

Doh - I need 20 characters to send my +1…


#3

Same should be possible for data objects. As it works for text-annotations.


#4

Thanks guys for your valuable feedback on this!

The BPMN 2.0 standard mandates that data store references (visible representation of data stores) have to be embedded inside pools / processes.

For the initial version of data store support we decided to be strictly compliant, i.e. allow nothing else.

Regarding data objects: They are different from text associations as they are not artifacts and therefor must be embedded inside a process / pool, too. This is nothing we make up but part of the BPMN 2.0 standard.

Your feedback is important for us to understand whether or not we need to allow more than the standard if users can build better models like that.

If you would like to model shared persistent data according to the BPMN 2.0 spec you’d need to do one of the following:

Copy data store / identify by name

This is the preferred behavior in strict accordance with the BPMN 2.0 spec.

Reference data store across pool boundaries

It is possible to create data associations across pool boundaries. While this has unclear semantics (data flow wise) we left this option open to reuse data store references.


#5

Nikku,

thank you for clarifing the bits and pieces of the bpmn standard.

Sometimes it feels better to break the standard, but it’s worth to know that it is a mistake.

Cheers, Ingo


#6

Hi Nikku,

Thanks for your extensive explanation. I would vote to “break” the standard a bit here.

  1. Your first example is a no go for me - from a practical point of view.

  2. I agree that your second example has “unclear semantics (data flow wise)” - but let me add that nobody would typically combine message flow and data association like shown in this example. Rather, one would like to show just the data association. By doing that one would express, that one process influences shared data (if you want, call it “create, update, delete” for a shared persistent object) and another process works with that shared data by actively pulling it (call it “read” for a shared persistent object).

This is a very typical pattern to “loosely couple” processes / pools via data instead of “directly couple” them via message flow. When expressing that pattern visually, it is often really annoying to suggest to the reader that this shared data somehow “belongs” to one of those processes more than to the other one.

  1. Maybe it’s worth to have a second set of eyes to discuss the interpretation of the standard? Do you have more specific references to the standard?

Many greetings,
Martin.


#7

Maybe it’s worth to have a second set of eyes to discuss the interpretation of the standard? Do you have more specific references to the standard?

Regarding the standard it is basically defined in the meta model / XSD schema shipped with the standard itself. Additionally the standard document itself states that data store references must be contained in a process or pool (something like that).

I remember we did an exception to this regarding data store handling in the Camunda Eclipse Plug-in back in the days, too.


#8

Hey guys.

I remember that this already caused a lot of trouble in the roundtrip/pool-extraction with Camunda Cycle. I think we can agree from a practitioners point of view that this is a severe weakness of the BPMN standard. It should be possible to place data stores OUTSIDE of a pool. I can give Falko a ping to have it on the agenda if there is movement on BPMN in future.

Signavio is breaking the standard here as well by the way, if I remember it right they “just” add the data store to one of the pools when exporting to produce valid XML - and this pool was rather random. Not sure what we did in Eclipse.

So my first question would be: Is there a possible workaround which produced 100% valid BPMN XML but still renders the data store outside of the pool? Maybe by having some wired coordinates (like y > pool height)?

Quite a pitty!

Cheers
Bernd


#9

See this example for a typical example where the Data Store should not be part of one pool because it is basically used to connect the pools!


#10

Hi all - do we already have a feasibility opinion or maybe even a GitHub issue for that? This is an issue which pops up in every BPMN training, because it contradicts what we currently teach with our own slides and what is really needed from a practitioner’s point of view. Naturally it also causes that some people in the training wonder whether the tool is practically usable already - or not yet. :smirk:

Option 2 shown above (“Reference data store across pool boundaries”) would be a doable workaround for the moment, but we seem to have a bug here, because most of the time the data association is partly invisible. See the following example which shows a data store associated to two tasks. One association is partly invisible:

I see this not just on Mac, also on Windows. Having said that, there is a workaround to display it correctly, though… a) Place the data store in a pool which makes sure that the first data association you want to draw goes across pool borders, the second one stays within the same pool, then place the data store in the other pool as a last step… it seems to depend on the order of pools and order of data association directions whether the data associations are fully shown or not, but I still do not fully grasp the underlying logic of the tool behaviour…


#11

Feels like we got enough feedback on this to bring the good old “DataStore in the middle” behavior back. I created a ticket for that in the Camunda Modeler.


#12

Just to make sure: I assume that the visualization issue I described in my last comment above (partly hidden data associations) goes away with implementing the “Data Store in between Participants” feature? (Cause if yes: If the implementation of this feature will take rather longer, I would love to see a fix for this relatively small issue rather sooner - just always display data associations on top of everything else?)


#13

We could ship a fix for the visualization issue earlier.

Could you add a bug ticket for that in the bpmn-js issue tracker?


#14

I opened an issue, but cannot mark it as a bug due to lack of rights. Maybe you can do that for me… many thx!


#16

Perhaps I’m reading the semantic.xsd wrong, but it says that DataStore is a extension of Root Element. This would further suggest that placing a DataStore outside a pool is the correct thing to do. Further to that, the spec indicates that a DataStore exists outside the lifetime of a process, it really should not exist within the pool. A reference to that DataStore (perhaps thought of like a particular connection to a DB) would exist within the lifetime of the process.

DataObjects must be placed within a pool, which is logical, as they represent internal variables to the process.

This all makes sense when you consider that a BlackBox pool should be able to write into a common datastore, and then your process reads from that datastore.

Apologies if it were incorrect to reopen this thread this long after the last comment, but I ran into this issue in Modeler 1.8.2, causing issues for how to express it correctly.


#17

This topic is still open. It should be possible to place DataStores - at least visually - outside of pools. As @nikku writes. Analysts still have to work around that limitation, which is annoying. :slight_smile: Sorry to be so blunt, I know well that software development is hard!


#18

Data stores exist outside pools, however they do not have a graphical representation outside a pool.


#19

I 100% agree. Unfortunately it is not a priority at the moment.

I suggest you to put your :+1: on the issue so we can tackle this eventually, some day.


#22

Done. However, I also suggest that you look at the visualisations here, too. This issue has 5.800 views. How many others have? It is the “all time” highest ranked issue in the user category! :slight_smile:


#23

Implemented and shipped with bpmn-js@2.0.0.