Bug report: Outer set expression is not inherited ... - Qlik Community

sergiogiogio

I would like to report what I believe is a bug in inner and outer set expressions feature (May 2023 patch 1), not sure if this forum can be a suitable location.

Given the the script,

Fruits:
LOAD * Inline [
Type,Color,Quantity
apple,green,1
apple,green,1
apple,green,1
orange,orange,1
orange,orange,1
];

Consider the below formulae outputs, where we select apples (which exist in the data set) and kiwis (which do not exist):

Sum({<Type={'apple'}, Color={'green'}>} Quantity)	3	OK
{<Type={'apple'}>} Sum({<Color={'green'}>} Quantity)	3	OK
{<Type={'apple'}, Color={'green'}>} Sum(Quantity)	3	OK
Sum({<Type={'kiwi'}, Color={'green'}>} Quantity)	0	OK
{<Type={'kiwi'}>} Sum( {<Color={'green'}>} Quantity)	3	WRONG!! The outer set analysis is not inherited!
{<Type={'kiwi'}, Color={'green'}>} Sum(Quantity)	0	OK
Sum({<Type={'orange'}, Color={'green'}>} Quantity)	0	OK
{<Type={'orange'}>} Sum( {<Color={'green'}>} Quantity)	0	OK
{<Type={'orange'}, Color={'green'}>} Sum(Quantity)	0	OK
Sum({<Type={'kiwi'}>} Quantity)	0	OK
{<Type={'kiwi'}>} Sum( Quantity)	0	OK

As per the results above: outer set expression is not inherited if

the outset set expression selects an inexistant value,
there is an inner set expression

Could someone confirm this is indeed a bug and whether it has already been fixed in the recent version? Thanks!

sergiogiogio

The same issue has been reported at least 3 other times since Aug 2023 without resolution or explanations.

At this moment, I understand the issue is triggered when:

a merge of set expressions is performed (inner or outer does not matter)
an outer set expression "exclusively" resolves into an empty set. In other words, that outer set expression taken in isolation would never return anything anyway.

under these 2 conditions, the merge operation will simply ignore the outer set.

If no merge is needed, the result will be correct, so it really seems to be a bug with the merge operation.

I cannot stress enough how important this issue is - it might lead to incorrect results completely silently and very difficult to debug.

The longer this issue is delayed, the more difficult it will become to resolve, as changing the behavior over time will disrupt an increasing number of applications.

Tagging @Jennell_McIntire who wrote the introduction blog post for help in escalating the issue.

marcus_sommer

IMO it's not a bug else behaved as designed. Essentially is that inner- and outer-statements are not regarded as a single statement else it are independent statements which are performed in an order.

You could imagine your "wrong" example as two selections. The first selects 'kiwi' and then the selection of 'green' happens - which deselects 'kiwi' again.

The outer statements and especially the combination of inner- and outer-statements are not a simplification or a replacement for a normal set statement else it is an advanced feature for special scenarios.

sergiogiogio

Thanks @marcus_sommer for your reply.

You could imagine your "wrong" example as two selections. The first selects 'kiwi' and then the selection of 'green' happens - which deselects 'kiwi' again.

'kiwi' does not exist in the dataset. Your scenario matches the test below where 'orange' exists in the data. The result of this test is 0, which is correct in my opinion. If we go by your proposed explanation, it should show 3. This would hopefully convince you that there is something deeper to investigate here.

{<Type={'orange'}>} Sum( {<Color={'green'}>} Quantity)

The outer statements and especially the combination of inner- and outer-statements are not a simplification or a replacement for a normal set statement else it is an advanced feature for special scenarios.

Can you please provide references for the special scenarios it should only be used for? The documentation simply says "Use a set expression outside the aggregation function if you have expressions with multiple aggregations and want to avoid writing the same set expression in every aggregation function."

One important use case is useing it for measures as explained in the documentation?

{<Year={2021}>} [Master Measure]

If [Master Measure] does not contain an inner expression, the above expression will always return the expected result.
If [Master Measure] does contain an inner expression, and if the year 2021 exists in the dataset, the above expression will always return the expected result.
If [Master Measure] does contain an inner expression, and if the year 2021 does not exist in the dataset, the year selection will be ignored.

You might not have control over the way the [Master Measure] are defined. That definition might also change overtime. If there is no consistency in the behavior (as is illustrated above), using outer expression for master measures is then not advisable, which then contradicts the documentation.

marcus_sommer

I have no examples for it because I don't use these features in our productive environment and I doubt that I would implement them as a general approach within a business release in the future.

Yes, the documentation gives the impression that it are must-use features but IMO in goes in the wrong direction of implementing the essential business logic within the UI and the data-model isn't really important anymore and is only a kind of data-lake and the smart developer just picked the wanted data.

In my experience it's much harder and needs much more efforts to develop the logic within the UI instead of the data-model. Ideally the most calculations are possible with simple sum() and count() without much set analysis and the users just select what they want to see: Let the User Select - Qlik Community - 1463978. Each kind of own usability beyond the green-white-grey selection-logic will increase the complexity and needed efforts - therefore be careful with it.

Bug report: Outer set expression is not inherited if 1) the outset set expression selects an inexistant value, 2) there is an inner set expression

Advanced Authoring