Differences between In-Memory Fields and Direct Discovery Fields with example.Thanks

[#1]

Differences between In-Memory Fields and Direct Discovery Fields with example.

[arulsettu]

Hi Read this article it will be helpful

QlikView Direct Discovery enables users to perform visual analysis against “any amount of data, regardless of size”. With the introduction of this unique hybrid approach, users can associate data stored within big data sources directly alongside additional data sources stored within the QlikView in-memory model. QlikView can “seamlessly connect to multiple data sources together within the same interface”, e.g. Teradata to SAP to Facebook allowing the business user to associate data across the data silos. Data outside of RAM can be joined with the in-memory data with the common field names. This allows the user associatively navigate both on the direct discovery and in memory data sets.

QlikView developer should setup the Direct Discovery table on the QlikView application load script to allow the business users to query the desired big data source. Within the script editor a new syntax is introduced to connect to data in direct discovery form. Traditionally the following syntax is required to load data from a database table:

To invoke the direct discovery method, the keyword “SQL” is replaced with “DIRECT”.

In the example above only column CarrierTrackingNumber and ProductID are loaded into QlikView in the traditional manner, other columns exist in the data table within the Database including columns OrderQty and Price. OrderQty and Price fields are referred as “IMPLICIT” fields. An implicit field is a field that QlikView is aware of on a “meta level”. The actual data of an implicit field resides only in the database but the field may be used in QlikView expressions. Looking at the table view and data model of the direct discovery columns are not within the model (on the OrderFact table):

Once the direct discovery structure is established, the direct discovery data can be joined with the in-memory data with the common field names (Figure 3). In this example, “ProductDescription” table is loaded in-memory and joined to direct discovery data with the ProductID field. This allows the user to associatively navigate both on the “direct discovery” and in memory data sets.

Direct Discovery will be much slow then in-memory processing and this is is expected, but it will take away from Qlikview its usual claim that is is faster then competitors. QlikView Direct Discovery can only be used against SQL compliant data sources. The following data sources are supported;

• ODBC/OLEDB data sources – All ODBC/OLEDB sources are supported, including SQL Server, Teradata and Oracle.
• Custom connectors which support SQL – Salesforce.com, SAP SQL Connector, Custom QVX connectors for SQL compliant data stores.

Due to the interactive and SQL syntax specific nature of the Direct Discovery approaches a number of limitations exist. The following chart types are not supported;
• Pivot tables
• Mini charts
And the following QlikView features are not supported:
• Advanced aggregation
• Calculated dimensions
• Comparative Analysis (Alternate State) on the QlikView objects that use Direct
Discovery fields
• Direct Discovery fields are not supported on Global Search
• Binary load from a QlikView application with Direct Discovery table

[r_wroblewski]

Hi Hammad and Arul

First of all Arul I`m sorry but your example is a little bit old.

For QV11.2 a direct discovery table looks like that:

TableName:

DIRECT QUERY

DIMENSION

     CustomerID

     ItemID

MEASURE

     Price

     PurchaseAmount

     Discount

DETAIL

     CountryName

     StoreName

     CustomerName

FROM "...";

in-memory:

All data is loaded in-memory and directly available

(if you load e.g. a whole table from an eternal database into QV every single info will be loaded into QV)

Direct Discovery:

Here you have 3 different kind of fields:

DIMENSION fields are loaded as distinct load (means every unique information will be loaded in-memory)

this fields are normally used as key fields with in-memory data for your query.

MEASURE fields are for calculation only (no data is loaded in-memory)

you can use e.g. "sum(PurchaseAmount)" and use this set analyse inside your application

DETAIL fields are to show information in a table (no data loaded in-memory)

you can use this fields to show additional information in a straight table without loading the data directly.

Direct Discovery has advantages and disadvantages.

Please see following documents for more detailed information:

QlikView Direct Discovery FAQ 11.2 SR7.pdf

QlikView 11 2 Direct Discovery Technical Addendum 11.2 SR7.pdf

Best regards,

Ronny

PS: Please mark this discussion as answered if this is what you need.

[jagan · Accepted solution]

Hi,

The data for Direct Discovery Fields are retrieved from the database everytime and In-Memory Fields are already fetched and stored in RAM as qlikview normally works.  Refer examples in the above posts.

Regards,

Jagan.