Since its inception, we have all watched Qlik Sense grow. From desktop to cloud, from custom extensions to embedded visualizations to mashups using Enigma.js. Qlik Core is the next phase of this growth, and it is exactly what developers ordered.
There’s no debating that Qlik Sense Desktop is an incredibly powerful tool, giving business analysts and decision makers everything they could want: easy ETL, data governance, security, self-service analytics, and more. Over the most recent years, Qlik Sense has upped the ante with new features, unlocking powerful new capabilities while maintaining the ease-of-use that makes Qlik so appealing.
But. We’ve all had that one client who wants the power of Qlik with their branding, their look-and-feel, in their custom web application. Of course, there are mashups, and embedded visualizations, but you still have to work within the parameters of the Qlik Sense Enterprise Edition. While you get the power of Qlik, you also get the heavy architecture in the background, even if you don’t want it.
It can also be hard to get the client to bite on a heavyweight data visualization application given the fact that the existing project or application could already be functional and in production (not to mention, the price and time to stand up a fully loaded enterprise software package).
Enter Qlik Core. In its simplest form, Qlik Core is a container that includes the QIX Engine, its APIs, and nothing more. It is lightweight, accessible, and easily fits into an existing architecture without demanding to be the centerpiece. Qlik stripped away the data governance, access control, UI, and data visualization pieces of the application; these are the pieces you don’t necessarily need if you are integrating high powered data analytics into an existing application or data warehouse.
So, how does it work? Qlik Core is containerized, so it scales up and down easily. This also makes it simple to create session applications, where data is loaded on the fly. It is designed without much infrastructure, so you simply spin up Qlik Core behind your existing security layer, plug in your data, and then begin hitting the endpoints from your application layer.
I recently attended the Qlik Data Revolution Tour’s Qlik Core breakout session to get hands-on with the new technology. The presenters admitted that with the power of Qlik Core, you could create a competitor to Qlik Sense itself.
There are multiple use cases that Qlik highlighted with this new ‘spin-off’ of Qlik. Some clients use Qlik Core to add self-service analytics to a pre-existing application, adding real-time filtering and querying to enhance an existing user interface. Others use it as a data pipeline. Feed data from the database through the QIX engine to transform and organize the data schema, and then feed that data into another system for long term storage.
A recent project I was on had the perfect use case for Qlik Core. We had an existing workflow application that managed documents through an editing, publication and dissemination process. The bulk of the data within the application included product metadata (author, coauthor, country, topic) and action metadata (when the product moved from draft to editor to publication, etc). The clients were interested in allowing each author to create their own data tables, enabling them to add any data columns and rows, and also permitting them to group by or aggregate on any column they would like. Want to see how many products are in your office, aggregated by author and grouped by type? Sure thing. Want to see the average time it took your products to get through the editing stage, grouped by topic? Go ahead.
In less words, the client wanted the click-and-drag capability of Qlik Sense Desktop built within our existing application. A strict requirement was that it needed to have the same look and feel as our existing application and be embedded in the existing UI. This is what made it complex. Our data was stored in an extremely de-normalized schema in a database that was run under suboptimal conditions. Solely leveraging our database was out of the question for multiple reasons.
We explored Tableau as a solution for some time, but realized that the Tableau Server API does not let you query the underlying data without first constructing and publishing a visualization for that data. I found this extremely frustrating. The real power of an analytical tool such as Tableau or Qlik (for us full stack developers, anyway) is that it associates the data for you. Need a bar chart between column 1 in table A and column 4 in table B? No problem, as long as you informed the tool of the schema when you loaded the data. Unfortunately, with Tableau Server, you need to publish a view of the data in order to query it via the API. In the terms of our use case, this would have required us to completely flatten our schema into one normalized table, and then publish that table. Given our schema, this was not a realistic solution.
As I was sitting in the Qlik Core breakout session, I realized this new technology would be the perfect tool for the job. Load the data schema via the API (or re-use your existing data load scripts, those work too) and then begin querying. The QIX Engine takes care of the behind the scenes joins, associations, and aggregations, just as it does when you’re using Qlik Sense. In fact, every API endpoint is available to you via Qlik Core, which encompasses every endpoint Qlik Sense uses to process, store, manage, and query your data sources.
Another great benefit of Qlik Core is the fact that it plays very nicely with lightweight Javascript libraries. One of the Qlik Core developers shared his enthusiasm for pairing Qlik Core with React. Qlik Core manages all of the data’s state for you, and knows to re-query the engine when necessary based on filter selections. Simple demo code (that was shared with us at the conference) can be found here: https://github.com/rjriel/qlik-core-workshop/tree/final
Another cool thing introduced to us during the session was the Catwalk tool: https://catwalk.core.qlik.com/. This is an open source project that provides users a way to navigate the data structure and dig into the data fields without actually having to access the data in its raw form. Once the data is loaded into the QIX engine, you plug the websocket information into Catwalk and begin exploring. We won’t get into how cool this is here, but checkout the code and give it a run! We thought it was pretty neat.
In a world where everyone wants their data shown in their application in their way (and don’t want to spend copious amounts of money on the heavy overhead of an entire enterprise solution to do it), Qlik Core is the answer to our prayers. The power of the QIX associative engine that powers Qlik Sense itself, unlocked for use.