Perficient Business Intelligence Solutions Blog

Blog Categories

Subscribe via Email

Subscribe to RSS feed

Posts Tagged ‘business intelligence’

Qlik leadership – vision, guts and glory… hopefully

“It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is most adaptable to change.” – Supposedly Darwin from ‘Origin of Species’… or NOT

According to the most recent report from Gartner, no one vendor is fully addressing the critical space in the market for “governed data discovery”. Governed data discovery means addressing both business users’ requirements for ease of use and enterprises’ IT-driven requirements – nobody is really doing that. So, who will be the most adaptable to change and embrace the challenges of an ever-changing and increasingly demanding BI and Analytics market?

Qlik leadership – vision, guts and glory… hopefullyThis year, Qlik plans to release a completely re-architected product – QlikView.Next – that will provide a new user experience, known as ‘Natural Analytics’. The lofty goal of QlikView.Next and this Natural Analytics approach is to provide both business-user-oriented and IT-friendly capabilities. According to Gartner, this approach has ‘the potential to make Qlik a differentiated and viable enterprise-standard alternative to the incumbent BI players.’

Will QlikView.Next be able to deliver the combination of business user and IT capabilities that are currently lacking in the market? Will Qlik be able to reinvent itself with Natural Analytics and deliver the “governed data discovery” solution that the market needs so desperately? Only time will tell; however Qlik is definitely showing all the traits of a real leader in the BI and Analytics space – once again, setting the bar pretty high. The vision and the guts are definitely there and accounted for. Will glory follow? That will depend on execution and delivery.

However, QlikView.Next is more than a year behind its scheduled release… so I guess we’ll have to rely on past behavior for now. Back in 2006, when Qlik carved its space on Gartner’s Magic Quadrant for Analytics and BI platforms (BusinessWire article), it positioned itself right in the ‘Visionary’ quadrant and it has been delivering on its vision ever since. For about eight years, Qlik has been delivering on its vision for Business Intelligence, i.e. user-driven BI – Business Discovery. Given this track record, I’ve reasons to believe that Qlik will be able to deliver on its vision once again.

I also believe that leadership is all about having a vision, along with the guts and ability to execute on that vision. That is probably one of the reasons why Gartner came up with quadrants that organize technologies along two dimensions – ‘Completeness of Vision’ and ‘Ability to Execute’. For the past few years, thanks to its ability to execute and deliver on its vision, QlikView has been able to work its way to the leaders quadrant and secure its position (GMQ 2014) – by demonstrating excellence in both vision and execution. So, how is Qlik planning on executing on its vision over the next few months – what’s .Next?

Well, there are several features worth mentioning… but, we’d be able to review only a few here, namely:

Read the rest of this post »

QlikView… QlikTech… Qlik…

Several years ago, when I started using QlikView (QlikTech’s flagship product), I had a strong preference for more traditional BI tools and platforms, mostly because I thought that QlikView was just a visualization tool. But after some first-hand experience with the tool, any bias I had was quickly dissipated and I’ve been a QlikView fan and fulfilling the role of Senior QlikView Architect on full lifecycle projects for a while now.

QlikViewToday, Qlik Technologies (also known as QlikTech or simply Qlik) is the 3rd fastest growing tech company in the US (according to a Forbes article) but my personal journey with QlikView, and probably QlikTech journey as well, has not always been easy – a paradigm shift in the way we look at BI is required. Most importantly, I understood along with many others, that this isn’t a matter of QlikView or SAP BI, of QlikView Agile approach to BI or Traditional BI – it is NOT a matter of ORs, but rather a matter of ANDs.

It is a matter of striking the right balance with the right technology mix and do what is best for your organization, setting aside personal preferences. At times QlikView may be all that is needed. In other cases, the right technology mix is a must. At times ‘self-service’ and ‘agile’ BI is the answer…. and at times it isn’t. Ultimately, it all revolves around the real needs of your organization and creating the right partnerships.

So far, QlikTech has been able to create a pretty healthy ecosystem with many technology partners, from a wide variety of industries and with a global reach. QlikTech has been able to evolve over time and has continued to understand, act on and metabolize the needs of the market, along with the needs of end-users and IT – I wonder what’s next.

That’s one of the reasons why Qlik has been able to trail-blaze a new approach to BI; user-driven BI, i.e. Business Discovery. According to Gartner ‘Qlik’s QlikView product has become a market leader with its capabilities in data discovery, a segment of the BI platform market that it pioneered.’

Gartner defines QlikView as ‘a self-contained BI platform, based on an in-memory associative search engine and a growing set of information access and query connectors, with a set of tightly integrated BI capabilities’. This is a great definition that highlights a few key points of this tool.

In coming blogs, we’ll explore some additional traits of QlikTech and its flagship product QlikView, such as:

Ø  An ecosystem of partnerships – QlikTech has been able to create partnerships with several Technology Partners and set in place a worldwide community of devotees and gurus

Ø  Mobility – QlikView was recently named ‘Hot Vendor’ for mobile Business Intelligence and ranks highest in customer assurance (see WSJ article here) with one of the best TCO and ROI

Ø  Cloud – QlikView has been selected as a cloud-based solution by several companies and it has also created strong partnerships with leading technologies in Cloud Computing, such as Amazon EC2 and Microsoft Azure

Ø  Security – provided at the document, row and field levels, as well as at the system level utilizing industry standard technologies such as encryption, access control mechanisms, and authentication methods

Ø  Social Business Discovery – Co-create, co-author and share apps in real time, share analysis with bookmarks, discuss and record observations in context

Ø  Big Data – Qlik has established partnerships with Cloudera and Hortonworks. In addition, according to the Wall Street Journal, QlikView ranks number one in BI and Analytics offering in Healthcare (see WSJ article here), mostly in connection with healthcare providers seeking “alternatives to traditional software solutions that take too long to solve their Big Data problems”

 

In future posts, I am going to examine and dissect each of these traits and more! I am also going to make sure we have some reality checks set in place in order to draw the line between fact and fiction.

What other agile BI or visualization topics would you like to read about or what questions do you have? Please leave comments and we’ll get started.

Primary Practices for Examining Data

SPSS Data Audit Node

z1

 

 

 

Once data is imported into SPSS Modeler, the next step is to explore the data and to become “thoroughly acquainted” with its characteristics. Most (if not all) data will contain problems or errors such as missing information and/or invalid values. Before any real work can be done using this data you must assess its quality (higher quality = more accurate the predictions).

Addressing issues of data quality

Fortunately, SPSS Modeler makes it (almost too) easy! Modeler provides us several nodes that can be used for our integrity investigation. Here are a couple of things even a TM1 guy can do.

Auditing the data

After importing the data, do a preview to make sure the import worked and things “look okay”.

In my previous blog I talked about a college using predictive analytics to predict which students might or might not graduate on time, based upon their involvement in athletics or other activities.

From the Variable File Source node, it was easy to have a quick look at the imported file and verify that the import worked.

z2

 

 

 

 

 

 

 

 

Another useful option is run a table. This will show if field values make sense (for example, if a field like age contains numeric values and no string values). The Table node is cool – after dropping it into my stream and connecting my source node to it, I can open it up and click run (to see all of my data nicely fit into a “database like” table) or I can do some filtering using the real-time “expression builder”.

z3

 

 

 

 

 

 

 

 

 

 

 

 

 

The expression builder lets me see all of the fields in my file (along with their level of measurement (shown as Type) and their Storage (integer, real, string). It also gives me the ability to select from SPSS predefined functions and logical operators to create a query expression to run on my data. Here I wanted to highlight all students in the file that graduated “on time”:

z4

 

 

 

 

 

 

 

 

 

 

You can see the possibilities that the Table node provides – but of course it is not practical to visually inspect thousands of records. A better alternative is the Data Audit node.

The Data Audit node is used to study the characteristics of each field. For continuous fields, minimum and maximum values are displayed. This makes it easy to detect out of range values.

Our old pal measurement level

Remember, measurement level (a fields “use” or “purpose”)? Well the data audit node reports different statistics and graphs, depending on the measurement level of the fields in your data.

For categorical fields, the data audit node reports the number of unique values (the number of categories).

For continuous fields, minimum, maximum mean, standard deviation (indicating the spread in the distribution), and skewness (a measure of the asymmetry of a distribution; if a distribution is symmetric it has a skewness value of 0) are reported.

For typeless fields, no statistics are produced.

“Distribution” or “Histogram”?

The data audit node also produces different graphs for each field (except for typeless fields, no graphs are produced for them) in your file (again based upon the field’s level of measurement).

For a categorical field (like “gender”) the Data Audit Node will display a distribution graph and for a continuous field (for example “household income”) it will display a histogram graph.

So back to my college’s example, I added an audit node to my stream and took a look at the results.

z5

 

 

 

 

 

 

 

 

 

First, I excluded the “ID” field (it is just a unique student identification number and has no real meaning for the audit node). Most of the fields in my example (gender, income category, athlete, activities and graduate on time) are qualified as “Categorical” so the audit node generated distribution graphs, but the field “household income” is a “Continuous” field, so a histogram was created for it (along with the meaningful statistics like Min, Max, Mean, etc.).

z6

 

 

 

 

 

 

 

 

 

 

 

 

Another awesome feature – if you click on the generated graphs, SPSS will give you a close up of the graph along with totals, values and labels.

Conclusion

I’ve talked before about the importance of understanding field measure levels. The fact that the audit data node generates statistics and chart types are derived from the measurement level is another illustration of how modeler uses the approach that measurement level determines the output.

 

Data Consumption – Cognos TM1 vs. SPSS Modeler

In TM1, you may be used to “integer or string”, in SPSS Modeler, data gets much more interesting. In fact, you will need to be familiar with a concept known as “Field Measurement Level” and the practice of “Data Instantiation.

In TM1, data is transformed by aggregation, multiplication or division, concatenation or translation, and so on, all based on the “type” of the data (meaning the way it is stored), with SPSS, the storage of a field is one thing, but the use of the field (in data preparation and in modeling) is another. For example if you take (numeric) data fields such as “age” and “zip code”, I am sure that you will agree that age has “meaning” and a statistic like mean age makes sense while the field zip code is just a code to represent a geographical area so mean doesn’t make sense for this field.

So, considering the intended use of a field, one needs the concept of measurement level. In SPSS, the results absolutely depend on correctly setting a field’s measurement level.

Measurement Levels in Modeler

SPSS Modeler defines 5 varieties of measurement levels. They are:

  • Flag,
  • Nominal,
  • Ordinal,
  • Continuous and
  • Typeless

Flag

This would describe a field with only 2 categories – for example male/female.

Nominal

A nominal field would be a field with more than 2 categories and the categories cannot be ranked. A simple example might be “region”.

Ordinal

An Ordinal field will contain more than 2 categories but the categories represent ordered information perhaps an “income category” (low, medium or high).

Continuous

This measurement level is used to describe simple numeric values (integer or real) such as “age” or a “years of employment”.

Typeless

Finally, for everything else, “Typeless” is just that – for fields that do not conform to any other types –like a customer ID or account number.

 

Instantiation

Along with the idea of setting measurement levels for all fields in a data file, comes the notion of Instantiation.

In SPSS Modeler, the process of specifying information such as measurement level (and appropriate values) for a field is called instantiation.

inst1

 

 

 

 

 

 

Data consumed by SPSS Modeler qualifies all fields as 3 kinds:

  • Un-instantiated
  • Partially Instantiated
  • Fully Instantiated

Fields with totally unknown measurement level are considered un-instantiated. Fields are referred to as partially instantiated if there is some information about how fields are stored (string or numeric or if the fields are Categorical or Continuous), but we do not have all the information. When all the details about a field are known, including the measurement level and values, it is considered fully instantiated (and Flag, Nominal Ordinal, or Continuous is displayed with the field by SPSS).

It’s a Setup

Just as TM1’s TurboIntegrator “guesses” what field (storage) type and use (contents to TM1 developers) based upon a specified fields value (of course you can override these guesses), SPSS data source nodes will initially assign a measurement level to each field in the data source file for you- based upon their storage value (again, these can be overridden). Integer, real and date fields will be assigned a measurement level of Continuous, while strings area assigned a measurement level of Categorical.

inst2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

This is the easiest method for defining measurement levels – allowing Modeler to “autotype” by passing data through the source node and then manually reviewing and editing any incorrect measurement levels, resulting a fully Instantiated data file.

Importing Data into SPSS Modeler for the TM1 Developer

If you have a TM1 background it is a quick step to using SPSS Modeler -if you look for similarities in how the tools handle certain tasks like, for example, importing data.

With TM1, source data is transformed and loaded into cube structures for consolidation, modeling and reporting using its ETL tool TurboIntegrator. In SPSS Modeler, source data is loaded and transformed through a “logic stream of nodes” for modeling and reporting.

Here is a closer look:

Sourcing Data

Cognos TM1 uses TurboIntegrator as its data import mechanism. TurboIntegrator (referred to as “TI”) is a programming or scripting tool that allows you to automate the data importation into a TM1 application. Scripts built with TurboIntegrator or “TI”, can be saved, edited and, through the use of chores, be set up to run at regular intervals.

Through the use of TI’s Data Source feature, you can import data or information external to TM1, such as:

  • Industry standard comma-delimited (CSV) text files including ASCII files.
  • Information stored in relational databases (accessible through an ODBC data source).
  • Other OLAP cubes, including Cognos TM1 cubes and views.
  • Microsoft® Analysis Services.
  • SAP via RFC.

Rather than using scripts, SPSS Modeler utilizes data import nodes, which are all found on the SPSS Sources palette.

Read the rest of this post »

Three Attributes of an Agile BI System

In an earlier blog post I wrote that Agile BI was much more than just applying agile SDLC processes to traditional BI systems.  That is, Agile BI systems need to support business agility.   To support business agility, BI systems should address three main attributes:

  1. Usable and Extensible –  In a recent TDWI webinar on business enablement, Claudia Imholf said “Nothing is more agile than a business user creating their own report.”   I could not agree more, with Ms. Imholf’s comments.   Actually, I would go farther.  Today’s BI tools allow users to create and publish all types of BI content like dashboards, and scorecards.  They allow power users to conduct analysis and then storyboard, annotate, and interpret the results.   Agile BI systems allow power users to publish content to portals, web-browsers, and mobile devices.  Finally, Agile BI systems do not confine users to data published in a data warehouse, but allow users to augment IT published data with “user” data contained in spreadsheets and text files.  Read the rest of this post »

Serviceability Auditing

Serviceability

What is application serviceability?

 “Serviceability (also known as supportability,) is one of the -ilities or aspects (from IBM’s RAS(U) (Reliability, Availability, Serviceability, and Usability)). It refers to the ability of application support personnel to install, configure, and monitor an application, identify exceptions or faults, debug or isolate faults to root cause analysis, and provide maintenance in pursuit of keeping an application current and/or solving a problem and restoring the product into service. Incorporating serviceability facilitating features typically results in more efficient product maintenance and reduces operational costs and maintains business continuity.”

Serviceability Levels

serviceme

An application serviceability audit should be conducted and have the objective of classifying all support procedures within the application as having either a low, medium or high serviceability level. Read the rest of this post »

Creating your First Report in SSRS – Part 1

I am splitting this article in 2 parts.  In the first part, I want to talk about all the tools you need to create your first report.  Before reading the 2nd part of this post, make sure you have:

1.)    Microsoft’s Business Intelligence Development Studio

2.)    Report Builder 3.0 (I am going to use Report Builder 2.0)

3.)    SQL Server or Oracle.

4.)    Revised SQL queries.

All the 4 items listed above are extremely important in creating your reports.  SSRS is a pretty user friendly tool and if an analyst can write SQL queries, he/she can start creating attractive SSRS reports in no time.

I am going to use the table given below in all my posts on SSRS.  We will be querying this table and creating different types of reports.

Consider this table to be a sales table for a store such as Bath and Body works.  This table has the following information:

Fields Description Data Type
ID Identifier (Primary Key) Number
Product _Type Candles, hand sanitizers, perfumes, etc Varchar2
Product Detail Fragrance of the product   such as Strawberry, Vanilla, Eucalyptus, etc Varchar2
In Store Timestamp of the product arrival in store Date
Sold Timestamp of product   sold Date

 

Please dedicate sometime in installing all the software listed above and creating SQL table on your machine before we dive into the second part of this article

Business Intelligence Tools and Agenda

Hello Readers and fellow Perficient bloggers,

This is my first official blog post on Business Intelligence forum.  Before I tell you what my blog is about and the agenda for upcoming blog posts, I would like to give you a quick introduction.

JuventusI am Sujay Nadkarni and I joined Perficient in March 2012 as a Business Consultant after completing my Master’s Degree in Information Systems from University of Texas, Dallas.  I am a Sun Certified Java Developer, Certified Scrum Master and have a certification in SAS.  Besides technology, I love soccer.  I am probably the biggest Juventus FC fan on this planet and my dream is to watch at least one Juventus home game.  I also love traveling and that is primarily the reason I joined Perficient (kidding).  Italy, Machu Pichu and Bora Bora are on the list of places I would love to visit before I die.  As a kid, I also acted in several Bollywood movies and Television series.

I am confident that everyone knows the importance of data in today’s world and the importance of presenting meaningful data to our clients.

Petabytes and Terabytes of data are collected every day from millions of users.  Check out this interesting link

It has become important to analyze this data and try to make sense out of it.  I intend to write several short posts on several business intelligence tools and I hope these posts will prove to be immensely helpful to consultants.

Read the rest of this post »

Top 5 Best Practices for an Actionable BI Strategy

In an earlier blog post, I pointed out a number of companies complete a BI Strategy but only to shelve it shortly after its completion.   One main reason is that companies let their BI Strategy atrophy by not maintaining it; however, the other main cause of shelving a BI Strategy is that it was not actionable. That is, the BI Strategy did not result in a roadmap that would be funded and supported by the organization.  As you formulate your BI Strategy, there are 5 best practices that will help result in a BI Strategy that is actionable and supported by your business stakeholders.  These best practices are:BI Strategy

1.       Address the Elephants in the Room – Many times if management consultants are brought into help with a BI Strategy, their objectivity is needed to resolve one or more disagreements within an organization.   For example, this disagreement could be a DW platform selection, architectural approach for data integration, or the determination of business priorities for the BI program.   The BI Strategy needs to resolve these issues or the issues will continue to fester within the organization, eventually undermining the support for the BI Strategy.    Read the rest of this post »