Subscribe via Email

Subscribe to RSS feed

Handle Slowly Changing Dimensions with Pentaho Kettle – Part1

In this blog we will talk about how to implement various types of slowly changing dimensions (SCDs) with Kettle in details. And I will introduce the examination of SCD Type I and Type II in Part2.

Types of Slowly Changing Dimensions

Following Kimball, we distinguish two main types of slowly changing dimensions: Type I, Type II:

  • Type I: Changes in the source system result in corresponding overwriting in the target dimension.
  • Type II: Changes in the source system result in inserting tamped versions of dimension rows multiple times in the target dimension.. This allows you to find whichever version of the dimension rows at any given point timely.

Read the rest of this post »

Posted in BI&Database

Understanding Extract Strategy in Oracle DAC

Almost every type of ETL tool has its extract and load strategy with its configurations. In general the user should be concentrated in full load (aka initial load) and incremental load options in whatever tools to utilize. Oracle BI Apps component provides straightforward options to let developer configure appropriate approaches to achieve data load in right time. But the extract technology is a little bit different for various sources – if the source is Siebel, we should configure table trigger and get records produced in an additional image table for insert, delete and update operations; if the source is Oracle EBS, we should utilize the extract query to compare last extract date in DAC with last update date in some specific source tables. Read the rest of this post »

How to Setup Local Standalone Spark Node

From my previous post, we may know that Spark as a big data technology is becoming popular, powerful and used by many organizations and individuals. The Spark project was written in Scala, which is a purely object-oriented and functioning language. So, what can a Java developer do if he or she wants to learn about Spark?

Here, I share my quick way of installing Apache Spark in the local machine. Read the rest of this post »

The influence of updating metadata by sub TI

Basing on my last blog (IBM Cognos TM1 updating metadata in TI submit time explore), we will explore the influence of updating metadata by sub TI here.

Please see below example as scenario 1: Read the rest of this post »

Posted in BI&Database

IBM Cognos TM1 updating metadata in TI submit time explore

Recently, when we update the dimension elements, we find the value of the cube which includes the dimension are missing. So we need to explore which time moment cause this. Please see below experiment: Read the rest of this post »

Posted in BI&Database

jsPDF in Web Portal

What is jsPDF?

jsPDF is an open-source library for generating PDF documents using JavaScript. It provides multiple options to generate PDF files, with custom properties. It has numerous plugins to support various ways of PDF generation.

The Challenge in portal web application for PDF generation:

In portal, aggregation of data from various sources plays an important part. In such cases, generating PDF with different visibility rules specific to a particular user, requires client side solution. Also, the solution should facilitate to render the generated PDF in different ways – open in new window, open in same window, save the file by default with default name, save the file with user given name, print the generated file automatically etc.

A simple algorithm to do so completely at the client side may be

  1. Capture the entire html page (may be body section / a particular section identified by an id attribute in the html) as an image.
  2. Generate a blank PDF with required document properties.
  3. As per the requirement, add the necessary top level content of the PDF document – date of PDF generation, an identifier, title to the document etc.
  4. Add the image with necessary parameters into the PDF below the custom content added.
  5. Render the PDF as per the requirement

JsPDF JavaScript library provides this solution through one of its plugin – addHTML plugin. Rendering the pdf in different ways also supported through other plugins of jsPDF.

AddHTML plugin from jsPDF, renders an HTML element to canvas object which is then added as an image to the PDF.

This plugin requires html2canvas ( or rasterizeHTML ( to perform the image conversion. Read the rest of this post »

Generate Performance Testing Report with JMeter-Plugins and Ant

JMeter is a popular open source performance test tool. However, its shortage of comprehensive report functionality, affects performance test result analysis, e.g., performance monitor of server, hits per second, response code over time, and so on. This paper will introduce how to create more analysis reports in both JMeter GUI Mode and Non GUI mode. Read the rest of this post »

Posted in Testing

Hangzhou Apache Spark Meetup

Similar to the Hadoop project, the Apache Spark project is a fast evolving in-memory engine for large-scale data processing platform. Particularly in recent years, Spark was widely used in many organizations and its community is being committed by many contributors. Perficient China GDC colleagues attended a recent Spark technology meetup in Hangzhou, during the meetup the developers shared the challenge, best practice and technical insight when to apply Apache Spark.
Read the rest of this post »

Generate code coverage report for iOS apps in Jenkins server

Code coverage is a good method to describe the degree to which the source code of a project is tested and how well the project is tested. In iOS develop world, around 25%  test code coverage is common required in business. Unfortunately, iOS native develop IDE (aka Xcode) hasn’t provided a default build-in code coverage feature yet. So here is my solution for it. Read the rest of this post »

KnockoutJS MVVM Script Folder Structure

In the past couple of years, we have delivered several projects successfully which were based on MVC + KnockoutJS, they all applied the same structure or pattern, so I was thinking to get them out from several projects which we might reuse in the future projects, here is what I’d like to share, you may adopt it, or question it.

Read the rest of this post »

Posted in BI&Database