Posts by this Author

In 2016, when I did my first in-depth comparison, the resulting TCOs were usually very close. Usually, the OpEx was slightly higher for the cloud TCO versus the on-prem TCO required substantial capital investment. However, our most recent estimate was eye-opening to our client.  We were assessing a green-field implementation for a Data Warehouse at […]

Data Lakes have been around since the early part of this decade as most Fortune 500 companies have a Data Lake or are building a Data Lake. The drive to lake data has predominately been driven by analytical use cases where Data Scientists can wrangle and prepare data for their study or model building. However, […]

Big Data is one of the enabling technologies for companies to digitally transform either their operations and/or customer  interactions.  However the open source world can be complicated, especially in the red hot Big Data arena. There are a myriad of technologies; some compete with one another, others overlap, some are complementary, and worse of all, […]

Previously, I reviewed why Spark will not by itself replace Hadoop, but Spark combined with other data storage and resource management technologies creates other options for managing Big Data.  Today we will investigate how an enterprise should proceed in this new, “Hadoop is not the only option” world.  Hadoop, Spark, Cassandra, Oh My!  Open source Hadoop and […]

Yesterday, IBM threw its weight behind Spark. This announcement is significant because it is a leading indicator of a transition from IT-focused Big Data efforts to business-driven analytics and Big Data investments. If you are interested in learning more about this announcement and what it means in the bigger picture, I wrote a blog entry on our […]

  Right after I posted my blog on Spark and Hadoop, I came across this article. IBM made a big announcement that they are putting their weight behind Spark.  They are committing more than 3,500 developers and programmers to help move Spark forward. This combined with significant support from the Big 3 Hadoop distributors (HortonWorks, Cloudera, […]

I have seen a number of articles asking the question of whether Apache Spark will replace Hadoop.   This is the wrong question!  It is like asking if your your DVD player will replace your entire home theater system, which is pretty absurd.  Just like a home theatre system has many components, a TV or Projector, a Receiver, […]

As companies transform their businesses to be data-driven and leverage the benefits of Big Data, they quickly realizing that lack of Big Data centric data scientists and wranglers is blocking their value attainment.  One of the limiting factors in Big Data resources, are the skills that have typically been required to leverage Big Data.  Java and […]

The strategy is complete, implementation of the mobile application and analytical system is finished, data scientists are providing useful analytical research.  But is your enterprise getting the value out of your digital transformation investments? A company’s culture, people, and business processes usually provide the largest barrier to realizing the value from digital investments.    Yes, we […]

What is broke?  If I drive a pickup truck around that has a small, unobtrusive crack in the windshield and a few dings in the paint, it will still pull a boat and haul a bunch of lumber from Home Depot. Is the pickup broke if it still meets my needs? So, when is data […]

A few years ago, Big Data/Hadoop systems were generally a side project for either storing bulk data or for analytics. But now as companies  have pursued a data unification strategy, leveraging the Next Generation Data Architecture, Big Data and Hadoop systems are becoming a strategic necessity in the modern enterprise. Big Data and Hadoop are technologies […]

Data integration has changed.  The old way of extracting data, moving it to a new server, transforming it, and then loading into a new system for reporting and analytics is now looking quite arcane. It’s expensive, time consuming, and does not scale to handle the volumes we are now seeing in the digitally transformed enterprise. We […]

With the advent of Splice Machine and the release of Hive 0.14 we are seeing Hadoop’s role in the data center continue to grow. Both of these technologies support limited transactions against data stored in HDFS. Now, I would not suggest moving your mission-critical ERP systems to Hive or Splice Machine, but the support of […]

Tomorrow I will be giving a webinar on creating business cases for Big Data. One of the reasons for the webinar was that there is very little information available on creating a Big Data business cases. Most of what is available boils down to a “trust me, Big Data will be of value.” Most information […]

Last night I had a opportunity to see a demo of Splice Machine which was pretty cool from a technology perspective.  Splice Machine took Apache Derby, a lightweight ANSI SQL standard database and “spliced” it into HBase. This essentially created an SQL interface into Hbase. This product illustrates the power of combining different open source […]

One of the key points I make in our Executive Big Data Workshops is that effective use of Big Data analytics will require transforming both business and IT organizations.   Big Data with access to cross-functional data will transform the strategic processes within a company that guide long term and year to year investments. With the […]

A few years back I worked for a client that was implementing cell level security on every data structure within their data warehouse. They had nearly 1,000 tables and 200,000 columns — yikes! Talking about administrative overhead. The logic was that data access should only be given on a need-to-know basis. The idea would be […]

In the Hadoop space we have a number of terms for the Hadoop File System used for data management. Data Lake is probably the most popular. I have heard it called a Data Refinery as well as some other not so mentionable names. The one that has stuck with me has been is the Data […]

The data warehouse has been a part of the EIM vernacular for nearly 20 years. The vision of the single source of the truth and a single repository for reporting and analysis are two objectives that have resulted in a never-ending journey.   The data warehouse never has had enough data and the quality required for […]

Traditionally, in our information architectures we have a number of staging or intermediate data storage areas / systems.   These have taken different forms over the years, publish directories on source systems, staging areas in data warehouses, data vaults, or most commonly, data file hubs.   In general, these data file staging solutions have suffered from two […]