Agile Microsoft BI: Enabling a Healthy Feedback Cycle

The IT Leader's Guide to Multicloud Readiness

This guide provides practical key insights and important factors to consider to make informed decisions in your multicloud journey.

Download the Guide

I recently worked with a client who brought on the Perficient BI team to do some very interesting data warehousing work. This client has had a data warehouse for many years, however their current solution is difficult to maintain and use. Frustrated with long cycle times for additional features and report requests, this client opted to build a new data warehouse on Microsoft’s BI stack using an Agile methodology.
Sounds great, right? Let’s get in there and plan an iteration. And it seemed pretty straightforward. The team set up a really interesting software factory complete with automated builds and test automation. Working with a steering committee they cranked out the models, built ETL, and spit out the early dimensions. And here’s where things went off the rails a little.
It was time to get feedback, and that feedback needed to be detailed. The steering committee had approved high-level models but these models needed to be pressure-tested with real data. This steering committee suffered from the same disease I’ve seen afflict many stakeholders new to Agile: “hands-off syndrome”. As much as business owners dislike the misalignments that can result from a waterfall methodology, they seem to dislike devoting consistent time to the project even more. Even my primary project sponsor had underestimated the amount of time he needed to set aside for the project, and attended daily standups only sporadically. When the first big modeling error was discovered I started asking some uncomfortable questions about how steering committee members were checking functionality. Eventually I found blocking issues standing in the way of both end-user delivery mechanisms we had targeted for this project.
Excel 2010, with or without PowerPivot: Insufficient RAM combined with the 32-bit version of Office running standard on local machines prevented reasonable use of this tool. Requests to develop a more powerful, 64-bit loaner test machine were rebuffed by the infrastructure manager.
SharePoint: Budget constraints had prevented the client from requesting hardware for even the development farm on schedule. When the hardware eventually arrived, its build was so delayed that it missed the boat on the first two data warehouse releases.
In the end, this client was forced to limp along using radically smaller data sets viewed via vanilla Excel 2010 until their SharePoint implementation spun up. As is so often the case, the fact that we ran into an issue soliciting feedback was not the schedule-killer. It was the fact that we weren’t communicating regularly about progress that let the problems compound.
If I knew then what I know now, I would have been much more purposeful about setting up good collaboration patterns with the business owner group.  It sounds so old-fashioned, but I truly believe that I should have asked a business owner to physically sit with the team one day a week. We would have seen their frustrations with Excel’s slowness on their underpowered hardware immediately, while there was still enough time to get the SharePoint hardware expedited.