Data & Intelligence

Cognos TM1 Bulk Data Loading

Bulk Absorption Options

From a pure architectural perspective, what is sometimes referred to as “bulk uploading” sometimes is (mistakenly) considered to be part of reporting or included in the data consumption component of a TM1 application design.  Loading and/or updating data is part of an applications absorption or configurations components and should be handled with a best practice approach of leveraging TM1 TurboIntegrator scripts and/or active WebSheets (designed based upon user and environmental requirements and constraints).

The 3 best approaches to loading data in substance are:

  • Bulk Paste
  • File Route
  • Off-Line Template

bulkpaster

Bulk Paste

Data Intelligence - The Future of Big Data
The Future of Big Data

With some guidance, you can craft a data platform that is right for your organization’s needs and gets the most return from your data capital.

Get the Guide

If data volumes are manageable, using bulk past is most likely the best option. Bulk paste utilizes a TM1 Websheet that allows the business user to paste from the clip board all data to be loaded into the TM1 application (rather than inserting the data line by line). To make this effective, the worksheet must not include formatting that would prevent a “single paste” – this would include subtotals, “spacer” rows and/or columns or actually any Excel formatting that requires data to be manually (or programmatically) translated (back to a “raw” form). The general “rule of thumb” here is that the use of bulk paste should only be used when the data to be loaded fits on a single screen (little or no horizontal or vertical scrolling is required).

filerouter

File Route                 

For larger volumes of data, the File Route option should be considered. With this option, network administration or IT support is required to set-up (and maintain) a controlled network folder location where user “load files” can be placed. From this location, TurboIntegrator scripts can be written to identify the file, load the data and then move the file to an archive location. This option works best when “near real time” loading is acceptable. (In addition, a mechanism should be implemented to inform the appropriate personnel when or if load errors occur).

offline

Off-Line Template

The Off-Line template is certainty the most user-friendly. With this option, an Excel worksheet is programmed to allow a user to download data from a TM1 application (if applicable) and then automatically disconnect (from TM1) – this allows a business user to work locally and in a non-connected mode in a static (with no TM1 functions or formulas) Excel worksheet. Once the data is “ready”, the worksheet can validate and/or format the data and then connect to the TM1 application and load the information. Real time statuses are given. The drawback to using this option is that unless the business user is restricted to a local network connection, a virtual desktop must be utilized (such as Citrix or Terminal Services).

jm

About the Author

Mr. Miller is an IBM certified and accomplished Senior Project Leader and Application/System Architect-Developer with over 30 years of extensive applications and system design and development experience. His current role is National FPM Practice Leader. His experience includes BI, Web architecture & design, systems analysis, GUI design and testing, Database modeling and systems analysis, design, and development of Client/Server, Web and Mainframe applications and systems utilizing: Applix TM1 (including TM1 rules, TI, TM1Web and Planning Manager), dynaSight - ArcPlan, ASP, DHTML, XML, IIS, MS Visual Basic and VBA, Visual Studio, PERL, Websuite, MS SQL Server, ORACLE, SYBASE SQL Server, etc. His Responsibilities have included all aspects of Windows and SQL solution development and design including: analysis; GUI (and Web site) design; data modeling; table, screen/form and script development; SQL (and remote stored procedures and triggers) development and testing; test preparation and management and training of programming staff. Other experience includes development of ETL infrastructure such as data transfer automation between mainframe (DB2, Lawson, Great Plains, etc.) systems and client/server SQL server and Web based applications and integration of enterprise applications and data sources. In addition, Mr. Miller has acted as Internet Applications Development Manager responsible for the design, development, QA and delivery of multiple Web Sites including online trading applications, warehouse process control and scheduling systems and administrative and control applications. Mr. Miller also was responsible for the design, development and administration of a Web based financial reporting system for a 450 million dollar organization, reporting directly to the CFO and his executive team. Mr. Miller has also been responsible for managing and directing multiple resources in various management roles including project and team leader, lead developer and applications development director. Specialties Include: Cognos/TM1 Design and Development, Cognos Planning, IBM SPSS and Modeler, OLAP, Visual Basic, SQL Server, Forecasting and Planning; International Application Development, Business Intelligence, Project Development. IBM Certified Developer - Cognos TM1 (perfect score 100% on exam) IBM Certified Business Analyst - Cognos TM1

More from this Author

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Subscribe to the Weekly Blog Digest:

Sign Up