Building analytics apps? Remember IM!
Thursday, April 26, 2018
By Steve Cavolick
During my college years, there were rumors the food we were eating was not of the highest quality. Stories abounded how boxes of beef labeled “Grade D But Edible” were seen being delivered to the dormitory commissary. Of course this was not true, as the USDA does not grade meat with letters; beef cuts are Prime, Choice, Select, Standard, and Commercial.
For some students, these stories became reality, and they refused to eat certain meals because of it. That’s the problem with bad information: the wrong conclusions can be drawn and incorrect actions taken. A similar scenario can occur in analytics: if data is inaccurate, old, or aggregated incorrectly, analytical applications will deliver bad results. The way to eliminate this is through information management.
While not the sexy part of analytics, Information Management, or IM, is easily the most important part and cannot be an afterthought. As you consider platforms to provide your users with trusted data and enable collaboration between lines of business and IT, there are some capabilities those platforms should provide at a minimum. Those are:
- Data Discovery: Whether the data is loaded into a single repository or resides in multiple locations through virtualized data warehouses, you must understand all relationships that exist among data elements in your source systems. Automating this is the first step in ensuring your environment will have accurate and complete data. Often overlooked in this step, but something that must be included, is the discovery of sensitive and private data.
- Data Movement and Data Quality: Moving data in batch or in near real time is a given with a good Information Management platform, but those that stand out can validate data in real-time against defined data rules to ensure your data can be trusted for reporting.
- Business Glossary: Using a common vocabulary to define Key Performance Indicators, or KPIs, creates a trusted language for all to use and eliminates time spent in meetings debating the definition of “employee turnover,” for example. The glossaries will allow the business users to create the common vocabulary which technical resources can use to build data movement jobs. All of this should be easily accessible by both IT and the Lines of Business through a web interface.
- Lineage: User confidence in data is key to the success of an analytical application. Most users will not go back to the rows and columns of data to verify the accuracy of their report, but giving them access to the source of their data, its definition, and any transformations applied to it will give users the trust they need. The tool should also provide the name of a business steward who can provide additional information if questions still remain.
- Impact Analysis: More of an IT function, wouldn’t it be great to know what is going to be affected if a DBA changes the name of a column in your repository? Understanding the relationship between data structures, ETL jobs, and reports reduces unplanned downtime and increases user confidence in the environment.
Using the right IM platform can help you increase time to value of your data, keep staffing numbers down, and successfully deploy an analytics application.
The LRS Big Data and Analytics team has 20 years of experience in analytics and data warehousing, including Information Management and Governance. If you are interested in understanding how we can help enable you to provide trusted data to create a competitive advantage, please fill out the form below and let us know how we can help.
About the author
Steve Cavolick is a Senior Solution Architect with LRS IT Solutions. With over 20 years of experience in enterprise business analytics and information management, Steve is 100% focused on helping customers find value in their data to drive better business outcomes. Using technologies from best-of-breed vendors, he has created solutions for the retail, telco, manufacturing, distribution, financial services, gaming, and insurance industries.