Enterprise Information Management Report Mining 'The easy way'

An Easier Way to Access Corporate Information

The most frequently used corporate information remains that old standby, the printed report.  ERP and other enterprise information systems typically offer a large library of existing reports, which can be run at anytime. 

Paper reports have many obvious shortcomings, including massive paper use in businesses worldwide.  A recent University of California, Berkeley study analysed paper consumption; “Contrary to notions of paperless offices ... the consumption of office paper has gone up substantially in recent years,” the report said.  The Worldwatch Institute estimated the average American office worker ploughs through 12,000 sheets of paper per year.   Experts believe global paper use will rise by 50% over this decade.

Hard Copy Reports: Still a Staple Diet of Corporate Information

Fortunately, existing reports are usually quite useful and embody a tremendous investment in programming work.  Reports must access and join data from multiple tables.  Business rules, including certain critical calculations performed when the report is run, are applied to the data, which may also be sorted, summarised and formatted to provide value to its recipients.

However, reports often contain too much information.  Also, the information is frozen on the printed page. There’s no way to get at that data, without digging through mountains of printouts and wasting time manually re-keying data from reports into spreadsheets or other applications.  Such re-keying is an error-prone process that wastes employee time and severely limits the ability to act upon data in a timely manner. 

Using Database Query Tools against Production Databases

SQL-based Business Intelligence (BI) applications seek to provide direct access to data within production databases, so managers can then request the exact information they need on an ad hoc basis. 

Using BI tools in this way was supposed to end the need to run and print most existing reports but organisations are printing more reports than ever. One problem is the data is sitting in operational systems, which are designed to quickly and efficiently process and store data for individual business transactions, not to quickly and efficiently enable analysis of data and making decisions.  Constant data-digging upon operational databases often slows applications.  

Complex data query tools can also create substantial end user frustration.   While suppliers of BI  applications have been saying for years that their products bring data analysis to front line workers, many say a degree in statistics and experience in SQL programming are required to use the software.
 
Often, direct end user access to data yields unsatisfactory results.  Operational systems suffer under the heavy load of query processing, frustrated managers spend too much time trying to build the correct query for the information they need and the data query burden often falls on the IT department.

Welcome to the Data Warehouse…Hardhats Required

To reap the promised benefits of high-end BI solutions, companies might implement a data warehouse or a data mart.  Both are intended to provide access to data for analysis and decision making without impacting on operational systems. 

To start building the data warehouse, “raw” data is extracted from operational databases at pre-determined intervals and is then combined into just a few monolithic tables.  This data cleansing and migration process is referred to as Extraction, Transformation and Loading (ETL).

ETL ensures that data from different operational data tables and perhaps completely different operational systems, is combined together accurately.  ETL is the most complex, longest-lasting, and error-prone part of a data warehousing project.

A key ETL challenge is the fact that the data going from operational systems into the warehouse or mart is “raw” – the data does not yet have any integrity because business rules have not yet been applied.  This is in contrast to existing reports, which already include certain calculations to make the raw data useful and correct for analysis and decision making.  All business rules must be included by data warehousing builders as part of the data ETL process. 

Organisations can yield a compelling return on investment and even reap tremendous business-transforming insights from a data warehouse.  However, data warehousing projects can also fail completely, often due to a faulty ETL process. Data from different systems is mapped together improperly; some needed data is not included in the data warehouse at all or wrong data is included into the data warehouse due to miscommunication between end users and IT personnel. 

However, Report Mining offers data warehouse and data mart projects a far greater and faster, opportunity for success, by dramatically eliminating much of the risks within the ETL process.

Putting it All Together: The Report Mining Alternative

Decision makers still resort to printed reports for information, despite huge efforts to provide direct access to data, either from operational systems or new data warehouses.  Yet, even after all of this powerful technology, one fact is undeniable: Printed reports steadfastly refuse to leave the business landscape.  End users still cannot do without them!

This fact leads us to some key questions:

• Given that reports are so pervasive, why not try to gain some brand new value from them? 
• What if the tried and true data buried within the report files themselves could be transformed into live, actionable data?
• What if end users could easily access the data within report files and then easily sort, filter, analyse and export the data, without programming?

The idea is simple: instead of running existing ERP, accounting or other enterprise reports to a printer, they are instead run to a print file or PDF file, and mined into actionable data by end users.  Datawatch’s Monarch software is the de facto standard in Report Mining. 

Using Report Mining, end users of any IT technology skill level can mine the massive amounts of data already captured in the organisation’s existing reports, immediately. In fact, no additional programming work of any kind is required to get actionable data into the hands of managers and workers. Users can, on their own, transform existing reports into live, actionable data and start discovering new business intelligence insights immediately.

“The real value of Report Mining tools,” says Gartner Group analyst Howard Dresner, “comes from their ability to promote access and analysis of report information directly, without an intermediary transformation process.”

Report Mining best enables managers and workers to utilise their business knowledge immediately.  For example, Report Mining tools leverage a manager’s knowledge of the company’s accounting reports; transaction reports; inventory reports, etc., as well as how the data within these reports is used to accomplish specific business processes. 

Report Mining not only serves as a means to convert text-based reports and PDF files into data, but also facilitates interactive analysis of that newly created data by the end user, such as:

• Sort and filter data
• Add new calculated fields of data, using formulas and functions
• Produce summaries of data, with subtotals, grand totals, and automatic graphs
• Open multiple copies of the same report in one Report Mining session for trending and other analysis (such as analysing three monthly reports at once to perform a quarterly analysis)
• Combine data from reports with data from another sources, such as spreadsheet, database files, ODBC data sources, etc.
• Export desired data to MS Excel, Access, PDF, and other applications

 

People often comment, “All of the data I need is buried somewhere in our paper reports. If only there was some easy way to get at it.”

That “easy way” is Report Mining.

 

To find out more about Datawatch Monarch Products, please click here to visit the product pages.

 

 

More information

Please use the menu above right for more information. Alternatively you can call us on 01962 835053 or email sales@avanquest.co.uk.