Trends in Data Warehousing

 (adopted from Business Times in my own personal archive)

Warehousing or data mart, Olap (on-line analytical processing), decision support systems -- whatever the term, it is the fastest growing segment of commercial IT spending. Studies show that in the US itself, the business for 1997 was worth US$16.8 million (S$28.5 million) and the forecast is that it will continue to see an annual growth of 20 to 30 per cent per year.

Companies are investing in data warehouses because they see it as an important business tool to increase competitiveness. It is not surprising to note then that business drivers, not technology issues, continue to be the most prominent driving force behind the proliferation of data warehousing.

The corporation must know its business needs and focus on business drivers -- for example, profitability -- which may be attained by increasing market share, better management of assets or lowering costs. Data warehouse implementations are successful when the business becomes more focused and when it knows how to increase its competitiveness. Then it can use IT to improve its competitiveness even more.

But most end-users cannot wait for their IT departments to figure out how to build a data warehouse in the way they want. In their haste, they may build a data mart for each of their business functions. The multiple number of independent data marts erected across the organisation could lead to information chaos.

This problem is starting to fizzle out though. Data warehouse applications are getting more mature. The time frame for their implementation, too, has been greatly reduced. From building their own applications from scratch, users have started to buy utilities and applications from vendors like SAP and PeopleSoft, who roll out application-specific packages for data warehousing.

The more mature data warehouses comprise a two-stage approach where there is a corporate data warehouse in which enterprise business rules are applied and general data cleansing/transformation is conducted.

Then there is the data mart level, where departmental business rules relevant only to that department apply.Some of the technologies that are beginning to emerge in the data warehouse market include web-based deployment and closed-loop decision support.

Web-based deployment is a fast growing segment within the data warehouse space. With web-based deployment, users do not have to worry about the different platforms for hardware and software, and even software distribution. They can even dial-in to their server to download the information that they need. Closed-loop decision support will grow in significance in the future where software solutions will be made available to automate routine follow-ups, usually based on the fixed analysis of information.

In a nutshell, companies will be focusing on their core business. Automation will let companies use data warehousing even more effectively to help them be more competitive. Telecommunications and financial services companies -- which operate in a very competitive field -- are already latching on to this technology.

Companies are using data warehouses to produce up-to-date statutory reports and to ensure that there is transparency of information both inside and outside the organisation. Repository sales have increased. With a repository, users know exactly where their assets are, and what business rules apply to them. The companies mainly using data warehouses in this region are in the banking and telecommunications sectors, due perhaps to the increased competition with the opening up of the markets here.