SkillRary

A Data Warehousing (DW) is process for collecting and managing data from varied sources to provide meaningful business insights. A Data warehouse is typically used to connect and analyse business data from heterogeneous sources. The data warehouse is the core of the BI system which is built for data analysis and reporting.

It is a blend of technologies and components which aids the strategic use of data. It is electronic storage of a large amount of information by a business which is designed for query and analysis instead of transaction processing. It is a process of transforming data into information and making it available to users in a timely manner to make a difference.

Data warehousing is the electronic storage of a large amount of information by a business or organization. Data warehousing is a vital component of business intelligence that employs analytical techniques on business data.

The concept of data warehousing was introduced in 1988 by IBM researchers Barry Devlin and Paul Murphy. The need to warehouse data evolved as computer systems became more complex and handled increasing amounts of data. A key book on data warehousing is W. H. Inmon's "Building the Data Warehouse," which was first published in 1990 and has been reprinted several times since.

Data warehousing is used to provide greater insight into the performance of a company by comparing data consolidated from multiple heterogeneous sources. A data warehouse is designed to run query and analysis on historical data derived from transactional sources.

Once the data has been incorporated into the warehouse, it does not change and cannot be altered since a data warehouse runs analytics on events that have already occurred by focusing on the changes in data over time. Warehoused data must be stored in a manner that is secure, reliable, easy to retrieve and easy to manage.

There are certain steps that are taken to create a data warehouse. The first step is data extraction, which involves gathering large amounts of data from multiple source points. After the data has been compiled, it goes through data cleaning, the process of combing through the data for errors and correcting or excluding any errors found.

The cleaned-up data is then converted from a database format to a warehouse format. Once it’s stored in the warehouse, the data goes through sorting, consolidating, summarizing, etc. so that it’s more coordinated and easier to use. Over time, more data is added to the warehouse as the multiple data sources are updated.

Some benefits of a data warehouse

Organizations that use a data warehouse to assist their analytics and business intelligence see a number of substantial benefits:

• Better data — Adding data sources to a data warehouse enables organizations to ensure that they are collecting consistent and relevant data from that source. They don’t need to wonder whether the data will be accessible or inconsistent as it comes in to the system. This ensures higher data quality and data integrity for sound decision making.
• Faster decisions — Data in a warehouse is in such consistent formats that it is ready to be analysed. It also provides the analytical power and a more complete dataset to base decisions on hard facts. Therefore, decision makers no longer need to reply on hunches, incomplete data, or poor quality data and risk delivering slow and inaccurate results.

What a data warehouse is not

1. It is not a database

It’s easy to confuse a data warehouse with a database, since both concepts share some similarities. The primary difference, however, comes into effect when a business needs to perform analytics on a large data collection. Data warehouses are made to handle this type of task, while databases are not.

2. It is not a data lake

Although they both are built for business analytics purposes, the major difference between a data lake and a data warehouse is that a data lake stores all types of raw, structured, and unstructured data from all data sources in its native format until it is needed. By contrast, a data warehouse stores data in files or folders in a more organized fashion that is readily available for reporting and data analysis.

3. It is not a data mart

Data warehouses are also sometimes confused with data marts. But data warehouses are generally much bigger and contain a greater variety of data, while data marts are limited in their application.

Data marts are often subsets of a warehouse, designed to easily deliver specific data to a specific user, for a specific application. In the simplest terms, data marts can be thought of as single-subject, while data warehouses cover multiple subjects.

The future of the data warehouse: move to the cloud

As businesses make the move to the cloud, so too do their databases and data warehousing tools. The cloud offers many advantages: flexibility, collaboration, and accessibility from anywhere, to name a few. Popular tools like Amazon Redshift, Microsoft Azure SQL Data Warehouse, Snowflake, Google BigQuery, and have all offered businesses simple ways to warehouse and analyse their cloud data.

The cloud model lowers the barriers to entry — especially cost, complexity, and lengthy time-to-value — that have traditionally limited the adoption and successful use of data warehousing technology. It permits an organization to scale up or scale down — to turn on or turn off — data warehouse capacity as needed. Plus, it’s fast and easy to get started with a cloud data warehouse. Doing so requires neither a huge up-front investment nor a time-consuming (and no less costly) deployment process.

The cloud data warehouse architecture largely eliminates the risks endemic to the on-premises data warehouse paradigm. You don’t have to budget for and procure hardware and software. You don’t have to set aside a budget line item for annual maintenance and support. In the cloud, the cost considerations that have traditionally preoccupied data warehouse teams — budgeting for planned and unplanned system upgrades — go away.