|
New technologies such as data warehouse appliances provide an alternative solution to the high costs of traditional data warehouse infrastructures. CIO’s are increasingly under pressure to provide more services, capacity and capabilities at a lower cost. Indeed, the TCO or Total Cost of Ownership is of increasing importance as significant investments are made. An increasing area of expense in many IT budgets is that of data warehousing. In fact, 97% of respondents in the 2005 IT Toolbox Data Warehouse Survey (conducted 10/24/05 through 11/1/05) reported their data warehouse budgets are continuing to grow in 2006. Managing the investments and TCO of investments in data warehousing is critical in containing costs and keeping budgets from going out of control.
The increase in data warehousing budgets is not without reason. Data warehousing has become main stream and in many cases mission critical for companies. The acceptance of data warehousing has generated increasing numbers of users and increasing volumes of data. Data volumes are being driven by the need for more frequent updates, new data sources and longer data retention requirements. In the Financial Services Industry data volumes are also being driven by compliance initiatives to support the requirements of legislative acts such as the Gramm-Leach Bliley Act, Sarbanes-Oxley, Basel II and SEC rules 17a-3 and 17a-4. A revealing data point from the 2005 IT Toolbox Data Warehouse survey is that more than 45% of the respondents expect their data warehouse size to increase at least 49%. 7.1% of the respondents expect this growth to be between 75 and 100%.
This growth in demand and data volume is outstripping the capacity of many existing data warehouse infrastructures. Upgrading existing infrastructure is very expensive and in many cases is cost prohibitive. This is especially so given that data warehouse environments will continue to grow on an annual basis. Therefore, IT organizations need to evaluate alternative technologies and solutions that can augment and/or retire their existing data warehouse infrastructure.
Data Warehouse appliances provide a complete data warehouse infrastructure including the hardware, storage, database and operating system. Using commodity components, these appliances are specifically configured and tuned for the demands of data warehousing workloads. Data warehouse appliances provide superior performance and pricing over traditional environments. Pricing for high performance data warehouse appliances can be as low as $90k per TB and $18k per TB for high capacity systems. Beyond pricing, there are many other benefits of an appliance that drive down the overall TCO. These benefits include low implementation costs, lower administration costs, lower upgrade costs, and lower maintenance costs. Let’s take a look at each of these costs as they apply to the data warehouse appliance.
Implementation of appliances tends to be a straight forward process. The two major activities of an implementation are the physical installation of the appliance and the migration or population of data in the appliance. Data migration or population can be done using existing data models (star schema or normalized). In many cases both of these activities can be accomplished in 30 days or less. Most vendors provide fixed price service packages for each of these implementation activities.
Beyond ease of implementation, one expects an appliance to be easy to administer. Indeed, in an appliance environment there is a blur between traditional system administrator and DBA tasks. This is because these roles are simplified in an appliance paradigm. For example, failover is built into the appliance and automated. There is no work required on the part of the system administrator to set this up. Backups are non-intrusive as they are executed in parallel using high speed interconnects such as Infiniband. On the software side, all upgrades to the appliance software and database components are bundled into a single upgrade.
For DBAs there is reduced workload in terms of tuning queries, creating indexes, creating temporary tables, maintaining aggregate tables and managing space. In general, queries run on an appliance will execute ten to one hundred times faster than in an existing environment. Therefore, queries require little to no tuning. Indexes are rarely used in an appliance. Rather, appliances scan data at a high rate of speed as they maximize sequential I/O. Appliances can reach scanning speeds of one terabyte per minute. Aggregation and temporary tables are not often required because of the extreme query performance. When aggregation tables are required, they are defined once and then automatically maintained by the appliance. This reduced workload for system administrators and DBAs translates into substantial labor savings.
Appliances use commodity hardware components in their MPP architecture. The use of commodity components drives down the initial hardware costs as well as upgrade costs. Hardware components can be upgraded as advancements in technology become available. As additional capacity is required, the size of an appliance can be expanded without replacing the entire appliance.
An additional benefit of appliances is that they have a smaller footprint in the data center. This results in reduced data center space requirements and cost allocations. For example, a 5 TB high performance appliance can fit in a single standard data center rack.
Another real tangible benefit with an appliance is the reduced fees a company pays for maintenance. With an appliance, there is only one maintenance fee. This fee covers the maintenance for hardware, software and database licensing. The low initial price of the appliance provides a baseline on which the maintenance fee is calculated. The result is a single maintenance fee for the entire data warehouse infrastructure at a fraction of the price of a traditional data warehouse infrastructure.
Data warehouse appliances provide a new solution for IT organizations that need to contain or reduce the costs of their data warehouse infrastructure. These appliances can be used to augment or over time replace an existing data warehouse infrastructure. The benefits of this approach are increased query performance, an initial investment that is a fraction of the cost of traditional approaches, and a lower TCO. The extreme value proposition of data warehouse appliances must be considered when facing data warehouse infrastructure investments or the need to provide more with less.
By Mark Theissen
Mark Theissen is a respected data warehouse and business intelligence expert. In his 22 years in the industry, Mark has worked in a variety of key roles including Vice President and Research Lead at META Group (now Gartner Group) for the Enterprise Analytics Practice. In his current position of Vice President of Professional Services at DATAllegro, Mark helps companies implement affordable and innovative data warehouse solutions using DATAllegro’s data warehouse appliances. Prior to working at Meta Group and DATAllegro, Mark was VP of Professional Services at Accruent and ran the professional and education services groups at Brio Technology (now Hyperion). Mark also worked as Data Warehouse Practice Director at Prism Solutions (now Ascential). You can reach Mark at mtheissen@datallegro.com |
