|
So you did it right. You brought in the right brain-power. You listened carefully to your end users. You designed the perfect data warehouse. And you implemented an award-winning business intelligence application. Your execution was flawless …
Why then are queries taking so long to run? Why does it take so long to load the data? Why is it costing so much to maintain your data warehouse? More users are asking for more data … more quickly, but how will your data warehouse scale to meet ever-increasing demands? Price quotes to upgrade your existing infrastructure are keeping you awake at night. Have you created a monster?
Well, if you’re like the majority of large-scale data warehouse implementations; you can expect your monster (read: data warehouse) to increase in size by at least 49%. Some actually expect this growth to be between 75 and 100%! To handle these growing volumes, you will probably see your existing budget grow by over 10% over the next year (source: TDWI).
This growth in system demand and data volume is outstripping the capacity of many existing data warehouse infrastructures. Upgrading existing infrastructure is very expensive and, in many cases, cost-prohibitive. This is especially true given that data warehouse environments are expected to continue to grow on an annual basis. Therefore, IT organizations need to evaluate alternative technologies and solutions that can augment and/or retire their existing data warehouse infrastructure.
The solution: data warehouse appliances provide a complete data warehouse infrastructure including the hardware, storage, database and operating system. Using commodity components, these appliances are specifically configured and tuned for the demands of data warehousing workloads which include large-scale data loading and high-speed concurrent data access.
Data warehouse appliances such as those offered by DATAllegro provide superior performance and pricing over traditional environments. Pricing for DATAllegro's high-performance data warehouse appliances can be as low as $24k per TB. Beyond pricing, there are many other benefits of an appliance that drive down the overall Total Cost of Ownership (TCO). These benefits include low implementation costs, lower administration costs, lower upgrade costs, and lower maintenance costs.
Let’s take a look at each of these costs as they apply to the data warehouse appliance:
- Implementation of data warehouse appliances tends to be a straight-forward process. The two major activities of an implementation are the physical installation of the appliance and the migration or population of data in the appliance. Data migration or population can be done using existing data models (star schema or normalized). In many cases both of these activities can be accomplished in 30 days or less. Vendors often provide fixed-price service packages for each of these implementation activities.
- In most data warehouse appliance environments there is a blur between traditional system administrator and DBA tasks. This is because these roles are simplified. For example, failover is often built into the appliance and automated so that no work is required on the part of the system administrator to set it up. Backups are non-intrusive as they are executed in parallel using high speed interconnects such as Infiniband. On the software side, all upgrades to the appliance software and database components are bundled into a single upgrade.
- For DBA’s appliance technology means reduced workload in terms of tuning queries, creating indexes, creating temporary tables, maintaining aggregate tables and managing space. In general, queries running on a data warehouse appliance will execute ten to one hundred times faster than in an existing environment. Therefore, queries require little to no tuning. Indexes are rarely used but can be when they can help performance. Instead, data is scanned at a high rate of speed and sequential I/O is maximized. For example, a five (5) Terabyte appliance can offer a scanning speed of one (1) Terabyte per minute. Aggregation and temporary tables are not often required because of the extreme query performance. When aggregation tables are required, they are defined once and then automatically maintained by the appliance. This reduced workload for system administrators and DBA’s translates into substantial labor savings.
- Some data warehouse appliances such as those offered by DATAllegro use commodity hardware components configured into a massively parallel processing (MPP) architecture. The use of commodity components drives down the initial hardware costs as well as upgrade costs. Hardware components can be upgraded as advancements in technology become available. As additional capacity is required, the size of an appliance can be expanded without replacing the entire appliance.
- Appliances have a smaller footprint in the data center. This results in reduced data center space requirements and cost allocations. For example, a 5 TB high performance appliance can fit in a single standard data center rack.
- Maintenance fees are dramatically reduced. With appliances, there is only one maintenance fee. This fee covers the maintenance for hardware, software and database licensing. The low initial price of the appliance provides a baseline on which the maintenance fee is calculated. The result is a single maintenance fee for the entire data warehouse infrastructure at a fraction of the price of a traditional data warehouse infrastructure.
Obviously, TCO is not the only factor to consider when evaluating data warehouse appliances … you should also consider system performance. There have to be substantial performance improvements for both end-user queries and data loads.
Some data warehouse appliance vendors very compelling evidence with regards to query performance. The chart below shows benchmark results from a number of queries run on a DATAllegro P-series appliance compared to another popular data warehouse platform. The data warehouse size, for this benchmark, was 3 Terabytes. Each of the queries varied in complexity, but each contained various co-related sub-queries, inner joins as well as complex scalar functions such as “rank”.
Larger Image
Note that each query ran in remarkably less time on a DATAllegro appliance, but more importantly, no indexes were used and the time to actually build and run the benchmark was less than one week!
Clearly, data warehouse appliances provide a whole new solution for IT organizations that need to contain or reduce the costs of their data warehouse infrastructure. They can be used to augment or, over time, even replace an existing data warehouse infrastructure. The benefits of this approach are increased query performance, increased query concurrency, an initial investment that is a fraction of the cost of traditional approaches, and a lower TCO.
Of course, benchmark results with your data may vary, but some data warehouse appliance vendors are so sure that their appliances will provide optimal performance at the lowest cost that they are willing to put it to the test … for free! If you are seriously considering the use of data warehouse appliances, call one of the vendors and ask to try it out. They’ll be happy to prove to you how they can help you sleep a little easier by taming your data warehouse monster.
By Jesse Fountain
Jesse Fountain is currently Senior Sales Consultant at DATAllegro. Jesse has over 25 years of experience it information technology, covering such areas as Decision Support, ETL and Business Intelligence. At DATAllegro Jesse works with customers from all industries to help them improve their TCO and leverage the benefits of data warehouse appliances. You can reach Jesse at jessef@datallegro.com
|
