DATAllegro v3 data warehouse appliances signify the birth of a new generation of data warehouse technology based on open, flexible platforms. The initial generation of data warehouse appliances disrupted the data warehouse market with affordable high performance that was achieved by using proprietary hardware. The second generation is likely to be just as disruptive. Proprietary technologies usually only succeed in the early stages of a new market. In the medium to long term, they are usually replaced with standards-based solutions. Second generation appliances use best-of-breed platforms to leverage technological advances and extend the already impressive price/performance advantages of this new paradigm in data warehousing.

POWERFUL PARTNERS
Gartner Dataquest estimates the size of the data warehouse market at $30BN. The sheer size of the market, coupled with relatively high growth, makes it very attractive to major vendors. As a leader in the storage arena, EMC® has been actively looking for a suitable partner among the new DWA vendors and found that DATAllegro provided a quality match with its open and flexible architecture. The partnership with EMC allows DATAllegro appliances to use EMC CLARiiON storage to provide high performance and enterprise level reliability.

DATAllegro is also partnering with Dell™ & Intel® for Quad-Core Xeon® servers and Cisco® for the latest technology in InfiniBand networking.

For its database engine, DATAllegro has partnered with Ingres®, which offers an ideal combination of a proven, enterprise-class DBMS with an open source business model.

These partners helped bring to fruition the DATAllegro v3 platform.

FLEXIBLE OPEN ARCHITECTURE
DATAllegro v3 offers a flexible, open architecture, allowing customers to scale up or scale out their data warehouse as needed. The separation of storage and compute nodes allows for flexible price / performance combinations while balancing disk and CPU workloads. DATAllegro v3 offers two ranges - Single Rack Appliances (SRA) for companies with data warehouses that won’t exceed 12TB and Multi-Rack Appliances (MRA) for companies with large scale data warehouse requirements.

The SRA is a fully contained data warehouse within a standard rack, complete with control, data and storage nodes as well as warm spares and a specialized backup node. The SRA12 contains six compute nodes, three EMC CX3 series storage nodes and offers 12TB of user data capacity for under $500K. The SRA12 cannot be expanded beyond 12TB. In addition to being deployed as a data warehouse, the SRA12 can be used as a high capacity test, development or QA environment, as well as a sandbox for ad-hoc querying.

The MRA product line allows multiple data storage racks (DR) to be combined with a single control rack (CR) to create appliances that can scale from 15TB to a petabyte. Data racks in the MRA range contain eight compute nodes and four EMC CX3 storage nodes.

The MRA product line has two sizes of data racks. The DR1530 offers 15TB of user data capacity per rack and can be expanded up to 30TB with the release of on-demand storage. The DR2550 has 25TB user data capacity and can be expanded to 50TB. Up to twenty DR units can be combined to create a single DWA with up to 1PB capacity.

Moreover, multiple appliances can be combined on a common InfiniBand backbone to create large scale and extremely powerful multi-tier or hub-and-spoke data warehouses with rapid, parallel data movement between the various appliances.

ARCHITECTED FOR PERFORMANCE
DATAllegro v3 offers improved performance over previous platforms through hardware and software optimization:

• Increased CPU Throughput: DATAllegro v3 increases throughput of compute nodes through the use of Intel's Quad-Core Xeon CPUs. Multi-core processors provide improved throughput and parallel processing, resulting in a significant performance increase over previous single and dual core processors. Data for each compute node is partitioned into six files on dedicated disks with a shared storage node. Multi-core allows each of these six partitions to be read in parallel. Data is streamed off these partitions using DATAllegro Direct Data Streaming™ (DDS) technology that maximizes sequential reads from each disk in the array. DDS ensures the appliance architecture is not I/O bound and therefore pegged by the rate of improvement of storage technology. As a result, read rates of over 1.2GBps per compute node are possible. The massively parallel processing architecture and DDS produces table scans ranging from 0.5TB / minute to 10.5TB / minute. These rates are likely to improve as DATAllegro rides the multi-core wave.
  
  
• I/O Rationalization: DATAllegro v3 separates workspace and user data space into dedicated storage areas. Workspace I/O is inherently random, while user data space is ideally accessed using sequential I/O as in the DATAllegro architecture. Workspace is housed on each compute node using six 146GB 15k rpm drives. User data space is stored on the EMC storage node using RAID1. The physical separation improves performance by reducing disk head movement.
  
  
• Compression expands throughput: Within each node, two of the multi-core processors are reserved for software compression. This increases I/O throughput from 800MBps from the shared storage node to over 1.2GBps for each compute node.
  
  
• Improved write efficiency: The EMC storage unit provides RAID1 mirroring and a hot spare to provide enterprise-class reliability. As a result, DATAllegro is able to optimize the write process to write data once, relying on EMC to ensure fault tolerance. In first generation appliances, the software on the appliance takes responsibility for fault tolerance and has to write each record twice on different nodes or SPUs.

HIGH AVAILABILITY
Unlike first generation competitors, DATAllegro v3 has no single point of failure and does not suffer from significantly reduced performance during any failover recovery processes. Redundancy is built in at all levels resulting in enterprise-class reliability and high availability.

• Best-of-Breed Servers with Hot Spare: DATAllegro v3 uses standard 2U Dell servers for compute nodes. In the MRA series, each compute node has two Quad-Core Intel Xeon processors running at 2.66GHz, 16GB of RAM and six hot-swap enterprise-class SAS high speed disks used for workspace. If one of the drives fails, the compute node still operates with no loss of service. In addition, if a server should fail for any reason, the appliance will automatically switch over to one of the hot spare servers that is included with every data rack.
  
  
• Enterprise-Class Storage: Each pair of compute nodes in a DATAllegro v3 appliance is directly connected to a shared EMC CX3 storage server. These units include dual 4Gb FC controllers, hot-standby disks, dual power supplies and hot-swappable components. Disk failure does not result in outage of service or significant performance degradation as the disk array rebuilds itself using the hot spare.
  
  
• Redundant Power Domains: Each rack in the DATAllegro v3 DWA includes dual, fully redundant power domains.
  
  
• InfiniBand Interconnect: The nodes within a DATAllegro v3 appliance are connected together using dual 10Gbps InfiniBand Interconnect. The RDMA protocols in use result in minimal CPU loading on servers during large data transfers. The interconnect has over twenty times the bandwidth of the GigE networks found in many first generation appliances.
  
  
• Open Source Operating System: Each compute, master and backup node runs the open source CentOS Linux 64 bit operating system.
  
  
• Disaster Recovery: Active / Active disaster recovery over multiple appliances offer site autonomy and options to deal with a catastrophic site failure.
  
  

SIMPLICITY WITH SOPHISTICATION
The marketing for first generation data warehouse appliances stresses the simplicity of the solution. Unfortunately, this only satisfies business needs that are well defined and can be solved with a simple solution. In reality, appliances require a level of sophistication in order to be able to solve multiple problems. Data warehouses are commonly required to provide enterprise and ad-hoc reporting, aggregation, concurrent loading and mixed work loads. The DATAllegro v3 platform provides the level of sophistication required to address these needs. DATAllegro v3 offers new abilities to handle complex workloads that are a mixture of near real-time loads, long analytical queries and short quick-hit queries. Workload management is improved, where short queries automatically obtain higher priority, reducing the overall workload on the server.

In addition, the unique ability of DATAllegro v3 to support multiple appliances in a single, easy to manage data warehouse platform makes it easy to divide and conquer complex workloads.

ARCHITECTED FOR CHANGE
Proprietary solutions create roadblocks in adapting to change and often require forklift upgrades. Standards-based platforms are architected for change and are able to embrace technological advance. DATAllegro v3 reduces data warehousing risks and protects data warehouse assets by being able to accommodate workload changes, scalability or user concurrency changes. DATAllegro v3 marks a shift in the data warehouse appliance market to second-generation, standards-based platforms that enable even further reductions in total cost of ownership. Finally, DATAllegro has secured partnerships with major vendors such as EMC, Intel, Dell & Cisco that will enable the company to outpace its proprietary-based competitors over the next few years.