SQL Server

An OLAP Requirements Example: CompSales International (part 15) - SSIS

- Free product key for windows 10
- Free Product Key for Microsoft office 365
- Malwarebytes Premium 3.7.1 Serial Keys (LifeTime) 2019
12/16/2010 11:39:07 AM

SSIS provides a robust means to move data between sources and targets. Data can be exported, validated, cleaned up, consolidated, transformed, and then imported into a destination of any kind. With any OLAP/SSAS implementation, you will undoubtedly have to transform, clean, or preprocess data in some way. You can now tap into SSIS capabilities from within the SSAS platform.

You can combine multiple column values into a single calculated destination column or divide column values from a single source column into multiple destination columns. You might need to translate values in operational systems. For example, many OLTP systems use product codes stored as numeric data. Few people are willing to memorize an entire collection of product codes. An entry of 100235 for a type of shampoo in a product dimension table is useless to a vice president of marketing who is interested in how much of that shampoo was sold in California in the past quarter.

Cleanup and validation of data are critical to the data’s value in the data warehouse. The old saying “garbage in, garbage out” applies. If data is missing, redundant, or inconsistent, high-level aggregations can be inaccurate, so you should at least know that these conditions exist. Perhaps data should be rejected for use in the warehouse until the source data can be reconciled. If the shampoo of interest to the vice president is called Shamp in one database and Shampoo in another, aggregations on either value would not produce complete information about the product.

The SSIS packages define the steps in a transformation workflow. You can execute the steps serially and in combinations of serially, in parallel, or conditionally.

OLAP Performance

Performance is a big emphasis of SSAS. Usage-based aggregation is at the heart of much of what you can do to help in this area. In addition, the proactive caching mechanism in SSAS has allowed much of what was previously a bottleneck (and a slowdown) to be circumvented.

When designing cubes for deployment, you should consider the data scope of all the data accesses (that is, all the OLAP queries that will ever touch the cube). You should only build a cube that is big enough to handle these known data scopes. If you don’t have requirements for something, you shouldn’t build it. This helps keep things a smaller, more manageable size (that is, smaller cubes), which translates into faster overall performance for those who use the cube.

You can also take caching to the extreme by relocating the OLAP physical storage components on a solid-state disk device (that is, a persistent memory device). This can give you tenfold performance gains. The price of this type of technology has been dramatically reduced within the past year or so, and the ease of transparently applying this type of solution to OLAP is a natural fit. It affects both the OLAP data population process and the day-to-day what-if usage by the end users. You should keep these types of surgical incisions in mind when you face OLAP performance issues in this platform. They are easy to apply, the gains are huge, and you quickly get a return on your investment.

MPP Data Warehouse Option from Microsoft

A few years ago, Microsoft acquired DATAllegro’s massively parallel data warehouse appliance company. This basically lifted any limitations for data warehousing that SSAS or SQL Server 2008 R2 itself had. Massively parallel means to scale horizontally on CPU and storage to grow with your size and processing needs. There is no practical limit here. The underlying architecture relies on standards-based technologies. Essentially, there is a separation of storage and compute nodes that allows you to spread out your data across vast storage (EMC storage) so that it is very shallow (easy to get to quickly across all data storage). The compute power is also horizontally scalable and allows any query to process data access in parallel to surface data needed by any query (and assemble it for delivery). Figure 65 shows the high-level architecture of Microsoft’s DATAllegro v3 offering.

Figure 65. The DATAllegro v3 MPP architecture.

Not only is the DATAllegro v3 architecture massively parallel and fast, but the multinode architecture also makes it highly available. If any node fails, hot spares kick in to pick up the load. Any failed node can easily be replaced and brought online with zero processing interruption. Moreover, multiple appliances can be combined on a common InfiniBand backbone to create large-scale and extremely powerful multitier or hub-and-spoke data warehouses with rapid, parallel data movement between the various appliances. Believe it or not, there is an Ingres SQL engine at the heart of the database portion of this appliance.

Master Data Services

Completing the business intelligence picture is a new focus on the data quality that is needed at all tiers of data information delivery. Microsoft has been pouring an enormous amount of effort (and money) into creating and embedding master data services throughout its BI and transactional platforms. By using Microsoft’s Master Data Services, organizations can align operational and analytical data across the enterprise and across lines of business systems with a guaranteed level of data quality for most core data categories (such as customer data, product data, and other core data of the business).

Microsoft has created data stewardship capabilities complete with workflows and notifications of any business user who might be impacted by core data change. Managing hierarchies is also an important part of mastering data that has a natural hierarchical structure, such as customer hierarchies (parent company to subsidiaries and so on). Each master data change within the system is treated as a transaction; and the user, date, and time of each change are logged, as well as pertinent audit details, such as type of change, member code, and prior versus new value. In addition to being a very useful audit trail, the transaction log can be used to selectively reverse changes. Customizable data quality rules create default values, enable data validation, and trigger actions such as email notifications and workflows. Rules can be built by IT professionals or business users directly from the stewardship portal.

Microsoft is still getting the kinks out of Master Data Services, so you should look for much maturing to come in the next few years. Other competing products that have many years’ headstart provide this capability to companies around the globe, but Microsoft is catching up fast.

Other -----------------
- SQL Server 2008 Analysis Services : An Analytics Design Methodology
- SQL Azure : Other Considerations
- SQL Azure : Sample Design - Application SLA Monitoring
- SQL Azure : Combining Patterns
- SQL Server 2008 Analysis Services : Understanding the SSAS Environment Wizards (part 2)
- SQL Server 2008 Analysis Services : Understanding the SSAS Environment Wizards (part 1)
- SQL Server 2008 Analysis Services : Understanding SSAS and OLAP
- SQL Azure : Design Patterns (part 3)
- SQL Azure : Design Patterns (part 2) - Sharding
- SQL Azure : Design Patterns (part 1)
- SQL Azure : Design Factors (part 2)
- SQL Azure : Design Factors (part 1)
- Limitations in SQL Azure
- SQL Server 2008 : Performance Data Collection (part 2)
- SQL Server 2008 : Performance Data Collection (part 1)
- SQL Server 2008 : Performance Tuning - Partitioning
- SQL Server 2008 : Guide to the DYNAMIC Management Views (DMVs)
- SQL Server 2008 : Managing Security - Service Accounts and Permissions
- SQL Server 2008 : Managing Security - Security and SQL Agent
- SQL Server 2008 : Implementing Transactions - Transaction Traps
Top 10
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Finding containers and lists in Visio (part 2) - Wireframes,Legends
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Finding containers and lists in Visio (part 1) - Swimlanes
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Formatting and sizing lists
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Adding shapes to lists
- Microsoft Visio 2013 : Adding Structure to Your Diagrams - Sizing containers
- Microsoft Access 2010 : Control Properties and Why to Use Them (part 3) - The Other Properties of a Control
- Microsoft Access 2010 : Control Properties and Why to Use Them (part 2) - The Data Properties of a Control
- Microsoft Access 2010 : Control Properties and Why to Use Them (part 1) - The Format Properties of a Control
- Microsoft Access 2010 : Form Properties and Why Should You Use Them - Working with the Properties Window
- Microsoft Visio 2013 : Using the Organization Chart Wizard with new data
- First look: Apple Watch

- 3 Tips for Maintaining Your Cell Phone Battery (part 1)

- 3 Tips for Maintaining Your Cell Phone Battery (part 2)
programming4us programming4us