Agile Data Warehouse Design

Agile techniques emphasise the early and frequent delivery of working software, stakeholder collaboration, responsiveness to change and waste elimination. They have revolutionised application development and are increasingly being adopted by DW/BI teams. This course provides practical tools and techniques for applying agility to the design of DW/BI database schemas – the earliest needed and most important working software for BI.

The course contrasts agile and non-agile DW/BI development and highlights the inherent failings of traditional BI requirements analysis and data modeling. Via class room sessions and team exercises attendees will discover how modelstorming (modeling + brainstorming) data requirements directly with BI stakeholders overcomes these limitations.

Learning objectives

You will learn how to:

• Model BI requirements with BI stakeholders using inclusive tools and visual thinking techniques
• Rapidly translate BI requirements into efficient, flexible data warehouse designs
• Identify and solve common BI problems – before they occur – using dimensional design patterns
• Plan, design and incrementally develop BI solutions with agility

Who Should Attend

• Business and IT professionals who want to develop better BI solutions faster.
• Business analysts, scrum masters, data modelers/architects, DBA’s and application developers, new to DW/BI, will benefit from the solid grounding in dimensional modeling provided.
• Experienced DW/BI practitioners will find the course updates their hard-earned industry knowledge with the latest ideas on agile modeling, data warehouse design patterns and business model innovation.

You receive a free copy of the book Agile Data Warehouse Design by Lawrence Corr.

Course description

Day 1: Modelstorming – Agile BI Requirements Gathering

Agile Dimensional Modeling Fundamentals

• BI/DW design requirements, challenges and opportunities: the need for agility
• Modeling with BI stakeholders: the case for collaborative data modeling
• Modeling for measurement: the case for dimensional modeling, star schemas, facts & dimensions
• Thinking dimensional using the 7Ws (who, what, when, where, how many, why & how)
• Business Event Analysis and Modeling (BEAM✲): an agile approach to dimensional modeling

Dimensional Modelstorming Tools

• Data stories, themes and BEAM✲ tables: modeling BI data requirements by example
• Timelines: modeling time and process measurement
• Hierarchy charts: modeling dimensional drill-downs and rollups
• Change stories: capturing historical reporting requirements (slowly changing dimension rules)
• Storyboarding the data warehouse design: matrix planning and estimating for agile BI development
• The Business Model Canvas: aligning DW/BI design with business model definition and innovation
• The BI Model Canvas: a systematic approach to BI & star schema design

Day 2: Agile Star Schema Design

Star Schema Design

• Test-driven design: agile/lean data profiling for validating and improving requirements models
• Data warehouse reuse: identifying, defining and developing conformed dimensions and facts
• Balancing ‘just enough design up front’ (JEDUF) and ‘just in time’ (JIT) data modeling
• Designing flexible, high performance star schemas: maximising the benefits of surrogate keys
• Refactoring star schemas: responding to change, dealing with data debt
• Lean (minimum viable) DW documentation: enhanced star schemas, DW matrix

How Much/How Many: Designing facts, measures and KPIs (Key Performance Indicators)

• Fact types: transactions, periodic snapshots, accumulating snapshots
• Fact additivity: additive, semi-additive and non-additive measures
• Fact performance and usability: indexing, partitioning, aggregating and consolidating facts

Day 3: Dimensional Design Patterns

Who & What dimension patterns: customers, employees, products and services

• Large populations with rapidly changing dimensional attributes: mini-dimensions & customer facts
• Customer segmentation: business to business (B2B), business to consumer (B2C) dimensions
• Recursive customer relationships and organisation structures: variable-depth hierarchy maps
• Current and historical reporting perspectives: hybrid slowly changing dimensions
• Mixed business models: heterogeneous products/services, diverse attribution, ragged hierarchies
• Product and service decomposition: component (bill of materials) and product unbundling analysis

When & Where dimension patterns: dates, times and locations

• Flexible date handling, ad-hoc date ranges and year-to-date analysis
• Modeling time as dimensions and facts
• Multinational BI: national languages reporting, multiple currencies, time zones & national calendars
• Understanding journeys and trajectories: modeling events with multiple geographies

Why & How dimension patterns: cause and effect

• Causal factors: trigging events, referrals, promotions, weather and exception reason dimensions
• Fact specific dimensions: transaction and event status descriptions
• Multi-valued dimensions: bridge tables, weighting factors, impact and ‘correctly weighted’ analysis
• Behaviour Tagging: modeling causation and outcome, dimensional overloading, step dimensions