Fishing the Data Lake
Whether you have a “data lake” or a “data warehouse” or just plain old data, fishing for value is the next step. For procurement analysis, the value step can be tricky, because chances are:
- The “lake” is missing the specific data needed
- The data needs additional transformation
- There’s too much data for a spreadsheet model
Examples of missing data include contracts and terms, price x quantity detail, and rate tables. Often the only source of invoice-level detail is the vendor; and if contract rates and dates are unavailable, so is any hope of auditing contract compliance. Org charts and employee counts are usually missing as well, but are critical for demand management analysis.
Even when data is available, additional transformation is always required. Are there preferred vendors? This is information with a high rate of change that has no analog in corporate data systems. Are vendors grouped properly? Mapped properly to commodity? This is also information with a high rate of change (don’t believe anyone who tells you otherwise). Data lakes don’t transform data – they just store it, and at best dump it into a read-only BI tool.
Even if all of the above is readily available in the data lake (it isn’t, but let’s pretend), the “uh oh” moment comes when model complexity and/or data volume exceeds the practical limitations of the models built to analyze it – typically spreadsheet models.
As one of our customers put it, “Basic systems can only answer simple questions.” When you’re looking for savings, you need the ability, in real time, to:
- Add additional data easily and quickly
- Create new dimensions and re-derive old ones
- Transform or modify data with understandable rules
- Model at database scale with spreadsheet flexibility
To catch fish, you need Spendata.