Now imagine you are the Risk Officer at a $30 Billion/year Enterprise that services some of the most sensitive transactional data in the world. This could be Social Security numbers, medical records/lab results, credit card numbers, account balances. Changes to this data are under constant scrutiny by regulatory bodies in each industry sector. Many organizations devote significant financial and technical resources on risk management. For example, internal governance rules may require housing 20+ years of historical records in case of a law suit. Audits related to government regulations (HIPAA, SEC Rule 17a-4) may not only require maintenance of historical data, but also a view of all data changes. In order to do this, organizations may:
- Transform all transactional ‘Update’ operations into ‘Insert’ and ‘Delete’ pairs to retain before-and-after images of records.
- Employ procedural code (e.g. triggers) to keep track of changes.
- Create copies of the historical data on external systems which may increase the liability of data breach and lead to additional costs related to data copying, transformation, storage, and maintenance.
- Keep an entire history of changes to the data without manually changing the transactions themselves (i.e. without requiring code to transform updates into insert/delete pairs)
- Automatically maintain beginning and end timestamps for each row of data where the timestamps indicate the “life” of the data (i.e. without requiring procedural code)
- Access and analyze this data via the transactional systems (without impacting resources on these transactional systems)
- Create a snapshot of the data as it existed at any point in time or range(s) of time with massive parallelism (without creating separate data connections and credentials
