Introducing Intermediate Data Retention (IDR) for SAP Central Finance: What is it and why should it be used?
Authors:
Abhisek Talukder , SAP Financials Senior Consultant and Team Lead, IBM
Philip Croom, Principal Consultant, TruQua, an IBM Company
Matt Montes, SAP Financials Managing Consultant, TruQua, an IBM Company
Introducing Intermediate Data Retention (IDR) for SAP Central Finance: What is it and why should it be used?
Intermediate Date Retention (IDR) is a new feature introduced by SAP for SAP Central Finance to provide the benefits of real time replication during the document load phase of SAP Financial Accounting (FI) documents. IDR addresses a lot of the past challenges of loading history into SAP Central Finance. A key benefit will be that future implementations can be more aggressive in scoping the amount of history to load. A common financial requirement is to load two years of history to have current and prior year comparisons in detail. However, many implementations struggled with the technical challenges it presented or descoped much of it, if not entirely.
Prior to the introduction of IDR, SAP provided the “Traditional Approach” for the FI document load. Under the Traditional Approach the starting balances, including open items, are initially loaded and then transactional history is extracted as part of the document load process. As the SAP Central Finance system wasn’t in place when the transactional history was first posted, the historical postings are reconstructed from various source system accounting tables and transferred to the SAP Central Finance system. The reconstruction process does its best to emulate what replication would have captured (had it been in place at the onset) but there is some data loss (after the fact) that potentially leads to complications.
With the IDR approach, the initial load of the starting balances and the open items is still the same; however, the document load of the subsequent transactional history changes as IDR captures and stores those postings in real-time in its full original form. By using the same replication mechanism as ongoing postings (i.e., SAP Landscape Transformation or “SLT”), the IDR approach is able to capture postings in flight without data loss.IDR eliminates the potential challenges inherent within the reconstruction process of the Traditional Approach by replacing the step entirely. By using the same mechanics as ongoing real-time replication, the IDR document load achieves greater data consistency and completeness for transaction history.
Architecture Overview: What Systems are Needed?
The IDR system needs to be a separate SAP ERP system. It can be either SAP ERP (same or higher version compared to the source) or an SAP S/4HANA system. It is our recommendation to have IDR as a separate client in the SAP Central Finance system.
Initiating IDR: – How is IDR Activated in SAP Central Finance?
The activation of IDR is done from the SAP Central Finance system and needs the baseline SAP Central Finance functions in place. It can be activated before the system is fully configured. The relevant program and overall IDR cockpit can be accessed in the SAP Central Finance system via program RFINS_CFIN_MANAGE_REPLICATION. Once the step to begin recording to the IDR system is activated, all the documents posted in the source gets replicated real time in the IDR system. Any changes made to the accounting documents are also captured in the IDR tables.
Relevant tables in IDR- CFIN_ACCHD_IDR and CFIN_ACC*
Once the “Block Previous Posting Periods” step is activated it will prevent any documents from posting in the source system for the periods prior to the one for which the IDR is activated. This step is in line with the approach that IDR is real time replication, and it becomes the source system for document load during the initial load step at later stage.
IDR vs Traditional Approach Background:
When conducting an SAP Central Finance implementation a load of FI data must be executed to bring the transactional data from the source system(s) into the SAP Central Finance system.
Traditionally, the FI load is a Remote Function Call (RFC) based approach in which the relevant data is extracted from the source system(s) and brought into the SAP Central Finance system for mapping, simulations, and eventually the posting of the data. The Traditional Approach has the benefit of being refined throughout the latest versions (1511 – 2020) of SAP S/4HANA while simultaneously providing an iterative framework to ensure that all errors can be analyzed and resolved.
Despite the refinement over the years the Traditional Approach still has some limitations with how the FI Document Load as part of the overall load process is reconstructed. As a result, SAP now provides the IDR approach that is available as of SAP S/4HANA 2020 Feature Pack 1. The IDR architecture leverages another SAP system to record a copy of the raw data until SAP Central Finance is ready to receive the document load.
Each approach, either Traditional or IDR, has pros and cons; ultimately both will need to be evaluated on a project-to-project basis. In the following section, both approaches will be discussed in detail along with key considerations and use cases to help discern between the Traditional Approach and IDR.
IDR vs Traditional Approach – Details and Considerations
Details:
Now that there are two FI Document Load approaches available it is important to understand how each approach works and the use cases associated with them. The Traditional Approach is based off a Remote Function Call (RFC) connection. This connection uses manually pre-determined settings to dictate the period of time in which data should be extracted from the source system(s). The IDR approach uses the same manually pre-determined time settings as the Traditional Approach with the exception that data is recorded in an IDR system instead of extracted to the SAP Central Finance system. This is an important nuance because it is one of the key differences between these approaches.
When the Document Load data is extracted for the Traditional Approach, it is reconstructed from the various source system accounting tables which means that the data will not contain all the transactional detail (clearings, resets, reversals, FI-CO reconciliation) available at the time of posting. In contrast, all the posting detail that is available for those documents gets transferred into SAP Central Finance via online replication. This occurs because the raw data in memory is being directly captured upon posting. Unlike the extraction that occurs as part of the Traditional Approach, activating Intermediate Data Retention (IDR) enables an immediate recording of the inflight FI data in the IDR system from the date of activation on wards. The enabling of recording in the IDR system mimics the exact behavior of online replication, which means all posting detail will be available for loading into the SAP Central Finance system.
Understanding the level of detail available in the Traditional Approach compared to IDR is a critical distinction and one of several items that need to be considered when deciding upon an approach for the FI Document Load.
Transactional Considerations:
As outlined in the previous section, a big distinction between the Traditional Approach and IDR is the completeness of data associated with the transactional data being loaded into the SAP Central Finance system. However, just because more detail is available that doesn’t necessarily mean one approach should be chosen over the other. There is a myriad of factors that need to be considered prior to deciding on a Document Load approach.
- How much transactional history is required to be loaded into SAP Central Finance?
- Is there a high number of clearings, resets, and reversals?
- Is there a level of complexity to the data that may require detailed testing?
- Is there a high number of FI Documents generated from integrated logistical processes, such as Materials Management and Sales & Distribution?
Things to Consider when choosing between IDR and the Traditional Approach
As outlined in the previous section, there are some transactional considerations when deciding between the Traditional Approach and IDR. Additionally, there are some technical constraints and some functional considerations to keep in mind.
Technical Considerations
Limitations of the Current Versions per SAP:
- Support only for cross-company postings if all companies are in the same Initial Load Group
- This means that if you are only loading some Company Codes from an instance then you must ensure you are loading all Company Codes that have cross-company activity and that you have grouped them all in the same Initial Load Group
- Transactions which result in more than one posting where at least one of those postings is before the start of the recording are not supported.
Beyond these hard limitations, the activation of IDR Recording in the Source System requires that a partially configured SAP Central Finance System is connected to that Source, so you must plan your system build schedule early enough in the project to record all transactions from your Migration Date forward.
Functional Considerations
Source System Migration Date Changeability
Under the current IDR behavior, once you have activated IDR Recording you cannot change the VCFIN_SOURCE_SET Migration Date (Start Year – Balances, Start Year – Transactions, and Start Period – Transactions) in the Source System. This means that you must have finalized your scope and migration date very early in the project and cannot use an earlier date for your test cycles where Production is your source system.
For example, if you are planning to go live with a Migration Date of January 1st, 2022 but your final test cycle begins in November, 2021 you will not be able to use a PRODUCTIVE source system and a SAP Central Finance test system as recommended by SAP in the Central Finance help documentation .
Changes in Configuration vis-a-vis Transaction History
As part of either approach, when you perform the SAP Central Finance Document Load, the system posts everything under the current Configuration, Mapping, and Master Data settings in the SAP Central Finance system. The longer the length of transaction history in the FI Document Load, the greater chance that there have been Configuration or Master Data changes in the Source System at some point during that history. As a result, there becomes a split in which “after-change” documents that matches current configuration posts, and “before-change” documents that doesn’t post as it is tied to obsolete configuration.
Under the Traditional Approach, there is no ability to change individual documents. Under the IDR approach, there is a way to correct documents, albeit one posting at a time, using “Emergency Correction” functionality available in the Application Interface Framework or “AIF” monitor.
To address the issue under the Traditional Initial Load Approach, temporary configuration changes sometimes had to be made during cutover. With the new IDR Approach, we have the additional option to resolve the conflict via Emergency Correction in AIF.
User Experience in Handling Error Volume
In the Traditional Initial Load approach, there is specialized functionality to isolate:
- Unique generic error messages and their counts (for example, how many Missing Mapping Errors are there), and
- Unique error values (for example, the list of missing G/L Account Mappings)
These tools work well even with very the higher error volume generally experienced in the first Sandbox or Development loads.
In the IDR Approach, all historical transactions are posted through AIF which means you only have the AIF Error Handing functionality available. The AIF Message Summary provides the unique generic error messages and their counts, but the only way to get the actual unique error values is by drilling into the AIF Error Details which is much slower when there are more than a few thousand documents in error.
Conclusion
In summary, there are two possible approaches to conducting an FI Initial Load for SAP Central Finance – the Traditional Approach and the Intermediate Data Retention (IDR) Approach. Each approach has its benefits and will need to be evaluated on a project-to-project basis to determine which approach makes the most sense for the project in question. There are both functional and technical considerations that need to be carefully considered as there will be an impact on the timing, quality, and type of data that is loaded.
About the Authors
Abhisek Talukder is an experienced SAP FICO consultant currently working as an SAP Financials Senior Consultant and Team Lead at IBM. His has 8 years of financial consulting experience which includes 2 full lifecycle SAP S/4HANA implementations and 1 small scale SAP Central Finance implementation project. He is certified in both SAP S/4HANA Professional and SAP Central Finance. Besides that, he has also led teams providing SAP Finance support to Fortune 500 organizations. Abhisek Talukder has conducted two internal trainings on SAP Central Finance. His areas of interest include SAP Central Finance, SAP Group Reporting, Asset Accounting and Product Costing.
Philip Croom is an SAP S/4HANA Central Finance Solution Architect for TruQua, an IBM Company, focused on Finance Transformations. Philip has experience with SAP Central Finance on SAP S/4HANA 1709, 1809, 1909, and 2020 and has implemented SAP Central Finance for numerous SAP ERP and SAP S/4HANA Source Systems.
Matt Montes is a Finance and Data-driven professional currently working as an SAP Financials Managing Consultant for TruQua, an IBM Company. His professional experience is comprised of Financial Transformation engagements which includes accounting, financial analysis, budgeting, consolidation, and data analytics. Matt Montes has 6.5 years of financial consulting experience utilizing SAP Central Finance, SAP S/4HANA, SAP ERP, SAP Fiori, SAP Analytics Cloud, SAP Group Reporting, and SAP Business Planning and Consolidation. In addition to 4 full lifecycle implementations, product development, and 6 POC’s, Matt Montes also works as a trainer, content developer, and international speaker.