General presentation

Objective of the application

BW Demand Planning Scheduling (DPS)

Tool Leader: Antoine Roy

IT leader of the application: Guillaume Thevenet 

Name of project: Dynasys

PMO Project: 6046 S&OP with Dynasys - Core model and pilot

UR: UR335513 DYNASYS Sales Forecast / UR333365 DYNASYS Demand Planning

Reporting Coordinator: GONZALEZ-ROUX, Célia <celia.gonzalez-roux@solvay.com>

Usage information

Number of users: tbd

Critical period: none

Geographical perimeter: worldwide

InfoArea:

• Demand Planning Scheduling
  IA_DPS
  

Process chain Display Component:

• 127 - PROJECT - DPS - Demand Planning Scheduling
  COMP_DPS_PC
  

Roles & Access

Roles and access

See also file maintained by Authorization team : BW Catalog of Roles / link: https://drive.google.com/open?id=10GEfKYqrT1eeTO_uHYAheL1GX7L5y_pvH0KQU64qh5I

Authorization objects

List of autorisation objects mandatory for the application.

Dataflow presentation

Overview

Reporting Part DataFlow

Functional and Technical rules on Workbench

General presentation

Most DSO / Cubes are used to extract data to the Dynasys tool. 

Data are extracted from a source outside BW dynasys (it could be another BW application, ECC table, query etc), loaded into DSO and/or Cubes. Then data are transformed through an APD and loaded into flat files which are loaded into the Dynasys tool.

Useful file to understand dataflow from APD to source, and check filters applied all along the dataflow:

File generated by APD specification :

• Files linked to DP process : DP Process specification
• Files linked to DIP-PP process : DI-PP Process specification
• Files linked to Master data process : Master Data Process specification

Conversion in KG unit

Only conversion in two steps are managed in Dynasys (UOM then T006; conversion in Three or more steps are not managed in Dynasys and should be updated by the business)

The different step of the routine are the following:

Creation of internal table

Creation of internal table between T006and T006D (T006D provide the fields MSSIE, base unit of a dimension DIMID)
Creation of internal table for DSO UOMCMAT2, filled with both conversion from UOM to UNIT and UNIT to UOM and joined with T006 to get Dimension DIMID

Conversion in KG (if possible)

Quantity needs to be converted to KG.

The routine check:

1. if Quantity is already in KG, nothing is done
2. if Quantity is equal to 0, Unit is set to KG
3. if Unit is in dimension MASS (the same dimension as the one for KG), then conversion is done in KG using T006
4. if Unit is not in MASS, we check in UOMCMAT2 if we have a direct conversion from UNIT to KG
5. if no direct conversion exists, we check if there is a conversion from Unit to any MASS unit and after conversion we apply another conversion from MASS to KG using T006

If no conversion exists, Unit remains empty and will be manage as an error (see below)

Catch missing conversion

The flow uses Error DTP. When Unit can't be converted, the routine raise an exception "cx_rsrout_skip_val". The error is loaded in the error queue with text from the message class ZDYN_CONV_ERROR

Not all error will be fixed. Most of these error are linked with BU outside of Dynasys perimeter.

Transformation from ZWPPTDYS ECC table

Documentation

Functional specification are available in the following spreadsheet:

In the following tabs:

ECC process

The ZWPPTDYS is loaded on ECC side through the transaction ZWPPTMRPOUT (which load the ZWPPMRPOUT program)

This transaction use the program RMDMRPEXTRACT01 which create a list of material to extract.

Then, for each material, the program RMDMRPEXTRACT01 run through the exit ZCL_PP_IM_MD_SR_LIST_EXTRACT.

This exit reads the database for each material and loads in memory several table among which is table IT_MDPSX.

Then, through specific code, the table ZWPPTDYS is loaded from IT_MDPSX using method IF_EX_MD_SR_LIST_EXTRACT~EXTRACT_SR_LIST_INFOS

BW Process

This table is loaded from WP1 into WBP in the DPDYN02 DSO (Propagation) and then in the DBDYN14 (Propagation).

This table is loaded from PF1 into WBP in the DPDYN05 DSO (Propagation) and then in the DBDYN14 (Propagation).

All rules are available in following transformations :

ODSO DPDYN02 -> ODSO DBDYN14

ODSO DPDYN05 -> ODSO DBDYN14

See short description of this rules below

Begin of global

1. Declaration of types
2. Declaration of table & variable

Begin of routine

1. Declaration of field-symbols
2. Refresh internal table
3. Clear variable
4. Get Source system using class oref_dynasys_utils, method get_source_systems
5. Get Systems extensions using class oref_dynasys_utils, method get_system_extension

Phase 0: Table preparation

Load each required table into hashed table.

Phase 1: DSO Enrichment

Depending on value of field c_delkz:

1. Step 1: Ship/Rec Shipto ID (PARTNER_CUST_ID)
2. Step 2: Ship/Rec plant ID (PARTNER_PLANT_ID)
3. Step 3: Complement ID (COMPL_ID)
4. Step 4a: BU Source
5. Step 4b: BU Destination
6. DOC_TYPE
7. ITEM_CATEG

Remarks:

• Step 1 to 4 depends on result of previous step. No simplication is done in the code.
• If in Step 1, PARTNER_CUST_ID equals source_field-X, in step we will use PARTNER_CUST_ID and not source_field-X
• This means any error in the code will raise error in further rules. This is a choice in all the routine and this logic must be kept.
• For Step 1,2,3, value are loaded without modification in variable ended by*_origin (and later in the script, these value are saved in dedicated infooject to ease debugging)
• Even useless rules are loaded to keep homogeneity in the code, example: loading a field at null even if this field is already null.
• For step 3, a master have been created C_CUSTASS to apply a specific Complement ID. This case happen only on flow "Plan consignment" :
  

Example :

If the PARNET_PLANT_ID is 8254,  normally COMPL_ID  linked to this plant is 61281 (by reading master data C_PLANT).

But for this flow, we previously read master data C_CUSTASS to see if this plant is a specific case :

And if it's the case, we apply the transcodification => in the example, NDC-66852 replace NDC-61281.

For the moment this master data is maintained manually.

Between phase 1 and 2:

Fields are assigned for expert routine (source to result)

System extension is added to field (Material, Plant, Partner plant ID etc) and remove unneeded 0 using function conversion_exit_alpha_output. These new value are stored in dedicated infooject to e used later in Phase 2 of script

A check is done to find any correspondance with content of specific table ZMATPLANT and ZMATVENDOR (see below) :

Two specific table have been created in SM30 to manage following rules

Two customised transaction have been created in SE93:

• ZMAINT_MATPLANT (maintain table ZMATPLANT)
• ZMAINT_MATVENDOR (maintain table ZMATVENDOR)

Phase 2: Flag determination

Once Phase 1 is finished, depending on DELKZ value, several fields are loaded:

• File flag fields: these flag will be used in APD to dettermine which data needs to be loaded in flat file
• Flowid identify which rule has been applied regarding the specification
• SKU R in FS (C_FLGSRF), filled from PARTNER_CUST_ID / PARTNER_PLANT_ID / COMPL_ID depending on flow id value.

All rules are checked one after the other. Once one check is validated, the check stop

Remarks: c_pfctr2 = BU SOURCE and c_recptbu = BU DESTINATION

Phase 3: DBDYN14 data loading with process chain

DSO DBDYN14 is loaded by 4 DSO "Source" :

DSO DPDYN02 & DPDYN05 are loaded in full mode due to modification done on table ZWPPTDYS in ECC systems (adding and deletion lines everyday).

DSO DSO_DLV1 & DPSD001 are loaded in delta mode.

To not to have to delete data everyday DSO DBDYN14 and reload it from the four sources, we'll use a flag on object C_FLGZWDY (Flag ZWPPTDYS data) wich is equal to 'X' for all data coming from DSO DPDYN02 & DPDYN05.

Program ZBW_DYN001 with variant ZBW_DBDYN14 will be launched everyday (in process chain) before DBDYN14 loading in order to delete data with flag ZWPPTDYS data = 'X'.

After that DTP full will run for DPDYN02 & DPDYN05 and DTP delta for DPSD001 & DSO_DLV1.

Specific rules on Dynasys dataflow

Nota fiscal 

Plant 7525 is not recognized on Dynasys side, deletion or modification are done on BW side following rules like below :

These rules are generally applied in transformation (by routine) before DSO source of each file.

Interco rule 

For DIP files below, delete line where "Receiving site" = "source site" :

Field with extension C_DYN_***

In Dynasys file exported, fields are generaly with an extension S or R corresponding of source system :

• *R => Rhodia (WP1)
• *S=> Solvay (PF1)

These fields are calculated and created in transformation before to be added in their DSO target.

It's also possible in specific dataflow these fields are directly calculated in APD.

Main fields with extension :

• C_DYN_001 : Plant with System Extension
• C_DYN_003 : Customer with System Extension
• C_DYN_006 : Distribution Channel with System Extension
• C_DYN_005 : Material with System Extension
• C_DYN_008 : Division with System Extension

Use of the Global Filter (C_GLBFILT)

Stream: DPS

Main filter : C_PFCTR2 (BU) and  PLANT

These 2 filter may be extended regularly due to the extension of Dynasys perimeter (RCS in january 2016, Soda in june 2016)

Other filters: C_MGN_ACC, DEL_TYPE, ITEM_CATEG, MATL_TYPE, REASON_REJ, REC_TYPE

Be careful, it's forbidden to have the same filter value twice for a global filter field (Active = Y only one time).

Reload data :

• If a new line is added in Global filter, all delta data flow have to be reinitialized

See below all DSO wich are to be reload :

1. DBDYN06
2. DPDYN21
3. DPDYN22
4. DBDYN14

Refer to Paragraph Recurring Procedures for detailed process

• Concerning BU, in APD there are BU (Field C_PFCTR2) filters values "hardcoded".Some modifications have to be done to update these filters if necessary.

      => See Dynasys dataflow document (BW Layer = APD / field =*C_PFCTR2) :

https://docs.google.com/spreadsheets/d/1Hs8ZtMGL_Fq7ed-IxlfmVaqdjOPXKXX_dyuf6xHDqWU/edit#gid=49959650

Usage of error stack

Error stack are used in relation with Unit and Currency conversion.

Most if not all error are linked with BU outside current perimeter of project and won't be fixed.

Each DTP has the following settings:

DTP

Error DTP

Package Size is lower than Maximum Number of Errors per Pack as the number of error won't decrease (example in may 2016: 2567 records from DP_PUSL2 to DPDYN22, 17618 records from DPSD001 to DPDYN21  and 530 from DSO_DLV1 to DPDYN21)

Connection with UD CONNECT

Dynasys is a MSSQL tool, BW WBP is on an AIX server. No ODBC driver exists between BW and MSSQL server in Linux / AIX.

This problem was solved setting an UD connect connection

Main servers:

• Dynasys Production server: WDCCLSLBV01\SLV1_DBPRD  => connected to WBP
• Dynasys Development server: WDCCLSLBV01\SLV1_DBDEV => connected to WBD
• Dynasys Testing server: WDCCLSLVBV01\SLV1_DBTST => connected to WBQ
  

Important:

• Adagio Team is in charge of UD Connect connexions. Admin Change to be created by Guillaume T for any change or issue.
• Wintel Operations is in charge of DynaSys servers (Contact Abdelmôti KHABABA)

Management of file through APD

Files are created in the following folder in AL11 :

• /exploit/depart/Dynasys

An archive is created everyday with generated file in following folder :

• /exploit/BW/archive/DYNASYS

To avoid problem with quotes, routines have been added in APD to encapsulate existing quotes in description fields

Negativ sign have to be at the end of number in file, this is treated directly by routine in APD :

Interface between BW and Dynasys

Documentation about Dynasys process and Interface here.

Data loadings

Loading frequency

Daily: launched by the RSP_DAILY process chain around 6:00 AM each day.

• PC_DPS_DYNASYS_01
  DPS Dynasys: META - D - Daily Meta-Chain
  

Weekly: launched on saturday, 9:00 AM

• PC_DPS_DYNASYS_03
  DPS Dynasys: META - W - Weekly Meta-Chain
  
• Remark: Dynasays tool is awaiting data between 6:00 AM and 8:00 AM

Yearly: launched the 15th of January at 9:00 AM

• DPS Dynasys: META - Y - Cleaning Historical Data
  PC_DPS_DYNASYS_04
  

Flag end of process chain: For both daily and weekly process chain, at the end of APD execution, a command is sent to the Dynasys tool. 

Script sftp_put_dynasys_bw.ksh is executed to send files from BW repository to Dynasys server

This one is saved in repository /exploit/TA/programs/script/sftp_put_dynasys_bw.ksh.

List of file copied on the FTP is stored in  /exploit/TA/programs/env

For each new file, these lists need to be udpated refer to Paragraph Recurring Procedures for detailed process

After execution, files are archived in following repository : /Exploit/BW/archive/DYNASYS and keeped during one rolling month.

Average performance

Most DSO & Cubes are loaded in full mode due to the low amount of data and/or calculation rules.

Exception:

• DPDYN21: 6,4 M lines loaded in delta mode (init)
• DPDYN22: 2,2 M lines loaded in delta mode (init)
• DBDYN06 : 150 000 lines loaded in delta mode (init)
• DBDYN14 (only from DPSD001 / DSO_DLV1) : 5 M lines loaded in delta mode (init)
  
  

NB : Statistics dating March, 2017.

Average duration (March 2017):

• Daily chain usually run for approximately 1,5 hour 
• Weekly chain usually run for 40 minutes  to 1,5 hour (= Daily Run + Material data flow loading and Material.csv file generation)

Remark : On going evolution will change Daily run to take into account the various dependences of DSO source wich are out of Dynasys perimeter (Deliveries, Sales order, Stock...).

Process will be cut in various process : DI-PP / DP / Master data & Reporting. (See part Planned evolution)

Historization

No data below 3 year before beginning of current year are kept. Ex: in Aprl 2016, only data greater than or equal to 01.01.2013 are loaded in Dynasys.

It involves that the volume of data loaded increase during the year, until arrived at its maximum at the end of december.

Reporting

Queries Documentation or user guide

No dedicated documentation on queries

Reporting Specification here.

Main queries

Main functionnalities

n/a

Broadcast

n/a

Maintenance

Known bugs

ShipTo missing in file ShipTo.csv

In end routine DPDYN22 => DBDYN31 & DBDYN18 => DBDYN31, only ShipTo with a filled country is loaded in target.

To fix the issue, follow the steps bellow :

1. Check if country is correctly filled in C_CUSTOMR master data for missing ShipTo.
2. If not filled, reload this ShipTo from ECC source system.
3. Check if country is filled
4. Delete data in DSO DBDYN31 and reload it from DPDYN22 and DBDYN18.
5. Generate file ShipTo.csv, the ShipTo would be appear.

Customer Sales empty or inexact (C_CUSTSAL or 0CUST_SALES)

Due to loading routine in the propagation level of DPSDSO05 / DSO_SD02 which are loaded in delta mode, this field is not correctly updated

Customer Sales attributes empty or inexact (0CUST_SALES & C_CUSTSAL)

Correction done for 0CUST_SALES (see below), the same dataflow must be done for C_CUSTSAL Master data.

Modification on ZZ fields are not taken by the delta. Only full loading can fix this problem.

A specific extractor has been created based on KNVP table from WP1_400 in order to feed ZZ fields correctly.

This data loading is done everyday with the process chain ZPC_MD_ATTR_RCS.

After that we have to retreat data stored in DSO DPDYN22 to update ZI PArtner only if we have a modification. To do that a new transformation has been created :

A new DTP has been created too, launching it ZI partner is reading into master data 0CUST_SALES and updated in DPDYN22.

This DTP is in daily and weekly Dynasys process chain :

• PC_DPS_DYNASYS_08

• PC_DPS_DYNASYS_12

Update of UD Connect datasource

Data loaded from Dynasys tool are using "UD connect" connection type (Java).

Datasource updated in Udconnect can encounter problem in WBQ / WBP due to Metadata cache (whereas everything is ok in WBD)

S:RSSDK:300 Field GHFF_QUANTITY is not a member of /BIC/CADTS_ EXP000010000001

Message no. RSSDK300

Ask for a « cache clearance for quality / production system (BW) on this JCO connection” 

This issue can also appear in WBD on an existing datasource, when a new field is added in the Dynasys table.

Ask for cache clearance the cache in all systems, otherwise you will get the issus when moving the datasource in WBQ/WBP

For production issue : cache to clear on BPJ

For quality issue : cache to clear on BQJ

For development issue : cache to clear on BDJ

New procedure :

Send an emai to CGI (SBS-IS-Appli-Techno-Report-BW.CGI@solvay.com), they will then create a ticket for IBM

Repository /EXPLOIT in AL11 is full

If you have the following DUMP when you execute an APD :

It means the repository where the file generated will be stored is full (/EXPLOIT).

In September 2016, ADAGIO Team added 50GB on WBP server : wbpsapr3.ibm.be.solvay.com 

If the issue appears again please contact ADAGIO Team.

Maintenance

Recurring procedure

Ask update on lists of flat files to be transfered to DynaSys server

The lists are stored in AL11 directory /exploit/TA/programs/env

• BW_DYN_COMPLETE.filelist
• BW_MASTER_COMPLETE.filelist

If a flat file namee need to added or removed, send a email to IS-Adagio-SAP-Operations@solvay.com

Example of message :

Current lists (2018/02/23)

Adding a plant to DynaSys scope

CAUTION : all delta loading are impacted 

Step1) Check the daily DynaSys chain (PC_DPS_DYNASYS_01) is finished via ST13

Step2) Add a new line in Global Filter (C_GLBFILT) for Stream DPS / Rule PLANT

Please indicates in the description the date and user code to track changes

Step3) Delete contain of DSOs using loaded via delta process

• • DPDYN21 / DYN - Sales Deliveries
  • DPDYN22 / DYN - Sales Orders   (removed because delta process without plant restriction)
  • DBDYN14 / DYN - MRP Element

Step4) Launch the Process Chain PC_DPS_DYNASYS_12 (around 3-4h).

Adding an ERP BU  to DynaSys scope

Same procedure thant 'Adding a plant to DynaSys scope' but with Rule C_PFCTR2

Please indicates in the description the date and user code to track changes

Adding a BFC BU  to DynaSys scope

Step1) Check the daily DynaSys chain (PC_DPS_DYNASYS_01) is finished via ST13

Step2) Add a new line in Global Filter (C_GLBFILT) for Stream DPS / Rule CPFCTR2_2

Please indicates in the description the date and user code to track changes

Step3) No more action requiered as BFC BU filter is only used in FULL loadings 

Adding new line in C_GLBFILT

• Modification of C_GLBFILT to add a new plant for example.

Make sure the new line created is not a duplicate of an existing entry.

• Check that the loading of DPS dataflow (PC_DPS_DYNASYS_01) in the night jobs ended successfully.

• Delete the content of the following DSO:
  • DBDYN06 (useless now as it's done in the PC)
  • DPDYN21
  • DPDYN22
  • DBDYN14
• Launch the PC PC_DPS_DYNASYS_12 (around 3-4h).

Exchange Rates : adding extra currency to existing list

Exchange rates are send daily to DynaSys via flat file DYS_BW_CURR.csv.

There are limited to Exchange Rate types ZRH2 and CAR4, and several currencies (ZAR, VEF, USD, TWD...). For all, conversion to EUR only.

It is possible to add currency to the current list (From currency / FCURR).

Steps :

1) in WBD : modify InfoPackage "IP: DTS_TCURR (BW) - Full with Filters for DynaSys " / ZPAK_53G6IDJIO274KOY0IAR54C016

2) transport till WBP.

 Caution : It is not possible to change Exchange Rate Type (KURST) or To-currency (TCURR) without impact on the loading flow, and file generation.

Planned Evolution

Several minor evolution managed by the Dynasis project team.

Process chain planning : Take into account various dependences of DSO Source wich are out of DYnasys perimeter (Deliveries, sales order, Stock...)

See the  New Dynasys dataflow to implement here.

Field C_PFCTR2 in master data C_MATPNT2

Originaly, there was a filter on field C_PFCTR2 in DTP linked to transformation between master data C_MATPNT2 and DSO DBDYN08 :

Filter value applied was OLAP variable V_0G_CWWE03_0001.

But the issue is that field C_PFCTR2 is not filled for source system PF1.

When we applied the filter, only data from source system WP1 was loaded in DSO DBDYN08.

To fix the issue, filter value was deleted and a start routine was added in transformation.

In this one, we read the master data C_MATPNT3 to keep C_PFCTR2 value with key Plant & Material.

If an evolution is done to fill C_PFCTR2 value for PF1 System in C_MATPNT2 master data, start routine could be deleted and filter value in DTP will be applied again.