SAP XI Tutorials | SAP XI Training | SAP PI Tutorials|SAP XI Step by Step Senarioas

HTTP Client for SAP Process Integration

noreply@blogger.com (SAP XI Tutorials) — Thu, 26 Jan 2012 00:59:51 PST

Project Description
SAP Process Integration (PI) is SAP's SOA Middleware and Integration Platform.
This tool makes it easier for SAP PI Consultants to post messages to SAP PI.
Features like Saving and Loading a project are provided.
It's developed in .net.
Screenshot

Download

SAP PI ABAP Transaction codes

noreply@blogger.com (SAP XI Tutorials) — Thu, 26 Jan 2012 00:45:09 PST

SAP PI is one of the core components of NetWeaver.It stands for Process Integration. Here is a list of important transaction codes used in SAP PI ABAP.

Administration Transaction codes

SXMB_IFR: Start Enterprise Service Integration Builder
SXMB_ADM: Integration Engine Administration
SXI_SUPPORT: Test all Repository and Directory Objects
SMQR: qRFC monitoring to display the status of QIN Scheduler and Registration.
SMQ1: qRFC monitoring the Outbound Queue.
SMQ2: qRFC monitoring the Inbound Queue.
SLDAPICUST: Customizing API for System Landscape Directory.
AL11: SAP Directories Display to check the profile parameter DIR_TRANS.
SU01: User Address Maintenance
SICF: HTTP Server Configuration
RZ20: CCMS Monitoring
AL08: Logged in user list screen.
SM21: System Error Logs
ST22: ABAP Runtime Errors
SLDCHECK: Check the connection to the System Landscape Directory (SLD).
SM58: tRFC error Logs
SMICM: ICM Monitor
STRUST: Trust Manager
RZ70: SLD Administration

Web Services Transaction codes

SUDDIREG: Maintaining UDDI Registry
WSADMIN: Administration tasks for Web services
WSCONFIG: Configuration of Web Services in the Web Service Framework
SOAMANAGER: Start SOA Manager
WSPUBLISH: Publish Web Services

IDOC transaction codes in PI

IDX1: Assign Port (RFC destination) in IDOC Adapter
IDX2: IDOC Metadata administration
IDX5: IDoc tracking like display IDOC XML Messages
WE02, WE05: IDOC Display and IDOC list respectively

Business Process Management (BPM) Tcodes

SWF_XI_ADM_BPE: Start and Stop BPE
SXMB_MONI_BPE: Monitoring tool for Business Process Engine (BPE)
SWF_XI_PBUILDER: Detailed BPM Process Builder
SWF_XI_CUSTOMIZING: Check Integration Process prerequisite
SWF_XI_ADM_BPE_DISP: Display the status of BPE
SWELS: Switch Event Trace On/Off

Monitoring transactions in PI

SMGW: Gateway Monitor
SXMB_MONI: Monitoring Integration Engine
SXI_MONITOR: Monitor for Processed XML Messages
SXI_CACHE: PI Runtime Cache

Migration to SAP PI

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 10:56:51 PST

Disclaimer: These are all my own views only, and do not represent that of my employer's or anybody else's

I am fortunate enough to get to the vicinity of a couple of migration projects, where we had been replacing some other middleware(eg. eGate , Mercator) with SAP PI. During the process, I have learned that migration requirements pose a different set of challenges than implementation requirements. During implementations, our main focus remains on getting clarity on the requirement, which generally changes, and gets refined in some form or the other, till the point of go live and beyond, and then use the middleware tool's features to best design the interfaces. Thus, best design is driven by the tool's abilities and out of box features provided. Over that custom codes are written to supplement the gaps and fulfill the requirement.

However, when we are talking about migration, the requirement is already known and functional with a different middleware tool. The present design is based on the present available tool's capabilities and best practices. These capabilities and best practices will differ from tool to tool , so, what is best designed on one tool might not be the best design for PI, thus posing frequent challenges to the migrating tool's capability. Thus an as is migration might not be the best possible way to choose, and in most cases might not be possible altogether. These technical challenges will lead to greater complexities, and might lead to a very clumsy situation unless dealt with properly from the very beginning of the project.

Thus approaching a migration project turns out to be a vital parameter for the success of the project. Below are some of the factors which should be taken into account from the Blue Printing phase.:

1. It is important that a comparative study between the tools is handy.

2. Study the present requirement and try to classify them (based on the visible PI design). examples might be "Control M" - Signifying interfaces which works along with the scheduler Control M; another example may be "External Web services" , denoting interfaces interacting with external Web services. Below are the few patterns /"pointers to define patterns" that I can think about now:

Synchronous / Asynchronous
Interfaces working along with other scheduling tools.
EDI Interfaces
Stateful Requirements / potential ccBPM requirements
Special Scheduling requirement
Dependency requirements
High Volume interfaces
High Frequency
Real time requirement
Protocol support required which are not provided by PI standard adapters
Special security requirements, eg: involves digital certificate / sFTP requirements etc.
Publish-Subscribe pattern
Business Critical Interfaces
Any requirement for High availability for specific set of interfaces
Any other patterns specific to the requirement
Third Party systems -- in most cases 3rd party systems pose unique(specific to the 3rd party) interface challenges

Next, carry out a high level design and feasibility study of each pattern. Incase adequate time is provided , a deeper look at the individual interfaces is worth. Try to narrow down the classifications into PI design patterns.

In the process, certain limitations of the tool is expected to crop up. There are chances that these limitation will have a certain common solution and should be studied carefully for something which is generic / at least can apply to a group of interfaces as generic-parameterizable solution. This solution might range from --a design pattern, writing custom Java or ABAP Maps/ proxies / Adapter Modules / operating system scripts , creating a map convertor etc. A cost analysis needs to be carried out to gauge the advantages / disadvantages of building the generic solutions - it is more likely that certain generic solutions will pass and will be applicable for a significant mass of the interfaces.

At this point of time, we should have some generic solutions available and some specific problems to be addressed which looks to be much simpler and less clumsy to be dealt with than where we started from. A lengthy Blue print/ requirement analysis / design phase is worth invested!!

PI 7.3 - Adapter User-Defined Message Search

noreply@blogger.com (SAP XI Tutorials) — Sun, 25 Dec 2011 10:35:49 PST

As all of us know one of the interesting feature enabled with PI 7.1 EHP1 dealt with User-Defined Message Search. But since (as mentioned in Note 1247043) PI monitors in NetWeaver Administrator (NWAPI) is not supported for production scenarios most of us could not use this feature any further. With PI 7.1 EHP1 Support Package 6 the user defined search could only be configured in & used for the integration server ABAP client. As all of must be aware of by now the PI 7.3 comes with a JAVA only installation option (AEX). So does this mean we cannot use user defined message serach with AEX. Well the good news is now the user defined message search can be configured and monitored from an AEX installation. The user defined message search has been integrated with the new configuration and monitoring tool (pimon).

In this blog I will show how to configure and monitor user defined message search for an AEX installation (PI Adapter User-Defined Message Search). This holds true for a dual stack installation as well.

Configuration:

Navigate to PI Adapter User-Defined Message Search either via pimon->Configuration and Administration from the directory start page or via NWA -> SOA ->Monitoring -> PI Adapter User-Defined Message Search Configuration

Now click on the new button to create a filter and Provide the values for required fields along with sender and receiver components as shown below. All the values will be available via drop down (search help).

Now we need to create the search criteria for this filter (in the details section). This means we need to enter the fields we want to monitor on a given payload. You can monitor more than one field of a payload at once, only thing you need to create separate search criteria for them. You have an option of either monitoring a payload value (XPATH expression) or a field from the SOAP dynamic header. Please note the XPATH expression must yield a single value.

I am using the following test payload for this blog

Now we need to provide the namespace for the prefix value we have used in our XPATH expression. The only restriction is the prefix value cannot be longer than 8 characters.

Now if needed we can test our filter configuration by using the Test Search Criteria button.

Monitoring:

Now we can process a message which satisfies the filter criteria and navigate to the Message Monitoring section to check for the payload and the filter. In the Message Monitoring->Database we can use the user defined search criteria under Advanced option to provide our filter and corresponding values. Here you can provide more than one search criteria for a payload search which you have defined in the configuration step earlier.

You should see all the message that satisfy the filter condition.

Note: Now you can see a new tab has been added to the messages that satisfy a given filter with the set attribute and their corresponding values as shown below.

The user-defined search is one of the many ways to find messages in the Advanced Adapter Engine using the message monitor without the use of TREX. Optionally the user-defined search is also available for searching messages in the Integration Engine. You can configure it using transaction SXMS_LMS_CONF in a dual stack installation as well.

Top 5 Things to Learn at SAP TechEd 2011 about SAP NetWeaver Process Integration (PI)

noreply@blogger.com (SAP XI Tutorials) — Sun, 25 Dec 2011 10:43:58 PST

In May 2011, SAP NetWeaver PI 7.3 was made generally available after a successful ramp-up earlier this year. SAP TechEd 2011 is a good time to learn about ramp-up experience, new and enhanced features, and our plans going forward – to shape your plans and gain firsthand experience. We have compiled a comprehensive program covering SAP’s integration technology with 12 hours of hands-on sessions and more than 10 hours of lectures (all being offered at least twice in Las Vegas). You can find the full agenda and schedule on the SAP TechEd websites for Las Vegas, Madrid, and Bangalore.

It’s hard to pick the best sessions as this depends on your role and experience with SAP NetWeaver PI (unless you decide to simply attend all). Still, here’s a list of the top 5 things to learn at SAP TechEd 2011 with respective sessions:

1. Gain firsthand experience with the latest release (our hands-on sessions – you’ll get your own Java-only PI instance running in the TechEd cloud!)

SAP NetWeaver PI - Integration Flows Deep Dive (PMC164)
SAP Netweaver PI - AEX (Improvements & New Features) (PMC266)
Build system centric processes using the combined BPM/PI offering (PMC165)
Combining SAP NetWeaver BRM and SAP NetWeaver PI (PMC166)
The SAP NetWeaver PI Java API (PMC360)

2. Learn about improvements for operating and maintaining SAP NetWeaver PI

Monitoring Enhancements for SAP NetWeaver PI (PMC108)
Near Zero Downtime Maintenance for SAP NetWeaver PI (ALM220)

3. Prepare your upgrade to SAP NetWeaver PI 7.3

SAP NetWeaver PI 7.3: Overview and Feedback from Ramp-up (PMC203)
Moving productive landscapes to PI 7.3 single-stack (PMC300)

4. Get up to date on our road to an integrated stack (and other enhancements)

SAP NetWeaver PI Roadmap (PMC202)
Deep Dive into what is New in SAP NetWeaver PI (PMC201)
BPMN for System Integration (PMC106)
Running system centric processes on SAP NetWeaver BPM and SAP NetWeaver PI (PMC204)

5. Learn from SAP NetWeaver PI customers and partners (ASUG sessions in Las Vegas; Madrid still in planning phase)

SAP NetWeaver Process Integration 7.1 Performance, Scaling, and Security (TEC115)

Moreover, we’ll be around for pod and networking sessions.

Proxy Configuration for PI 7.3 Java only

noreply@blogger.com (SAP XI Tutorials) — Sun, 25 Dec 2011 10:39:19 PST

The purpose of this document to highlight new changes and steps required to configure proxy communication between SAP PI 7.3 (Java only installation) and backend SAP systems like ECC, CRM etc.

PI7.3 is coming with 2 installation options- Java only installation (AEX) and Dual stack (ABAP + Java) installation. In Java only installation i.e AEX we do not have ABAP stack on PI system hence the configuration changes are required in PI as well as Backend systems to enable proxy communication between systems.

In case of PI7.3 dual stack implementation the proxy configuration will remain same as of PI earlier versions.

Steps to be followed in PI system

1. Go to PI NWA using Http://<Hostname>:port/nwa

2. Go to Configuration->JCo RFC Provider

3. Create 2 RFC JCO providers

SAPSLDAPI_<PI SID>
LCRSAPRFC_<PI SID>

Create SAPSLDAPI_PEX:

Note the Application Server host and Gateway Service.

Here Application system is used to register the program id SAPSLDAPI_PEX.

Reason: In Dual stack we generally give PI ABAP stack details, however for Java only installation we need any ABAP stack details(in landscape) to register the Program ID. I have used for ECC application system that PI is exchanging data.

Perform the similar steps for LCRSAPRFC_PEX

Steps to be followed in Application system

a) Create 3 RFC destinations (SM59)

- SAPSLDAPI (Application system to SLD connection)

- LCRSAPRFC (Application system to Exchange profile connection)

- SAP_PROXY_ESR (Application system to ESR connection)

b) Configure SLDAPICUST and test connections through TA: SLD Check

c) Test SPROXY transaction

d) Configure SXMB_ADM transaction for runtime proxy message exchange

Create RFC destinations

- Create RFC destination SAPSLDAPI through SM59 of type T (Start External Program Using TCP/IP)

The gateway Host and service are blank as in PI NWA (JCo RFC) we have registered the program ID in same application system gateway.

Note: In dual stack generally we give SAP PI ABAP stack details, same details we used while creating JCo RFC Provider.

- Create RFC destination LCRSAPRFC through SM59 of type T (Start External Program Using TCP/IP) similar to RFC destination SAPSLDAPI and use program ID as LCRSAPRFC_<PI SID>

Note : In case we have more application system connected to PI using SPROXY than we need to create above RFC destinations in another application system and will pass Gateway host and Gateway service as provided in JCo RFC provider.

- Test both RFC destinations.

- Create RFC destination SAP_PROXY_ESR through SM59 of type G (HTTP Connection to External Serv)

Target Host: PI host name

Port: Http port (50XX00) where X X- System number

Long Details: with PIAPPL<SID> user authorizations

b) Configure SLDAPICUST and test connections through TA: SLDCheck

The user should have SAP_SLD_CONFIGURATOR role.

Generally we use SLD_CL_<SID> user

c) Test proxy connection with ESR for transaction: SPROXY

Note: Here all the config were working but somehow I am not getting exchange profile values using RFC destination LCRSAPRFC. Hence I had to create RFC destination SAP_PROXY_ESR to make transaction SPROXY work.

d) Configure SXMB_ADM transaction for runtime proxy message exchange

Create RFC destination <any name> of type G (HTTP Connections to External Server)

Target Host: PI host name

Port: Http port (50XX00) where XX- System number

Path Prefix: /XISOAPAdapter/MessageServlet?ximessage=true

Log on Details:

Go to Transaction: SXMB_ADM->Integration Engine configuration

Note: For dual stack we generally give ABAP stack RFC destination, since we want to use ICO object we need to connection using SOAP adapter hence the path prefix used for RFC destination is ( /XISOAPAdapter/MessageServlet?ximessage=true)

Introduction to Process Orchestration

noreply@blogger.com (SAP XI Tutorials) — Sun, 25 Dec 2011 10:36:48 PST

If you analyse common business practices, you will find that they are generally a series of relatively simple activities that interact with each other to accomplish a larger, more complex result. While these smaller processes are the core of day-to-day business activities, many businesses only consider the large and complex aspects of their operations to be "processes". In truth, any chain of business activities that are carried out in order to achieve a specific goal should be considered a business process, regardless of how simple or small. Often costs are reduced, quality is increased and efficiency improved by simplifying and refining even the small processes. Keep in mind that a business process is not restricted to human interaction - it often includes interaction between various software applications as well.

Business Process Management

Business Process Management (BPM) is the way in which you plan, implement, and ultimately improve the processes that run your business. BPM tools such as SAP NetWeaver BPM, help to model and enable the automation of business processes - these tools offer a clear view into processes through graphical representations, in-depth analysis and testing, process tracking, and continual process improvement. The beauty and power behind BPM is the capability to deliver the right work to the right people at the right time - everything can be automated and monitored. Key Process Indicators (KPI's) can be measured and Service Level Agreements (SLA's) can be enforced with automatic escalation rules - all of this independent of the underlying applications. In other words, a good BPM tool can work seamlessly across all other enterprise applications - including on premise, web, cloud and mobile.

Process Integration

While BPM focuses more on the human interaction of business processes, there are application integration tools such as SAP Process Integration (PI) that bring together various enterprise applications through a unified integration approach. These tools focus specifically on the automated communication between software applications. As an example, you could use SAP PI to facilitate the integration between Oracle Business Suite running Financials and SAP ERP running Human Resources. A well thought out integration product should never be technology or application dependent.

Business Rules Management

A huge problem with most enterprise software is that very often the logical conditions for business logic (business rules) are "hard-coded". What this means is that if the rules ever change, software developers are required to update all of the effected software components. Often complex rules are difficult to comprehend and implement - it can take days or even weeks for the business and developers to finally understand and correctly code the rules. Typically the developers understand the rules differently to what the business has tried to communicate and the change process takes a lot longer than it should. A risky but very common practice is that many business rules are only mapped out in the minds of specific business owners and very badly documented, if at all. If a key business owner leaves the company for any reason, the understanding of those rules go with him/her.

A solution to this is what we call Business Rules Management (BRM). Essentially, all business rules are stored in the business rules engine and enterprise software simply taps into these rules when required. The rules are managed centrally so that when they change, none of your software components need to be recoded. Once again it is very important that a BRM tool (such as SAP BRM) is not application dependent and that all of your software applications can tap into the rules engine and leverage its power.

A good example of a business rule is the calculation of a customer's credit limit at banks. It may sound like a simple rule, but there are many aspects taken into consideration - credit ratings, transaction history, age, level of education, current financial commitments, current income, assets and more.

Process Orchestration

The orchestration of business processes can be a complex task - one has to consider human interaction, application integration, rules management, KPI measurement, SLA monitoring, continuous improvement, user experience, business involvement, exception handling, and much more.

The SAP Process Orchestration bundle includes SAP BPM, SAP PI and SAP BRM. These are enabling tools that pass process ownership on to the business and allow IT to support the technology. All of these tools are completely independent and work across any other applications - this bundle is something that even non-SAP customers should be considering.

SAP has built their BPM and BRM tools in a way that the business can design the processes and rules with graphic modelling. Once a BPM model has been designed, it is simply passed on to the developer who quickly ties up application functionality to the process. The processes and rules that the business designs are the processes and rules that are ultimately executed. No more miscommunications or misunderstandings - simple.

Synchronous calls with the PI SOAP Adapter

noreply@blogger.com (SAP XI Tutorials) — Sun, 25 Dec 2011 10:30:40 PST

I have not worked a lot with synchronous calls and modules. So I had to learn a bit about it.

In July I launched the SAP PIArchiving solution, to make it easier to archive messages in the adapter framework. It could be EDIFact or XML messages that you want to find easily and store safely.

One question I was asked was how the system worked with synchronous calls for instance for a web service. I guess that the adapter would figure it out and it was possible to make it work.

Now I finally had time to test how to put the adapter modules on a SOAP adapter, to archive the messages for both ways. It proved really easy. The only change was that the adapter module should be placed before and after the SAP standard module.

The processing sequence should just look like this.

The modules can be configured independent of each other using the module key.

What seems to happen when this occurs is that the two messages are being processed.

When looking at the processing logs it is possible to see the connection. In the adapter communication logs the following information exists. There is a link to the continuing processing of a message as well as continuing from this guid. It is thereby possible to find the corresponding message.

What I like about this is how the framework works, is that the modules do need to know anything that it is a synchronous message. My archiving modules still works, which is just really nice.

Chem XML Message eStandards and CIDX Scenario Part III

noreply@blogger.com (SAP XI Tutorials) — Sun, 25 Dec 2011 10:29:01 PST

In my earlier blogs efforts are made to explain about CIDX standards, how to design and configure the object to support CIDX communication.

Blog1

Blog2

I would like to make your experience pleasant and fruitful with CIDX communication through this blog.

This blog covers those intricate details in regard to security, certificates through simple steps, focusing on PI 7.1

You have already selected CIDX adapter with Transport Protocol as “HTTPS” and Message Protocol as “RNIF 1.1” for communication. Selecting the message protocol to RNIF 1.1 means you are configuring the scenario to handle Preamble, Service Header, Service Content, Digital Signatures etc…

We will focus on achieving HTTPS communication which is a combination of HTTP with SSL/TLS protocol to provide encrypted communication and secure identification of a network web server.

Step 1: Since the CIDX adapter is available through adapter engine, you need to set your server through Java stack to receive and send secured messages.

SSL Communications are handled by ICM (Internet Communication Manager) for both the Java and ABAP servers. You need to perform the configuration to use one of it, navigate to RZ10, select the profile <SYSID>_DVEBMGS00_<host> and configure the profile parameter.

ssl/pse_provider = JAVA

Step 2: Restart the server to notice automatic creation of Keystore views in SAP NetWeaver Administration (NWA).

Navigate to NWA >> Configuration Management >> Certificate and Keys.

Identify the new Keystore View named after ICM_SSL_<instance ID>

Create the private key in the specified keystore view using “Create” and follow the wizard.

Notice that “Generate CSR Request” is enabled and use it generate CSR Request. Basically this step is needed to get your certificate issued by 3^rd party Authority, to be identified as secure partner to carry out secure online transactions and conduct the business over internet.

When you purchase the certificate that is considered as CSR Response. Select the private key that you have just created and import it as “Import CSR Response”.

Copy these certificates into Trusted CAs and secure_ssl keystore Views.

Step 3: Load the public key of your partner with entire certificate chain (Public Key, Intermediate and Root) into keystore Views “ICM_SSL_XXXX”, Trusted CAs.

In the following screenshot, you can view Verisign as Certificate Authority and chain of certificates.

They can be recognized as Verisign as root, Verisign Class 3 Secure Server CA – G3 as intermediate and business.partner.com as public key.

At times your partners provide self signed certificates, however PI supports.

Step 4: Choose how you want to enable your partner log into your server to come up with processing a message request.

a) In some cases, the certificate is issued with CN = <user id>, then provide necessary authorizations to the user.

b) In most cases, the certificate is issued after host name for eg., business.partner.com. In this case to support the certificate log in, you need to perform additional settings.

i. Create a certificate user say PICERTUSER with adequate authorizations (One of it is ‘XI_AF_RECEIVE’).

ii. Navigate to NWA >> Configuration Management >> Security >> Authentication

Go to Login module, ClientCertLoginModule.

Edit to maintain Name as “Rule1.getUserFrom” and Value as “wholeCert”.

iii. Navigate to NWA >> Configuration Management >> Security >> Identity Management.

Display PICERTUSER, Modify to load the partner certificate(Only Public key).

It basically means when message comes from remote server, certificate is authenticated and then accept to login through PICERTUSER.

Internally the message is checked whether valid or not by comparing the certificate Authority in ICM_SSL_XXX and partner through TrustedCAs. If it exists, it passes and then next part of steps is to select the matching user with the certificate.

iv. As an additional optional step, you may want to restrict the processing of scenario to this user through Business component >>Assigned Users in Configuration.

Step 5: Follow my previous blog 2 for configuration.

Wait for another blog that focuses on Troubleshooting CIDX communication.

For additional help use SAP resources

CIDX Adapter http://help.sap.com/saphelp_nwpi711/helpdata/en/90/1d17c56bb01f4382342294b9a4a308/frameset.htm
Maintaining the User’s Certificate Information

http://help.sap.com/saphelp_nwpi711/helpdata/en/a7/1cd08ffe25e34799cbbe1a7ecdb8ed/frameset.htm

3. You may use diagtool provided by SAP, more details are available in note#1045019 – Web Diagtool for collecting traces. This is a very good tool that provides you the visibility on how the message is being processed.

Handling patterns and manipulating hierarchies through XSLT

noreply@blogger.com (SAP XI Tutorials) — Sun, 25 Dec 2011 10:24:27 PST

XSLT can handle complex and tricky requirements specially when copying patterns and manipulating hierarchies. I came across a tricky requirement in my project. It was an IDoc to File scenario where source IDoc xml was to be converted into a third party specific format.

Here we have an invoice Idoc (INVOIC.INVOIC02) which needs to be converted into a specific format. This transformation is required for a correction invoice Idoc, which means that Idoc would have even number of E1EDP01 segments in it. For n number of original E1EDP01 segments, there will be another n number of correction E1EDP01 segments. Source structure of the idoc is as follows:

IDOC

EDI_DC40

E1EDKxx segments

E1EDP01 segments (Even number, n original, n correction)

E1EDSxx segments

Third party requires the Idoc xml but in a bit different format as explained below, Let us introduce some notations:

E1EDKxx_Original - These are E1EDKxx segments which are present in the source idoc.So lets call them the E1EDKxx_Original segments.

E1EDKxx_Changed - This is modified version of source E1EDKxx segments. One of the segment in E1EDKxx segment would be changed.

Trailer - Copy of E1EDSxx, this would be used as trailer record

Target structure required by third party --

IDOC

EDI_DC40

E1EDKxx_Original (As is original E1EDKxx).

E1EDP01 (item number 1, original item)

E1EDSxx (Trailer)

E1EDKxx_Changed

E1EDP01 (item number 1, correction item)

E1EDSxx (Trailer)

E1EDKxx_Original (As is original E1EDKxx)

E1EDP01 (item number 2, original item)

E1EDSxx (Trailer)

E1EDKxx_Changed

E1EDP01 (item number 2, correction item)

E1EDSxx (Trailer)

……………………………..

We have a relation here between original and correction item (E1EDP01).

Original E1EDP01-POSEX = Correction E1EDP01-HIPOS.

Given the above information, we need to convert the source IDoc xml into target structure as explained above. Achieving this transformation via Graphical, Java or ABAP mapping would be quite difficult. So we will be discussing XSLT transformation here.

We would approach the target xml as mentioned below:

Loop on all E1EDP01

If ( Current E1EDP01-POSEX = Any of other E1EDP01-HIPOS)

write E1EDKxx_Changed;

write current E1EDP01;(original item)

write trailer;

write E1EDKxx_Original;

write correction E1EDP01; (Correction Item)

write trailer;

End If.

Endloop

First we have to copy the mulitple E1EDKxx segments in a variable. We will do that using an XSLT function - starts-with(name(), 'E1EDK') in a loop.

<xsl:variable name="e1edkxx_original">
            <xsl:for-each select="//IDOC/*">
                
                <xsl:if test="starts-with(name(), 'E1EDK')">
                <xsl:copy-of select="."/>
                </xsl:if>
            </xsl:for-each>
        </xsl:variable>

Note: For checking every segment name inside IDOC, we need to run a loop on * (<xsl:for-each select="//IDOC/*"> )

Now we need E1EDKxx segment same as in the source idoc but with a change E1EDK14-ORGID = 'G2O' (where qualifier = '015'). We will store this data in another variable e1edkxx_changed.

<xsl:variable name="e1edkxx_changed">
            <xsl:for-each select="//IDOC/*">
                
                <xsl:if test="not (starts-with(name(), 'E1EDK14') and QUALF = '015') and (starts-with(name(), 'E1EDK'))">
                    <xsl:copy-of select="."/>
                </xsl:if>
                
                <xsl:if test="starts-with(name(), 'E1EDK14') and QUALF = '015'">
                    <E1EDK14>
                        <xsl:attribute name="SEGMENT"><xsl:value-of select="1"/></xsl:attribute>
                        <QUALF>015</QUALF>
                        <ORGID>
                            <xsl:value-of select="'G2O'"/>
                        </ORGID>
                    </E1EDK14>
                </xsl:if>
            </xsl:for-each>
        </xsl:variable>

We will store all the E1EDP01 segments in a variable all_e1edp01.

<xsl:variable name="all_e1edp01">
            <xsl:for-each select="//IDOC/*">
                
                <xsl:if test="starts-with(name(), 'E1EDP') ">
                    <xsl:copy-of select="."/>
                </xsl:if>
            </xsl:for-each>
        </xsl:variable>

E1EDSxx segments would be used as trailer. A variable trailer will hold all the E1EDSxx segments.

<xsl:variable name="trailer">
            
            <xsl:for-each select="//IDOC/*">
                
                <xsl:if test="starts-with(name(), 'E1EDS') ">
                    <xsl:copy-of select="."/>
                </xsl:if>
            </xsl:for-each>
            
        </xsl:variable>

We now have E1EDKxx_original, E1EDKxx_changed, all_e1edp01 and trailer records ready. Other requirement remains for relating the original E1EDP01 segments and correction E1EDP01 segments. We have been given following condition for relating original E1EDP01 segments and correction E1EDP01 segments.

HIPOS of correction E1EDP01 segments = POSEX of original E1EDP01 segments

This can be done using two for loops. But we will be using an XSLT key() function. The key() function returns a node-set from the document, using the index specified by an <xsl:key> element.

For using the key function, we first need to declare a definition of key function in the beginning of XSLT code.

<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:ns0="urn:test:xi:Sales:100">
    <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>
    
    <xsl:key name="E1EDPXX_HIPOS" match="E1EDP01" use="HIPOS"/>

Declaration of key in the beginning of XSLT, informs the processor that we would be using key() function somewhere in the code.Now we just need to run a loop on POSEX of the E1EDP01 and compare the key function output if:

E1EDP01-HIPOS from Key Function = Current E1EDP01-POSEX.

And then place these segments in the required format using copy-of function.

<IDOC>
                <xsl:copy-of select="$controlRecord"/>    
                <xsl:for-each select="$all_e1edp01/*">     
                    
                    
                    <xsl:if test="POSEX = key('E1EDPXX_HIPOS', POSEX)/HIPOS ">
                        <xsl:copy-of select="$e1edkxx_changed"/>    
                        <xsl:copy-of select="."/>                    
                        <xsl:copy-of select="$trailer"/>                
                        <xsl:copy-of select="$e1edkxx_original"/>     
                        <xsl:copy-of select="key('E1EDPXX_HIPOS', POSEX)"/>    
                        <xsl:copy-of select="$trailer"/>
                    </xsl:if>
                </xsl:for-each>
                </IDOC>

Here is the complete code which will transform the IDoc into required format.

<?xml version="1.0" encoding="UTF-8"?>

<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:ns0="urn:test:xi:Sales:100">
    <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>
    
    <xsl:key name="E1EDPXX_HIPOS" match="E1EDP01" use="HIPOS"/>
    <xsl:template match="/">
                
        <xsl:variable name="controlRecord" select="//IDOC/EDI_DC40"/>
        
        <xsl:variable name="e1edkxx_original">
            <xsl:for-each select="//IDOC/*">
                
                <xsl:if test="starts-with(name(), 'E1EDK')">
                <xsl:copy-of select="."/>
                </xsl:if>
            </xsl:for-each>
        </xsl:variable>
        
        <xsl:variable name="e1edkxx_changed">
            <xsl:for-each select="//IDOC/*">
                
                <xsl:if test="not (starts-with(name(), 'E1EDK14') and QUALF = '015') and (starts-with(name(), 'E1EDK'))">
                    <xsl:copy-of select="."/>
                </xsl:if>
                
                <xsl:if test="starts-with(name(), 'E1EDK14') and QUALF = '015'">
                    <E1EDK14>
                        <xsl:attribute name="SEGMENT"><xsl:value-of select="1"/></xsl:attribute>
                        <QUALF>015</QUALF>
                        <ORGID>
                            <xsl:value-of select="'G2O'"/>
                        </ORGID>
                    </E1EDK14>
                </xsl:if>
            </xsl:for-each>
        </xsl:variable>
        <xsl:variable name="trailer">
            
            <xsl:for-each select="//IDOC/*">
                
                <xsl:if test="starts-with(name(), 'E1EDS') ">
                    <xsl:copy-of select="."/>
                </xsl:if>
            </xsl:for-each>
            
        </xsl:variable>
        
        <xsl:variable name="all_e1edp01">
            <xsl:for-each select="//IDOC/*">
                
                <xsl:if test="starts-with(name(), 'E1EDP') ">
                    <xsl:copy-of select="."/>
                </xsl:if>
            </xsl:for-each>
        </xsl:variable>
        
            <IDOC>
                <xsl:copy-of select="$controlRecord"/>    
                <xsl:for-each select="$all_e1edp01/*">     
                    
                    
                    <xsl:if test="POSEX = key('E1EDPXX_HIPOS', POSEX)/HIPOS ">
                        <xsl:copy-of select="$e1edkxx_changed"/>    
                        <xsl:copy-of select="."/>                    
                        <xsl:copy-of select="$trailer"/>                
                        <xsl:copy-of select="$e1edkxx_original"/>     
                        <xsl:copy-of select="key('E1EDPXX_HIPOS', POSEX)"/>    
                        <xsl:copy-of select="$trailer"/>
                    </xsl:if>
                </xsl:for-each>
                </IDOC>
    </xsl:template>
</xsl:stylesheet>

The source file in Altova

On executing XSLT, target file is generated

Important Points to Note:

1) For running a loop on IDoc data, we choose //IDoc/* since we have to copy multiple segments.

2) Use of xslt function starts-with(name(), 'Pattern') to copy data.

3) Copying the required segments in variables and then putting them in desired position.

4) Using Key Function to relate two segments. Key function is quite efficient when dealing with large XSLT.

A Performance Analysis between point to point Web Service call Vs Brokered Web Service call via PI

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 11:43:58 PST

We once had a requirement to integrate a 3rd party IVR system with SAP ECC. As a standard practice we approached the solution to be brokered , ie interfacing via SAP PI. The interfacing solution agreed upon was a synchronous webservice call. We would be exposing the web service and IVR would be consuming it.

The performance requirement we had was 3 secs. That is the request-response cycle should complete within 3 seconds. This led us to consider a statistical analysis between the following cases:

Using PI-AAE -Proxy -- Here we had exposed the Webservice out of PI, and IVR would Call PI webservice. PI will call ABAP proxy on the ECC end using Advance Adapter Engine's local processing capability.
Direct ECC- RFC Webservice By-Passing PI -- This case, an RFC was written on the ECC end and exposed directly out of ECC as WebService which IVR would Call.

We tested both the scenarios using SOAP UI and below is the statistics which we got. Whatever cases crossed the SLA of 3 secs is highlighted in yellow.

Summany of the analysis:

Conclusion:

Looking at the above statistics, we find that Direct ECC calls performed better(but not significantly) than being routed through PI AAE. We tested with a small set of data, and having simplest coding, involving one select statement only. However, with increase of data volume, and with further complex logics involved, this observation / ratio of response times might get changed

Can SAP PI and Seeburger Business Integration Server (BIS) co-exist in an Integration landscape?

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 11:42:43 PST

SEEBURGER Business Intelligence server (BIS) is one of the most common EDI translators in most SAP Implementation projects. SAP PI is also typically available in such a landscape being extensively to support various Integration needs.

SEEBURGER BIS has distinct features to support B2B communication/Integration. A few of them are as follows

Managed File Transfer
Out of the Box/Standardized EDI translation mechanisms
Support for secured protocols like AS2, SFTP required for critical and sensitive financial/Customer transactions.
Support for High Volume transactions and large message sizes.
Out of the box features for Partner/Agreement management.

With that said, BIS has certain limitations in its native integration with SAP systems. The integration mechanisms available are IDOC and File systems. SAP PI can bridge the gap to produce an efficient implementation.

The following are a few use cases for SAP PI and SEEBURGER BIS to be used together in a landscape

Native communication with SAP systems

In majority of SAP implementations, a common integration strategy used is to avoid creation of custom IDOC types. More so in cases where the number of source fields on the interface is not large and custom logic needs to be implemented to populate such fields. ABAP Proxy using the XI adapter is an obvious choice in most cases.

SEEBURGER BIS does not support a proxy like communication. In order to achieve the same a pass-through or a hop interface can be built with SAP PI passing the message to SEEBURGER BIS, without any data transformation. BIS can then transform this message into an acceptable target format and transmit it to the business partner via a secure channel.

BIS supports inbound and outbound HTTP calls. SAP PI can connect to SEEBURGER BIS via a receiver HTTP adapter.

This would reduce the development effort of an SAP system by avoiding creation of custom IDOC types. This should only be done for interfaces in which the message sizes are not too large and the additional delay in message processing due to the hop interface is acceptable. This would avoid failures in PI due to message sizes.

Transmission of data to business partners using secured protocols like AS2 or SFTP

This is a case in which, message sizes are small and the transformations are easily achievable within SAP PI or is delivered as predefined content within SAP PI e.g. SAP ECC and Experian Integration for Credit Checks. Here the requirement is to deliver files (non-EDI) to business partners using secured mechanisms like SFTP or with encryptions not supported out of the box in SAP PI.

BIS supports an out of the box FTP server. This server has 2 nodes a Central or Internal Node and an external node on the DMZ. SAP PI can generate a file onto the central BIS node. BIS can then deliver the file via secured mechanisms like SFTP to business partners.

This is a pass-through process in BIS.

These are a few patterns which I implemented in one of my implementation and was able to achieve reduced development times and additional security benefits.. The PI version used was SAP PI 7.11 and the SEEBURGER version used was SEEBURGER BIS 6.3.3

Data Cross Reference Strategies in SAP PI

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 11:40:13 PST

Data cross reference or value mapping is a critical component of any SAP PI Integration Project.

In very simplistic terms, the above is a technique to map data values from a source produced format to a target acceptable format. Some very common examples are country code conversions, UoM conversion etc.

In one of my implementation project, I had the following use cases.

Basic value mapping requirements –

This requirement was to map basic data objects like Country Code, UoM, and Plant to Company Code and Company codes to bank accounts. The data volume for these objects was under 5000 cross references.

Master Data Cross reference –

The requirement was to convert Vendor Numbers between SAP and 3rd party systems. The data volume was huge in the range of 30,000 - 50,000 records. This cross reference was not available in ECC, and the only source of the cross reference was MDM. The frequency of use for this cross reference was also high.

Master Data Cross reference SNC

The requirement here was to have a cross reference between an SAP Vendor/Plant to SNC business partners. All non-standard SNC interfaces would need the business partner to be populated for successful message processing. The volume for such data was high as well as the frequency.

Multiple options were weighed to determine a cross reference strategy. The driving factors were defined storage location, high performance use as well as ease of maintenance and replication.

The following were the major options which were considered

Translate at source -

The value translation can happen at the source. The converted value needs to be available at source. This occurrence of this scenario is rare. This is the most optimized option as this avoids any additional overhead to message processing.

Lookup at source -

In case the converted value is available at source, and the source is incapable of passing transformed value as a part of the interface (mostly in case of Standard Idocs, with a strategy to minimize custom exits) another option is to lookup the value from SAP PI mapping programs.

This lookup is done via cross system RFC or SOAP lookup from the PI mapping.

It should be noted that the above could cause performance issues during effecting interface runtime, especially in cases where the frequency and runtime requirements of these interfaces are high.

Replicate to PI Cache (Value Mapping Replication) –

In this case the mapping values needs to be replicated to the PI cache from a consolidation system. PI supports out of the box features using ABAP and JAVA proxies to replicate value mapping data to PI value mapping cache from an ABAP based consolidation system. Data can be maintained in the consolidation system.

This mapping can be read from the PI mapping, via out of the box optimized functions.

The best practice on a fairly sized PI system is to have the number of records in the PI cache to be limited to less than 20,000 entries. This option was best suited for basic data objects mapping.

Replicate to PI ABAP Stack –

In case the cross reference data volume is high, the PI ABAP stack can be used to store the cross reference.

This data can be read from the mapping program via local RFC call to the PI ABAP stack.

This is case in which data is not available at source, data storage requirements are high and lack in performance due to cross system look-ups needs to be avoided.

Based on the above options the following flow chart could be defined.

Based on the above chart, the scenarios were decoded in the following manner.

· Basic value mapping requirements –

This is case in which the mapping was not available on the source. The volume of data was around 5000 records and the frequency of use was high. Thus a consolidation system like SAP ECC was chosen to maintain the cross reference; this was then replicated to PI value mapping cache using the standard value mapping replication techniques.

This is resulted in high performance of frequently run interfaces.

Master Data Cross reference –

This is the use case in which the data volume was high, data was not available at source and performance requirements were high. The value mapping cache was not the option to go with. The data was replicated to the ABAP Stack tables, and a local RFC look-up was done from PI mapping.

This RFC calls were more optimized than cross system RFC/SOAP look-up calls.

As the cross reference was available at MDM, a simple replication interface was built from MDM to PI (using Receiver RFC adapter) to keep the data up to date.

The PI version used here was PI 7.11.

The Code Life Cycle - Managing Custom Code?

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 11:23:42 PST

In majority of projects that implement SAP PI, developments that require custom adapter modules and java proxies are predominantly less in number. But there are projects in which the most of the developments are based on such custom codes. Also with the single stack concept gaining importance, these developments will gather more storm.

How do you manage your Java code when it comes to SAP PI?

I assume most of us don't really bother. We get our coding done on NWDS, compile the code locally, export the Jar or EJB, load them into the server and Voila, we are done.

But as the code base increases, managing this gets tricky. PI, out of the box doesn't provide you any help either. You can edit your code in ESR but only hope for it to compile on itself. If it could, it would have been wonderful, aint it?

One of the recommended ways is to implement NWDI and have the code centrally maintained. But I personally haven't found it easy configuring NWDI and start hosting my code. Ignoramus? Mea Culpa! (Maybe SAP can indeed host a good how to guide to help us with this?)

With Eclipse integration into ESR, I believe this will open a new dimension of possibilities. Imagine code management via the IDE integrated with a check in - check out version management feature.

I hope this is what SAP is aspiring toward. In any case, I have already posted this on Idea place and will wait to see if anyone picks this up.

Question:

How do you manage your custom Java code? Do you use NWDI extensively for SAP PI projects? Or is there any other way you are achieving this in your SAP PI projects?

Flexible Serialization in SAP PI

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 11:41:43 PST

Let us start by defining what strict serialization is - Serialization during message processing is a feature by which messages are delivered to the target in the same sequence as generated by the source via a middleware.

The quality of service defined for the same in EOIO (Exactly Once in Order).

Serialized messages are processed in a specified queue both on the middleware and the target system. In case of a processing error on any particular message in the queue, the message queue gets blocked; all subsequent messages go on a wait state, till the error is resolved.

Some implementations have a unique requirement for Flexible serialization. In this case, there is a need of serialization of messages and in case of error the other messages can continue, staying serialized, ignoring the failed one which can be attended manually at a later point of time.

The following are the advantages of flexible serialization.

1. The message processing remains serialized in majority of error-free cases.

2. The message processing for an interface does not halt due to the failure in one message on the queue.

3. An error notification, informs the support team of the failure and the subsequent manual action can be performed by them.

It should be noted, that Flexible Serialization is implemented only in cases where, the percentage forecast for failures in less, and manual re-conciliation is possible.

SAP PI, like most middleware systems does not support this out of the box. A simple WATCHER report can be written based on QRFC APIs in PI and connected ABAP Based SAP systems to achieve the same. The purpose of this watcher program would be to move the error message out of the active LUW SMQ2 to a saved LUW SMQ3 for all queues requiring Flexible serialization.

The following would be the details of the watcher operation.

1. Watcher to Monitor all specified Serialized queues on SMQ2

2. The Watcher looks for specific failure statues like SYSFAIL or CPIC

3. Upon determination of an error the watcher checks for the message depth on SMQ2 (running/active LUW). . The business team can define the active queue depth, for flexible serialization to work

4. The watcher also looks at the depth on the saved queue or SMQ3. This would be important to prevent message movement if there is a generic issue on the interface, which can impact every flowing message.

5. The watcher based on the above parameters would then move the message to SMQ3.

6. Finally an alert notification should be send out to the respective users, specifying the movement.

The following is a selection screen.

The following is the code snippet for the same.

TYPE-POOLS: salrt.
* Get all errored inbound queues
TABLES: trfcqin.
SELECT-OPTIONS: s_qname FOR trfcqin-qname,
                s_eqname FOR trfcqin-qname,
                s_qstate FOR trfcqin-qstate,
                s_iqname FOR trfcqin-qname.
PARAMETERS: p_pdf AS CHECKBOX DEFAULT 'X',
            p_qdepth TYPE int4,
            p_sqdep TYPE int4.
DATA: v_err_qs TYPE sy-tabix,
      v_err_qss TYPE string,
      v_err_msg(200) TYPE c,
      v_msg TYPE string,
      v_trfcmess TYPE t100-text,
      v_qstate TYPE trfcqin-qstate,
      o_cont TYPE REF TO if_swf_cnt_container,
      o_err TYPE REF TO cx_root,
      w_qin TYPE trfcqin,
      w_qstate TYPE trfcqstate,
      w_tid TYPE arfctid,
      t_no_pdf_cont TYPE soli_tab,
      t_qin TYPE TABLE OF trfcqin,
      t_qstate TYPE TABLE OF trfcqstate,
      v_q_deep TYPE sy-index,
      ls_message TYPE zsca_string,
      lt_message TYPE zttca_string,
      lt_qview TYPE TABLE OF trfcqview,
      ls_qview LIKE LINE OF lt_qview.
*CONSTANTS: C_INT_ID TYPE ZOBJID VALUE 'ECC_BLOCKED_QUEUES'.
START-OF-SELECTION.
*Identify the queues
CALL FUNCTION 'TRFC_QIN_GET_ERROR_QUEUES'
    EXPORTING
      client      = sy-mandt
      with_qstate = 'X'
    TABLES
      qtable      = t_qin
      qstate      = t_qstate.
DELETE t_qin WHERE NOT qname IN s_qname.
* DELETE T_QIN WHERE QNAME IN S_IQNAME.
IF NOT s_qstate[] IS INITIAL.
    LOOP AT t_qin INTO w_qin.
      CLEAR: v_qstate.
      CALL FUNCTION 'TRFC_QIN_STATE'
        EXPORTING
          qname                   = w_qin-qname
        IMPORTING
          qstate                  = v_qstate
*         QLOCKCNT                =
          qdeep                   = v_q_deep
*         QRESCNT                 =
*         WQNAME                  =
*         ERRMESS                 =
*       EXCEPTIONS
*         INVALID_PARAMETER       = 1
*         OTHERS                  = 2
                .
      IF sy-subrc <> 0.
      ENDIF.
      IF v_q_deep < p_qdepth.
        DELETE t_qin.
      ELSE.
        CHECK NOT v_qstate IS INITIAL.
        CHECK v_qstate NOT IN s_qstate.
      ENDIF.
    ENDLOOP.
ENDIF.
*   * Move the first LUW to the saved queue
LOOP AT t_qin INTO w_qin.
    CLEAR: w_tid.
    CALL FUNCTION 'TRFC_QINS_OVERVIEW'
      EXPORTING
        qname = w_qin-qname
        client = sy-mandt
      TABLES
        qview = lt_qview.
    READ TABLE lt_qview INDEX 1 INTO ls_qview.
    IF ls_qview-qdeep < p_sqdep.
      CALL FUNCTION 'TRFC_QIN_GET_FIRST_LUW'
        EXPORTING
          qname                   = w_qin-qname
          client                  = sy-mandt
*     NO_READ_LOCK            = ' '
       IMPORTING
         tid                     = w_tid
*     QSTATE                  =
*     WQNAME                  =
          errmess                 = v_trfcmess
*     FDATE                   =
*     FTIME                   =
*     FQCOUNT                 =
*     SENDER_ID               =
*   TABLES
*     QTABLE                  =
       EXCEPTIONS
         invalid_parameter       = 1
         OTHERS                  = 2
                .
      IF sy-subrc <> 0.
* MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
      ELSEIF w_tid IS NOT INITIAL.
*      IF NOT S_EQNAME[] IS INITIAL AND W_QIN-QNAME IN S_EQNAME.
*        WRITE: / '*', AT 3 W_QIN-QNAME, AT 40 W_TID, AT 70 V_TRFCMESS.
*        CONTINUE.
*      ENDIF.
        SUBMIT rstrfcdk AND RETURN WITH tid = w_tid.
* Activate the QIN scheduler for this queue.
        CALL FUNCTION 'QIWK_SCHEDULER_ACTIVATE'
          EXPORTING
            qname = w_qin-qname.
        WRITE: / w_qin-qname, AT 40 w_tid, AT 70 v_trfcmess.
      ENDIF.

The above queue movement should be notified to the business user, with the help of any alerting mechanism.

Idea Place and a case for SAP PI

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 10:57:55 PST

Almost a year ago, I posted a blog titled 'Contribute to better a Product? Do you have ideas for SAP PI?'. The objective was to get a dedicated page for Process Integration so that consultants, users and customers, if interested could post their thoughts and suggestions on improving the product.

Currently Process Integration and the future is under review. The wait has exceeded a year now but the interest of many fellow community members in this area has not diminished. Recently we had Prateek Raj Srivastava, open a wiki page for suggesting improvements to PI. I also met with a customer who said he has initiated a dialogue with SAP in suggesting improvements to the product. There are user groups who are focused on product improvements to PI but then it is a closed network and not all can discuss and contribute their ideas.

We need a place for ideas and the question I have is 'Is Idea place the right place for this specific need?'. I hope someone managing Idea place can answer it. I personally believe that it is since it will be open enough for consultants, customers and other stakeholders. But if I am proven wrong, then the wiki page started by Prateek will need to be publicized more so as to bring it to the attention of contributors and get the ideas flowing.

PI has matured over the years but there is still a lot that has to go into the product to make it an ideal fit for a multitude of requirements. The community is keen to contribute and I am sure SAP will be happy to listen.

Handling two different messages within mapping

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 10:55:57 PST

In one of our project assignments, we had a kind of strange, or a rather tricky requirement. This is was supposed to be a normal XML file to a web service scenario, but the situation was that, the source system gives an XML file, which contains customer’s complaints / acknowledgments. The file naming scheme for both complaints and acknowledgements was the same, and we had to process only those XML files that had complaints in them. There was no way to identify the type of data the files had, as both the complaints and acknowledgements had same file name, but different root tags.

After a lot of research, the only solution we came up with is to have two different outbound interfaces, one for complaints, and the other for acknowledgements. Firstly, pick the file, convert it into a string using a java mapping, then use the graphical mapping to split it into complaints / acknowledgements.

Below are the steps:

First, we have to create a dummy data type with just one element in it, as shown in the below screenshot.

Then create the Message Type and Message Interface for this data type and set the direction of the message interface as Outbound.

This interface is going to be used as the Sender Message Interface.

Then import the XSD / create the data type & message types for the complaint and acknowledgments. Create their message interfaces accordingly with direction as inbound.

In the next step, we have written a simple java mapping program to convert the given XML input into String and import it into the “Imported Archives”. The output of this java mapping is passed to a graphical mapping, which creates the XML message for complaint / acknowledgement based on the input.

Java Mapping:

The graphical mapping is a one-to-many mapping, with the one outbound interface, and two inbound interfaces, with the occurrence of the inbound interfaces defined as “0..1”. The inbound interface is our dummy inbound message interface, and the inbound interfaces would be the Complaints and Acknowledgments interfaces.

The graphical mapping takes the XML input in String format, uses a substring UDF fetch the values.

The UDF “getString” in the above screenshot takes the XML string, the index of the beginning and ending tags of the element “messagetype” as input and returns the actual value of this element called “messagetype”. If this value is “Complaint” then, the target structure “ichicr”, which is for complaint is created, else, the other structure “ichicsrack”, which is an acknowledgment.

Code for getString():

The same logic is used to retrieve the values of other elements, and are populated into the target element as shown in the below screenshot.

Then, create the operations mapping as shown below:

This mapping output returns either of Complaint / Acknowledgement messages at a time and the other message would be empty. Create the configuration scenario in the Integration Directory, with one sender and two receivers.

Result of Operations Mapping:

1. For Complaints:

2. For Acknowledgments:

Hope this blog is helpful.

Idea Place for SAP NetWeaver Process Orchestration - Process Integration

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 10:55:02 PST

Do you have ideas and feedbacks to improve PI and BPM? Now, you can submit them and solicit opinions from others.

When working with PI and BPM, I am sure we all come across situations where we feel if there are certain features or capabilities in PI and BPM, it will improve our usage of the products. These ideas can be in areas of development, monitoring, administrations, or anything.

Now, there is a place where we can submit our ideas and feedbacks. We can also view and comment on ideas and feedbacks submitted by others...and, even vote for our favorite ones. And, it is possible we can even see our ideas actually to become part of the product.

To submit and view ideas and feedback, you can access the links below:
Idea Place for PI
https://cw.sdn.sap.com/cw/community/ideas/sap_netweaver/sap_netweaver_process_orchestration_process_integration

Idea Place for BPM
https://cw.sdn.sap.com/cw/community/ideas/sap_netweaver/sap_netweaver_process_orchestration_business_process_management

SCADA Integration with SAP using SAP PI – Part 3

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 11:03:00 PST

SCADA Integration with SAP using SAP PI – Part 3 –Interface

In my previous blogs, we saw Systems Integration and how they communicate in exchanging Tag values using a testing tool. Let me discuss about the Interface with actual systems involved. I mean Sap ERP TSW, SAP PI, OPC Gateway Systems, and SCADA Servers (OPC Server). Based on the two previous blogs I hope you got an idea on systems involved.

If you look in to the Business Case, Interface was demanding communicate with Decentralised SCADA servers. They were 10 instances of SCADA Servers in the Landscape, primarily provided by INVENSYS and TELVENT. All the instances will contain same Tags and Values based on the periodic synchronization of the instances. Our requirement was to fetch the values for specific tags based on the availability of the instances. There were some primary instances which will be given priority and rest of them as redundant instances.

In ERP TSW module this Read Interface has to be executed at a scheduled time every day to get the Tag values and store it in Z Tables. These values are utilized by the module to finalise Allocations, based on which Ticket Creation, Delivery Creation and Billing Creation happens [Refer IMG 1.1: Business Process] in my first blog of this series.

So to fetch the tag values in ERP all the Tags relevant to the SCADA server instances are stored in Z Table and used to create a proxy request for Read operation with the relevant tags.

Synchronous Communication

Onward Journey -->

In ERP - TSW module a background program which executes every day at scheduled time sends a client proxy request containing SCADA TAG’s to SAP PI.
In SAP PI Read Interface is executed which receives client proxy request and transforms message using operation mapping.
Resultant of this mapping will be mapped as an Operation – Message called ReadSoapIn of Web Service provided by OPC GATEWAYs.
Using conditional routing in Read Interface, SAP PI sends to ReadSoapIn operation to available OPC GATEWAY using a SOAP request.
In OPC GATEWAY a corresponding Web Service will receive SAP PI’s ReadSoapIn request, and sends the request to OPC SERVER in SCADA using .NET program and COM/DCOM technology.

Synchronous Communication

<-- Return Journey

OPC SERVER processes the request and reads the values related to the requested TAG’s, and sends response back to OPC GATEWAY.
OPC GATEWAY will send the response using Operation – Message ReadSoapOut to SAP PI.
SAP PI receives the response using ReadSoapOut Message, this message gets transformed to ERP client proxy response.
ERP client proxy program update relevant table with TAG Values for all OPCSERVERS available.

TAG Values in table will be used for finalizing allocations, ticket creation, delivery and billing.

Now will see abstract of objects which are needed in SAP PI.

Sender and Receiver Message Structures with Mappings in PI:

Receiver Determination:

In ERP a program controls the flow of Interface, to which Gateway and to which SCADA (OPC Server) request is sent. Based on the source field value the receiver OPC Gateway is determined, and Web Service specifically running for particular OPC Server will be triggered in OPC Gateway based on the request XML.

Receiver Communication Channel:

XML - Request:

XML – Response

I hope this blog series helped you in understanding SCADA systems integration with SAP.

SCADA Integration with SAP using SAP PI – Part 2

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 11:02:05 PST

SCADA Integration with SAP using SAP PI – Part 2 – Systems Integration

In my previous blog, I briefly explained about the systems involved and the business case. Let me share how systems Integration happens in this part. On the high level SAP ERP system will request Tag values for particular SCADA OPC Server Instance, in response OPC Server will reply with Tag values, Quality of the Tag value, Quality of the Tag, OPC Server Status, and Time Stamp.

A deep drive in to the Systems Integration:

As SAP ERP system makes a request and gets the response, so we made this as Client Proxy communication to SAP PI. On the other hand SAP PI system is connected to OPC Gateway (Technically: OPC Client System) across the Firewall. In OPC Gateway system a standard Web Service is exposed, this will be provided by vendors who supply .NET Wrapper software like MatrikonOPC/Advosol. So SAP PI will consume this Web Service at receiver end. A SOAP request sent by SAP PI will be received by service available in OPC Gateway, now this request has to be passed to OPC Server. In OPC Gateway Web Service request is mapped to a .Net program which will convert HTTP request to COM/DCOM Request and sends the same through ISS to OPC Server. OPC Server will process the request and reply with the necessary Tag Values to OPC Gateway, OPC Gateway will map the response to SOAP response, which will be reached to SAP ERP through Client proxy response. This is how synchronous communication happens between Integrated Systems.

SAP ERP:

ERP makes a business request consists of Tag Names:

“A tag name in terms of SCADA represents a variable to hold various quantitative measures like GCV (Gross Calorific Value), NCV (Net Calorific Value), Pressure, Volume, and Temperature. Each quantitative measure will have its own tag name”

Because all the data stored in SCADA will be in the form of Tags, ERP system should send relevant Tag Names to get the values. Sample tag names and specific codes for quantitative measures are given in my previous blog.

As ERP sends a Proxy request this could be manipulated as a SOAP request by using SOAP Message Protocol as XI, which will point to AAE (Advance Adapter Engine).

OPC Gateway:

Client’s landscape has 2 OPC Gateway systems. Communication happens with either of the Gateway’s based on the availability. Each Gateway will be linked to available Instances of SCADA.

OPC Gateways are manipulated to handle SOAP request coming from SAP PI and a .Net Client Application (Wrapper Software) in OPC Gateway converts the call in to D[COM] request and sends to OPCSERVER.

.NET applications can access OPC servers only through a software layer that is usually called a .NET Wrapper. There are no OPC standard specifications for a .NET interface, so different vendors offer .NET wrapper software with much different interfaces and features.

One of the .NET wrapper software we used was ADVOSOL XMLDA.NET wrapper. This supports access to OPC DA and XML DA servers. The Helper classes of wrapper software provide user friendly server access methods for features such as browsing and reading item values.

OPC DA Sample Server & Matrikon OPC Explorer Client Application

OPC Gateway (OPC Client Machine) system with ISS installed and .NET wrapper running can be monitored using application called Matrikon OPC Explorer.

I have simulated OPC Server and OPC Gateway in my desktop using OPC .NET Sample DA Server –Tags and Tag values are maintained here (generally updated by RTU’s) and Matrikon OPC Explorer (for OPC Gateway) to view the tags values by installing .NET framework and ISS on Windows OS.

In this OPC DA Sample Server, we can add New Tags, and edit existing Tag properties like Data type, Quality. If this OPC DA Sample Server is connected to RTU’s will have real Tags names as I mentioned in my first Blog.

Since it is manipulated on my machine to show case the Systems, I could load some dummy Tag names and provide dynamic values in to the Server Instance. [Please note Tag Name in terms of OPC Server is Item ID]

Now let’s see how to access the Tags available in OPC DA Sample Server in to Matrikon OPC Explorer application:

Since I have installed both (OPC DA Sample Server and Matrikon OPC Explorer) on the same host machine, when I open Matrikon OPC Explorer it will automatically detects available OPC Servers on the same host machine.

Now we will see how to add the Tags to view the values available in the OPC DA Sample Server Instance in Matrikon OPC Explorer application.

Press Add Tags which will open another window, this will allow adding of new Item ID: Tag Name to monitor:

This application (Matrikon OPC Explorer) need to be connected to the OPCSample.OpcDa.20Server.1, to achieve this press Connect button. Once we press this button you can see the Server is connected and Status as Running on the bottom left window with refresh rate.

Now we will see how to add the Tags to view the values available in the OPC DA Sample Server Instance in Matrikon OPC Explorer application.

Press Add Tags which will open another window, this will allow adding of new Item ID: Tag Name to monitor:

Press Browse Menu Item and select Flat Browse as follows:

You can see all the available Tags/Item ID’s:

Double click relevant Tags/Item ID you wish to explore, which will be added in to the Tag to be added window.

Now press File -> Update and Return

With this action it will add the Tag to the Matrikon OPC Explorer main window and fetches the value available in: OPCSample.OpcDa.20Server.1 as shown below:

This is how we monitor the Tag values in Matrikon OPC Explorer in OPC Gateway (OPC Client). Other than Matrikon OPC Explorer, you can use any other tools available.

Now we see how to get this value with the Web Service request from a web client like SOAPUI testing tool.

Web Service and Functions:

With the available Web Service we can Read / Write Tag Values available in OPC Gateway’s. We can utilize GetProperties function to get the Tag’s Properties like Quality and Time Stamp etc. We can utilize GetStatus function to check the status of the Tag value like Active/In-Active.

Once the Web Service is available, this can be used in SAP PI on receiving end. Let us see the Interface details with Read Operation, which will be triggered from SAP ERP TSW in my next blog. I hope this blog provides you little bit knowledge on OPC Gateway and OPC Server Instance.

SCADA Integration with SAP using SAP PI – Part 1

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 11:00:52 PST

SCADA Integration with SAP using SAP PI – Part 1 - Business case & Systems Involved

Before starting my previous project, I was little sceptic about SCADA (Supervisory Control and Data Acquisition) integration with SAP using PI, whether it is achievable or not. But I had an iota of confidence that this can be done somehow. With the help of SAP's XMII expert Pandiarajan .P and .NET consultant Mahesh Patel, I could successfully implement several interfaces integrating SCADA. Since there is no reference about this topic in SDN, so I thought of putting my experience in words as a series of blogs about the Integration.

1. SCADA Integration with SAP using SAP PI – Part 1 - Business case & Systems Involved.

2. SCADA Integration with SAP using SAP PI – Part 2 – Systems Integration.

3. SCADA Integration with SAP using SAP PI – Part 3 – Interface.

Business case:

Our client has India’s biggest pipeline connectivity network in Gas distribution, (Downstream – Delivery of Natural Gas through Pipeline & Gas Tankers) which is working under Central Government of India. They have Decentralized SCADA Servers providing by different vendors like INVENSYS and TELVENT. These will get Pressure/Volume/Temperature/GCV/NCV etc values from RTU’s (Remote Terminal Units)/Flow Meters attached across pipelines. Based on the values SCADA will control the GAS flow in Flow Meters remotely. Business case was to fetch these values in to SAP –TSW module and schedule the quantity of the Gas to be delivered on particular day.

Now let me introduce you with the landscape systems involved and brief introduction about them.

SAP ERP 6.0 EHP 4 TSW Module.
SAP PI 7.1 EHP 1.
SCADA with OPC DA Server (2.0) – Decentralized
OPC Gateways - OPC Clients.

SAP ERP TSW module:

TSW stands for Traders and scheduler workbench. Trader's and Scheduler's Workbench (TSW) provides functions for stock projection and for planning and scheduling bulk shipments using nominations. TSW provides the relevant master data to model the supply chain. The stock projection, planning, and nomination processes enable the scheduler to schedule bulk shipments while taking into account supply, demand, and available transportation.

Processes and Functions

The following planning tools are provided by the TSW application:

Planning (Replenishment Proposals and Safety/Target Stock Calculation)
Stock Projection Worksheet (SPW)
Location Balancing
Worklist

The following scheduling tools are provided by the TSW application:

Three-Way Pegging (3WP)
Berth Planning Board (BPB)
Nomination
Ticketing
Worklist

Since Client’s business is to deliver Gas to various dealers, using Pipeline & Gas Tankers, for daily nominations and scheduling the quantity to delivery, this module has been used in the project. [REF: help.sap.com]

SCADA with OPC DA Server (2.0) - Decentralized:

What is SCADA?

SCADA stands for Supervisory Control And Data Acquisition. As the name indicates, it is not a full control system, but rather focuses on the supervisory level. As such, it is a purely software package that is positioned on top of hardware to which it is interfaced, in general via Programmable Logic Controllers (PLCs), or other commercial hardware modules. SCADA systems are used not only in most industrial processes: e.g. steel making, power generation (conventional and nuclear) and distribution, chemistry, but also in some experimental facilities such as nuclear fusion. SCADA systems used to run on DOS, VMS and UNIX; in recent years all SCADA vendors have moved to NT.

SCADA systems are widely used in industry for Supervisory Control and Data Acquisition of industrial processes. Companies that are members of standardisation committees (e.g. OPC, OLE for Process Control) and are thus setting the trends in matters of IT technologies generally develop these systems. SCADA Server’s which our client own was provided by following vendors:

There were multiple instances of these SCADA servers which are decentralized. But servers get synchronized with real time data on periodical intervals.

How to communicate with SCADA?

These SCADA systems include OPC (OLE for Process Control) Servers inside, other than its own Data Sources. OPC Servers are software applications (drivers) that comply with one or more OPC specifications as defined by the OPC Foundation. OPC Servers communicate natively with one or more Data Sources on one side and with OPC Clients on the other. In an OPC Client / OPC Server Architecture, the OPC Server is a Slave while the OPC Client is the Master.

Versions of OPC Servers released so far:

Data Format in SCADA:

Data in SCADA is available in the form of TAG’s, something like:

For each tag corresponding value will be updated to SCADA database by the RTU’s (Remote Terminal Units) connected to Gas Pipeline. Based on which SCADA will control the flow of Gas through Flow Meters attached to Gas Pipeline. OPC Client machines will request these tag values by specifying the tag name. Client machines got the facility to subscribe the functionality to get updated information of tag values, every second or even shorter time interval.

For further information on OPC Server please refer to MatrikonOPC.com

OPC Gateways (OPC Clients)

OPC Gateways systems are Client machines that will communicate to OPC Server in SCADA system. These Gateway systems with Windows Operating System and ISS installed & running. Communication between the OPC Client and OPC Server is bi-directional meaning the OPC Clients can read from and write to OPC Servers. Classic OPC Servers utilize the Microsoft Windows’ COM/DCOM infrastructure as a means of exchanging data. This means these OPC Servers must run on the Microsoft Windows operating system. An OPC Server can support communications with multiple OPC Clients simultaneously. But latest releases of OPC Servers supports HTTP Communications so Web Service are provided by OPC Servers can be directly consumed by client applications like SAP PI or ERP directly.

OPC Client – OPC Server Communication

[Ref: MatrikonOPC.com]

These are all the systems and brief information about them.

Let us discuss about the Architectural connectivity that we followed in our Systems Integration in my next blog. I hope this blog series helps you get an idea on integration between SCADA and SAP.

PI/XI: NWDS not only for java development but also for PI administration.

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 11:04:51 PST

In this article I'd like to show two common ways of using SAP NetWeaver Developer Studio (NWDS) for administration purposes of SAP PI.
1. How to deploy JDBC/JMS external drivers (SDA files) without JSPM ?
If you'd like to use any external JDBC/JMS driver in your PI landscape you need to deploy the drivers first using an SDA file. In the latest PI releases you can only do that using JSPM (as SDM is no longer available). However you can also try deploying SDA files using NDWS and below you will find the necessary steps to do that.
Step 1
Create an EAR project.
Step 2
Import the SDA file into this project.
Step 3
Make sure to set the J2EE server in Window - Preferences - SAP AS Java menu.
Step 4
Right click on the SDA file and deploy it.

As you see the server will get restarted and your file will be deployed.

2. How to use NWDS for nice log file monitoring ?
In case something goes wrong with PI you may need to check the log files. You can do that from multiple places like NetWeaver Administrator (nwa) or even from transaction AL11 from your ABAP stack. The fact is that I don't like those two tools at all because they are either slow or show the log files in a very difficult to read format. What about using NDWS for that purpose?
Step 1
Once you see your J2EE servers in the deployment view you can select "Open SAP management console" link.

Step 2
Then in the "log files" link you can see all of your most important logs and their content (nicely formatted) below.

Step 3
The feature I like the most is the automatic refresh. Once you enable it you will see instantly the latest entries to your log files and in most cases this will enable you to find the cause of the issue very quickly.

The conclusion of this short article is - even if you're not a developer you can start using NDWS for other purposes. This will benefit you also in the future as in the future (even as of PI 7.3) you can do some PI configuration in NWDS already as shown in one of my previous articles:
PI/XI: PI 7.3 - accessing PI with NWDS - teaser

Automating Import of IDoc Metadata in Transaction IDX2

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 11:06:25 PST

Despite all of the advances in Web service and proxy technologies over the course of the past few years, I still find that many customers prefer to use the tried-and-true ALE/IDoc technology stack. Consequently, a frequent administrative headache is the upload of IDoc metadata using Transaction IDX2. In this blog, I will demonstrate a simple report program that can be used to automate this task.

What is IDoc metadata, anyway?

If you haven't worked with the PI IDoc adapter before, then a brief introduction is in order. As you know, all messages that flow through the PI Integration Engine pipeline are encoded using XML. Thus, besides providing raw connectivity, adapters such as the IDoc adapter also must perform the necessary transformations of messages so that they can be routed through the pipeline. In the case of the IDoc adapter, the raw IDoc data (you can see how the IDoc data is encoded by looking at the signature of function module IDOC_INBOUND_ASYNCHRONOUS) must be transformed into XML. Since the raw IDoc data does not provide information about segment field names, etc., this metadata must be imported at configuration time in order to enable the IDoc adapter to perform the XML transformation in an efficient manner.

From a configuration perspective, all this happens in two transactions:

In Transaction IDX1, you create an IDoc Adapter Port which essentially provides the IDoc adapter with an RFC destination that can be used to introspect the IDoc metadata from the backend SAP ALE system.
In Transaction IDX2, you can import IDoc types using the aforementioned IDoc adapter port. Here, you can import standard IDoc types, custom IDoc types, or even extended types.

If you're dealing with a handful of IDocs, then perhaps this isn't such a concern. However, if you're dealing with 10s or 100s of IDocs and a multitude of PI systems, then this process can become tedious in a hurry.

Automating the Upload Process

Now, technically speaking, the IDoc adapter is smart enough to utilize the IDoc port definition to dynamically load and cache IDoc metadata on the fly. However, what it won't do is detect changes to custom IDocs/extensions. Furthermore, if you have scenarios during cutover which block RFC communications, not having the IDoc metadata in context can lead to unexpected results. The report below can be used to automate the initial upload process or execute a kill-and-fill to pull in the latest and greatest changes. In reading through the comments, you can see that it essentially takes two inputs: the IDoc adapter port defined in IDX1 and a CSV file from your frontend workstation that defines the IDoc types to import. Here, you just need to create a two-column CSV file containing the IDoc type in column 1 and the extension type (if any) in column 2.

REPORT zidx_idoc_load_metadata. *&---------------------------------------------------------------------* *& Local Class Definitions * *&---------------------------------------------------------------------* CLASS lcl_report DEFINITION CREATE PRIVATE. PUBLIC SECTION. CLASS-METHODS: "Used in the selection screen definition: get_frontend_filename CHANGING ch_file TYPE string, "Public static method for running the report: execute IMPORTING im_idoc_types_file TYPE string im_idoc_port TYPE idx_port. PRIVATE SECTION. "Class-Local Type Declarations: TYPES: BEGIN OF ty_idoc_type, idoc_type TYPE string, ext_type TYPE string, END OF ty_idoc_type, ty_idoc_type_tab TYPE STANDARD TABLE OF ty_idoc_type. "Instance Attribute Declarations: DATA: idoc_port TYPE idx_port, idoc_types TYPE ty_idoc_type_tab. "Private helper methods: METHODS: constructor IMPORTING im_idoc_port TYPE idx_port, upload_idoc_types IMPORTING im_idoc_types_file TYPE string RAISING cx_sy_file_io, import_idoc_metadata, remove_idoc_metadata IMPORTING im_idoc_type TYPE string. ENDCLASS. CLASS lcl_report IMPLEMENTATION. METHOD get_frontend_filename. "Local Data Declarations: DATA: lt_files TYPE filetable, lv_retcode TYPE i, lv_user_action TYPE i. FIELD-SYMBOLS: <lfs_file> LIKE LINE OF lt_files. "Present the user with a dialog box to select the name "of the file they want to upload: CALL METHOD cl_gui_frontend_services=>file_open_dialog EXPORTING default_extension = 'csv' file_filter = '.csv' CHANGING file_table = lt_files rc = lv_retcode user_action = lv_user_action EXCEPTIONS file_open_dialog_failed = 1 cntl_error = 2 error_no_gui = 3 not_supported_by_gui = 4 others = 5. IF sy-subrc EQ 0. IF lv_user_action NE cl_gui_frontend_services=>action_cancel. READ TABLE lt_files INDEX 1 ASSIGNING <lfs_file>. IF sy-subrc EQ 0. ch_file = <lfs_file>-filename. ENDIF. ENDIF. ELSE. MESSAGE 'Could not determine target filename!' TYPE 'I'. RETURN. ENDIF. ENDMETHOD. " METHOD get_frontend_filename METHOD execute. "Method-Local Data Declarations: DATA: lo_report TYPE REF TO lcl_report, lo_exception TYPE REF TO cx_root, lv_message TYPE string. TRY. "Create an instance of the report driver class: CREATE OBJECT lo_report EXPORTING im_idoc_port = im_idoc_port. "Upload the set of IDoc types to load into the IDoc adapter; "This file should be a CSV file on the local workstation: CALL METHOD lo_report->upload_idoc_types EXPORTING im_idoc_types_file = im_idoc_types_file. "Import the set of IDoc types: CALL METHOD lo_report->import_idoc_metadata( ). CATCH cx_root INTO lo_exception. lv_message = lo_exception->get_text( ). MESSAGE lv_message TYPE 'I'. ENDTRY. ENDMETHOD. " METHOD execute METHOD constructor. me->idoc_port = im_idoc_port. ENDMETHOD. " METHOD constructor METHOD upload_idoc_types. "Method-Local Data Declarations: DATA: lt_csv_file TYPE string_table, lv_idoc_type TYPE edipidoctp, lv_ext_type TYPE edipcimtyp. FIELD-SYMBOLS: <lfs_csv_record> LIKE LINE OF lt_csv_file, <lfs_idoc_type> LIKE LINE OF me->idoc_types. "Upload the selected file from the user's workstation: CALL METHOD cl_gui_frontend_services=>gui_upload EXPORTING filename = im_idoc_types_file CHANGING data_tab = lt_csv_file EXCEPTIONS file_open_error = 1 file_read_error = 2 no_batch = 3 gui_refuse_filetransfer = 4 invalid_type = 5 no_authority = 6 unknown_error = 7 bad_data_format = 8 header_not_allowed = 9 separator_not_allowed = 10 header_too_long = 11 unknown_dp_error = 12 access_denied = 13 dp_out_of_memory = 14 disk_full = 15 dp_timeout = 16 not_supported_by_gui = 17 error_no_gui = 18 others = 19. IF sy-subrc NE 0. RAISE EXCEPTION TYPE cx_sy_file_io EXPORTING filename = im_idoc_types_file. ENDIF. "Copy the CSV file into the IDoc types table: LOOP AT lt_csv_file ASSIGNING <lfs_csv_record>. SPLIT <lfs_csv_record> AT ',' INTO: lv_idoc_type lv_ext_type. APPEND INITIAL LINE TO me->idoc_types ASSIGNING <lfs_idoc_type>. <lfs_idoc_type>-idoc_type = lv_idoc_type. <lfs_idoc_type>-ext_type = lv_ext_type. ENDLOOP. ENDMETHOD. " METHOD upload_idoc_types METHOD import_idoc_metadata. "Method-Local Data Declarations: DATA: lv_count TYPE i. FIELD-SYMBOLS: <lfs_idoc_type> LIKE LINE OF me->idoc_types. "Process each of the selected IDoc types in turn: LOOP AT me->idoc_types ASSIGNING <lfs_idoc_type>. "Check to see if the metadata already exists: SELECT COUNT(*) INTO lv_count FROM idxsload WHERE port EQ me->idoc_port AND idoctyp EQ <lfs_idoc_type>-idoc_type AND cimtyp EQ <lfs_idoc_type>-ext_type. IF lv_count GT 0. "If it does, go ahead and remove it: CALL METHOD remove_idoc_metadata( <lfs_idoc_type>-idoc_type ). ENDIF. "Now, import the IDoc metadata using IDX_FILL_STRUCTURE: SUBMIT idx_fill_structure WITH port = me->idoc_port WITH idoctyp = <lfs_idoc_type>-idoc_type WITH cimtyp = <lfs_idoc_type>-ext_type AND RETURN. ENDLOOP. ENDMETHOD. " METHOD import_idoc_metadata METHOD remove_idoc_metadata. "Remove the IDoc metadata using IDX_RESET_METADATA: SUBMIT idx_reset_metadata WITH port = me->idoc_port WITH idoctyp = im_idoc_type AND RETURN. ENDMETHOD. " METHOD remove_idoc_metadata ENDCLASS. *&---------------------------------------------------------------------* *& Selection Screen Definition * *&---------------------------------------------------------------------* PARAMETERS: p_idxprt TYPE idx_port OBLIGATORY, p_ifile TYPE string LOWER CASE OBLIGATORY. *&---------------------------------------------------------------------* *& AT SELECTION-SCREEN Event Module * *&---------------------------------------------------------------------* AT SELECTION-SCREEN ON VALUE-REQUEST FOR p_ifile. CALL METHOD lcl_report=>get_frontend_filename CHANGING ch_file = p_ifile. *&---------------------------------------------------------------------* *& START-OF-SELECTION Event Module * *&---------------------------------------------------------------------* START-OF-SELECTION. CALL METHOD lcl_report=>execute EXPORTING im_idoc_port = p_idxprt im_idoc_types_file = p_ifile.

Final Thoughts

SAP NetWeaver 7.3 Goes Green

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 11:07:18 PST

The concept of energy savings through SAP NetWeaver was first raised during the SAP Influencer Summit in December 2010 by Peter Graf, who used similar data to show how SAP development takes advantage of energy savings possibilities, which result in lower cost for operating our software and reduces carbon emissions.

In this blog I want to explain a bit more in detail how this calculation was done and on which assumptions it is based on. To make my case, I will use the example of SAP NetWeaver Process Integration (PI) software, as it is the area I worked on for many years and which I am most familiar with.

To demonstrate this idea, SAP carried out testing of its new SAP NetWeaver Platform using the SAP Application Performance Standard (SAPS). SAP defined a medium-sized landscape as being 37,500 SAPS for an SAP NetWeaver Process Integration (PI) customer.

Based on this definition, they found that the savings potential for PI is in the region of:

60% less energy consumption or around 32,000 kWh/yr
16 tons of CO2 savings per landscape/year
€6.5k saving potential per landscape/year

Medium-sized PI landscapes

As we do not have 100% correct customer data on sizes of PI installation systems, we decided to undertake an expert evaluation of small, medium, and large PI customer installations. With input from the SAP field organization, PI product management, PI sizing experts, and regional implementation team experts, we decided to make the assumption that a small PI customer has installed 5.000 SAPS for production system, 15.000 SAPS for medium and 30.000 SAPS as large customer in production.

Customer landscapes consist not just of of the production environment, but also development and quality assurance systems; so we decided to take the factor 2,5 times of the production system for the complete customer landscape. Usually the quality assurance environment is of the same size of production, whereas development has half the size of production. The result led to the conclusion that a medium sized PI landscape is about: 37.500 SAPS.

Why we calculated with "SAPS"

With the help of SAP Standard Application Benchmark results at www.sap.com/benchmark, statements can be made as to CPU consumption and memory consumption of particular software components. SAPS is an artificial calculation to decouple software from hardware influences. That gave us the possibility to exclude improvements in the hardware environment from the improvements of the software.

How these savings are reflected in the SAP (Quick) Sizing

Improvements achieved on the software level can either mean customers can reduce their hardware demand, or process more messages in regards to PI usage. We did the calculation in a way where we kept the number of messages the same and expected the customer will reduce his hardware. The calculation is based on a mix of scenarios executed as integrated scenarios on a PI system. For the PI centric folks: 75.000 asynchronous messages of 100 KB size and 25.000 synchronous messages of 20 KB size with simple mapping and receiver determination executed in one hour executed on standard PI dual stack set up. That gave a SAPS demand of 15.000 SAPS in a classic PI dual stack usage with 66% hardware usage. Now, that can be executed on AEX as the Java only deployment option of PI with 61% less SAPS and hardware demand. These numbers have been tested and verified therefore the customer and partners will see the same results using SAP PI sizing tool.

How to transfer SAPS to energy consumption, costs of energy, and carbon emissions

All hardware needs different amounts of power resources to achieve a certain number of SAPS. By introducing the SAP Power Benchmark, the approach is that a hardware vendor does not only show that his hardware set up can provide the requested SAPS, but although how much energy input is needed to achieve this. As we could not calculate for all hardware vendors, we decided to take the average measured values for up to date hardware from 2010. With further improvements on hardware these values have to be adopted, but as hardware is in place for several years and older hardware is much more energy-intensive, we think things will be fine for this year.

To calculate the energy costs and potential customer cost savings we took the price of 0,2 € per one kilo watt per hour (kWh) in Germany, which is certainly different for other countries and business customers buying and consuming high numbers of energy. The same is true for the carbon emissions created to produce one kWh, because that is dependent on the energy mix of nuclear power, natural gas, coal, renewal energy in the individual country. For Germany that was 506 grams of carbon per kWh according to BDEW and as the energy mix is not unusual we took this number for the calculations. Result: 16 tons of CO2 savings per landscape/year.

How savings can be achieved

The main reason is that with integrated configurations all steps can be executed on one stack in our case Java. That reduces the persistency steps from 3 and more in "dual stack times" (ABAP + Java) to in average to one now. As we learned that especially write operations to hard discs are in average close to two times more power demanding then cpu and memory usage that already gives the major part. Additionally, there is no further need to connect one or two times from ABAP to Java stack at runtime, which results in further savings in resource consumption.

Another very big advantage with PI 7.3 and the Java only deployment option is that it only brings the software resources with it which are absolutely needed for PI. That reduces installation times, software patching times and restart times of the Java server. The times measured SAP internally on hardware with good read write operations are around two hours for AEX installation and one and a half to two minutes for a restart of Java server. Due to advanced installer and new PI 7.3 ctc scripts (SAP service Marketplace Logon Required) these numbers will be achievable for customers, too. Customers can get the benefits immediately as the product is generally available since 31. May 2011.

What we can learn for other SAP application developments

Process tasks in one step and in one technical stack together as long as possible, as the AEX does the main part. Reducing write operations to hard discs significantly reduces power consumption. Therefore, you shoul limit the amount of time for processing in memory and write only to disc your logs and application date where it is really necessary!

What the future holds

There are many areas where SAP can improve the carbon footprint of its software solutions. On the company level, on demand solutions could help eliminate the need for each customer to deploy and operate their own system and system landscape, and then provide the hardware and training for people to run these applications. Another possibility could be the in-memory approach to reduce read/write operations.

Individuals can also play a role. Many improvement possibilities are already known in people’s respective areas of expertise. Maybe you have own findings and best practices you’d like to share. Feel free to post and discuss!

This publication is part of SAP’s Green IT initiative, which is based on the following three pillars:

Energy - reducing energy consumption
E-waste –minimizing waste through sustainable sourcing and recycling
Dematerialization –substituting high carbon products and activities with low carbon alternatives

SAP is committed to optimizing these three areas in its own IT operation, with its product development organization, as well as offering compelling solutions and services to our customers. If you need more information please refer to www.sap.com/greenit

How to solve JMS reconnection issue between SAP PI and Oracle Fusion Application Server (BEA/WLS)

noreply@blogger.com (SAP XI Tutorials) — Sat, 24 Dec 2011 10:53:39 PST

A permanent issue has been identified in integration scenario’s using JMS based interfaces between SAP PI and the Oracle Service Bus – OSB (formerly known as BEA Aqualogic Service Bus). The issue normally arises when the connection of a JMS receiver-or sender-channel becomes temporary unavailable.

When the instance or server hosting the JMS queue is restarted, the SAP PI JMS communication channel is not able to re-establish the JMS connection. As a result existing or new messages placed on the remote JMS queue(s) are not picked up by SAP PI.

The workaround to solve this problem involves in most cases restarting the communication channels and in some times it requires restarting the entire Java stack of SAP PI. After many unsuccessful attempts trying to solve this problem including but not limited to; researching possible applicable SAP notes, requesting support at both SAP and Oracle organizations, we did finally find the source of the problem on the Oracle side of this scenario. In the next lines I will explain how we solved it.

The Oracle Application Server also called the WLS (Weblogic Server) hosting the JMS queues provides several parameters that can be configured for different purposes for instance; performance, security, availability and persistency. See figure 1.

Figure 1.

One of those parameters in particular is the “Reconnect Policy” which is applicable for the JMS Connection Factory used by JMS clients (such as SAP PI) responsible for managing the way JMS clients should react in case a connection with the JMS server has to be re-established.

The parameter can be found on the tab “Client” Client under Messaging\Summary of Services\JMS Modules\..

Figure 2.

Figure 3.

As shown in the screenshot above the parameter can be set to either None, Producer or All. By default the value of the parameter is set to “Producer”.

Figure 4.

We changed the value from “Producer” to “None”, save the new settings and restarted the Oracle Application Server to test the new configuration. The results after the change were positive and until now we have not seen the issue anymore.