xenoterracide

Adventures with SOAP using Perl: Part 1 ( SOAP::Lite )

2011-10-28T15:03:14-07:00

The most prevalent of SOAP libraries for Perl is SOAP::Lite it is the oldest and most documented. Though for all of its documentation it can be quite painful to figure out how to use it.

First make sure you've read Part 0 to set up the server. Once that's done let's look at the most simple way to interface with this server.

In our first example we need to send a request to getCountries, which is a method provided by the API. You can determine this by reading namesservice.wsdl and looking for the operations to see what's available. Essentially this means we need to send SOAP request with a Body of <getCountries />. First we need to import SOAP::Lite.
If you look at our SOAP::Lite import you'll notice that were are passing the arguments +trace => [ 'debug' ]. There are various levels and options for +trace, but this turns on full debug printing which will be sent to stderr. You don't normally want to have debug running in production code, but it will be useful to illustrate our examples and the request they send and receive.

Now let's look at creating an actual SOAP::Lite request object. The first option we pass in is readable => 1, adds whitespace to the request sent so that it's easier to read when you're looking at the debug output, you should not enable this in production, as it makes the request bigger, and I believe it is not considered correct SOAP as I've been told something about extra whitespace in SOAP being considered invalid. The second option is proxy => 'http://localhost:8877' This specifies the hostname and port that the HTTP request is sent to. ns => 'http://namesservice.thomas_bayer.com/' is the namespace, which you can find by looking for namespace in the namesservice.wsdl.

Now we need to actually create and send an actual request. For this trivial request we simply need to call the method that we need on the remote server and then return the object. You can see that SOAP::Lite is generating a namespace for your request to use with the XML <namesp1:getCountries xsi:nil="true">, which is just fine in this case.

Of course we want to do something with our response. Please note that I've modified the code to use 5.10, but if you want to use print instead of say this code will work fine on 5.6 and up. valueof, which is documented in SOAP::SOM, returns the first element in scalar context, and an array in array context. So in my code I've shown both. The syntax used in the parameters to valueof is XPath, so an even simpler way to call it in this case would be $res->valueof(//country); and it would do the same thing with this XML.

Next let's look at the getNameInfo method, it's a bit more complex so let's look at the XML in the XSD. Here's the snippet that is really important. This means that we need to send a request with a body that looks like ( note: you can look at the sample data in MyExampleData.pm for other names. ) Set let's take a stab at writing some Perl. There are some important differences to note from our previous script. You'll notice that I call ->getNameInfo() directly on the request object, instead of passing it as a parameter to ->call. This functions basically the same as call and it will end up making the first tag inside of body. We could have doen this in our first example as $req->getCountries; and that would have been it. Now that we've covered the slight differences in calls, let's go over the completely new things.

SOAP::Data objects are used to create any further data structures. Obviously the hash key of name defines the element name, and value defines what you want to go into it, here I have hardcoded "Mark".

If you run this code you'll notice that it returns a faultstring (among other fault properties) "operation getNameInfo for SOAP11 called with invalid data", and details the error as "element `c-gensym3' not processed at {http://namesservice.thomas_bayer.com/}getNameInfo". Now go back and look at the request, you'll see a c-gensym3 element, where did that come from? Well, SOAP::Lite will generate elements for anonymous elements but we can fix this.

The only difference between this and the previous code is that we aren't putting a \ in front of SOAP::Data. I wrote it the first way because I had seen examples of that all over the place, and could not find a solution to getting rid of the gensyms until I asked this question on stackoverflow.

Unfortunately this is the most complex example that our server API has implemented. As an exercise to the reader I suggest Implementing a request for the method getNamesInCountry, which is no more complex but available.

Adventures with SOAP using Perl: Part 0 ( prelude )

2011-10-15T12:00:03-07:00

This is a prelude to a series on working with SOAP Requests using Perl. For the past 3 months I have been working on a Perl API for CyberSource's Simple Order API which uses SOAP (I should note, that although I believe most of the API is now stable some area's still need work, and thus I don't expect it to reach 1.0.0 anytime soon).

First I used SOAP::Lite to do my requests, but I found it confusing to construct the requests that I needed to make. I even discovered a bug that lead to the current ( 0.714 ) release of SOAP::Lite.

Next I started using SOAP::Data::Builder to make it easier to build my SOAP::Lite requests. This was good, but frustrating that I had to add my data in a specific order.

Finally I came upon XML::Compile::SOAP. A glance at it's API which used a hash to build requests seemed much better. however, it took me a few weeks and some help from Mark Overmeer (the author) and an update to XML::Compile::SOAP::WSS to get it to work.

If you're planning on starting a new project that requires SOAP I definitely recommend using XML::Compile::SOAP if you have a .wsdl and a .xsd to work with.

I will be covering all 3 of these methods in Parts 1, 2 and 3 of the series.

To get started we'll need a SOAP Server since I haven't been able to find any reliable public services. To do this you can install XML::Compile::SOAP::Daemon (which probably could use some Plack/PSGI love ). You'll also want to download it's tarball directly, extract it and cd examples/namesservice. Once you've done that you can do perl server.pl . Now you should have a Server running on http://localhost:8877 which we can use for testing our client examples.

Please be advised, these tutorials will not be explain XML, XSD, WSDL, or SOAP, but simply the Perl interfaces.

Experience with having a non Dist::Zilla user contribute to a Dist::Zilla project (it's not hard for them or me)

2011-07-04T23:07:21-07:00

I've heard many times that Dist::Zilla makes it harder for people to contribute to the project. This is not true, it is either unfortunately either ignorance or FUD (much like saying Linux is harder to use than Windows). Truly, there are things that some dzil users do that can make it harder, but it doesn't have to be that way. Michael Schwern recently contributed to one of my dzil projects without ever realizing I was using dzil, until I told him. He more recently stated on twitter, "While your solution works, it seems like it makes more work for you to shield contributors from dzil". This was true in this case because I wasn't sure how to effectively move a series of multiple patches, I now know it's easy to do with git. Here's how you can allow contributors to contribute to your dzil project without causing you or them undo pain.

Don't use anything that changes the line numbers of your source

Examples: don't use DZP::Prepender. Use modules like OurPkgVersion to insert VERSION, and make sure your # ABSTRACT and any pod is below the 1; at the end of your module. This will keep the line numbers of errors in your code from being different from the final build. It's still ok to use PodWeaver as it will save you way more time than it'll hurt, so long as you follow the rules about pod being at the bottom

Use a plugin that commits builds to your source control software

I personally recommend DZP::Git::CommitBuild

Make your build branch your default branch

I'm not sure how to do this with just git, but on github you can go to the admin section of your repository and change the default branch there. This makes it so that when someone clone's your repository the initial checkout is of your build branch. Your build branch shouldn't require dzil, it's the final build.

Now that you have your final build branch as the default branch anyone who wants to contribute can simply clone your repo and start hacking. Their are a couple of mistakes they could make, They could either change or add files that are maintained, or pruned by dzil. This did happen when Schwern sent me patches, one of them was the addition of a .gitignore, which I already had, but is being pruned out of the build branch. I can also see it happening to meta's and makefiles. These patches can simply be rejected as unnecessary, dzil-ified, or if they are truly a bug, then they can be reported and fixed upstream.

So what if someone sends you a pull request from build/master? well if it's just one or two patches, you add their remote, and do a git remote update and then you can git log remote build/master to find the sha1 of this patch. Now that you have the sha1 all you have to do is git cherry-pick [sha1] and it should apply. If there are any conflicts you may have to resolve them with git mergetool. However, the only conflict with Schwern's patches for me was the .gitignore, all other patches applied without assistance and applied correcly, surprisingly even the pod patch applied without issue.

If you have more patches than is comfortable with git cherry-pick then you need git rebase. The command you want is git rebase -i --onto master [sha1 before first sha1 in series] [tip of remote branch checkout]. So in my case git rebase -i --onto master 3f1e3748 schwern. What this appears to do is ends up rewriting my local schwern/build/master checkout and removes all the build commits, and then applies the patches on top. This means I can now do git merge schwern from the master branch, and all of his patches that I want will be successfully merged. For more on this strategy you may want to read this stackoverflow question.

given this is not quite as easy as a git pull that's a fast forward, but reality is it's not that hard once you know how to do the rebase and how it works. Of course this isn't ideal for constant contributors, those should simply learn to use dzil, but for the random contributor it should be ok.

My solution to not using PluginBundle:AUTHOR for dzil is git

2011-06-05T12:47:38-07:00

I am now tired of updating my dist.ini's for my Dist::Zilla projects. For many people this is when they start using a PluginBundle with their authorname. I discussed why you shouldn't do this a year ago. Now that I'm tired of managing my dist.ini's on an individual basis I'm going to show you how I'm going to solve the same problem everyone else is, which is getting tired of updating your dzil configuration for all of your projects. I'm using git to do it. You probably haven't considered this, or am thinking I'm wrong because you believe that git can't merge branches without a common history, which is not true. I did it with Regen2, Funtoo, Portage, and Sunrise, which have way more files than any perl repo, including Perl.

First let's talk about the advantages and disadvantages of doing this with git.

I'll start with the disadvantages. You'll have another repository to manage. Git isn't completely automatic, you'll have to remember to merge your changes. You'll also have to add the repository to your existing repositories. You will have to resolve merge conflicts at least once, and probably occasionally more, though most should be fast-forwards.

The advantages are... now the changes to your dist.ini are getting recorded in your history. You can now have a master dist.ini, but remove items without filters in your individual modules. You can share even more configuration as merged differences allow you to maintain differences in downstream commits. It doesn't automatically update all of your modules. Wait didn't I list that last one in disadvantages? Yep, it's an advantage because what if you update your PluginBundle and that update breaks one of your modules, but you don't know it because you haven't worked on that module in a while. You can use this for more than just dist.ini.

Ok so the first thing you you want to do is create your master dzil repo. This is not a git or dzil tutorial so go do that. I did it by creating a dzil new project and removing the files I couldn't use and making a few tweaks and amending all of those changes to the initial commit. Check out my repo for inspiration. Remember the directory structure has to be the same as a dzil repo structure for any common files.

Now add a remote from your new repo to an existing dzil project. Next you need to merge the branch from the remote into your project. The first time you do this you'll have to resolve conflicts. If they're like mine then they'll be easy and obvious. Once that's done future changes will be fast-forwards unless you change something in your perl module repo, then you might end up with a simple merge conflict. One thing to remember NEVER EVER try to merge from your module repo to your dzil repo, it will cause you extreme pain in the future. cherry-picking that direction is possible but not merging.

After that, all that is left to do is run your dzil tests and fix any breakage in your module. Happy merging.

Regen2 accounts transferred

2011-06-05T09:57:17-07:00

I killed regen2 about 2 years ago. Today I was contacted by Rafal Kupiec asking about the project. Since it's dead he asked if he could take over the project. I have transfered the github account, and freenode channel #regen2 to him. The google groups are remaining archived. At some point in the future there should be a site at asiotec.org I have no idea what he plans on doing with it.