Fabien Sanglier: Going Further With ALUI / Webcenter

Adding JavaScript includes to all portal browsing pages

noreply@blogger.com (Fabien Sanglier) — Wed, 10 Jun 2009 03:34:00 +0000

In almost every portal deployment, we usually have the need to create some JavaScript libraries that can be used by various components…possible functions could be window openers, url encoder/decoder or any other utility functions that you might not want to include in all your portlets.

Now, the question is: how do you add it to the portal page as (a) global include(s)? 2 possibilities:

Add the JavaScript include(s) to the portal header/footer portlets (which is usually done in publisher, and displayed on all pages)
Create a small UI customization that does the job more reliably.

You might have guessed it now: option 1 is not the one I am going to explain here. Why?

First, option 1 would not be worth a blog post since it is fairly easy to implement. But more seriously, depending on header or footer portlets is not reliable in order to include global resources. Indeed, different portlet headers/footers are displayed based on the current experience definition, and/or the current community you are in. Also, the header/footer portlet content can easily be modified (especially if implemented in publisher for example) which increase the chances of removing the resource include(s).

Ok, let’s dive into option 2!

By customizing the 2 main browsing portal DP (stands for Display Page) classes (MyPortalDP and GatewayHostedDP), you will be able to include all the JavaScript resource you need on virtually all end user portal pages: all portal community pages and on the gateway page in hosted display mode.

Luckily, those DP classes provide a method to override: DisplayJavaScriptFromChild. This method returns an HTMLScriptCollection object that will be read in order to add the JavaScript to the header tag (in between the portal <head></head> tags).

The following could be what you might want to have in this method override (it gets the list of JavaScript files to include from a Varpack, and iterate through this list in order to add it to the returned HTMLScriptCollection object)

protected override HTMLScriptCollection DisplayJavaScriptFromChild()
{
    HTMLScriptCollection scriptCollection = base.DisplayJavaScriptFromChild();
    try
    {
        //get all the JS to include
        XPArrayList arrJSToInclude = ...Getting this from Varpack is a good idea, and as such, recommended...;
        if(null != arrJSToInclude && arrJSToInclude.GetSize() > 0)
        {
            IXPEnumerator jsEnum = arrJSToInclude.GetEnumerator();
            string src = "";
            while(jsEnum.MoveNext())
            {
                src = (string)jsEnum.GetCurrent();
                if(!"".Equals(src))
                {
                    HTMLScript script = new HTMLScript(HTMLScript.TYPE_JAVASCRIPT);
                    src = src.Replace("pt://images/", ConfigHelper.GetImageServerRootURL(m_asOwner));
                    script.SetSrc(src);
                    log.Debug("Adding JS external file with src: {0}", src);
                    scriptCollection.AddInnerHTMLElement(script);
                }
            }
        }
    } 
    catch (Exception exc)
    {
        log.Error(exc, "An exception occurred while adding the custom javascript to the page head.");
    }
    return scriptCollection;
}

I never recommend customizing directly the portal classes...that way it is a bit simpler if you need to upgrade the portal version: you custom code is not everywhere.

So instead of customizing directly MyPortalDP and GatewayHostedDP, create new custom classes that inherit from those 2...then make sure your custom classes are loaded properly in the related Activity Spaces (PlumtreeAS and GatewayAS)...

Hope that is helpful!

ALUI Webcenter Grid Search: Maintaining the search cluster repository without loss of service

noreply@blogger.com (Fabien Sanglier) — Mon, 11 May 2009 20:33:00 +0000

Sometimes, for maintenance reasons, the cluster repository has to be unavailable for a short amount of time (I.e. the NAS on which it is hosted is patched and need to be restarted, or the central search cluster repository needs to be moved from one server to another, etc…).

The main problem is that search relies heavily on the availability of search cluster repository in order for the portal content to be properly indexed (for example, the cluster registers any new index delta and ensures that these delta are redistributed to each node’s local index, that way the search nodes are always in sync with one another). Unfortunately, I’ve noticed many times that when the cluster becomes not available for as short as a couple of seconds, the search infrastructure does NOT handle this gracefully…

At best, the nodes all go in “read-only” mode automatically (meaning the nodes act only as query service instead of query+index service), corrupting most of the time the process handles of the node being the “indexer” at the time of disconnection (you’ll see “invalid handles” errors in the search status screen for example)…at worse, all the nodes shutdown with a great “out of memory” error. Both scenario is not a good one, since it will not go back in run mode automatically after the disruption is over, and will probably require a overall restart of nodes.

If you need to do this in PROD where uptime is usually a strong requirement, then the idea is to do such an operation without jeopardizing the search capabilities of your portal site(s). Indeed, search being so central to portal, when down, many portlets or components relying on search will be down, and overall service will be pretty degraded.

Fortunately, search comes with a powerful admin utility: cadmin.exe. You can find it on any of your search nodes, usually at the following path:

<pt_search_home>/bin/native/cadmin.exe

Using the tool, you can gracefully put all the search nodes in “read-only” mode before the maintenance operation. Indeed, when the nodes are in read-only mode, each node act as a “disconnected” query service, providing search results solely based on their local index. While in that state, the search cluster can be fully unavailable… and apart from any new content not being indexed, end users will not see any search disruption.

So here are the commands you would want to perform, either manually or in a batch:

cadmin runlevel readonly –-> this puts all the nodes of the cluster in readonly mode
cadmin status –-verbose –-> this give you the status of the cluster. useful to make sure the previous operation worked as expected.

…perform your maintenance operation…

cadmin runlevel run –> put all the nodes on the cluster back in run mode
cadmin status –verbose –> sanity check…

What you would want to do before any search maintenance is perform a search checkpoint (in other word search backup). 2 options: doing it manually using the Admin UI, or using the cadmin tool as follows:

cadmin checkpoint --create

As an extension of this, you can check out the other operations you can perform with the cadmin tool (pretty much everything you can do with the search cluster admin UI, with more power added to it) by entering cadmin –help

Hope that helps. Until next time, take care!

ALUI Publisher: Easy bug fix with portlet templates…

noreply@blogger.com (Fabien Sanglier) — Sun, 15 Mar 2009 20:38:00 +0000

I know ALUI Publisher is on the down slope, but until then, it does not mean I should not share some of my findings…Lately, I found something in publisher that does not make sense to me. It has to do with the creation of portlets based on publisher portlet templates. Let me explain the problem…and then the quick fix…

The problem:

Out of the box, when you create a portlet based on a publisher portlet template (i.e. announcement, news, etc…), there is a screen where you are asked to choose a publisher folder where the portlet publisher content should be…When clicking on the “chose publisher folder”, a publisher tree pops up, and you can pick the publisher folder. The tree picker will only show you the folders where you have “producer” access…until now it makes sense since only the “producer” role and above can create folders in publisher.

But what if a user does have “producer” access to a subfolder Z located in the tree structure at X > Y > Z…but does not have producer access to the parent folder X and Y?? Then the tree picker would simply stop showing the tree at X, hence not showing the subfolder to which the user has actually access to…hence problem.

A perfect (and probably common) use case is where you have various publisher sites (i.e. an intranet site, and internet site, etc…) and within each of these, you have various “community-related” publisher folders… (folders that contain the web content of each community). You will want your community administrators (in the portal) to also have “producer” or “folder administrator” role in their “community-related” publisher folder…but not have these roles in the parent publisher folders…they should have “reader” access on these parent folders…

The solution:

By reverse engineering publisher one more time, I found out that this problem can be fixed really easily…basically the publisher tree picker is opened with the following url:

“../folderpicker_frame.jsp?sid="+sessionId+"&showItemCategory=0&itemId="+parentFolderId+"&isMultiSelect=false&rootIsCheckable=true&minRoleId=12”

What is interesting in this url is the last parameter: minRoleId=12. What it probably means is “show only the folders to the users who have at least a role 12 – producer role – assigned to it.” That’s it, we have our solution…By removing this parameter altogether, the tree will now show the folders to which the user has minimum access to (“reader” access), hence fixing the user case explained above:

A producer in folder X > Y > Z will now be able to browse down to the Z folder, where he will be able to create a folder.

And don’t worry, it does not impact core security at all. If the user tries to select folder X or Y as a container for his “announcement” portlet, he will simply receive an error message saying ”you do not have enough access to create a folder”…so no problem.

Detailed instructions:

Make a backup of the publisher application files \bea\alui\ptcs\6.4\webapp\ptcs.ear and \bea\alui\ptcs\6.4\webapp\ptcs.war (obvious, no?)
Unpack the publisher archive (ptcs.war)

navigate to \bea\alui\ptcs\6.4\webapp folder
create a new dir: ptcs
navigate to that new dir and execute the following jar command: jar –xvf ../ptcs.war

edit the extracted file: ./portlet_packages/portlet_create.jsi
Find the JavaScript functions “ChooseParentFolder” and “NewParentFolder”
Remove the “minRoleId=12” from the “var url = …” line (1st line in each functions)
Still in \bea\alui\ptcs\6.4\webapp\ptcs folder, repackage the war by executing: jar -cvf ptcs.war *
move the newly created war to the \bea\alui\ptcs\6.4\webapp folder: move ptcs.war ../
navigate to \bea\alui\ptcs\6.4\webapp: cd ../
update the ptcs.ear file with the following command: jar -uvf ptcs.ear ptcs.war

That’s it…

Before restarting publisher, clean the temp files from the publisher container folders just to make sure your modification is loaded properly.

\ptcs\6.4\container\tmp\deploy
\ptcs\6.4\container\work\jboss.web\localhost

As usual, do this at your own risk…and test it well before deploying to production :)

New Portal Tools On ALUI Toolbox

noreply@blogger.com (Fabien Sanglier) — Tue, 24 Feb 2009 03:29:00 +0000

In my efforts of improving the ALUI/Webcenter portal, and especially enhancing its admin management capabilities, I created over time a set of utilities that I think could be useful to the ALUI/Webcenter community. In one of my previous post, I already talked about the "PT URL Replace" utility (refer to: http://fsanglier.blogspot.com/2008/01/alui-administration-tool-for.html) which was already on the ALUI Toolbox Google code project as a download only. What I did some days ago is updating the ALUI Toolbox Google project (http://code.google.com/p/alui-toolbox/) with the "PT URL Replace" code, as well as a couple of new applications/utilities:

Portlet Caching Clearer (web portlet - c#)
Object Identifier (web portlet - c#)
Web Service Changer (web portlet - c#)
Page Lister (web portlet - c#)
ALUI Knowledge Directory Security Agent (java app runnable from console and/or scheduled task such as ALUI jobs) – feature download at http://code.google.com/p/alui-toolbox/
Improvements for PT URL Replace utility – http://code.google.com/p/alui-toolbox/

All the above apps use the Server API (for Server API introduction, refer to previous posts:) because the tasks performed would not be possible by simply using the IDK. It has been written for ALUI 6.1.x versions and might not work fully on version 6.5 and above without minor changes (because the server API does change between portal releases). When I get the chance, I will update the Google code project with 2 extra branches that follows the more current portal versions.

The code is released under the GPL license (you can find a copy the the GPL license in the root folder of the apps, or go to http://www.gnu.org/licenses/gpl.html), and off course is provided without any kind of warranty...

I'll go quickly over each of these utilities in order for you to understand why they might be useful.

Portlet Caching Clearer:
In order to maximize performance, each portlet in the portal can have output caching enabled to a particular timeframe. The drawback of such caching mechanism is that the updates performed by content managers are not instantaneously viewable to end users. This portlet that can be added on your "My Page" allows you to clear portlet caching in 3 different ways:

Using the portal tree picker, select the portlet(s) you want to clear.
Using the portal tree picker, select the "Community Page(s)" whose portlets you want to clear. The utility will find all the portlets currently added to the selected pages, and clear the cache for each of them.
Using the portal tree picker, select the "Community(ies)" you want to clear. The utility will find all the portlets currently added to the selected pages of the selected communities, and clear the cache for each of them.

That way the content managers can easily clear caching for a set of pages, communities, or portlets...

Object Identifier:

Let's say that you have a portlet application that identify a portal object in its config file by its UUID (or its ID for that matter)...Now let's say you come back to that portlet 6 month later because you need to perform an improvement and/or fix something...Unless you clearly documented what object corresponds to this ID/UUID (and where it is in the portal), it will not be easy to find it (unless you can easily run a DB query...). This portlet basically answer that needs:

Provide a UUID and it will tell you the corresponding Classid/ObjectID pair + Object Name + Object Location in the Portal Admin Hierarchy.
Provide a Classid/ObjectID pair, and it will tell you the corresponding UUID + Object Name + Object Location in the Portal Admin Hierarchy.

Web Service Changer:

Have you ever noticed that you cannot change the webservice attached to a portlet once you initially picked it and created the portlet? Now let's say that you have a bunch of publisher portlets that are all tied to the same "Publish Content Web Service" object. But all a sudden, you change your mind and decide to have some portlets that should be tied to 2 different "Publish Content Web Service" objects, each one with a different caching timeframe (i.e. a long caching for the content that hardly change, and a shorter caching for the content that changes often). In the portal out of the box, you cannot do that easily, and will probably have to recreate all the portlets and re-attach them to the same publisher content etc...(big pain).

Well this portlet allows you to easily change the "portlet web service" OR the "portlet template" attached to a particular portlet or group of portlets:

Using the portal tree picker, pick the portlet you want to change.
Using the portal tree picker, pick the web service OR portlet template that you want these portlet to be assigned to.
Click Submit...Done.

Page Lister:

This simple utility allows to display as a list of HTML links (simple <a href=""></a> tags) all the Community Pages in the portal that a certain users have access to...

Why doing this? it is a simple way of creating a Sitemap that a web "crawler" (either ALUI web crawler, or Google appliance etc...) could hit in order to be dynamically aware of all the ALUI pages of your site that are accessible to the guest users for example...Or that could also be used as a security monitoring tool in order to verify which pages are accessible to a particular user (i.e. Guest) and ensure it is not a security mistake etc...

ALUI Knowledge Directory Security Agent:

This utility, written in java (we have to think about our linux/unix user base too :) ), was created as a scheduled task agent in order to act as a security cop in the knowledge directory. What it does is go through all the KD folders the agent user has access to, and automatically assign the found cards with their parent folder security.

All you need to provide are:

KD folder ID to start with,
A user ID / password (or session token if you run as a portal "external operation") the agent should impersonate with,
(Optional) CrawlerIDs (if you want to change only the cards that were brought into KD by a specific set of crawlers)

2 main use cases I see for this tool:

Use it as a "cop" background schedule job in order to ensure the security on the cards is always right, based on the security of the folder.
In the event you use a single crawler with various filters that organize your content in various folders. Without this agent, the crawled cards will get the security defined in the crawler's "crawled content permission" section...and that might not be in phase with the security of the various KD folders the content will be organized into (due to filters). If you run this cop security agent after each crawl, then you are all set based on the destination folder, not on the crawler's "crawled content permission" section...

That's it for now. I hope you'll test them out and let me know if you think these are useful or not.
Don’t hesitate to give ideas and/or wish lists of things which would be good to have in the portal...and don't hesitate to share the cool utilities you've done too

I or others will be updating the project every so often so keep informed or you might miss out on some good resources :)

WebCenter Native API Development: Advanced Search Query explained (and applied to WebSite Search)

noreply@blogger.com (Fabien Sanglier) — Sun, 18 Jan 2009 18:26:00 +0000

In his article about "Dot Com Portals: Smart Searching", Jordan Rose already explained really well how to easily implement with Webcenter Interaction an efficient and accurate website-like search (enter a search term, and expect in the results either website pages, or documents within these pages).

To summarize the challenge:

The very powerful WebCenter search component will index everything the user has access to (i.e. web content items, documents, crawled third party websites, etc…) independently from their real presence on the web site pages. For example, the search results would present web content items instead of the website pages where the web content item is displayed through a portlet.

To summarize a bit the solution:

Using WebCenter interaction “web crawler” capability (google like spider that follows links on a page and index its content for future searches) coupled with experience definition features (to hide the part of the page that we don’t want the crawler to index, like the top/left navigation, the banner, etc…), it is easy to actually implement a website-like search with accurate portal page results (Refer to Jordan's blog post:Dot Com Portals: Smart Searching)

But one thing that was not there yet in the solution was: "How can the crawler navigate from page to page" if the navigation is not there? What we did at first (we did not have time to do better) is create manually this HTML file that would contain all the pages of the website, and direct the web crawler to that page, instead of to the root of the public portal website url.

This would work ok, but would require a manual update of this file each time you create a new page...not super practical. Anyway, I finally took the time to improve it, and created this "Page Listing" code that basically render a list of pages located within a specific folder...Basically, you simply create a request with "topfolderid", "includesubfolders", and "openerhost" (http:////PageLinkListing?topfolderid=123&includesubfolders=true&openerhost=yourdomainhost) and the dotnet page will render all the portal page links that correspond to these values.

In this article I'd like to use this example in order to focus on the Native Search API (refer to my previous article about the native API) because the dotnet frontend is pretty simple:

DotNet front end page
Native Portal API
Webcenter search API to query the pages

First as always, it all starts with the native session creation…then, that’s when you can start creating the search request object:

IPTSearchRequest req = m_ptSession.GetSearchRequest();

From there, the PTSearchRequest object allows you to set all sorts of setting that will define the search you want to make. Simply call the SetSettings method. This method takes a setting ID and a value (that can be a string, int, or array of objects). The main problem is the non-documentation of this API (native API is non documented)…but luckily, the setting IDs are all available through the PT_SEARCH_SETTING class, and each name is relativelly straightforward (not always though). Check out the example below that sets the fields to return, specify not to execute best bet and spell check, and the maximum number of results to bring back:

   1: int[] arPropIDs = { PT_INTRINSICS.PT_PROPERTY_OBJECTID, PT_INTRINSICS.PT_PROPERTY_OBJECTNAME, PT_INTRINSICS.PT_PROPERTY_OBJECTSUMMARY};

   2: req.SetSettings(PT_SEARCH_SETTING.PT_SEARCHSETTING_RET_PROPS, arPropIDs);

   3: req.SetSettings(PT_SEARCH_SETTING.PT_SEARCHSETTING_INCLUDE_USUAL_FIELDS, false);

   4: req.SetSettings(PT_SEARCH_SETTING.PT_SEARCHSETTING_KWIC, false);

   5: req.SetSettings(PT_SEARCH_SETTING.PT_SEARCHSETTING_BESTBETS, false);

   6: req.SetSettings(PT_SEARCH_SETTING.PT_SEARCHSETTING_SPELLCHECK, false);

   7: req.SetSettings(PT_SEARCH_SETTING.PT_SEARCHSETTING_SKIPRESULTS, 0);

   8: req.SetSettings(PT_SEARCH_SETTING.PT_SEARCHSETTING_MAXRESULTS, 10000);

You can also specify the admin folders (or KD folders) within which the search should be performed:

   1: req.SetSettings(PT_SEARCH_SETTING.PT_SEARCHSETTING_ADMINFOLDERS, new int[] { adminfolderid });

and the object type the search should be dealing with (here we want to search only community pages, but very similarly to the object type checkboxes in the advanced search interface, you could pick several object type to search for):

   1: req.SetSettings(PT_SEARCH_SETTING.PT_SEARCHSETTING_OBJTYPES, new int[] { PT_CLASSIDS.PT_PAGE_ID });

Finally, you can create all sorts of filters statements that you can add to this search request. It works very similarly to the snapshot query interface: A filter can contain several “Filter Clauses” and each clause can contain several “Filter Statements”. Clauses and Statements can be put together using “OR” or “AND” operations.

Here for this exercise, we will look for objects with ID greater than 230 and name containing “Test”… (kind if useless query…but that’s not the point here…)

   1: // Create a filter for the search request which will "AND" together each filter clause.

   2: IPTFilter ptFilter = PortalObjectsFactory.CreateSearchFilter();

   3: ptFilter.SetOperator(PT_BOOLOPS.PT_BOOLOP_AND);

4:

   5: //Create the clause that will contains the statements we need for the query

   6: IPTPropertyFilterClauses ptFilterClause = (IPTPropertyFilterClauses) ptFilter.GetNewFilterItem(PT_FILTER_ITEM_TYPES.PT_FILTER_ITEM_CLAUSES);

   7: // The filter clause should "AND" each of the statements.

   8: ptFilterClause.SetOperator(PT_BOOLOPS.PT_BOOLOP_AND);

9:

  10: //Statement 1: ObjectID > 230

  11: IPTPropertyFilterStatement statement1 = (IPTPropertyFilterStatement)filter.GetNewFilterItem(PT_FILTER_ITEM_TYPES.PT_FILTER_ITEM_STATEMENT);

  12: statement1.SetOperand(PT_INTRINSICS.PT_PROPERTY_OBJECTID);

  13: statement1.SetOperator(PT_FILTEROPS.PT_FILTEROP_GT);

  14: statement1.SetValue(230);

15:

  16: //Statement 2: Object Name contains the text "Test"

  17: IPTPropertyFilterStatement statement2 = (IPTPropertyFilterStatement) ptFilter.GetNewFilterItem(PT_FILTER_ITEM_TYPES.PT_FILTER_ITEM_STATEMENT);

  18: //search on the name property.

  19: statement2.SetOperator(PT_FILTEROPS.PT_FILTEROP_CONTAINS);

  20: statement2.SetValue("Test");

21:

  22: //add statements to clause

  23: ptFilterClause.AddItem(statement1, ptFilterClause.GetCount());

  24: ptFilterClause.AddItem(statement2, ptFilterClause.GetCount());

25:

  26: //add clause to filter

  27: ptFilter.SetPropertyFilter(ptFilterClause);

As you can see it is very powerful and straightforward, and allows you to perform all sort of searches that fit your needs.

Finally, when you are done with the search parameters and filters, you simply need to execute the query, and get the results back…

   1: IPTSearchQuery query = req.CreateAdvancedQuery(filter);

   2: IPTSearchResponse ptPagesResponse = req.Search(query);

   3: int nResultCount = ptPagesResponse.GetResultsReturned();

   4: for (int nIndex = 0; nIndex < nResultCount; nIndex++) {

   5:     //do something with the data...

   6:     ptPagesResponse.GetFieldsAsInt(nIndex, PT_INTRINSICS.PT_PROPERTY_OBJECTID));

   7:     ptPagesResponse.GetFieldsAsString(nIndex, PT_INTRINSICS.PT_PROPERTY_OBJECTNAME));

   8:     ptPagesResponse.GetFieldsAsString(nIndex, PT_INTRINSICS.PT_PROPERTY_OBJECTSUMMARY));

   9: }

Here it is, I hope you see the endless possibilities you now have using the native search API in your various Native API Utilities (Portlet, Console application, etc…). I will soon post on the ALUI Toolbox google project the integrality of this code plus many other extras. Stay tune, and Happy new year! :)

ALUI / WebCenter Interaction: Introduction to Native API development (Part 1)

noreply@blogger.com (Fabien Sanglier) — Sun, 16 Nov 2008 20:51:00 +0000

Most of the time, creating a remote portlet application using the standard IDK is enough. Indeed, commonly, you just need to access user profile information, portlet ID, page ID etc... or perform simple operations exposed by the IDK PRC API.

But if you hit the limits of the IDK, it is not the end of the road. In the ALUI/Webcenter development documentation, they explain how to perform various UI customizations such as view replacement, PEI development, Activity Spaces creation or extensions, etc... All these are created within the portal application itself...Another option is to create what I call a "Native API Portlet Application". As its name indicates, this portlet application accesses directly the portal API and can perform virtually any operation that the portal can do...very powerful (but non supported :) so make sure your application is well written so that you can change the portal API access layer easily if you upgrade the portal version for example)

First, the 2 main requirements in order to create a Native API application are:

The application needs to be on the same server as Portal, or Automation, or API service
The application needs to reference various portal DLLs (located in /ptportal/6.1/bin/assemblies for dotnet, or /ptportal/6.1/lib/java for java). You don’t need them all. Here is the minimum list:

ptportal\6.1MP1\bin\assemblies\opencache.dll

ptportal\6.1MP1\bin\assemblies\openconfig.dll

ptportal\6.1MP1\bin\assemblies\opencounters.dll

ptportal\6.1MP1\bin\assemblies\openfoundation.dll

ptportal\6.1MP1\bin\assemblies\openhttp.dll

ptportal\6.1MP1\bin\assemblies\openkernel.dll

ptportal\6.1MP1\bin\assemblies\openkernelsearch.dll

ptportal\6.1MP1\bin\assemblies\openkernelsearchimpl.dll

ptportal\6.1MP1\bin\assemblies\openlog-framework.dll

ptportal\6.1MP1\bin\assemblies\openprocman.dll

ptportal\6.1MP1\bin\assemblies\opensharedcache.dll

ptportal\6.1MP1\bin\assemblies\opentempfile.dll

ptportal\6.1MP1\bin\assemblies\openusage.dll

ptportal\6.1MP1\bin\assemblies\openusage-api.dll

ptportal\6.1MP1\bin\assemblies\openusage-impl.dll

ptportal\6.1MP1\bin\assemblies\plumtreeserver.dll

ptportal\6.1MP1\bin\assemblies\portal.dll

ptportal\6.1MP1\bin\assemblies\pthome.dll

ptportal\6.1MP1\bin\assemblies\ptportalobjects.dll

Important note: It is perfectly allowed to also include the IDK Dlls here too... and I find it particularly recommended if you are writing a native API portlet (for instance getting the current login token for example is fairly easy using the IDK API)

Second, you need to create the native session (everything starts from here, really):

1: String strServerConfigDir = ConfigPathResolver.GetOpenConfigPath();

2: IOKContext configContext = OKConfigFactory.createInstance(strServerConfigDir,"portal");

3: PortalObjectsFactory.Init(configContext);

4: IPTSession ptsession = PortalObjectsFactory.CreateSession();

From there you actually need to connect this session with a particular user identity. That way, the session object will be aware of your identity and especially the security and permission associated with your identity. In other words, even if you are using the native API, you cannot do more than you are allowed to.

To connect, 2 main options:

Use the login token that you can simply get from the IDK API (that is prefered option if you can)

1: String loginToken = m_portletRequest.GetLoginToken(); //IDK call

2: ptsession.Reconnect(loginToken);

Use explicit Username (or user ID) / Password to connect.

1: String username = "myusername";

2: String pwd = "mypassword";

3: ptsession.Connect(username, pwd, null);

Third (and Final), you access the right ObjectManager Class depending on which type of portal object you want to interact with.

When the session is created (that was pretty easy, right?), that is when you can actually start interacting with the portal internals...and a majority of actions goes through the PTObjectManager objects (IPTObjectManager interface). For instance:

1: IPTCommunityManager ptCommManager = ptSession.GetCommunities(); //for community objects

2: IPTPageManager ptPageManager = ptSession.GetPages(); //for community page objects

3: IPTObjectManager ptGadgetManager = ptSession.GetGadgets(); //for portlet objects

4: //etc...

You noticed that for portlets, there is not a IPTGadgetManager insterface....that’s ok as all the "Manager" interfaces are children of the base IPTObjectManager interface.

Although the vast majority of manager objects are accessible through a direct ptsession.Get() call, you can also get the right manager using the generic call below (using the classID of the object type you want)

1: IPTObjectManager ptCrawlerManager = ptSession.GetObjectManagers(PT_CLASSIDS.PT_CRAWLER_ID);

2: //using     the class id of the object type you want

From there, the freedom is yours... and various operations will be available depending on the manager you called. One call for instance that all managers have is the open object:

1: ptObjectManager.Open(objectid, lockObject);

With this call, you get directly access a particular object in the portal, and interact with it as if you were in the UI

In the next article, I will show you how I package these API call in a standalone library to minimize as much as possible any API call within the presentation layer code...

ALUI Publisher - Part 3: No Redirect Bug Fix of Bug Fix :)

noreply@blogger.com (Fabien Sanglier) — Tue, 28 Oct 2008 03:02:00 +0000

I have been relatively lazy in regards of my blog lately, and I have plenty of articles that are stacking up...But in the meantime, I thought this one is pretty urgent.

In one of my previous article, ALUI Publisher - Part 2: Increase Performance by enabling REAL Caching - No Redirect Bug Fix, I was explaining how to fix the Publisher published_content_noredirect.jsp (see in that serie the benefits of using this instead of published_content_redirect.jsp).

Well, I found out a small little bug on my part, and it is definitely worth fixing if you have not done so already. In my corrective code, I was trimming the content string from its end spaces (just to optimized the HTML output)...and I did not think of the likely negative effects, such as if the buffered content does finish on a meaningful space character (such as a sentence separator etc...)

So the updated code with bug fix is: (specifically the trim that is removed)

//if there is content, forward to the requesting client
int buffersize = 2000;
int charread = 0;
char[] content;

//read until no byte is found in the input stream because request content length is no reliable
// UTF-8 is necessary
BufferedReader bisr = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));

do {
               content = new char[buffersize];
               charread = bisr.read(content);
               out.write(new String(content).trim());
} while(charread > -1);

bisr.close();
bisr = null;
content = null;

Better to find it late than never! :)

The code should be updated on the famous (yeah right...) ALUI Toolbox Google project (or should be soon)

So Long.

ALUI Enhancement: Get/Set All Preferences from Javascript

noreply@blogger.com (Fabien Sanglier) — Mon, 29 Sep 2008 05:02:00 +0000

As you probably already know, ALUI Portal personalization features are mostly guarantied by the possibility of setting/getting "preferences" with various scopes. Not to myself extend too much on this concept, you can refer to the ALUI Development Documentation (Link HERE) to have further explanations. While you read this documentation, you'll learn that preferences can be set and retrieved through:

the IDK API (most commonly),
the server API (much less common since Server API should mostly be used only if the IDK cannot fulfill your needs), or
the Javascript API (but only for preferences of scope "session")

What is usually done consists in building a JAVA or Dotnet application that uses the IDK API to get/set preferences within a portlet, allowing for the presentation or behavior of the portlet to change based on the preference chosen.

But as you build ALUI intranet or even Internet websites, you often realize that most of the portlets are either content-specific with some kind of personalization (i.e. a link showing only if this preference has been set for that user...), or have a pretty simple behavior and presentation.

Would not that be cool to be able to use "Publisher" for example to build those simple portlets? In most cases, with the power of the ALUI Portal Adaptive Tags coupled with the Publisher PCS Tags (this could apply to any other Web Content Management Systems, but why not use Publisher as an example since it is still alive :) ), you could build such simple portlets without the need for programming a portlet in JAVA or Dotnet...

But what about preferences? How could you set and get preferences, since they are available only through an API that can be used only using those programming languages? Well, ALUI has already thought it through, and integrated within their Javascript API the possibility to set/get "Session" preferences. That was great thinking indeed. But I kept (and still keep) asking myself: Why having developed this idea only half way through?? Why allowing to only set/get "Session" preferences through Javascript? All I know is that there is no reason why not...and I just wanted to share this essential feature.

Javascript Session preference: How does this work?

So first, let's understand how the Javascript preference access works. Basically an activity space has been developed and its code is found in the following package: com.plumtree.portalpages.browsing.sessionprefs. As the portal follows a strict MVC (Model View Controller) architecture, an activity space is mostly composed of a "Control" class (implementing IControl interface), a "Model" Class (implementing IModel interface) and a "Presentation" class (implenting IDisplayPage interface). You can refer to the "ALUI Portal Customization Package" to see the code of this sessionPrefs activity space. To request the activity space functionnality, simply create a HTTP request with the right parameters, like this:

http://your.host.domain.com/portal/server.pt?space=SessionPrefs&control=SessionPrefs&action=getprefs&_preferencename=

http://your.host.domain.com/portal/server.pt?space=SessionPrefs&control=SessionPrefs&action=setprefs&_preferencename=preferencervalue

From there, all there is to do is to allow that request to be called from Javascript, through an AJAX HTTPWebRequest...The javascript methods getSession() and SetSession() which initiate that AJAX HTTPWebRequest will use the PTPortalContext javascript object defined on every page to get the right ActivitySpace url as follows:

PTPortalContext.GET_SESSION_PREFS_URL = 'http://localhost/portal/server.pt?space=SessionPrefs&control=SessionPrefs&action=getprefs';
PTPortalContext.SET_SESSION_PREFS_URL = 'http://localhost/portal/server.pt?space=SessionPrefs&control=SessionPrefs&action=setprefs';

...and that's it...Plumtree/BEA delivered your ALUI Javascript-Enabled "Session" Preference behavior.

Javascript preference of "All Type": How does this work?

All I did was extending on this principle in order to make all type of preferences available through Javascript, and as you understood now, through the behind-the-scene Preference Activity Space. Since I post the code on "ALUI Toolbox Google Project", I will pass on the very details of the code...

The main outline is:

Create a new Activity Space called "PortalPrefs" instead of SessionPrefs
Update the PortalPrefControl class to take into account a new parameter "type" which allow user to specify the type of preference you want to get/set (possible values are: portlet, admin, session, user, community, communityportlet).
Update the PortalPrefsModel class with the corresponding Preference Getter/Setter methods for each preference type (or scope)

Now, after installing this new code, the urls to access the AS are (differences in bold)

http://your.host.domain.com/portal/server.pt?space=PortalPrefs&control=PortalPrefs&action=getprefs&type=PreferenceType&_preferencename=

http://your.host.domain.com/portal/server.pt?space=PortalPrefs&control=PortalPrefs&action=setprefs&type=PreferenceType&_preferencename=preferencervalue

Then you have the choice as far as Javascript is concerned: either you create your own layer that performs the call to this new activity space, or you plug this new bahavior into the already existing Javascript framework, without any modification to the framework itself.

I personally chose the later, since I always try not to modify the portal existing libraries (just to allow for easier future upgrades of those libraries). All you have to do is to make sure to override on your portal page the PTPortalContext URL properties with the new AS URL end point BEFORE you call the already existing GetSession()/SetSession() javascript methods. To do that, you could simply create the following Facade JS methods to be called from your portlets:

GetPortalPreference(prefType, prefname){

PTPortalContext.GET_SESSION_PREFS_URL = 'http://your.host.domain.com/portal/server.pt?space=PortalPrefs&control=PortalPrefs&action=getprefs&type=' + prefType;

GetSession(prefname); (this is the call to the already existing JS framework for session preference)

}

SetPortalPreference(prefType, prefname, prefvalue){

PTPortalContext.SET_SESSION_PREFS_URL = 'http://your.host.domain.com/portal/server.pt?space=PortalPrefs&control=PortalPrefs&action=setprefs&type=' + prefType;

SetSession(prefname); (this is the call to the already existing JS framework for session preference)

}

</script>

That's it! Feel free to browse the code (only JAVA version for now) at the Google Code Project I created (I also created for you the JAR that contains this code...that way, you can easily install and test this on your portal) and please let me know your thoughts! Hopefully, this will be integrated in the future releases of ALUI.

ALUI Best Practice - Don't Hardcode ObjectIDs: Use UUIDs Instead

noreply@blogger.com (Fabien Sanglier) — Tue, 05 Aug 2008 02:28:00 +0000

In ALUI, there are 2 ways to identify uniquely an object:

ClassID (the type of object) + ObjectID (The ID of the object within the classID family)
Global UUID (unique ID throughout the environment)

The ClassID/ObjectID combination is used throughout the Portal API to query/open/manage the ALUI objects, as well as navigate to communities and pages. The ALUI ADAPTIVE TAGS are no exception: you will notice that they require ObjectID / ClassID to perform their tasks, like for example the opener tag:

<pt:standard.openerlink pt:objectid="219" pt:classid="514" pt:mode="2" target="myWindow">view community page</pt:standard.openerlink>

The main problem is not in the fact the ObjectID / ClassID is a bad way to identify an object, but in the fact that the ObjectID WILL NOT NECESSARILY (very improbable actually) be the same when you migrate objects from one environment to another. And that's where it can hurt...

Indeed good practice is to test out your creation in DEV / TEST / STAGING etc... and then migrate it using the ALUI migration tool. Since objectIDs will be different after migration in the new environment, all the objectIDs used in Adaptive Tags will have to be changed...hassle indeed.

Fortunately, UUID does NOT change with migration (or very improbable) from environments to environments. So that would seem a good option if you need to hardcode IDs, especially in Adaptive Tags, and I'd recommend that option everywhere you can.

In order to make it possible, I built a Adaptive Tags that does just this: Transform a UUID into its corresponding ClassID/ObjectID pair. All you need to do is use that Adaptive Tag before using the opener link tag for example. The object and class IDs are stored in shared memory (adaptive tag framework) with the specified scope and can be reused with any other tags that require objectID / ClassID.

<pt:taglibname.convertuuidtoid pt:uuid="{UUID}" pt:objectid="objectIDKey1" pt:classid="classIDKey1" pt:scope="portlet request" />

<pt:standard.openerlink pt:objectid="$objectIDKey1" pt:classid="$classIDKey1" pt:mode="2" target="myWindow">view community page</pt:standard.openerlink>

What's even better is that you can use this tag even for your remote portlet applications, like any other tags.

The code to change a UUID to the ObjectID / ClassID pair is fairly simple. Just get the Migration manager and call the convert method UUIDToObjectID to get the job done. I created a helper method that package it altogether:

public Object[] getClassObjectID(String uuid){
        Object[] oClassIDObjectID = null;
        if (null != uuid) {
            try {
                IPTMigrationManager oPTMigrationMgr = (IPTMigrationManager) oPTSession
                        .OpenGlobalObject(
                                PT_GLOBALOBJECTS.PT_GLOBAL_MIGRATION_MANAGER,
                                false);
                oClassIDObjectID = oPTMigrationMgr.UUIDToObjectID(uuid);
            } catch (Exception exc) {
                oClassIDObjectID = null;
            }
        }
        return oClassIDObjectID;
    }

The first item in the returned array is the ClassID (oClassIDObjectID[0]), the second item is the object ID (oClassIDObjectID[1]).

You can download the code on my newly created subversion project (http://alui-toolbox.googlecode.com) on Google code (I'll be updating/maintaining this as I see fit - feel free to suggest at will :)):

svn checkout http://alui-toolbox.googlecode.com/svn/trunk/ alui-toolbox-read-only

No more mocking around with IDs during migration :)

Hope that helps!

Inside ALUI Grid Search: Redundancy Bug (6.1 on window at least)

noreply@blogger.com (Fabien Sanglier) — Thu, 24 Jul 2008 03:07:00 +0000

With ALUI 6.1, BEA introduced a completely revamped search component for ALUI, allowing for better redundancy and better throughput: Grid Search. The main advantages of that new search component are:

Multiple search nodes to provide redundancy for serving search requests.
Search index can be split in multiple partitions, each attached to various search nodes., to increase throughput.

Every nodes on the same partition automatically replicate locally their search index to guaranty redundancy and performance.

Although there is capability for multiple nodes that guaranty redundancy, all the nodes need to access a central "cluster" data repository located somewhere on the network (through file share). It is located by default at <ALUI_HOME>/ptsearchserver/6.1/cluster. What is usually done is to share that folder (simple network share if you are on windows) and set up the other nodes to access that share as their cluster repository. This cluster repository holds the cluster information (nodes and partitions info) and the multiple search checkpoints that allow for search index backup.

One main problem that I personally experienced with that design consists in the fact that this cluster repository represent a single point of failure... if the cluster share is suddenly not available (hard disk, server, or network failure), all the nodes are not able to talk to the cluster and there might be problems happening.

And actually, a huge problem occurs in that case: if the cluster share is not available, all the nodes are suddenly experiencing an "Out Of Memory" exception and shutting down abruptly. Thus, although you deployed multiple nodes and partitions, if the cluster share is down, your search architecture is...down.

It is pretty easy to test (at least I successfully reproduced the bug on ALUI 6.1 MP1 Patch 1 on windows server 2003): have your nodes all running, and simply remove the share from your cluster folder...all your nodes will go down (apart from the one that accesses the share locally if the cluster share is installed on the same server as one of the nodes)

2 options from there:

make sure the share is never down (windows clustering, redundant NAS cluster, or polyserve technologies)
install that critical fix from BEA that fixes this bug

If you don't have an infrastructure that provides the first expensive option, you might want to look seriously into the 2nd one...and contact your sales rep asap. Basically, the critical fix allow for the nodes to continue serving requests even if the cluster share is no longer available. All the nodes switch automatically in read only mode without the "out of memory" exception that was occurring before.

Although it is much better, some problems are still present with that critical fix. When in read-only mode, the nodes are no longer indexing new content...your search index is then blocked at the point in time when the cluster share did actually go down, and any new object or document will not show up in the search as long as the cluster share is not restored. The second problem is that the nodes will NOT automatically roll back to read/write mode whenever the cluster is available again. It will require a manual restart.

But compared to a total shut down of search, these problems seem less important indeed!

I am not 100% sure this fix has been pushed to ALUI 6.5 but I sure hope so. And by fix, I am talking about a total fix including auto rollback to "normal" mode when share is available anew, or even allowing for TOTAL continuity of service when this share goes down...

Please let me know (leave comment) if you have that information on 6.5, or if you reproduce this with other versions of the portal.

ALUI Tool: URL (or text) Migration within Publisher Items

noreply@blogger.com (Fabien Sanglier) — Wed, 16 Jul 2008 03:06:00 +0000

Following my previous article "ALUI Administration Tool for Environment Refresh: String Replacing for URLS" talking about migration between environments, here is an extra piece that you might find very useful (I surely use it all the time)

Basically, as explained in the previous article, it is common to have different DNS aliases set up per environment...I.e for publisher remote server, you could have:

http://publisher.domain.com for production
http://publisher-stg.domain.com for staging
http://publisher-dev.domain.com for development

Similarly, the publish browsing URL is not an exception to this rule:

http://publisher-content.domain.com/publish for production

http://publisher-content-stg.domain.com/publish for staging

http://publisher-content-dev.domain.com/publish for development

When you add an image or a link in the free text editor of content items in publisher, it will most of the time create an absolute URL to that resource...thus you can imagine that there will be a lot of DNS aliases within a lot of publisher items throughout the environment.

What happen when you migrate the publisher DB from one environment to another? Well you will have a lot of DEV dns aliases within your Staging environment (in case of a DEV promotion to Stage); or a lot of production DNS aliases within your dev environment in the case of a production refresh to DEV.

In my previous article "ALUI Administration Tool for Environment Refresh: String Replacing for URLS", I was mostly talking about migrating URL within portal objects, but nothing really about migrating urls within publisher items.

Thus, I created some DB scripts (SQL Server only for now) that do just that...

puburls-PTCSDIRECTORY-nvarchar-replace.sql: Script to change a particular string within the PUBLISHEDTRANSFERURL and PUBLISHEDURL columns (which is mapped out in the DB to a column of type VARCHAR)
puburls-PTCSVALUE-ntext-replace.sql and puburls-PTCSVALUEA-ntext-replace.sql: Scripts to change a particular string within the "long text" property of a publisher item (which is mapped out in the DB to a column of type TEXT)

PCSVALUES.LONGVALUE (hosting the long text of the currently published item)
PCSVALUESA.LONGVALUE (hosting the long text values of all the previous versions of the item)

For the first script, PUBLISHEDTRANSFERURL and PUBLISHEDURL columns are of type VARCHAR and thus it is easy to replace a string within those columns using the REPLACE MS SQL Function. Thus, a simple SQL statement is good here.

The main challenge was really with the 2nd scripts...indeed, within a column of type TEXT, the SQL "REPLACE" function cannot be used...The workaround is to use the PATINDEX and UPDATETEXT functions within a Transact-SQL (T-SQL) script. To give the credit to to the right person, I adapted a script that I found at ASP FAQ - How do I handle REPLACE() within an NTEXT column in SQL Server?

DISCLAIMER: ALTHOUGH I PERSONNALY USE THIS SCRIPT ALL THE TIME, THERE IS NO GUARANTY; SO USE THIS TOOL AT YOUR OWN RISK blah blah blah AND USE IT ONLY IF YOU ARE PROFFICIENT ENOUGH WITH ALUI PORTAL TECHNOLOGIES.

Attached is the zip file package that contains the 3 scripts:

Don't forget to change the string to look for, and the string to replace it with

puburls-PTCSDIRECTORY-nvarchar-replace.sql

UPDATE [dbo].[PCSDIRECTORY]
SET 
[PUBLISHEDTRANSFERURL]=REPLACE([PUBLISHEDTRANSFERURL],'-DEV.DOMAIN.COM','-TST.DOMAIN.COM'), 
[PUBLISHEDURL]=REPLACE([PUBLISHEDURL],'-DEV.DOMAIN.COM','-TST.DOMAIN.COM')
WHERE 
publishedtransferurl like '%-DEV.DOMAIN.COM%' 
or publishedurl like '%-DEV.DOMAIN.COM%'

puburls-PTCSVALUE-ntext-replace.sql and puburls-PTCSVALUEA-ntext-replace.sql

SET @oldString = N'por-pubcontent-dev.domain.com'; -- remove N 
SET @newString = N'por-pubcontent-tst.domain.com'; -- remove N

That's it! Let me know if you find it as useful as I do! Enjoy!!

ALUI Publisher - Part 2: Increase Performance by enabling REAL Caching - No Redirect Bug Fix

noreply@blogger.com (Fabien Sanglier) — Mon, 30 Jun 2008 18:59:00 +0000

Previously (http://fsanglier.blogspot.com/2008/06/alui-publisher-part-2-increase.html) , I've been talking (and proving I hope) that using a "no redirect" mechanism for serving published content from publisher is the best option to enable portal caching. Publisher 6.4 offers already such a possibility (although not publicized a lot): using in the portal Published Content Web Service object published_content_noredirect.jsp instead of the standard published_content_redirect.jsp.

Unfortunately, if you start using this, you are going to start seeing a weird behavior: the publish content is getting truncated in some special cases...and this is due to the way the JSP has been coded. Several options for you: either you wait for a Critical fix to be issued to you by BEA (i am not aware of one yet), or you upgrade to ALUI 6.5 (I hear that this has been fixed in 6.5...have not verified though), or you simply do it yourself, as this is a simple fix to implement (ultimately, that might be the same type of code that would be issued by a CF I imagine)

By looking at the JSP within the publisher web application archive (ptcs.war - explode the war using jar command), we can see what's wrong and why the content is truncated in some case:

HttpURLConnection conn = (HttpURLConnection)url.openConnection();
                  
// make the request
conn.connect();

//read the content length
int contentLength = conn.getContentLength();

//if there is content, forward to the requesting client
if( contentLength > 0 ){
  // UTF-8 is necessary
  InputStreamReader isr = new InputStreamReader(conn.getInputStream(), "UTF-8");
  char[] content = new char[contentLength];
  isr.read(content);
  isr.close();
  out.write(content);
}

As you can see, an HTTP GET request is made, and the content length of the response is gotten from the "getContentLength()" method. This call is going to get the content length number fro mthe response header rather than actually count all the bytes that are contained in the response content. Thus, since the code base itself on this number to output the content to the JSP output stream (see above: char array of length equal to contentlength), the content will indeed be truncated if the contentlength number is not correct...

A simple correction (and more robust code) is actually to make sure ALL the content is pushed to the output stream, independently from the contentlength number returned by the response header. Here is my code below that fixes that issue, and also increase performance by using the preferred BufferedReader wrapper class instead of the bare InputStreamReader:


------EDITED 3/12/2009--------
BufferedReader bisr = null;
            try {
                bisr = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));
                String line;
                while ( (line = bisr.readLine( ) ) != null ) {
                    out.println(line);
                }
            }
            catch(Exception exc){
                throw exc; //to be caught by the global try catch
            } finally {
                if(bisr != null)
                    bisr.close();
                bisr = null;
            }
            return;
------END EDITED 3/12/2009--------

Basically, the code will read by chunks of 2000 chars (this is a chunk size that I think is appropriate) the entirety of the content until the last character...and write it all to the output stream...This does not rely on the contentlength at all, and thus is more reliable and robust.

After changing the published_content_noredirect.jsp as above, you can repackage the ptcs.war with the new corrected JSP (within the root of the previously extracted ptcs.war folder, run jar -cvf ptcs.war * command) and redeploy to ALL redirector and publisher instances...

Voila, you have your perfect solution for ALUI 6.1 and Publisher 6.4 (and previous versions too).

ALUI Publisher - Part 2: Increase Performance by enabling REAL Caching

noreply@blogger.com (Fabien Sanglier) — Mon, 30 Jun 2008 17:21:00 +0000

In my previous post http://fsanglier.blogspot.com/2008/02/alui-publisher-increase-performance.html(man, already couple of month ago...I know I've been sucked into a black hole since then :) ) I was talking about how to best design a scalable and redundant ALUI Publisher architecture. But what I had not pushed enough in the last article was performance.

In my opinion, the Publisher standard behavior for serving content has some performance flaw (at least in ALUI 6.1 - Publisher 6.4) related to caching... Since there are ways around this (that's what we do, right?), I think you might benefit from this a lot, and in my last implementation, the change explained below increased performance under load to complete new levels (in the meantime reducing DB requests and DB and publisher redirector CPU utilization)...So here it goes...

As I explained in previous post, each published content portlet within a portal page are going to make a request to the Publish Content Redirector component, and particularly the JSP page in charge of performing the redirect: published_content_redirect.jsp (you can see that defined in the Published Content portal web service object). As the name says, this Java Server Page (JSP) performs a 302 redirect to the published content item (as seen in previous post, this location should be served by your favorite web server, apache or IIS for example...). But BEFORE making this redirect, it must know where to redirect to... so for that, the code makes a DB request to the publisher Database in order to get the browsing path to the published content item (passing the publisher content item ID that was saved when you created your published content portlet earlier).

Ok so here is the first flaw I was taking about in previous paragraph: To be able to see some content through the publisher portlets within the portal page, you can see from the above explanation that multiple calls to the publisher DB will be made. Let's say we have 5 publisher portlets on the page (not uncommon), for each page rendering for 1 user, 5 DB calls will be made to the publisher DB (in addition of multiple other DB calls for portal and analytics). If we are now talking about thousands of users, we are talking about too many DB calls to simply see some content that does not change often. This has an unnecessary impact on DB load of your infrastructure, and will increase the page load consequently since DB calls are inherently slower that simple content rendering...

While reading, some of view are already thinking for a good reason: CACHING! Yeah, indeed, caching is the secret to scaling and performance (not always necessary to take out the big bucks and supercharge even more the DB infrastructure). And great for us, the portal offers a great out-of-the-box caching capability within the web service object: simply set the minimum caching to 2 hours, max to 20 days, and normally, you would think that the portal should simply cache the published content for that amount of time...removing the need to call the published_content_redirect.jsp altogether, and thus the need to make a DB call!! ...but it does not happen this way. You don't believe me? Enable access logging on publish content redirector components (un-comment "Access logger" section within <ALUI HOME>\ptcs\6.4\container\deploy\jbossweb-tomcat50.sar\server.xml) and you will clearly see that even though your page contains only published content portlets that should be cached, the published_content_redirect.jsp is still constantly called...and thus caching is not really....caching.

Why does this happen? It is because of the redirect mechanism for serving content...From www.w3.org, 302 is explained this way: "The requested resource resides temporarily under a different URI. Since the redirection might be altered on occasion, the client SHOULD continue to use the Request-URI for future requests. This response is only cacheable if indicated by a Cache-Control or Expires header field." Thus, the portal is doing is job perfectly (it acts as a client here) and will never cache a response with a temporary redirect 302 status code.

Ok so what can make it better? Changing the redirect mechanism to something that does not redirect...Well, using this other JSP actually already present and developed within ALUI 6.4 (I have not verified if present in earlier versions of publisher): published_content_noredirect.jsp. Instead of issuing a redirect to the published content, this jsp performs an HTTPGetRequest to it and write its content to the JSP output stream. The response code is now a simple 200 OK ("The request has succeeded" - www.w3.org) that can be cached by the portal. To enable this, simply change the HTTP url of the Published Content Web Service Object to be published_content_noredirect.jsp instead of published_content_redirect.jsp. Of course, check out the publisher redirector access logs to see the dramatic difference...under load, you will initially see a bunch of requests to published_content_noredirect.jsp, but very very fast, the access log becomes silent, all the content being really cached by portal...

Result?? You can increase the load even more, and the page response satys the same (or is even better), and the DB utilization is not altered by that load...you simply have a site that is so much more performant. Our initial results showed that under constant intensive load (with and without the change) the publisher infrastructure would not crash anymore, the CPU usage of both DB and publisher redirectors would be consequently diminished, the number of DB requests would drop, and the load could actually be increased to new levels without loss of performance and functionality, thanks to caching.

Since this post is already long, I am going to stop here for now...Please read next post (http://fsanglier.blogspot.com/2008/06/alui-publisher-part-2-increase_30.html) to understand the second flaw: the published_content_noredirect.jsp has a truncation bug...that is fixable of course :)

So long!

Technorati Tags: Aqualogic,ALUI,Publisher,Web CMS,ALUI Architecture

ALUI Publisher: Increase Performance, Scalability and Availability

noreply@blogger.com (Fabien Sanglier) — Wed, 27 Feb 2008 01:00:00 +0000

ALUI publisher is the component that provides out of the box Web Content Management for the ALUI portal. Depending on the nature of your portal, Publisher can be an important or even critical part of your portal infrastructure (ALUI-powered Internet sites for example, or intranet with a lot of static content displayed through portlets).

So if this component is really critical, the architecture design for that component must provide performance, scalability and redundancy. This article is meant to show you how to design Publisher Architecture to maximize performance, scalability and redundancy, and ultimately provide the best experience to the end user.

Published Content portlets - How it works:

First, let's lay out all the key components and understand the request flow for each Publish Content Portlet on a single portal page:

It is important to reiterate that this flow is not per portal page, but per portlet. In other words, a portal page with 4 published content portlets will initiate 4 simultaneous request flows like the one above. From that, you can understand that depending on the load of these portal pages, the number of requests to the redirector and the publish content services could easily skyrocket, which is a strong proponent of clustering these components.

The redirector, as its name states, will simply redirect the request to the right content url, based on the content ID that is given. Since this component actually makes more intensive DB calls and is accessed for every pub content requests (when content is not cached by the portlet of course), we definitely would want to cluster it and allow for future addition to that cluster when the load increase.

The published content piece is a very simple "static web content serving solution". Although JBOSS or any other application servers are able to serve static content, none are as performant as true web servers optimized for the task: Amongst others, Apache or IIS are good possibilities.

The problems:

The main problem consist in the fact that the installer does not provide any tip to help you achieve that. By default, Publisher installer provides 2 options: Full Component (it means publisher engine, workflow engine, and the embedded redirector) or Publish Content Redirector Only.

A couple of common shortfall if out of box design is followed:

Publisher engine (explorer) cannot be clustered.
Publisher internal application server (JBOSS) is serving publisher, workflow, redirection, and published content.
Publish Content Redirector standalone is not optimized for serving static content (it is used mainly to redirect to the actual publish content url, based on the requested publisher item ID)

The Solutions:

Ultimately, what we want to do is decouple each component from each other in order to scale it independently from the others. Thus, what we want to improve through architecture design consist in:

Publisher and workflow engine set up in failover mode. (Hot-Warm or Hot-Cold failover)
Publish Content Redirector Load balanced using Hardware (F5 BigIP, Cisco LB) or software load balancer.
Publish Content Served by appropriate web server (apache or IIS), optimized to best deliver static content (best performance)...instead of the publisher internal jboss app server (better optimized for dynamic J2EE content delivery).
Publish Content Web Server (apache or IIS) to be load balanced using Hardware (F5 BigIP, Cisco LB) or software load balancer.

Following that design, the redirector can be scaled independently from the publish content serving mechanism, as well as from the publisher admin. And also, if the server that hosts publisher is down, the redirector services and publish content services are still able to provide content without any problem. Similarly, since the redirectors and pub content web servers are load balanced, they are fully redundant, providing maximum availability of static content on your ALUI powered internet site for example.

So how do we tie everything together? Here is an overview of tasks that corresponds to what we talked about until now: (here using 4 servers in my example...could use more or less depending on what hardware is available to you...with a minimum of 2 servers of course.)

Install full Publisher on Server A (contains redirector capability by default)
Import the publisher objects in the portal (pte file)
Install redirector component on Server B
Install Apache/IIS on server C and D
Create the 3 load balancing pools, 1 for Publisher (useful for HOT-WARM failover), 1 for Redirector, 1 for publish content web server.
In publisher explorer, set the "publishing target" to a fault tolerant and fully redundant file share. (clustered NAS for example)
Still in "publishing target" screen, set the browsing url to make sure it access the publish content web server pool (and as such access the Apache/IIS web servers)
Setup Apache/IIS web servers on C and D in order to expose the published content that is at the "publishing target" set up previously
We need to modify the publisher portal objects in order to make them access the correct load balancing pools
1. Change PubContent Web Service Url to make it access the redirector load balancing pool.
2. Also, change accordingly the gateway prefixes of the PubContent Web Service, especially the ones related to published content (publish/preview)
3. Make sure the publisher url is actually going to the LB instead of going directly to publisher.

One last thing you might want to do to still increase performance: Make sure you publish the images to the image server instead of the publish content repository. (Since the image server is publicly accessible from the internet, do this only if the images you are working with are not exposing information that should not be publicly accessible!). This is explained by the fact that serving binary content through the portal gateway component is much slower than serving the content directly to end user.

Et Voila! When you are done configuring all that, you have a publisher infrastructure that is completely redundant in every aspects, offers maximum scalability for each critical components, as well as maximum performance for serving static content.

Technorati Tags: ALUI Portal,Architecture,ALUI Component,ALUI Publisher,Best Practices

ALUI Administration Tool for Environment Refresh: String Replacing for URLS

noreply@blogger.com (Fabien Sanglier) — Tue, 29 Jan 2008 00:58:00 +0000

Any company that owns a portal infrastructure such as ALUI will usually also plan for extra portal environments for troubleshooting and test purposes. Any portlet developments, portal customizations, or administrative changes will be implemented and tested in those environments. This is very good practice indeed. But you will quickly notice something happening: Your production portal is no longer in sync with your other staging / QA / Test environments. Although not necessarily a huge deal in the immediate future, that could become a problem when your test environment is really too different from the production one: Imagine the test team trying to execute a test plan in Staging environment, but half the production communities are simply not present in the test environment...etc...etc...etc...

I am sure you figured where I am going: Sooner or later, a clean refresh of your tests environments will be needed. Such a refresh is best done by a full database copy from production to your test environment (ALUI built-in export / import functionality is not suited for a large amount of objects, and thus is totally inappropriate and not recommended in the present case)

Besides restoring all the appropriate repositories (documents, published content, search indexes etc...), a major problem remains: All the various service' urls specified in the various ALUI objects (Remote servers, web services, KD cards, server urls, etc...) are no longer accurate in the refreshed test environments (of course: those urls are the production ones). Thus your test environment is far from working yet.

2 possibilities from here (1 good and 1 less good):

You use host files in your test environments. These hosts files will direct the same production urls to the appropriate test servers (not good in my point of view because it is dangerous: if the hosts files is removed, there is a huge risk of having your test environment accessing production - thus not good :) )
You use environment-specific DNS entries for each services. (Much better because no risk of inter-environment communication)

First, it is much better practice to call each service through DNS name rather than directly to through the server name. And secondly, it makes it much easier when it comes to environment refresh.

Why? Because all the DNS entries follow a global "intelligent" pattern (i.e. PUBLISHER.COMPANY.COM / PUBLISHER-TEST.COMPANY.COM / PUBLISHER-DEV.COMPANY.COM / etc...), it is much easier to create a tool (or a DB query for those DB gurus out there) that will allow for automatic replace of these DNS names.

That's where I wanted to lead you:

I created a tool that allow just that: for all the portal objects that contain URLs, the tool can replace a specific pattern with another one that you define. Although I could have created a set of DB queries that could do the job (I actually started to do that when I had to do my first environment refresh), I realized that a utility written in JAVA (most portable language) and using ALUI server API would guaranty portability, extensibility and reusability. The main advantage is that it is completely independent from backend technologies, and can work with any ALUI portal (.NET or JAVA), as well as any database (MS SQL or Oracle). It has been tested with JRE 1.4 and used by me on all G6 portal versions (until 6.1 MP1 included)

DISCLAIMER: ALTHOUGH I PERSONNALY USE THIS TOOL, THERE IS NO GUARANTY; SO USE THIS TOOL AT YOUR OWN RISK blah blah blah AND USE IT ONLY IF YOU ARE PROFFICIENT ENOUGH WITH ALUI PORTAL TECHNOLOGIES.

2 smart moves if you are not the easily scared type of person: First, make a DB backup of the test environment you are going to refresh, and second, test it in your local environment first, and see with your own eyes that it works perfectly well :)

For the moment, this migration utility only updates the main portal objects that contain urls:
    -Remote servers
    -Web Services
    -Portal Settings

When I have time, I'll improve the utility and include publisher content items (especially urls embedded in them) in the list of migration objects.

Shell scripts (.bat and .sh) are created in order to facilitate and secure the usage of this utility. The user must provide several key information for the program to run:
    -Application name (by default: portal)
    -Administration username (must be an ALUI local database user)
    -Username password
    -Pattern to look for (any valid regular expression)
    -Replacement String
    -Optional: Debug mode? (if true, details of the parsed objects is output to screen)

The executable JAR along the appropriate launch scripts (bat for windows, sh for unix/linux) must be copied into the <PT_HOME>/bin folder in order to access the required libraries (otherwise the shell script -especially the java classpath- can be changed appropriately)

I zipped this tool (scripts and jar) for your convenience and uploaded it (pturlreplace-1.0.zip) to my google code project (http://code.google.com/p/alui-toolbox/).

Let me know what you think of it after you tested it. If you think it is useful, I'll update this post with new version of the tool as I come up with it. Enjoy!

Technorati Tags: ALUI Portal,Portal Maintenance,Technical Tips,Tools

ALUI Portlet Monitoring Tool

noreply@blogger.com (Fabien Sanglier) — Sun, 13 Jan 2008 00:52:00 +0000

Most ALUI Portal environments will usually contain a large number of portlets, either "out-of-the-box" or custom portlets. These portlets, which range from simple bookmark links to sophisticated applications that interact with third party products such as Siebel or SAP, are indeed the major components that serve the high value functionality of your ALUI portal. Thus, it is vital that these portlets are monitored properly, so that a problem can be detected as soon as possible (preferably before your customer calls you to report the error that you had no idea was occurring...) and of course corrected asap.

Developed by Project Performance Corporation (www.ppc.com), Portlet Monitor® is a utility designed to track the operational status of all portlets registered in any communities or mypages of your ALUI Portal environment. Although Rakesh Gupta from PPC is the creator of this product, I personally redesigned and upgraded it for version 6.X of ALUI, and thus know the internals of it inside out.

You can contact me (fsanglier-at-ppc-dot-com) if you need additional information and pricing.

What is it?

Portlet monitor is a stand alone application (developed in .NET) that uses the Port

al Server API to read the portal data, especially the community and portlet objects. It can be installed on any server that have the following components: Portal, Automation, or API Service (to be able to use the server API). Using windows automated tasks (or even ALUI automation server) portlet monitor can be configured to run at any pre-specified time.

How does it work?

In plain English, Portlet Monitor connect as a portal user (by initiating a portal session) and iterate through all communities, mypages and finally portlets (the ones that are actually used on the community or "My" pages) that this user has access to. For each of the portlet found, Portlet monitor will "fake" a gateway call to the remote server hosting the portlet, passing all the required portal preferences and other ALUI-specific settings to the remote portlet application.

Error Found?

Various types of error can be caught by portlet monitor:

Various HTTP Errors (500 / 401 / etc...) returned by the remote portlet application/web server.
Portlet Timeout errors
Network errors (remote portlet server has some network problems)
Portal exceptions (portal settings or portlet preferences missing, problem with gateway component or any other portal internal component, etc...)
Specific error text pattern found in the portlet (if the above errors are not found, an error can be triggered if a specific text patterns is found in a specific portlet)

Results?

The result of each run is locally saved in an XML file that contains the following:

The portlet ID
The community / mypage of the analyzed portlet
Processing Time
If "Broken", the last date/time the portlet was found "Working"
If "Broken", the cause of the error (exception trace or error code)

After each run, an email containing the above XML report is automatically sent to the specified email list. (usually portal administrator and specific portlet developer(s))

It is also possible to eventually serve the XML report directly in the portal through a portlet (with a XSLT applied of course)

Performance?

Portlet monitor can be installed on any windows server that contains either the portal, automation or API service component (It is important to note that the portal component service does not need to be running) Thus, to minimize the performance impact of portlet monitor successive runs, it is recommended to install it on a server that is not part of the live portal cluster.

Why using this Tool versus Enterprise Monitoring Tools?

While most enterprise monitoring tools (i.e. NETIQ) work from the component service (or process) point of view, Portlet Monitor works from the portal point of view. What does it mean exactly?

Basically, enterprise monitoring tools are able to to monitor windows services, and eventually restart them if something wrong is happening. They are also able to monitor component log files and search for any text pattern that are relevant to an error, and if found, perform a specific action. (like restart the service) They are also able to perform a lot of other various operations...

But what they are not able to do in a portal context like ALUI (versus Portlet Monitor) are:

Detect the errors before the end-user finds it (by running the portlet monitor scans at scheduled intervals) - compared to finding the errors in the log files when the end-user actually triggered the error himself...
Detect point of failure(s) between the portal and the portlet remote server
Detect portal application or database defects (i.e. gateway problem) that could induce the portlet in error (from portal point of view), even if the remote application is just working fine.
Detect misconfiguration of the portlet that could induce the portlet in error (from portal point of view), even if the remote application is just working fine.
Detect any infrastructure or software errors that would induce a timeout of the portlet.
Detect any portlet errors even if the monitoring agent is not running (sometimes the monitoring agent can have defect too...)
Detect any portlet errors even if the portlet application service is in a "Frozen" state (meaning the service is still up, the log are not showing any error, and thus the monitoring system is not reporting anything wrong, but the application is actually not responsive anymore)

It is important to note that portlet monitor is just a monitoring and notifying tool: It does not perform any action (like restarting services etc...) other than notifying the right people.

Last Words

If you don't have a monitoring infrastructure in place, this tool will offer you monitor effectively the most important and valuable components of your portal (portlets).

If you have already a monitoring infrastructure in place, this tool is not meant to replace it, but rather to be a valuable addition that completes your monitoring infrastructure already in place.

Technorati Tags: Aqualogic User Interaction,ALUI Portal,Tools

ALUI and Java Server Faces Portlet development - Part 2 (Continued)

noreply@blogger.com (Fabien Sanglier) — Wed, 05 Dec 2007 23:25:00 +0000

In my previous ALUI JSF post ALUI and Java Server Faces Portlet development - Part 1, I talked about the problems you might encounter while using JSF and ALUI, and I also started to explain how to fix those problems. I will continue with that post...

Form Unique Id

It is standard (best) practice to make sure every ids and every javascript function names in a HTML page are unique. That way, Javascript functions are targetted correctly when called, and Javascript functions can effectively use "getElementById()" to access the right element within the portlet. When you are in a Portal environment, it is difficult to ensure that all the portlets on the page have unique ids. And thus it is possible to have a defective portlet when rendered with others on the same page, due to those kind of interferences (same element ids or same javascript function names). The probably of error reach 100% of chance if you put 2 portlets with identical content on the page.

Traditionally, with web applications where you have control over every elements and javascript function throughout the page (i.e. standard JSP), you would surely use the pt:common.namespace adaptive tag, in order to append to every JS functions and form ids the token string (i.e. $$PT_TOKEN$$)

But unfortunately, that won't work with JSF. Why is that?

First, quick reminder: It is important to note that "namespace" tag (like every ALUI Adaptive Tags) is transformed by the GATEWAY component. In other word, if you use the tag $$PT_TOKEN$$ throughout your JSF form, the remote JSF application DO see literally "$$PT_TOKEN$$", not the actual portlet IDs. When the portlet response goes through the gateway component, that is when the tag "$$PT_TOKEN$$" is changed on the fly with the real portlet id number.

Since JSF is a component oriented framework that generates the HTML markup, we don't have direct control over the HTML elements...and also, the element "name" (this is posted back to the server) is usually the same as the element "id". Thus, when you use:

<h:form id="editaddress_$$PT_TOKEN$$">
<h:inputText id="fullname" value="#{bean_backing.fullname}" />
</h:form>

You HTML markup for the response is:

<form id="editaddress_$$PT_TOKEN$$" ="">
<input id="editaddress_$$PT_TOKEN$$:fullname" 
name="editaddress_$$PT_TOKEN$$:fullname" type="text" value="" />
<form>

And when you visualize the portlet source code (as rendered by the portal, where 289 is the portlet id that renders your web application), you get:

<form id="editaddress_289" ="">
<input id="editaddress_289:fullname" 
name="editaddress_289:fullname" type="text" value="" />
<form>

So great, we do have unique HTML elements within each portlets, but you start to get the problem, right? Okay I am still going to explain :)

So the JSF component is waiting for a posted request attribute with literal name "street_$$PT_TOKEN$$", but actually, the posted attribute is "street_289"...and thus the JSF component cannot decode the posted value, and thus the form post won't work.

So the solution is to include the real portlet Id on the remote application side directly. I will use the same technique that I explained in the previous post: Custom renderer for the Form JSF component, overriding this time the base method "convertClientId(FacesContext context, String clientId)". Within the overridden method, I get the portlet id by simply using the IDK and then append the id to the component id.

    @Override
    public String convertClientId(FacesContext context, String clientId) {
        IPortletContext ctx = PortletContextFactory.createPortletContext(
                (HttpServletRequest)context.getExternalContext().getRequest(), 
                (HttpServletResponse)context.getExternalContext().getResponse()
                );
        IPortletRequest portletRequest = ctx.getRequest();
        //add the portlet id to the form
        if(portletRequest.isGatewayed())
            clientId += ctx.getRequest().getPortletID();
        return super.convertClientId(context, clientId);
    }

That's it!! That way, even if you let JSF generate the form element id, the "portlet id" WILL be appended if you are viewing the application through the portlet. And since the component is aware of the portlet id, the post will work fine. And finally, since the <h:form> is very often the top element of your JSF page, all the sub elements will also be unique since the form id is the prefix for all the children component ids.

Although you probably don't want to create a custom specific renderer for every components you want to use, this technique seems the best to me because it is completely non-intrusive within your JSF page...and thus, in the future, if BEA corrects all those JSF problems, you just disable the custom renderers and everything will work without any change.

ALUI and Java Server Faces Portlet development - Part 1

noreply@blogger.com (Fabien Sanglier) — Sun, 02 Dec 2007 01:25:00 +0000

Lately I have been looking into JSF and especially the Myfaces implementation as a development framework for ALUI portlets, Similar to ASP.NET, I personally do think JSF is also an awesome and powerful framework to develop all sort of cool portlets and applications. But similar to ASP.NET portlets, JSF portlets have some problems when integrated in ALUI portal.

Mainly, the problems lie in the fact that those 2 frameworks are component oriented: We don't have a lot of control on the rendition of those control, and sometimes, the way they render is not fully compatible with the way ALUI portal behave:

Some urls in generated javascript are not always successfully gatewayed.
The portal does not enforce generated Html components to have unique IDs throughout the portal pages (i.e. 2 portlets on the same page can possibly have 2 form tags with same ids...which would very likely mess with the functionality of the portlets)
If you don't enable ALUI inline refresh, the portlets will go in the "Gateway" space (or the maximized state) on any navigation event. Although that can be useful in certain case, it won't to perform portlet-to-portlet communication (since the other portlets disappear in this maximized state).
If you DO enable inline refresh, you will hit other problems due to the fact that the Javascript layer added to the "inline-refresh" enabled portlet can interfere with the good behavior of some of the JSF/DOTNET components...

Ok...having painted such a "Dantesque" picture of the portlet development within ALUI portal, I now want to reassure you: Because ALUI is so extensible and powerful, all those problems can be resolved, and that's the goal of this post: Showing you some of the trick to make it all work.

For you DotNet users, you are in luck :) BEA introduced the "BEA AquaLogic .NET Application Accelerator" to offer a remedy to all the problems above. Since it is all there at http://www.bea.com/framework.jsp?CNT=index.htm&FP=/content/products/more/accelerator/ I won't talk to much about it.

But what about you poor JSF users? Well unfortunately for you (and I), we will have to get to work :)

But before getting to work, some of you might say: What about the "Java Portlet Tool" (http://dev2dev.bea.com/pub/a/2006/05/java-portlet-tools.html)? Well I looked at that quite thoroughly. I must admit that it is a pretty involved and clever piece of code...and it makes things work a little better on the JSF side. I especially do like the bean support for IDK functionality. Unfortunately, the following points are making me very hesitant and uneasy about this framework:

Unlike "BEA AquaLogic .NET Application Accelerator", the "Java Portlet Tool" (which is written by a BEA resource) is not supported by BEA. (Even though I could go with that - after all since the Java portlet tool code is provided, we could eventually support it ourselves - I would not recommend it to my clients who pay a lot of money to get support),
In addition to that, the java portlet tool, which rely on the modification of the pretty involved inline refresh jsxml javascript libraries (PTXML.js), seems to have been developed and tested with Plumtree 5.x, and does not seem to have been upgraded for 6.x versions of the portal. (thus, it might or might not work with 6.x versions)
Finally, the java portlet tool project has not been modified since 2006, which makes me think there are not a lot of people using it, or keeping it up-to-date.

Okay...so no supported portal framework for JSF...but still I want to use inline refresh...

Does it mean that I have to develop my JAVA portlets the way I use to develop them 5 years ago, without the popular frameworks out-there? Well I personally refuse to do that, and you should too :)

So it is time to have some fun finally!

JSF Navigation

A major problem I found with the ALUI inline refresh and JSF is that HTTP POSTs with JSF navigation is not working. JSF framework navigation (similarly to struts etc...) is controlled by the framework itself, based on the navigation rules you define in the JSF config file. (faces-config.xml). In other words, when the form is posted back to the server, the framework will deal with the navigation based on the action define on the command button that triggered the HTTP POST. Nothing unusual until now, and such a navigation behavior will work fine if you don't enable inline refresh...but the next page will be displayed in the maximized state we want to avoid.

When you enable inline refresh though, the ALUI portal will intercept the button submit action and perform its own submit action using its AJAX/Javascript framework (PTXML.js). On the html side, you will see:

<form id="viewaddress" name="viewaddress" onsubmit="pt_280.formRefresh(this); return false;" method="post"
action="http://localhost:7001/portal/server.pt/gateway/PTARGS_0_0_280_377_0_43/http%3B/localhost%3B7001/MyJSFDemo/viewAddress.jsf">

We can particularly notice how the submit is intercepted: the OnSubmit event handlers of the form tag does a "return false", which is the equivalent of saying: the submit button has never been clicked, the form has never been submitted. The "pt280.formRefresh(this);" is the statement that will post the form using an AJAX call. How this works is interesting: Using JavaScript to go through the DOM, the portal reconstructs the post request using all the input fields in the form (which would have been posted to the server). But the problem is: during that reconstruction, the <input type="button" /> and <input type="submit" /> are purposely left out. Although this usually makes sense in a standard web application (those buttons don't have a particular meaning on the server side, they are just useful to trigger a post back to the server), it does not make sense in the JSF world: the button name/value pair should be posted back to the server because the navigation rules won't happen otherwise. The result is that with inline refresh, navigation initiated by a form post is NOT working.

That is inconvenient...but we can correct that easily: if we create a hidden input field that has the same name and value as the button clicked, then this hidden field WILL be included in the posted request and the navigation will then work! Great! But you don't want to manually add yourself the hidden field in the form...for the simple reason that you want the value posted only when a particular button is clicked.

So the simple fix is to use javascript to dynamically add the input field in the form:

<pt:namespace pt:token="PORTLET_ID" xmlns:pt='http://www.plumtree.com/xmlschemas/ptui/' />
function setHiddenInputPORTLET_ID(form, name, value){
	var newInput = document.createElement('input');
	newInput.setAttribute('type','hidden');
	newInput.setAttribute('name',name);
	newInput.setAttribute('value',value);
	form.appendChild(newInput);
}
<input id="viewaddress:_idJsp12" name="viewaddress:_idJsp12" type="submit" value="Change" 
onclick="if(typeof window.oamSetHiddenInputPORTLET_ID!='undefined'){setHiddenInputPORTLET_ID(this.form,'viewaddress:_idJsp12','Change');}" />

That way, when you click the button, the equivalent hidden input is added to the form, which will then be effectively posted by the "pt280.formRefresh(this);" call. You noticed that I also use the ALUI adaptive tag to generate a unique token (portlet id) and thus make the JavaScript function "setHiddenInput" unique (that way, no interferences can occur if multiple portlets are on the same page)

In addition, you certainly don't want to force your developers to create that by hand for every buttons, (as well as introducing in your presentation code such a specific bug fix). So in order to be the least intrusive possible, I chose to customize a little bit the rendering of the <h:commandButton> element, which is the one responsible for postback and navigation:

<h:panelGroup>
<h:commandButton action="editaddress" value="Change"></h:commandButton>
</h:panelGroup>

To do that, you just have to create a Class that extends the HtmlButtonRenderer class, and overrides 2 methods:

encodeEnd to generate the javascript function setHiddenInput281()

buildOnClick so add the call to the generated function setHiddenInput281()

That way, your JSF page are exactly the same...but the rendering of it is a little different when viewed through the portal. You can find the full custom renderer here.

That fix alone will take care of a good majority of the problems you would have had with using JSF and inline refresh inside ALUI. On a next post, I will show you how to deal with form ids when several JSF portlets are on the same page.

So long!

BEA Aqualogic Portal (ALUI) - Single Sign On (SSO) Concerns and Techniques

noreply@blogger.com (Fabien Sanglier) — Sun, 18 Nov 2007 00:11:00 +0000

As explained very well on this great post "Changing SSO Settings" from Function1, it is highly recommended to protect only the "SSOServlet" (default path= /portal/SSOServlet) when you want to SSO-enable the ALUI Portal. As I worked recently with an integration of ALUI with CA Siteminder, I am going to go a little bit further so that the SSO solution is complete.

But first, the 2 benefits of protecting only SSOServlet are:

Performance (as explained in Function1's post)
Enable guest access to the portal with same url (the portal is the gate, not the SSO agent)

But the main concerns are related to the SSO Session versus the Portal Session. As explained in Function1's post, the timeouts of portal and SSO agent should be in sync so that the user is actually logged out when the portal session times out (20 minutes by default). As it is well explained, you want both timeout to be very close, the portal timeout being trigged a bit before the SSO timeout.

But that's where I want to go just a little further...timeout sync is not enough to make the system perfect. Indeed, 2 extra use cases exist (and are not covered yet):

SSO session "keep-alive"
SSO session Logged out.

SSO session "keep-alive"

Basically, after the user successfully logged into ALUI portal (using SSO login screen), a SSO token has been created for its session (SSO cookie embedded in the browser session). But the problem is that after a successful login, the portal won't call SSOServlet again, and thus the SSO agent (generally on the web server front end, or the application server hosting the portal) has no longer anything to say about the traffic going through its agent since it is all unprotected from its point of view. Actually, it is as if the agent is not even seeing that traffic at all. Why do we care?? Well, the SSO session timeout is only refreshed if the agent is actually seeing any protected traffic. When the agent is not seeing any protected traffic, the timeout count down starts.

So to sum up, when the user successfully logged in the portal and is browsing the portal protected content, it is as if the session was idle from the Siteminder point of view. That is perfectly fine (at least you wont notice the problem) if you only use SSO for the portal. But imagine you use SSO for any other application in your enterprise (which is the use of SSO by the way :) ), or even portlets within your ALUI portal, then you probably don't want your SSO session to timeout while you are browsing the portal. Otherwise, after 20 or 30 minutes (whatever is set up on your SSO system for SSO timeout), you will still be able to browse the portal fine (since you are not idle from the portal point of view), but your SSO Session WILL HAVE TIMED OUT...and you will have to login again if you go to another of your favorite enterprise SSO protected sites...

So how can we make that work?? How can we ensure that the SSO session (or SSO token) is "kept alive" while we browse the portal?? Very simple trick actually: I basically created a dummy SSO protected content on the front end web server (for example http://portal.com/sec/dummy.html) and embedded that url in an invisible iframe in every portal pages (well suited for the footer portlet of your portal, or better, put that in your favorite liquidskin component - see my liquidskin ongoing serie "Overview of BEA ALUI LiquidSkin: Part 1" and "Overview of BEA ALUI LiquidSkin: Part 2")...

Basically, you have <iframe src="http://portal.com/sec/dummy.html" width="0" height="0"></iframe> at the bottom of every page you wish...it seems low tech, but works pretty well: every portal pages that have that footer portlet will initiate a request to that protected url, letting the SSO agent know: Hey, i am not idle, my session is still alive!!

Ok, good...but I am still not happy yet :)

SSO Log out

When you logout from the portal, you also want the SSO session to be logged out too. this is easy enough with CA siteminder: in the SSO policy you just provide the log out pattern to look for (usually something like /portal/server.pt?open=space&name=Login&control=Login&doLogout=&clearsession=true">/portal/server.pt?open=space&name=Login&control=Login&doLogout=&clearsession=true">/portal/server.pt?open=space&name=Login&control=Login&doLogout=&clearsession=true">http://<portal>/portal/server.pt?open=space&name=Login&control=Login&doLogout=&clearsession=true) and the SSO agent (remember it is on your front end web server or application server) will kill the SSO token if it intercept such a pattern in the request url. Cool!! That way, when you click log off in the portal, you are actually logged out.

But again, let's imagine we have multiple systems that uses the SSO session. You first loggedin the portal, browse it a little, and then decide to go to this other SSO-protected application. Then you decide you are finished and logout from there. What happens if you log out from another system apart from the portal?? Well if you don't do anything special, you will logoff from this other application, which will normally kill the SSO session (same "finding logout url pattern" technique)...but if you go back to the portal after that (and you were previously logged in)...oh surprise, you can go in no problem!! Well that's normal, your portal session is not over yet (provided that you were not away from the portal more than the portal timeout) and you did not do anything to finish it...

But that is not how it should work, right? If the SSO session is over (by logging off from any SSO protected system), you should not be able to go in any other SSO protected system!! Just imagine if your favorite banking web site was working this way...you logoff from your savings account, but your checking account interface is not logged off...I would not particularly like that from a security point of view :)

So we have to deal with it: I see 2 options in order to make sure all your systems are logging out when 1 is logging out:

Having an independent logout page that initiates logout requests for all the protected applications in your enterprise (for example an HTML page containing as many <img> tags or <iframe> tags as you have application to log out) -- I don't like this method so much because I will have to maintain this central logout page as soon as I add / remove a protected application in my environment,
Making applications log out on their own if the SSO token is not there. (or at least forcing any requests to SSO login if the SSO token is not present in the request)

The second solution is to me the best since it works once and for all. But it is a little more complex than the first one since we are protecting only the SSOServlet...(if the site was fully protected, that would work without doing anything...the agent would not see the SSO token and would not let you pass as long as you don't authenticate successfully)

How can we do that in the portal? I chose to implement a JAVA filter (I was working with JAVA portal with that client) that checks for SSO token presence (you can implement the same filter functionality in DOTNET too: it is called http module). The simplified logic is:

If SSO token is present and valid, do nothing,
Otherwise, force portal logout (if you actually previously logged in)...which will make the request being redirected to SSO login...

Et Voila! That way, the portal will logout nicely, on his own, when no SSO token is found.

I hope this helps you better understand the different concerns of implementing Single Sign On with your favorite ALUI portal (would actually apply to other portals too)

How to Change ALUI Components Passwords

noreply@blogger.com (Fabien Sanglier) — Sun, 04 Nov 2007 04:09:00 +0000

Each ALUI component requires Database information (host, schema, user, password, etc…) in order to interact with their respective Database.

When DBAs change the password of a database user (in some companies, it can actually happen pretty often) that is used by an Aqualogic User Interaction component, a similar “password change” procedure has to be done for each the concerned components’ configuration files.

If that was just it, that could be easy...but thanks to security, every passwords in the ALUI configuration files are encrypted...and thus, you need to encrypt the password too. But how do you encrypt those passwords? Where is the encryption utility?

ALUI provides several encryption mechanisms that you can apply to each component. I will try to walk you through what I call the encryption maze so that changing ALUI passwords becomes a breeze. (Note: files *.bat or *.cmd exist also in the unix/linux world...extension sh)

ALI Portal

DB Configuration File(s) path :

<PORTAL_HOME>\settings\common\serverconfig.xml

Encryption Utility path:

<PORTAL_HOME>\ptportal\<VERSION>\bin\ptconfig.exe (this utility will change the password directly in the serverconfig.xml file, OR
<PORTAL_HOME>\ptportal\<VERSION>\bin\cryptoutil.bat (must be executed in a shell or dos prompt -- provides the encrypted version of the text provided)

ALI publisher / Workflow

DB Configuration File(s) path:

<PORTAL_HOME>\ptcs\<VERSION>\settings\config\database.content.properties (publisher DB connection)
<PORTAL_HOME>\ptcs\<VERSION>\settings\config\database.portal.properties (portal DB connection)
<PORTAL_HOME>\ptworkflow\<VERSION>\settings\config\application.conf (workflow DB connection)

Encryption Utility path:

<PORTAL_HOME>\ptcs\<VERSION>\bin\native\pcsencrypt.cmd

Note: You can use that same utility to encrypt the publisher basic authentication user password too.

ALI Collaboration

DB Configuration File(s) path:

<PORTAL_HOME>\ptcollab\<VERSION>\settings\config\database.xml (collaboration DB connection)
<PORTAL_HOME>\ptnotification\<VERSION>\settings\config\database.xml (notification DB connection)

Encryption Utility path:

<PORTAL_HOME>\ptcollab\<VERSION>\bin\passwordChanger.bat

The utility passwordChanger.bat will change the password in both config file and database directly. This is very useful indeed, but what if the DBA only can change the password, or what if he already did change it...What now? Well another solution is available to you...you can use another encryption utility that works (weirdly): The publisher encryption utility <PORTAL_HOME>\ptcs\<VERSION>\bin\native\pcsencrypt.cmd

Note: You can use that same utility to encrypt the collaboration basic authentication user password too.

Studio

DB Configuration File(s) path:

<PORTAL_HOME>\ptstudio\<VERSION>\settings\config\PTStudioConfig.xml (studio and portal DB connections)

Encryption Utility path: No encryption utility is provided in the studio directory, so believe it or not...you guessed it...again the publisher encryption utility can be used for both studio and portal DB passwords.

<PORTAL_HOME>\ptcs\<VERSION>\bin\native\pcsencrypt.cmd

Analytics

<PORTAL_HOME>\ptanalytics\<VERSION>\settings\config\security\securityservice-config.xml (portal DB connection)
<PORTAL_HOME>\ptanalytics\<VERSION>\settings\config\hibernate.properties (analytics DB connection)
<PORTAL_HOME>\ptanalytics\<VERSION>\settings\config\configurator.properties (analytics, portal, collaboration and publisher DB connections)
<PORTAL_HOME>\ptanalytics\<VERSION>\settings\config\jobs.properties (portal, collaboration and publisher DB connections)

Encryption Utility path: Analytics provide a web configuration interface for changing DB connections and passwords in all the configuration files. But if you need to do it yourself without this web interface, again a solution exist: this time use the ... portal encryption for all the passwords!!

<PORTAL_HOME>\ptportal\<VERSION>\bin\cryptoutil.bat

That's all I got for now! Hope this helps to navigate in ALUI Password encryption hell! :)

Overview of BEA ALUI LiquidSkin : Part 1

noreply@blogger.com (Fabien Sanglier) — Sun, 28 Oct 2007 03:49:00 +0000

On one of my past project, I worked quite a bit with this (fairly) new BEA offering for BEA Aqualogic Interaction (ALUI) Portal: LiquidSkin. But what is it exactly?
Well, LiquidSkin is a pluggable component (or pluggable navigation in the ALUI jargon) that will transform your standard ALUI Portal Interface into a web-like easy-to-use easy-to-update interface.
LiquidSkin can be installed on any G6 portal (6.0, 6.1, 6.1 MP1) and supports rapid development of custom pluggable navigations.
It is important to note that this component DOES NOT come free with your portal purchase(inquire with BEA in regards of the pricing), but once bought, it includes installation/training by a BEA professionnal, as well as fairly good support.

All in all, I've been quite impressed with this component and want to give an overview of the great features that it provides to your ALUI portal.

First what is an ALUI pluggable navigation?

The pluggable navigations are ALUI software components that create the content around the portlets in your portal (for example links to mypages, my home, my communities, mandatory communities, my account, help, etc...)

By default, ALUI provides a couple of standard navigation options:

Even though you can already do good things with those out-of-the-box navigations, it does not offer you a great flexibility. Indeed, if you want to customize you navigation (i.e. putting an extra link somewhere outside of your portlets), you will go through a great deal of code...(unless you are a real ALUI guru of course)

So what does LiquidSkin give me??

LiquidSkin fills that gap by offering a pluggable navigation "on steroid", with a long list of navigational components that you can use easily in order to build the ALUI interface of your dream :)

To give you a better idea of what LiquidSkin gives you, here are the parts of the screen that are the responsability of LiquidSkin (and thus that you can customize easily)

List of Interesting features

I dont want to sound too marketing (I am neither a BEA employee, nor have shares in the company :)), but below is a list of the feature that I most appreciate in LiquidSkin:

Complete configurable Pluggable Navigations that allows rapid development of custom navigations.
Construct and deploy custom navigations by modifying simple XML configuration files.
Only skills required: HTML (table-less) and CSS. No Coding!!
Each skin tied to an experience definition: provide multiple intranet or .com-like navigation experiences within 1 portal.
The navigation is available in both .Net and Java versions, and it is easy to install (15-30 minutes)
Both the code and the configuration options are well documented, and there are multiple configuration examples.
No Portal Restart when modifying or installing a new skin.
Skin loader: Configurable schedule.
Easy deployment from development to production: Copy only folders, images and xml documents (no compilation of code needed)
Make it easy to internationalize your nav customizations.
Optimize performance by selectively caching components output.
Hide page display for communities with page name starting with specified word.
Styles possibly targeted to different browsers
Etc...(you will see for yourself when you start working with it)

A couple of components you might like

Top Nav and Breadcrumb

Left Navigation

All together

On this last one, we can see one portlet on the page (Portal Login) and the rest is displayed by LiquidSkin (Top DHTML Nav, Top Sub Nav for pages and subcommunities, Left Nav, and buttons for quick functionnality access)

What's under the hood?

I spared you the technical part ... until now. But dont worry, it is not that complex. Basically, to add component to a view, or edit component properties, or delete components from a view, you have to edit the "LiquidSkin.xml" that is located in your image server, within the LiquidSkin folder (ptimages\imageserver\plumtree\portal\custom\LiquidSkin\<your liquid skin name>\)

After you edited that file, the Liquid Skin timer task that runs every X seconds (you specify the reload interval in the same Liquidskin.xml file) will reload the file in the portal memory, which will make your changes visible on the screen.

To give you a basic idea, this Liquidskin.xml configuration file is similar to the following:

As you can see, nothing fancy: Just a well formatted XML that contains all the views you want to use, all the components that you want to add in each view, and all the parameters for each components...

But how do I link my portal experience to the liquidskin??

You just have to specify in the property attached to your portal experience definition the name of the LiquidSkin you are developping. (see below). This name will correspond to the liquid skin that is one your image server, at the foloowing path: ptimages\imageserver\plumtree\portal\custom\LiquidSkin\<your liquid skin name>\LiquidSkin.xml

Final Words

Even though this component offers great flexibility and a fairly long number of components to use, the real challenge will be to build the LiquidSkin.xml file, which requires a good knowledge of all the components available to you (and how they works), as well as a fairly good understanding of tableless HTML (every thing is DIV-based) and Cascading Style Sheets. Nothing hard for a seasoned web-designer :)

So long...see you next time to talk about how easy it is to install that component!

Overview of BEA ALUI LiquidSkin: Part 2

noreply@blogger.com (Fabien Sanglier) — Sat, 27 Oct 2007 04:06:00 +0000

In my previous post "Overview of BEA ALUI Liquidskin", I talked about the great features that this offering provides. Now I want to talk about the installation of the component.

But first, I might not have been clear on my previous post: The LiquidSkin engine is running on the portal server, NOT on the image server. On the ALUI image server(s), you will find the different configuration files (all called LiquidSkin.xml in their own folder) that contains the settings for each skin in the various portal experience definitions. The xml files on the image server are of course not accessed every time a page is displayed, but loaded in the portal memory at specific intervals (very similar to Varpacks loading for those of you who are familiar with Varpacks).

LiquidSkin comes in 2 flavors: .NET or Java (like the portal). Depending on the portal you are running, you will chose the appropriate version.

For .NET, those are:

LiquidSkin.dll (core LiquidSkin code that contains the base classes and the various utility classes)
LiquidSkinComponents.dll (contains the majority of Liquidskin components, i.e. LeftNavigation, Breadcrumb, Links, etc... Each of those components extends a base class defined in the core LiquidSkin.dll)
LiquidSkinImpersonate.dll (optional component -for security purpose- that allows an administrator to impersonate any user in the portal. When activated, the administrator is logged in the portal as that user.)
LiquidSkinFilteredCommunities.dll (optional component that creates a list of "My Community" links which are filtered to include and/or exclude certain 'types' of communities. A community 'type' is defined by a combination of property values associated with the community. Note: Expensive process. Enable Caching.)

For Java, you just replace dll with jar for the names.

Installation Steps:

It is said in the documentation that it should not take more than 30 minutes to install Liquidskin. This is TRUE!

Stopping/Starting portal server(s)
Copying a the jar/dll files to each portal server
Editing the CustomActivitySpaces.xml and portalconfig.xml files on each portal server
Copying the skin folders to each image server
Importing a .pte file to create the custom properties used by LiquidSkin

Jar / Dll Installation

In order to install those jars / dlls in your portal, you just need to put them in the library folders of the portal installation, as well as in the portal web application itself.

Thus, you need to copy them in <pt_home>/ptportal/6.0/lib

And in the portal web application (method differs slightly between .NET and Java)

For .NET, it is easy since the portal application is not packaged in an archive, as it is for java. Thus, you just need to copy all the dlls in the <pt_home>/ptportal/6.0/webapp/portal/web/bin folder.

For Java, you will need to explode the portal.war and copy the LiquidSkin Jars in the newly exploded folder WEB-INF\lib\. Then, just repackage the portal.war and update the portal.ear.

Little Reminder for Jar operations:

Exploding the war: jar -xvf portal.war
Repacking the new war: jar -cvf portal.war * (to be done at the root of what you want to package)
Update the ear: jar -uvf portal.ear portal.war

Portal Configuration

To enable the LiquidSkin pluggable navigation, you must edit the "<PORTAL_HOME>\settings\portal\CustomActivitySpaces.xml" file and add in it the LiquidSkin libraries:

<AppLibFiles>
<libfile name="LiquidSkin"/>
<libfile name="LiquidSkinComponents"/>
</AppLibFiles>

Portal Object Import

You must import the PTE package that contains 2 properties:

"NavConfigFolder" property: This is attached (Global Object Property Map) to experience definition objects. You will save in that property the name of the liquidskin that corresponds to the experience definition you want "LiquidSkin enabled"
"Subcommunity Display Order" is attached (Global Object Property Map) to the community objects. This is to allow custom sorting of communities within the Top or Left navigation components.

Optional: Modify margins and view width

As any other portal navigation. it is possible to change margins and view width in the PortalConfig.xml (section "portal:Navigation"). The only thing you need to know is that the the id of the LiquidSkin navigation framework is 71. This ID is to be appended to the navigation properties so that you can get for example a specific left navigation column width.

Finally, restart the web application server that host the portal. DONE!! In 18min 32 seconds!! See I did not lie :)