Recently I had been wondering if Project Management was a questionable career choice. I have spent the last couple of years surrounded by talented individuals who seemed to be able to work fine without me. The test is always when the project manager (me) goes on vacation. Has the team fallen apart? Are they forgetting all of the practices they were doing? Is stuff getting delivered to clients? I was finding that I had a backlog of chores when I came back, but overall things were still humming along.

The Agile 2009 session reminded me of the road I took from software developer to a project manager and why. Early in career I generally found project managers provided little value. I asked a lot of questions and I caused a lot of trouble for those “managing” me. Eventually somebody said the equivalent of, “If you think you could do it better, do it.” No introduction to the subject tells you that, simply, good project managers are good leaders. They optimize what resources they have available, point out areas for improvement, and occasionally carry a stick. An Agile project manager, freed from the burdens of heavy process, can’t stay busy with chores long and must produce a good team quickly. I found that through focusing on how the team could work best together and gently pushing during retrospectives, I left a trail of happy teams on successful projects.

It may not be the most glamorous job, but each team needs one. To continue the sports theme from my last post: Most teams populated by very talented athletes. However, its take a good coach to help each of them reach potential together as a team. Now, back to leading my team.

Related Services: Agile Development, Custom Software Development

Related posts:

1. Building a High Performance Agile Team: Assume You Will Be a One Hit Wonder
2. Getting a team over the fear of daily scrums
3. What’s the value of Agile out of the box?

Although I’ve seen so many players rise and fall over the years, what has inspired this post is the irony of what is happening to Microsoft. In the early years of my technology career it was Microsoft’s ownership of the PC O/S market that primarily allowed other compatible hardware manufacturers to create innovation and eventually marginalize IBM’s dominance of that market.

In the early years of PCs, the applications on top of the O/S weren’t even Microsoft applications. It was only later that Microsoft started developing their own applications and with them the predatory practice of squeezing out other application providers to rule the desktop.

Fast forward twenty years, and In many ways, they're a textbook case for two phenomena:  suffering the effects of incumbency (in Clayton Christensen's theory of disruptive innovation as articulated in The Innovator's Dilemma)  as well as case of technical debt.

Through some of their strong-handed practices and deep pockets Microsoft has been able to fend off and delay some of the threats from other vendors in the past but, in my opinion, it’s just a matter of time before that convergence of competitive forces brings them down a notch or two. In fact in my opinion it has already been happening on a number of fronts.

On the desktop and especially the mobile side Apple is making significant inroads. The iPhone is taking the high-end cellular phone market by storm, as essentially what will become a PC in a smaller handheld package and certainly owns the “cool factor” now. Although Apple on the desktop and laptop market still is primarily run only on its own hardware, it has adopted the Intel hardware architecture allowing Leopard to run LINUX or Windows under a VM.

The option of running legacy applications under a VM affords users more flexibility of applications while maintaining legacy ties. This is eroding the market for Microsoft’s Desktop Suites, as Open Office and other less expensive alternatives have matured and provide that basic functionality.

Microsoft’s gargantuan O/S suffers immensely from Worm and Viruses and from the negative effects imposed but the software layered on top to offer protection from those ills. On the server side Microsoft long-ago squeezed out Novell years ago but LINUX is still gaining market share here as well as a less expensive, more reliable, and more efficient O/S with less “bloatware”.

As far as the pressure from the Internet side of the equation, Steve Balmer said last week in Chicago that if he had one do over, it would be in the area of search technologies. Google and Amazon, both facilitated by the availability of Internet access from virtually everywhere, are in the process of moving as much functionality as possible to the Internet or the cloud.  Bing is a step in the right direction for Microsoft, but the jury is still out on whether it's too little too late.

While things are certainly changing, Microsoft is still very dominant player now and I’m not advocating that Microsoft is irrelevant or will be soon irrelevant any time soon. What I am saying is there is more than one player on the field now and it is having an impact regarding Microsoft’s market share and what Microsoft is able to charge for their applications. If not for Microsoft’s response to the competition that was initially started by Netscape, I doubt we’d be looking the fact today that any Vendor’s Browsers are free to download. Microsoft isn’t really falling but just losing its stranglehold on the IT world.

Related posts:

1. Contrasting Apple and Microsoft’s Product Strategy: A Tale of Two Spills
2. Sanity Amid the Tablet Hype
3. Betting Your Business on the iPhone

I recently had an issue in flex where I needed the click event for a component in an ItemRenderer to not bubble up to the parent list. Since the click event was being dispatched by Flex SDK code, I could not directly do this by passing false into the event's constructor. I had hoped to find a property I could set on the object, something like "clickBubbles", which I could set which would accomplish this.

Either that property doesn't exist or I could not find it. My solution was to basically hijack the event and send an impersonator in its place.

What I ended up doing was writing an event listener function which would stop propagation on the event. It would then dispatch another event with the same properties as the original, except it would set bubbles to false. This will prevent the event from bubbling up to parent components, but still allow event listeners to be added to the component itself.

The following is the function I wrote:

 
private function removeMouseEventBubble(event:MouseEvent):void {
	// if the event already does not bubble, we don't
	// need to do anything.
	// also, this will prevent any infinite loops from occurring
	if (event.bubbles) {
		// stop the event from being handled by other code
		event.stopImmediatePropagation();
 
		// dispatch a new event which does not bubble
		event.target.dispatchEvent(
			new MouseEvent(
				event.type, false, event.cancelable,
				event.localX, event.localY,
				event.relatedObject,
				event.ctrlKey, event.altKey,
				event.shiftKey, event.buttonDown,
				event.delta
			)
		);
	}
}
 

Now, all you need to do is set the click handler on the component:

 
<components:Foo click="removeMouseEventBubble(event)"/>
 

In order to make sure that this code prevented the original event from propagating and to ensure that it dispatched a similar event which does not bubble, I wrote the following FlexUnit test. N.B. This code is adapted from an actual test, I wouldn't recommend having the code under test defined in the test case itself, nor do I like the fact that the removeMouseEventBubble function is being added in the test.

 
public function testShouldRemoveBubbleOnClickButAllowEventListenersToStillFire():void {
	var component:UIComponent = new UIComponent();
	component.addEventListener(
		MouseEvent.CLICK, removeMouseEventBubble
	);
 
	// if we cannot add other event listeners, then the
	// async function will not get fired and the test will fail.
	component.addEventListener(MouseEvent.CLICK, addAsync(
		function(event:MouseEvent):void {
			assertFalse(event.bubbles);
 
			// assert that the event was translated properly
			assertEquals(MouseEvent.CLICK, event.type);
			assertTrue(event.cancelable);
			assertEquals(8, event.localX);
			assertEquals(9, event.localY);
			assertEquals(renderer, event.relatedObject);
			assertTrue(event.ctrlKey);
			assertTrue(event.altKey);
			assertTrue(event.shiftKey);
			assertTrue(event.buttonDown);
			assertEquals(5, event.delta);
		}, 1000)
	);
 
	component.dispatchEvent(
		new MouseEvent(
			MouseEvent.CLICK, true, true, 8,
			9, renderer, true, true, true, true, 5
		)
	);
}
 

I then wrote a test to verify that the event did not bubble up to the parent:

 
public function testShouldNotBubbleClickEventToParent():void {
	var parent:UIComponent = new UIComponent();
	parent.addEventListener(MouseEvent.CLICK,
		function(event:MouseEvent):void {
			// this won't cause the test to fail per say,
			// but it will cause an exception to be thrown
			// and this will show up when executing the
			// test.  This will fail the build, especially
			// if run on a CI server like hudson or cruise.
			fail('did not expect event to bubble');
		}
	);
 
	var child:UIComponent = new UIComponent();
	child.addEventListener(
		MouseEvent.CLICK, removeMouseEventBubble
	);
	parent.addChild(child);
 
	child.dispatchEvent(new MouseEvent(MouseEvent.CLICK));
}
 

This solution works, but I was hoping there was something more elegant and less hacky which accomplished the same thing.

Related posts:

1. Notification Testing with PureMVC and FlexUnit
2. Getting Rid of the “handler.call: handler undefined” Error
3. Grails: Custom Parent/Child Aware Tags

I've been spending some time with our internal sales and marketing team to hash out some of our goals for the year, and it became quite clear to me that non-developers are on their computers all day long facing some of the same technical challenges we do.

Some of the tasks they have to do:

• "take the data out of the spreadsheet for last quarter and compare it to this quarter"
• "gather the bounced emails from our newsletter posting, and update our list, pulling out duplicates"
• "replace all the names and addresses from our NDA agreement each time it is sent to a new client"
• "slice and dice google ad-words and google analytics data"

So I've resolved to take some time each week to 'Adopt a non-techie', and help them spend less time 'screwing around with the computer' and more time on the most valuable tasks they do.

In the same way that developers need to be as efficient as possible with the tools they use, so do the rest of the people at your company. At Pathfinder a good number of us have attended Neal Ford's Productive Programmer talks at the NofluffJustStuff conference, which covers a series of strategies for becoming extremely efficient as a developer. For those on our team that haven't attended the presentations, they pick up the best tricks through pairing and internal developer brownbags, but now I suggest we take the same approach with our business team.

While the benefits of having two developers pair on a technical task has been covered a million times, I would say it also applies to pairing with your UI designer, QA and BA resources as well. Whenever they need to take lots of text and reformat it, search/replace, etc, they might be tackling that problem manually, when you could help them solve it with a quick script or finding the plugin they need.

A few areas you might be able to help them with:

1. Forms and mail merge in word
2. Advanced searching and replacing in word
3. Google docs now has more powerful regular expression capabilities for searching and replacing (Actually, I could do a whole post on the benefits of getting your team to use GoogleDocs, would anyone be interested in that?)
4. Building a quick survey with google forms or surveymonkey
5. General text manipulation
6. Searching through all of their emails or local documents for anything related to the 'Jelly Belly vs. M&Ms' case
7. Excel spreadsheet manipulation, charts, etc.
8. Web searching and data extraction
9. Managing and sharing bookmarks and research data with others
10. Even tools like Selenium or iMacros could help your Business team automate repetitive form filling, web searching or web site testing.

Goals (First do no harm!):

1. Find the right problem. Just like we see in Product development, sometimes the most difficult problem is also the rarest, and not the best use of your time. In fact, the greatest area of efficiency is more likely to come from something they know how to do, but could be doing faster. (Instead of saving them 30mins off of a task they do twice a year, you might find something that saves 15mins every day!)
2. Don't expect them to become a developer. The goal is to reduce the time these things get in their way, so they can focus on the higher value stuff.
3. Make it easy for them to repeat the process (let them document it in their words, or even make a screencast while you're doing it)
4. Share your successes with the rest of the team (notes and screenshots/videos up on the wiki)

Use the right tools for the job!

While its best if you can help them to become more efficient with the tools they use everyday, you may find that they are just not using the right tool for the job, or if its a one-time thing, and you need to bring your toolkit with you.  For that I recommend a usb drive with your favorite PortableApps (notepad++ is great for text editing), or my new favorite PortableUbuntu (which lets you run linux off the usb drive, while accessing the files on the windows system)

What gets in your way?

To the business folks out there, what are some of the tasks you do to manipulate data? What frustrates you or eats up a lot of your time?

Reach out to your nearest techie, I bet they can help, or post your problem here and we'll make some recommendations.

Related posts:

1. Can your Selenium do that? Testing flash/flex and silverlight in web apps with iMacros
2. Stick with ERB or move to Haml
3. Does your project have Code Ownership Culture?

What does this mean for software developers and system integrators? Back when the aquisition plans were first announced, I joked that we would soon be using WebSphere rules. And wouldn't you know it, the aquisition announcement mentions ILog and WebSphere in the same sentence several times.

So, what does it mean for us? It means that IBM is likely going to give the ILog business rules products the WebSphere treatment, i.e. polute a perfectly good package with all sorts of WebSphere goodness until it isn't fit for integrator consumption. I'm sure there are going to be at least twice as many xml configurations files and an enourmous Java runtime that takes 10 minutes to start up.

Actually, IBM isn't even the worst of the big software vendors in regard to infecting aquisitions. Oracle is the undisputed champ in this regard. Don't get me wrong, I love their database product, but they buy a best-of-breed product, they get their hooks into it and turn it into something that you'd be crazy to run without the database. It makes sense from a product management perspective -- if you want to sell more databases, make sure that people who buy your other products need to use your database. Ergo, rewrite 30% of the application in PL/SQL.

Here's hoping I'm wrong about ILog and IBM.

Related posts:

1. IBM to acquire ILOG and other rumors
2. IBM and ILOG playing nice
3. GWT Showcase – BRMS for JBoss Rules

Related posts:

1. The Blogs Have Moved
2. PureMVC, Spanning the Platform Spectrum?
3. Touch Screen Kiosk in Adobe Air

One of the things that has differentiated commercial BRE's like Blaze Advisor and ILOG JRules from the Open Source JBoss Rules is the tools. The commercial BRE's were part of a suite of development and management tools -- IDE's, "natural" language support, Business Rules Management Systems (BRMS) -- that made their use in a corporate IT "ecosystem" much easier.

With version 4 of JBoss Rules, you now get a powerful workbench plugin for Eclipse, support for Domain Specific Languages (DSL) that mimick natural language, and a web-based BRMS. With it you can deploy, roll back, and report on your rules.

The BRMS has a nice Ajax interface and is in fact written in GWT. This is a natural for a Java-based system like JBoss Rules. If the developers who use and develop JBoss Rules can also tinker with the BRMS interface (GWT==Java in the browser), the odds are better that the BRMS will evolve apace with the core system.

Compare this with a framework like Echo2/3 where there is only a small cadre of developers who develop the JavaScript piece of what is essentially a Java centric framework, and consequently the development has lagged relative to other frameworks. So, good decision for JBoss Rules and probably a good decision if you are adding a web interface to your own Java system.

Related posts:

1. JBoss Rules Wiki – A Business Rules Community of Practice
2. Will JBoss Rules 3.1 Do for BRE’s What mySQL Did for RDBMS’?
3. Business Rules Resources

If you run into any issues (which I'm sure you will, given the contortions we had to go through to deal with Typepad's funky url's), drop us a line at info@pathf.com.

Related posts:

1. Category and UI changes on Pathfinder blogs
2. Two Interesting Blogs
3. “Ajax Overhaul, Part 3: Retrofit existing sites with jQuery, Ajax tabs, and photo carousels” Up on IBM Developerworks

Technorati Tags: bre, ruby, rails, rete, rools, SIE, Ruleby

Related posts:

1. Explaining RETE
2. Do You Really Need RETE?
3. IntelliJ IDEA and Ruby on Rails

Assert all your objects as facts: Rule engines usually know how to handle a fairly heavy load of facts, so let the engine do its magic. If you want to reason over a nested set of objects, assert all of them (not only the top level ones) into the engine and make sure your business model models the relationship between them. After that, writing your rules will be easier and the engine will be able to optimize execution (when compared to using eval and other constructs to access deeply nested structures). Example: if you have an Order fact that contains a list of OrderItems, assert both Order and OrdemItems as facts (if you need to reason over OrderItems too, obviously).

Why, by all that is object oriented, do you have to turn your domain model inside out, exposing what was hidden, unencapsulating what was encapsulated? Isn't this defeating the purpose of object orientation? The answer is, in short, yes. You are defeating the purpose of OO when you expose the innards of your model to the BRE. But it is a tradeoff. You get something for your trouble. Specifically, you get the super efficient rule matching of RETE and it's decendents.

This sort of exploding of the object model happens all the time in systems, and it isn't necessarily always bad. Alen Hollub (author of Why getter and setter methods are evil and other Edsger Dijkstra-like articles) has identified the "procedural boundary layer" and explains it nicely in his book Holub on Patterns: Learning Design Patterns by Looking at Code:

One big exception exists to the no-getter-or-setter rule. I think of all OO systems as having a "procedural boundary layer." The vast majority of OO programs run on procedural operating systems and talk to procedural databases, for example. The interfaces to these external procedural subsystems are by their nature generic. The designer of JDBC hasn't a clue about what you'll be doing with the database, so the class design has to be unfocused and highly flexible. UI-builder classes such as Java's Swing library are another example of a "boundary-layer" library. The designers of Swing can't have any idea about how their classes will be used; they're too generic. Normally, it would be a bad thing to build lots of flexibility that you didn't use because it increases development time. Nonetheless, the extra flexibility is unavoidable in these boundary APIs, so the boundary-layer classes are loaded with accessor and mutator methods. The designers really have no choice.

BRE's are not OO. Yes, the facts can be viewed as objects and the BRE's can be implemented using and OO language and design, but really the internal working of the BRE are most definitely not OO, the more the rule engine knows about each and every object and its innards, the more efficient it can be. Thus the PBL (procedural boundary layer) applies.

As an example why you need to expose all object, consider an order that is made up of line items.

public interface Order {
[...]
void addItem(OrderLineItem item);
boolean removeItem(OrderLineItem item);
[...]
}

Now normally you would hide the details of all the line items inside the class that implements the Order interface. Let's suppose now that we haven't added any of the encapsulated line items to the working memory of the BRE's, just the order. Now let's say we update one of the line items through an operation. The only way we have to telling the BRE that something has changes is by flagging the enclosing Order object. That will trigger a reevaluation of every rule in which the Order plays a part, even ones where the line items play no part.

There are all sorts of ways of dealing with exposing the internals of your domain model to the BRE without breaking encapsulation. One way is to implement both the business interface and a bean interface (getters, setters, etc.) in the implementation (see diagram). The business interface is used within the domain model, the bean interface in the BRE.

Another approach is to wrap the domain objects in bean containers that access the private instance variables through reflection.

Of course lots of this depends on the specific BRE you are using. Some require you to expose your instance variables directly and don't deal well with method calls. I'd suggest not using those.

Next time I'll go through a simple example using JBoss Rules showing how to convert a domain model to a Collaboration Cop rule base.

Technorati : bre, business rules, holub, ooad, patterns, procedural boundary

Related posts:

1. BRE Patterns III: Collaboration Cop, Part I
2. BRE Patterns I: The Fact Harvest
3. JSR-94 – Getting Your Feet Wet

rule "Find interesting crash events, with 10 or more port scans within a 1 day window"
when
$crash : OutageEvent( eventType == OutageEvent.SERVER_CRASH)
ArrayList( size > 9) from collect ( HackEvent(hackType == HackEvent.PORT_SCAN, $date : eventDate -> (DateHelper.withinDayWindow($crash.getEventDate(), $date, 1))))
then
suspicious.add($crash);
end

There is also the accumulate operator, which is even more powerful than the collect operator. The documentation has lagged a little behind (it isn't a full on release, after all), but you can see some examples of it's use here. I find that I use these sorts of collection operations quite a bit. That being said, the above rule isn't terribly efficient. If you have about 50 of those rules and lots of events of both categories, you will run into some cross-product hell (sort of like a poorly thought out database join). I'll show some ways of making this sort of rule more efficient in an upcoming entry.

Finally, the system now in theory allows better debugging of rulesets. The various views into the working memory have been around for a while. See below for some of my issues.

A few nits that I'm sure the JBoss Rules team will work out:

• The system desperately needs better syntax error handling in rule files. Right now the stack traces you get out of the parser/builder are pretty cryptic.
• While you can now set breakpoints in the rule files, it doesn't seem that they are recognized by the debugging environment. As of now, you can only set breakpoints on a WorkingMemory fireAllRules call. Not that useful for debugging rule issues. I'm not sure if this mod is supposed to address that problem, but regardless, it would be handy to have a fireSingleRule call so one can control the execution of the rule engine more closely.
• A real-time audit view to inspect engine events would be handy.
• The graphical RETE display would be more useful if the nodes were labeled. A display showing the facts in each node during execution would be too much to hope for, but one can dream.

The best thing about this latest version of JBoss Rules is, of course, that I can now start developing non-trivial examples and publishing them without having to constantly caveat that "this feature is present in Blaze and ILog, but not here." An even more interesting question is this: since JBoss Rules is getting close on being a reasonable alternative to some of the commercial products, will what happened in the database arena with mySQL also happen in the BRE arena? Will costs come down, interoperability increase? Will the commercial vendors concentrate on service, support and high end features? Will these same vendors become more open and share more information with the public? I don't think BRE's will ever be as widespread as RDBMS's, but I think the same dynamics might apply.

Some things to look forward to that I hope the JBoss Rules team is thinking about: rule management, specifically rule repositories and deployment management, e.g. revision control for rule sets and deployment of rule modifications to distributed, live production engines; Rule analytics, e.g. analysis of a rule set to see if it contains likely problems (infinite loops, possibly bad logic, etc.), and stats on actual rule execution, performance, etc.

Technorati : bre, business rules, drools, jboss rules

Related posts:

1. JBoss Rules Wiki – A Business Rules Community of Practice
2. GWT Showcase – BRMS for JBoss Rules
3. Business Rules Resources

This is a somewhat involved pattern, and I'll provide some more examples and references in part II. Also in part II, I'll run down the differences between the rule classifications that come out this pattern and the ones you commonly see in works by Ross.

Pattern Name: Collaboration Cop

Synopsis
Extract the business logic that is in an OO domain model -- collaboration rules, property rules and controller logic -- and express it as a set of business rules. Activity in the system is triggered by the UI inserting events into the working memory and firing the rules. Objects can be persisted by something like and ORM framework.

Slightly longer version: if you strip away the user interface and the persistence mechamism, a typical OO application is designed around a domain model and controller or service classes that set object collaborations within the domain model in motion in response to external events. The methods of the controller/service classes often correspond to steps in a Use Case. Embedded in the various domain classes is logic about how object can be modified and accessed, what collaborations can be created or desolved, etc. With this pattern, you extract all the service/controller and validation and collaboration logic into business rules. You populate the working memory with the objects that comprise the context for a Use Case. Finally, in response to an external event, you inject an event object into the working memory and fire the rules that modify the state of the object model (and persist it).

Context
Take the somewhat simplified Use Case where a customer is trying to rent a tape (he's a sucker for punishment and doesn't like DVD's). In a OO system, we would try to create a collaboration as follows (yes, very simplified):

customer.addRental(new Rental(title.getTape()));

The customer object knows how many active rentals a customer has and knows hat he cannot have more than three. This attempted collaboration would throw an exception to indicate that it is not permitted.

In the Collaboration Cop version we would create an event and insert it into the working memory:

VideoStoreEvent event = new VideoStoreEvent();
event.type = VideoStoreEvent.RENT_TAPE;
event.value = "Gone With the Wind";


IF event.type == VideoStoreEvent.RENT_TAPE AND title.getTitle() == event.value AND title.isInStock()
AND customer.activeRentals() < 3
THEN
customer.addRental(new Rental(title.getTape()));
externalObjectSet.add(customer);
externalResponse.message = "Rented tape.";
externalResponse.success = true;
ELSE
externalResponse.message = "Unable to rent tape.";
externalResponse.success = false;

You can wrap a transaction around the sequence of event injections, then save (using an ORM) all of the modified objects in the externalObjectSet.

-----------

This pattern is complex and powerful, and it's implementation is very much dependent on the capabilities of the chosen BRE, how you choose to deal with concurrency, etc.. In part II, I'll also construct a simple example using JBoss Rules.

Technorati : bre, business rules, patterns

Related posts:

1. BRE Patterns III: Collaboration Cop, Part II
2. BRE Patterns I: The Fact Harvest
3. BRE Patterns II: The Tranformer

The first is Pychinko, a native Python RETE-based business rule engine. This points out that Java, C# and C are not the only languages that can benefit from a business rules approach. In fact, as you'll see in my next BRE patterns entry, I'd argue that object orientation and business rules are a natural fit, regardless of the specific language used. If your requirements call for any sort of embedded scripting, this solution might be worth considering.

I'm experimenting with a new way of finding resources on the Internet. With this new method, I was able to find the second item, a thorough introduction to business rules and RETE over at InfoQ, entitled Real-World Rule Engines. The article covers quite a bit of territory. Among the topics addressed are:

• What are rule engines
• How do you use them
• Architecting with Rule Engines
• Managing Rules
• Rule Engine Patterns and Anti-Patterns
• Governance
• Thoughts on rule writing

Definitely a good resource for those wanting to get started with business rules engines.

Last, I wanted to pass on the news that James Taylor has moved his blog over to a new url: http://www.edmblog.com/weblog. If you are a regular reader of his blog, you already probably know this. If you aren't a regular reader of his blog, you really should be.

 
  Technorati : RETE, business rules

Related posts:

1. Business Rules Resources
2. Exaggerated Claims and the Importance of Hands-on Evaluation
3. Do You Really Need RETE?

The guys over at JBoss Rules are starting to build a little community of practice over in their Wiki.

There's a ton of good stuff over there, including a link to this tutorial by Arnaud Blandin that teaches in some great detail on how to integrate the JBoss Rules engine into your business process. Even if you're not open to open source or leaning toward a commercial product like Blaze or JRules, JBoss Rules is a great way to get your feet wet and get up the steep BRE learning curve. It certainly helps knowing what the heck a BRE is and what it can do when you are making decisions on how to integrate it into your software ecosystem. (Imagine trying to buy a database if your company has never used a database before. Would you know what to look for and what was important?)

Technorati : jboss rules, usage patterns

Related posts:

1. GWT Showcase – BRMS for JBoss Rules
2. Will JBoss Rules 3.1 Do for BRE’s What mySQL Did for RDBMS’?
3. Business Rules Resources

I have to apologize in advance. I'm in the middle of squeezing out a book chapter and haven't had time to tackle any of the BRE articles on my writing TODO list. So today I'm going to have to do what I usually hate to do -- become a blog echo chamber. The post I want to alert you to is one posted almost a year ago by Charles Young, entitled Microsoft's Rule Engine Scalability Results - A comparison with Jess and Drools. The article claims to show that Microsoft's BRE cannot be criticized about not implementing RETE properly based on MS's published test results. That's a long way to go for such an ephemeral point.

I don't recommend the article because of the claims it makes, but rather because it both demonstrates the use of three different BRE's through actual code samples, and it provides an approach to benchmarking and comparing BRE's for scalability and performance. I'm a firm believer in benchmarking BRE's based on your actual use cases rather than on abstract worst-case scenarios like Miss Manners. Read through this blog entry. Even if his approach is flawed, as I believe it is, he deserves full credit for publishing the full code to his test and opening it up to public scrutiny.

Technorati : drools, jboss rules, jess, microsoft, performance, scalability

Related posts:

1. Drools – Getting Your Feet Wet with Open Source
2. Drools 3.0 – Intriguing
3. Will JBoss Rules 3.1 Do for BRE’s What mySQL Did for RDBMS’?

OpenCyc 1.0 was quietly released today. It doesn't bill itself as a Business Rule Engine. Instead, from the FAQ:

What is OpenCyc?

OpenCyc is the open source version of the Cyc technology, the world's largest and most complete general knowledge base and commonsense reasoning engine. Cycorp, the builders of Cyc, have set up an independent organization OpenCyc.org, to disseminate and administer OpenCyc, and have committed to a pipeline through which all current and future Cyc technology will flow into ResearchCyc (available for R&D in academia and industry) and then OpenCyc.

What is included with the first release of OpenCyc?

Release 1.0 of OpenCyc will include the following:

• 6,000 concepts: an upper ontology for all of human consensus reality.
• 60,000 assertions about the 6,000 concepts, interrelating them, constraining them, in effect (partially) defining them.
• A compiled version of the Cyc Inference Engine and the Cyc Knowledge Base Browser.
• A suite of "RKF" tools for rapidly extracting knowledge from a domain expert (e.g., a physician or oil drilling specialist), tools which operate by carrying on a clarification dialogue with that individual; hence: tools for answering questions via English dialogue.
• Documentation and self-paced learning materials to help users achieve a basic- to intermediate-level understanding of the issues of knowledge representation and application development using Cyc.
• A specification of CycL, the language in which Cyc (and hence OpenCyc) is written. There are CycL-to-Lisp, CycL-to-C, etc. translators.
• A specification of the Cyc API, by calling which a programmer can build an OpenCyc application with very little familiarity with CycL or with the OpenCyc KB.
• A few sample programs that demonstrate use of the Cyc API for application development.

According to the docs, the inference engine is not RETE but instead a mix of specialized techniques. Also, said engine is not actually open source but instead distributed under a perpetually free license.

I'm going to reserve comment on the usefulness or quality of the software, but the ontology and the knowledge-base are probably worth some study.

Update 1: There are some comments that give a little bit more texture on Cyc from a IEEE mailing list.

Technorati : cyc, inferencing engine, knowledge base, opencyc

Related posts:

1. Framework Watch – HSE 2.0 Beta Released
2. GWT Libraries Released – XML, SOAP, Crypto, Amazon, etc.
3. Ext JS 1.1 Released

Turing Complete

First the easy question: when is a system Turing Complete? From the Wikipedia:

In computability theory, an abstract machine or programming language is called Turing complete, Turing equivalent, or (computationally) universal if it has a computational power equivalent to (i.e., capable of emulating) a simplified model of a programmable computer known as the universal Turing machine. Being equivalent to the universal Turing machine essentially means being able to perform any computational task - though it does not mean being able to perform such tasks efficiently, quickly, or easily.

So, what is a Turing machine? Again you can look at the Wikipedia, but in a nutshell, it's a tape with an infinite number of cells, a read/write head, a finite alphabet that the head can write to a cell, a finite number of 'states' that the machine can be in, and a finite number of rules that say things like "if you are in state 3 and the symbol on the tape is '0', the write a '1', move the head one cell to the right and change your state to 8."

While this may seem like a silly sort of machine (just try to write a word processor with it), it can do anything, given the right number of states and the right set of rules, that a von Neuman style of computer -- essentiall the sort of computer with which you are reading this blog -- can do. It's handy for proving certain theorems about computability and complexity. If you can write any Turing machine with your device, then it is said to be a Universal Turing Machine, i.e. if you can compute it with an algorithm, you can do it with your machine. It should be obvious that a BRE, even without First Order Logic, is a UTM. We can have a fact that tells us what our state is, another fact that tells us where our read/write head is, a set of facts that represent all of the cells we've seen so far, and our rules that govern how our state, head and cells are modified. Therefore a BRE is Turing Complete.

Of course there's a small fly in the ointment. Up above we said that a turing machine had an "infinite" number of cells. Of course your computer can't accomodate an infinite number of cells, since it has a finite amount of memory and storage. This is a rather important point, as it turns out. Anything program running on a modern computer with finite memory and storage is at best a Finit State Machine, basically the most primite of the interesting algorithms. But in principle, a BRE with infinite memory is Turing Complete.

First Order Logic

I'll try to stay a little non-technical here, mostly because all of my math text books are in storage. First Order Logic (or First Order Predicate Logic) is an extension of regular propositional logic with the addition of the universal and existential quantifiers. That means we can write conditions in our rules that are true for all facts of a particular type in our fact base or which are true for at least one fact. I've written before why having First Order Logic built into your BRE is a good thing. In short, you can work around your BRE not having it, but you'll lose the performance advantage of your rule execution algorithm in the process, such as RETE. And there are plenty of scenarios where you are looking at an event history of a customer or client that call for FOL.

You can see, however, that our simulation of a Turing machine above never called for a quantifier. Thus even a BRE without FOL is Turing Complete (if it has infinit memory, yada yada). I hope this explanation I hope this entry has been useful and not too geeky. Behind all of the hype behind BRE's, it is sometimes useful to understand what exactly they are and what they can do.

Related posts:

1. Executives Writing Spaghetti Code
2. Higher Order JavaScript
3. First Order Logic and Why Your BRE Should Have It

Let's suppose you are designing a system that decides whether or not to recommend prisoners for early release. Your facts inlcude the prisoners' criminal records, the details of their behavior behind bars, recommendations by prison officials and the district attourney, enrollment in treatment and rehabilitation programs, and mitigating circumstances such as aged parent or infirm spouses or children. You formulate the state's sentencing and release guidelines as business rules and implement them in your vendor platform of choice. When a feed of data comes from the department of corrections, you add it to your database and run it through your rule system which issues recommendations to a board charged with making the final decision. This board makes a judgement call based on the information you provide them.

When they look at your recommendation, they don't just want to see a yes or no, they want to see an explanation or justification. Something like "Prisoner Joe Smith, #54-56, is not recommended for release because he had at least 3 incidents of misbehavior in the last 6 months and was convicted of at least a Class C felony." Ideally, you'd want to spell out exactly what the issues were:

Prisoner: Joe Smith
Number: 54-56
Recommended for Early Release: No
Reasons: At least 3 incidents of misbehavior in the last 6 months (stabbing of guard, 4/5/2006; assault of fellow prisoner, 4/22/2006; contraband in cell, 5/17/2006) AND convicted of at least a Class C felony (Criminal Possession of Stolen Property, 6/11/2004)

Now as a board member, I can look at the prisoner and see that while he was convicted of a nonviolent crime, he looks to have been violent while in prison. If he had just been flagged for contraband, maybe I might consider releasing him.

OK, so justification is useful in this context, as it is in many other fields, like healthcare, insurance underwriting and claims, criminal science, etc. How do we produce a justification with our rule engine? The sad truth is that none of the commercial vendors of forward-chaining BRE's will do that for you automatically. You have to roll your own. One approach is to use the Fact Harvest pattern to construct and build a ReleaseRecommendation object. This fact would tell us whether a particular prisoner was recommended for release (true or false) and all of the conditions that contributed to this recommendation. It might contain a list of all of the misbehavior's over the last 6 months and the crime for which he was convicted.

Here we come to the first reason for using a wizard to write rules: if you are testing a particular condition in the 'IF' part of your rule and then updating your ReleaseRecommendation object with the details of that same condition in the 'THEN' part of the rule, you're essentially recapitulating the condition in the action part of the rule. Aside from being a pain in the neck, it's also a maintenance problem. Every time you update a condition, you have to make sure to update the corresponding action. If you don't, Charles Manson gets released by accident -- or, rather, because of your bug. It voilates the DRY principle (Don't Repeat Yourself). One way of handling this is to have your wizard generate the rule for you, with both the condition and action parts. Now you make one change and the rule is generated in a way that is guaranteed to maintain consistency.

The second reason for using a wizard has to do with the short-circuited OR of the title. If our rule above actually tests for

has had at least 3 incidents of misbehavior in the last 6 months OR is a drug addict

then what happens with our rule? If you've programmed much, you know that most languages use what is known as "short-circuited OR," i.e. if you are testing A OR B and A evaluates to true, then B is never tested. If you are depending on side effect of B (such as the building of a list of incidents, etc.), you can forget about it ever happening. The 'is a drug addict' part of the condition is never tested. To make things worse, if we are constructing a justification, we have to test in the action which parts of the OR clause in the condition actually evaluated to true. Imagine now a complex condition with lots of OR, NOT's, AND's and parentheses. Yuck.

A way around this mess is to break apart the OR clause into seperate rules and accumulating them in the ReleaseRecommendation object. The drawback, of course, is that now we have to create several rules where before we had only one. Again, generating these rules with a wizard is a more or less elegant way out.

If anyone has a niftier solution to justification or non-short-circuited OR, I'm more than happy to hear about it.

Update 1: Michael Chermside correctly points out in the comments that it would be more elegant to have a justification engine that worked directly with the BRE's internals. I agree. Unfortunately, none of the forward-chaining BRE's I've worked with have very good support for justification or supply hooks into their internals to support the building of a justification engine through AOP or something equivalent. If you work with an open source engine like JBoss Rules, you can build in your own justification logic.

Related posts:

1. OpenLaszlo Considered Harmful
2. CSS considered harmful by Ajaxian commentators?
3. Drools – Getting Your Feet Wet with Open Source

I've published an article over at the RealRules Blogzine on programmatic generation of rules, i.e. why and under what circumstances this makes sense. Check it out.

Related posts:

1. Prova – Son of Mandarax (Backward Chaining Rule Engines)
2. Your Business Rule System is a Fool
3. Happy New Year and Business Rule Books

He who knows not, and knows not that he knows not, is a fool - shun him.
He who knows not, and knows that he knows not, is a child - teach him.
He who knows, and knows not that he knows, is asleep - wake him.
He who knows, and knows that he knows, is a wise man - follow him.

The above is either an Arabic, Persian or Chinese proverb. I known not, so take your pick. But you could learn something from it for your business rules project.

Even after you've gone through the trouble of pulling your business rules out of existing applications and processes and centralizing them in a rule repository, you could still face problems of a logical or informational variety. You're business rule system may not have the necessary data, or it may have wrong data, or it may have the wrong kind of facts. The system may further not have all the right rules, or some bad rules or it may even come up with some unpleasant conclusions you didn't expect at all. In short, it may be a fool, a child, asleep or even a wise man, all at once, and even a few other choice epithets not included in the above proverb.

To make sure your business rule system behaves well, there are a few specific factors you should consider. I'll blog about them in the next few weeks, digging a little deeper and proposing some remedies along the way:

• Data (or Fact) Consistency - If you've got Joe being both 16 and 45 years old, you've got a problem. If that situation seems absurd to you, I tip my hat. You're clearly one of the chosen few whose data is consistent across all of your firms databases. The kinds of data inconsistencies can go much farther, such as a 5-year-old having a husband, or a dead man requiring a liver transplant. The solutions to these data consistency problems are for the most part very unglamorous techniques from the world of data modeling.

• The Wrong Kinds of Facts - If you've ever design OO software or designed a database, you know what I'm talking about. If your entities are the wrong kind, the wrong size, or incomplete, you will kill yourself writing awkward code. These entities and associated behaviors are like your nouns, verbs and adjectives. Pick the wrong words and your sentences are hard to write. If you capture only the final bill in a system and not the details of the transaction that incurred the bill, you may still be able to extrapolate the transaction information, but you have to work harder to do it. Too much hard work and the system limits rather than enables your ability to describe your business rules.

• Rule Consistency - What if given a clean, consistent set of facts, your rule set allows you to arrive at A and Not A -- "Joe is a minor" and "Joe is an adult?" With a few hundred or a thousand rules in your set, this problem may be hard to discover. (I'd add rule execution order independence to this category.) In fact, tracking down rule inconsistencies can be a bit tricky, with a couple of not quite satisfactory solutions available.

We'll leave off from things like decidability, as they probably don't have much bearing on BRE's, and cross product matching, as that's really more of a performance issue. I would advise you to be prepared, however, for your cauldron of rules and facts to sometimes deliver you some unexpected results. If you've done more than just 1 + 1 = 2, they can surprise you and perhaps grow past your own abilities. You've built a child and taught it to be a wise man -- or maybe it's just a fool that's mouthing off.

Related posts:

1. Happy New Year and Business Rule Books
2. Commandment IV: Keep Business Rules Simple Unto All The Rules of Your Life
3. Business Rules Management and Testing – Avoid the “Mystery Box”