Code Fusion - All purpose Symbolic Instruction Code

Deploying Cold Fusion to Google's App Engine and Big Table

2011-11-01T21:52:00.000-05:00

At the ColdFusion User Group in Minneapolis Wednesday November 3, 2011 I will be presenting on the topic of "Deploying Cold Fusion to Google's App Engine and Big Table".

Learn the basics of app engine, the deployment methodologies required to develop a Cold Fusion application with Open Bluedragon on gae, and the nuances of working with 'big table', Googles distributed object data store.

Visit ColderFusion.com to RSVP!

Looking forward to seeing you there

Update:
Those interested in the files (pdf, some code minus the jar files) can download it here. If you're looking for the magic bits for the working OpenBD version, see the pdf link for the enlist project link.

(Minneapolis) Application Architect / Developer Job - Jobs2Web, Inc.

2011-01-24T16:54:00.000-06:00

Location: Minnetonka, MN, USA

Are you looking for an opportunity to maximize your career with one of Minnesota’s fastest growing internet startups? Jobs2Web is one of the few venture backed Minnesota startups which made the Inc 500 list in 2009 and 2010 for our fast growth, and now we’re looking to expand our team of passionate professionals in the human capital software marketplace.

Jobs2Web helps employers maximize their interactive recruiting strategy using our recruitment marketing platform. The Jobs2Web platform helps major employers to attract, capture, and communicate with quality talent all while measuring recruiting effectiveness using our patent-pending solutions which dramatically reduce dependency on expensive and less effective sourcing mediums.

Working within our company will put you close to our award winning technology that combines the best of recruiting and marketing solutions including Search Engine Marketing (SEO), SEM (Pay-Per-Click), Social Networks and Social Media Marketing, and Email Marketing.

The Architect / Developer enhances and maintains the Jobs2Web Recruitment Marketing Platform and associated applications.

Specifically, the Application Architect / Developer role will have impact through these responsibilities:

Create and maintain the application architecture for the Jobs2Web platform, provide application architecture guidance to the development group and other Jobs2Web departments, and engage in hands-on development as required.
Participate in a dynamic team environment, including frequent team feedback, collaboration, training and code review.
Professionally communicate with our internal clients, including Client Services and Sales.
Demonstrate ability to manage multiple priorities in a fast-paced environment.

Desired Qualifications and Competencies:

Bachelors Degree or equivalent work experience, with at least 7 years of application development experience required.
Experience with Java required. Desired areas of expertise include
- Java EE 6
- Apache Solr/Lucene
- JMS
- JPA/Hibernate
- GlassFish 3
- JSF
- EJB 3.1
Experience in creation and maintenance of enterprise-grade application architecture, including SOA and messaging systems is required
Experience in SQL language and database technologies required.
Application of OO principles and use of best practices in web development required, with a strong background in application architecture.
ColdFusion MX experience desired, with experience using application frameworks and ColdFusion components.
Exposure to Adobe Flex 3 applications desired.
Competent in use of HTML, XML, CSS, and AJAX technologies.
Experience working in an Agile Scrum environment is desired.
Proficient in use of source control, defect management, and release management tools.
Consistently demonstrates strong team and individual performance.
Effective written and oral communication skills as well as strong interpersonal skills are essential.
Builds positive and effective relationships with individuals both internal and external to the organization.
Staffing/HR industry knowledge a plus.

Jobs2Web is an entrepreneurial employer that challenges and rewards all employees to succeed at every opportunity! Your talents are fully utilized as we are building this rapidly growing business. Our competitive market driven package includes a total rewards package of competitive pay, bonus, health insurance coverage, 401(k), career development, a very exciting marketplace and a really great work culture.

Nearest Major Market: Minneapolis
Job Segments: Architecture, ColdFusion, Computer, Creative, Database, Developer, Engineer, Engineering, Information Technology, Interactive Marketing, Java, Legal, Marketing, Patent, Product Development, Programmer, QA, Quality, Quality Assurance, Research, SEO, SQL, Technology, Web Design, XML

Apply Now

mxUnit Testing with Hudson CI Server @ CFUG Minneapolis

2010-11-03T00:44:00.000-05:00

I'll be speaking Wednesday Nov 3, 2010 @ the ColdFusion User Group in St. Paul/Minneapolis on mxUnit Testing with Hudson Continuous Integration Server.

I'll be talking about basic Assertions, as well as going into some advanced mocking topics with the new 2.x release of the mxUnit Framework.
The second part of the evening we'll look at the Hudson Server to help execute our Unit Tests and provide jUnit reports that show execution trends as well as some special surprises.

See you there!

JDBC Connection String and Unicode

2010-08-25T16:33:00.001-05:00

So we have a SQL Server 2005 JDBC Driver in production using the type 4 driver from Microsoft (Not the DataDirect Driver that ships with ColdFusion 8 Enterprise) and have it configured using the JDBC URL to specify the Host and some other parameters. You do this by going into your ColdFusion Administrator and under Data & Services - Data Sources - Entering in a new connection specifying a Name and choosing 'Other' as the Driver:
Format:

jdbc:sqlserver://xxx.xxx.xxx.xxxx;databaseName=MyDatabase;applicationName=Myclient;responseBuffering=adaptive;loginTimeout=5;lockTimeout=60000

Now the problem was that we had some other 'Connection String' information under the 'Advanced Settings' for this connection that we had configured to turn Unicode characters OFF for this dsn (thus using asii characters and varchar vs. unicode and nvarchar). This was for performance as we had varchar fields on columns, and when we use cfquery param, and cf_sql_varchar, the dsn connection setting drives whether you are sending 'varchar(8000)' (non-unicode) or a' nvarchar(4000)' parameter to the database.

Database Indexes: If your index has to convert this nvarchar type to a varchar type of the column, then it will more than likely skip using the index all together and you loose the benefit of having an index on that column in your database (i.e. an email address field).

With Profiling ON we were able to watch the requests come through by manipulating the jdbc URL connection string as follows:
sendStringParametersAsUnicode=false will send the cfquery param data through as a varchar(8000) field
sendStringParametersAsUnicode =true (which is the default) will send it through as a nvarchar(4000) field

What we noticed is that those types of queries with:
<cfqueryparam value="#arguments.email#" cfsqltype="cf_sql_varchar">
would not use the index.

Another thing we noticed was that we had specified sendStringParametersAsUnicode=false in the 'Connection String' area under Advanced Settings. This was NOT being taken into effect when sending parameterized queries to the db. We HAD to move this string into the JDBC URL portion in order for it to work. Bug in ColdFusion?? We're not sure, but we hope this helps someone out there.

This is how the dsn should look:

Executing Remote MXUnit Tests in CFBuilder/CFEclipse

2010-02-19T10:47:00.004-06:00

I had some interesting problems the other week while working on some Model-Glue (gesture) RemoteService requests. I had posted to the Model-Glue Google Group about a few things I ran into when using the AbstractRemotingService.cfc, and getting url variables to work properly through the service, which they have addressed (Thank you!).

Obviously you need to read the wiki documentation first.

When you Create your RemoteService.cfc that exists at the root of your application, which will be your remoting front controller if you will, you need to tell it where your main model-glue template file is located. (download the zip file from www.model-glue.com and find the modelglueapplicationtemplate directory for the sample RemotingService.cfc)

Once you have a simple working version up / running by calling /RemotingService.cfc?method=executeEvent&event=my.event&format=json and pass in the data you need to, then you need a good way to test it right? If you use MXUnit then you have more than likely configured your IDE to execute these tests.

Figure 1.0

ISSUE:
The thing I ran into was that I was using model-glue's event security where I could specify certain events to be 'authentication' needed etc. My remoting service was calling an event that required prior authorization (i.e. session.isLoggedin() or some other means). Now, when I executed a test for this event in mxUnit through the Eclipse plugin for mxunit, it would Fail the test and kept telling me I was unauthorized to execute this event etc, and return the source code for my login screen instead (which would be correct if I was not logged in). However in my unit Test, I was making sure to Call my AuthorizationFacade and log me in in the 'setup' method and verified that 'isLoggedIn()' returned true.

RESOLUTION:
So in my unit test I am doing a cfhttp 'post' (or now that they supported url variables better in the service a 'get' ) request to the RemoteService.cfc passing in my data. HOWEVER, I needed to also pass in the following through my cfhttp call:

<cfhttpparam ....="formfield" name="CFIDE" value="#cookie.CFIDE#">
<cfhttpparam ....="formfield" name="CFTOKEN" value="#cookie.CFTOKEN#">
<cfhttpparam ....="formfield" name="JSESSIONID" value="#session.sessionid#">

Once this was in place, the calls worked as expected, and returned proper json data.

Mpls CFUG - Active MQ, Cluster Messaging, Topics and

2009-11-20T09:41:00.005-06:00

I'll be presenting at the Minneapolis / St. Paul ColdFusion User Group again in December on Active MQ, Topics, Queues and Cluster Messaging for those interested. I'd love to pack the room before Christmas and show off all the exciting things Apache's Active MQ messaging can do with your ColdFusion applications, especially as it relates to cluster messaging. I'll be doing live demo's and code shows, as well as discussing how our company has used these techniques to help us address things like cache updating/invalidating, message driven events, maintenance queues, workflow etc.

Visit the ColderFusion.com website for more details but hope to see you out December 2, 1009, in the mean time, check out my new keyboard!

Updated: Download the code samples and presentation (+code) (requires activemq and blazeds deployed on tomcat as separate downloads) OR download the Entire thing with blazeDS, tomcat, amq and code in one large 200MB file

New Bluetooth Keyboard

2009-11-20T09:29:00.006-06:00

I broke down and got one of the new Logitech deNovo Edge Keyboards from Amazon this week. I'm just breaking it in but so far it's very sweet. The sleek lazer cut plexiglass and brushed aluminum frame is very 'edgy', and at 11 mm thick, you'll feel like your on a laptop even when docked. It's even got a little touchpad so you can throw your mouse out if you want (but I wouldn't recommend that as the left/right click buttons are not large enough to make it full time usable). It will be perfect for the home theater remote keyboard when you have a PC hooked up to your system.

As you could tell from a prior post, I was wearing the 'C' letter out on my last keyboard, and having a quieter keyboard with cool functionality was something I was definitely interested in.
If you're out shopping this Black Friday, maybe you'll get lucky and see one on sale, but Amazon has it for a discount right now.
Happy coding - !

Model Glue - Variable CTRLINST is undefined

2009-09-23T23:12:00.005-05:00

I'm playing around with Model Glue 3 (gesture) and event-types and message handling etc, and I ran across a nasty little productivity stopper.

I had defined an 'init' method which did nothing in my Model-Glue Controller for one of my controllers. I was getting a very obscure 'Variable CTRLINST is undefined.' message as a result.

Removing the 'init' method from the controller solved the problem, and the error message went away - leaving me to run my application once again.

I had become so accustomed to putting an 'init' method regardless in all cfc's that If I had not had some prior controller code to look at - I would have been sitting on this error for a long time.

Hope this helps someone out there - Maybe a more appropriate message here would help (framework guys).

100+ Downloads for CFProperty Inspector ColdFusion Builder Extension

2009-08-24T08:30:00.002-05:00

I made it past the 100+ downloads barrier for the ColdFusion Builder (pre-release) IDE with my CFProperty Inspector Extension. Thanks to all who've downloaded it and found it useful. Not bad numbers (10% download to view ratio) considering the IDE is not going to be released for another month or more.

Keep an eye open for more updates on the future as requests come in for the next version of the extension which allows you to write cfproperty tag meta data to existing components.

Search Engine Death Match (Pt. 3) MPLS CFUG

2009-07-23T11:06:00.005-05:00

I'll be speaking to the Minneapolis/St. Paul ColdFusion User Group (CFUG) on Aug. 6, 2009 about the Solr Integration in ColdFusion 9 (Centaur Beta). If you're in the area we'd love to see you, session starts at 6:30 - more information at ColderFusion.com.

Preview:
Search Engine Death Match: Solr (Luncene) vs. Verity

Search is King.
See how the 2 engines that ship with ColdFusion 9 stack up.
Learn more about the newly integrated Solr (based on Java Lucene) engine and how it stacks up against Verity.
Listen to real world stories about Solr's performance in a large scale Enterprise application.
Preview the features of Solr in Centaur Beta and determine if you're company could benefit from a better, more scalable, more reliable search solution.

Here is a preview of the session (pdf).

Hope you see you there!

cf_PropertyInspector Bolt Extension Released!

2009-07-09T09:44:00.008-05:00

Version 1 of the cf_PropertyInspector ColdFusion Builder/Bolt Extension for your IDE has been released on riaforge (pending).

Features Include:
For ease of Flex Integration, CFProperty tags are needed to have your methods available to ActionScript classes. Also complex webservices need this metadata defined for them. With existing components already created, having to go back in and add in all these cfproperty tags for all your 'accessors' is a pain. How about just right clicking -> 'Inspect CFC' and be done with it?? Best used if components exist under your webroot.

Features:
- Searching subdirectories
- Alpha sorting properties
- Wverwriting/keeping existing properties
- Lower case properties
- Exclusion of Methods (i.e getInstance, getBinaryNull etc)

Version 2 of this application will include the following:
- Backups of cfc's before modification
- Fine Tune properties before wizard completes
- Mapping support if your components are not located under your webroot.

Requirements:
ColdFusion 8+
ColdFusion Builder (Beta 2+) Build: 240677

Download NOW Available on riaforge.

Final Result!

Disable CFC Type Check Proof

2009-05-17T00:41:00.007-05:00

I usually question a lot of the things I hear, and this one I just had to set the record straight. I can stand being corrected on things when I'm proven wrong so I'm hoping this will come across as simply a clarification and not any kind of 'I told you so'.

But in the CFADmin in CF8 'Settings' there is a 'Disable CFC type Check' option that allows you to turn ON the ability to ignore the argument 'types' of CFC's. For example, if I have a component of type 'com.adobe.reactor' and I pass that to a method expecting a 'com.adobe.transfer', if this option is ON, then there would be NO type checking and the argument would be successfully passed in (I'd assuming something else would fail after that however).

At cfObjective this year it was stated at a session that if this option was turned ON, a datatype like 'numeric' would be able to accept a 'string' value as there would be no type checking. This is simply not true

Proof:

Visit your cfadmin and turn ON the 'Disable CFC Type Check' option and restart your server.
Run the following script:

You'll notice that this works just fine. Notice that I'm passing a string of 'hi' into the test method that is expecting a 'com.adobe.transfer' object. This works just fine with Disable CFC Type Checking ON as it simply treats the argument type as an 'ANY' type.

The CFAdmin states:

"When checked, UDF arguments of CFC type is not validated. The arguments are treated as type "ANY". Use this setting in a production environment only."

In this case 'com.adobe.transfer' is a UDF or user defined argument - I defined it in a directory of com/adobe/transfer.cfc.

However, when I change my test method to accept a 'numeric' value instead of this UDF value, you will see it will throw an exception:

"The NUM argument passed to the test function is not of type numeric."

This proves that ONLY UDF arguments are changed to 'ANY' types when 'Disable CFC Type Check' is turned ON in cfadmin (oh no I've gone cross eyed)

Imagine the security holes created by such an option if indeed the native datatypes were all treated as 'ANY'? SQL Injection could occur if you had typed the cfargument as a 'numeric' datatype. A malicious user could simply pass in a string value and bypass your initial line of attack - that being your cfargument data type. This would not be a good situation (more information on security and cfml see Jason Dean's blog at 12Robots.com).

The fact is, regardless of whether or not 'Disable CFC Type Check' is ON or OFF in your CFAdmin Setttings, native Datatypes still need to be adhered to, or an exception will be thrown.

A ColdFusion Developers Keyboard

2009-05-04T09:24:00.005-05:00

How can you tell what kind of developer you are by your Keyboard??

Check out my keyboard - Notice the 'C' key is all but worn right off - This has been from about 2 years of programming on this external keyboard - My previous laptop was the same way!

Just for fun, shoot me a picture of your keyboard - whether ColdFusion Programmer, .Net Programmer or dba (would your 'S' key be worn down from all the SELECTS?).

Enjoy

Clustered Application Needs Re-Loading? Here's How

2009-05-01T13:50:00.006-05:00

A question came up at last years cfObjective in Mike Brunt's presentation about reloading your application/model/ColdSpring config in mid-flight on a clustered application (your application has 2 or more CF instances running it).
We've spent some time investigating on our own and mostly just ignoring the issue and going through the tedious process of RDP-ing into the server and MMC-ing into each Servers Services Panel and restarting each instance and verifying they come back online - staggering them throughout a period of time until all cluster members were synced. This sucks.

Some of the proposed solutions included:
1.) Using ActiveMQ and a CF Gateway to Broadcast a 'reload' message and have the cluster members stagger their execution of it so they all don't reload at once (taking the app offline) via a thread sleep randomly based on the JVM instance etc.
2.) Appreloading via the url X times until you think you've HIT them all
3.) Use JRun's internal Web Server

I figured I'd give number 3 a try as I knew it would be the quickest and easiest to implement if it worked. Turns out it does.

What would happen is this: each CF instance has it's own dedicated IP and port that is running the application that can be accessed via a browser or over http (which can get automated but we'll get into that later).

The IP:Port is normally used if you wanted to get to a particular cluster members cfide to do some configurations etc i.e. 127.0.0.1:8303/cfide/administrator/index.cfm. However, if you browse to just 127.0.0.1:8303, you'll notice a nice directory structure that comes form your installation instance directory i.e. C:\JRun4\servers\{instancename}\cfusion.ear\cfusion.war if on Jrun.

So what we'll do is modify your jrun-web.xml file by adding the virtual root path to your application AND make sure if you use a separte jvm.config for your instance, that the default jvm.config is the same (more on that later).

Navigate to C:\JRun4\servers\{instancename}\cfusion.ear\cfusion.war\WEB-INF and open jrun-web.xml (if you don't have one you can try borrowing one from the default cfusion instance).

Add the following to the file and save:

<virtual-mapping>
<resource-path>/</resource-path>
<system-path>C:/webroot/approot</system-path>
</virtual-mapping>

This will make JRUN's internal web server root be your application's root directory.

Restart your CF instance.

If you are like most shops and use some kind of query parameter to execute your application reload procedure i.e. ?reloadsite=true or the like, then you can execute it via the ip:port?param and reload each cluster member by changing your ip:port.

Advanced Note: if you're like us and have a separate jvm configured for your CF instances (different than the stock jvm.config found in c:\JRun4\bin) then you'll need to make sure that
a.) either the jvm.config is the same as your CF Instance jvmconfig file
b.) make the jvm.config (default) the same as the CFInstance jvmconfig file, and then reconfigure that instance to use the 'jvm.config' file. This is key, as the internal JRun webserver will use the default jvm.config on startup, so if you have separated these jvm's out, reloading your app from ip:port will do nothing, as you have only just setup 2 separate jvm's running your application, and the one has NO knowledge of the other.

This is one method anyway - just make sure you have tested this thoroughly, especially if you have environment specific configurations that get loaded up depending on the url - as the IP address block may load up an unexpected set of configuration data from your application.

We'll be trying this out over the next few weeks and likely integrate it into the application in some kind of threaded reload routine if some configuration needs to be done mid-flight.

Enjoy!

Search Engine Death Match - Solr Wins (Pt. 2)

2009-05-01T08:49:00.004-05:00

Abstraction:
They key to the integration of Solr and removal of Verity in our model was to abstract the search functionality of the application. The process started by creating a 'CollectionManager' which held the methods that were needed to interact with the Verity Server through cfsearch and consolodated all calls to 'cfsearch' through this service.

Coldspring:
Once that was done and working fine - we were ready to dive into Solr and figure out the nuances of the product and ultimately create a 'SolrManager' which would replace the collectionManager. Since we were using ColdSpring, this was a mere class path change in the coldspring.xml and that was it (I love ColdSpring).

Solr Data-Config:
I'll get a more formalized presentation together about swapping out Verity in favor of Solr, but in general, our data was in a SQL Database vs. xml etc, so we used the 'data-config.xml' to basically come up with our main query that populates the index (each solr instance has a single instance or collection vs. verity where there is 1 verity and many collections) as well as delta and delete queries for updating and removal of the collection data through JDBC. This data-config.xml file is simply an xml file in which you write sql to select your columns based on some criteria.
Note: it's best to have created, lastmodified, id, active columns in your database table you wish to index, as this will help Solr in determining what data has changed, and what has been deleted. Hint: keep these queries simple and use a 'view' if your sql is complex at all, as it will hide the complexity, making the file easier to read, and abstract the data in a way that you can change the view w/o having to change the data-config.xml file each time. It's also worth pointing out that the 'server time' should be exactly the same on both your database and solr machine (if not on the same box) as Solr stores internally the last time it indexed your data in a value referenced in your sql as '${dataimporter.last_index_time}'.
** If you have a 'smalldatetime' value for lastmodified where sql will 'round' the seconds of the field, then you may run into issues where Solr won't pick up on changes that have happened since it's last_index_time as compared to your database timestamps etc.

Solr Schema:
This is basically the data definition in a single schema.xml of your data. You define your 'fields' which are your columns of data from the data-config and the datatypes etc. This is very flexible including the ability to 'group' multiple fields together as a single field (copyfield) as well as defining what is the default field that's searched on if none is specified in the field (i.e. search for 'MN jobs' could search job titles, job desc, locations etc).

Solr Config:
There was a one line change here just to tell Solr to use the data-config for it's data:

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst>
</requestHandler>

ColdFusion:
Wayne Graham has published an open source component out there at Ria Forge to assist in the low level cfhttp calls called SolManager. I've added this as a helper to my main 'SolrManager' component and i just call it to do the cfhttp calls to Solr and then I can do what I need with the data it returns.
Since the data is returned as xml, and the 'schema' of the data defines which columns / nodes are returned, I then have full control over the result. I convert the xml to a query using query new, xmlparse, and xmlsearch - and for our purpose as long as the query is in the same format as the one returned by a cfsearch through verity, the application won't know any difference.

Performance:
FAST! Solr does it's own caching and warming of searches and results and from my metrics, depending on the xml result set (which can EASILY be adjusted to limit the size of the xml as a '&rows=x' query string attribute in the search string) the performance is negligible if not better across the board for searches.

There's a lot more to discuss such as search syntax, replication, admin dashboard etc, but I'll leave a few stones un-turned for you to dive in and get your feet wet. They have good documentation on the site as well as an Extremely easy and efficient Jetty package that you can start up, execute a few commands on through a tutorial on their site, and get a feel for what to expect.

All in all, we were to the point where Verity was NO LONGER AN OPTION or a viable solution for our company and was causing nearly hourly baby sitting, so this solution was implemented JIT (Just in Time) and has saved us from feeling the effects of daily search stress.

Have fun and stay tuned for a full run down with code samples at MN CFUG soon.

Search Engine Death Match - Solr Wins

2009-04-30T09:41:00.002-05:00

It's now been over a month since we rolled out our Lucene / Solr powered search engine replacement for Verity in our application and let me sum it up this way: Zero maintenance.

Even though the OEM Verity licensing states 250K record max, we ran into a ceiling on a few of our machines when the number of Verity collections reached 82-88. We were doing constant rcadmin fixes to drop the thread counts for each collection down to 2 from 3 - but on each restart or at random points in time, the thread counts would return to 3. Instability and random collection switches were the main reasons we moved off the platform, and we'll never look back.

Solr took some configuration, but it was more than flexible enough for our needs, and with the delta updates and deletes, it made the maintenance near zero.

As far as parsing the resulting xml, that was done efficiently through coldfusion's xmlparse and xmlsearch to walk the nodes and return a 'like' verity recordset to the application. With ColdSpring we were able to swap out implementations easily and everything just worked. Great planning and execution resulted in a seamless rollout with Solr being run on RedHat on a dedicated machine.

I will most likely do a presentation at our next/upcoming CFUG (ColdFusion User Group) in Minneapolis/St.Paul in the near months about the transition and talk about integration points before ColdFusion 9 comes out and makes it all available with a single tag ;)

Long Live Open Source software

Query of Queries is Inconsistent (Adobe CFML Runtime)

2009-02-03T11:57:00.012-06:00

I've never been a big fan of a half baked solution, but I will acknowledge that coldfusion's query of queries has come a long way from it's beginning and does do a good job on simple queries to get data from an existing query.

However, there are times when it simply sucks at doing what you think it should do, especially when it comes to multiple tables and joins, as well as converting datatypes to be alike.

I'll show a few examples of it's inconsistencies and attempt to point out some problems with it's implementation.

Example 1: Q-of-Q (Query of Queries) and Proper variable scoping inside a cffunction.

When inside a cffunction you always want to fully qualify your variables as best practice. If it's an arguments, prepend the value with 'arguments.' etc. The same goes for Q-of-Q. But I will show you where q-of-q forces you to break it's own best practice advice by not allowing you to do this simple task.

I will be referencing the following 2 queries:

<cfset q = QueryNew('id,name','Integer,varchar')>
<cfset queryaddrow(q,1) >
<cfset querysetCell(q,'id',1,1) >
<cfset querysetcell(q,'name','test',1) >


<cfset q2 = QueryNew('id,age','Integer,Integer')>
<cfset queryAddRow(q2,1) >
<cfset querySetCell(q2,'id',1,1) >
<cfset querySetCell(q2,'age',33,1) >

This is a simple query that creates a single row of data

What I'm going to do is merge the results of these 2 queries (as one may have come from a cfdirectory, and the other from a cfsearch, or cfquery etc).

My method takes 2 query arguments and joins them - but for now I'll just return the value of the 1st argument 'qry'.

<cffunction name="merge" output="false" access="public" returntype="query" hint="">
<cfargument name="qry" type="query" required="true"/>
<cfargument name="qry2" type="query" required="true"/>
<cfset var Private=StructNew()>

<cfquery name="private.q" dbtype="query">
select *
from arguments.qry
</cfquery>

<cfreturn private.q />
</cffunction>

Resulting in a simple query returned - nothing special.

Now when i wish to merge the 2 queries - I would assume that I could do something like this then if 'arguments.qry' worked to reference the qry argument.

WRONG!
Query Of Queries syntax error.
Encountered "arguments . qry .. Incorrect conditional expression, Expected one of [like|null|between|in|comparison] condition,

Ok fine - I won't use proper best practices and just reference the query w/o the arguments proper fully scoped value.

<cffunction name="merge" output="false" access="public" returntype="query" hint="">
<cfargument name="qry" type="query" required="true"/>
<cfargument name="qry2" type="query" required="true"/>
<cfset var Private=StructNew()>

<cfquery name="private.q" dbtype="query">
select *
from qry, qry2
where qry.id= qry2.id
</cfquery>

<cfreturn private.q />
</cffunction>

Results: works fine

Example 2: Using QofQ (query of queries) to reference a column from the table using dot notation

The ColdFusion LiveDocs states the following:

Example

If a structure named A contains a field named B, which contains a table named Products, you can refer to the table with dot notation, as follows:

SELECT tape_ID, length
FROM A.B.Products;

The 'structure' in my case is 'arguments', the field name is 'id', and the table name is 'qry'. With that in mind:

A=arguments
B=id
Products=qry

Therefore you could deduce that the following is true:

SELECT id, name
FROM arguments.id.qry

Results:

Query Of Queries syntax error.
Encountered ". Incorrect Select List, Incorrect select column, arguments.id cannot be followed by '.'

However, you would normally reference this data as 'arguments.qry.id'.

Results:

Query Of Queries syntax error.
Encountered ". Incorrect Select List, Incorrect select column, arguments.qry cannot be followed by '.'

The kicker is - Neither of these work.

Killer WDDX - Beware!

2009-01-28T22:10:00.000-06:00

I came across a rather interesting 'feature' tonight while working on some logging and alerting processes. We have data in a structure that holds raw information about various processes that have occurred. In one of the structure keys there is an XMLObject (coldfusion parsed xml object). See the code snippet below and DO NOT run this on production equipment!!

<cfset stcStruct = StructNew()/>

<cfset xmlDoc = "<myxml><name>Kevin Penny</name></myxml>">
<cfset stcStruct.xml = xmlParse(xmlDoc)/>
<cfset stcStruct.id = 1>

<cfwddx action="cfml2wddx" input="#stcStruct#" output="tmp"/>

This takes a Structure, adds a key of 'xml' and places a parsed xml object in it's position. I'm then serializing the data into wddx via cfwddx as cfml2wddx.

I originally thought it was the date fields that I had in the structure, so I removed those, but the problem was the xml object's serialization.

Here's the error I get along with a partial dump:
java.lang.StackOverflowError

....
at java.beans.Introspector.instantiate(Introspector.java:1438)
at java.beans.Introspector.findExplicitBeanInfo(Introspector.java:410)
at java.beans.Introspector.(Introspector.java:359)
at java.beans.Introspector.getBeanInfo(Introspector.java:222)
at java.beans.Introspector.(Introspector.java:368)
at java.beans.Introspector.getBeanInfo(Introspector.java:222)
at java.beans.Introspector.(Introspector.java:368)
at java.beans.Introspector.getBeanInfo(Introspector.java:222)
at java.beans.Introspector.(Introspector.java:368)
at java.beans.Introspector.getBeanInfo(Introspector.java:222)
at java.beans.Introspector.(Introspector.java:368)
at java.beans.Introspector.getBeanInfo(Introspector.java:222)
at java.beans.Introspector.(Introspector.java:368)
at java.beans.Introspector.getBeanInfo(Introspector.java:222)
at java.beans.Introspector.getBeanInfo(Introspector.java:208)
at coldfusion.wddx.BeanSerializer.writeObject(BeanSerializer.java:48)
at coldfusion.wddx.WddxOutputStream.writeObject(WddxOutputStream.java:310)
at coldfusion.wddx.BeanSerializer.writeObject(BeanSerializer.java:120)
at coldfusion.wddx.WddxOutputStream.writeObject(WddxOutputStream.java:310)
at coldfusion.wddx.BeanSerializer.writeObject(BeanSerializer.java:120)
at coldfusion.wddx.WddxOutputStream.writeObject(WddxOutputStream.java:310)

I've submitted this as a bug to Adobe #75230.

Update to CFPROPERTY Inspector Utility

2009-01-14T09:32:00.001-06:00

I implemented a new UI for the CFPROPERTY Inspector as well as small bug fixes etc.

I used the JQuery tab UI which helps the flow of the steps being that the process is just one large form. Here are the new screen shots.

CFProperty Inspector on RiaForge

2009-01-06T09:59:00.000-06:00

I've released the CFPROPERTY Inspector on riaforge and google code for those interested in it.

This Utility is designed to introspect components and auto-create CFPROPERTY tags based on the function meta data.
Specify the mapping to your components, and choose which cfc's you want to have CFPROPERTY Tags defined on them as well as what properties of your components that you wish to have CFPROPERTY tags defined for. (for assisting in getting your components well defined for Flex integration etc.)

For Screen Shots and more checkout cfproperty.riaforge.org and to download the code visit Google Code here.

Let me know what you think.

Manipulating CFC Properties through getMetaData

2008-12-16T20:10:00.000-06:00

I started toying with the idea of creating a generic utility that would populate a cfc with ‘property’ as in cfproperty data and I have something work now – (this would make the compatibility between flex and the model that much better and more ‘well defined’)

Would this be of benefit to us – i.e. running our model through it which would then create all cfproperty tags for a given object based on the getter methods w/in the object – and determine datatype of string, numeric etc – based on the getter methods data etc –

What sort of things do I have to be careful of ??

Order of the cfproperty tags counts right – and Case sensitivity of the names obviously ??

So the process would either do it alpha – or by order that was returned by getMetaData on the object –

An interesting thing happens however when I’m doing this – I could actually build this into coldspring likely where these properties are autogenerated on the fly and never actually have to exist w/in the cfc (written back to the cfc)

The strange thing happened was this:

Once I do a create object on a cfc – I can dump it and it shows that there are NO properties defined in the meta data – which is good

So I run it through my process where I’m introspecting the cfc and create this ‘properties’ key and append the array to this key.

This works well and now a side by side before and after shows the left object with NO properties and the right with Properties – great.

However the next time I run it – the left side also shows that the object now HAS properties –

This is strange – b/c cfproperty tags don’t actually exist in the cfc – but yet I’m able to fake cf (caching likely) into thinking / assigning these properties to the object.

For Example:

Here my Job Object has NO cfproperty tags:

I’m going to make ColdFusion ‘think’ it has these properties defined on it by manipulating the metadata about the object.

Simple App/UI

There are No properties on the left – On the right below, shows the ‘manipulated’ meta data based on the getter methods in the object.

Now after I run this again – you see both left and right are identical (cached metadata about the object)

The left side ‘thinks’ that the object now has these properties available to it- which if called or introspected by Flex, they should be available.

This feels a bit like cheating, but if it doesn’t work – we can always complete this thought by having the process ‘write’ the cfproperty tags back to the object.

Interesting huh?

JRun4 wsconfig Error Loading jvm.dll

2008-11-10T20:18:00.000-06:00

Recently I attempted to patch our JRun4 servers with updater 7. Unfortunately after the fact, I found out that there is no 64 bit installer for it on windows. "64-bit Support – JRun4 Updater 7 provides 64-bit support on Solaris-SPARC. Other 64-bit operating systems including Windows 64-bit and Solaris-x86 64-bit are not supported."

So after a quick panic attack and reverting to the backed up copy (yes I did back up some recommended things) I felt that I was out of the woods. Twas not the case.

The next time I went to add a site to the Coldfusion instance through wsconfig command line - I received a dreaded "Error Loading jvm.dll". So I could no longer add sites and associate them to the webserver etc.

Here was the old version:
wsconfig.exe -ws IIS -site "jobs-jobs.com" -coldfusion -server "cfcluster" -v -ws64 -norestart
(Error Loading jvm.dll)

The new version of this has be using another technique using a java properties file instead. :
java -jar wsconfig.jar -f c:\j2w_IISconfig\jrun.properties

This called a jrun.properties file that had the arguments that i was executing directly to the wsconfig.exe prior

The Properties looks somehing like this:

ws=IIS
site=jobs-jobs.com
ws64=true
norestart=true
server=cfcluster
coldfusion=true

Bingo - Hope this helps some other soul out there - as it stopped me in my tracks for a bit.

Separating the JVM config files - A Cluster Conversation

2008-10-30T10:16:00.000-05:00

I had an interesting discussion with a fellow who was looking to solidify his stand with his boss on jvm configuration separation and the need for it. Below is the IM conversation the ensued - I think it helps to show some of the various points that could be made arguing FOR having the separation of the config files per logical JVM (names changed to protect the innocent and guilty)

[09:20] cluster_help: Hello, i was just talkin to Jared about some jvm clustering stuff and he said you were way smarter than him on that stuff
[09:20] kpenny: sure ok -
[09:20] cluster_help: we're setting up a few clustered instances for a large app on a single vmware box
[09:21] kpenny: cool - 32 bit?
[09:21] cluster_help: i know i should have each instance running its own jvm config so we (yeah, 32) can lower the max heap etc so all the instances aren't trying to use ALL the available memory
[09:22] cluster_help: i can find lots of posts and articles on HOW. Do you know of any that mention WHY?
[09:22] cluster_help: need to help my infrastructure guy convince his boss
[09:22] kpenny: so 'why' separate the instances?
[09:22] cluster_help: load balance and failover
[09:23] cluster_help: its a custom installation of our main app for a large, high traffic customer
[09:23] cluster_help: we're isolating them from all our "regular" customers
[09:23] kpenny: yup - both excellent reasons - keep the app UP -
[09:24] kpenny: so you'll have x vm's
[09:24] cluster_help: yea
[09:24] cluster_help: and a gig of RAM to allocate among them
[09:25] kpenny: i also recommend if you have say 8 processor box - not to give each vm virtual procs that add up t o 8
[09:25] kpenny: as the ESX host for example would need cpu 2
[09:25] cluster_help: right, it is an ESX host
[09:25] kpenny: k - we noticed that when we dedicated 2 cores per vm
[09:25] kpenny: we were cpu strapped -
[09:26] kpenny: also you have to think about memory utilization - if you have 4g ram on each vm
[09:26] kpenny: and only a single jvm - likely the jvm max is 1.2G -
[09:26] kpenny: and the os isn't going to use 2.8g ram
[09:26] cluster_help: u know of anything published anywhere that describes WHY its a good idea to lower the max (allocate memory judiciously) across each instance
[09:26] kpenny: so you're also leveraging your hardware better
[09:26] cluster_help: right
[09:27] kpenny: describe - lower the max -
[09:27] cluster_help: when you create new instances, they all use the same jvm.config, so they all get allocated whatever the default is
[09:27] kpenny: right - separating them gives you more control that's all
[09:28] kpenny: and also allows you to do custom class paths
[09:28] kpenny: etc, as your apps grow -
[09:28] cluster_help: so you create a sep jvm.config for each instance, and lower the max heap size, so they all aren't trying to use all the available
[09:28] kpenny: and/or as you allocate resources - you have more granular control
[09:28] kpenny: well even though you're using the same jvm.config
[09:28] kpenny: its still a separate unique jvm
[09:28] kpenny: that they are running w/in -
[09:28] kpenny: which can be a misconception -
[09:29] kpenny: the jvm.config just configures the values for each of the individual jvms in use -
[09:29] kpenny: by separating out the configs, then you're able to have more control over 1 jvm out of the 4 say
[09:29] kpenny: vs. one change in one file for 'all' jvms
[09:30] cluster_help: roger that. but other than the obvious, if you have 8 instances and 8 x maxheap ends up being more than the available memory on the host, don't you risk having a problem?
[09:31] kpenny: yes
[09:31] kpenny: that's when you'd need either 64 bit - or multiple machines - etc -
[09:31] kpenny: but yeah you never wanna allocate 2 much
[09:31] cluster_help: so why my infrastructure guy's boss is looking for is a technote or a blog post or something that says you SHOULD lower the max in the .config
[09:31] kpenny: but with separation of the configs you can control that -
[09:32] kpenny: well really the max should be determined by the app
[09:32] cluster_help: yeah. Like I told my boss, I know HOW to do it. (I did it yesterday on my laptop!)
[09:32] kpenny: so if you have large mem requirements -
[09:32] kpenny: then it should be higher - i guess i'd say there's no set rule - just
[09:32] kpenny: that depending on performance tsting what's best for the app at a given time -
[09:33] kpenny: i.e. if you have multiple applications using frameworks and cs etc -
[09:34] cluster_help: here's the background on my company - the dev manager and the infrastructure manager constantly bicker and fight little political battles. the infrastucture guy and me have about 10 years of exp in each of our areas of expertise
[09:34] cluster_help: but both our bosses want us to prove via external sources what we recommend
[09:34] kpenny: ok -
[09:35] cluster_help: and I am saying, and my infrastructure guy concurs, that we need sep config files for each instance in the cluster, (for granular control) of the max memory allocations
[09:35] kpenny: whats your max set to now - ? and is there a memory problem i.e. using 2 much memory
[09:35] cluster_help: but we need to show them WHY we recommend that
[09:35] cluster_help: the cluster isn't even in production yet, this is a first time set up
[09:35] cluster_help: we're actually clustering the staging env first for load testing
[09:36] cluster_help: then we'll do it in production
[09:36] cluster_help: we've never clustered or had very good front end load balancing here
[09:36] kpenny: i guess one more good point to throw in the mix is this
[09:37] kpenny: some apps perform better on different jvm versions - heck even sun vs. ibm etc -
[09:37] kpenny: so - with that - w/o granular control you're stuck using the same version that all other apps are using
[09:37] kpenny: in a single config -
[09:37] kpenny: vs what we went through were were we had an app that uses coldspringn and
[09:37] cluster_help: right
[09:38] kpenny: benefited from the bug fix in 1.6.v10
[09:38] kpenny: so we were able to just change that config and run the app
[09:38] kpenny: others can use stock or whatever
[09:38] cluster_help: this is going to be 3 or 4 instances of a single app deployed as an .ear
[09:38] kpenny: so 3 jvm's
[09:38] kpenny: clustered?
[09:39] kpenny: i guess i'd argue for it having the separation b/c it's easier to do it from the start -
[09:39] kpenny: and you really can't predict how things will be configured 5 months from now -
[09:39] kpenny: it's not a big deal to separate them -
[09:39] kpenny: and you can always do it at a later date if needed 2
[09:40] cluster_help: yeah
[09:40] cluster_help: but as far as you know, there isn't anything out there in a kb article or a tech note that says WHY you should do it that way, its just kind of a common knowledge/common sense thing?
[09:41] kpenny: well i know for dev its essential
[09:41] kpenny: for reasons like control over debugging ports
[09:41] kpenny: and you're likely working with multiple applications each with their own requirements
[09:42] kpenny: so for dev i'ts a no brainer - but if you have production separated out where x apps are all on y vm's etc.
[09:42] kpenny: and the other ones are on separate ones and they never cross over
[09:42] kpenny: then you can likely get away with stock setup
[09:42] kpenny: however -
[09:42] kpenny: we have some 'maintenance' things that happen with the app
[09:42] kpenny: and we found it most beneficial to have this as a separate instace - even though tis the same app -
[09:43] kpenny: but run schedules through it -
[09:43] kpenny: for batch processes etc - off on it's own - which
[09:43] kpenny: has it's own requirements for mem etc -
[09:43] kpenny: and we benefited from having them separate configs
[09:43] kpenny: also -
[09:43] kpenny: if you wanted to setup a scheduler 'instance' where all it did is kick off
[09:43] kpenny: cf schedules to your app
[09:43] kpenny: you don't wanna allocate 512 mb ram for that guy
[09:44] kpenny: likely small 128 via config
[09:44] kpenny: and run it alongside or whatever -
[09:44] kpenny: so there are many use cases - and they all depend on your preference
[09:44] kpenny: debugging locally with multiple instances w/o having separate jvm configs is impossible
[09:44] kpenny: so that's #1 reason I started doing it -
[09:45] kpenny: then after that i noticed all these other possibilities
[09:45] cluster_help: right, but if we're doing the multiple instances so we can have load balance and failover, each instance should be set to the max we can afford given the machines total available RAM and taking into account the OS's needs and stuff like that?
[09:45] cluster_help: in a production environment?
[09:46] kpenny: if you're app performs best with the max at that level -
[09:46] kpenny: some apps won't -
[09:46] kpenny: and typically if you allocate over what 1.8gb?
[09:46] kpenny: the service won't start?
[09:46] cluster_help: so really the load testing we're going to do will let us know if the settings will work
[09:46] kpenny: yup
[09:46] cluster_help: i think its 1.2gb.
[09:46] kpenny: right ok -
[09:47] kpenny: yeah try it at various levels -
[09:47] cluster_help: cool - hey, thanks for the info man, it really helps
[09:47] kpenny: np - ! best of luck -
[09:47] cluster_help: thanks, i appreciate it

Lowering the Number of Verity Threads on Servers with Large Number of Collections

2008-10-23T12:47:00.000-05:00

With our 55+ collections - we have some suggested ways to better manage the amount of threads that Verity has to manage. Below is the suggestion direct form Adobe Tier 3 ColdFusion/Jrun support consultants.

So, the suggested workaround to prevent the issue of collectionnames being ‘switched’ is to lower the number of threads assigned to each collection in Verity. Verity support had indicated this is an issue they are familiar with. They see it when the ratio of (total number of threads/CPU) gets high. Your total threads is about 165 now (55 collections * 3 threads). They suggest lowering the threads per collection to 2. Note, it is generally not recommended you use less than 2 threads per collection. One thread per collection may work for collections that are not heavily used.

You can update the threads per collection by running Verity RCAdmin. I have pasted below the output from my updating a “linuxarchive” collection. You will run the rcadmin from your verity_root\k2\_nt40\bin directory in a DOS window.
1. type in rcadmin
2. at rcadmin prompt type in indexattach
3. type in collectionname for index alias
4. enter ‘c’ for collection
5. enter the search server alias (ColdFusionK2_server1)
6. modify type is update = 0
7. index state is online = 2
8. Threads is the update you are making. All collections have 3 threads currently, which you should change to 2.
9. save changes=y
You should do this for each of your collections. Once you have updated all the collections you will need to stop and restart the Verity search service.

I have recently done this on our beta environment and the numbers fell from about 270 threads (task manager - Processes - view - Select columns - thread count) to 210 or so. We'll see how this effects overall management of the collections. This is not necessary on servers with lower number of managed collections.

64 Bit Microsoft Windows ODBC Drivers (Excel)

2008-10-21T11:13:00.000-05:00

If you've moved from a 32 bit environment to a 64 bit one and are using the Microsoft ODBC drivers for things like Excel (reading a worksheet) and Access, I'm afraid you're out of luck (this includes even text files etc).

I just found out the hard way that code written to pull data using the 'jdbc:odbc:Driver={Microsoft Excel Driver (*.xls)}' no longer works in Windows Server 64 bit edition. Microsoft apparently feels that they don't need to provide 64 bit drivers at all for Office connectivity. Which I guess means that we as developers and software designers should no longer support Office, which is OK by me.

I've hated working with Excel to import/export data to / from, and have had nothing but trouble either direct to the Database with OPENDATASOURCE and having to move uploaded files to the database server, or simply on the server through a jdbc odbc call.

I'm fine with Eliminating Excel as a valid import option - I just need to find / create a fast alternative that will do CSV or other formats (not using CF to do the heavy work with those files as they can be large). There is one ray of hope - in an opencsv format found here.