Search Engine

Oracle Endeca Commerce Installation On Openshift

2014-08-06T02:53:00.000-07:00

Oracle Endeca Commerce Installation On Openshift

Components involved in Oracle Endeca Commerce

o Oracle Commerce Guided Search

o Oracle Commerce Experience Manager

Oracle Commerce Guided Search and Oracle Commerce Experience Manager are the most effective way for your customers to dynamically explore your site and find relevant and desired items quickly.

OpenShift?

OpenShift is Red Hat's Platform-as-a-Service (PaaS) that allows developers to quickly develop, host, and scale applications in a cloud environment. With OpenShift you have choice of offerings, including online, on premise, and open source project options.

Steps involved in install Oracle Endeca Commerce on Openshift

Creating a gear on Openshift

a. Click on Add Application in Openshift Application page

b. Scroll down and select “Do-It-Yourself 0.1”

c. In “Public URL” enter “demoendeca”

d. In “Gear” select ATG

e. Click Create Application

f. For Remote Access of gear use ssh provide.

Installing Oracle ToolsAndFramework on gear

Note: Make sure you have unzip ToolsAndFramework installer at “"$OPENSHIFT_DATA_DIR”

We need to do a silent installation on Openshift as it’s doesn’t support display.

1. Create a file “command.sh” in "$OPENSHIFT_DATA_DIR" folder

2. Save below lines to it

"$OPENSHIFT_DATA_DIR"cd/Disk1/install/runInstaller.sh ORACLE_HOSTNAME=127.156.121.31 -invPtrLoc "$OPENSHIFT_DATA_DIR"test/oraInst.loc -silent -waitforcompletion -responseFile "$OPENSHIFT_DATA_DIR"cd/Disk1/install/silent_response.rsp ORACLE_HOME_NAME=ToolsAndFrameworks ORACLE_HOME="$OPENSHIFT_DEPLOYMENTS_DIR"/endeca/ToolsAndFrameworks/ PASSWORD_VALIDATION="$1"

Make sure ORACLE_HOSTNAME is “OPENSHIFT_DIY_IP”, which can be found in .env folder of gear or use command “env | grep IP” and it will show OPENSHIFT_DIY_IP value.

3. Create a test directory at "$OPENSHIFT_DATA_DIR"

4. Create “oraInst.loc” file and add location “inventory_loc=/var/lib/openshift/12d13141516m17p18s1920a2/app-root/data/oracleInst”

Note: “12d13141516m17p18s1920a2” is gear Home location, which can be found in .env folder of gear or use command “env | grep HOME”.

5. Now run “command.sh”

sh command.sh admin

“admin” is used a password for login to Endeca Experience Manager.

[endeca-demo.ose.dev.global.demo.org data]\> sh command.sh admin

Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB. Actual 7999 MB Passed

Preparing to launch Oracle Universal Installer from /tmp/OraInstall2014-07-15_01-42-20PM.

Please wait ...Oracle Universal Installer, Version 12.1.0.1.0 Production

You can find the log of this install session at:

/var/lib/openshift/12d13141516m17p18s1920a2/app-root/data/test/logs/installActions2014-07-15_01-42-20PM.log

For this silent installation it is assumed that you have read and accepted the license agreement located in /var/lib/openshift/12d13141516m17p18s1920a2/app-root/data/cd/Disk1/install/./license.txt. If this is not the case, please cancel this installation

and read the licence agreement before resuming the installation.

.................................................................................................... 100% Done.

Loading Product Information

............................................................... 100% Done.

Analyzing dependencies

............................................................... 100% Done.

-----------------------------------------------------------------------------

Summary

Global Settings

Source: /var/lib/openshift/12d13141516m17p18s1920a2/app-root/data/cd/Disk1/stage/products.xml

Oracle Home: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework (ToolsAndFrameworks)

Installation Type: Complete Installation

Product Languages

English

Space Requirements

/var/lib/ Required 529MB : Available 193.80GB

/tmp/ Required 226MB (only as temporary space) : Available 193.80GB

New Installations (2 products)

Oracle Commerce Tools and Frameworks With Experience Manager 11.0.0.0.0

Reference Applications 11.0.0.0.0

-----------------------------------------------------------------------------

Installation in progress (Tuesday, 05 August 2014 13:42:26 o'clock BST)

............................................................... 44% Done.

............................................... 78% Done.

Install successful

Linking in progress (Tuesday, 05 August 2014 13:42:43 o'clock BST)

Link successful

Setup in progress (Tuesday, 05 August 2014 13:42:43 o'clock BST)

.........Checking Workbench Environment Settings

Using ENDECA_TOOLS_ROOT: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0

Using ENDECA_TOOLS_CONF: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace

Using CATALINA_HOME: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/apache-tomcat-7.0.42

Using CATALINA_BASE: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace

Using JAVA_HOME: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/j2sdk

Using JRE_HOME: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/j2sdk

Using WORKING_DIR: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/state

Using CATALINA_OPTS: -Xms256m -Xmx1024m -XX:MaxPermSize=512m -Dorg.apache.el.parser.SKIP_IDENTIFIER_CHECK=true -Dcom.endeca.manager.endecaroot=/var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace

Using JAVA_OPTS: -Djava.security.auth.login.config=/var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/conf/Login.conf -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,getLock -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,addLock -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,isGrantable -XX:+UnlockDiagnosticVMOptions -XX:-DisplayVMOutput -d64

Starting Workbench (background process)

.Checking Workbench Environment Settings

Using ENDECA_TOOLS_ROOT: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0

Using ENDECA_TOOLS_CONF: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace

Using CATALINA_HOME: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/apache-tomcat-7.0.42

Using CATALINA_BASE: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace

Using JAVA_HOME: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/j2sdk

Using JRE_HOME: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/j2sdk

Using WORKING_DIR: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/state

Stopping Workbench

.. 100% Done.

Setup successful

Saving inventory (Tuesday, 05 August 2014 13:43:55 o'clock BST)

Saving inventory complete

Configuration complete

End of install phases.(Tuesday, 05 August 2014 13:44:15 o'clock BST)

The installation of Oracle Commerce Tools and Frameworks With Experience Manager was successful.

Please check '/var/lib/openshift/12d13141516m17p18s1920a2/app-root/data/test/logs/silentInstall2014-07-15_01-42-20PM.log' for more details.

Install CAS on gear

Note: Make sure you have unzip CAS installer at “"$OPENSHIFT_DATA_DIR”

Run below command

sh cas11.0.0-Linux64_173709REL.sh --target /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/

Note: target is “OPENSHIFT_DEPLOYMENTS_DIR” which can be found in .env folder of gear or use command “env | grep OPENSHIFT_DEPLOYMENTS_DIR” and it will show OPENSHIFT_DEPLOYMENTS_DIR value.

[endeca-demo.ose.dev.global.demo.org data]\>

sh cas11.0.0-Linux64_173709REL.sh --target /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/

Content Acquisition System 11.0.0 install for x86_64pc-linux

Oracle and Java are registered trademarks of Oracle and/or its affiliates.

Other names may be trademarks of their respective owners.

UNIX is a registered trademark of The Open Group.

This software and related documentation are provided under

a license agreement containing restrictions

on use and disclosure and are protected by intellectual property

laws. Except as expressly permitted in your

license agreement or allowed by law, you may not use, copy,

reproduce, translate, broadcast, modify, license,

transmit, distribute, exhibit, perform, publish, or display any

part, in any form, or by any means. Reverse engineering,

disassembly, or decompilation of this software, unless required

by law for interoperability, is prohibited.

The information contained herein is subject to change without

notice and is not warranted to be error-free.

If you find any errors, please report them to us in writing

U.S. GOVERNMENT END USERS: Oracle programs, including any

operating system, integrated software,

any programs installed on the hardware, and/or documentation, delivere

d to U.S. Government end users are “commercial computer

software” pursuant to the applicable Federal Acquisition

Regulation and agency-specific supplemental regulation

s. As such, use, duplication, disclosure, modification, and

adaptation of the programs, including any operating system,

integrated software, any programs installed on

the hardware, and/or documentation, shall be subject to

license terms and license restrictions applicable to the

programs. No other rights are granted to the U.S. Gove

rnment.

This software or hardware is developed for general use in a

variety of information management applications.

It is not developed or intended for use in any inherently d

angerous applications, including applications that may create

a risk of personal injury. If you use this software

or hardware in dangerous applications, then you shall

be responsible to take all appropriate fail-safe, backup,

redundancy, and other measures to ensure its safe use.

Oracle Corporation and its affiliates disclaim any liability for

any damages caused by use of this software or hardware in dangerous applications.

This software or hardware and documentation may provide

access to or information on content, products,

and services from third parties. Oracle Corporation and its

affiliates are not responsible for and expressly disclaim all

warranties of any kind with respect to third-party

content, products, and services. Oracle Corporation and it

s affiliates will not be responsible for any loss, costs, or damages

incurred due to your access to or use of

third-party content, products, or services.

Verifying archive integrity...

All good.

Uncompressing Content Acquisition System 11.0.0 for x86_64pc-linux

................

Please enter the port for the CAS service. The typical default is 8500.

:2100

Please enter the shutdown port for the CAS service. The typical default is 8506.

:18506

Creating the CAS service workspace.

Configuring the CAS host and port in the workspace located at /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/endeca/CAS/11.0.0/bin/../../workspace

Validating the ENDECA_TOOLS_ROOT location for the Workbench installation.

is not a full path.

Please enter a full path to an existing directory.

: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/

Validating the ENDECA_TOOLS_CONF location for the Workbench installation.

is not a full path.

Please enter a full path to an existing directory.

: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace

ENDECA_TOOLS_ROOT is set to: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/.

ENDECA_TOOLS_CONF is set to: /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace.

Please enter the fully qualified name, including domain information, of the CAS server.

:127.156.121.31

Installing casconsole.xml into /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/conf/Standalone/localhost.

Installing casconsole-infocenter.xml into /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/conf/Standalone/localhost.

Installing war files into /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0//server/webapps.

Installing casconsole.properties into /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/conf.

Installing CAS console into /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/conf/ws-extensions.xml

Backing up file /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/conf/ws-extensions.xml to

/var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/conf/ws-extensions.xml.07-15-2014-13-58

Installing CAS console into /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/conf/ws-mainMenu.xml

Backing up file /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/conf/ws-mainMenu.xml to

/var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0/server/workspace/conf/ws-mainMenu.xml.07-15-2014-13-58

Installing casStubs.jar to /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0//reference/discover-data-cas/lib/casStubs.jar

Installing casStubs.jar to /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0//deployment_template/app-templates/common/config/lib/java/casStubs.jar

Installing casStubs.jar to /var/lib/openshift/12d13141516m17p18s1920a2/app-deployments/toolsandframework/11.0.0//reference/media-mdex-cas/lib/casStubs.jar

CAS installation is complete.

[endeca-demo.ose.dev.global.demo.org data]\>

Installing MDEX on gear

Note: Make sure you have unzip MDEX installer at “"$OPENSHIFT_DATA_DIR”

Run below command

sh mdex6.5.0-Linux64_781329REL.sh --target /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/

[toolsandframeworks-demo.ose.dev.global.demo.org data]\>

sh mdex6.5.0-Linux64_781329REL.sh --target /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/

MDEX 6.5.0 install for x86_64pc-linux

Oracle and Java are registered trademarks of Oracle and/or its affiliates.

Other names may be trademarks of their respective owners.

UNIX is a registered trademark of The Open Group.

This software and related documentation are provided under a license

agreement containing restrictions on use and disclosure and are

protected by intellectual property

laws. Except as expressly permitted in your license agreement

or allowed by law, you may not use, copy, reproduce, translate,

broadcast, modify, license, transmit, di

stribute, exhibit, perform, publish, or display any part, in any form,

or by any means. Reverse engineering, disassembly, or

decompilation of this software, unless req

uired by law for interoperability, is prohibited.

The information contained herein is subject to change

without notice and is not warranted to be error-free.

If you find any errors, please report them to us in writing

U.S. GOVERNMENT END USERS: Oracle programs, including

any operating system, integrated software, any programs

installed on the hardware, and/or documentation, delivere

d to U.S. Government end users are “commercial computer

software” pursuant to the applicable Federal Acquisition

Regulation and agency-specific supplemental regulation

s. As such, use, duplication, disclosure, modification, and

adaptation of the programs, including any operating system

, integrated software, any programs installed on

the hardware, and/or documentation, shall be subject to

license terms and license restrictions applicable to the

programs. No other rights are granted to the U.S. Gove

rnment.

This software or hardware is developed for general use in

a variety of information management applications. It is

not developed or intended for use in any inherently d

angerous applications, including applications that may

create a risk of personal injury. If you use this software

or hardware in dangerous applications, then you shall

be responsible to take all appropriate fail-safe, backup,

redundancy, and other measures to ensure its safe use.

Oracle Corporation and its affiliates disclaim any li

ability for any damages caused by use of this software or

hardware in dangerous applications.

This software or hardware and documentation may

provide access to or information on content, products,

and services from third parties. Oracle Corporation and its affi

liates are not responsible for and expressly disclaim all

warranties of any kind with respect to third-party content,

products, and services. Oracle Corporation and it

s affiliates will not be responsible for any loss, costs,

or damages incurred due to your access to or use of

third-party content, products, or services.

About to extract 526508 KB in

/var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/.

Available disk space in

/var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/: 191336352 KB.

Uncompressing MDEX 6.5.0 for x86_64pc-linux.........................................

Configuring Endeca MDEX 6.5.0 for x86_64pc-linux .....

Configuration of Endeca MDEX 6.5.0 for x86_64pc-linux done

Please run the following command to set your environment variables:

Bourne, Bash or Korn:

source /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/endeca/MDEX/6.5.0/mdex_setup_sh.ini

csh or tcsh:

source /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/endeca/MDEX/6.5.0/mdex_setup_csh.ini

[toolsandframeworks-demo.ose.dev.global.demo.org data]\> source /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/endeca/MDEX/6.5.0/mdex_setup_sh.ini

Installing PlatformServices on gear

Note: Make sure you have unzip PlatformServices installer at “"$OPENSHIFT_DATA_DIR”

Run below command

sh platformservices11.0.0-Linux64_781523REL.sh --target /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/

[toolsandframeworks-demo.ose.dev.global.demo.org data]\>

sh platformservices11.0.0-Linux64_781523REL.sh --target /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/

Endeca Platform Services 11.0.0 install for x86_64pc-linux

Oracle and Java are registered trademarks of Oracle and/or its

affiliates. Other names may be trademarks of their respective owners.

UNIX is a registered trademark of The Open Group.

This software and related documentation are provided under a license

agreement containing restrictions on use and disclosure and are

protected by intellectual property laws. Except as expressly permitted

in your license agreement or allowed by law, you may not use, copy,

reproduce, translate, broadcast, modify, license, transmit, distribute,

exhibit, perform, publish, or display any part, in any form, or by any

means. Reverse engineering, disassembly, or decompilation of this

software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice

and is not warranted to be error-free. If you find any errors, please

report them to us in writing.

U.S. GOVERNMENT END USERS: Oracle programs, including any operating

system, integrated software, any programs installed on the hardware,

and/or documentation, delivered to U.S. Government end users are

"commercial computer software" pursuant to the applicable Federal

Acquisition Regulation and agency-specific supplemental regulations.

As such, use, duplication, disclosure, modification, and adaptation

of the programs, including any operating system, integrated software,

any programs installed on the hardware, and/or documentation, shall be

subject to license terms and license restrictions applicable to the

programs. No other rights are granted to the U.S. Government.

This software or hardware is developed for general use in a variety

of information management applications. It is not developed or

intended for use in any inherently dangerous applications, including

applications that may create a risk of personal injury. If you use

this software or hardware in dangerous applications, then you shall

be responsible to take all appropriate fail-safe, backup, redundancy,

and other measures to ensure its safe use. Oracle Corporation and its

affiliates disclaim any liability for any damages caused by use of this

software or hardware in dangerous applications.

This software or hardware and documentation may provide access to or

information on content, products, and services from third parties.

Oracle Corporation and its affiliates are not responsible for and

expressly disclaim all warranties of any kind with respect to

third-party content, products, and services. Oracle Corporation and

its affiliates will not be responsible for any loss, costs, or damages

incurred due to your access to or use of third-party content, products,

or services.

Verifying archive integrity...

All good.

Uncompressing Endeca Platform Services 11.0.0 for x86_64pc-linux.........

Configuring Endeca Platform Services 11.0.0 for x86_64pc-linux ....

Configuring the Endeca installed Perl 5.8.3 on your environment.

Configure the ports your Endeca Instance will utilize.

Please enter the port number of the Endeca HTTP service. The typical default is 8888.

: 27000

Please enter the shutdown port number of the Endeca HTTP service. The typical default is 8090.

: 18090

Please enter the port number of the Endeca Control System JCD, or nothing if you do not plan to use a Control System environment. The typical default is 8088.

: 18088

Would you like this installation configured to run the Endeca Application Controller (EAC)? (Y/N)?

(Please note: this will also install the EAC Agent)

: Y

Please enter the root of your Endeca MDEX Engine installation. The root directory path typically includes the version number. Leave blank if there is no Endeca MDEX Engine installed. (default: /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/endeca/MDEX/6.5.0)

: /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/endeca/MDEX/6.5.0

Would you like to install the reference implementations? (Y/N)?

: Y

Please run the following command to set your environment variables:

Bourne, Bash or Korn:

source /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/endeca/PlatformServices/workspace/setup/installer_sh.ini

csh or tcsh:

source /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/endeca/PlatformServices/workspace/setup/installer_csh.ini

[toolsandframeworks-demo.ose.dev.global.demo.org data]\> source /var/lib/openshift/53c7bec6982e97fc1d000052/app-deployments/endeca/PlatformServices/workspace/setup/installer_sh.ini

Configuring Oracle Endeca Commerce

Configuration involves configuring start/stop scripts, tomcat and jetty server settings, for making Oracle Endeca Commerce to work on Openshift.

Configure start/stop scripts

Go to start/stop script directory “/var/lib/openshift/12d13141516m17p18s1920a2/app-root/runtime/repo/.openshift/action_hooks”

Open start script

Remove ruby references and add Oracle Endeca Commerce reference

#!/bin/bash

# The logic to start up your application should be put in this

# script. The application will work only if it binds to

# $OPENSHIFT_DIY_IP:8080

nohup sh "$OPENSHIFT_DEPLOYMENTS_DIR"endeca/ToolsAndFrameworks/11.0.0/server/bin/startup.sh

echo "Starting ToolsAndFrameworks"

export ENDECA_ROOT="$OPENSHIFT_DEPLOYMENTS_DIR"endeca/PlatformServices/11.0.0/

export ENDECA_CONF="$OPENSHIFT_DEPLOYMENTS_DIR"endeca/PlatformServices/workspace/

nohup sh "$OPENSHIFT_DEPLOYMENTS_DIR"endeca/PlatformServices/11.0.0/tools/server/bin/startup.sh

echo "Starting PlatformServices"

nohup sh "$OPENSHIFT_DEPLOYMENTS_DIR"endeca/CAS/11.0.0/bin/cas-service.sh

echo "Starting CAS"

Open stop script

Remove ruby references and add Oracle Endeca Commerce reference

#!/bin/bash

source $OPENSHIFT_CARTRIDGE_SDK_BASH

# The logic to stop your application should be put in this script.

nohup sh "$OPENSHIFT_DEPLOYMENTS_DIR"endeca/ToolsAndFrameworks/11.0.0/server/bin/shutdown.sh

echo "completed toolsandframeworks shutdown"

export ENDECA_ROOT="$OPENSHIFT_DEPLOYMENTS_DIR"endeca/PlatformServices/11.0.0/

export ENDECA_CONF="$OPENSHIFT_DEPLOYMENTS_DIR"endeca/PlatformServices/workspace/

nohup sh "$OPENSHIFT_DEPLOYMENTS_DIR"endeca/PlatformServices/11.0.0/tools/server/bin/shutdown.sh

echo "shutdown PlatformServices"

nohup sh "$OPENSHIFT_DEPLOYMENTS_DIR"endeca/CAS/11.0.0/bin/cas-service-shutdown.sh

echo "shutdown CAS"

exit 0

Updating Endeca ToolsAndFramework

- Go to “ENDECA_TOOLS_CONF/conf”

- Edit server.xml

Update

<Server port="<port>" shutdown="SHUTDOWN">

<Server port="<port>" address="$OPENSHIFT_DIY_IP" shutdown="SHUTDOWN">

Here we are adding “OPENSHIFT_DIY_IP” to server shutdown

Update

<Connector port="8080" protocol="HTTP/1.1"

connectionTimeout="20000”

maxPostSize="0"

redirectPort="18446"

URIEncoding="UTF-8"/>

<Connector port="8080" protocol="HTTP/1.1"

connectionTimeout="20000"

maxPostSize="0"

redirectPort="18446"

URIEncoding="UTF-8"/>

<Connector port="8080" protocol="HTTP/1.1"

connectionTimeout="20000"

address=" $OPENSHIFT_DIY_IP"

maxPostSize="0"

redirectPort="18446"

URIEncoding="UTF-8"/>

<Connector port="8080" protocol="HTTP/1.1"

connectionTimeout="20000"

address="localhost"

maxPostSize="0"

redirectPort="18446"

URIEncoding="UTF-8"/>

Here we are doing port binding

Update

Here we are adding “OPENSHIFT_GEAR_DNS” to defaultHost

Update

<Host name="localhost" appBase="webapps"

unpackWARs="true" autoDeploy="true"

xmlValidation="false" xmlNamespaceAware="false">

<Host name="$OPENSHIFT_GEAR_DNS" appBase="webapps"

unpackWARs="true" autoDeploy="true"

xmlValidation="false" xmlNamespaceAware="false">

Here we are adding “OPENSHIFT_GEAR_DNS”

- Edit “webstudio.properties”

# The host name of this Workbench instance

com.endeca.webstudio.hostname=$OPENSHIFT_DIY_IP

# This must be set to a non-SSL port, even if you are using the

# SSL version of Oracle Commerce Workbench

com.endeca.webstudio.port=$OPENSHIFT_DIY_PORT

# The EAC Central Server that this Workbench uses

com.endeca.webstudio.eac.hostname=$OPENSHIFT_DIY_IP

com.endeca.webstudio.eac.port=27000

- Go to “ENDECA_TOOLS_CONF/conf/Standalone”

- Create a folder “demoendeca-demo.ose.dev.global.demo.org”

- Now add all files from localhost folder in the same directory to “demoendeca-demo.ose.dev.global.demo.org”

- Update war files/directories location in all the xml to point it to gear directories.

Updating Endeca PlatformServices

- Go to “ENDECA_CONF /conf”

- Edit server.xml

Update

<Server port="<port>" shutdown="SHUTDOWN">

<Server port="<port>" address="$OPENSHIFT_DIY_IP" shutdown="SHUTDOWN">

Here we are adding “OPENSHIFT_DIY_IP” to server shutdown

Update

<Connector port="27000"

maxHttpHeaderSize="8192"

maxThreads="150"

minSpareThreads="25"

enableLookups="true"

redirectPort="15443"

acceptCount="10"

connectionTimeout="60000"

protocol="HTTP/1.1"

disableUploadTimeout="true"

URIEncoding="UTF-8"/>

<Connector port="27000"

maxHttpHeaderSize="8192"

maxThreads="150"

minSpareThreads="25"

enableLookups="true"

redirectPort="15443"

acceptCount="10"

connectionTimeout="60000"

protocol="HTTP/1.1"

disableUploadTimeout="true"

URIEncoding="UTF-8"/>

<Connector port="27000"

maxHttpHeaderSize="8192"

maxThreads="150"

minSpareThreads="25"

enableLookups="true"

redirectPort="15443"

acceptCount="10"

connectionTimeout="60000"

address="$OPENSHIFT_DIY_IP"

protocol="HTTP/1.1"

disableUploadTimeout="true"

URIEncoding="UTF-8"/>

<Connector port="27000"

maxHttpHeaderSize="8192"

maxThreads="150"

minSpareThreads="25"

enableLookups="true"

redirectPort="15443"

acceptCount="10"

connectionTimeout="60000"

address="localhost"

protocol="HTTP/1.1"

disableUploadTimeout="true"

URIEncoding="UTF-8"/>

Here we are doing port binding

Update

Here we are adding “OPENSHIFT_GEAR_DNS” to defaultHost

Update

<Host name="localhost" appBase="webapps"

unpackWARs="true" autoDeploy="true"

xmlValidation="false" xmlNamespaceAware="false">

<Host name="$OPENSHIFT_GEAR_DNS" appBase="webapps"

unpackWARs="true" autoDeploy="true"

xmlValidation="false" xmlNamespaceAware="false">

Here we are adding “OPENSHIFT_GEAR_DNS”

- Edit “eaccmd.properties”

Update

host=node9.ose.dev.global.demo.org

port=27000

host=127.156.121.31

port=27000

- Go to “ENDECA_CONF/conf/Standalone”

- Create a folder “demoendeca-demo.ose.dev.global.demo.org”

- Now add all files from localhost folder in the same directory to “demoendeca-demo.ose.dev.global.demo.org”

- Update war files/directories location in all the xml to point it to gear directories.

Updating Endeca CAS

- Go to “CAS_CONF /conf”

- Edit jetty.xml

Update

<Arg>

<Set name="statsOn">false</Set>

</New>

</Arg>

</Call>

<Arg>

<Set name="host"><SystemProperty name="jetty.host" default="127.156.121.31 "/></Set>

<Set name="port"><SystemProperty name="jetty.port" default="16000"/></Set>

<Set name="statsOn">false</Set>

</New>

</Arg>

</Call>

Updating Endeca application

Update Endeca application configuration files to point EAC host and port of openshift.

Relevance Ranking Modules

2014-04-22T02:22:00.000-07:00

Following are relevance ranking modules which decide ordering of records in Endeca.

Exact: The Exact module provides a finer grained, but more computationally expensive alternative to the Phrase module. It groups results into three strata: 1. Results whose complete text matches the search terms exactly. 2. Results that contain the search terms as an exact substring. 3. Other hits, such as normal conjunctive matches. Any match requiring query expansion lands in this lowest stratum

Field: The Field module ranks records based on the field priority defined for the search interface in which it matches. Only the best field in which a match occurs is considered.

First: The First module ranks records based on how close a match is to the beginning of a field. It is primarily for unstructured data.

Freq: The Frequency (Freq) module provides result scoring based on the frequency (number of occurrences) of the search terms within a record. Results with more occurrences of the user search terms are considered more relevant.

Glom: The Glom module ranks single-field matches ahead of cross-field matches. It serves as a useful tie-breaker in combination with the Maximum Field (Maxfield) module

Interp: The Interpreted (Interp) module scores records based on the processing techniques used to obtain the match. The techniques considered include partial matching, cross-field matching, spelling correction, thesaurus and stemming.

Maxfield: The Maximum Field (Maxfield) module behaves identically to the Field module, except in how it scores cross-field matches. Unlike Field, which assigns a static score to cross-field matches, Maximum Field selects the score of the highest-ranked field that contributes to the match.

Nterms: The Number of Terms (Nterms) module ranks matches according to how many search terms they match. For example, in a three-word query, results that match all three words will be ranked above results that match only two, which will be ranked above results that match only one.

Numfields: The Number of Fields (Numfields) module ranks records based on the number of complete matching fields defined in the applied search interface. Only whole-field, rather than cross-field, matches count. Therefore, a result that matches two fields matches each field completely, while a cross-field match typically does not match any field completely.

Phrase: The Phrase module considers the search terms as an exact phrase, or a subset of an exact phrase, more relevant than matches simply containing the search terms scattered throughout the text.

Proximity: The Proximity module ranks based on how close the search terms in a multi-term query appear to each other in a field. It is primarily for unstructured data.

Spell: The Spell module ranks true matches ahead of spelling-corrected matches.

Static: The Static module ranks records by an attribute in either ascending or descending order.

Stem: The Stem module ranks true matches ahead of matches due to stemming.

Stratify: The Stratify module is used to boost or bury records in the result set into various strata using the Endeca Query Language (EQL).

Thesaurus: The Thesaurus module ranks true matches ahead of matches due to thesaurus entries.

Wfreq: Like the Frequency (Freq) module, the Weighted Frequency (Wfreq) module scores results based on the frequency of search terms within a record. Additionally, the Weighted Frequency module weights the terms for each result by the overall frequency in the complete data set. Less frequent terms (that is, terms that would result in fewer search results) are weighted more heavily than more frequently occurring terms.

ATG-Endeca: Range Dimension

2013-10-01T23:23:00.000-07:00

Creating Range dimension

This documentation will help you on getting the Price as Range dimension using ATG-Endeca. Ease while creating range price dimension using developer studio UI. But for creating these dimension used for ATG-Endeca integration is a bit of work.

An example:

demo.price

Under $100

$100 - $500

$500 - $1000

Over $1000

For this we need to create mapping in “dimension_values.csv” and “schema.csv”.

schema.csv

Define required dimensions and its source mappings

attribute.name	attribute.type	attribute.display_name	attribute.offline_sort	attribute.record_filterable	attribute.rollup_key	attribute.search_interface
sku.margin	property		TRUE
demo.price	dimension	Price				All

attribute.search.searchable	attribute.search.wildcard	attribute.show_with_list	attribute.show_with_record	attribute.source_name	attribute.property.data_type	attribute.dimension.autogen
				sku.margin	decimal
				activePrice

attribute.dimension.compute_refinement_counts	attribute.dimension.dyn_ranking_enabled	attribute.dimension.dyn_ranking_refinement_count	attribute.dimension.dyn_ranking_more	attribute.dimension.dyn_ranking_sort	attribute.dimension.hidden	attribute.dimension.hierarchy_dim_search	attribute.dimension.hierarchy_record_search

attribute.dimension.multiselect_type	attribute.dimension.refinements_sort_order

	numeric

dimension_values.csv

dimval.dimension_spec	dimval.spec	dimval.display_name	dimval.parent_spec	dimval.match_type
demo.price	demo.price	Price Range
demo.price	under_100	Under $100	demo.price	range
demo.price	100_to_500	$100 - $500	demo.price	range
demo.price	500_to_1000	$500 - $1000	demo.price	range
demo.price	over_1000	Over $1000	demo.price	range

dimval.display_order	dimval.classify_synonym	dimval.search_synonym	dimval.range.comparison_type	dimval.range.lower_bound	dimval.range.lower_bound_inclusive	dimval.range.upper_bound	dimval.range.upper_bound_inclusive

1			decimal			100	FALSE
2			decimal	100	TRUE	500	FALSE
3			decimal	500	TRUE	1000	FALSE
4			decimal	1000	TRUE

If we miss to add entries in “schema.csv” we get following error

FORGE {baseline}: (AdapterRunner): Unable to find corresponding attribute definition for taxonomy with root node spec 'demo.price', you must define a dimension in the schema feed

com.endeca.edf.adapter.AdapterException: Unable to find corresponding attribute definition for taxonomy with root node spec 'demo.price', you must define a dimension in the schema feed

at com.endeca.itl.fcm.integration.cadk.ifdi.TaxonomyGenerator.execute(TaxonomyGenerator.java:261)

at com.endeca.itl.fcm.integration.cadk.ConfigurationAdapter.execute(ConfigurationAdapter.java:118)

at com.endeca.esource.springadapter.SpringDelegateAdapter.execute(SpringDelegateAdapter.java:60)

at com.endeca.edf.adapter.AdapterRunner.run(AdapterRunner.java:168)

at com.endeca.edf.adapter.AdapterRunner.main(AdapterRunner.java:43)

Once the changes are done run the baseline using component “/atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin”

Oracle Endeca Tech Performance Tools Guide

2013-09-01T09:30:00.000-07:00

How to perform performance test both for throughput and latencies for my MDEX engine

Note: The below information is for testing MDEX engine performance only.
Additional tools is required to test performance running through your front end application.

To test MDEX engine performance, we use a program called eneperf. The first is a command line tool included with Endeca. Essentially send requests at a given host and port where you can ramp up concurrent connections to evaluate load and throughput.

The great thing about eneperf is that it add almost no overhead to the query at all. It do nothing with the response object so very little application overhead is added.

Other tools e.g. loadrunner can add latency to the round trip times that can skew your results. These times will show up in an admin stats as "send time" and in the request log as the delta between engine time and response time minus queue time. For testing the performance of the MDEX engine only using a test harness with no overhead is optimal.

Use your production traffic to generate the queries you are going to send through your MDEX engine. Take a production request log and run it through the Request Log Parser utility which strips out all of the non-request information from each line. This gets you a log to use with eneperf or mdex perf.

For example, when you apply request log parser to a request log;

1322705649 127.0.0.1 - 178456 346.67 249.47 200 5 0 10 /graph?node=196&group=0&offset=0&nbins=50&offset=0&sort=price_demo|desc

Becomes

/graph?node=196&allgroups=1&group=0&offset=0&&nbins=50&offset=0&sort=price_demo|desc

We then send these queries at the MDEX engines in different configurations of threading, mdex flags, Endeca configuration and options using eneperf.

Your request log should have enough queries as to fill up the Endeca cache.
If not, and you loop your log using <numiterations> you will get all of your responses after the initial run back from the Endeca cache and the engine will be not be under its normal load.

Depending on your site and type of queries a good starting point is to have at least 10,000 unique queries and preferably 20-40,000 queries in your log for a good test.

Eneperf uses the flags of <num connections> and <num iterations> which do the following
<num connections> Maximum number of outstanding (unreplied) requests to allow before waiting for replies, i.e., the number of simultaneous HTTP connection streams to keep open at all times.

This number emulates multiple clients for the target server. For example, using <num connections> of 16 emulates 16 concurrent clients querying the target server at all times. <num iterations> Number of times to replay the URL log.

Example

You have a dgraph you want to test that is configured for 4 threads.You run eneperf with the following command line:

eneperf <mdexHost> <mdexPort> parsed.log 8 1

Here you see numconnections 8. This will mean all 4 threads of your dgraph will always be busy and you will always have 4 queries in queue. (actually in practice you will have just under 4 queries in queue as there will be a lag between when the eneperf tool sends a response and when it actually hits the mdex queue where there will be 3 queries in queue. Typically this will be only a few ms per query depending on network speed etc.) That said, the way we typically test *throughput* performance (queries/second) is to make num connections = num MDEX threads +1 which will create a slight queuing environment.

To test for *latency* (engine processing time OR engine processing time plus travel time to and from the MDEX engine) performance we make num connections = num MDEX threads so we don't create a queue and therefore skew latency numbers.

Once you have run the test, you can then analyze the results:

Eneperf will give some excellent statistics on the speed of the test.

Example

Running iteration 1... Done:

29422 sent, 29421 received, 1 errors.

51 minutes, 50.62 seconds for iteration 1, ****9.45823 req/sec.***** average throughput for the iteration (from the num iterations flag)

51 minutes, 50.62 seconds elapsed (user: 1.54 seconds, system: 5.43 seconds).

Net: 2.15397e+06 KB (692.455 KB/sec).

Page Size: avg=73.21 KB, std dev=80.51 KB, max=259.27 KB, min=0.20 KB.

Eneperf completed:

29422 sent, 29421 received, *****1 errors.***** Number of errors see the err output for details on errors.

51 minutes, 50.62 seconds elapsed (user: 1.54 seconds, system: 5.43 seconds).

Net: 2.15397e+06 KB (692.455 KB/sec).

Page Size: avg=73.21 KB, std dev=80.51 KB, max=259.27 KB, min=0.20 KB.

Best iteration time: 51 minutes, 50.62 seconds.

Peak rate: 9.45823 req/sec

Avg iteration time: 51 minutes, 50.62 seconds.

Avg rate: 9.45823 req/sec.

*****Total rate: 9.45823 req/sec.***** Total rate for the entire run.

Second, you can use cheetah and run it against the request log you produce in your eneperf test. This, especially combined with the --showAll flag, will break your traffic down into some sub categories and show you your worst engine times, average round trip times queuing information etc.

Thirdly we look at an admin?op=stats page on the mdex engine after the test to see if we can understand where there are possible performance bottlenecks. Especially in the hotspots section of the "Details" tab.

Fourth: Frequently we will monitor things like ram usage, virtual memory statistics, disk I/O and cpu usage to determine if there is not some other bottleneck as well. In windows you can use perfmon to set up counters to analyze various things. In *nix you can output top and iostat and vmstat etc. to files for analysis.

A typical performance testing procedure will go something like this.

· Get a baseline of current performance

· Make one or two changes

· Test performance changes and compare against baseline

· Make second changes

· Test performance changes and compare against baseline and test 2

· Iterate

Example

1. Set up an environment that is a hardware mirror to production. (or as close as you can get)

2. Set up system monitoring software e.g. perfmon in windows or top, iostat or vmstat in Linux etc. record the output of these tests.

3. Stop your MDEX engines and clear all request and error logs.

4. Start your MDEX engines with regular settings

5. Run eneperf using the parsed logs from your production environment.

6. Save the output from eneperf with a descriptive label of this particular test

7. Save off a copy of the request AND error logs also with the same descriptive label of the test

8. go to http://<dgraphhost>:<dgrapport>/admin?op=stats and save the resulting file as <descriptive_test_name.xml >

9. Stop your MDEX engines

10. Make the first changes you are implementing. Example addition of a new thesaurus, a removal of a significant query option, or a change in MDEX flags.

11. Clear your request and error logs again.

12. Start your MDEX engines with new configuration and settings

13. Run eneperf using the parsed logs from your production environment.

14. Save the output from eneperf with a NEW descriptive label of this particular test

15. Save off a copy of the request AND error logs also with the same descriptive label of the test

16. go to http://<dgraphhost>:<dgrapport>/admin?op=stats and save the resulting file as <NEWdescriptive_test_name.xml >

17. Repeat making small changes each time to be able to easily determine what changes had an impact.

Some Examples of things you would want to run performance analysis:

to test removing "allgroups=1" (setNavAllRefinements=true) from your application you can remove this query option from all of the queries in the perf log before using eneperf to send traffic at the engine.

You can test adding a new thesaurus with 10000 new entries

You can test the performance impact of a new server or vs. your current hardware.

You can test the impact of a new data source or new dimensions added to your project

Remove Endeca Application

2013-08-04T21:23:00.000-07:00

Do not simply remove the Endeca application directories or delete EAC state directories (ENDECA_CONF/state/eac*) from in the file system.

This leaves application artifacts, which can cause naming conflicts when creating a new project with the same name.

Steps to remove an Endeca application:

Invoke the deployment template command to remove the application.

For example:

<Endeca-app-dir>\control\runcommand.sh|bat --remove-app

Demo\control\runcommand.sh|bat --remove-app

Drop the corresponding Record stores from CAS.

Check names of application record stores registered in CAS.

$CAS_ROOT\bin>component-manager-cmd.bat/.sh list-components

Remove the record stores corresponding to your Endeca application. The record store names will be prefixed with the application name.

For example:

$CAS_ROOT/bin>component-manager-cmd.bat/.sh delete-component -n myAppName_en_data

$CAS_ROOT/bin>component-manager-cmd.bat/.sh delete-component -n myAppName_en_schema

$CAS_ROOT/bin>component-manager-cmd.bat/.sh delete-component -n myAppName_en_prules

$CAS_ROOT/bin>component-manager-cmd.bat/.sh delete-component -n myAppName_en_dimvals

Finally, remove the Endeca application directory from the file system.

ORACLE ENDECA - JSP Reference Implementation not working

2013-07-26T04:56:00.001-07:00

Error:

Once we are done will all smooth Endeca installation, application creation and baseline.

If we try to run the endeca jsp ref application we get following error

com.endeca.navigation.ENEException: Navigation Engine not able to process request 'http://localhost:15000/graph?node=0&offset=0&nbins=10&irversion=620'.

Cause:

This is because API version is not matching the Endeca installation version. The API version getting referred is Endeca 6.2.0 and the required version of Endeca API is 6.4.1 [Whatever is the the latest version]

Solution:

Copy the correct version of jars.

From Location: C:\Endeca\ToolsAndFrameworks\3.1.2\reference\endeca_jspref\WEB-INF\lib

Files: endeca_navigation-6.4.1.jar [Whatever is the the latest version], endeca_logging-6.4.1.jar [Whatever is the the latest version] and eacclient.jar

To Location: C:\Endeca\PlatformServices\6.1.3\tools\server\webapps\endeca_jspref\WEB-INF\lib

And restart the PlatformServices [EndecaHTTPservice] from services.

Oracle Endeca some common question and errors

2013-04-04T04:46:00.001-07:00

How to troubleshoot if a thesaurus entry is working

I am using a thesaurus entry but I'm not sure it is firing. How can I tell whether this entry is being used by the MDEX Engine?

Enable the --whymatch option on the MDEX Engine. Using an Endeca Reference application, search for the term that should trigger the thesaurus entry. The reference application will display a new property on each record called "DGraph.WhyDidItMatch" and the value for this property will include a tag specifying whether the match was produced by the thesaurus.

For example, in the sample wine data application, creating a one-way thesaurus entry for "French-> France" and searching for "French" causes the following property value to be returned:

DGraph.WhyDidItMatch: P_WineType: France (Thesaurus)

Partial update fails with "Record change references unknown dimension value id" error

A partial update fails with a "Record change references unknown dimension value id" error, along these lines:

ERROR 11/19/12 02:26:09.348 DGRAPH {dgraph,update} Record change references unknown dimension value id 4294952004. [rec_spec=`p_record_id:1202']

This error usually indicates a failure in an earlier partial-update operation which attempted to define a new autogenerated dimension value (with ID "4294952004", in the example). Because of this failure, this dimension value ID, which is defined in the Forge autogen state file, is never defined in the MDEX Engine to which the partial update is applied.
The upshot of this earlier failure is that the MDEX Engine has no definition for the associated dimension value ID; as a result, subsequent record-update operations which reference this ID, such as adds or updates of records tagged with the new dimension value, will also fail with the above error.
Forge will not include the failed dimension-value update operation in later updates: it has no way of knowing which operations failed on application to the MDEX Engine.

Re-run your baseline update to synchronize dimension-value IDs between the Forge autogen state file and the MDEX Engine, then generate and apply new partial updates normally.
Re-applying just the dimension-value definition operation (by copying the update file back into the MDEX Engine's updates directory and issuing an admin?op=update command) is not normally practical: the MDEX Engine will not apply further updates from a file which has already been read in once. And Forge will not include the dimension-value definition in later updates: because the ID is already present in the Forge autogen state file, Forge believes that it's already defined and that it can be relied upon for further record updates. So a baseline update is generally the best approach for ensuring that all dimension value IDs defined in the Forge state are also defined in the MDEX Engine.

Troubleshooting large MDEX Engine response sizes

How to determine if a large query response size is contributing to performance issues?

While troubleshooting a performance issue with Endeca, your Cheetah analysis might indicate that the worst queries all have large response sizes ("large" is a relative term, but it is best practice for an MDEX Engine's response size to be no larger than approximately 200-300 Kb). The following example from a Dgraph request log is an example of a large response size:

1173115134 127.0.0.1 13172736 493811.11 463253.40 200 1933 -2 9 /graph?node=33&group=0&offset=0&nbins=25&irversion=480

The third column from the left shows the byte size of the response. In this example, the byte size of this query is 13172736, which equals to just over 13 MB and is causing poor performance because of its large size.

Note: In 6.x, the response size column is now 5th in position.

The following is a list of troubleshooting questions and resolutions that you can take to determine why the response size is large:

1. Is "nbins" too high? This is the maximum amount of records that the MDEX Engine will return in the response for this query and can be found in the request log entry for a given query. In the example above, the nbins=25 statistic is reasonable. If nbins=1000 were in a request log query (for example), this is likely to be too high and needs to be restricted in the API code via the ENEQuery.setNavNumErecs() method (from the Java API, in this example) and paging control methods need to be used. For information on these paging control methods, see the section "Paging through a record set" in the "Endeca Developer's Guide".

2. Is there a merchandising rule being returned that is promoting a large amount of rules? It is possible to check the result of a query in an Endeca reference application to see the supplemental objects returned (which would contain the merchandising data) and how many records the object contains. It is also possible to use the Dgraph --merch_debug flag and verify this information in the Dgraph error log.

In the case that too many records are being returned with a rule, it is likely the rule's Style "Max Records" setting needs to be lowered to return no more than "x" amount of records.

3. Are large properties (such as long description fields or a crawled document's Endeca.Document.Text property) being returned in the record list? It is generally not recommend to include properties like this in the record list, but only within the individual record page, as it allows for a potentially large document to be returned unnecessarily in the list of results.

To avoid returning large property files in the record list, open your project in Developer Studio and use the "Show with Record List" checkbox in the Dimension or Property editor for a given Dimension or Property. It is also recommended to use snippeting to expose portions of large properties on the record list.

To Baseline Or Not To Baseline

Reloading Configs that needs Baseline

Stop words

Dimension ordering

Doesn't require baseline

Thesaurus entries

Key Properties (usap.key_props.xml)

Business rule changes (within existing zones and styles)

Keyword redirects

Automatic phrases

NOTE

When you add new zones or styles, you can avoid a baseline by copying the new XML file to the dgraph_input directory of any live dgraph and restarting that dgraph.

Is there a maximum depth for a hierarchical dimension?

The maximum depth for a dimension hierarchy is 64 levels.

Are dimension value properties searchable?

I have properties assigned to my dimension values. If I search for a term contained in those property values, will the dimension be returned?

No, by default properties on a dimension value are not searchable.

How to check status of running components

Checking status

Go to /root/data/endeca/apps/<appName>/control

./runcommand.sh --print-status

Sample output:

Status of components defined in AppConfig.xml.

Forge: NotRunning

PartialForge: NotRunning

Dgidx: NotRunning

Dgraph1: Running

LogServer: Running

WeeklyReportGenerator: NotRunning

DailyReportGenerator: NotRunning

WeeklyHtmlReportGenerator: NotRunning

DailyHtmlReportGenerator: NotRunning

Fail to obtain lock Error

Check for lock:

/data/endeca/PlatformServices/6.1.0/bin/eaccmd.sh localhost:8888 list-flags --app app_name

If the output is

baseline_data_ready

update_lock

Run following to Release lock:

/data/endeca/PlatformServices/6.1.0/bin/eaccmd.sh localhost:8888 remove-flag --app app_name --flag update_lock

Oracle Endeca Cluster Setup

2013-02-12T00:17:00.002-08:00

This post provide details on creating cluster environment during Oracle Endeca setup. CAS is not included in the guide.

Hardware Requirements

Recommended hardware from Oracle Endeca

Linux on x64 or Windows on x64

Minimum hardware requirements:

• x64 processor, minimum 1.8 GHz

• At least 2 GB of RAM, depending on the size of the application data set

• 80 GB hard drive

Recommended hardware requirements:

• x64 3.0+ GHz processors; Oracle recommends Intel Xeon or AMD Opteron processors

• 8 GB of RAM or more, depending on the size of the application data set

• One 72GB or 146 GB hard drive per CPU, minimum 10,000 or 15,000 RPM spindle speed

• Gigabit ethernet

• RAID 10 or 6 configuration

Software Requirements

As this is upgrade following packages are required.

· Oracle Endeca MDEX Engine 6.3.0

· Oracle Endeca Platform Services 6.1.3

· Oracle Endeca Tools and Frameworks with Experience Manager 3.1.0

Setup Diagram

For demo we have only three server as show below

Details:

Component	Servers
9.10.11.1	EAC server
9.10.11.2	MDEX server
9.10.11.3	MDEX server

Install following package on EAC server

· Oracle Endeca MDEX Engine 6.3.0

· Oracle Endeca Platform Services 6.1.3

· Oracle Endeca Tools and Frameworks with Experience Manager 3.1.0

Install following package on MDEX server.

· Oracle Endeca MDEX Engine 6.3.0

· Oracle Endeca Platform Services 6.1.3

Deploying application using ToolsAndFramework deployment template

[root@9.10.11.1 bin]# ./deploy.sh

------------------------------------------------------------------------------

Found version 6.1 of the Endeca IAP installed in directory

/data/endeca/PlatformServices/6.1.3. If either the version or location are

incorrect, type 'Q' to quit and adjust your ENDECA_ROOT environment variable.

Press enter to continue with these settings.

Continue?

------------------------------------------------------------------------------

Deployment Template installation script.

This script creates the directory structure for your

deployment and installs configuration files and scripts

into the directory structure.

2/5/2013 23:58:43 [deploy.pl] INFO: Starting deployment template

installation.

2/5/2013 23:58:43 [AppDescriptorReader] INFO: Parsing application descriptor

file

/data/endeca/ToolsAndFrameworks/3.1.0/deployment_template/lib/../app-templates/base_descriptor.xml.

------------------------------------------------------------------------------

Enter a short name for your application.

Note: The name must conform to this regular expression: ^[a-zA-Z0-9]+$

Application name:

test

------------------------------------------------------------------------------

Specify the path into which the application will be deployed. The specified

directory must exist and cannot contain spaces.

For example, to deploy into /localdisk/apps/test, specify the path as

/localdisk/apps.

Deployment directory:

/data/endeca/apps/

------------------------------------------------------------------------------

Specify the port on which the Endeca Application Controller is running. This

is configured in the server.xml file in the workspace of the Endeca software

install and should be the same for all applications deployed in this

environment. Ports must be in the range 1024-65535 [default: 8888].

EAC port:

8888

2/5/2013 23:59:00 [deploy.pl] INFO: Deploying application into

/data/endeca/apps/test

------------------------------------------------------------------------------

What port is the Workbench running? [Default: 8006]

8006

------------------------------------------------------------------------------

What port should be used for the Live Dgraph? [Default: 15000]

15000

------------------------------------------------------------------------------

What port should be used for the Authoring Dgraph? [Default: 15002]

15002

------------------------------------------------------------------------------

What port should be used for LogServer? [Default: 15010]

15010

2/5/2013 23:59:18 [AppDescriptorReader] INFO: Parsing application descriptor

file

/data/endeca/ToolsAndFrameworks/3.1.0/deployment_template/lib/../app-templates/base_descriptor.xml.

2/5/2013 23:59:18 [deploy.pl] INFO: Processing install with id 'Dgraph'

2/5/2013 23:59:18 [deploy.pl] INFO: Application successfully deployed.

[root@9.10.11.1 bin]# pwd

/data/endeca/ToolsAndFrameworks/3.1.0/deployment_template/bin

Changes to be made

If you are trying to import your application following are few points that need to be taken care. Also please make sure all the custom changes are included at this point [For example any pipeline changes, Appconfig changes and others]

Config XML changes

If you are upgrading from MDEX 6.1.4 to MDEX 6.4.0 there are number of changes in the Appconfig.xml which is now split to many other XML. Following are the details.

AppConfig.xml

Cross verify this file to check appName, eacHost, eacPort, dataPrefix, sslEnabled and lockManager settings are correctly specified.

<!--

########################################################################

# EAC Application Definition

-->

<app appName="test" eacHost="9.10.11.1" eacPort="8888"

dataPrefix="test" sslEnabled="false" lockManager="LockManager">

<working-dir>${ENDECA_PROJECT_DIR}</working-dir>

<log-dir>./logs</log-dir>

</app>

AuthoringDgraphCluster.xml

Specifies the Dgraphs used in the authoring environment and a script that pushes configuration from Workbench to each Dgraph in the authoring cluster. Here we will define our environment specific combination of Authoring Dgraph.

For example:

<!--

########################################################################

# Authoring MDEX Hosts - The machines used to host all MDEX processes

# for the 'authoring environment' MDEX cluster.

-->

<host id="AuthoringMDEXHost" hostName="9.10.11.1" port="8888" />

<host id="AuthoringMDEXHost2" hostName="9.10.11.2" port="8888" />

<!--

########################################################################

# Authoring Dgraph Cluster - The 'authoring environment' MDEX cluster.

-->

<dgraph-cluster id="AuthoringDgraphCluster" getDataInParallel="true" enabled="true>

<dgraph ref="AuthoringDgraph" />

<dgraph ref="AuthoringDgraph2" />

</dgraph-cluster>

<!--

########################################################################

# Authoring Dgraph Process Definitions

-->

<dgraph id="AuthoringDgraph" host-id="AuthoringMDEXHost" port="15002"

post-startup-script="AuthoringDgraphPostStartup" >

<properties>

<property name="restartGroup" value="A" />

<property name="DgraphContentGroup" value="Authoring" />

</properties>

<log-dir>./logs/dgraphs/AuthoringDgraph</log-dir>

<input-dir>./data/dgraphs/AuthoringDgraph/dgraph_input</input-dir>

<update-dir>./data/dgraphs/AuthoringDgraph/dgraph_input/updates</update-dir>

</dgraph>

<!--

########################################################################

# Authoring Dgraph Process Definitions

-->

<dgraph id="AuthoringDgraph2" host-id="AuthoringMDEXHost2" port="15002"

post-startup-script="AuthoringDgraphPostStartup">

<properties>

<property name="restartGroup" value="B" />

<property name="DgraphContentGroup" value="Authoring" />

</properties>

<log-dir>./logs/dgraphs/AuthoringDgraph2</log-dir>

<input-dir>./data/dgraphs/AuthoringDgraph2/dgraph_input</input-dir>

<update-dir>./data/dgraphs/AuthoringDgraph2/dgraph_input/updates</update-dir>

</dgraph>

DataIngest.xml

We will update any specific custom code related to baseline update or partial update here. Specifies data processing scripts, including the baseline update script, partial update script, and the components to perform data processing such as Forge and Dgidx. Setup any Dgidx flags if required.

<!--

########################################################################

# Dgidx

-->

<dgidx id="Dgidx" host-id="ITLHost">

<properties>

<property name="numLogBackups" value="10" />

<property name="numIndexBackups" value="3" />

</properties>

<args>

<arg>-v</arg>

<arg>--compoundDimSearch</arg>

</args>

<log-dir>./logs/dgidxs/Dgidx</log-dir>

<input-dir>./data/forge_output</input-dir>

<output-dir>./data/dgidx_output</output-dir>

<temp-dir>./data/temp</temp-dir>

<run-aspell>true</run-aspell>

</dgidx>

DgraphDefaults.xml

Specifies default values that are inherited by all Dgraph components. These values include host IDs, data processing paths, and Dgraph flags.

<!--

########################################################################

# Global Dgraph Settings - inherited by all dgraph components.

-->

<dgraph-defaults>

<properties>

<property name="srcIndexDir" value="./data/dgidx_output" />

<property name="srcIndexHostId" value="ITLHost" />

<property name="srcPartialsDir" value="./data/partials/forge_output" />

<property name="srcPartialsHostId" value="ITLHost" />

<property name="srcCumulativePartialsDir" value="./data/partials/cumulative_partials" />

<property name="srcCumulativePartialsHostId" value="ITLHost" />

<property name="srcDgraphConfigDir" value="./data/workbench/dgraph_config" />

<property name="srcDgraphConfigHostId" value="ITLHost" />

<property name="numLogBackups" value="10" />

<property name="shutdownTimeout" value="30" />

<property name="numIdleSecondsAfterStop" value="0" />

</properties>

<directories>

<directory name="localIndexDir">./data/dgraphs/local_dgraph_input</directory>

<directory name="localCumulativePartialsDir">./data/dgraphs/local_cumulative_partials</directory>

<directory name="localDgraphConfigDir">./data/dgraphs/local_dgraph_config</directory>

</directories>

<args>

<arg>--threads</arg>

<arg>2</arg>

<arg>--whymatch</arg>

<arg>--spl</arg>

<arg>--dym</arg>

<arg>--dym_hthresh</arg>

<arg>5</arg>

<arg>--dym_nsug</arg>

<arg>3</arg>

<arg>--stat-abins</arg>

</args>

<startup-timeout>120</startup-timeout>

</dgraph-defaults>

InitialSetup.xml

Specifies scripts to perform initial setup tasks, such as uploading initial configuration to Workbench.

LiveDgraphCluster.xml

Specifies the Dgraphs used in the live environment and a script that pushes configuration from Workbench to each Dgraph in the live cluster.

<!--

########################################################################

# Live MDEX Hosts - The machines used to host all MDEX processes

# for the 'live environment' MDEX cluster.

-->

<host id="LiveMDEXHost" hostName="9.10.11.1" port="8888" />

<host id="LiveMDEXHost2" hostName="9.10.11.2" port="8888" />

<host id="LiveMDEXHost3" hostName="9.10.11.3" port="8888" />

<!--

########################################################################

# Live Dgraph Cluster - The 'live environment' MDEX cluster.

-->

<dgraph-cluster id="LiveDgraphCluster" getDataInParallel="true" enabled="true">

<dgraph ref="Dgraph1" />

<dgraph ref="Dgraph2" />

<dgraph ref="Dgraph3" />

</dgraph-cluster>

<!--

########################################################################

# Live Dgraph Process Definitions

-->

<dgraph id="Dgraph1" host-id="LiveMDEXHost" port="15000"

post-startup-script="LiveDgraphPostStartup">

<properties>

<property name="restartGroup" value="1" />

<property name="DgraphContentGroup" value="Live" />

</properties>

<log-dir>./logs/dgraphs/Dgraph1</log-dir>

<input-dir>./data/dgraphs/Dgraph1/dgraph_input</input-dir>

<update-dir>./data/dgraphs/Dgraph1/dgraph_input/updates</update-dir>

</dgraph>

<!--

########################################################################

# Live Dgraph Process Definitions

-->

<dgraph id="Dgraph2" host-id="LiveMDEXHost2" port="15000"

post-startup-script="LiveDgraphPostStartup">

<properties>

<property name="restartGroup" value="2" />

<property name="DgraphContentGroup" value="Live" />

</properties>

<log-dir>./logs/dgraphs/Dgraph1</log-dir>

<input-dir>./data/dgraphs/Dgraph1/dgraph_input</input-dir>

<update-dir>./data/dgraphs/Dgraph1/dgraph_input/updates</update-dir>

</dgraph>

<!--

########################################################################

# Live Dgraph Process Definitions

-->

<dgraph id="Dgraph3" host-id="LiveMDEXHost3" port="15000"

post-startup-script="LiveDgraphPostStartup">

<properties>

<property name="restartGroup" value="3" />

<property name="DgraphContentGroup" value="Live" />

</properties>

<log-dir>./logs/dgraphs/Dgraph1</log-dir>

<input-dir>./data/dgraphs/Dgraph1/dgraph_input</input-dir>

<update-dir>./data/dgraphs/Dgraph1/dgraph_input/updates</update-dir>

</dgraph>

ReportGeneration.xml

Specifies the hosts used for logging and report generations, and several scripts that produce log files at different time intervals.

WorkbenchConfig.xml

Specifies the IFCR component, the Workbench Manager component, and a script that promotes content from the authoring environment to the live environment.

Run initialize_service.sh

Run initialize service and initialize the test application.

Ø /root/data/endeca/apps/testdemo/control

Ø sh initialize_services.sh

Copy State file

Copy the state file folder to /data/endeca/test/data to make sure that Dimension ID are constant.

Run baseline

Once we have the application setup, run the baseline get data indexed and ready to use records.

Oralce Endeca Match Modes

2012-12-04T04:51:00.000-08:00

Introduction

Oracle Endeca, by default, will return results that match all of the users search terms for a given query to the engine. In most cases this is not desirable as it will require all the terms entered by a user match before a result is returned. For any query the match mode can be specified so that more desirable results can be obtained depending on the situation. In this section we will describe the match modes available.

Match Modes

MatchAll

This is the default match mode (if none specified in the query) and requires that the results match all terms submitted.

MatchPartial

Results here must match a partial set of the terms entered according to the rules set for partial matches. The following 2 rules are available in Developers Studio by selecting the "Options" button in the Search Interface dialog box:

• "Match at Least" - this setting specifies the minimum number of user query terms that each result must match. If there are not enough terms in the original query to satisfy this rule, then the entire query must match. The default setting is 2 which mean at least 2 terms must match. Note - setting this to 1 will in effect create a match mode of matchany.

• "Omit at Most" - setting specifies the maximum number of user query terms that can be ignored in the user query. If "Omit at most" is zero, then any number of words can be ignored.

This match mode will return all results that a MatchAll would return and possibly more. Also, if any Stop Words are included in the search terms then they will reduce the count required to match a result by the number of stop words included.

MatchAny

Using this mode requires that only one term match for a result to be considered. So for a 3 term search, results will be returned that match any of the 3 terms. This mode is not recommended for catalog applications and should be used with care.

MatchAllPartial

This mode tries to match on all the terms entered and will return only that set if any are found. If no matches are found then the search will try for matches using the MatchPartial mode. This is the recommended mode for catalog applications.

MatchAllAny

Like the MatchAllPartial mode, this mode will try to match all of the terms first. If no matches are found then a MatchAny is performed.

Setting Up Tools And Frameworks

2012-11-09T01:09:00.000-08:00

This is Linux version of setup.

Error encounter while making Tools and Framework up and running

Following are some of the errors and there fixes we got for Tool And Framework setup.

[root@endeca bin]# sh deploy.sh --app /endeca/ToolsAndFrameworks/3.1.0/reference/discover-data/deploy.xml

Can't locate strict.pm in @INC (@INC contains: ./../lib /tmp/original_perl_build_dir/lib/5.8.3/i686-linux /tmp/original_perl_build_dir/lib/5.8.3 /tmp/original_perl_build_dir/lib/site_perl/5.8.3/i686-linux /tmp/original_perl_build_dir/lib/site_perl/5.8.3 /tmp/original_perl_build_dir/lib/site_perl .) at ./../lib/deploy.pl line 13.

BEGIN failed--compilation aborted at ./../lib/deploy.pl line 13.

=================================================================================================================================================

Run following command to remove these error if occur, these setup the environment

export ENDECA_ROOT=/endeca/PlatformServices/6.1.3

export ENDECA_CONF=/endeca/PlatformServices/workspace

export ENDECA_TOOLS_ROOT=/endeca/Workbench/2.1.2

export ENDECA_TOOLS_CONF=/endeca/Workbench/workspace

export PERLLIB=$ENDECA_ROOT/lib/perl:$ENDECA_ROOT/lib/perl/Control:$ENDECA_ROOT/perl/lib:$ENDECA_ROOT/perl/lib/site_perl

export PERL5LIB=$ENDECA_ROOT/lib/perl:$ENDECA_ROOT/lib/perl/Control:$ENDECA_ROOT/perl/lib:$ENDECA_ROOT/perl/lib/site_per

=================================================================================================================================================

Rerunning the deployment command will run smoothly.

[root@endeca bin]# sh deploy.sh --app /endeca/ToolsAndFrameworks/3.1.0/reference/discover-data/deploy.xml

------------------------------------------------------------------------------

Found version 6.1 of the Endeca IAP installed in directory

/endeca/PlatformServices/6.1.3. If either the version or location

are incorrect, type 'Q' to quit and adjust your ENDECA_ROOT environment

variable. Press enter to continue with these settings.

Continue?

Y

------------------------------------------------------------------------------

Deployment Template installation script.

This script creates the directory structure for your

deployment and installs configuration files and scripts

into the directory structure.

11/09/2012 23:55:02 [deploy.pl] INFO: Starting deployment template

installation.

------------------------------------------------------------------------------

The following app modules were specified on the command line argument:

/endeca/ToolsAndFrameworks/3.1.0/reference/discover-data/deploy.xml

Modules may be deployed standalone or on top of a base deployment type. Do

you want to install a base deployment with the specified module(s)?

Install base deployment? [Y/N]:

Y

11/09/2012 23:55:44 [AppDescriptorReader] INFO: Parsing application descriptor file

/endeca/ToolsAndFrameworks/3.1.0/reference/discover-data/deploy.xml.

11/09/2012 23:55:44 [AppDescriptorReader] INFO: Parsing application descriptor file

/endeca/ToolsAndFrameworks/3.1.0/deployment_template/lib/../app-templates/base_descriptor.xml.

------------------------------------------------------------------------------

Enter a short name for your application.

Note: The name must conform to this regular expression: ^[a-zA-Z0-9]+$

[default: Discover]

Choosing a different application name may require additional configuration on your web application.

Application name:

Discover

------------------------------------------------------------------------------

Specify the path into which the application will be deployed. The specified directory must exist and cannot contain spaces.

For example, to deploy into /localdisk/apps/Discover, specify the path as

/localdisk/apps.

Deployment directory:

/endeca/apps

------------------------------------------------------------------------------

Specify the port on which the Endeca Application Controller is running. This is configured in the server.xml file in the workspace of the Endeca software install and should be the same for all applications deployed in this environment. Ports must be in the range 1024-65535 [default: 8888].

EAC port:

8888

11/09/2012 23:56:30 [deploy.pl] INFO: Deploying application into

/endeca/apps/Discover

------------------------------------------------------------------------------

What port is the Workbench running? [Default: 8006]

8006

------------------------------------------------------------------------------

What port should be used for the Live Dgraph? [Default: 15000]

13000

------------------------------------------------------------------------------

What port should be used for the Authoring Dgraph? [Default: 15002]

13002

------------------------------------------------------------------------------

What port should be used for LogServer? [Default: 15010]

13010

11/09/2012 23:57:07 [AppDescriptorReader] INFO: Parsing application descriptor file

/endeca/ToolsAndFrameworks/3.1.0/deployment_template/lib/../app-templates/base_descriptor.xml.

11/09/2012 23:57:07 [deploy.pl] INFO: Processing install with id 'Dgraph'

11/09/2012 23:57:07 [AppDescriptorReader] INFO: Parsing application descriptor file

/endeca/ToolsAndFrameworks/3.1.0/reference/discover-data/deploy.xml.

11/09/2012 23:57:07 [deploy.pl] INFO: Processing install with id 'DefaultTemplateManagerApp'

11/09/2012 23:57:08 [deploy.pl] INFO: Application successfully deployed.

=================================================================================================================================================

Running initialize will generate error, you need to fix some hostname in Appconfig.xml and other xml files if deployment process pulls machine name as host name, you need to make it IP for working properly in some cases.

[root@endeca control]# sh initialize_services.sh

[11.09.12 00:00:47] SEVERE: Caught exception while querying for defined application list.

Occurred while executing line 18 of valid BeanShell script:

[[

15|

16| // If the application is already defined

17| // log an error and exit with code 1

18| if (app.isDefined()) {

19| log.severe("An application already exists with the name, \"" + provObj.getAppName() + "\". " +

20| "Please use the '--force' option if you want to replace all existing configuration.");

21| System.exit(1);

]]

[11.09.12 00:00:47] SEVERE: Caught an exception while invoking method 'run' on object 'AssertNotDefined'. Releasing locks.

Caused by java.lang.reflect.InvocationTargetException

sun.reflect.NativeMethodAccessorImpl invoke0 - null

Caused by com.endeca.soleng.eac.toolkit.exception.AppControlException

com.endeca.soleng.eac.toolkit.script.Script runBeanShellScript - Error executing valid BeanShell script.

Caused by com.endeca.soleng.eac.toolkit.exception.EacCommunicationException

com.endeca.soleng.eac.toolkit.application.Application isDefined - Caught exception while querying for defined application list.

Caused by org.apache.axis.AxisFault

org.apache.axis.AxisFault makeFault - ; nested exception is:

java.net.ConnectException: Connection refused

Caused by java.net.ConnectException

java.net.PlainSocketImpl socketConnect - Connection refused

=================================================================================================================================================

Following command will help you find out where to change.

[root@endeca script]# find /root/apps/Discover/ -type f -exec grep -i "endeca.local" {} \; -print

"host": "endeca.local",

/endeca/apps/Discover/config/editors_config/services/dataservice.json

/endeca/apps/Discover/config/script/WorkbenchConfig.xml

/endeca/apps/Discover/config/script/DataIngest.xml

/endeca/apps/Discover/config/script/ReportGeneration.xml

/endeca/apps/Discover/config/script/LiveDgraphCluster.xml

/endeca/apps/Discover/config/script/AuthoringDgraphCluster.xml

<app appName="Discover" eacHost="endeca.local" eacPort="8888"

/endeca/apps/Discover/config/script/AppConfig.xml

emgr_update --host "endeca.local:8006" --app_name "${ENDECA_PROJECT_NAME}" --action remove_templates

emgr_update --host "endeca.local:8006" --app_name "${ENDECA_PROJECT_NAME}" --action set_templates --dir "${WORKING_DIR}/../config/cartridge_templates"

/endeca/apps/Discover/control/set_templates.sh

emgr_update --host "endeca.local:8006" --action get_templates --prefix "${ENDECA_PROJECT_NAME}" --app_name "${ENDECA_PROJECT_NAME}" --dir "${WORKING_DIR}/../config/cartridge_templates"

/endeca/apps/Discover/control/get_templates.sh

Change “endeca.local” to IP of the machine ex: “23.232.325.323”

=================================================================================================================================================

Make sure that the Tools and Framework is running properly and required directories are given permission and access for “endeca” user.

[root@endeca control]# sh initialize_services.sh

Setting EAC provisioning and performing initial setup...

[11.09.12 00:16:59] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.

[11.09.12 00:16:59] INFO: Setting definition for application 'Discover'.

[11.09.12 00:16:59] INFO: Setting definition for host 'AuthoringMDEXHost'.

[11.09.12 00:16:59] INFO: Setting definition for host 'LiveMDEXHostA'.

[11.09.12 00:17:00] INFO: Setting definition for host 'ReportGenerationHost'.

[11.09.12 00:17:00] INFO: Setting definition for host 'WorkbenchHost'.

[11.09.12 00:17:00] INFO: Setting definition for host 'ITLHost'.

[11.09.12 00:17:00] INFO: Setting definition for component 'AuthoringDgraph'.

[11.09.12 00:17:00] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/dgraphs/AuthoringDgraph could not be created for component Discover:AuthoringDgraph.

[11.09.12 00:17:00] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/dgraphs/AuthoringDgraph/dgraph_input could not be created for component Discover:AuthoringDgraph.

[11.09.12 00:17:00] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/dgraphs/AuthoringDgraph/dgraph_input/updates could not be created for component Discover:AuthoringDgraph.

[11.09.12 00:17:00] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/dgraphs/AuthoringDgraph could not be created for component Discover:AuthoringDgraph.

[11.09.12 00:17:00] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_dgidx-output'.

[11.09.12 00:17:00] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_dgidx-output' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host AuthoringMDEXHost.

[11.09.12 00:17:00] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_forge-output'.

[11.09.12 00:17:00] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_forge-output' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host AuthoringMDEXHost.

[11.09.12 00:17:01] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_cumulative-partials'.

[11.09.12 00:17:02] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_cumulative-partials' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host AuthoringMDEXHost.

[11.09.12 00:17:02] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_dgraph-config'.

[11.09.12 00:17:03] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_dgraph-config' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host AuthoringMDEXHost.

[11.09.12 00:17:03] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_local-dgraph-input'.

[11.09.12 00:17:04] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_local-dgraph-input' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host AuthoringMDEXHost.

[11.09.12 00:17:04] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_local-cumulative-partials'.

[11.09.12 00:17:05] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_local-cumulative-partials' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host AuthoringMDEXHost.

[11.09.12 00:17:05] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_local-dgraph-config'.

[11.09.12 00:17:06] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_local-dgraph-config' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host AuthoringMDEXHost.

[11.09.12 00:17:06] INFO: Setting definition for component 'DgraphA1'.

[11.09.12 00:17:07] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/dgraphs/DgraphA1 could not be created for component Discover:DgraphA1.

[11.09.12 00:17:07] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/dgraphs/DgraphA1/dgraph_input could not be created for component Discover:DgraphA1.

[11.09.12 00:17:07] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/dgraphs/DgraphA1/dgraph_input/updates could not be created for component Discover:DgraphA1.

[11.09.12 00:17:07] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/dgraphs/DgraphA1 could not be created for component Discover:DgraphA1.

[11.09.12 00:17:07] INFO: [LiveMDEXHostA] Starting shell utility 'mkpath_dgidx-output'.

[11.09.12 00:17:08] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_dgidx-output' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host LiveMDEXHostA.

[11.09.12 00:17:08] INFO: [LiveMDEXHostA] Starting shell utility 'mkpath_forge-output'.

[11.09.12 00:17:09] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_forge-output' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host LiveMDEXHostA.

[11.09.12 00:17:09] INFO: [LiveMDEXHostA] Starting shell utility 'mkpath_cumulative-partials'.

[11.09.12 00:17:10] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_cumulative-partials' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host LiveMDEXHostA.

[11.09.12 00:17:10] INFO: [LiveMDEXHostA] Starting shell utility 'mkpath_dgraph-config'.

[11.09.12 00:17:11] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_dgraph-config' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host LiveMDEXHostA.

[11.09.12 00:17:12] INFO: [LiveMDEXHostA] Starting shell utility 'mkpath_local-dgraph-input'.

[11.09.12 00:17:12] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_local-dgraph-input' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host LiveMDEXHostA.

[11.09.12 00:17:12] INFO: [LiveMDEXHostA] Starting shell utility 'mkpath_local-cumulative-partials'.

[11.09.12 00:17:13] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_local-cumulative-partials' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host LiveMDEXHostA.

[11.09.12 00:17:13] INFO: [LiveMDEXHostA] Starting shell utility 'mkpath_local-dgraph-config'.

[11.09.12 00:17:14] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_local-dgraph-config' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host LiveMDEXHostA.

[11.09.12 00:17:14] INFO: Setting definition for script 'PromoteAuthoringToLive'.

[11.09.12 00:17:14] INFO: Setting definition for custom component 'WorkbenchManager'.

[11.09.12 00:17:14] INFO: Updating provisioning for host 'ITLHost'.

[11.09.12 00:17:14] INFO: Updating definition for host 'ITLHost'.

[11.09.12 00:17:14] INFO: [ITLHost] Starting shell utility 'mkpath_temp'.

[11.09.12 00:17:16] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_temp' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ITLHost.

[11.09.12 00:17:16] INFO: Setting definition for custom component 'IFCR'.

[11.09.12 00:17:16] INFO: Updating provisioning for host 'ITLHost'.

[11.09.12 00:17:16] INFO: Updating definition for host 'ITLHost'.

[11.09.12 00:17:16] INFO: [ITLHost] Starting shell utility 'mkpath_temp'.

[11.09.12 00:17:17] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_temp' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ITLHost.

[11.09.12 00:17:17] INFO: Setting definition for component 'LogServer'.

[11.09.12 00:17:17] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/logservers/LogServer could not be created for component Discover:LogServer.

[11.09.12 00:17:17] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/logserver_output could not be created for component Discover:LogServer.

[11.09.12 00:17:17] INFO: [ReportGenerationHost] Starting shell utility 'mkpath_input'.

[11.09.12 00:17:18] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_input' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ReportGenerationHost.

[11.09.12 00:17:18] INFO: Setting definition for script 'DaySoFarReports'.

[11.09.12 00:17:18] INFO: Setting definition for script 'DailyReports'.

[11.09.12 00:17:18] INFO: Setting definition for script 'WeeklyReports'.

[11.09.12 00:17:18] INFO: Setting definition for script 'DaySoFarHtmlReports'.

[11.09.12 00:17:18] INFO: Setting definition for script 'DailyHtmlReports'.

[11.09.12 00:17:18] INFO: Setting definition for script 'WeeklyHtmlReports'.

[11.09.12 00:17:18] INFO: Setting definition for component 'WeeklyReportGenerator'.

[11.09.12 00:17:18] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/report_generators/WeeklyReportGenerator could not be created for component Discover:WeeklyReportGenerator.

[11.09.12 00:17:18] WARNING: Provisioning warning: Directory /endeca/apps/Discover/reports/weekly could not be created for component Discover:WeeklyReportGenerator.

[11.09.12 00:17:18] INFO: [ReportGenerationHost] Starting shell utility 'mkpath_input'.

[11.09.12 00:17:20] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_input' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ReportGenerationHost.

[11.09.12 00:17:20] INFO: Setting definition for component 'DailyReportGenerator'.

[11.09.12 00:17:20] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/report_generators/DailyReportGenerator could not be created for component Discover:DailyReportGenerator.

[11.09.12 00:17:20] WARNING: Provisioning warning: Directory /endeca/apps/Discover/reports/daily could not be created for component Discover:DailyReportGenerator.

[11.09.12 00:17:20] INFO: [ReportGenerationHost] Starting shell utility 'mkpath_input'.

[11.09.12 00:17:21] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_input' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ReportGenerationHost.

[11.09.12 00:17:21] INFO: Setting definition for component 'DaySoFarReportGenerator'.

[11.09.12 00:17:21] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/report_generators/DaySoFarReportGenerator could not be created for component Discover:DaySoFarReportGenerator.

[11.09.12 00:17:21] WARNING: Provisioning warning: Directory /endeca/apps/Discover/reports/daysofar could not be created for component Discover:DaySoFarReportGenerator.

[11.09.12 00:17:21] INFO: [ReportGenerationHost] Starting shell utility 'mkpath_input'.

[11.09.12 00:17:22] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_input' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ReportGenerationHost.

[11.09.12 00:17:22] INFO: Setting definition for component 'WeeklyHtmlReportGenerator'.

[11.09.12 00:17:23] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/report_generators/WeeklyHtmlReportGenerator could not be created for component Discover:WeeklyHtmlReportGenerator.

[11.09.12 00:17:23] WARNING: Provisioning warning: Directory /endeca/apps/Discover/reports/weekly could not be created for component Discover:WeeklyHtmlReportGenerator.

[11.09.12 00:17:23] INFO: [ReportGenerationHost] Starting shell utility 'mkpath_input'.

[11.09.12 00:17:24] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_input' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ReportGenerationHost.

[11.09.12 00:17:24] INFO: Setting definition for component 'DailyHtmlReportGenerator'.

[11.09.12 00:17:24] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/report_generators/DailyHtmlReportGenerator could not be created for component Discover:DailyHtmlReportGenerator.

[11.09.12 00:17:24] WARNING: Provisioning warning: Directory /endeca/apps/Discover/reports/daily could not be created for component Discover:DailyHtmlReportGenerator.

[11.09.12 00:17:24] INFO: [ReportGenerationHost] Starting shell utility 'mkpath_input'.

[11.09.12 00:17:25] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_input' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ReportGenerationHost.

[11.09.12 00:17:25] INFO: Setting definition for component 'DaySoFarHtmlReportGenerator'.

[11.09.12 00:17:25] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/report_generators/DaySoFarHtmlReportGenerator could not be created for component Discover:DaySoFarHtmlReportGenerator.

[11.09.12 00:17:25] WARNING: Provisioning warning: Directory /endeca/apps/Discover/reports/daysofar could not be created for component Discover:DaySoFarHtmlReportGenerator.

[11.09.12 00:17:25] INFO: [ReportGenerationHost] Starting shell utility 'mkpath_input'.

[11.09.12 00:17:25] INFO: Setting definition for script 'BaselineUpdate'.

[11.09.12 00:17:26] INFO: Setting definition for script 'PartialUpdate'.

[11.09.12 00:17:26] INFO: Setting definition for component 'Forge'.

[11.09.12 00:17:26] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/forges/Forge could not be created for component Discover:Forge.

[11.09.12 00:17:26] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/processing could not be created for component Discover:Forge.

[11.09.12 00:17:26] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/forge_output could not be created for component Discover:Forge.

[11.09.12 00:17:26] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/processing could not be created for component Discover:Forge.

[11.09.12 00:17:26] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/temp could not be created for component Discover:Forge.

[11.09.12 00:17:26] INFO: [ITLHost] Starting shell utility 'mkpath_incoming'.

[11.09.12 00:17:27] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_incoming' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ITLHost.

[11.09.12 00:17:27] INFO: [ITLHost] Starting shell utility 'mkpath_temp'.

[11.09.12 00:17:28] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_temp' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ITLHost.

[11.09.12 00:17:28] INFO: Setting definition for component 'PartialForge'.

[11.09.12 00:17:29] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/forges/PartialForge could not be created for component Discover:PartialForge.

[11.09.12 00:17:29] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/partials/processing could not be created for component Discover:PartialForge.

[11.09.12 00:17:29] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/partials/forge_output could not be created for component Discover:PartialForge.

[11.09.12 00:17:29] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/partials/processing could not be created for component Discover:PartialForge.

[11.09.12 00:17:29] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/temp could not be created for component Discover:PartialForge.

[11.09.12 00:17:29] INFO: [ITLHost] Starting shell utility 'mkpath_incoming'.

[11.09.12 00:17:29] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_incoming' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ITLHost.

[11.09.12 00:17:29] INFO: [ITLHost] Starting shell utility 'mkpath_cumulative-partials'.

[11.09.12 00:17:29] WARNING: Failed to create custom directory: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Utility 'mkpath_cumulative-partials' failed. Refer to utility logs in [ENDECA_CONF]/logs/shell on host ITLHost.

[11.09.12 00:17:29] INFO: Setting definition for component 'Dgidx'.

[11.09.12 00:17:29] WARNING: Provisioning warning: Directory /endeca/apps/Discover/logs/dgidxs/Dgidx could not be created for component Discover:Dgidx.

[11.09.12 00:17:29] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/forge_output could not be created for component Discover:Dgidx.

[11.09.12 00:17:29] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/dgidx_output could not be created for component Discover:Dgidx.

[11.09.12 00:17:29] WARNING: Provisioning warning: Directory /endeca/apps/Discover/data/temp could not be created for component Discover:Dgidx.

[11.09.12 00:17:29] INFO: Definition updated.

[11.09.12 00:17:29] INFO: Provisioning site from prototype...

[11.09.12 00:17:30] SEVERE: Precondition Failed (412): Cannot copy JcrNodeResource, type=endeca/site, superType=null, path=/prototype/default to /sites/Discover: destination exists

Occurred while executing line 3 of valid BeanShell script:

[[

3| IFCR.provisionSite();

4| WorkbenchManager.updateWsConfig();

]]

[11.09.12 00:17:30] SEVERE: Caught an exception while invoking method 'run' on object 'InitialSetup'. Releasing locks.

Caused by java.lang.reflect.InvocationTargetException

sun.reflect.NativeMethodAccessorImpl invoke0 - null

Caused by com.endeca.soleng.eac.toolkit.exception.AppControlException

com.endeca.soleng.eac.toolkit.script.Script runBeanShellScript - Error executing valid BeanShell script.

Caused by com.endeca.soleng.eac.toolkit.utility.IFCRUtility$HttpStatusException

com.endeca.soleng.eac.toolkit.utility.IFCRUtility execute - Precondition Failed (412): Cannot copy JcrNodeResource, type=endeca/site, superType=null, path=/prototype/default to /sites/Discover: destination exists

=================================================================================================================================================

Make sure entries in set_template.sh are correct regarding the host and port for the below error

[root@endeca control]# sh initialize_services.sh --force

Removing existing application provisioning...

[11.09.12 00:56:22] INFO: Removing application. Any active components will be forced to stop.

[11.09.12 00:56:22] INFO: Removing Workbench config files.

[11.09.12 00:56:22] INFO: [ITLHost] Starting shell utility 'emgr_update_remove_all_settings'.

[11.09.12 00:56:24] INFO: Removing definition for custom component 'WorkbenchManager'.

[11.09.12 00:56:24] INFO: Updating provisioning for host 'ITLHost'.

[11.09.12 00:56:24] INFO: Updating definition for host 'ITLHost'.

[11.09.12 00:56:24] INFO: Removing definition for custom component 'IFCR'.

[11.09.12 00:56:24] INFO: Updating provisioning for host 'ITLHost'.

[11.09.12 00:56:24] INFO: Updating definition for host 'ITLHost'.

[11.09.12 00:56:24] INFO: Removing site Discover from IFCR...

[11.09.12 00:56:25] INFO: Node did not exist, no deletion necessary: sites-live/Discover

[11.09.12 00:56:25] INFO: Finished removing site from IFCR.

[11.09.12 00:56:25] INFO: Removing definition for application 'Discover'.

[11.09.12 00:56:25] INFO: Application 'Discover' removed.

Setting EAC provisioning and performing initial setup...

[11.09.12 00:56:28] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.

[11.09.12 00:56:28] INFO: Setting definition for application 'Discover'.

[11.09.12 00:56:28] INFO: Setting definition for host 'AuthoringMDEXHost'.

[11.09.12 00:56:29] INFO: Setting definition for host 'LiveMDEXHostA'.

[11.09.12 00:56:29] INFO: Setting definition for host 'ReportGenerationHost'.

[11.09.12 00:56:29] INFO: Setting definition for host 'WorkbenchHost'.

[11.09.12 00:56:29] INFO: Setting definition for host 'ITLHost'.

[11.09.12 00:56:29] INFO: Setting definition for component 'AuthoringDgraph'.

[11.09.12 00:56:29] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_dgidx-output'.

[11.09.12 00:56:30] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_forge-output'.

[11.09.12 00:56:32] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_cumulative-partials'.

[11.09.12 00:56:32] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_dgraph-config'.

[11.09.12 00:56:32] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_local-dgraph-input'.

[11.09.12 00:56:33] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_local-cumulative-partials'.

[11.09.12 00:56:34] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_local-dgraph-config'.

[11.09.12 00:56:35] INFO: Setting definition for component 'DgraphA1'.

[11.09.12 00:56:36] INFO: Setting definition for script 'PromoteAuthoringToLive'.

[11.09.12 00:56:36] INFO: Setting definition for custom component 'WorkbenchManager'.

[11.09.12 00:56:36] INFO: Updating provisioning for host 'ITLHost'.

[11.09.12 00:56:36] INFO: Updating definition for host 'ITLHost'.

[11.09.12 00:56:36] INFO: [ITLHost] Starting shell utility 'mkpath_temp'.

[11.09.12 00:56:37] INFO: Setting definition for custom component 'IFCR'.

[11.09.12 00:56:37] INFO: Updating provisioning for host 'ITLHost'.

[11.09.12 00:56:37] INFO: Updating definition for host 'ITLHost'.

[11.09.12 00:56:38] INFO: Setting definition for component 'LogServer'.

[11.09.12 00:56:38] INFO: [ReportGenerationHost] Starting shell utility 'mkpath_input'.

[11.09.12 00:56:39] INFO: Setting definition for script 'DaySoFarReports'.

[11.09.12 00:56:39] INFO: Setting definition for script 'DailyReports'.

[11.09.12 00:56:39] INFO: Setting definition for script 'WeeklyReports'.

[11.09.12 00:56:39] INFO: Setting definition for script 'DaySoFarHtmlReports'.

[11.09.12 00:56:39] INFO: Setting definition for script 'DailyHtmlReports'.

[11.09.12 00:56:39] INFO: Setting definition for script 'WeeklyHtmlReports'.

[11.09.12 00:56:39] INFO: Setting definition for component 'WeeklyReportGenerator'.

[11.09.12 00:56:40] INFO: Setting definition for component 'DailyReportGenerator'.

[11.09.12 00:56:40] INFO: Setting definition for component 'DaySoFarReportGenerator'.

[11.09.12 00:56:40] INFO: Setting definition for component 'WeeklyHtmlReportGenerator'.

[11.09.12 00:56:40] INFO: Setting definition for component 'DailyHtmlReportGenerator'.

[11.09.12 00:56:40] INFO: Setting definition for component 'DaySoFarHtmlReportGenerator'.

[11.09.12 00:56:41] INFO: Setting definition for script 'BaselineUpdate'.

[11.09.12 00:56:41] INFO: Setting definition for script 'PartialUpdate'.

[11.09.12 00:56:41] INFO: Setting definition for component 'Forge'.

[11.09.12 00:56:41] INFO: [ITLHost] Starting shell utility 'mkpath_incoming'.

[11.09.12 00:56:42] INFO: Setting definition for component 'PartialForge'.

[11.09.12 00:56:42] INFO: [ITLHost] Starting shell utility 'mkpath_incoming'.

[11.09.12 00:56:43] INFO: Setting definition for component 'Dgidx'.

[11.09.12 00:56:43] INFO: Definition updated.

[11.09.12 00:56:43] INFO: Provisioning site from prototype...

[11.09.12 00:56:44] INFO: Finished provisioning site from prototype.

[11.09.12 00:56:44] INFO: Uploading config files to Workbench.

[11.09.12 00:56:44] INFO: [ITLHost] Starting shell utility 'emgr_update_update_mgr_settings'.

[11.09.12 00:56:45] INFO: Finished uploading config files to Workbench.

Finished updating EAC.

Importing sample content...

[11.09.12 00:56:46] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.

[11.09.12 00:56:47] INFO: Definition has not changed.

[11.09.12 00:56:47] INFO: Packaging contents for upload...

[11.09.12 00:56:47] INFO: Finished packaging contents.

[11.09.12 00:56:47] INFO: Uploading contents to: http://23.232.325.323:8006/ifcr/sites/Discover/

[11.09.12 00:56:49] INFO: Finished uploading contents.

Finished importing initial content

Importing editors configuration...

[11.09.12 00:56:51] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.

[11.09.12 00:56:51] INFO: Definition has not changed.

[11.09.12 00:56:51] INFO: Packaging contents for upload...

[11.09.12 00:56:51] INFO: Finished packaging contents.

[11.09.12 00:56:51] INFO: Uploading contents to: http://23.232.325.323:8006/ifcr/sites/Discover/configuration/tools/xmgr

[11.09.12 00:56:52] INFO: Finished uploading contents.

Finished importing editors configuration

Importing templates...

Removing existing cartridge templates for Discover

ERROR: <head>

ERROR: Failed to acquire lock.

ERROR: Failed to remove app config. Make sure you can connect to http://23.232.325.323.

=================================================================================================================================================

Clean Run

[endeca@endeca control]$ sh initialize_services.sh --force

Removing existing application provisioning...

[11.09.12 19:45:06] INFO: Removing application. Any active components will be forced to stop.

[11.09.12 19:45:06] INFO: Removing Workbench config files.

[11.09.12 19:45:06] INFO: [ITLHost] Starting shell utility 'emgr_update_remove_all_settings'.

[11.09.12 19:45:08] INFO: Removing definition for custom component 'WorkbenchManager'.

[11.09.12 19:45:08] INFO: Updating provisioning for host 'ITLHost'.

[11.09.12 19:45:08] INFO: Updating definition for host 'ITLHost'.

[11.09.12 19:45:08] INFO: Removing definition for custom component 'IFCR'.

[11.09.12 19:45:08] INFO: Updating provisioning for host 'ITLHost'.

[11.09.12 19:45:08] INFO: Updating definition for host 'ITLHost'.

[11.09.12 19:45:08] INFO: Removing site Discover from IFCR...

[11.09.12 19:45:09] INFO: Node did not exist, no deletion necessary: sites-live/Discover

[11.09.12 19:45:09] INFO: Finished removing site from IFCR.

[11.09.12 19:45:09] INFO: Removing definition for application 'Discover'.

[11.09.12 19:45:09] INFO: Application 'Discover' removed.

Setting EAC provisioning and performing initial setup...

[11.09.12 19:45:12] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.

[11.09.12 19:45:12] INFO: Setting definition for application 'Discover'.

[11.09.12 19:45:12] INFO: Setting definition for host 'AuthoringMDEXHost'.

[11.09.12 19:45:13] INFO: Setting definition for host 'LiveMDEXHostA'.

[11.09.12 19:45:13] INFO: Setting definition for host 'ReportGenerationHost'.

[11.09.12 19:45:13] INFO: Setting definition for host 'WorkbenchHost'.

[11.09.12 19:45:13] INFO: Setting definition for host 'ITLHost'.

[11.09.12 19:45:13] INFO: Setting definition for component 'AuthoringDgraph'.

[11.09.12 19:45:13] INFO: Setting definition for component 'DgraphA1'.

[11.09.12 19:45:14] INFO: Setting definition for script 'PromoteAuthoringToLive'.

[11.09.12 19:45:14] INFO: Setting definition for custom component 'WorkbenchManager'.

[11.09.12 19:45:14] INFO: Updating provisioning for host 'ITLHost'.

[11.09.12 19:45:14] INFO: Updating definition for host 'ITLHost'.

[11.09.12 19:45:14] INFO: Setting definition for custom component 'IFCR'.

[11.09.12 19:45:14] INFO: Updating provisioning for host 'ITLHost'.

[11.09.12 19:45:14] INFO: Updating definition for host 'ITLHost'.

[11.09.12 19:45:15] INFO: Setting definition for component 'LogServer'.

[11.09.12 19:45:15] INFO: Setting definition for script 'DaySoFarReports'.

[11.09.12 19:45:15] INFO: Setting definition for script 'DailyReports'.

[11.09.12 19:45:15] INFO: Setting definition for script 'WeeklyReports'.

[11.09.12 19:45:15] INFO: Setting definition for script 'DaySoFarHtmlReports'.

[11.09.12 19:45:15] INFO: Setting definition for script 'DailyHtmlReports'.

[11.09.12 19:45:15] INFO: Setting definition for script 'WeeklyHtmlReports'.

[11.09.12 19:45:15] INFO: Setting definition for component 'WeeklyReportGenerator'.

[11.09.12 19:45:16] INFO: Setting definition for component 'DailyReportGenerator'.

[11.09.12 19:45:16] INFO: Setting definition for component 'DaySoFarReportGenerator'.

[11.09.12 19:45:16] INFO: Setting definition for component 'WeeklyHtmlReportGenerator'.

[11.09.12 19:45:16] INFO: Setting definition for component 'DailyHtmlReportGenerator'.

[11.09.12 19:45:16] INFO: Setting definition for component 'DaySoFarHtmlReportGenerator'.

[11.09.12 19:45:17] INFO: Setting definition for script 'BaselineUpdate'.

[11.09.12 19:45:17] INFO: Setting definition for script 'PartialUpdate'.

[11.09.12 19:45:17] INFO: Setting definition for component 'Forge'.

[11.09.12 19:45:17] INFO: Setting definition for component 'PartialForge'.

[11.09.12 19:45:17] INFO: Setting definition for component 'Dgidx'.

[11.09.12 19:45:18] INFO: Definition updated.

[11.09.12 19:45:18] INFO: Provisioning site from prototype...

[11.09.12 19:45:18] INFO: Finished provisioning site from prototype.

[11.09.12 19:45:18] INFO: Uploading config files to Workbench.

[11.09.12 19:45:18] INFO: [ITLHost] Starting shell utility 'emgr_update_update_mgr_settings'.

[11.09.12 19:45:19] INFO: Finished uploading config files to Workbench.

Finished updating EAC.

Importing sample content...

[11.09.12 19:45:21] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.

[11.09.12 19:45:21] INFO: Definition has not changed.

[11.09.12 19:45:21] INFO: Packaging contents for upload...

[11.09.12 19:45:21] INFO: Finished packaging contents.

[11.09.12 19:45:21] INFO: Uploading contents to: http://23.232.325.323:8006/ifcr/sites/Discover/

[11.09.12 19:45:22] INFO: Finished uploading contents.

Finished importing initial content

Importing editors configuration...

[11.09.12 19:45:24] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.

[11.09.12 19:45:24] INFO: Definition has not changed.

[11.09.12 19:45:24] INFO: Packaging contents for upload...

[11.09.12 19:45:24] INFO: Finished packaging contents.

[11.09.12 19:45:24] INFO: Uploading contents to: http://23.232.325.323:8006/ifcr/sites/Discover/configuration/tools/xmgr

[11.09.12 19:45:25] INFO: Finished uploading contents.

Finished importing editors configuration

Importing templates...

Removing existing cartridge templates for Discover

Setting new cartridge templates for Discover

Finished setting templates

Finished importing templates

[endeca@endeca control]$

================================================================================================================================================

Running baseline

[endeca@endeca control]$ sh load_baseline_test_data.sh

Setting flag 'baseline_data_ready' in the EAC.

[endeca@endeca control]$ sh baseline_update.sh

[11.09.12 19:46:56] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.

[11.09.12 19:46:57] INFO: Definition has not changed.

[11.09.12 19:46:57] INFO: Starting baseline update script.

[11.09.12 19:46:57] INFO: Acquired lock 'update_lock'.

[11.09.12 19:46:57] INFO: [ITLHost] Starting shell utility 'move_-_to_processing'.

[11.09.12 19:46:57] INFO: [ITLHost] Starting copy utility 'fetch_config_to_input_for_forge_Forge'.

[11.09.12 19:46:58] INFO: [ITLHost] Starting backup utility 'backup_log_dir_for_component_Forge'.

[11.09.12 19:47:00] INFO: [ITLHost] Starting component 'Forge'.

[11.09.12 19:47:05] INFO: [ITLHost] Starting backup utility 'backup_log_dir_for_component_Dgidx'.

[11.09.12 19:47:06] INFO: [ITLHost] Starting component 'Dgidx'.

[11.09.12 19:47:29] INFO: [AuthoringMDEXHost] Starting copy utility 'copy_index_to_host_AuthoringMDEXHost_AuthoringDgraph'.

[11.09.12 19:47:30] INFO: Applying index to dgraphs in restart group 'A'.

[11.09.12 19:47:30] INFO: [AuthoringMDEXHost] Starting shell utility 'mkpath_dgraph-input-new'.

[11.09.12 19:47:31] INFO: [AuthoringMDEXHost] Starting copy utility 'copy_index_to_temp_new_dgraph_input_dir_for_AuthoringDgraph'.

[11.09.12 19:47:32] INFO: [AuthoringMDEXHost] Starting shell utility 'move_dgraph-input_to_dgraph-input-old'.

[11.09.12 19:47:32] INFO: [AuthoringMDEXHost] Starting shell utility 'move_dgraph-input-new_to_dgraph-input'.

[11.09.12 19:47:34] INFO: [AuthoringMDEXHost] Starting backup utility 'backup_log_dir_for_component_AuthoringDgraph'.

[11.09.12 19:47:35] INFO: [AuthoringMDEXHost] Starting component 'AuthoringDgraph'.

[11.09.12 19:47:41] INFO: Publishing Workbench 'authoring' configuration to MDEX 'AuthoringDgraph'

[11.09.12 19:47:41] INFO: Pushing authoring content to dgraph: AuthoringDgraph

[11.09.12 19:47:42] INFO: Finished pushing content to dgraph.

[11.09.12 19:47:42] INFO: [AuthoringMDEXHost] Starting shell utility 'rmdir_dgraph-input-old'.

[11.09.12 19:47:42] INFO: [LiveMDEXHostA] Starting shell utility 'cleanDir_local-dgraph-input'.

[11.09.12 19:47:43] INFO: [LiveMDEXHostA] Starting copy utility 'copy_index_to_host_LiveMDEXHostA_DgraphA1'.

[11.09.12 19:47:44] INFO: Applying index to dgraphs in restart group '1'.

[11.09.12 19:47:44] INFO: [LiveMDEXHostA] Starting shell utility 'mkpath_dgraph-input-new'.

[11.09.12 19:47:44] INFO: [LiveMDEXHostA] Starting copy utility 'copy_index_to_temp_new_dgraph_input_dir_for_DgraphA1'.

[11.09.12 19:47:46] INFO: [LiveMDEXHostA] Starting shell utility 'move_dgraph-input_to_dgraph-input-old'.

[11.09.12 19:47:46] INFO: [LiveMDEXHostA] Starting shell utility 'move_dgraph-input-new_to_dgraph-input'.

[11.09.12 19:47:47] INFO: [LiveMDEXHostA] Starting backup utility 'backup_log_dir_for_component_DgraphA1'.

[11.09.12 19:47:49] INFO: [LiveMDEXHostA] Starting component 'DgraphA1'.

[11.09.12 19:47:55] INFO: Publishing Workbench 'live' configuration to MDEX 'DgraphA1'

[11.09.12 19:47:55] INFO: Pushing live content to dgraph: DgraphA1

[11.09.12 19:47:55] INFO: Finished pushing content to dgraph.

[11.09.12 19:47:55] INFO: [LiveMDEXHostA] Starting shell utility 'rmdir_dgraph-input-old'.

[11.09.12 19:47:55] INFO: [ITLHost] Starting copy utility 'fetch_post_forge_dimensions_to_ws_temp_dir_-root-data-endeca-apps-Discover-data-workbench-temp'.

[11.09.12 19:47:55] INFO: Uploading post-Forge Dimensions to Workbench.

[11.09.12 19:47:55] INFO: [ITLHost] Starting shell utility 'emgr_update_set_post_forge_dims'.

[11.09.12 19:47:56] INFO: [ITLHost] Starting backup utility 'backup_state_dir_for_component_Forge'.

[11.09.12 19:47:58] INFO: [ITLHost] Starting backup utility 'backup_index_Dgidx'.

[11.09.12 19:47:59] INFO: [ReportGenerationHost] Starting backup utility 'backup_log_dir_for_component_LogServer'.

[11.09.12 19:48:00] INFO: [ReportGenerationHost] Starting component 'LogServer'.

[11.09.12 19:48:01] INFO: Released lock 'update_lock'.

[11.09.12 19:48:01] INFO: Baseline update script finished.

[endeca@endeca control]$

=================================================================================================================================================

Running promote script

[endeca@endeca control]$ sh promote_content.sh

[11.09.12 19:48:53] INFO: Checking definition from AppConfig.xml against existing EAC provisioning.

[11.09.12 19:48:54] INFO: Definition has not changed.

[11.09.12 19:48:54] INFO: Promoting IFCR content...

[11.09.12 19:48:55] INFO: Finished promoting IFCR content.

=================================================================================================================================================

Oracle Endeca Common Performance Problems

2012-11-01T00:45:00.003-07:00

Info from MDEX Engine Performance Testing Best Practices

Engine Feature/Query function	Comments
setNavAllRefinements / allgroups=1	Automatically expands all of the available dimensions and calculates available values for each dimension. The operation is very computational intensive as it requires the MDEX Engine to calculate not only which dimension values are available, for all possible dimensions, but depending on configuration of the index, calculates the number of available records for each dimension value. The problem can be seen in the MDEX Engine statistics page HotSpots section. The Navigation Query Total line will have the highest percent time spent out of the total time spent inside the engine. It is easy to confirm whether this is an issue by creating a new version of the ENEPerf query log, replacing allgroups=1 with allgroups=0. The new ENEPerf log should result in significant improvement in the MDEX Engine performance.
Exact, Phrase, and Proximity relevance ranking modules	Exact, Phrase, and Proximity relevance ranking modules can be expensive. The lower in your strategy you place them, the better.
Large response sizes	Large response sizes can expose network bottlenecks or cause responses to take longer to travel over the network to the application server. A rule of thumb is that average response sizes >500k in your Cheetah reports may be cause for concern, especially in a high traffic site or 100 megabit Ethernet environment.
Complex analytics queries	Complex analytics queries can be computationally expensive.
Text searching against the entire record set.	Text searching against a very large (> 30 million) record set can be expensive. It is better to apply a record filter or force the user to navigate to a more manageable set before allowing text searches.
Large Flat Dimensions (no hierarchy or sifting)	Dimensions that have a flat list of thousands of dimension values can degrade performance of the MDEX Engine, especially if the dimensions are expanded. Create a hierarchy or configure precedence rules to reduce the number of calculations required and to ensure that thousands of values are not returned at once
Wildcarding	Wildcarding, especially on large text properties or very broad queries (such as ab) can be extremely computationally expensive.
Interactions of large thesaurus + spelling + stemming on large datasets	The interaction of the various query expansion modules when the dataset and thesaurus are large can be expensive.
Frequent Partial Updates	Applying partial updates flushes the MDEX Engine cache forcing subsequent queries to be fully recalculated. Very frequent partial updates can essentially cause the MDEX Engine to operate without a cache, which can dramatically impact performance.
Geospatial range filters on the entire record set	Geospatial filtering is an expensive operation depending on the number of records it is applied against. Whenever possible, use record filters before applying geospatial filters. For example, first filter to a region (e.g. state, country, or range of postal codes) prior to allowing geospatial filtering within that region.
Not enough physical RAM on server	Not enough physical RAM can cause a dramatic degradation in performance due to OS paging.

Analyze Cheetah Output

Cheetah will produce a lot of useful metrics and information. The complete list is available in the Cheetah documentation that is provided with the Cheetah installation package. It is important to review the entire report for any problem areas, although a few common issues are mentioned below.

To ensure that the overall test is valid, it is useful to first validate that the number of records returned on average is around the expected value. This is especially important if multiple tests are run where the query log is modified for each test. The average number of records can be found in the Response Profiling section (see example below) of the cheetah output under “Number of Base Records in Result Set”. Also verify that the “Number of Requests Returning Zero Results” did not significantly change from one test to another. It is also possible to find potential network issues by comparing engine-only processing time to total response time. The “Response Differential” section of the Cheetah report contains the statistics about the difference in time between the total roundtrip versus MDEX Engine processing. If the Average Differential is significant (>200ms), that could be a sign of a network bottleneck.

Analyze MDEX Engine Stats Page

If you have gotten to this stage of analysis and the performance looks sufficient for your requirements, then this step may not be required. However, if the previous results have indicated that the performance is not sufficient and the cause is not environmental, then the MDEX Engine stats page is a useful to understand how the MDEX Engine spends its processing time. It can help determine which features cause slower than desired performance.

There are a number of questions that this tool can help answer:

· Is the MDEX Engine Total Processing Time too high?

Details -> Server-> Scheduler: Processing Time

This section will help identify if the performance issues are uniform and affect all of the queries (where Average Processing Time is high and Standard Deviation is low) or if the performance numbers are skewed due to high variance in the individual query performance (where Average Processing Time is high and Standard Deviation is high). If the latter is true, it is useful to check the Most Expensive Queries section. This section lists the queries that take the longest to execute. In this situation, it is worthwhile to consider another test case in which the most expensive queries are removed from the ENEPerf query log and the test is rerun. This test will help determine whether the longest running queries have any significant impact on the average performance of the MDEX Engine. Sometimes the list of Top 10 queries provided by the MDEX Engine stats page is not enough. It is always useful to load the resulting MDEX Engine request log into a spreadsheet and sort the queries by their engine processing times.

If the average performance is indeed affected by the few long running queries, the next step is to understand what causes their slow performance. For example, the queries can be extracted out into a separate ENEPerf query log and a short performance test can be run using that log.

· Are MDEX Engine requests queuing?

Details -> Server-> Scheduler: Queue Time Before Processing

“Queue time before processing” reports how long requests spend in the MDEX Engine’s queue waiting to be picked up by an MDEX Engine thread to be processed. If requests are queuing, the easiest resolution would be to expand CPU and MDEX Engine thread capacity. However, the performance impact should be analyzed since increasing the number of queues may actually cause performance degradation after a certain point.

· Which MDEX Engine feature(s) are requiring the most processing time?

Details -> Hotspots

The “Hotspots” section of the MDEX Engine stats page explains which MDEX Engine features require the most processing time. Please note that “Navigation/RecSearch query total” can further be expanded by clicking on the triangles on the right of the Total column (see the screenshot above).. The expanded view shows details that can help identify the root cause of the performance issue. The important Total column shows which MDEX Engine feature consumes the most time out of the total time spent.

See below for an explanation of the various features listed in the HotSpots section:

Navigation/RecSearch query total	Time taken by MDEX Engine to perform Refinements calculations and keyword search functionality combined
Record Search Performance	Time taken by MDEX Engine to perform Record Search (Ntk/Ntt parameters, excluding relevance ranking)
Navigation Performance	Time taken by MDEX Engine to perform Navigation (N parameter)
Navigation binlist	Time taken by MDEX Engine to calculate which records to return based on the intersection of search results, navigation results, range filters, etc.
Navigation Refinements (LCA/topdown/bottomup)	Time taken by MDEX Engine to calculate which dimension values to display
Navigation Refinements Multi-OR	Time taken by MDEX Engine to calculate which dimvals to display for Multi-OR dimensions
Navigation Refinements Multi-AND	Time taken by MDEX Engine to calculate which dimvals to display for Multi-AND dimensions
Refinement Record Count	Time taken by MDEX Engine to calculate refinement statistics
Clustering performance	Time taken by MDEX Engine to perform record clustering
Record Filter performance	Time taken by MDEX Engine to perform record filters(Nr parameter)
Range Filter performance	Time taken by MDEX Engine to perform range filters (Nf parameter)
Content spotlighting performance	Time taken by MDEX Engine to apply Merchandising Rules, including sorting of Merch Rules results
Dimension Search Performance	Time taken by MDEX Engine to perform dimension search (D parameters)
Spell Engine Performance	Time taken by MDEX Engine to calculate variations of the spelling and come up DYM or Autocorrect options
Page render total	Time taken by MDEX Engine to package up the page of results, including aggregation (e.g. applying rollup, paging, and relevance ranking sort)
Page render/record list	The above time minus packaging the result into a binary structure
Snippeting performance	Time taken by MDEX Engine to find and tag text that matched the keywords

· What was the cache hit ratio?

Cache ->Main Cache

The MDEX Engine cache statistics provide information about the cache Hit/Miss rates, the number and size of evictions, and the size of entries, broken down by functional category. If the results of the performance test show surprisingly good performance, and the MDEX Engine Stats page shows a high Сache Hit Rate (greater than 80%), ensure the performance query log does not contain a high frequency of repeated queries.

If the total number of evictions (# of evictions) is close to or just couple of orders of magnitude smaller than the total number of lookups (# of lookups) and the “Size of Entries” is close to the size of MDEX Engine Cache (--cmem), this may be a sign of cache thrashing. Cache thrashing is a state in which multiple threads are competing for the same cache, and data inserted by one thread is almost immediately evicted by another, resulting in a high number of misses and evictions. One of the side effects of cache thrashing is low CPU utilization due to constant cache synchronization and degraded performance of the MDEX Engine. Although the MDEX Engine implements various caching algorithms that should minimize thrashing, it can still occur and therefore it is important to watch for the symptoms.

The MDEX Engine allows you to tune the amount of RAM allocated to the cache, which can modified via the MDEX Engine --cmem flag. A balance must be found between the size of the MDEX Engine cache and the amount of RAM left for the operating system. Often, the best configuration is found experimentally.

Oracle Endeca Getting Started

2012-04-12T02:33:00.002-07:00

Backbone for all Oracle Endeca Solution is Oracle Endeca Server (formerly Endeca MDEX) which is used as indexing and query engine.

What Is Oracle Endeca & Where it is used

Endeca is a Information Access & Search Platform

Endeca for B2B eCommerce

Endeca for Web Site Search

Endeca for Product Data Navigation, etc.

Oracle Endeca Different Components

Endeca Information Transformation Layer (ITL)
Oracle Endeca Server (Formerly Endeca MDEX)
Oracle Endeca Presentation API

ITL is highlighted with dark color because it’s offline process where as Oracle Endeca Server and Oracle Endeca Presentation API is online process.

Understanding Different Components

Technical Overview

For this explanation we are considering source data as Flat File to ITL process. But it’s not just limited to that. Oracle Endeca accepts any of the following formats till date.

Forge in action: Once the data is in the ITL process Forge maps the source data to Oracle Endeca Properties and Dimension along with all configuration applied that were developed using Developer Studio.

Dgidx in action: Once data is forged, dgidx index the data and converts it to Endeca records which are finally given to Dgraph.

Dgraph in action: Once Oracle Endeca records are ready, enjoy ease and smooth Navigation and Search experience by query Oracle Endeca Server.

Records, Dimension and Properties

Records

Endeca records are the entities in your data set that users are navigating to or searching for. Below image show endeca record information

Dimensions

Dimensions provide the logical structure for organizing the records in your data set. For example below categories are user to narrow down the result to specific product user is looking for.

Properties

§ Properties mostly contain detail about a record than the higher-level dimension values used for navigation.

Guided Navigation

Guided Navigation is the presentation of valid follow-on refinement queries to the user.
You can think of Guided Navigation as the elimination of invalid refinement queries, or dead ends.

An example below shows when user navigates to Carpet Kit. User still has options to navigate to more targeted records he is looking for. Here user can further drill down on Brand, Price, etc.

Using Endeca API

The Endeca Presentation API is based on following three core classes:

HttpENEConnection class enables connections with Oracle Endeca Server.
ENEQuery class builds query to be sent to Oracle Endeca Server.
ENEQueryResults class contains results of Oracle Endeca Server.

Following is JAVA API example where we need to create a connection object to get

Accessing Properties from records

Oracle Endeca Presentation API URL Query Parameters

Endeca Presentation API URL Query Parameters

Oracle Endeca Enterprise Search

2012-03-29T08:30:00.000-07:00

What Is Enterprise Search?

An enterprise search system provides extensive capabilities for searching any number of structured and unstructured data sources with a single query.

So what are structured and unstructured data sources?

Data can be designated as unstructured or structured data for classification within an organization. The term unstructured data refers to any data that has no identifiable structure. For example, images, videos, email, documents and text are all considered to be unstructured data within a dataset. This is in contrast to a database, for example, which is a common example of “structured" data.

How Oracle-Endeca helps Enterprise Search?

Considering Endeca the first really modern platform, created in the late 1990s when it was clear that search was more than just a box on a page. They were just about the first platform to have a fully integrated console that a business user can actually understand and customize as per user search and navigation behavior. They made search relevance mode effective with their module. So it's not just search it's the powerful eCommerce.

On October 18, 2011, Oracle announced it has entered into an agreement to acquire Endeca, a leading provider of unstructured data management, web commerce and business intelligence solutions.

Oracle with Endeca plans to create a comprehensive technology platform to process, store, manage, search and analyze structured and unstructured information together enabling businesses to make stronger and more profitable decisions.

The combination of Oracle ATG Commerce with Endeca's InFront web commerce solutions is expected to enhance cross-channel commerce, merchandising, and online customer experiences. Adding Endeca Latitude software to Oracle Business Intelligence Enterprise Edition (OBI EE) is expected to provide a single unified user interface for analysis of structured and unstructured data.

What features does Oracle-Endeca provides?

Endeca InFront is a leading customer experience management platform that enables businesses to deliver highly targeted and relevant customer experiences online with advanced merchandising and content targeting tools for web commerce.

What this means?

Guidance To Relevant Results

Customers want flexibility to search/navigate based on their preferences, and view content relevant to them. With Oracle Endeca, we get superior quality Search capability to drill down to a specific content relevant to our Search and Navigation help us to get to what we are looking for. Therefore user feels the power of GUIDED SEARCH.

Content Spotlighting

Automatically promote contextually relevant information—such as related articles, promotions, or reviews—based on the search and navigation state.

Endeca Latitude is a technology platform that enables businesses to rapidly develop analytic applications bringing information from many unstructured and structured information sources together.

Information Transformation Layer

Integration with industry-leading ETL tools for data ingest (e.g. Informatica)
Integration with best-of-breed data enrichment tools (e.g. Lexalytics)
Connectors to leading enterprise applications
Best-of-breed crawlers for file systems and Web sites

Discovery Framework

Highly configurable and extensible application framework, built against industry standards
Rich component library with specialized visualization and dash-boarding tools

MDEX Engine

Hybrid search-analytical database for search, navigation, and analytics on structured and unstructured content
Interactive response time at scale