UrchinExperts Analytics Software Support Blog

Update: UTM Update for Remote Logging

noreply@blogger.com (Epik Phil) — Fri, 22 May 2009 18:48:00 +0000

Google is retiring the service.urchin.com server on 1 June, 2009. If your legacy Urchin 5.x installation references this server, you'll need to update the "__utm.js" that's being used on your web pages.

What to do: Check to see if your Urchin Traffic Monitor (UTM) file is referencing this old service. If it is, you must make a change to the javascript or replace it if you wish to continue logging traffic both locally and to the Google Analytics system.

You can either update or replace your existing UTM. Both options are detailed in the following help document at the Urchin Help Center.

Urchin Q and A: Recommended Environment for Andvanced Setup

noreply@blogger.com (Epik Phil) — Fri, 15 May 2009 21:43:00 +0000

Question: What would be the ideal condition to run Urchin without low speed: some servers with Urchin and database in all of them; some servers with Urchin and one master database; 2 or 3 servers with Urchin and some databases configured as master and slave?

Answer: Reporting data is stored in proprietary databases. Reporting data can be configured to be stored on your NAS. The MySQL or PostgreSQL database may also reside on a separate box. You will need a license for each Urchin installation that is crunching logs. Start with the minimum system requirements to see how Urchin's log processing performs. With a baseline measurement, you'll be able to easily scale from there.

Server specifications and configuration are really dependent upon your unique situation. You must set up a test environment to see what kind of resources you are going to need.

Urchin Q and A: Special Data Storage Circumstance

noreply@blogger.com (Epik Phil) — Fri, 15 May 2009 21:38:00 +0000

Question: How much disk space, memory and processing is required if we have to store 1-year's of data information for 50.000 domains with about 50 million pageviews per day?

Answer: Server specifications are really based on size of log data being crunched, and resulting amount of reporting data. Report data is going to be ~10% of raw log data. Try starting with the minimum system specifications, and scale from there:

* 2.5Ghz Pentium dual-core
* 2GB RAM per Core (i.e. dual-core = 4GB; quad-core = 16GB)
* Ultra2/Wide SCSI hard disk (such as a Seagate Cheetah)
* Backup utility external to Urchin

Urchin Q and A: How Urchin 6 Licensing Works

noreply@blogger.com (Epik Phil) — Mon, 11 May 2009 16:19:00 +0000

Question: How does Urchin 6 licensing work?

Answer: The Urchin activation process is as follows:

When installing a serial code, your Urchin machine sends the serial code and key code to the Urchin licensing server.

Urchin verifies these two values and returns a license key enabling the customer to complete the installation through the following process:

1. ugetlicense utility automates the sending of the serial code and key code to the Urchin license center

2. A unique license key is returned and written to the Urchin configuration. You can use uconf-export to export your Urchin configuration which contains this value

3. The key code is computed using:

* Inode numbers of configuration database files (Unix systems)
* System identifiers (Windows systems)
* Note that the IP address is NOT used in the license key!
* Also note that copying Urchin config databases or moving Urchin to another disk or partition will change the computed key code

4. Once activated no further network-based license authentication is done by any Urchin component.

Urchin 6.501 Changelist (Mar 26, 2009)

noreply@blogger.com (Epik Phil) — Fri, 10 Apr 2009 20:02:00 +0000

From the Urchin Help Center:

Urchin 6.501 is a minor release fixing the following four issues in the Urchin 6.500 release. (Scroll down to see the Urchin 6.500 changelist.)

Downloading CPC data in v6.500 frequently took a long time.
- o v6.501 fixes a performance issue that affected accounts with a very large CPC history. v6.501 can download up to 256 MB of GZIPPED historical CPC data and process it much more quickly. Due to Google AdWords limitations, however, downloads over this threshold are not supported.
Proxy Settings
- o Fixed v6.500 proxy issue when downloading CPC data. All traffic is now being redirected through proxy (with or without authentication).
  o For new proxy setup instructions, see Downloading CPC Data via proxy (6.501+) in CPC Data Import Utilities.
64-bit Visitor ID overflow issue
- o We continue to recommend that you use "urchin.js", not "ga.js". However, for ga.js users, v6.501 addresses the issue of the utma cookie containing a 64-bit value that is used during visitor_id generation. The Urchin parser has been modified to handle both 32 and 64 bit values in the utma cookie.
uconf-manager
- o The v6.501 uconf-manager can perform 'add', 'delete' and 'update' operations on the uprofiles table.

Download 6.501

Urchin Q and A: Differences in U5 and U6 Data Results

noreply@blogger.com (Epik Phil) — Fri, 10 Apr 2009 19:48:00 +0000

Question: We see differences in Urchin 5 versus Urchin 6 results. There is also some information that we used to get in urchin 5 that we no longer see in urchin 6. Are you able to speak with us about a few examples?

Answer: In comparing any two web analytics packages (including U5 vs U6), a difference in data should be expected. It's not always easy to compare U5 to U6 because of the change in report organization and naming. In addition, there are a lot of reasons why the numbers may not be exact. Here's some of the things you should check:

1. Is the Pageview Mimes Match field in the Reporting tab of the profile's configuration the same for both profiles?
2. Is the Session Timeout the same for both profiles?
3. Do you have Session Requires Pageview on in Urchin 5 (It's always "on" in Urchin 6)?
4. Is Log Tracking on for both profiles?
5. Are you using the same Visitor Tracking Method for both installations?
6. Are you processing the log source from both profiles, or are you processing in Urchin 5 and then migrating the data to Urchin 6?
7. Are metrics different for the most recent day, week, and month and for historic days, weeks, and months?
8. Are any other metrics the same?

Urchin 6 has changed a good deal since Urchin was originally acquired (U5.7). Thus, proprietary back-end algorithms may also be different.

Urchin Q and A: Fixing Daylight Savings Error

noreply@blogger.com (Epik Phil) — Wed, 11 Mar 2009 22:40:00 +0000

Question: Ever since daylight savings, we get the following error when Urchin tries to process logs:

ERROR: (9003-942-883) Failed to query the database
DETAIL: Incorrect datetime value: '2009-03-08 02:59:50' for column 'utpt_data_start_time' at row 1

How do we fix this?

Answer: Reconfigure your MySQL installation and uncheck the checkbox for Enable Strict Mode.

Then, stop and restart the Urchin service.

If that doesn't work, from the MySQL Command Line
Prompt enter the following command:

mysql>SET @@global.sql_mode='MYSQL40'

Urchin 6 Software Changelist (v. 6.500)

noreply@blogger.com (Epik Phil) — Tue, 23 Dec 2008 19:54:00 +0000

Google recently released v.6.500 update for Urchin 6 software. Here is a summary of the changelist:

New Cost Per Click (CPC) Data Import Manager allows to create 'CPC Sources'

Reporting: Urchin now recognizes "Chrome" as a browser and "Android" as a platform

Profiles: Global Settings for DB Tables are now properly appled. Reporting data is deleted correctly from a file system on Windows when "Delete All Data" is clicked or profile is deleted.

Scheduler and History: Profile scheduled weekly no longer runs several times. Geo-DB moved to separate screen

Users and Groups: Override Dashboard option on Group Management screen fixed

Installer: 404 error when a different port was specified during Windows upgrade fixed

Utilities: Functionality of uconf-manager utility has been extended. Works on several other tables in addition to "uusers" table

Running Urchin Profile Scheduler in console mode on Windows fixed

Other: Updated geo database up until 12/2008 geo data, various localization issues repaired, Urchin embedded help topics and updated articles on Urchin Help Center.

For further information, refer to the Urchin Help Center.

Go to the Urchin Download page to get v.6.500.

Urchin Q and A: Rolling Back to Urchin 5 from 6

noreply@blogger.com (Epik Phil) — Tue, 16 Dec 2008 19:05:00 +0000

Question: Is there a way to manually rollback from Urchin 6 to Urchin 5?

Answer: One way to rollback from Urchin 6 to Urchin 5 is to restore from backup. Another way you can restore Urchin 5 is, if you still have the log files, you can reinstall Urchin 5 and reprocess log files.

Urchin Q and A: Report Conversion from Urchin 5 to 6

noreply@blogger.com (Epik Phil) — Tue, 16 Dec 2008 19:01:00 +0000

Question: I have reports going further back than my logs. How can I convert these reports from Urchin 5 to Urchin 6?

Answer: Urchin provides a script, convert-u5data, to convert report data from U5 to U6. As with the configuration database, the structure of reporting database (that stores profile information) has also been changed in Urchin 6. This utility, convert-u5data, provides an option for importing the reporting database directly from Urchin 5 installation into the Urchin 6 installation. convert-u5data is located in the util directory of the Urchin 6 distribution.

Urchin Q and A: File Migration Slow Going

noreply@blogger.com (Epik Phil) — Tue, 16 Dec 2008 18:57:00 +0000

Question: I am migrating from Urchin 5 to Urchin 6. Why is uconf-import going so slow?

Answer: The migration script chokes on archived files. You have to unpack and delete the archive; then, run the migration script.

Urchin Q and A: Utmlinker Function and UTM Cookies Across Domains

noreply@blogger.com (Epik Phil) — Tue, 16 Dec 2008 18:53:00 +0000

Question: Can you describe in detail how the utmlinker function and utm cookies work across domains?

Answer: Urchin cookies are 1st party. Thus, if you intend to track visitors across a primary domain and/or sub-domains of that primary domain, you have to set cookies to the primary domain. We recommend you set _udn equal to whatever the primary domain is the page resides on. The reason for __utmLinker() is to copy values from one set of cookies to another set, usually between different primary domains all together. The uLink option enables Urchin to execute utmLinker() and also to look for utmLinker values in the URI.
If your site spans multiple primary domains, you will also want to set _uLink=1.

Urchin Q and A: Urchin and *.shtml Extensions

noreply@blogger.com (Epik Phil) — Tue, 16 Dec 2008 18:42:00 +0000

Question: Does Urchin support *.shtml extensions?

Answer: There are no limitations in Urchin with respect to .shtml file extensions. Try adding the extension to the Pageview Mime Match field under Profile Settings > Reporting.

Urchin Q and A: Urchin and Microsoft SQL Server 2005 Support

noreply@blogger.com (-EpikLabs-) — Mon, 24 Nov 2008 17:11:00 +0000

Question:
Does Urchin support Microsoft SQL Server 2005 instead of MySQL?

Answer:
Currently, Urchin does not support MS SQL Server 2005.

Urchin Q and A: What's Taking Up Storage Space

noreply@blogger.com (-EpikLabs-) — Mon, 24 Nov 2008 17:03:00 +0000

Question: Is the main consumer of storage space the raw data and Urchin report data folders?

Answer: Yes, the main consumer of storage space is the raw data and reports. You can estimate the size of the proprietary Urchin databases to be ~10% of the size of the raw logs that were processed to populate the Urchin databases.

Urchin Q and A: MySQL Database Size

noreply@blogger.com (-EpikLabs-) — Mon, 24 Nov 2008 16:57:00 +0000

Question: Can we expect the MySQL database size to grow?

Answer: You can expect the SQL database to grow, only as much as you add users/groups/accounts and profiles to your Urchin configuration.

Urchin Q and A: What's Stored in the MySQL Database

noreply@blogger.com (-EpikLabs-) — Mon, 24 Nov 2008 16:51:00 +0000

Question: What is stored in the MySQL database?

Answer: Urchin configuration data is stored in the SQL database.

Urchin Q and A: Data Storage Format

noreply@blogger.com (-EpikLabs-) — Mon, 24 Nov 2008 16:45:00 +0000

Question: What format are the Urchin reporting data stored in?

Answer:
Urchin report data is stored in proprietary databases.

Urchin Q and A: Traffic Breakdown

noreply@blogger.com (-EpikLabs-) — Mon, 24 Nov 2008 16:42:00 +0000

Question: Does Urchin reporting provide a breakdown of average traffic by hour of the day?

Answer: No, Urchin does not provide a breakdown of average traffic by hour of the day. We lost this reporting after version 3.X.

Urchin Q and A:Report Window Configuration Limit

noreply@blogger.com (-EpikLabs-) — Mon, 24 Nov 2008 16:33:00 +0000

Question: In the report window, there is a drop down where the number of results to display can be changed. The upper limit seems to be 500. Is this upper limit configurable to a larger number?

Answer: Unfortunately, there is no built-in configuration option for the values available in the drop-down. These values are hardcoded into an HTML template (~/urchin/lib/reporting/templates/main_rform.tpl). You can increase this value at your own risk! Bear in mind, this change might slowdown Urchin output and/or cause Urchin server timeouts.

Urchin 6 and Filters

noreply@blogger.com (-EpikLabs-) — Mon, 24 Nov 2008 15:49:00 +0000

Two Filters We Often Recommend:

Filters are applied to the information coming into your Urchin reports in order to manipulate the data and provide accurate reports. Filters can be set up to exclude visits from particular IP addresses, to report only on a sub-domain or directory, or to take dynamic page URLs and convert them into readable text strings.

First, we highly recommend creating a filter to exclude internal traffic from your Urchin reports as this data set isn’t an accurate representation of how your visitors are using your site.

To exclude internal traffic from your Urchin reports, follow these steps:

1 - Navigate to Configuration -> Urchin Profiles -> Filter Manager in the Urchin administration interface.
2 - Click the "Add" button to create a new filter.
3 - Choose "Filter Out Internal Traffic" from the "Select Example Filters" dropdown menu.
4 - Replace the sample IP address in the "Filter Pattern" box with the IP address you wish to filter out.
5 - Click "Finish".
6 - On the next window, click "More Options".
7 - Select all the Profiles you want to apply the filter to in the "Available Profiles" box, and then click the right arrow to transfer them to the "Applied To" box.
8 - Click "Update" to apply the filter to the selected Profiles.

Second, in order to ensure Urchin is not capturing unique page views as multiple pages due to different character cases being used, we highly recommend implementing this filter that forces the URI to lowercase.

As an example, Urchin will think “Default.aspx” and “default.aspx” are two separate pages, thus dividing pageview data.

Urchin has an advanced log setting that can force the URI to lowercase thus consolidating hits to the same page.

To force the URI to lowercase, follow these steps:

1 - Navigate to Log Manager in your Urchin GUI.
2 - Select a log source by clicking on the Edit icon.
3 - Click the "Advanced Settings" tab. There will be an option for "URI Stem to Lower Case". Select the "On" radio button. This will force all URIs in the reports to lower case, thus making the data uniform and easier to analyze.

Implementing these filters will help make the data in your Urchin reports uniform and easier to analyze, and more accurately represent visitor interaction with your website

Urchin Q and A: Moblie Device Activity Tracking and UserAgent Detection

noreply@blogger.com (-EpikLabs-) — Mon, 24 Nov 2008 15:35:00 +0000

Question: What is your recommended solution for mobile device activity tracking to include user agent detection?

Answer: You can use the IP+UserAgent tracking method for mobile browsers. However, you would have to inspect your log files to see if they log device specific information.

Urchin Q and A: Urchin6 and data storage

noreply@blogger.com (-EpikLabs-) — Mon, 24 Nov 2008 15:16:00 +0000

Question: What types of data is stored in the Urchin6 database? Is it only the configuration data or both configuration data and report data?

Answer: The SQL database is used solely for the purposes of storing the configuration data. Report data is still stored in Urchin Software's proprietary databases. .

Urchin Q and A: "Individual Record" Defined

noreply@blogger.com (Epik Phil) — Fri, 24 Oct 2008 21:40:00 +0000

Question: Can you define what an "individual record" is? Is it a single hit from the access log?

Answer: A 'record' is a table row which contains a whole host of information based on the visit. It is not a single hit from the access log.

Urchin Q and A: Urchin 6 Installation and Windows 2000

noreply@blogger.com (Epik Phil) — Wed, 22 Oct 2008 21:59:00 +0000

Question: I have tried to install Urchin 6 on Win2000 server using PostgreSQL and MySQL, and it does not install correctly. The Urchin services crash and the bundled Apache web server service won't start. Do you have any suggestions or tips on how to get this successfully installed?

Answer: There are no known issues with installing Urchin on the platform you described. This leads us to believe there is something at the system level that is interfering with the installation of Urchin. For example, if you are using a password generator, the passwords created may contain special characters that are interfering with Urchin's configuration file (urchin.conf). Urchin stores its database login information unencrypted in the urchin.conf file.

For example, if the password created contains a '#' sign, this is also a comment character in Urchin's configuration file. Thus, your database login will fail, and the Urchin services will keep failing.

Unfortunately, Urchin does not log error messages to aid in troubleshooting installation and runtime errors.