Nir Levy - Eureka

"Type 'Newtonsoft.Json.Linq.JToken' is a recursive collection data contract" While Adding Service Reference in VS2012

noreply@blogger.com (Nir Levy) — Thu, 23 Aug 2012 05:14:33 PDT

Trying to add a service reference in Visual Studio 2012 to Bing Maps SOAP service I got the following error message:

Cannot import wsdl:portType
Detail: An exception was thrown while running a WSDL import extension: System.ServiceModel.Description.DataContractSerializerMessageContractImporter
Error: Type 'Newtonsoft.Json.Linq.JToken' is a recursive collection data contract
...

Following Getting “Recursive collection data contract” when referencing a WCF service with a slightly complex method and Visual Studio 2012 Error Reference.svcmap throwing Error I've had to change the "reuse types in referenced assemblies" from "all" to everything except Newtonsoft.JSON and RestSharp.

Might not be something to write home about, but I'm sure I'll forget where it is was in a year or so.

PS. This seems to be VS2012 only issue, VS2010 worked just fine.

Tech-Ed Eilat 2010

noreply@blogger.com (Nir Levy) — Wed, 17 Nov 2010 23:37:06 PST

So happy to be presenting at Tech-Ed Eilat 2010.
See you all at the Herods Hotel, Tue 30/11 at 11:15 where we would share our lessons from the field on enterprise web content management with SharePoint 2010.

You can read more about our session at the realcommerce web site.

IIS JNLP 404 Problem

noreply@blogger.com (Nir Levy) — Tue, 26 Jan 2010 02:18:59 PST

If your IIS 6 returns a 404 Not Found when you request a Java Web Start jnlp file, then you probably need to add the jnlp mime-type to the IIS configuration.

IIS Settings -> Right click the machine -> mime-types -> new Extension: .jnlp MIME Type: application/x-java-jnlp-file

How to do rewrites in an IIS ISAPI

noreply@blogger.com (Nir Levy) — Sun, 04 Oct 2009 03:13:38 PDT

Note to self: If ever you need to do a rewrite in an IIS ISAPI Filter you just need to re-set the pHeaderInfo's url header. Something like this:

 
pHeaderInfo->GetHeader(pCtxt->m_pFC,
              "url",strUrl.GetBuffer(dwUrlSize+1),&dwUrlSize);
strUrl.ReleaseBuffer(); 
strNewUrl = GET_REWRITE_FOR_URL(strUrl); 
if ( strNewUrl.IsEmpty() == FALSE ) {  
    pHeaderInfo->SetHeader(pCtxt->m_pFC, 
              "url", (LPTSTR)(LPCTSTR)strNewUrl);  
} 
return SF_STATUS_REQ_NEXT_NOTIFICATION;

This is working in IIS4 and above.

Note that if you really need a good re-write/redirect filter for IIS you should look at IIRF ,but since it's not working for IIS4 i had to do this manually

Bing, Google, Nutch and Canonical URLs

noreply@blogger.com (Nir Levy) — Thu, 18 Jun 2009 01:10:38 PDT

Google announced a few months back that they started supporting canonical urls. This is a great feature that we already adopted in a couple of sites (mostly to keep tracking codes from messing with our search engine results.

Unfortunelty, altough Microsoft announced they will support this feature in Live, this was not yet implemented in Bing as of now (Jun 2009).

Nutch, on the other hand, had this on thier TODO list for 1.1.

Also, Google Search Appliance (GSA), is currently assumed by all to not support this (although nobody really knows if it does).

More about canonical URLs:
http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html
http://searchengineland.com/canonical-tag-16537
http://janeandrobot.com/library/url-referrer-tracking

many group by strategies to select single rows per group

noreply@blogger.com (Nir Levy) — Sun, 14 Jun 2009 23:53:18 PDT

Xaprb gives a good explanation for selecting a single row from a group: How to select the first/least/max row per group in SQL at Xaprb.

apc futex_wait lockdown make your apache freeze over

noreply@blogger.com (Nir Levy) — Tue, 04 Aug 2009 00:46:51 PDT

We had the typical LAMP setup going on, with Drupal as the base CMS and APC for bytecode cache. We needed a good caching engine so I figured why not use APC's user cache. Well, we tried the APC Cache Drupal Module which, with minor fixes proved to work very nicely. That is, until we actually put this all thing on production.

The first thing we had was having our apache hang and not respond to any user requests. We susspected network issues, especially since netstat -na showed that all the apache processes were hanging on SYN_WAIT. However, since apache restart solved the issue i started to suspect this was something else.

To make a long (very long) story short, I got strace on our prod machines to find out that apache was either hanging on futex_lock(....FUTEXT_WAIT...) or doing infinte loops on the same functions.

To make even a longer story short, I got gdb installed on those machines and the backtrace clearly indicated that the locks were from APC user-cache calls.

We decided to abandon APC user-cache and switch to memcached which proved faster and had less lockdowns.

The funny thing is that when we talked about this over dinner the same evening a developer from another team just pointed me to this article by one of the APC leaders: (or something) How to Dismantle an APC Bomb which has been around for over a year. I am supprised and shocked that such a information is hidded so well and not mentioned anywere in the docs. Moreover, I went through the APC code again after reading this post (I went through it once when i started analysing the problem) and it seems that this is not even close to being resolved. there are no patches and no TODOs and nothing of the sort. From reading the code the entire user-cache needs a major re-write. What gives?

(this is a post i wrote a couple of months ago, never had time to finish it. ~~Unfortunately this is still not fixed afaik~~)
EDIT (07/2009): http://pecl.php.net/bugs/bug.php?id=15179 reports this to be fixed. If anyone can confirm this please send me a note so that I could update this post

GA_googleAddAttr is not defined error

noreply@blogger.com (Nir Levy) — Thu, 14 May 2009 02:10:20 PDT

If you are using Google Ad Manager you can define custom attributes for targeting.

If you are getting a javascript error GA_googleAddAttr is not defined just make sure to put the call(s) to GA_googleAddAttr in a seperate <script> line, after the GS_googleEnableAllServices.


<script type="text/javascript">
GS_googleAddAdSenseService("ca-pub-0000000000"); 
GS_googleEnableAllServices();
</script>
<script type="text/javascript">
GA_googleAddAttr("ATTRB1", "whatever");
</script>
<script type="text/javascript">
GA_googleAddSlot("ca-pub-0000000000", "slotname_216x311");
</script>

V6 DECODE64 not working properly

noreply@blogger.com (Nir Levy) — Mon, 23 Mar 2009 00:21:19 PDT

It turns out that after all these years, V6's built-in DECODE64 and ENCODE64 actually do not work properly with some binary data.


proc COMPARE_BASE64 {} {
 set code  {2lIU1ZNw5BYvKi79j4L/+GGmjAQK7uiBQG7elDdKZcE=}
 set vgn_result [DECODE64 $code];         #VGN built-in function
 set tcl_result [::base64::decode $code]; #TclLib tcl-only implementation 
 return [join [list \\
   [BIN2HEX $vgn_result] \\
   [BIN2HEX $tcl_result] \\
   [string compare $vgn_result $tcl_result ] \\
  ] "\\r\\n"]
}
proc BIN2HEX { text }  { binary scan $text H* result; return $result }
COMPARE_BASE64

--- result ----
da52145370e4162f2a2efd8f82fff861a68c040aeee881406e94374a65c1
da5214d59370e4162f2a2efd8f82fff861a68c040aeee881406ede94374a65c1
1

If you need to encode/decode base64 for use in non-vignette systems make sure you use tcllib's base64.

This was tested on Vignette's V6 but I am sure it's true for Storyserver 4.2 and V5 as well.

MySQL indexes behaving badly

noreply@blogger.com (Nir Levy) — Thu, 19 Mar 2009 14:15:08 PDT

If you have a MySQL query that for no apparent reason stops working or no longer behaves as it used to, the first thing to do is EXPLAIN it. If the explain is unreasonable (for example, using different indexes then it used to or using an index but still scanning millions or rows) then you should try to CHECK TABLE.

One thing that I accidently found out was that CHECK TABLE also updates the key statistics. This made many problems magically disappear.

A few things that should be noted:

CHECK TABLE will only update key statistics for MyISAM tables

InnoDB

OPTIMIZE TABLE

If you use OPTIMIZE TABLE to update the key statistics you should be aware that OPTIMIZE TABLE does other (good) things as well, so it usually takes much longer and it locks your table while doing it. Also, for InnoDB it actually re-builds the table using ALTER TABLE.
The CHECK TABLE have several options. Only the MEDIUM (the default) or EXTENDED update the key statistics. (i think).

So, if your indexes stop working, or just mis-behave, try to check them.

Vista fast user switching disappeared

noreply@blogger.com (Nir Levy) — Fri, 20 Feb 2009 11:46:51 PST

My fast user switching option in Vista Home machine suddenly disappeared. I remember it was there, then it was not. I don't know why this happened, but here is how to fix it.

A rough translation:

Run regexit.exe
Go to HKEY_LOCAL_MACHINE > SOFTWARE > Microsoft > Windows > CurrentVersion > Policies > System
Look for a key named HideFastUserSwitching. If it exists, change it's value to 0 (zero). If it does not exist create it as a DWORD and set it to zero.
You might need to log off before the change takes effect.

utf8 and hebrew in tomcat

noreply@blogger.com (Nir Levy) — Sun, 08 Feb 2009 07:24:47 PST

In tomcat 5.0 and above, if your UTF-8 request parameters are received as gibberish you might need to do the following:

In your server.xml add the URIEncoding="UTF-8" and useBodyEncodingForURI="true" to the Connector tag(s):


    <Connector 
     useBodyEncodingForURI="true" 
     URIEncoding="UTF-8" 
     acceptCount="100" 
     enableLookups="false" 
     maxSpareThreads="75" 
     maxThreads="150" 
     minSpareThreads="25" 
     port="8080" 
     redirectPort="8443" />

This should make GET requests work properly.

For some reason the above does not work for POST requests. If you ask the tomcat people they'll mumble something about W3C, RFC, and RTFM. The short way to have this work for POST requests is to write a small filter to set the request encoding properly. We are using something similar to this:


package com.realcommerce.filters;
import javax.servlet.*;
import java.io.*;
public class RequestEncodingFilter implements Filter  {
    public void init(FilterConfig filterConfig) throws ServletException {
         //Do Nothing
    }
    public void destroy() {
        //Do Nothing
    }
    public void doFilter(ServletRequest request,ServletResponse response,
        FilterChain chain) throws IOException, ServletException 
    {
        request.setCharacterEncoding("UTF-8");
        chain.doFilter(request, response);
    }
}

This made POST requests pass Hebrew (or any UTF-8) parameters properly.

slow lstat, slow php, slow drupal

noreply@blogger.com (Nir Levy) — Mon, 02 Feb 2009 04:27:17 PST

If you are short on time go to bottom line.

Ok, the lstat() and stat() system calls system call are not really slow per se. But take a look at an apache strace -r dump from a drupal installation I was testing:


[user@host ~]# strace -o strace.load -r -s 256 -p <PID>
...
0.000081 getcwd("/mnt/var/www/html/drupal", 4096) = 32
0.000048 lstat("/mnt", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
0.000805 lstat("/mnt/var", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
0.001596 lstat("/mnt/var/www", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
0.000104 lstat("/mnt/var/www/html", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
0.000362 lstat("/mnt/var/www/html/drupal", {st_mode=S_IFDIR|0755, st_size=8192, ...}) = 0
0.000804 lstat("/mnt/var/www/html/drupal/sites", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
0.057575 lstat("/mnt/var/www/html/drupal/sites/all", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
0.000297 lstat("/mnt/var/www/html/drupal/sites/all/modules", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
0.014762 lstat("/mnt/var/www/html/drupal/sites/all/modules/Internet", {st_mode=S_IFDIR|0755, st_size=4096,
0.000214 lstat("/mnt/var/www/html/drupal/sites/all/modules/Internet/date", {st_mode=S_IFDIR|0755, st_size=4
0.000112 lstat("/mnt/var/www/html/drupal/sites/all/modules/Internet/date/date.module", {st_mode=S_IFREG|064
0.010816 open("/mnt/var/www/html/drupal/sites/all/modules/Internet/date/date.module", O_RDONLY) = 20
0.000835 fstat(20, {st_mode=S_IFREG|0644, st_size=44118, ...}) = 0
0.000069 lseek(20, 0, SEEK_CUR)    = 0
0.000063 stat("././sites/all/modules/Internet/date/date.module", {st_mode=S_IFREG|0644, st_size=44118, ...}) = 0
0.000159 close(20)                 = 0
...

As you can see, for every file included in the system there are around 10-15 lstat calls. this means that if we have ~200 files included we have ~2500 lstat calls per page. A simple sum on the strace results gave me ~8 seconds per page of lstat time, which was over a third of my test page time. see?


[user@host ~]# cat strace.single | awk '{printf " " $1 "\n " $2}' |sed 1d | sed \$d |awk '{printf $2 "+"}'  | sed s/.$/\\n/ | bc
21.204960
[user@host ~]# cat strace.single | awk '{printf " " $1 "\n " $2}' |sed 1d | sed \$d | fgrep lstat | awk '{printf $2 "+"}'  | sed s/.$/\\n/ | bc
8.361994

if you really want to understand this shell script let me know. It deserves a post by itself

Now we have 3 options: a) make lstat() faster. b) make php stop doing all those lstats(). c) make drupal include less files.

Well, (a) is not an option as our infrastructure team (aka sysadmins) insist there is nothing wrong with the filesystem and nothing can be done to enhance it's performance. Since I gave up being a sysadmin when it was still call 'sysadmin' it would be hard for me to prove them wrong (especially since they might be are right).

As (c) would require us to refactor most of our code in a very very 'not drupal way' and create very large php files, I was hoping to find a better solution.

By a rare chance of good fortune i came to learn about realpath_cache_size. Once I changed the foolish default to a reasonable 1M in php.ini, I lost most of the lstat() calls and over 10 seconds of processing time.


[user@host ~]# cat strace.2.single | awk '{printf " " $1 "\n " $2}' |sed 1d | sed \$d |awk '{printf $2 "+"}'  | sed s/.$/\\n/ | bc
9.752309
[user@host ~]# cat strace.2.single | awk '{printf " " $1 "\n " $2}' |sed 1d | sed \$d | fgrep lstat | awk '{printf $2 "+"}'  | sed s/.$/\\n/ | bc
.040935

Here is what to do in php.ini


; ...php.ini...
; Determines the size of the realpath cache to be used by PHP. This value should
; be increased on systems where PHP opens many files to reflect the quantity of
; the file operations performed.
 realpath_cache_size=1M

; Duration of time, in seconds for which to cache realpath information for a given
; file or directory. For systems with rarely changing files, consider increasing this
; value.
realpath_cache_ttl=300

For normal pages under normal load I was able to get a 10%-25% performance improvment. I think that's quite a bit for a configuration change.

IE6 freeze with MOSS 2007

noreply@blogger.com (Nir Levy) — Wed, 31 Dec 2008 23:12:19 PST

emposha wrote about how to handle a situtation where IE6 chokes on MOSS preview pages.

Hebrew SEO: גוגל, עברית, וה"א הידיעה

noreply@blogger.com (Nir Levy) — Thu, 11 Dec 2008 13:50:31 PST

נראה ש-google לא מתייחס נכון לה"א הידיעה.
אם נחפש לדוגמא הממשלה נקבל בתוצאות את "פורטל השרותים של ממשלת ישראל" אבל גוגל מסמן בבולד רק את המילה הממשלה ולא את המילים ממשלה או ממשלת שהן הטיות חוקיות לחלוטין.
בכיוון ההפוך, אם נחפש ממשלה נקבל תוצאות אחרות לגמרי, כאשר המילים ממשלת וממשלה מודגשות אבל המילה הממשלה אינה מוגדשת.
לפי ההבדלים בתוצאות החיפוש נראה שהבעיה קיימת לא רק בהדגשה אלא גם באינדוקס.
בדקתי גם בחיפוש לפי הילדים מול ילדים ומילים נוספות. הדבר נראה נכון גם לגבי live של מיקרוסופט (בלי לדבר על yahooשבכלל לא טרחו לתמוך במורפולוגיה עברית)
המסקנה,לצרכי אופטימיזציה במנועי החיפוש של גוגל כדאי להגדיר את מילות המפתח של האתר עם ובלי ה"א הידיעה.

drupal memcache items disappearing from the cache

noreply@blogger.com (Nir Levy) — Thu, 20 Nov 2008 14:21:14 PST

The scenario: you are using memcached as your Drupal cache system. You do a cache_set for some item. You do a cache_get for this item. The item is there. You do a cache_get again after a few seconds/minutes, the item is not there.

You might be experiencing this issue of the memcache module: In short it means that the memcache module flushes the entire cache cluster whenever someone does a wildcard cache_clear_all. Since it's very likely that all your bins are in a single cluster you get to a situation that whenever someone-on-the-other-side-of-the-world does something that is probably completely unrelated to you, your item will still get flushed from the cache.

Note that since flush does not activly removes the items from the cache but rather marks them as expired you will not see any memcached stats that will hint to this issue.

I have commented on the issue. Here are some highlights:

...we are currently using gaolei's patch which is fast albeit expensive. We are using it on a large production system that depends heavily on memcache and we did not see any problems yet. (thank you gaolei).

We have tried to implement a lock-add-unlock scenario such as was suggested in several comments but this will be a definite nightmare for high-traffic/high-update sites...

I would suggest letting the user choose which flush mechanism to use: the current one, or the salt one. I see no reason why the module's developers should decide for me. On a site with little updates and little memory i would prefer to have the whole memcache flush; On a site with many updates and tons of memory i'm willing to sacrifice space for the sake good performance. a simple memcache_flush_method variable would do the job just fine.

MYSQL Query Optimization: Avoiding ORs

noreply@blogger.com (Nir Levy) — Mon, 10 Nov 2008 11:57:20 PST

So, you've probably heard that using ORs in your queries is heavy and should be avoided when possible. Here is a live example. We had the following query running on a medium sized table in MYSQL (around 200K rows):


SELECT count(*)  FROM interactions p
   WHERE (
       (p.employer = 69 AND p.flag1 = 1)
       OR
       (p.employee = 69 AND p.flag2 = 0)
       )

Let's think of interactions as a table holding the interaction between employers and employees, and this query should tell the user the total number of interactions where user number 69 is involved.

The query looked pretty innoccent on an idle MYSQL server taking around 200ms, but when the server got busy the query execution time was reaching 5-6 seconds.

Altough we did have indexes on interactions.employer and interactions. employee doing EXPLAIN showed that MYSQL was not using them.

After some digging and fiddling and trying we ended up with:


SELECT count(*) as count FROM (
   SELECT 1  FROM interactions p
       WHERE (p.employer = 69 AND p.flag1 = 1)
   UNION ALL
   SELECT 1  FROM privatemsg p
       WHERE (p.employee = 69 AND p.flag2 = 0)
) t2

This query took on the busy server less then 250 milliseconds and less then 80ms on the idle server. Doing EXPLAIN showed that MYSQL was doing two queries, and using the correct index on each one.

After some more digging we noticed that if we EXPLAIN the original query on the idle server the optimizer occasionally converts it to a UNION, but for some reason this did not always happen and in any case took about twice the time then the UNION query.

To sum up the results here is a simple table

Query type	Server activity	Query Time
Using OR	Idle	200ms
Using OR	Busy	5500ms
Using UNION	Idle	78ms
Using UNION	Busy	189ms

Conclusion: always consider alternatives to OR, but make sure you check them well against real-time examples.

A Further Note: When timing queries in MYSQL alway use the optimizer hint /*! SQL_NO_CACHE */ to make sure you are not getting results from the query cache (if you have one set up).

Internet Explorer does not support http vary header

noreply@blogger.com (Nir Levy) — Wed, 10 Sep 2008 11:48:54 PDT

It was always assumed, but I just saw this quote from Microsoft:

Internet Explorer does not fully implement the VARY header per Requests for Comments (RFC) 2616. The Internet Explorer implementation of VARY is that it does not cache any data except for Vary-Useragent

Internet explorer cannot open the internet site: operation aborted

noreply@blogger.com (Nir Levy) — Wed, 10 Sep 2008 12:24:40 PDT

During an integration with a third party provider we were unfortunate enough to get the error

Internet explorer cannot open the internet site http://localhost: operation aborted

Yes, we have read http://support.microsoft.com/kb/927917 and did move the 3rd-party's script to be just before the </BODY> tag but to no avail.

After a long and hard effort by this provider they pointed out that while the view-source indeed looked like the script is just before </BODY> when looking at the DOM itself (they used Dominspector, I used the IE Developer Toolbar) it showed that the script was inside a <DIV> element.

As it turns out there was some code on the page that wrote an unclosed <DIV>. This made IE "fill in the blanks" and guess (wrongly) where it should place the closing </DIV> Fixing this javascript made the error go away on most pages. But not all.

I found out that the broken pages used some JQuery plugins (tooltips, jqmodal, etc.) to produce fancy decorations and effects. These scripts attached to $document.ready and did $('body').appendTo(...). This effectively added a couple of items to the DOM between the 3rd-party script and the </BODY>. I am still not quite sure why this should cause IE to choke but since we have a tight schedule we simply changed those scripts to add their stuff to some other elements. This indeed solved out problem.

They funny (or maybe sad) thing was that on Firefox we did not get any error but the page simply disappeared. Chrome worked just fine.

Misplaced elements with position:relative

noreply@blogger.com (Nir Levy) — Tue, 26 Aug 2008 23:34:37 PDT

We had a situation where we had three <DIV> elements in a column and the middle element had content dynamically loaded into it after the page has completed loading. When the middle element finished loading, some of the content of the last element seemed to forget its parenthood and floated nicely over the second element.

Naturally this only happened in Internet Explorer (what else).

After a lot of digging we noticed that the misplaced content of <DIV>#3 had a position:relative style (indicated as #4 in the image above). It is "well known" that IE does not handle relative positioning properly when the layout of the page change.

Luckily I chanced upon an article by Holly Bergevin et al. called On having layout which was enlightening. Understanding that IE will only respect the element's state if it have the hasLayout property quickly solved the problem.

In our specific case I used display:inline-block to force <DIV>#3 to have hasLayout set to true, which made IE respect the element's style and re-render the element's content (i.e. <DIV>#4) after re-positioning it. My only concern now is what effect this might have on rendering performance, but it will have to do for now.

Tip: you can use IE Developer Toolbar to check if an element hasLayout. It will show the hasLayout property as set to -1 if hasLayout is true.

Further credit is due to Ingo Chao who wrote relatively positioned parent and floated child – disappearance.

Absolute vs. Relative URLs and SEO

noreply@blogger.com (Nir Levy) — Tue, 26 Aug 2008 11:46:50 PDT

I've seen several places where people suggest that one should use absolute URLs (http://domain.com/page) to do internal linking instead of relative URLs (/page). Those people relate to a google article where it is quoted that

We also suggest you link to other pages of your site using absolute, rather than relative, links with the version of the domain you want to be indexed under. For instance, from your home page, rather than link to products.html, link to http://www.example.com/products.html . And whenever possible, make sure that other sites are linking to you using the version of the domain name that you prefer

This seems to me a classic example of taking things out of context. If you read the entire article you can see that Ms. Fox is talking about a specific case where your site can be accessed by more then one domain name (e.g. http://www.domain.com and http://domain.com).

There is nothing google ever wrote that i could find that say that absolute URLs are better if your site is only accessed by one domain name.

There is one exception i can think of: if your domain is "coolstuff.com" for example and you do use absolute URLs, then the word "coolstuff" will appear in your pages alot. This might be something that may boost your ranking with regards the the word "coolstuff". But this is just a guess.

More reading: Google Canonicalization Problems? - Crawling, indexing, and ranking | Google Groups

Please note: this is my personal opinion. I've had at least two SEO experts that claim that absolute URLs will give better performance on google ranking. Since I'm strong-headed I will hold to my opinion until I'll be proven (with numbers) otherwise. Feel free to comment with supporting or conflicting opinions and data.

Save binary file in Tcl under V6

noreply@blogger.com (Nir Levy) — Sun, 06 Jul 2008 02:36:43 PDT

This issue pops up every couple of years. It's nothing new, but still worth documenting.

The Challenge: To save a posted file to a file system, not using SUBMIT_STATIC_FILE.

The Problem: Vignette V6 pre-process the form submitted data so that any <input type="file"> are encoded as an hex string. If you ERROR_TRACE the variable for the file, you will see a string that looks like 0x1D00ABADD1D9....

The Solution: binary format to the rescue. and with a bit of trimming, you get it all in a few lines:


proc save_files { } {
#FileData,FileExtention,Field1 are posted from the <form>

set filename [generate_filename]
set bin_filename "${filename}.[SHOW FileExtention]"
set xml_filename "${filename}.xml"

### This will save a binary file.
set file [open $bin_filename "w"]
fconfigure $file -translation binary
puts -nonewline $file [binary format H* [string range [SHOW FileData] 2 end-1]]
close $file

### This will save a utf-8 xml (text) file
set file [open $xml_filename "w"]
fconfigure $file -encoding "utf-8"
puts $file "<?xml version='1.0' encoding='utf-8' ?>"
puts $file "<formdata>"
puts $file "<Field1>[HTML_ESCAPE [SHOW Field1]]</Field1>"
#...more fields...
$puts $file "<Attachment>$bin_filename</Attachment>"
puts $file "</formdata>"
close $file
}

* Remember to use H* and not h*.
* The example above also saves a text file in utf-8 with extra data from the form
* Make sure your form is enctype=multipart/form-data
* This will not work in Storyserver 4.2.

Make a Footer Stick to the Bottom of the Page

noreply@blogger.com (Nir Levy) — Thu, 05 Jun 2008 12:32:29 PDT

Ryan Fait wrote a great piece of code that makes a footer stick to the bottom of the page with css only. no javascript, no hassle. genius, absolutly genius.

Apache Rewrites and SetEnv

noreply@blogger.com (Nir Levy) — Mon, 02 Jun 2008 10:21:49 PDT

It seems that variables set with Apache's SetEnv (and hence SetEnvIf and BrowserMatch) are being ignored by RewriteRule and RewriteCond. The rumor has it to be a design issue. The only way i could find around this problem is to use the [E= rewrite rule directive.

#Use RewriteRule & RewriteCond as a replacment for SetEnv and BrowserMatch
#SetEnv browser="-ie"
RewriteRule ^(.*)$ - [E=browser:-ie]
#BrowserMatch "IE7" browser="-ie7"
RewriteCond %{HTTP_USER_AGENT} "MSIE 7"
RewriteRule ^(.*)$ - [E=browser:-ie7]
#BrowserMatch "Firefox" browser="-moz"
RewriteCond %{HTTP_USER_AGENT} "Firefox"
RewriteRule ^(.*)$ - [E=browser:-moz]

RewriteCond %{DOCUMENT_ROOT}/cache/index%{ENV:browser}.html -f
RewriteRule ^(.*)$ cache/index%{ENV:browser}.html [L]

PS. If you need to debug rewrite rules you can add the following lines to your httpd.conf:


RewriteLog logs/rewrite.log
RewriteLogLevel 5

Annoying FireBug Bug

noreply@blogger.com (Nir Levy) — Mon, 08 Dec 2008 23:53:40 PST

I came across an annoying bug in firebug. If there is a global variable with the same name as a local variable then firebug does not correctly show the value of the local variable in the tooltips and the watch window.

The most annoying thing is that it has been reported several months ago, but still no fix.

UPDATE: Confirmed Fixed in 1.2 on FF3.