Het Bijstere Spoor

SabreDAV reaches 1.0

2009-11-02T12:45:02-08:00

After lots of testing, coding and integrating SabreDAV in a few systems, SabreDAV has reached 1.0.

What's that?

For those of you randomly stumbling on this blog, SabreDAV is a tool I've been working on to allow easy integration of PHP and WebDAV. While looking at WebDAV implementions out there for PHP, there weren't really any standards-compliant and flexible solutions out there, so I decided to write my own.

Since then it has gained some popularity, and the beta version has been tested and implemented in a number of places. It's open source (under the BSD license), so anybody can grab it and use it.

If you don't really know what WebDAV is, it is basically a filesystem protocol that works on top of HTTP. By implementing WebDAV (or SabreDAV specifically) you are able to expose your web application as if it was a filesystem to the users. WebDAV is supported by any major operating system.

If you're interested in SabreDAV, check out the example, or tutorial.

Changes

The last round of changes since 0.12.1 are mostly cleanups, but the most important are the following:

SimpleDirectory class
Authentication plugin is no longer included. The plugin was not yet production ready, and in order to allow myself to make API changes, I needed to remove it for 1.0. This does not mean you can no longer use the recommended way to implement authentication.
The unknownProperties event was removed, replaced by the afterGetPropertiesEvent (docs).
Addition of more events for plugin writers.
Lots of cleaning up, unittests.

Full changelog and download.

Future plans

Although the 1.0 version should be a bit of a milestone, the release had actually been sitting ready for a while. So while 1.0 is cool and all, I've already spent a great deal of time implementing RFC 3744 a.k.a. ACL extensions for WebDAV. Support for ACL will be added as a separate module in the future.

While these changes are going on, no backwards compatibility breaking changes should be made to the API's. For more details on the BC and versioning policy, check here.

Thanks!

It's been a lot of hard work to get here, but seeing some uptake is very rewarding. I would probably have given up on this if it weren't for the people asking questions, making suggestions and filing reports. Thanks for that, it makes it worth it!

Comments are back!

2009-10-13T07:46:47-07:00

Thanks to a suggestion from Mohammad Lahlouh, all deleted comments are restored. Thanks!

I was able to do this with the mysql binlog, which is a binary log containing all statements executed on the server. If you face a similar problem, here are the steps to restore the DB:

See if you're actually storing binlogs. For me this happened to be the case, although I never actually payed attention (My friend Jens Meijer is so kind to provide hosting for me). In my case they were stored in /var/log/mysql.
Grab your latest backup. In my case it was from October 2008.
Make sure you also create a new backup, just in case this process goes wrong.
Restore your old back.
Find the binlog that corresponds to the date of your backup. I did a simple copy of the ~300 binlogs to a different directory.
Run the binlog through 'mysqlbinlog' and pipe into a new file. I had to append the --database argument, because I wasn't interested in all the other databases. The full command for me was mysqlbinlog mysql-bin.* --database evert2 > commentsbinlog.sql
Open the new file, and make sure the first queries are correct. Likely there are some updates/inserts/deletes on top of the file that were already part of your last backup.
Go all the way to the bottom of the file, and remove the offending queries. In my case I had to remove the query that deleted all my comments.
Apply the sql file to your database: mysql -u root -p evert2 < < commentsbinlog.sql

It's wise to assure nobody can access the DB while you are doing this. I hope this helps anyone else with this problem.

Deleted all comments by accident

2009-10-12T15:49:01-07:00

I just accidentally deleted all comments made on this blog.

This blog is homegrown and hand-made. It has a very simple spamfilter that catches most spam comments. The few that make it through though, I've been deleting myself using the mysql console.

I tend to write simple queries, ending in: .. AND id = 1234, but today I forgot the id =. I had a backup from October 2008, so some old comments are back now, but I'm still pretty sad.

Lesson from today: make backups! I'm pretty good with things I do for clients, but I tend to not uphold the same standards for my own projects.

Upgrading to Snow Leopard adventures

2009-09-15T10:41:51-07:00

I just got my copy of snow leopard in the mail, so I decided to upgrade my laptop. After doing a full backup, I didn't turn out to be as easy as I thought.

The installation process quickly showed a little yellow triangle, indicating I didn't have any startup disks. I quickly remembered I had a dual boot to Ubuntu, and used Refit, which is basically an alternative boot menu, to get into Linux and the likes.

So I looked up the removal instructions, but unfortunately this didn't change the situation. I then read that OS/X will restore the standard EFI loader (if that's the correct terminology), by opening the Disk Utility, and making the partition slightly smaller and bigger again. Trying this I got an odd error, namely "MediaKit reports no such partition". No dice.

So I ended up booting an Ubuntu Live CD, removing the linux ext3 and swap partitions there. After this was done, I was able to resize the main OS/X partition, which eventually fixed the problem.

If you ran into similar problems, I'm hoping there's enough keywords in here for you to find this solution :).

After all that installing Snow Leopard is a bit of an anti-climax. As advertised, not a lot changed, and I've yet to see a noticeable speed difference. The OS is using less disk-space, but 29 euro is still a high price for a glorified 'rm' statement.

I do notice small notification improvements here and there, and Address Book.app now supports CardDAV, which is nice.

OS/X internet sharing defaults to 192.168.2.* range.

2009-09-14T11:49:25-07:00

I was stumped, I was trying to hook up a second computer on the internet through ethernet. Normally it's easy enough to just turn on internet sharing. It turns out OS/X always uses 192.168.2.*, which happened to be in the same subnet as the WiFi network.

OS/X provides no way to change this as far as I could tell. Thankfully I was able to change the router's settings. If you ever get into this situation and you're not able to make this change, here's a simple guide to setup internet sharing using the console.

CalDAV - it's hard!

2009-09-13T14:35:48-07:00

Over the past little while I've been working on a CalDAV library, and it's tough!

It appears that the spec is very much implemented with the client, rather than the server in mind. The spec writers have left no stone unturned, and it's technically possible to query the server for any odd dataset you might need. Aside from that the authors made absolutely sure there were no duplicated efforts, and built heavily upon the existing WebDAV, WebDAV ACL extensions and iCalendar.

While I generally agree it's a wise thing to not reinvent the wheel, I'm a bit of a pragmatist and feel the complexity of all these layers make it hard to understand, and must have played/will play a factor in the arguably slow adoption rate.

To keep it short, I feel the authors have over-engineered/abstracted the specification. Instead of making simple API's that work for 90% of the usecases, they have gone a much more generic route. This makes it very hard to predict how CalDAV clients will be using the server. Also, it makes it quite difficult to optimize and fit in a simple relational database or existing calendaring software. CalDAV should IMHO just have been a simple CRUD system. Advanced features and reports could have been made optional, and should have been specialized for common usecases.

Incomplete view of Web/CalDAV dependencies

This was a graph I used in an earlier post. There's actually a few more specs that are relevant to the implementor.

GroupDAV

GroupDAV deserves a mention here, as it attempts to address most of the issues I have. What it lacks is adoption from ICal, Sunbird and MS Outlook. The only issue I have is the frowned upon use of the PUT method (spec), where POST really should have been used.

Some examples

These will be highly uninteresting unless you have an intimate knowledge of the specs.

New calendars/events/todo's are created using PUT. This means the client now determines the unique id given to any of these objects. Most people will want to integrate CalDAV with systems which already have a different primary key, these people will be required maintaining 2 different id's and ensure there are never any duplicates. The id generated by the client can be any url encoded string. Note that this wouldn't have to be a problem if some POST statement would have been used.

CalDAV requires support for the expand-properties report from rfc 3253. In a nutshell the expand-properties report will look for WebDAV properties containing <DAV:href>url<DAV:/href>, follow these locally and add specified properties from these resources to the DOM. The benefit of this REPORT query is that less HTTP requests/roundtrips have to be made to make this work. The usecase for CalDAV specifically might be a lot simpler though, as it could mean something like 'get me the information of the user owning this calendar'. If it was instead simply required to add functionality that meets exactly that (and a few other usecases), it would be reasonably easy for me to write an optimized SQL query for those cases. Now I should support these lookup queries everywhere, because there's no good way to restrict how clients might use it.

CalDAV also requires fully implementing the ACL spec. The main usecase for this is calendar sharing. While the ACL specification will give you infinite complexity in how to setup your access control, it's quite a bit of work to even send back all the appropriate information and error messaging if you're not actually interested in this feature. Making ACL an optional extension to CalDAV would have been easier for implementers IMHO.

SabreDAV 0.12

2009-09-10T09:38:05-07:00

I just released a new version of SabreDAV, 0.12. I've skipped on posting for the last few versions, because I didn't want to get too spammy on this blog.

These were mostly bugfixes, and a few added features. SabreDAV is also a PEAR-package again, so installing is as simple as 'pear install SabreDAV-0.12.0.tgz'.

Download it here.

Full list of changes

Added: Experimental PDO backend for Locks Manager.
Fixed: Sending Content-Length: 0 for every empty response. This improves NGinx compatibility.
Fixed: Last modification time is reported in UTC timezone. This improves Finder compatibility.
Fixed: Issue 13.
Added: now a PEAR-compatible package again, thanks to Michael Gauthier.
Added: Plugin to automatically map GET requests to non-files to PROPFIND (Sabre_DAV_Browser_MapGetToPropFind). This should allow easier debugging of complicated WebDAV setups.
Added: Ability to choose to use auth-int, auth or both for HTTP Digest authentication. (Issue 11)
Fixed: TemporaryFileFilter plugin now intercepts HTTP LOCK requests to non-existant files. (Issue 12)
Updated: Browser plugin now shows multiple {DAV:}resourcetype values if available.
Added: generatePropfindResponse now takes a baseUri argument.
Added: ResourceType property can now contain multiple resourcetypes.
Added: Sabre_DAV_Property_Href class. For future use.
Changed: Made more methods in Sabre_DAV_Server public.
Added: Central list of defined xml namespace prefixes. This can reduce Bandwidth and improve legibility for xml bodies with user-defined namespaces.
Changed: moved default copy and move logic from ObjectTree to Tree class.

Converting ICalendar to XML

2009-09-02T08:25:45-07:00

I've started working on a CalDAV implementation, which also requires analysis of ICalendar (rfc 2554) objects.

ICalendar objects have properties, components (such as VEVENT, VTODO) and attributes. This is awfully familiar to XML. So instead of trying to come up with a complicated parser and object structure, I decided to just convert it to XML and use PHP's simplexml.

This is my current script:


<?php



function iCalendarToXML($icalendarData) {



    // Detecting line endings

    if (strpos($icalendarData,"\r\n")) $lb = "\r\n";

    elseif (strpos($icalendarData,"\n")) $lb = "\n";

    else $lb = "\r\n";



    // Splitting up items per line

    $lines = explode($lb,$icalendarData);



    // Properties can be folded over 2 lines. In this case the second

    // line will be preceeded by a space or tab.

    $lines2 = array();

    foreach($lines as $line) {



        if ($line[0]==" " || $line[0]=="\t") {

            $lines2[count($lines2)-1].=substr($line,1);

            continue;

        }



        $lines2[]=$line;



    }



    $xml = '<?xml version="1.0"?>' . "\n";



    $spaces = 0;

    foreach($lines2 as $line) {



        $matches = array();

        // This matches PROPERTYNAME;ATTRIBUTES:VALUE

        if (preg_match('/^([^:^;]*)(?:;([^:]*))?:(.*)$/',$line,$matches)) {

            $propertyName = strtoupper($matches[1]);

            $attributes = $matches[2];

            $value = $matches[3];



            // If the line was in the format BEGIN:COMPONENT or END:COMPONENT, we need to special case it.

            if ($propertyName == 'BEGIN') {

                $xml.=str_repeat(" ",$spaces);

                $xml.='<' . strtoupper($value) . ">\n";

                $spaces+=2;

                continue;

            } elseif ($propertyName == 'END') {

                $spaces-=2;

                $xml.=str_repeat(" ",$spaces);

                $xml.='</' . strtoupper($value) . ">\n";

                continue;

            }



            $xml.=str_repeat(" ",$spaces);

            $xml.='<' . $propertyName;

            if ($attributes) {

                // There can be multiple attributes

                $attributes = explode(';',$attributes);

                foreach($attributes as $att) {

                    

                    list($attName,$attValue) = explode('=',$att,2);

                    $xml.=' ' . $attName . '="' . htmlspecialchars($attValue) . '"';



                }

            }



            $xml.='>'. htmlspecialchars($value) . '</' . $propertyName . ">\n";

          

        }



    }



    return $xml;



}



?>

This will convert:


BEGIN:VCALENDAR

VERSION:2.0

PRODID:-//Example Corp.//CalDAV Client//EN

BEGIN:VTIMEZONE

LAST-MODIFIED:20040110T032845Z

TZID:US/Eastern

BEGIN:DAYLIGHT

DTSTART:20000404T020000

RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4

TZNAME:EDT

TZOFFSETFROM:-0500

TZOFFSETTO:-0400

END:DAYLIGHT

BEGIN:STANDARD

DTSTART:20001026T020000

RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10

TZNAME:EST

TZOFFSETFROM:-0400

TZOFFSETTO:-0500

END:STANDARD

END:VTIMEZONE

BEGIN:VEVENT

DESCRIPTION:Hello Im evert

 Next line also

  Blabla

ATTENDEE;PARTSTAT=ACCEPTED;ROLE=CHAIR:mailto:cyrus@example.com

ATTENDEE;PARTSTAT=NEEDS-ACTION:mailto:lisa@example.com

DTSTAMP:20060206T001220Z

DTSTART;TZID=US/Eastern:20060104T100000

DURATION:PT1H

LAST-MODIFIED:20060206T001330Z

ORGANIZER:mailto:cyrus@example.com

SEQUENCE:1

STATUS:TENTATIVE

SUMMARY:Event #3

UID:DC6C50A017428C5216A2F1CD@example.com

X-ABC-GUID:E1CX5Dr-0007ym-Hz@example.com

END:VEVENT

END:VCALENDAR

To:


<?xml version="1.0"?>

<VCALENDAR>

  <VERSION>2.0</VERSION>

  <PRODID>-//Example Corp.//CalDAV Client//EN</PRODID>

  <VTIMEZONE>

    <LAST-MODIFIED>20040110T032845Z</LAST-MODIFIED>

    <TZID>US/Eastern</TZID>

    <DAYLIGHT>

      <DTSTART>20000404T020000</DTSTART>

      <RRULE>FREQ=YEARLY;BYDAY=1SU;BYMONTH=4</RRULE>

      <TZNAME>EDT</TZNAME>

      <TZOFFSETFROM>-0500</TZOFFSETFROM>

      <TZOFFSETTO>-0400</TZOFFSETTO>

    </DAYLIGHT>

    <STANDARD>

      <DTSTART>20001026T020000</DTSTART>

      <RRULE>FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10</RRULE>

      <TZNAME>EST</TZNAME>

      <TZOFFSETFROM>-0400</TZOFFSETFROM>

      <TZOFFSETTO>-0500</TZOFFSETTO>

    </STANDARD>

  </VTIMEZONE>

  <VEVENT>

    <DESCRIPTION>Hello Im evertNext line also Blabla</DESCRIPTION>

    <ATTENDEE PARTSTAT="ACCEPTED" ROLE="CHAIR">mailto:cyrus@example.com</ATTENDEE>

    <ATTENDEE PARTSTAT="NEEDS-ACTION">mailto:lisa@example.com</ATTENDEE>

    <DTSTAMP>20060206T001220Z</DTSTAMP>

    <DTSTART TZID="US/Eastern">20060104T100000</DTSTART>

    <DURATION>PT1H</DURATION>

    <LAST-MODIFIED>20060206T001330Z</LAST-MODIFIED>

    <ORGANIZER>mailto:cyrus@example.com</ORGANIZER>

    <SEQUENCE>1</SEQUENCE>

    <STATUS>TENTATIVE</STATUS>

    <SUMMARY>Event #3</SUMMARY>

    <UID>DC6C50A017428C5216A2F1CD@example.com</UID>

    <X-ABC-GUID>E1CX5Dr-0007ym-Hz@example.com</X-ABC-GUID>

  </VEVENT>

</VCALENDAR>

I hope this is useful to anyone else.

SabreDAV 0.9

2009-07-20T22:02:22-07:00

Just released the 0.9 version of SabreDAV.

This release mainly contains bugfixes and some API changes that will facilitate future extensions, such as ACL and CalDAV.

Although these changes might not be relevant to most people, upgrading is recommended in order to keep a solid upgrade path. Fortunately for most people the API breaks are minimal.

Flash ExternalInterface - Debug checklist

2009-07-10T12:24:00-07:00

Flash' ExternalInterface api allows you to communicate between Javascript and Actionscript. A very useful tool, and we've used it quite a few times. Often times it just doesn't work though, and it can be difficult to get to the root of the issue. This is a list of things you should be looking out for if ExternalInterface does not work for you:

Give your <object> tag an id

In order to call your flash application, your <object> tag needs to have an id attribute. Make sure this attribute is unique.

The id cannot collide with items in the document object

Internet Explorer will add any element with an id to the document object. So if you were to, say, give your object the name 'window', 'getElementById' or anything to that respect EI will not work.

The object cannot be embedded in a <form> tag

EI will not work in IE if the object sits in a <form>-tag. For some reasons IE will treat any element within a form slightly different.

Don't use 'play' or 'stop' for method names

Another IE issue. play and stop are somehow reserved for IE. This only applies to actionscript functions you're calling from javascript.

Check your domain names

If your swf is loaded from a different domain as your html document, you need to make sure you add this domain through flash.system.Security.allowDomain. In our case it was fine to just add Security.allowDomain('*').

If all else fails, know how to debug.

Much like with XMLHTTPRequest, errors will not show up in Firebug by default, but you can trace everything if you check everything in the Console-tab-submenu. This will also allow you to see some of the automatically generated javascript Flash produces.

Most of the above issues are in IE, but thankfully Microsoft's script debugger will actually catch these (rather cryptic) errors, which might help you find your problem.

We've run into every single one of these issues, if you have any others like this, please comment :). I figured it would be helpful to have one single source of documentation.