Adds and Dishes

Hm. I forgot you could tempt me with things I want.

2011-09-07T20:21:00.004+01:00

Since the Diablo III client has leaked, it's time to publicly release what used to be a small hack to automatically download WoW patches.

https://github.com/adys/patchdl

patchdl is a simple script that emulates the Blizzard Launcher's interaction with the patch servers and outputs a list of files to download.

Right now it outputs wget command lines because I don't really know what else to make it do - I don't really want it to download files automatically.
By default it checks for WoW MPQs, but it can download Diablo III MPQs as well. It "could" download Starcraft II MPQs but the whole thing is very different there.

Usage:

adys@azura ~/src/git/patchdl % ./main.py
WoW::enUS
No new files

Of course, it outputs "No new files" because I already have all the MPQs downloaded.
You can pass --client and --server parameters in order to change the client or server version - for WoW (and likely D3) that basically means --client=2 if you want to tell the server that your client is PTR, and --server=2 if you want the server to tell you about PTR files. (--client=1 --server=2 sort of means "My client is Live, I want the PTR files for it").

But I'm sure all the vultures reading this post just want to know how to download Diablo 3 MPQs ;) So it's as simple as this:

adys@azura ~/src/git/patchdl % ./main.py D3
D3::enUS
No new files

No, I won't give you the links to the MPQs. Install Python yourself. :)

Moving to Leicester in 12 days. Eeeek.

You can lie like that? That is so cool...

2011-07-01T00:58:00.002+01:00

Google+ is really nice. I won't go in details here, but overall, the integration is great and the feel of the whole website is solid. Finally a good excuse to get rid of facebook/twitter accs :)
I'm hoping for a better integration with Buzz over time, and account merging. And the black navbar is still terrible.

On to whines of the day.
First of all, MIME type pollution is stupid and annoying.


  <mime-type type="virtual/bluedevil-input"></mime-type>

  <mime-type type="virtual/bluedevil-audio"></mime-type>

  <mime-type type="virtual/bluedevil-sendfile"></mime-type>

Really? Come on.

Second of all, the kde-git docs are *still* terrible. It's improved a bit, but searching for them still leads to old, outdated gitorious repositories and all. Pretty sad.

Finally, Blogger still disgusts me in every possible way.
The interface is still ugly and unresponsive, signing in with a Google account is still a horrible process, it's still impossible to paste code properly and it still forces the user onto a specific language through geoip.

Bah.

I'm... I'm not... I'm.. not.. I'm not from Mars...

2011-05-20T19:08:00.000+01:00

I got really fed up with KMix.

I know it's just a sound applet. I know there are hundreds of different configurations to support. I know all that.
But the most common configuration should always be supported in the most simple way.

With that said, here comes Rattle. Right now it's a simple PulseAudio volume applet in PySide. I could make it in C and link it against no unnecessary libs, just make sure it's as efficient as possible, but I really don't care anymore. I grabbed libpulse from gnome-volume-applet and.. cleaned it up.
I'll probably continue clean it up and have something decent out of it, but right now this is all it is: A docked volume icon in PySide. It requires PySide, Python 2.7, a system tray, systray wheel events, xdg audio-volume- icons and Pulseaudio -- apart from PySide, this would be available by default in most distros. It only supports one volume channel with no bells and whistles. Main feature being that it doesn't consume 300 megabytes of memory like KMix does.

Anyway, enough about our bells.
Mentalist and Castle Season 3 finales are both up, they're both spectacular. Burn Notice and Futurama both return in June, Torchwood and White Collar both return in July. I'll be in Romania this summer; can't wait... that's going to be a fun backlog to get back to.

You know what else is coming this summer? Duke Nukem Forever.

Also interesting is the recent AOL move to XMPP, perhaps one day with a full protocol move too but for now I'm enjoying this quite a bit.

I was planning to go to BlizzCon 2011 but the events are really uninteresting this year, and a lot of people aren't going. So scratch it; I'll save for 2012 and more amazingness.

The dirt looks too much like cake.

2011-04-29T15:55:00.000+01:00

Natty being finally released, I decided to take a look at why all my GTK apps were still ugly as sin on KDE.

As it turns out, Qt-GTK was writing to ~/.gtkrc-2.0-kde4. export | grep gtk revealed the following:

declare -x GTK2_RC_FILES="/etc/gtk-2.0/gtkrc:/home/adys/.gtkrc-2.0::/home/adys/.kde/share/config/gtkrc-2.0"

declare -x GTK_RC_FILES="/etc/gtk/gtkrc:/home/adys/.gtkrc::/home/adys/.kde/share/config/gtkrc"

Moving ~/.gtkrc-2.0-kde4 to ~/.kde/share/config/gtkrc-2.0 solved the problem.

Svengali next month!

I'm in an ethical pickle.

2011-04-09T18:11:00.001+01:00

So by now it's pretty obvious to everyone that Minecraft is an interesting business target. I'm somewhat saddened the break I'm getting from working on WoW is on such a small, non-mmo game. In Java.

Anyone who would analyze it for more than 5 minutes would figure out why it's so successful. Ish. But it's also really addictive, and got me working on interesting mechanics I didn't have the occasion to dabble with before. =)

I wanted to take the opportunity to advertise the last couple of libraries I worked on a few weeks back.

https://github.com/Adys/python-mime
python-mime (mime) is a "cross-platform" API to handle MIME types. On *nix, it will try to follow the XDG spec. On Windows, it will read the registry and pray things will work out.
For now, it's read only. I find myself without enough time to implement writing, although I'll eventually need it, at least on Linux.

https://github.com/Adys/python-irc
python-irc (irc) is an IRC client library in Qt (PySide). It doesn't implement the entire protocol, but the commonly-used parts are already here. SSL is supported. A 30-LOC bot is available as example.
It offers a simple signal/slot-based API, with the main IRCClient class that will spit out IRCChannel and IRCUser instances in its signals.
Note that IRCUsers are never stored as instances, purely as string, for performance reasons.

How do I use the thing?

2011-01-18T23:22:00.000Z

One of the many challenges a designer has to overcome is the "thing" problem.
The more familiar you are with the inner workings of an object (or program), the higher the odds you'll be making assumptions about your users and their knowledge of the object. What may very well be a masterpiece then becomes "the thing".
In an engine where every cog is important, a designer is often tempted to drift into making some cogs more important, just to end up in a lot of unused work because users have no idea what they are dealing with. "What's the thing in the corner?"

Magicians are faced with a very similar issue when designing their own performances. Spend a few hours or days into the "getting from A to B is impressive" mindset, become blinded with the assumption the audience is still following. Usually ends in a disastrous applause of "Why am I impressed again?" eyebrows.
There are not enough reminders of how important starting with the effect is. Yet it's so simple. After years, many tricks begin with the "I know this - how do I use it?" mindset, instead of a more elegant "I want this to happen - let's figure out how".

You would think it as common sense, yet in design too, many start asking "what can I do with my knowledge?" instead of "how can I open a coconut with a swiss army knife?".

If it being obvious would make it child play, we would see less "I can make a Facebook clone" and more "I can make Google Cubed". And then maybe you, too, wouldn't have grown tired of seeing the reflection of your amusement in an overly polished Ace of Spades.

Sniffings are okay.

2010-09-30T15:22:00.000+01:00

Starting to prepare for Blizzcon. I'll be at the UI dinner and stay in Boston for three weeks after that.

Okay, first of all: The Mentalist S3 started. Hurray!
Second of all, I'm still working on multiple miniprojects at the same time. A desktop keyring-enabled authenticator, python bindings for StormLib (with the help of Teomyr), a patchqueue and a huge rework, starting to learn Werkzeug and getting away from Django.

White Collar is pretty good.

Iceland corrected!

2010-06-11T18:08:00.004+01:00

That's another year down, give or take a few days.

A lot of stalling, yet a lot of activity. These past two months especially have been a bit crazy (dare I say cataclysmic?).

Now what? A couple weeks and a cheap flight away from greener grass. I got my Blizzcon tickets (hurray! The queue has never been so bad...). Bought a new laptop and started packing, I've been staring at the open bag for a few days now. I even got my old PSU back (double-yay! Money not wasted...) and a new external HD.

Oh how I love github. My most recent addition is the open sourcing of python-bna, a low level Python module to interact with Battle.net mobile authenticator algorithms and servers. How it works is very simple (and less secure than the Blizzard version):
* Initialize a new token by contacting the blizzard servers, with authenticator = bna.getNewSerial()
* The returned value is a dict containing "serial" and "secret" values, which you'll need to store somewhere safe.
* To get an authenticator code, just call bna.getToken(secret)

You can find the repo here: http://github.com/Adys/python-bna - Maybe I'll document it when I'm bored. Also on the TODOs are time syncs.

OK, the real reason I wrote this post was about slugging. Dinnerbone kindly pointed out to me the security flaw in having URL parameters separated the same way as optional URL slugs. Take an object called "tooltip":
* /object/12345/tooltip/
* /object/12345/tooltip/tooltip/

An optional slug will of course confuse the engine and there is no fix for that without hitting the database (BAD!).
Now, let's imagine you really do want URI parameters to be stored after the slug (you don't want to store them in the GET). What's the only solution?
Changing the slug. In order not to break the style guidelines, you cannot use /object/12345-tooltip/tooltip/. That doesn't mean the idea is not good. My personal favourite is checking against a blacklist (eek, DRY warning) and, for exact matches only, prepending the id to the slug. Of course, this can lead to confusing URLs such as /object/12345/12345-tooltip/tooltip/... but given a small blacklist (GET helps) and no user input, that would happen extremely rarely (if at all).

That's all, folks. See you on the other side.

It's fake, mon.

2010-03-02T23:22:00.002Z

After a bumpy road, lots of coding, recoding, rewriting and polishing, the originally wdbc-intended structures turned into an abstract structure engine.

I'm also very happy with its current state. This is why I've decided to release it under the permissive BSD license.

What are its features?

* Extremely lightweight. 3 files, less than 10 KiB including one example file.

* Extremely extensible. You can easily and separately patch methods into the structure Skeletons, add new fields and so on.

* Describe rather than parse. One of the core ideas was to parse a file separately from their description. The description stays entirely abstract.

How add structures?

Create a structure file and from structures import *. Define a new structure like this:

>>> class MyStructure(Structure):

>>> ... fields = Skeleton(

>>> ... ... IntegerField("field_one"),

>>> ... ... IntegerField("field_two"),

>>> ... ... StringField("name"),

>>> ...)

How to use them?

Initialize your structure with MyStructure(build=0)

Build argument is optional. It's only used if you use multibuild structures.

To use it, iterate over the structure (just like a list) while reading/writing your file. Example:

>>> for field in MyStructure():

>>> ... value = struct.unpack(field.char, f.read(field.size))

>>> ... print field.to_python(value, None)

How to add new field types?

Just subclass Field, or any of its relevant current subclasses (IntegerField, StringField, ...).

For custom behaviour, you can implement from_python and to_python methods (there are two examples in the provided fields.py).

I will try to write some documentation and improve the module as much as I can. Meanwhile, I'm looking for feedback. There's probably stuff that's still tied to the wdbc parser, or features you'd like to see, ...

Enjoy it.

http://db.mmo-champion.com/adys/python-structures.tar.gz

Score One for Science.

2010-01-24T13:06:00.000Z

(Happy new year, ha ha.)

Sigrie 2.2.0 released. 3.0 is on the way, as soon as I finish integrating some of the user data. What's still to come?
- Modelviewer. Thanks to Vjeux, I have the necessary javascript to get working on my implementation. Plan is still to use WebGL, so far looks great.
- More userdata. Locations for example. Once a couple of bugs are ironed out, I'll push a bunch of them.
- More useless things. Flight paths for example.

Recently got a bunch of improvements done on the parser, too. Thanks to Andrei, logging and attributes are handled a lot better. I also reimplemented bitmasks after the changes. You can now do item[12345].required_classes.mage == true, stuff like that.
"But wait, there's more!".. preloading and maybe (haha) parsing in C++. What's still keeping me off is the linking hell it's going to be to have Python and C++ living together like I'd like it to. I should also use pySide for the gui, it's likely going to be without changes.

Oh I think my PSU is dead. Which really sucks. I'm still trying to figure it out though. New year means new hardware (no it doesn't!?), so I got myself a nice p55 and an i7 860. Will try them out tomorrow if I get the chance.

I'm taking this loop off.

2009-10-26T16:15:00.001Z

Wave is awesome, hope more people will get in soon. I've migrated a lot of stuff down there which I don't regret one bit. It's mostly stable, but still lacks features.

I was asked for it earlier on, since I had to use it elsewhere recently here's the snippet I use to slugify strings.

strin = """ '".,?!:;%*#&+=()[]<>/"""
strout = "---------------------/"
def slugify(txt):
... trans = maketrans(strin, strout)
... txt = unicode(txt)
... txt = normalize("NFKD", txt).encode("ascii", "ignore").strip().lower()
... txt = txt.replace("'s", "s").translate(trans)
... txt = "-".join(seg for seg in txt.split("-") if seg)[:50].rstrip("-")
... return txt

The way from 2.0

2009-09-26T11:04:00.000+01:00

Sigrie 2.0 is up and running since a few days.
Version numbers are usually of little importance. I like the way Chrome handles them.

BlizzCon obviously slowed me down with updates on Sigrie, which is why I ended up not releasing 1.2 and skipping to 2.0.

The changelog is up, many changes are backend. A lot of necessary cleanup. Slowly paving the way to a better tooltip engine; tooltips should be the best they possibly can.

What's to come? Well, high on priority list are heirlooms. Similarly to the new linkstrings, heirlooms will have support for tooltips of any level.
Loot tables are coming as well. I'm still trying to think of the sanest way to store them.
Maps/locations are on the short-term todo list. I'm working on them every day and should come just after heirlooms.

On a longer time frame, a WebGL modelviewer prototype is planned. It should make for a very strong testcase for both Gecko's and Webkit's WebGL engine.

Looking forward, now.

I don't think anyone has seen this before, we actually beat them to the punch!

2009-08-26T14:17:00.006+01:00

BlizzCon was amazing. Nothing unexpected there. I had lots of fun; visiting the US was an unique experience (although a bit short). The journey from Athens was... long, to say the least -- 14 hours wait in Heathrow, sleeping on chairs with aircon on.

"Expect something... Cataclysmic!"

The leak was a pretty big spoiler. I'm not going into Blizzard politics and whatever was behind it, but either way that didn't stop anyone from having fun. Cataclysm going to be amazing. I played it a bit, got some data and all... it's really gonna be great.

"Now you can panic."

The concert was slightly disappointing. Comparing it to the WWI, the ambiance wasn't the same. Twenty-seven thousand people doing less noise than a group of 6000. You could really feel it in Sam's presentation, too.

The panels were all great. There is so much I want to say, but not until september.

I'm going to work on Sigrie again. Tabs are still not pushed, and there's a bunch of fixes that need doing before the expansion. Got a pretty good area of work going on. Sigrie 2.0 will be out hopefully this week.

PS: Blogger Editor is awful.

BlizzCon 2009

2009-08-11T02:33:00.001+01:00

I've been getting prepared for BlizzCon. Most of everything is ready; I'll just have a 14 hour wait at Heathrow. Other than that, I'm pretty excited!

Sigrie 1.2.1 and 1.2.2 released. I've been working for a while on complete support for custom WDB structures. That, itself, wasn't hard. However, some refactoring is still necessary. Well okay, lots of cleaning up to do, but still!

Been getting some Webkit testcases done. I'll publish a dozen or so when I get some time (read: after BlizzCon).

I'll see you all there.

Don't trust QuerySets

2009-07-20T09:52:00.001+01:00

I messed up pretty bad on this one. I should've seen it coming, too; which is why I'm sharing it today.

>>> def get_prev_next_by_id(self):
>>> next = self.__class__.objects.filter(id__gt=self.id).order_by("id")
>>> prev = self.__class__.objects.filter(id__lt=self.id).order_by("-id")
>>> return prev and prev[0], next and next[0]

Apart from duplicate logic, this looks harmless, right?
QuerySets evaluate on list(), repr() and str()/unicode(). What's expected here is a LIMIT 1 when the query gets sliced.

What actually happens: The entire table gets queried in two lines.
Needless to say, this crashed pretty bad.
Problem is, bool() evaluates the query. Is it a bug? I don't know, I think bool() should use COUNT. Django devs think 10 different things.
COUNT or not, this query wouldn't be efficient either way. Best have a slice ([:1]) on it. The query will still get evaluated on bool(), but it'll be harmless.

Bottom line: Don't trust QuerySets. They're amazing but very sneaky.

Sigrie 1.1

2009-07-18T05:32:00.001+01:00

New release! Bigger one this time with the addition of latest additions and json serialization.

Added a Latest Additions page! Works with almost everything stored in the database, including items, spells, achievements,
quests, item sets, creatures, titles, zones and instances.
Added ability to lookup a model by its name (case-insensitive). Example.
JSON serialization for single models is available. Example.
Added __matches and __imatches lookup filters, which matches words rather than full strings. Example.
Added __in lookup filter, iterating through a comma-separated list of values. Example.
The URL backend has been rewritten and is more responsive and reliable.
Django master sync with 11268.
Completed forward-compatibility with Postgres 8.4.
Lots of bugfixes.

Sigrie 1.0.6

2009-07-12T01:30:00.000+01:00

Sigrie 1.0.6 released, with a few new end-features:

Spellstring conditionals are now supported. Examples here and here.
Tags are live and usable! See for yourself: twin-valkyrs-25-alliance
New "heroic" flag, filterable.
Screenshot thumbnails have been moved and are now a lot bigger.
Some graphical tweaks, mostly on IE.

See you next release.

Remote closed the connection

2009-07-05T01:10:00.000+01:00

Devcub has closed down after an uptime of 462 days (out of 554 full uptime, totalling 1 reboot).

Irrelevant, I know. But it's been quite an experience!

Oh, and Debian is ahead for once. Postgres 8.4 has been released and is already in sid!

Sigrie 1.0.5

2009-07-03T20:11:00.000+01:00

New minor release. Most of it in the changes were either internal or pure optimizations. A couple of things however:

Screenshots can be uploaded by admins. Example here.
Menus have been updated to match the 3.2 PTR categories
Canonical URLs are now supported. They should be on almost every page, but only detailviews actually make use of them. Examples here, here, here and here.
Django master sync with 11178.

I'm hoping to get 1.1 out by 3.2 live release, but I can't be as productive on this laptop. A bunch of JS fixes coming up hopefully soon, maybe a better filter interface, ...

Cheers.

Poor source code... can't even take it.

2009-06-28T23:10:00.001+01:00

Leaving for France in a few hours. I'll have two weeks of rest, might even be able to pull some stuff done.

Sigrie's been updated again, data updates are easier and I'll make them even easier in the days to come. There is still a great deal of work to do in the administration panel (thanks to Django, I didn't have to actually do anything to get a very good admin panel sorted).

For now, almost everything is up to date. I still need to implement some stuff in the spellstrings, though. For example, $floor.

Holiday reqs are up, too; and filterable. I might work on the filter/feature docs if I get time.

See you in two weeks.

Sigrie is Alive

2009-06-25T20:09:00.001+01:00

It's finally officially released.
http://db.mmo-champion.com

Creating a database working with WoW's format without breaking the design on the way is really not an easy task.
Some will know that the project has been in the works for almost two years now. It's been recoded three times from scratch. The last attempt in late 2008, using the Django framework, was the right shot.

However Django was not completely convenient. I had to modify the core for multiple features that... aren't really pretty. But hey, it's live and it's alive.

I'm so happy I got to the end of this design nightmare. I'm working on a C++ port of the wdbc reader, which will act as a Python module. It's a great experience.

Hope you'll like it.

Some KDE usability annoyances

2009-06-21T14:48:00.000+01:00

KDE is great, and really fast, but ...

* Why is Plasma's "Resize Widget" control so awkward? Why can't at least SOME widgets (such as folderview and webbrowser) have real corner-resizing? The current implementation makes it so you can only resize in two directions instead of four.

* Why, why, why reinvent the wheel?
=> Default behaviour in Dolphin is to show a context menu when I drag a file. I want to set it to "move" by default, and have the context menu on shift (which currently moves the file). Why's it so hard to do that?
=> Isn't there SOME way to get a real desktop back? Using a plasma widget for it is horrible; I got two screens, I want to make the best out of both.

* Speaking of dual-screen, plasma widgets are extremely buggy with them. it's near-impossible to drag a widget from one screen to another. That's alpha software though.

* Dolphin again, "Show Hidden Files" should at least be in the right-click context menu of the current folder.

* And Dolphin still and ever. File symlinks got a link arrow tag image; why not folders?

* Oh good god. The "Apply" stuff is eye-gounging. Please tell me there is a way to get the gnome "automatically applies" behaviour. Apply should be kept for extreme cases (such as an action requiring some kind of restart, ...). If I change the color of some element, I'm not going to suddenly change opinion in the second more I need to click Apply; and even if I will, that's what Cancel is for!

* Why in someone sane's name are Keyboard & Mouse options in "Computer Administration"... those are user-specific settings; or should be.

* System Settings again. The KDE team had some nice thing going with their control panel, but currently I can't do two things at the same time... such as changing a wallpaper while I'm reconfiguring the mouse options. The panel should either spawn new processes when opening a config app, or at least I should be able to open two different System Settings processes.

* The "Software Updates" is a prime example of developer logic in a GUI. "Apply all available updates" should NOT be clickable when there is no update to apply. Some other stuff is also confusing ...

Maybe more to come, when I'll be bothered to upload screenshots.

If you don't have a back pocket... improvise.

2009-06-07T14:40:00.003+01:00

Another year gone.

It felt slightly slower this time, with spikes I really enjoyed :)
I've gotten a lot of stuff done; some of my bigger projects are completed or nearing completion.
England was great.

New projects arising, hopes for 2010 and good news all around. Now if only I could enjoy a better climate :)

Smiles.

Hi it's Vince with Django

2009-05-31T10:26:00.000+01:00

This month has been *great*. Got a lot of work done, went to London (was absolutely lovely) and Dover (less lovely but still awesome!).

I really, really can't wait to go back to Britain. I'll be trying to get a flat around Bristol later this year I think.

Most of all I had lots of fun. Now I'm back in Athens, with the good old 40 degrees outside.. urgh =P

So I've worked on DurationFields in Django and they're almost ready to send to trunk. They won't make 1.2 release so I will wait until then to send anything.

The add-on is almost finished, the web interface is complete (pretty much, lacking navigation atm)... I should be busy fixing bugs, but I'm waiting for the js coder to finish some work (what an excuse, I know).

Lastly, I will be going to BlizzCon this year. Back in France for two weeks around the 29th of June, then to California the 20th of August (oh dammit, need to book the plane tickets already).

That's all for the unrelated news for today. Toodles.

DurationFields

2009-05-11T07:28:00.000+01:00

Worked on DurationFields lately, both in the WDBC parser and Django.
The parser implementation was simple enough...

>>> class DurationField(IntegerField):
>>>    units = {
>>>     "years": 31556925993600, # 52.177457 * (7*24*60*60*1000*1000)
>>>        "months": 2629743828768, # 4.34812141 * (7*24*60*60*1000*1000)
>>>       "weeks": 604800000000, # 7*24*60*60*1000*1000
>>>       "days": 86400000000, # 24*60*60*1000*1000
>>>       "hours": 3600000000, # 60*60*1000*1000
>>>       "minutes": 60000000, # 60*1000*1000
>>>     "seconds": 1000000, # 1000*1000
>>>        "milliseconds": 1000,
>>>       "microseconds": 1,
>>>     }
>>>    def __init__(self, name="", unit="seconds", **kwargs):
>>>     IntegerField.__init__(self, name, unit=unit, **kwargs)
>>>        if unit not in self.units:
>>>           raise ValueError
>>>       self.unit = unit
>>>       if hasattr(self, "value"):
>>>           self.duration = self.timedelta(self.value)
>>>
>>>     def timedelta(self, value):
>>>       return timedelta(microseconds=value*self.units[self.unit])

Django's was trickier though, cf ticket 2443. Well, not the implementation itself, but the UI design. The idea was to be able to specify a date the most natural way possible. A friend came up with the idea of parsing a string like this:

>>> def from_timedelta(value):
>>>   if not value:
>>>   return u"0 sec"
>>>   vals = []
>>>   mis = value.days * 24 * 3600 * 1000000 + value.seconds * 1000000 + value.microseconds
>>>   for k in values_in_microseconds:
>>>   if mis >= values_in_microseconds[k]:
>>>   diff, mis = divmod(mis, values_in_microseconds[k])
>>>   vals.append("%d%s" % (diff, k))
>>>   return u" ".join(vals)

What this lets you do is specify a string, such as "1w 3d 10h 50s", and get it parsed into 1 week + 3 days + 10 hours + 50 seconds.
Everything is working, I'm quite happy about it, but there is still some work to do on the Django side.