Last year I was very fortunate to have been selected as one of the scholars to attend the Electronic Entertainment Expo. I’ve written and given talks on my experiences there, so I’ll not rehash them, but needless to say that it was a phenomenal experience. I was very lucky to be selected, there were many, many applicants (I don’t have exact numbers, but I understand it was around a thousand), and only 15 scholarships were awarded. Words can’t express how great it was.

This year, I applied for a scholarship to attend GDC, the largest gathering of game developers in the world. There were more applicants than ever before – by quite a margin apparently – and still only 15 places available. And somehow, I managed to make the cut again! I was deeply honoured to have been selected yet again and massively excited to be attending GDC, which is really the premier place to meet top-notch game developers, and particularly AI guys.

Today I got some further news – I have won the “Eric Dybsand Memorial AI Scholarship”, presented by the IGDA Foundation, which is an even greater honour. The scholarship commemorates the late Eric Dybsand, something of a legend within the Game AI community. I never had the chance to meet Eric; he died in 2004, way before my career path took a lurch in the Game AI direction, but by all accounts he seems to have been a very talented AI developer and a seriously nice guy. I’m incredibly proud to be a part of his legacy, and overwhelmed at the additional AI-centric opportunities that this opens up for me, such as being an “honoured guest” at the AI Game Programmers Guild annual dinner.

Eric Dybsand

So on the 1st of March I’m heading off to San Francisco. My schedule is already starting to fill up with events, and as part of the IGDA Scholarship program, I know that it will be an amazing time, with barely chance to catch a breath between things. I’ve learned my lesson from last year and budgeted a couple of days either side of the conference to see some sights, not just dash from event to event before hopping on a plane. In just a few weeks, I’ll be enjoying one of the most important events in my field courtesy of the IGDA. I’ll be introduced to new people, and have a chance to chat with old friends. If it’s anything like last year’s E3 event, we’ll be treated like VIPs for pretty much the duration – “gamedev royalty” is a phrase I’ve used previously to describe the feeling it evokes.

I’ve heard it said that the scholarship program selects the “best and brightest” students, and also that those selected are the “future leaders of the games industry”. I can’t speak too much to that – only time will tell. What I do know is that the IGDA have given me yet another amazing opportunity; I’m incredibly grateful and intend to grab it with both hands.

It has become common for us to expect women to earn less, so when the analysis shows us that this is true yet again, we accept it no questions asked. Well, in the next few paragraphs I want to at least ask some questions. I want to be very clear though – this isn’t a post about gender, or workplace equality. This is about maths, and the importance of thorough, informed journalistic analysis of statistics.

So let’s begin.

The first assertion is that 6% of game developers in the UK are female. On the face of it, this is fine. There’s a definite selection bias here I think since the study was conducted online and we’ve seen in other areas that women are in general less likely to express an opinion online. All we can really say with certainty is the 6% of survey respondents were female. We can try to draw broader parallels to the industry, but there are other factors involved in this – does Develop cater to the whole industry or does its demographic have a gender-bias (intentional or unintentional), how and where was the survey publicised, did it take a completely gender neutral tone. All these things affect whether we can say that survey respondents are reflective of a wider group. If you are running a survey on where people shop, and stand outside Tesco to do it, you are obviously introducing a bias that will skew your results. Equally if the first page of an online survey had hardcore porn adverts (an extreme example) it would likely introduce a gender bias into the results, and make them unreflective of the wider group we might expect. These are issues that do have an impact and they do mean that all the conclusions you draw are already, if not suspect, then potentially inaccurate.

But let’s say that we let them off on this one, it’s a tricky issue to compensate for, and we don’t have a better system, so fine. Let’s move on, but bear it in mind.

The next issue is in the fourth paragraph :

Pay proportionality is another key issue for the sector, with British female game dev execs earning almost £3,000 less than their male counterparts. The difference is about £2,000 on a global basis.

This is a very fluffy statement. Execs is poorly defined, the number of each group isn’t displayed and we are assumedly talking about average salary without explaining how that average is calculated, or who exactly qualifies. The big problem of course, comes back to this issue with only 6% of the sample data being women. Out of a sample of 582, 35 were women, and these 35 now somehow represent all women across the sector. We know for high school science and maths that a sample size of 35 is not large, and then we are trimming it down further by talking in terms of female execs. How many were in that sample? Let’s say for the sake of argument that the 35 were evenly distributed between junior level employees, senior level employees and executives, we now have 11 2/3 of our women as executives. What can we deduce from these women? Probably nothing that is going to be accurate, because with so few datapoints, it’s unlikely we have a reflection of all circumstances or an accurate assessment of how many of each type of person is in each circumstance. All we can deduce is that these executives earn less on average than their male counterparts, not female executives in general, and without statements about confidence intervals and standard deviations, we can’t really say that with any certainty.

Let’s do a thought experiment. Suppose a class gets the results of the test they took and we ask them to fill in a survey. We find the following :

Sally 50%

Brian 45%

Tom 45%

Geoff 70%

In this example, Sally got the second highest mark. However, using our survey data we can also conclude that women under-perform men on average. The female average is just 50%, whilst the male average is (45+45+70 / 3) = 53.3 %. However, there is also a practice when calculating averages of discounting outlier results. In this case, Geoff is our outlier, so the male average is now 45% and we can confidently claim that women actually out-perform men!

Later we might get access to the teacher’s mark list and be able to see how close our survey results were :

Sally 50%

Brian 45%

Tom 45%

Geoff 70%

Garry 60%

Linda 80%

Steve 55%

Mike 70%

Laura 75%

Ben 60%

Jim 65%

Now men score 58% on average and women 68,3%. We can see that Brian and Tom have lower scores than everyone – in fact they’re dragging the average way down! Discounting them, the male average jumps to 63.333%, which is getting close to comparable to the women’s average.

Notice that it turned out that our sample, in part because it was so small, happened to be predominantly at the lower end of the marks. That meant that all of our calculations were way off when thinking about the sample as a whole.

So, getting back to the Develop survey we can be sure of one thing – the statistics for women, based on such a small sample size, are almost certainly wrong. And quite possibly very wrong.

Coming back to this concept of outlier results, and the way that Brian and Tom could drag the average around by having results different from the rest of the group. Develop say the following :

Of all female developers surveyed, not one said they earned more than £90,000 per year – 18 men surveyed said they did.

Firstly, let’s address this concept that there should be females at this paygrade. There are 18 male result paid this, out of a sample size of 547, meaning that it is something that occurs with 3,3% frequency. On that basis, out of 35 women surveyed, we should see 1.1 of them at this level – instead we see none. Statistically, this isn’t a major aberration – you can only get people in whole unit chunks, so a difference of one on a result is not really significant.

In fact, if you assume that these high earners fall into the “executive” category mentioned above, it becomes quite astounding that women only trail by £3,000. These outlier results should distort things dramatically – assuming that they were included in the calculation for the average. As I said earlier, it is often the case that outliers are ignored, but the article doesn’t indicate which method they used.

Despite this, the female respondents’ average industry experience was at three-to-five years, as opposed to two-to-three years for the males.

The inference here of course is that the females are more experienced, and are not being paid accordingly. However think about this for a second. We’re being told that females in general are more experienced, but that 18 men earn a wage that no female does. Let’s go back to the classroom, and look at a different test that the pupils took

Sally 50% 3hrs

Brian 45% 1hr

Tom 25% 1hr

Geoff 70% 3hrs

Garry 90% 7hrs

Linda 80% 5hrs

Steve 55% 2hrs

Mike 50% 4hrs

Laura 75% 3hrs

Ben 60% 2hrs

Jim 65% 3hrs

On average the females studied for 3 2/3 hours for the test, whilst the males only studied 2.5 hours. Does this have any bearing on Garry’s result of 90% or the fact that no female got a score of 90% or better? Yet this is exactly what Develop would have you believe.

Finally, in the next paragraph, immediately after the jump, Develop claims that 42% of game customers are women. They base this on ESA data without any comment, such as highlighting that the ESA is an organisation based in the US, and demographic data from the US does not necessarily track with demographic data in the UK, which was the target demographic of the survey data. The point they were trying to make was to link the 6% representation of women in the creation of entertainment for which women represent a much more significant chunk of the customers, evidently they didn’t have UK data, and glossed over this pretty quickly.

I don’t know if there’s a pay gap between men and women. It makes sense to me that there would be given the situation in other sectors. I’m not trying to say that there isn’t, or that there is. What I’m trying to show to you is that this article is absolute horse-shit, written by someone at best with a dubious grasp on statistics, and at worst pushing their own agenda and stretching the analysis as far as possible to make the desired points. We can get up in arms about the perception that there is if you want.

Or, we could get up in arms that this is the quality of statistical analysis we get from a news organisation covering our highly technical field.

SOPA and PIPA both seem very dangerous pieces of legislation, but realistically, I’ve got no say in it. This is an internal US issue and protests by non-US citizens/corporations are unlikely to have any sway on US representatives.

Ask me anything

I’d be delighted to give a presentation on any of this material pretty much anywhere, either online or in person. Please drop me a line using luke@lukedicken.com

1. Game Theory
2. Probability Theory
3. Decision Making for Games
4. Opponent Modelling
5. Procedural Content Generation
6. Procedural Content and Player Models
7. Experience Management
8. What is Game AI? Final Thoughts

• Franks Red Hot – This I think generates the most authentic Buffalo experience that I’ve made so far. 5/5
• Tabasco (Original) – I found that I didn’t get a good texture with Tabasco. Additionally there was too much of a pepper kick to the wings and not enough flavour. It’s not dire in a pinch if it’s the only sauce you have in. 2/5

My parents came over last night, and both agreed that my wings were really good and authentic, which is great to hear, particularly as my fat fryer died just as they arrived and we were about 1-2 hours delayed whilst trying to repair it, then popping to the store and getting a replacement. They were kept waiting for quite a while, but they seemed to leave pretty satisfied!

As some of you might know, I’m trying to pull together a jam-team for next weekend’s Ludum Dare. I’ve got a couple of great people on board so far, but we’ve still got a few slots we could do with getting more people for, and we’re very VERY stuck for art.

We’re getting together on Friday night, and hanging out / resting / getting to know each other until the theme of the weekend is released (around 2am our time). Then we have 72 hours to work on putting something together, although I’m expecting that we’ll actually wind down on Sunday evening to allow people to get some rest before work on the Monday.

Right now we have a producer, a coder and a writer, so whatever your talent, if you’re interested in joining us, I’m interested in hearing from you! As I said though, the thing we are definitely lacking is the ability to art our way out of a paper bag, so artists, please please do drop me a line. You can get hold of me either through FB or Twitter to @LukeD, or if email is your thing: lukedicken@gmail.com

First of all, I have to tell you that the secret to a good Buffalo recipe is the hot sauce. I heard this a lot and figured it was nonsense – hot sauce is hot sauce (give or take) right? Weeeeelll no. The hot sauce is key. Sorry. The good news is that in the UK, you can buy the good stuff now. It’s called “Frank’s Red Hot Sauce” and Tesco and Sainsbury have both started selling it. I don’t know why Franks is so good, and works so well. but it does. You can’t just throw Tabasco at it, or Tabasco Habanero or any of a hundred other sauces at it. You won’t get that Buffalo taste. Frank’s gets you there, but I hate being reliant on a single brand continuing to be sold, so I’ll probably look into alternatives at some point

OK, with that said, here’s what you do to make great at home Buffalo Wings.

* First off, if you need to, separate your wings up. I don’t know the technical term for them, but you’ll probably want three bits – the tip (which atm I’m discarding, but figure there are better things to do with), the “forearm” and the “upper arm”.

* Throw them in your deep fat fryer until they’re golden brown. You can’t really have too crispy a wing here, so err on the side of lengthy. 10-15m for 6-9 piece is probably a decent bet at high temperature.

* Towards the end of cooking, get a pan and melt some butter into it on a low heat. How much you use is going to depend a lot on how many wings you intend to have in the end. As one datapoint, I found around 15g of butter did good for around 15 wings.

* Once the butter is melted, you’re going to want to add your Franks. A lot of it. It’s not entirely economical but you need a decent amount of your “sauce” for what comes later. At 15g grams of butter I used around 1/2 a bottle of Franks – there’s probably a lot of scope for refinement here mind you.

* As your wings are complete, add them to the saucepan you have your sauce in, and get a good stir or toss on to make sure you get 100% coverage. You can leave wings say in this pan just now if you need to run a second batch of wings through your fryer to get the right quantity for the number of folks you have.

* So now you have a pile of wings, all coated in a slightly more sticky version of Franks. But they’re a bit wet as it is, so pre-heat your oven to around 100 Celsius, and put them on a grid inside a tray. The idea here is to drain off the excess while concentrating the mixture just a bit – and of course keeping everything hot still.

*After a few minutes, you should have chicken wings with a nice thick highlighter-pen orange glaze to them, and you’re ready to tuck in to what hopefully is one of the more accurate renditions of this classic American dish you’re going to find this side of the Atlantic.

Now some of you have asked if I served these with a proper blue cheese sauce. And although I’ve been working on it, tonight I lazied out. I had some sour cream and chive dip open and “that’s close enough right?”. It’s one of the many minor “fuuuuuuuu…” moments when I realised that I’d finally cracked it (or close), yet left the components of my nearest-approximation-to-blue-cheese-sauce in the fridge. Again, once I have this one nailed down completely, I’ll be back to talk about it.

• • Landmarks
    • “On the Extraction, Ordering and Usage of Landmarks in Planning” Porteous et al, ECP ’01
  • Abstraction
    • “Applying Clustering Techniques to Reduce Complexity in Automated Planning Domains” Dicken & Levine, IDEAL ’10
  • Relaxed Plan Graph
    • “The FF Planning System: Fast plan Generation Through Heuristic Search” Hoffman, JAIR Vol. 14
  • Landmark Heuristic
    • “The LAMA Planner Using Landmark Counting in Heuristic Search” Richter & Westphal, IPC ’08
  • HTNs
    • “SHOP2 : An HTN Planning System” Nau et al, JAIR Vol. 20
  • Execute/Replan
    • “FF-Replan: A baseline for probabilistic planning” Yoon et al, ICAPS ’07
  • Execution Monitoring
    • “T-REX: A Deliberative System for AUV Control” McGann et al, PPERWS ’07

If you find yourself unable to track down a copy of one of these, drop me a note and I will source it for you.

Returning from LA I managed to get very sick with one of the many bugs that was floating around the convention centre, which took me out of action for most of the subsequent week, and then I began preparing for the Paris Game/AI Conference which I’m at right now. I’m giving at least one session here and potentially a second one tomorrow as well.

I have managed to put together this month a couple of Tech Demos to show you the kind of direction my work is heading in :

- http://lukedicken.com/projects/AITest/WebPlayer.html is a demonstration of a flocking behaviour in a group of birds, based on the Reynolds approach to emergent behaviour.
- http://lukedicken.com/projects/AITest/DayNight/WebPlayer.html is a quick tech demo showcasing a day/night cycle. It isn’t properly finished but it’s something I’ve been working on.

Finally, we have just in the last day launched the new website for our research group. I’m very proud of this site, and we’ve had a lot of help from Epitome Solutions and @maniacyak . I’ve not put all of the content up yet, in particular I want to get more media up there showing off our work, but we pushed hard to ensure that the site launched before the Game AI Conference started, and we met the target! You can take a look at here.

I really really hope I can sit down and catch my breath in July – I’ve lost a stone dashing about getting all of this done!