A little while ago now, I looked at how d3dx is actually quite slow and looked at why that was and that it’s better to use Direct3D directly to do such things. Generally when one wants to take a screenshot, it needs to beveryfast as to not interfere with the gameplay. This post breaks down what to do when presented with an anti-aliased display so that screenshot taking is nice and quick:

LPDIRECT3DSURFACE9 pd3dsBack = NULL;

if ( SUCCEEDED ( device->GetBackBuffer(0, 0, D3DBACKBUFFER_TYPE_MONO, &pd3dsBack) )) {
    D3DSURFACE_DESC desc; pd3dsBack->GetDesc(&desc);
    LPDIRECT3DSURFACE9 pd3dsCopy = NULL;
    if (desc.MultiSampleType != D3DMULTISAMPLE_NONE) {
        if (SUCCEEDED(device->CreateRenderTarget(desc.Width, desc.Height, desc.Format, D3DMULTISAMPLE_NONE, 0, FALSE, &pd3dsCopy, NULL))) {
            if (SUCCEEDED(device->StretchRect(pd3dsBack, NULL, pd3dsCopy, NULL, D3DTEXF_NONE))) {
                pd3dsBack->Release();
                pd3dsBack = pd3dsCopy;
            } else
                pd3dsCopy->Release();
        }
    }
}

The first thing you’ll notice is we grab the back buffer and query whether it has multi-sampling. If it is, we create a new render target and then render the scene onto it. This is far quicker to do than trying to take a shot of a multi-sampled scene as you’ve forced the GPU to do the rendering down onto a surface with no multi-sampling. Job done! That way you can then continue with LockRect() etc. and take the shot as you would do normally.

This makes taking shots of multi-sampled scenes not more than 1–2ms longer than those that aren’t. Enjoy!

D3D Screenshots With Anti-Aliasing… Quickly was originally published in Lee Packham on Medium, where people are continuing the conversation by highlighting and responding to this story.

Docker is a hot topic at the moment in the DevOps world. I use it almost every day and want to look at how automation can be achieved in terms of security and monitoring.

Containers in computing aren’t new. In fact FreeBSD had containers before Google was using them in Linux; although it call them jails.

Docker is great in that it’s brought containers to the masses. Once the reserve of people with the patience to set up LXC on Linux or the painful jails on FreeBSD — side note: it’s very painful I might talk about that another time.

We can talk to Docker via it’s RESTful API and libraries exist for almost every language. The two popular obvious ones are Go and Python — I say obvious, but it’s more that I just prefer these two languages. I’m sure the Ruby one is awesome too.

The downside of Docker that’s coming up more and more is managing security of containers. People often just use official images without a second thought and these end up in production. There’s posts containing loads of FUD on the topic which exist already — but in general how do you ensure you keep your containers’ operating system packages up to date?

Sounds like a task for a script. I broke it down into the following tasks:

• Connect to Docker (boot2docker in my case)
• Get a list of installed packages in debian:jessie image
• Get a list of packages from security.debian.org
• Compare the two

I need to add I used Python 3.4 for this. This makes the syntax seem a little odd to a Python 2.x view so needed to say!

Let’s get connecting out of the way:

from docker.client import Client
from docker.utils import kwargs_from_env

# So we can use boot2docker
kwargs = kwargs_from_env()
kwargs['tls'].assert_hostname = False
kwargs['tls'].verify = False  # Workaround https://github.com/docker/docker-py/issues/465

# Set up the client
client = Client(**kwargs)

Took me a small while to figure out the issue where OpenSSL 1.0.2a causes problems with quite a few libraries and talking to APIs. To get out of it for now I disable the verify part of requests — It’ll complain a lot about it.

Now we’re connected we can make a container and get some stuff out:

# Create my Debian Jessie container
container = client.create_container(
    image='debian:jessie',
    stdin_open=True,
    tty=True,
    command="/usr/bin/dpkg-query -Wf '${Package},${Version}\n'",
)

# Launch it with the custom command
client.start(container)

# Grab the dpkg-query output
output = client.logs(container).decode("utf-8")

packages = {}

lines = output.split('\n')
lines.pop(0)
for line in lines:
    # Last line is a blank
    if not line:
        continue

k, v = line.split(',')
    packages[k] = v

We now have a stack of packages in a dictionary keyed by package name. To do this we make good use of dpkg-query to get a CSV like list of package,version.

What we want next is a similar dict for up to date packages. Now, I know a lot of people who might read this would launch into apt-get update and then query the global list of packages. Would you do that in production? Really? You just want a list of stuff… Let’s just get it from security.debian.org directly.

import gzip
import re

import requests

r = requests.get('http://security.debian.org/dists/jessie/updates/main/binary-amd64/Packages.gz', stream=True)
gz = gzip.GzipFile(fileobj=r.raw)

A small point here… We make use of the gzip library directly to ungzip the file downloaded via Requests. To do this we use ‘r.raw’ like a file which GzipFile can use without any issue.

Now the format of this file is a bit weird. It’s a list of key value pairs for each package with a blank line between packages. The two keys we’re interested in for each package are Package (the name) and Version.

kvregex = re.compile(r'(\w+): (.*)')
security_updates = {}

current_package = None
current_version = None

# Build up a security updates dict
for line in gz.readlines():
    m = kvregex.match(line.decode("utf-8"))
    if not m:
        security_updates[current_package] = current_version
        continue

g = m.groups()
    if g[0] == 'Package':
        current_package = g[1]
    elif g[0] == 'Version':
        current_version = g[1]

r.close()

Perfect! We now have a dict with all the security updates in Jessie keyed by the package name again.

With these two dicts we can intersect them and only get elements that are in both. If the version doesn’t match spit it out. I had to fake an update to exist to test this properly as when I tested there were no out of date packages.

def common_entries(*dcts):
    for i in set(dcts[0]).intersection(*dcts[1:]):
        yield (i,) + tuple(d[i] for d in dcts)

# Fake a security update for Sed... (cos Jessie 8.1 is quite up to date)
security_updates['sed'] = '4.2.2-4+b2'

for entry in common_entries(packages, security_updates):
    if entry[1] != entry[2]:
        print('%s is %s and a security update exists for %s' % entry)

And, there we have it (spot the whining from requests…):

$ python foobar.py
.../venv/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
.../venv/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
.../venv/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
sed is 4.2.2-4+b1 and a security update exists for 4.2.2-4+b2

Awesome! There we have it — a quick way to grab and compare packages against containers.

Experiments With Docker-py was originally published in Lee Packham on Medium, where people are continuing the conversation by highlighting and responding to this story.

It has been a while since I’ve used Node.js for anything serious. To give you an idea of how long go we’re talking… I originally hacked together the Green Man Gaming stock control system in Node.js 0.1.x and to this day it only runs on 0.2.x because of the way 0.4.x changed things way way back and it works so no one dare upgrade it.

Since then, the Node.js ecosystem has matured and although people make fun of it… Node.js is heavily used on things at massive scale. I’d also like to think I’ve learnt to write better Javascript since working on TweetDeck… but let’s not get ahead of ourselves!

So I wanted to get up to date on Node and figured I’d have a quick go at using the ‘parse the deb Packages.gz’ file as a quick thing to try out piping.

This is going to be very crude in places, I’m sure… but here was my quick hack around:

// There be lots of dragons...
var zlib = require('zlib');

var R = require('request'),
    _ = require('lodash'),
    through2 = require('through2'),
    es = require('event-stream');

var url = 'http://security.debian.org/dists/jessie/updates/main/binary-amd64/Packages.gz';

var all = [];

R.get(url)
    .pipe(zlib.createGunzip())
    .on('error', function() {
        console.log("it's all broke yo");
    })
    // Split by each 'object' in the packages file
    .pipe(es.split(/\n\n/))
    .pipe(through2.obj(function(chunk, enc, callback) {
        // We'll get an empty chunk at the end
        if (chunk === "") {
            callback();
            return;
        }
        // Create kv Pairs from each line
        var kvPairs = _.map(chunk.split('\n'), function(line) {
            return line.split(': ');
        });
        // Lower case the attributes - cos that's better
        var fixedAttribs = _.map(kvPairs, function(obj) {
            return [obj[0].toLowerCase(), obj[1]]
        });

this.push(_.zipObject(fixedAttribs));
        callback();
    }))
    .on('data', function(data) {
        all.push(data)
    })
    .on('end', function () {
        console.log(JSON.stringify(all));
    });

I know Streams were introduced way back in 0.8.x of Node.js — but… wow that code is far simpler to understand than counterparts in other languages. It also improved debugging — as I was able to debug each part of the pipe.

The only thing that took me a short while was ‘what if the request fails’. It turned out that I had to use the ‘on’ error bit at the point in the pipe where I was adding the gunzip streaming. If I threw up there — then I’d be all good.

Either way — as a post Dota2 TI5 final night bit of experimenting it was certainly worthwhile.

Node.js Piping was originally published in Lee Packham on Medium, where people are continuing the conversation by highlighting and responding to this story.