The Invisible Things Lab's blog

Qubes Beta 3!

2012-02-06T11:45:00.002+01:00

A new ISO with the just released QubesBeta 3 is now available for download here.

Beta 3 fixes lots of annoying problemsdiscovered in Beta 2 and earlier releases, and also implements abunch of useful feature:

This includes the qvm-block tool and infrastructure for easy attachment of block devices to any AppVM,no matter which system VM is handling the actual USB controller. So,this allows to have untrusted USB domain(s), almost seamlesslyintegrated in the desktop system. One can consider to use it in orderto prevent various USB attacks. Thenext release (the 1.0) will bring this feature to the Qubes GUImanager as well, making it easy to use for non-command-line userstoo.

Also, we have now introduced fullyautomatic Qubes build system, that allows to build all the Qubespackages, and also create the installation ISO, with just onecommand. More information on this system and on how to use it can be found inthe wiki.

We have also updated to Fedora 15-basedtemplate as a default. Unfortunately F16-based template would requiretoo much work to get all the Gnome 3 stuff working correctly. (The challenge here, is that we don't run a normal Windows and Desktop manager in every domain, in order to make the VMs light weight, and so we need to sometimes work around various problems this causes...).

Finally, we have added two newQubes-specific applications:

A plugin for Thunderbird (it isautomatically installed in the template), that allows for one clickopening of attachments in Disposable VMs, as well as one-click savingof the attachment to select VM.

And something we call “splitGPG”, that I will describe in a separate article later.

ThoseQubes-specific applications are based on our Qubes RPC, introduced in Beta 2.

This is likely the last release beforethe “final 1.0”, that is scheduled to follow soon(TM). The only major work for 1.0 is GUI manager improvements toexpose most of the Qubes functionality via clickable GUI, and commandline tools cleanup and documentation. Plus testing and bugfixing :)

Andthen, the next thing we will be working on will be support for HVMdomains, e.g. Windows. This work is starting actually just about now, but code will be released only after Qubes 1.0.

Thoughts on DeepSafe

2012-01-21T18:01:00.002+01:00

Several people asked me recently what Ithough about DeepSafe.So, below I present my opinion...

First, for any AV system (or Host IPS,or Personal Firewall, etc) to work effectively, there are three problems that must be addressed:

How to protect the AV agent (code and data) from tampering (from the rest of the OS)?
How can the AV agent get reliable access to (sensitive pieces of) the system memory and registers, and/or provide reliable memory protection for the (sensitive pieces of) the OS.
What are those "sensitive pieces of” memory that should be monitored or protected?

From reading various PR materials, itseems like the #1 above is the primary differentiation factor forDeepSafe (DS). So, let's consider this problem in the context of e.g.a Windows OS. In order to protect its code and data, DS uses, as itis heavily advertised, Intel VT-x virtualization technology. Now,that sounds really secure -- after all what can be more secure than ahardware virtualization, right? ;)

But VT-x (including EPT) is only aboutCPU virtualization, which in our case translates to protecting the DSmemory and registers from CPU-originating accesses. But, as everyregular to this blog knows, there is also another method of accessingmemory on any PC system, and this is through DMA transactions fromdevices. The OS (so also the kernel malware) is free to program oneof the many devices in the system to issue DMA reads or writes to anyphysical memory itwants...

Now, in order to protect some portionof the system memory (DRAM, cache) against DMA accesses, we have theIntel VT-d technology... So, one would think that DS must be alsousing VT-d in order to protect itself.

Very well, let's assume then that theDeepSafe is not a total ripoff,and that it implements also VT-d protection for its agent, although Ihaven't found this mentioned in any of the public papers or pressmaterials I found on the web...

This, however, would be a bit complexto do correctly, because the OS (so, also the kernel malware) stillhas a full control over the chipset (MCH), which is the entity...that controls the VT-d.

Now, in order toprevent the OS (or the kernel malware) from playing with the chipsetfor fun and profit, and e.g. disabling VT-d protection, DS would haveto virtualize the chipset.

If you look at some consumer VMMs, suchas VMware or Xen/Qemu, you would see that they all virtualize thechipset for their guests (of course), but that the chipset theyprovide this way is some kind of an ancient Pentium MCH. I don'tthink any of the consumers would be especially happy if they foundout that after installing DS on their brand new 2012 laptop, Windowssuddenly see a Pentium-era chipset... And this is not without areason – chipsets, specifically MCHs, are one of the most complexdevices, perhaps only beaten by GPUs in this category. There arevirtually hundreds of configuration registers exposed by the chipset,some of them control the VT-d, some other control system memory mapsand permissions, PCIe configuration, and many other things that Ieven have no idea about, and this all makes virtualizing the chipseta very challenging task.

So, it's either that McAfee and Intelfound some interesting way of how to securely virtualize the chipsetwhile preserving all of its (very rich) functionality, or thatthey... don't bother with VT-d protection and chipset virtualizationat all, assuming that even without VT-d, DeepSafe is good enough and“rises the bar” anyway (sarcasm intended).

(Can somebody from McAfee or Intelconfirm in the comments below what does DP really do?)

Anyway, let's assume they dohave VT-d protection and they do virtualize the chipsetsomehow...

Now, we're moving on to the #2 pointfrom the list of tasks above -- about the reliable

memory access or reliable protection.

So, let say that the DS agent decidedthat some part of the system memory, e.g. the IDT table, is sensitiveand should be monitored/protected. So it sets up EPT traps to triggeran VT-x/EPT intercept on any access to that memory (or IDT baseregister), in order to find kernel malware that tried to modify IDT.That sounds really nice, but what if the malware uses DMA to modifyIDT? DS would not be able to catch such access! (So far we consideredthe, hypothetical, use of VT-d only to protect the DS agent code).

One might think that DS is programmingVT-d to sandbox each and every device in the system (so includingGPU, USB controllers, NICs, SATA, etc) so they never beallowed to touch any of those sensitive parts of the system, such asIDT. Let's assume they do it this way...

And here we've arrived to the lastpoint from the list at the beginning: which of the system memoryconstitutes those "sensitive pieces" that should beprotected/monitored? IDT? Sure. What about all the code sections ofthe all the kernel modules? Yes. Are we fine now? Well, no, becausethe malware can hook some pointers other than the well known IDT.Some public NDIS data structure? Ok, we can add those to theprotected areas. But, what about some undocumented NDIS structures?And this is just NDIS subsystem, one of the many subsystems in theWindows kernel... When we think about it, it should be intuitivelyobvious that in a general purpose Operating System like Windows, itis not possible (at least for 3^rd party) to make asatisfactory list of all the sensitive pieces of memory that shouldbe monitored/protected, in order to detect all the systemcompromises.

Greg Hoglund, Jamie Butler, AlexTereshkin, and myself, have been researching this area actively inthe early years of this millennium. In addition to the Alex's paperlinked above, you might also check out one of my last slides fromthisperiod.

I don't think anything has changedsince that time. It was also the reason why I gave up on writingWindows compromise detectors, or forensic tools, and moved on toresearching other ways to secure OSes, which finally gave birth toQubes OS, many years later.

So, back to DS -- in order to provide asomehow satisfactory protection level for your general purpose OS,such as Windows, it would need to:

Use VT-d to protect its own memory,

Virtualize the chipset, at least some (sensitive) parts of it,

Program VT-d permissions for each device to exclude all the sensitive areas in the system that should be protected, and also protect one device from DMAing into/from another device memory (e.g. NIC stealing GPU framebuffer, or inserting instructions to the GPU instruction buffer, or keystrokes to USB controller buffer). Ideally, this could be done by programming VT-d to grant each device only access to its own DMA buffer, but as far as I know, this would be very hard to implement, if not impossible for a 3rd party, on a Windows OS (in contrast to Linux, which mostly support this model). Please correct me, if the recent Windows version allows for such use of VT-d.

Finally, and the most hard thing to solve, how to define all the "sensitive pieces of memory" in the system that should be protected and/or monitored? Although this is a somehow more generic problem, not specific to DS, but applying to any A/V, HIPS, or forensic tool.

So, is DeepSafe another piece of crap not worth any special attention, or has McAfeeand Intel came up with some novel methods, e.g. for chipsetvirtualization, and other problems? Unless I see some technical info to backup the latter, I would have to assume,unfortunately, the former. But I would like to be mistaken – afterall DeepSafe seems to be just a new incarnation of my Bluepill ;)

Trusted Execution In Untrusted Cloud

2011-12-13T20:25:00.000+01:00

Wouldn't it be nice if we couldactually own our data and programs in the cloud? By “owning” hereI mean to have control over their confidentiality and integrity. Whenit comes to confidentiality and integrity for the data,it's not much of a rocket since, as the classic crypto (andsecure client systems) is all that we need. Ihave already wrote about it in an earlier post.

But itwould also be nice, if we could somehow get the same confidentialityand integrity assurance for our programsthat we upload for the execution in the cloud...

Forexample, a company might want take theirdatabase application, that deal with all sorts of corporate criticalsensitive data, and then upload and safely run this application on e.g.Amazon's EC2, or maybe even to some China-based EC2-clone. Currently there is really nothingthat could stop the provider, who has a full control over the kernelor the hypervisor under which our application (or our VM) executes, from reading the contents of our process' memory and stealingthe secrets from there. This is all easy stuff to do from the technical point ofview, and this is also not just my own paranoia...

Plus,there are the usual concerns, such as: is the infrastructure of thecloud provider really that safe and secure, as it is advertised? Howdo we know nobody found an exploitable bug in the hypervisor and wasnot able to compromise other customer's VMs from within theattacker-hired VM? Perhaps the same question applies if we didn't decided to outsource the apps to a 3^rdparty cloud, but in case of a 3^rdparty clouds we really don't know about what measures have beenapplied. E.g. does the physical server on which my VMs are hosted also used to host some foreign customers? From China maybe? Youget the point.

Sometimesall we really need is just integrity, e.g. if we wanted to host anopen source code revision system, e.g. a git repository or a fileserver. Remember the kernel.org incident?On a side note, I find the Jonathan Corbet's self-comforting remarkson how there was really nothing to worry about, to be strikinglynaive... I could easily think of a few examples of how theattacker(s) could have exploited this incident, so that Linus &co. would never (not soon) find out. But that's another story...

But, how can one protect a running process, or a VM, from apotentially compromised OS, or a hypervisor/VMM?

Tosome extent, at least theoretically, Intel Trusted ExecutionTechnology (TXT), could be used to implement such protection.Intel TXT can attest to a remote entity, in that case this would bethe cloud customer, about the hash of the hypervisor (or kernel) that has beenloaded on the platform. This means it should be possible for the userto know that the cloud provider uses the unmodified Xen 4.1.1 binaryas the hypervisor and not some modified version, with a built-in FBI backdoor formemory inspection. Ok, it's a poor example, because the Xenarchitecture (and any other commercially used VMM) allow the administrator who controls Dom0 (or equivalent) to essentiallyinspect and modify all the memory in the system, also that belongingto other VMs, and no special backdoors in the hypervisor are needed for this.

But let's assume hypothetically that Xen 5.0 would change thatarchitecture, and so theDom0 would not be able to access any other VM's memory anymore. Additionally,if we also assumed that the Xen hypervisor was secure, so that it wasnot possible to exploit any flaw in the hypervior, then we should be fine.Of course, assuming also there were also no flaws in the TXT implementation, and that the SMM was properly sandboxed, or that we trusted (some partsof) the BIOS (these are really complex problems to solve in practice, but I knowthere is some work going on in this area, so there is some hope).

Such aTXT-bases solution, although a step forward, still requires us to trust the cloud provider abit... First, TXT doesn't protect against bus-level physical attacks– think of an attacker who replaces the DRAM dies with some kind ofDRAM emulator – a device that looks like DRAM to the host, but onthe other end allows full inspection/modification of its contents(well, ok, this is still a bit tricky, because of the lack ofsynchronization, but doable).

Additionallyfor Remote Attestation to make any sense, we must somehow know thatwe “talk to” a real TPM, and not to some software-emulated TPM.The idea here is that only a “real” TPM would have access to aprivate key, called Endorsement Key, used for signing during RemoteAttestation procedure (or used during the generation of the AIK key,that can be used alternatively for Remote Attestation). But thenagain who generates (and so: owns) the private endorsement keys? Well,the TPM manufacturer, that can be... some Asian company that we notnecessarily want to trust that much...

Now we see it would really be advantageous for customers, if Intel decidedto return to the practice of implementing TPM internally inside the chipset, as they did in the past for their Series 4 chipsets (e.g.Q45). This would also protect against the LCP bus-level attacksagainst TPM (although somebody told me recently that TPM in currentsystems cannot be so easily attacked from LCP bus, because of someauthentication protocol being used there – I really don't know, asphysical attacks have not been the area we ever looked atextensively; any comments on that?).

Butthen again, the problem of DRAM content sniffing always remains,although I would consider this to be a complex and expensive attack.So, it seems to me that most governments would be able to bypass suchTXT-ensured guarantees in order to “tap” the user's programsexecuting in the cloud provides that operate within theirjurisdictions. But at least this could stop malicious companies fromstaring up fake cloud services with an intent to easily harvest somesensitive data from unsuspecting users.

It seems that the only way to solve the above problem of DRAM sniffing attacks is to add some protection at the processor level. We can imagine two solutions that processor vendors could implement:

First,they could opt for adding an in-processor hardware mechanism forencrypting all the data that leave the processor, to ensure thateverything the is kept in the DRAM is encrypted (and, of course, alsointegrity-protected), with some private key that never leave the processor. This could be seen as an extension to the Intel TXT.

This would mean, however, we still needed to relay on:1) the hypervisor to not contain bugs, 2) the whole VMM architectureto properly protect VM's memory, specifically against the Dom0, 3)Intel TXT to not be buggy either, 4) SMM being properly sandboxed, oralternatively to trust (some parts of) the BIOS and SMI handler, 5)TPM's EK key to be non-compromised and verifiable as genuine, and 6)TPM bus attacks made impossible (those two could be achieved bymoving the TPM back onto the chipset, as mentioned above), and finally, 7) on theencryption key used by the processor for data encryption to be safelykept in the processor.

That'sstill quite a lot of things to trust, and it requires quite a lot of work to make it practically really secure...

The other option is a bit more crazy, but also more powerful. Theidea is that the processor might allow to create untrustedsupervisors (or hypervisors).Bringing this down to x86 nomenclature, it would mean that kernelmode (or VT-x root) code cannot sniff or inject code into (crypto-protected) memory of the usermode processes (or VT-x guests).This idea is not as crazy as you might think, and there has even beensome academic work done in this area.Of course, there are many catches here, as this would requirespecifically written and designed applications. And if we everconsidered to use this technology also for client systems (how niceit would be if we could just get rid of some 200-300 kLOC of the Xenhypervisor from the TCB in Qubes OS!), the challenges are evenbigger, mostly relating to safe and secure trusted output (screen)and, especially, input (keyboard, mouse).

Ifthis worked out, then we would need to trust just one element: theprocessor. But we need to trust it anyway.Of course, we also need to trust some software stack, e.g. thecompilers we use at home to build our application, and the librariesit uses, but that's somehow an unrelated issue. What is important isthat we now would be able to choose that (important) software stackourselves, and don't care about all the other software used by thecloud provider.

As Iwrote above, the processor is this final element we always need torust. In practice this comes down to also trusting the US government :) But we might imagine users consciously choosing e.g.China-based, or Russia-based cloud providers and require(cryptographically) to run their hosted programs on US-madeprocessors. I guess this could provide reasonable politically-based safety. And there is also ARM, with its licensable processor cores, where, I canimagine, the licensee (e.g. an EU state) would be able to put theirown private key, not known to any other government (here I assume thelicensee also audits the processor RTL for any signs of backdoors). I'm not sure if it would bepossible to hide such a private key from a foundry in Hong Kong, orsomewhere, but luckily there are also some foundries within the EU.

In anycase, it seems like we could make our cloud computing orders ofmagnitude safer and more secure than what is now. Let's see whetherthe industry will follow this path...

Exploring new lands on Intel CPUs (SINIT code execution hijacking)

2011-12-06T10:48:00.001+01:00

Today we're releasing a new paper where we describe exploiting a bug in Intel SINIT authenticated code module that allows for arbitrary code execution in what we call an “SINIT mode”. So, to the already pretty-well explored “lands” on Intel processors, that include ring 3 (usermode), ring 0 (kernelmode), ring “-1” (VT-x root), and ring “-2” (SMM), we're now adding a new “island”, the SINIT mode, a previously unexplored territory inhabited so far only by the Intel-blessed opcodes.

What is really interesting about the attack are the consequences of SINIT mode hijacking, which include ability to bypass Intel TXT, LCP, and also compromise system SMRAM.

It's also interesting how difficult was this vulnerability for Intel to patch, as they had to release not only updated SINIT modules, but also updated microcode for all the affected processors, and also work with the BIOS vendors so they release updated BIOSes that would be unconditionally loading this updated microcode (plus provide anti-rollback mechanisms for both the BIOS and microcode). Quite an undertaking...

You can get the paper here.

Intel also published an advisory yesterday, which can be downloaded from their website here. The advisory is peculiar in a few ways, however...

First, the advisory (I'm referring to the revision 1.0) never explicitly mentions that the attack allows to bypass TXT launch itself, only that the attack “may compromise certain SINIT ACM functionality, including launch control policy and additionally lead to compromise of System Management Mode (SMM). Intel also recommend to disable TXT altogether in the BIOS, as a preventive measure, in case the user doesn't “actively running Intel® TXT”... This reminds me how various vendors started actively disabling Intel VT-x after certain virtualization rootkits have been demonstrated some 5 years ago, and how many laptops still ship with this technology disabled today (or VT-d at least) to the questionable delight of many users.

Second, the advisory assigns only an “Important” rating to this vulnerability, even though another Intel advisory, published some two years ago for a problem also reported by us, and which which was strictly a subset of the current vulnerability in terms of powers that it gave to the attacker (in other words the current vulnerability provides the attacker with everything that the previous one did, plus much more), was given a “Critical” rating... This is called evolution, I guess, and I wonder what would be considered critical by Intel these days?

UPDATE (Dec 7th, 2011): Intel has just released an updated advisory (release 1.1) that now explicitly states that the vulnerability also bypasses Intel TXT.

This is the last paper co-authored with Rafal Wojtczuk, who recently decided to try some new things and to leave ITL. Rafal has been the most talented exploit writer I have worked with, and I will surely miss his ingenious insights, such as e.g. how to practically win an absolutely hopeless race condition with ICMP-delivered MSI! But then again, how many times can one break Intel technologies, before getting bored? At the same time ITL is really transforming now into a development company, with all our efforts around Qubes and architecting, rather than on breaking. I wish Rafal all the best with his new endeavors, and thank him for all the excellent contributions he made while working for ITL over the past 3+ years.

Playing with Qubes Networking for Fun and Profit

2011-09-28T16:36:00.002+02:00

Today, I would like to showcase some of the cool things that one can do with the Qubes networking infrastructure, specifically with all the new features that have been brought by the just released Qubes Beta 2. This will cover the use of multiple Net VMs for creating isolated networks, the use of a Proxy VM for creating a transparent Tor Proxy VM, as well as demonstration of how to use a Standalone VM with manually assigned devices, to create a “WiFi pen-testing” VM, which surely represents the “for fun” aspect of this post.

Qubes Networking Intro

From the networking point of view there are three types of VMs in Qubes:

Net VMs, that have networking devices assigned to them, such as e.g. a WiFi or Ethernet card. Each Net VM contains a Xen network backend that is used to provide networking to all VMs that are connected to this Net VM.
Regular VMs (AppVMs) that use the networking provided by Net VMs (so they have Xen network frontends that provide virtual interfaces that are backed by the backend in the corresponding Net VM.
Proxy VMs that combine both of the above: to Net VMs they look like regular AppVMs, because they are consumers of the networking they provide, but to other AppVMs they act as if they were Net VMs themselves, allowing other VMs to connect to them. Of course the Proxy VMs do not have directly assigned networking devices – they use the networking provided by the Net VM that they connect to. One can chain many Proxy VMs, as we will see below.

The virtual interfaces in client VMs are called ethX, and are provided by the xen_netfront kernel module, and the corresponding interfaces in the Net/Proxy VM are called vifX.Y and are created by the xen_netback module.

Each Net and Proxy VM implements NAT, specifically masquerading, for all the connected VMs. Additionally to this SNAT, each Net or Proxy VM provides also DNAT redirection for DNS resolutions, so that each VM behind a Proxy or Net VM thinks that it uses a DNS in the Net/Proxy VM, but in fact all the DNS request are DNAT-ed by all the Proxy and Net VMs down the original DNS that is provided to the final Net VM. This smart trick allows us to avoid running a DNS caching server in Proxy/Net VMs.

Also, any VM-to-VM traffic, among the VMs connected to the same Net/Proxy VM is blocked by default.

Additionally, each Proxy VM enforces system-wide firewaling rules, specifically the rules for all the directly connected VMs. Those firewalling rules are centrally managed in Dom0 and exposed to each Proxy VM through Xen store. One useful application of this firewalling mechanism is to limit certain VMs to only specific type of white-listed traffic to minimize likelihood of user mistakes. A good example could be a work VM that might be limited to network connectivity only with the select corporate servers and denied all other traffic. This way, when the user receives an email message with an embedded http link (possibly leading to a malicious website) and accidentally clicks on it, nothing wrong happens.

The current infrastructure doesn't support IPv6 routing, but we will likely add this support in the upcoming Beta 3.

The default networking topology in Qubes OS

When you proceed with the default installation of Qubes Beta 2, then your initial networking topology looks like on the diagram below:

The default network configuration in Qubes.

So, by default there is one Net VM, called 'netvm', that is automatically assigned all the networking devices in the system. There is also one Proxy VM, called 'firewallvm' that is directly connected to the default Net VM, and which provides networking to all other VMs in the system. This Proxy VM is used for firewall rules enforcement. Each such service VM consumes 200MB of RAM by default.

Network-isolated VMs

For some VMs it might be desirable to completely disconnect them from any kind of networking access. This can be easy done using the following command (issued from Dom0's konsole):

[dom0]$ qvm-prefs -s netvm none

For example I have a 'vault' VM that I use for keeping my master PGP keys, and other secrets, and this machine is not connected to any network.

Using multiple Net VMs for physically isolated networks

In some scenarios the machine might be connected to two or more physically separate networks (e.g. safe corporate intranet, reachable via ethernet cable on the user's desk, and the unsafe and evil Internet, reachable via WiFi card).

It is easy to use more than one Net VMs in Qubes, and assign different networking devices to different Net VMs, and also decide which VMs are connected to which Net VMs. The diagram below presents an exemplary such setup:

A simple setup with two isolated networks, and one fully isolated domain ('vault').

We could created such a setup using the following commands (issued in Dom0):

[dom0]$ qvm-create netvm1 --net --label red

[dom0]$ qvm-create netvm2 --net --label yellow

Currently qvm-create when used with the --net option automatically assigns all networking devices to the just created VM, so in the example above you would want to remove extra devices from each Net VM using qvm-pci -d, leaving only those you really want, e.g.:

[dom0]$ qvm-pci -l netvm1 # to get a list of currently assigned devices

[dom0]$ qvm-pci -d netvm1 02:00.0

Now we should create the Firewall VMs:

[dom0]$ qvm-create firewallvm1 --proxy --label green

[dom0]$ qvm-create firewallvm2 --proxy --label green

... and connect them to proper Net VMs:

[dom0]$ qvm-prefs -s firewallvm1 netvm netvm1

[dom0]$ qvm-prefs -s firewallvm2 netvm netvm2

And now, for any other VM, just set the appropriate Net VM (either firewallvm1 or firewallvm2, or 'none), to get it assigned to either of the isolated networks, e.g.:

[dom0]$ qvm-prefs -s banking netvm firewallvm1

[dom0]$ qvm-prefs -s xfiles netvm firewallvm2

[dom0]$ qvm-prefs -s vault netvm none

...

This configuration provides very strong isolation between the VMs belonging to network #1, and the VMs belonging to network #2. Specifically, this becomes significant if we fear about potential remotely exploitable bugs in the client code of the core TCP/IP stack (in this case the Net VM could potentially compromise all the connected VMs -- but the same problem applies to even physically separated machines that use the same network).

Setting up Tor Proxy using a Proxy VM

Let's now play a bit with Proxy VMs and see how we can use it to create a simple Tor proxy VM. Such a VM would provide anonymized networking to all its clients, so would allow to easily create VMs for anonymous Internet access. The simple setup we would like to prepare is depicted on the figure below:

The 'torvm' Proxy VM provides anonymized networking to 'anon-web' and 'anon-bitcoin' VMs. All the traffic generated by the VMs behind 'torvm' is either fed into the Tor network, or discarded. Furthermore, any app running in those VMs is not able to read any global system identifiers, such as the external IP, external MAC address, etc.

Our Tor proxy would forward only the Tor traffic, so we don't have tofear about some Tor-not-aware applications, or even intentionallymalicious ones to compromise the privacy of our connection. This isbecause such applications have no way to generate traffic to theoutside world without going through our Tor proxy (unless they couldexploit a hypothetical vulnerability in the Tor process running inthe Tor VM). Also, the applications running in any VM behind the Torproxy are not able to determine any globally identifiable IDs, suchas the user's external IP address, the real MAC address used by realNICs, etc.

Interestingly just after writing the above paragraph, I discoveredthat one of our xenstore keys had wrong permissions and, as a result,any VM could read it and get to know the actual external IP (the keyis used by a Net VM to communicate the external IP configuration tothe connected Proxy VMs, so they could know when to update thefirewall configuration). The fix for this problem is here,and the update (qubes-core-dom0-1.6.32) is now available for Dom0(just do qvm-dom0-updateto get it installed).

So, this represents a rather strong setup for use with Tor. Let's nowhave a look at how to practically create such a configuration, stepby step.

First, let's create the VM that will become our Tor proxy:

[dom0]$qvm-create torvm --proxy --label green

This will create a Proxy VM named 'torvm', based on the defaulttemplate. We will need to now start the template VM and install theTor client there:

[dom0]$ qvm-run-a fedora-14-x64 gnome-terminal

Alternatively, if we didn't trust the Tor client rpm package to benon-malicious, specifically for its installation scripts to be nonmalicious, we could have based this on a different template, e.g. oneused for less trusted VMs, or we could installed the Tor client in/usr/local,that is backed by the VM's private storage, but this would requirecompiling Tor from sources.

Now, in the just started template VM,lets install the Tor client and (optionally) the Vidalia graphicalfrontend:

[fedora-14-x64]$sudo yum install tor vidalia

And then power offthe template VM. Now, every VM based on this template, started afterthe template shutdown, will also see the Tor binary in itsfilesystem.

Let's now configureour torvm to properly start Tor proxying at boot:

[dom0]$ qvm-run-a torvm gnome-terminal

Now, we will createthe following script for starting up the Tor transparent proxy andsetting up traffic redirection using iptables:

[torvm]$ vim/rw/config/start_tor_proxy.sh

and now paste thefollowing into this file:

#!/bin/sh
killall tor
QUBES_IP=$(xenstore-readqubes_ip)
TOR_TRANS_PORT=9040

if [ X$QUBES_IP== X ]; then
echo"Error getting QUBES IP!"
echo"Not starting Tor, but setting the traffic redirection anyway toprevent leaks."
QUBES_IP="127.0.0.1"
else
/usr/bin/tor\
--SocksPort0 \
--TransListenAddress$QUBES_IP --TransPort $TOR_TRANS_PORT \
--DNSListenAddress$QUBES_IP --DNSPort 53 \
--RunAsDaemon1 --ControlPort 9051 \
|| echo"Error starting Tor!"

fi

echo “0” >/proc/sys/net/ipv4/ip_forward
/sbin/iptables-t nat -F
/sbin/iptables-t nat -A PREROUTING -i vif+ -p udp --dport 53 -j DNAT--to-destination $QUBES_IP:53
/sbin/iptables-t nat -A PREROUTING -i vif+ -p tcp -j DNAT --to-destination$QUBES_IP:$TOR_TRANS_PORT
/sbin/iptables-I INPUT 1 -i vif+ -p udp --dport 53 -j ACCEPT
/sbin/iptables-I INPUT 2 -i vif+ -p tcp --dport 9040 -j ACCEPT
/sbin/iptables-F FORWARD

echo “1” >/proc/sys/net/ipv4/ip_forward

Except for the“QUBES_IP=$(xenstore-readqubes_ip)” line that reads the torvm'sIP address, there is nothing Qubes-specific in the above listing.It's just a standard way of setting up transparent Tor proxy.

It is importantthat this file be located in the /rwdirectory, as this directory is backed by the VM's private storageand will survive VM reboots. The VM's root file-system is read-onlyand all the changes to it are lost on VM shutdown (VM gets anillusion of the root fs being writeable thanks to Copy-On-Writemechanism, but the actual COW backing device is cleared upon each VMshutdown).

We should alsomodify the /rw/config/rc.localscript, to ensure that our Tor proxy is automatically started -- justpaste the following into this script:

#!/bin/sh

# Uncommentthis if you would like to use a custom torrc file:
#rm -f/rw/config/log
#ln -sf/rw/config/torrc /etc/tor/torrc

chkconfigqubes_netwatcher off
chkconfigqubes_firewall off
/rw/config/start_tor_proxy.sh

Finally we shouldalso provide a script that would restart our proxy in case the userdynamically switched the NetVM, which would result in the completelydifferent routing. This could be done by creating a script withpredefined name qubes_ip_change_hookwithin /rw/config/directory:

#!/bin/sh
/rw/config/start_tor_proxy.sh

Make sure that all the scripts are executable (chmod +x). And that's all.Now, shutdown the torvm:

[dom0]$ qvm-run--shutdown --wait torvm

From now on, everytime you start the torvm (or when Qubes starts it in response tostart of some other VM that uses torvm as its Net VM), the Tortransparent proxy should be automatically started.

Let's test this bycreating a VM that would be using the just created Tor proxy:

[dom0]$qvm-create anon-web --label black

[dom0]$qvm-prefs -s anon-web netvm torvm

Now, every time youstart the anon-web VM (e.g. by clicking on the Web browser icon inthe anon-web's start menu), Qubes will also ensure that torvm is upand running, and this in turn would configure all the Tor proxyingfor this VM.

Fo additionalcontrol one might want to use Vidalia, the graphical front end forTor (this should be installed within the template VM that has beenused for torvm). We could easily start Vidalia by just typing:

[dom0]$ qvm-run-a torvm vidalia

We should howevermake sure to disable "Start the Tor software when vidaliastarts" option in Settings/General in Vidalia. Otherwise,Vidalia might kill your original Tor (that has transparent proxyopen) and start own without transparent proxy enabled.

The web browser runs in the 'anon-web' VM that uses 'torvm' for networking access, and thus all the traffic generated by 'anon-web' is routed through the Tor network, or discarded if it's a different traffic than TCP or DNS.

Of course one case easily create more VMs that would be using torvmas their Net VM, as so would have anonymized network access. Thebeauty of this solution is that in case one of my anonymized VM getscompromised, others do not. Plus, the already mentioned benefit, thatno matter whether apps in those VMs are buggy, or even intentionallymalicious, they would not be able to leak out the user's external IPaddress.

Creating a WiFipen-testing VM

Finally let's have some fun and create a WiFi pen-testing VM. Thedesired config is depicted below:

Because we would like to use all sorts of ~~l33t h4x0r t00lz~~pen-testing security software in this VM, it would make sense tocreate it as a StandaloneVM, which means thatit would get its own copy of the whole file-system (as opposed tojust the home directory, /rwand /usr/local,as it is the case with regular Qubes VMs). This would ease theinstallation of all the extra software we would need there, and alsoensure that even if the install/build scripts were malicious, thedamages would be contained only to this very VM and nothing else.Also, for some reason the standard Linux WiFi stack and drivers stilldon't support injection on (all?) most of the WiFi cards out of thebox, so we would need to patch the actual kernel drivers -- yetanother reason to use a Standalone VM in this case.

So, let's create the VM first, and assign a WiFi card to it:

[dom0]$qvm-create wififun --standalone --label yellow

[dom0]$qvm-prefs -s wififun memory 800 #ensure at least this mem at startup

[dom0]$qvm-prefs -s wififun kernel none #use own copy of kernel and modules

[dom0]$ qvm-pci-a wififun

Youcan easily find the BDF address of any device using the lspcicommand in Dom0 -- this would be something like e.g. “02:00.0”.You should make sure that this WiFi card is not used by any other VM,specifically by your default Net VM (called 'netvm' in a standardQubes installation). Ideally you could just use a dedicated ExpressCard-based WiFi card, leaving the built in WiFi assigned to yourdefault Net VM.

Because it's aStandalone VM, Qubes will make a copy of the whole root filesystem,and thus it would eat about 5GB of your disk (normal VMs would takeonly as much space as their private fs takes up).

Let's now start the VM...

[dom0]$ qvm-run-a wififun gnome-terminal

... and then install the prerequisite software there, starting withdownloading the reasonably new compat-wireless sources, together withthe required injection patches, and then building and installing thenew kernel modules. All actions below are now executed within the VM.This stuff here is really nothing Qubes- or Xen-specific -- one woulddo more or less the same on any Linux in order to get injectionworking (so, treat this as a free bonus WiFi hacking tutorial onLinux).

[wififun]$ wgethttp://linuxwireless.org/download/compat-wireless-2.6/compat-wireless-2011-07-14.tar.bz2

[wififun]$ wgethttp://patches.aircrack-ng.org/channel-negative-one-maxim.patch

[wififun]$wgethttp://patches.aircrack-ng.org/mac80211-2.6.29-fix-tx-ctl-no-ack-retry-count.patch

[wififun]$wgethttp://patches.aircrack-ng.org/mac80211.compat08082009.wl_frag+ack_v1.patch

[wififun]$ sudoyum install kernel-devel patch gcc

[wififun]$ tarxjf compat-wireless-2011-07-14.tar.bz2

[wififun]$ cdcompat-wireless-2011-07-14

[wififun]$patch -p1 < ../channel-negative-one-maxim.patch

[wififun]$patch -p1 < ../mac80211-2.6.29-fix-tx-ctl-no-ack-retry-count.patch

[wififun]$patch -p1 < ../mac80211.compat08082009.wl_frag+ack_v1.patch

[wififun]$ make

[wififun]$ sudomake unload

[wififun]$ sudomake install

Now, lets reboot the VM to ensure thatall the patched drivers will get properly loaded on each VM boot:

[dom0]$ qvm-run--shutdown --wait wififun

[dom0]$ qvm-run-a wififun gnome-terminal

Let's first see if the WiFi driver got properly loaded and if theinterface has been created (look for wlanXinterface):

[wififun]$ifconfig -a

If yes, then proceed with the steps below (if not, then have a lookinto dmesg and see what was the problem):

[wififun]$ sudobash

[wififun]# yuminstall aircrack-ng dnsmasq

[wififun]#airmon-ng start wlan0

[wififun]#iptables -F INPUT

[wififun]#iptables -F FORWARD

[wififun]# echo“1” > /proc/sys/net/ipv4/ip_forward

Note that you don't need to add anyexplicit masquerading rules, as they are applied by default on QubesVMs (you can take a look at the nat table in the VM if youwant to see by yourself).

Edit the /etc/dnsmasq.conf,so that it contains at least the following:

interface=at0

dhcp-range=192.168.0.50,192.168.0.150,12h

and then start the dnsmasq daemon -- wewill use it for providing DHCP to our fake AP (the at0 interface willbe created by airbase-ng and emulates the “uplink” of atraditional AP):

[wififun]#/etc/init.d/dnsmasq start

And finally the fake AP:

[wififun]#airbase-ng -e free_wifi mon0

and on another console (before anyclient connects, but after airbase-nggot started), configure the at0interface (make sure it matches what you wrote into dnsmasq.conf):

[wififun]#ifconfig at0 192.168.0.1 up

(you can also add an udev rule to thatautomatically).

and just to verify it really isworking:

[wififun]#tcpdump -i at0

... and now, just wait for a client toconnect to your AP. What you do next is only limited by yourimagination... But hey, this article is about Qubes networking andnot about 0wning client systems ;)

Here's an innocent example usingMoxie's sslstrip (amazing this attack still works so well at the endof 2011...):

My 'wififun' VM in action using a simple sslstrip attack, that surprisingly still works pretty nice...

Please note that as your wififun VM is a regular Qubes VM, it isautomatically connected to the default Net VM, which in turn providesnetworking to it. That's why it is so easy to create a fullyfunctioning fake AP.

When using custom driver domains, there are currently some catchesyou should be aware:

Catch #1: Whenyou start a driver domain lateafter system boot, so after some days of uptime and extensive use ofVMs, Xen might not be able to allocate enough continues (in terms ofMFNs) memory for a driver domain. And PV driver domains, unlikenormal domains or HVM driver domains, do require MFN-continuous memory for their DMA buffers (HVM domains do not need that, becauseIOMMU can create an illusion of this; even though IOMMU is also usedfor PV driver domains, for protection, it doesn't actively translatebus addresses into GMFNs).

This is usually not a bigproblem in practice, because in most cases all the driver domains arestarted early at system boot, when there is still plenty ofnon-fragmented memory available. However it might become a problemwhen one wishes to start e.g. the WiFi pen-testing at some latertime. The work around is to close as many VMs as possible beforestarting such driver domain, and then also reducing, for a moment,the amount of memory assigned to Dom0:

[dom0]$xm mem-set 0 1600m

andthen starting the driver domain should be fine. Now we can start allother domains, and that should no longer be problematic for thealready running driver domain.

Catch #2: Somenetwork cards, notably Express Cards, might not work well with the3.0.4 pvops kernel that we use in all VMs by default. In that caseyou might want to try to use the 2.6.38.3 xenlinux kernel in yourWiFi fun VM -- to do that, follow these steps:

[dom0]$sudo qvm-dom0-update kernel-qubes-vm-2.6.38.3-10.xenlinux.qubes

[dom0]$cp /var/lib/qubes/vm-kernels/2.6.38.3/*/var/lib/qubes/appvms/wififun/kernels/

[dom0]$qvm-prefs wififun -s kernelopts "swiotlb=force"

And then, in the VM:

[wififun]$ sudoyum install kernel-devel-2.6.38.3-10.xenlinux.qubes

And rebuild the compat-wireless,unload, install modules, and then load drivers again.

Summary

As you can see, Qubes Beta 2 now offers a very advanced networkinginfrastructure that allows more advanced users to create verysophisticated configurations, allowing for pretty good isolationbetween various domains and networks. Qubes leaves it up to the user(or admin) to figure out what would be the best configuration -- mostusers would be happy with the default simple setup with just one NetVM and one Firewall VM, while others would go for much more advancedsetups.

A bit more advanced networking setup. The usbvm has a 3G modem assigned, and it is possible to dynamically switch between the Net VMs without restarting any other VMs.

Qubes Beta 2 Released!

2011-09-19T12:52:00.000+02:00

I'm proud to announce that we have just released Qubes Beta 2! You can view installation instructions and download the ISO here.

We faced quite a few serious problems with this release that were caused by an upgrade to Xen 4.1 (from Xen 3.4) that we used in Beta 1. But finally we managed to solve all those problems and all in all I'm very happy with this release. It includes many performance optimizations compared to Beta 1 (CPU- and memory-wise) and also many bugfixes.

We also introduced a couple of new features:

Generic mechanism for inter-domain services with a centralized policy enforcement (more)

Network-less update mechanism for Dom0 (more)

VM management improvements: easy device assignment for driver domains, dynamic netvm switching, flexible VM kernel configuration, etc (see the new qvm-prefs utility)

Easy management of appmenus (shortcuts in the Start Menu)

Update to Xen 4.1 that offers, among other things, better VT-d support and more lightweight management stack (we have ported Qubes to use the new xl now, instead of the slow and heavy xend), and also to 2.6.38-xenlinux kernel for Dom0, and to 3.0.4 pvops kernel for VMs (better hardware compatibility, better power management)

I will write some more posts shortly that would present in detail some of the new features and what cool things one could do with them.

We have also created a dedicated wiki page that enumerates all the security-critical code for Qubes OS. We hope this page would be useful for security researchers that might attempt to find weaknesses in Qubes OS either in our code or in the 3rd party code that we rely on (Xen hypervisor, select Xen backends). Whether your motives are noble (gaining immortal fame, helping create a secure client OS), or not (proving ITL wrong), we would appreciate your efforts! And you might even get a job at ITL.

Speaking of which, I'm happy to announce that Marek Marczykowski, who has effectively become the key Qubes developer over the past few months, has now officially joined ITL :)

Anti Evil Maid

2011-09-07T23:56:00.004+02:00

Anti Evil Maid is an implementation of a TPM-based static trusted boot with a primary goal to prevent Evil Maid attacks.

The adjective trusted, in trusted boot, means that the goal of the mechanism is to somehow attest to a user that only desired (trusted) components have been loaded and executed during the system boot. It's a common mistake to confuse it with what is sometimes called secure boot, whose purpure is to prevent any unauthorized component from executing. Secure boot is problematic to implement in practice, because there must be a way to tell which components are authorized for execution. This might be done using digital signatures and some kind of CA infrastructure, but this gets us into problems such as who should run the CA, what should be the policy for issuing certificates, etc.

The adjective static means that the whole chain of trust is anchored in a special code that executes before all other code on the platform, and which is kept in a non re-flashable memory, whose sole purpure is to make the initial measurement of the next component that is going to be executed, which is the BIOS code. This special code, also known as Core Root of Trust for Measurement (CRTM), might be part of the BIOS (but kept on a special read-only memory, or implemented by some other entity that executes before the BIOS reset vector, such as e.g. Intel ME or the processor microcode even. Once measured, the BIOS code is executed, and it is now its turn to measures the platform configuration, Option ROM code, and MBR. Then the loader (stored in the MBR), such as Trusted GRUB, takes over and measures its own next stages (other than the MBR sector), and the hypervisor, kernel, and initramfs images that are to be loaded, together with their configuration (e.g. kernel arguments).

As explained above, trusted boot can only retrospectively tell the user whether correct (trusted) software has booted or not, but cannot prevent any software from executing. But how can it communicate anything reliably to the user, if it might have just been compromised? This is possible thanks to the TPM unseal operation that releases secrets to software only if correct software has booted (as indicated by correct hashes in select PCR registers).

So the idea is that if a user can see correct secret message (or perhaps a photo) being displayed on the screen, then it means that correct software must have booted, or otherwise the TPM would not release (unseal) the secret. Of course we assume the adversary had no other way to sniff this secret and couldn't simply hardcode it into the Evil Maid – more on this later.

Another way to look at it is to realize that Anti Evil Maid is all about authenticating machine to the user, as opposed to the usual case of authenticating the user to the machine/OS (login and password, decryption key, token, etc). We proceed with booting the machine and entering sensitive information, only after we get confidence it is still our trusted machine and not some compromised one.

Installing Anti Evil Maid

Anti Evil Maid should work for any Linux system that uses dracut/initramfs, which includes Qubes, Fedora and probably many other distros. You can find the Anti Evil Maid source code in a git repository here. You can also download a tarball with sources and prebuilt rpm packages from here (they all should be signed with the Qubes signing key). Qubes Beta 2, that is coming soon, will have those RPMs already per-installed.

To install Anti Evil Maid, follow the instructions in the README file.

Some Practical considerations

If you decided to use no password for your TPM SRK key (so, you passed '-z' to tpm_takeownership, see the README), then you should definitely install Anti Evil Maid on a removable USB stick. Otherwise, if you installed it on your disk boot partition, the attacker would be able to just boot your computer and note down the secret passphrase that will be displayed on the screen. Then the attacker can compromise your BIOS/MBR/kernel images however she likes, and just hardcode the secret passphrase to make it look like if your system was fine.

If you decided to use custom TPM SRK password (so, you did not pass -z to tpm_takeownership), then you can install Anti Evil Maid onto your regular boot partition. The attacker would not be able to see your secret passphrase without knowing the SRK password. Now, the attacker can try another Evil Maid attack to steal this password, but this attack is easy to spot and prevent (see the discussion in the next section).

However, there is still a good argument to install Anti Evil Maid on a separate USB stick rather than on your built-in disk boot partition. This is because you can use Anti Evil Maid as a provider of a keyfile to your LUKS disk encryption (as an additional file unsealable by the TPM). This way you could also stop adversary that is able to sniff your keystrokes (e.g. using hidden camera, or electromagnetic leak), and capture your disk decryption passphrase (see the discussion in the next section).

In any case it probably would be a good idea to make a backup stick that you might want to use in case you lose or somehow damage your primary stick. In that case you should have a way to figure out if your system has been compromised in the meantime or not. Use another stick, with another passphrase, and keep it in a vault for this occasion.

Finally, be aware that, depending on which PCRs you decided to seal your secrets to, you might be unable to see the secret even after you changed some minor thing in your BIOS config, such as e.g. the order of boot devices. Every time you change something in your system that affects the boot process, you would need to reseal your secrets to new PCR values as described in the installation instructions.

Attacks prevented by Anti Evil Maid

The classic Evil Maid attack is fully prevented.

If the attacker is able to steal your Anti Evil Maid stick, and the attacker gets access to your computer, then the attacker would be able to learn your secret passphrase by just booting from the stolen stick. This is not fatal, because user should get alarmed seeing that the stick has been stolen, and use the backup stick to verify the system (with a different secret messages, of course), and later create a new stick for every day use with a new secret message.

A variation of the above attack is when the attacker silently copies the content of the stick, so that the user cannot realize that someone got access to the stick. Attacker then uses the copied stick to boot the user's computer and this way can learn the secret passphrase. Now, the attacker can infect the computer with Evil Maid, and can also bypass Anti Evil Maid verification by just hardcoding the secret message into Evil Maid. So, even though TPM would know that incorrect software has booted, and even though it would not unseal the secret, the user would have no way of knowing this (as the secret would still be displayed on screen).

In order to protect against this attack, one might want to use a non-default SRK password – see the installation instructions. Now an extra SRK password would be needed to unseal any secret from the TPM (in addition to PCRs being correct). So the attacker, who doesn't know the SRK password, is now not able to see the secret message and cannot prepare the Evil Maid Attack (doesn't know what secret passphrase to hardcode there).

The attacker might want to perform an additional Evil Maid attack targeted at capturing this SRK password, e.g. by infecting the user's stick. This, however, could be immediately detected by the user, because the user would see that after entering the correct SRK password, there was no correct secret passphrase displayed. The user should then assume the stick got compromised together with the SRK password, and should start the machine from the backup stick, verify that the backup secret is correct, and then create new AEM stick for daily usage.

If an attacker is able to capture the user's keystrokes (hidden camera, electromagnetic leaks), the attacker doesn't need Evil Maid attack anymore, and so doesn't need to bother with compromising the system boot anymore. This is because the attacker can just sniff the disk decryption password, and then steal the laptop and will get full access to all user data.

In order to prevent such a “keystroke sniffing” attack, one can use an additional sealed secret on the Anti Evil Maid stick that would be used as a keyfile for LUKS (in addition to passphrase). In this case the knowledge of the sniffed LUKS passphrase would not be enough for the attacker to decrypt the disk. This has not been implemented, although would be a simple modification to dracut-antievilmaid module. If you decided to use this approach, don't forget to also create a backup passphrase that doesn't need a keyfile, so that you don't lock yourself from access to your data in case you lose your stick, or upgrade your BIOS, or something! You have been warned, anyway.

Attacks that are still possible

An adversary that is able to both: sniff your keystrokes (hidden camera, electromagnetic leak) and is also able to copy/steal/seize your Anti Evil Maid stick, can not be stopped. If a non-democratic government is your adversary, perhaps because you're a freedom fighter in one of those dark countries, then you likely cannot ignore this type of attacks. The only thing you can do, I think, is to use some kind of easy-to-destroy USB stick for keeping Anti Evil Maid. A digestible USB stick, anyone?

Another type of attack that is not addressed by Anti Evil Maid is an attack that works by removing the “gears” from your laptop (the motherboard and disk at the very least), putting there a fake board with a transmitter that connects back to the attacker's system via some radio link and proxies all the keyboard/screen events and USB ports back to the original “gears” that execute now under supervision of the attacker. Another way of thinking about this attack is as if we took the motherboard and disk away, but kept all the cables connecting them with the laptop's keyboard, screen, and other ports, such as USB (yes, very long cables). The attacker then waits until the user boots the machine, passes the machine-to-user authentications (however sophisticated it was), and finally enters the disk decryption key. In practice I wouldn't worry that much about such an attack, but just mentioning it here for completeness.

Finally, if our adversary is able to extract secret keys from the TPM somehow, e.g. using electron microscope, or via some secret backdoor in the TPM, or alternatively is able to install some hardware device on the motherboard that would be performing TPM reset without resetting the platform, then such an attacker would be able to install Evil Maid program and avoid its detection by SRTM. Still, this doesn't automatically give access to the user data, as the attacker would need to obtain the decryption key first (e.g. using Evil Maid attack).

Implementation Specific Attacks

In the discussion above we assumed that the trusted boot has been correctly implemented. This might not be true, especially in case of the BIOS. In that case we would be talking about attacks against a particular implementation of your BIOS (or TrustedGRUB), and not against Anti Evil Maid approach.

One typical problem might be related to how CRTM is implemented – if it is kept in a regular BIOS reflashable memory, than the attacker who can find a way to reflash the BIOS (which might be trivial in case your BIOS doesn't check digital signatures on updates) would be able to install Evil Maid in the BIOS but pretend that all hashes are correct, because the attacker controls the root of trust.

Another possible implementation problem might be similar to the attack we used some years ago to reflash a secure Intel BIOS (that verified digital signatures on updates) by presenting a malformed input to the BIOS that caused a buffer overflow and allowed to execute arbitrary code within the BIOS. For such an attack to work, however, the BIOS should not measure the input that is used as an attack vector. I think this was the situation with the logo picture that was used in our attack. Otherwise, even if there was a buffer overflow, the chain of trust would be broken and thus the attack detected. In other words, the possibility of such an attack seems to be rather slim in practice.

What about Intel TXT?

Intel TXT takes an alternative approach to trusted boot. It relies on a Dynamic instead of Static Root of Trust for Measurement (DRTM vs. SRTM), which is implemented by the SENTER instruction and special dynamic PCR registers that can be set to zero only by SENTER. Intel TXT doesn't rely anymore on the BIOS or CRTM. This offers a huge advantage that one doesn't need to trust the BIOS, nor the boot loader, and yet can still perform a trusted boot. Amazing, huh?

Unfortunately, this amazing property doesn't hold in practice. As we have demonstrated almost 3 years ago (!), it is not really true that Intel TXT can remove the BIOS away from the chain of trust. This is because Intel TXT is prone to attacks through a compromised SMM, and anybody who managed to compromise the BIOS would be trivially able to also compromise the SMM (because it is the BIOS that is supposed to provide the SMI handler).

Thus, if one compares SRTM with Intel TXT, then the conclusion is that Intel TXT cannot be more secure than SRTM. This is because if an attacker can compromise the BIOS, then the attacker can also bypass Intel TXT (via a SMM attack). On the other hand, a BIOS compromise alone doesn't automatically allow to bypass SRTM, as it has been discussed in a paragraph above.

It really is a pity, because otherwise Intel TXT would be just a great technology. Shame on you Intel, really!

Alternative approaches to mitigate Evil Maid Attacks

Various people suggested other methods to prevent Evil Maid attacks, so lets quickly recap and discuss some of them...

The most straight forward approach suggested by most people, has been to disable booting from external devices in BIOS, together with locking the BIOS setup with an admin password.

There are two problems with such an approach. First, all the BIOSes have a long history of so called default passwords (AKA maintenance passwords). You don't want to rely on the lack of BIOS default passwords when protecting your sensitive data, do you?

Second, even if your BIOS doesn't have a backdoor (maintenance password), it is still possible to just take your disk away and connect to another laptop and infect its boot partition.

Another suggested approach has been to keep your boot partition on a separate USB stick. This solution obviously doesn't take into account the fact that the attacker might install Evil Maid into your BIOS. Many consumer laptop BIOSes do not require digital signatures on BIOS firmware updates (my Sony Vaio Z, a rather high-end machine, is among them), making it simple to install Evil Maid there (the most trivial attack is to make the BIOS always boot from the HDD instead of whatever other device the user wanted to boot from).

Finally, some people pointed out that many modern laptops comes with SATA disks that offer ability to “lock” the disk so that it could only be used with a specific SATA controller. Using this, combined with setting your BIOS to only boot from your internal disk, plus locking access to BIOS setup, should provide reasonable protection. This solution, of course, doesn't solve the problem of a potential maintenance password in your BIOS. Also being skeptical and paranoid as I am, I would not trust this mechanism to be really robust – I would expect it would be fairly simple to unlock the disk so that it could be paired with another, unauthorized controller, and that this probably is a matter of NOP-ing a few instructions in the controller firmware... In fact it seems like you can buy software to unlock this mechanism for some $50... And apparently (and not very surprisingly) some drives seems to continue on the 'default passwords' tradition.

FAQ

Q: Bitlocker implemented this already several years ago, right?
A: No.

Q: But, two-factor authentication can also be used to prevent Evil Maid, right?
A: No.

Q: Does it make any sense to use Anti Evil Maid without a full disk encryption?
A: No.

Q: Are you going to answer 'no' for each question I ask?
A: No.

Q: Why there are no negative indicators (e.g. a big scary warning) when the unseal process fails?
A: The lack of negative indicators is intentional. The user should keep in mind that if somebody compromised their computer, then the attacker would be able to display whatever she wants on the screen, and especially to skip displaying of any warning messages. The only thing the attacker would not be able to display would be the secret message. Thus, it would make no sense to use negative indicators, as they would likely not work in case of a real attack. One solution here would be to use the unsealed secret as a keyfile for disk encryption (as discussed above), which would make it impossible to decrypt the user disk (and so generally proceed with the boot) without successfully unsealing the secret from the TPM.

Interview about Qubes OS

2011-08-30T23:06:00.001+02:00

Here is a recent interview with me for Tom's Hardware, where I talk about Qubes, why virtualization alone does not automatically bring much security, and why we need it for secure systems anyway, and all that kind of stuff. Nothing really new, but still might be of interest to some readers.

As for Qubes Beta 2 release -- it really is coming, but we've faced recently some very nasty, race-condition-related problems with new Xen (we bravely switched to Xen 4.1 in Beta 2) that seem to occur on machines with very fast SSDs and we're currently trying to see if we can solve them, or should we instead revert back to Xen 3.4 that we used previously in Beta 1. Except for that, Beta 2 is mostly ready, so we should be releasing it within coming weeks.

My SSTIC 2011 slides

2011-06-10T15:02:00.002+02:00

A few days ago I had a privilege to give an opening keynote at the SSTIC conference in Rennes, France, which is believed by many to be the most important security conference in France. You can find my slides here.

SSTIC seems to be a very interesting conference indeed, with a strong emphasis on system-level security, which is quite unusual these days where most conferences focus on networking, apps, and web-apps. What a pity all those interestingly-looking talks have been encoded in an obscure language used only by some 3% of the population of the planet...

Anyway, it was a pleasure to talk to some ANSSI people I met before the conference (one of the organizers of the event) who really seemed to understand well the challenges we face with building secure operating systems, and generally seemed well versed in the topic. Perhaps some other nations should learn from France, instead of proposing ridiculous and superficial means that can't really solve any real problem.

From Slides to Silicon in 3 years!

2011-06-03T17:16:00.003+02:00

Remember our Xen 0wning Trilogy at Black Hat in summer 2008, specifically the presentation on Detecting & Preventing the Xen Hypervisor Subversions?

One of the things we were discussing there was a proposal to include an additional restriction to Intel processors that would disallow execution of usermode pages from within supervisor mode (ring0). Such a feature, we argued, apart from obviously making many ring3-to-ring0 exploits much harder, including the very Xen heap overflow exploit we presented in the slides, would also bring us closer to efficient runtime code integrity checkers for kernels and hypervisors, as discussed in the slides.

Slide #97, Detecting and Preventing Xen Hypervisor Subversions, Black Hat USA, July, 2008

Fast forward 3 years. On June 1^st, 2011, an Intel engineer is submitting a patch for Xen to support a mysterious new processor feature called SMEP (Supervisor Mode Execution Protection). He writes the feature is not yet documented in SDM, but soon will be. In fact, the May 2011 update of Intel SDM already contains the details:

Intel SDM, vol. 3a, May 2011, source: intel.com

Some other people spotted this feature earlier, because of another patch submitted by another Intel engineer to Linux kernel a few weeks ago. Here's a good write up by Dan Rosenberg discussing how this patch makes writing Linux kernel exploits harder, and how it's still possible to write them.

The SMEP feature still doesn't seem to be present in the processors available on the market, including the latest Sand Bridge processors, but there's no question it's coming, now that the feature made it into SDM.

It is quite rewarding to see your idea implemented in a processor... I guess this is how physicists feel when they introduce a new particle as part of a new quantum model, and later discover evidences to support the existence of this very particle in the wild...

USB Security Challenges

2011-06-01T01:25:00.001+02:00

When we think about “USB Security” there are lots of things that come to mind...

First there are all the physical attacks that could be conducted with the help of USB devices. These are generally not so interesting, because if one includes physical attacks in the threat model, then it really opens up lots of possibilities of various attacks, and generally a physical attacker always wins. Still, there are a few very cheap and easy physical attacks that one would like to avoid, or make harder, such as the Evil Maid Attacks or the Cold Boot Attacks. Strictly speaking these are not problems inherent to USB itself, but rather with lack of Trusted Boot, or OS not cleaning properly secrets from memory upon shutdown. They are just made simple thanks to bootable USB sticks.

Much more interesting USB-related physical attacks are those that take advantage of the specifics of the USB standard. One example here would be a malicious USB device that exposes intentionally malformed info about itself in order to exploit a potential flaw in a USB Host Controller driver that processes this info upon each new USB device connect. Or a malicious USB device that would trick the OS (Windows at least) into downloading a known buggy USB driver (or even an intentionally malicious driver, legally submitted to WHQL by the attacker) and then exploit the driver.

Another class of physical attacks made possible by the USB specification are malicious USB devices that pretend to be a keyboard or mouse. The input devices, such as keyboard, are actually the most security sensitive devices, because an attacker who controls the keyboard can do everything the user can do, which basically means: can do everything, at least with regards to the user's data.

Finally, the USB, as the names stands, is a bus interconnect, which means all the USB devices sharing the same USB controller are capable of sniffing and spoofing signals on the bus. This is one of the key differences between USB and PCI Express standards, where the latter uses a peer-to-peer interconnect architecture.

Ok, so these all above were physical attacks. Let's now look at, much more fatal, software attacks.

The infamous class of attacks exploiting various autorun or auto-preview behaviors is the most known example, but also the easiest, at least in theory, to protect against.

Much more interesting are software attacks that attempt to exploit potential flaws in the USB stacks – similarly like the physical attacks mentioned above, just that this time not requiring any hardware-level modifications to the USB device. Exposing a malformed partition table is a great example of such an attack. Even if we have all the autorun mechanisms disabled, still, when we're inserting a storage medium the OS always attempts to parse the partition table in order to e.g. create devices symbolizing each partition/volume (e.g. /dev/sdbX devices).

Now, this is really a problematic attack, because the malformed partition table can be written onto a fully legitimate USB stick by malware. Imagine e.g. you have two physically separated machines (air-gapped), belonging to two different security domains, and you want to transfer files from one to another. You insert the USB stick into the first machine, copy files, and then insert the stick to the second machine. If the first machine was compromised, it could have altered the partition table on the USB stick, and now when this stick is inserted into the other machine its malformed partition table might exploit a buffer overflow in the code used by the second OS to parse the stick's partition information. Air-gapped systems, huh? We avoid this attack vector in Qubes by using a special inter-domain file copy mechanism that doesn't require any metadata parsing.

A variation of the above attack would be to expose a malicious file system metadata, but this time the target OS would have to actually mount the partition for the attack to work (and, of course, there would have to be bugs in the OS file system parsing code, although these seem to be quite common on most OSes).

Having quickly summarized the USB security-related threats, let's now think about how we could design an OS to mitigate most of those attacks, and at the very least the software-based attacks. This is, in fact, precisely the challenge we've been facing in Qubes, so the divagations below necessarily focus mostly on the Qubes architecture.

First we should realize that USB devices, unlike PCI Express devices, cannot be independently delegated to different domains (VMs). This is because IOMMU technologies, such as Intel VT-d, operate only on PCIe device granularity. This means we can only delegate a whole USB controller to a domain, including all of the USB devices connected to this controller/hub.

Imagine now two internal devices, both connected via internal USB bus: a keyboard, and a 3G wireless modem. Chances are high that you will have those two devices connected to the same USB controller – usually one controller is used for all the internal devices, like those I just mentioned, plus camera, fingerprint reader, etc, and the other controller is used for all the externally visible USB connectors (at least this is true for modern systems: Intel Series 5 chipsets and newer).

We would like to be able to delegate the 3G modem to the NetVM (an untrusted domain on Qubes where all the networking drivers and stacks are kept; it's considered untrusted because its compromise is equivalent to a compromise of a WiFi network or home router, or some other router, and any reasonable person always assumes that the network is compromised, and deals with that using crypto, such as SSL or SSH). But assigning the USB controller, to which the 3G modem is connected to, to the NetVM, would also assign the USB keyboard to the NetVM! And this is precisely what we don't want to do, because control over the keyboard is equivalent to the control over the whole system!

Currently, in Qubes Beta 1, we keep all the USB controllers assigned to Dom0. This, however, causes two annoyances:

First, the user cannot use any of the USB-connected networking devices, such as 3G modems (because there is no networking in Dom0).

Second, if somebody connects a USB disk and later delegates it to some domain (this could easily be done via block-attach mechanism, supported by the same backend that handles storage virtualization for domains), and this domain turns out to be compromised, it might alter e.g. the stick's partition table and later attack Dom0 as explained above.

We can eliminate the second problem by modifying the Dom0's kernel to not parse the partition table of any removable devices automatically, and instead expect some kind of explicit consent from user to actually do that (we still must allow to mount USB disks in Dom0 to allow easy backups of all domains at once).

To allow the use of USB-connected networking devices in NetVM, we could use a PVUSB backend that can virtualize single USB devices without moving the whole USB controller to the domain. But that would require introducing a whole lot of new code to Dom0 – code that would be directly reachable from VMs (in other words that would be processing lots of untrusted input coming from untrusted domains).

So another option is to delegate all the non-security-critical USB controllers, i.e. those controllers that don't have any security-sensitive USB devices connected, such as keyboard, to a dedicated “USB” domain, and later share the USB devices via PVUSB backend from this USB domain. This time, the extra PVUSB backend runs in the USB domain, not in Dom0, so we don't worry that much about potential bugs in this backend. Of course, this way you cannot delegate the USB controller to which the keyboard, and potentially also other security-critical devices, such as camera, are connected to, which in practice rules out the integrated3G modem. Fortunately many modern laptops do not use USB-connected keyboard and touchpad (they use PS/2-connected keyboards instead), and the face camera can be easily disabled with a piece of sticker (although that sucks, because it means we cannot really use the camera).

With this approach (a dedicated USB domain) you can now delegate your 3G modem to the NetVM, and other USB devices, such as removable disks to other domains, e.g. for file exchange. This seems the most reasonable setup, although it requires that either 1) your laptop doesn't have USB-connected keyboard, or 2) you don't use internal USB devices connected to the same controller that your USB keyboard/touchpad from other domains than Dom0 (in practice: no 3G modem in NetVM).

As we can see proper handling of USB devices is quite a challenge for OS architects. It might have been much less of a challenge if the engineers designing the USB, chipsets, and motherboards were a bit more security-conscious. Even such simple practice as never mixing security critical devices (keyboard, touchpad, camera, fingerprint reader), with non-security ones (3G modem), onto the same USB controller, would help tremendously. Or ability to somehow dynamically configure their connectivity, e.g. in BIOS?

(Un)Trusting the Cloud

2011-05-28T14:56:00.003+02:00

Everybody loves The Cloud these days, and it is not hard to understand why. When every person owns computers (devices), the cloud is really hard to beat when it comes to syncing all your digital life back and forth between all those devices, and also sharing with your family members, friends, and colleagues at work. From task lists, through calendars, through health & fitness data, to work-related documents. And I'm not even mentioning all the unencrypted email that is out there.

One doesn't need to be especially smart or security conscious to realize how much this might be a threat to security and privacy. How much easier would it be to attack somebody's laptop if I knew precisely in which hotel and when he or she is planning to stay? How much more expensive would my health and life insurance be, if they could get a look at my health and fitness progress? Etc.

But we're willing to sacrifice our privacy and security in exchange for easy of syncing and sharing of our data. We decide to trust The Cloud. What specifically does that mean?

First, it means we trust the particular cloud-based service vendor, such as the provides of our training monitoring app and service. We trust that this vendor is: 1) non-malicious and ethical, and so is not going to sell our private data to some other entity, e.g. insurance company, and 2) that the software written by this vendor is somehow secure, so it would not be easy for an attacker to break into their cloud service and download all the user's data (and then sell to health insurance companies).

Next, we trust the cloud infrastructure provider, such as Amazon EC2. We trust that the cloud provider is 1) non-malicious and ethical, and that they won't really read the memory of the virtual machine on which the previously mentioned cloud-service is running (and won't make it available to a local government officials, e.g. in China), and 2) that they secured their infrastructure properly (e.g. it wouldn't be easy for one customer to “escape” from a VM and read all the memory of the VMs belonging to other customers).

Finally we trust all the infrastructure that is in the middle between us and the service provider, such as e.g. the networking protocols, are safe to use (e.g. we trust all the engineers working in any of the ISP we use won't sniff/spoof our communication, e.g. by using some fake or quasi-fake SSL certs).

So, that's a hell of a lot of trusting! And the stake is high. Do we really need to make such a sacrifice? Do we really need to hand in all our private data to all those organizations? Of course we don't!

First, notice that in majority of cases, the cloud is only used basically as a on-line storage. No processing, just dump storage. Indeed, what kind of server-side processing does your task list or calender require? Or your freestyle swimming results? Or your conference slides? None.

And we know for very long how to safely keep secrets on untrusted storage, don't we? This is achieved via encryption (and digital signatures for integrity/authenticity). So, the idea is very simple: let's encrypt all the data before we send them to the cloud. The point here is, the encryption must be done by the app that is running on our client device. Not in the cloud, of course.

Ok, so let's say I have my calendar records encrypted in the cloud, how do I share it with my other devices and other people, such as my partner and colleagues at work? Very simple – you encrypt each record with a random symmetric key and then, for every other device or person who you want to grant access to your calendar you make the symmetric key available to this person, by encrypting it with their public key (if you're paranoid, you can even verify fingerprints using some out-band communication channel, such as phone, to ensure the cloud/service provider didn't do MITM attack on you). What if you want to share only some events (or some details) with some group of people (e.g. only your availability info)? Very simple – just encrypt those records you want to share in non-full access with some other symmetric key and publish only this key to those people/devices you want to grant such non-full access.

Implementing the above would require writing new end-user apps, or plugins for existing apps (such as Outlook), so that they do encryption/decryption/signing/verification before sending the data out to the cloud. But what stops the malicious vendor from offering apps that would be leaking out our secrets, e.g. the keys? Well, nothing actually. But this time, the vendor would need to explicitly build in some kind of backdoor into the app. The same could be done with any other vendor, and any other, non-cloud-based app. After all, how do we know that MS Word, which is not cloud-based yet, is not sending out fragments of our texts to Agent Smith? Note how different this is from a situation when the vendor already owns all our data, unencrypted, brought legitimately to their servers, and all they need to do is to read them from their own disks. No need to plant and distribute any backdoors!

In practice few vendors would be risking their reputation and would be willing to build in a backdoor into an app that is then made available to customers. Because every backdoor in such client-exposed code will sooner or later be found (You would really not believe what great lengths all those young people aimed with disassembler and debugger would go to, to win an economy class ticket to the middle of desert in the hottest summer season, just to be able to deliver a presentation on how evil/stupid a company X is ;).

One problem is, however, with accessing our encrypted cloud over a Web Browser. In contrast to apps, the web browser content is much less identifiable. An app can have a digital signature – everybody know its an App v 1.1, published by X. As explained above it would be rather stupid for X to plant a backdoor into such an app. But a Web-delivered Javascript is much more tentative, and it's very possible for X to e.g. deliver various versions of scripts to different customers. Digital signature on client-side scripts, paired with ability to whitelist allowed client-side-scripts, would likely solve this problem.

So, why we still haven't got client-side-encrypted cloud-services? The question is rhetorical, of course. Most vendors actually loves the idea of having unlimited access to their customers data. Do you think Google would be happy to give up an opportunity to data mine all your data? This might affect their ad business, health research, or just Secret Plan To 0wn The World. After our dead body, I can almost hear them yelling! After all they have just came up with Chrome OS to bring even more data into their data mining machine...

To sum it up, there is no technical reason we must entrust all those people with our most private data. Sooner or later somebody will start selling client-side-encrypted cloud services, and I would be the first person to sign up for it. Hopefully it will happen sooner than later (to late?).

This post also hopefully shows, again, one more aspect – that we can, relatively easy, move most of the IT infrastructure out of the “TCB” (Trusted Computing Base, used as metaphor here). In other words, we can design our systems and services so that we don't need to trust a whole lot of things, including servers and the networking infrastructure (except for its reliability, but not for its security). But, there always remains one element that we must trust – these are our client devices. If they are compromised, the attacker can steal everything.

Strangely most people still don't get it, or get it backwards. Just the fact that “information is not stored on the iPad but kept safe on the corporate network”, doesn't change anything! Really. If the attacker owns your iPad, then she also can do anything that the legitimate user could do from this iPad. So if you could get to the company's secret trade data from your iPad's Receiver, so would be able to do the malware/attacker.

The App-oriented UI Model and its Security Implications

2011-05-21T20:17:00.001+02:00

Most of the desktop OSes today, such as Windows or Mac, expose and encourage a File-oriented UI model. You pick a file in the file manager, click it, and then the file manager automagically determines the best app to handle the file, starts the app, and passes the file to it.

Back in the MS-DOS days we used a different model: an app-oriented model – you started an app first, e.g. Word Perfect, or Lotus 1-2-3, and then you opened a file from within the app (Norton Commander and similar programs somehow changed that later).

Interestingly this very same app-oriented model is now becoming popular again thanks to systems such as iOS and Android. There is no such thing as a global File Explorer or Finder on an iPad. Only the apps. One must first pick an app, and then it's the application's responsibility to expose an option for opening one of your “files”, if the app supports it (e.g. the calendar or task list apps would always open your default calendar or task list without asking for anything).

I actually like this app-oriented model a lot! It's much less confusing to the user. Just think about all those attacks in the past where an attacker could prepare a file with some innocently-looking extension but which in fact was an MZ executable. Or how many times people are not even aware which app they use! One might argue that user should not be distracted by such “unimportant” things as what app he or she uses for her work, but I disagree. Apparently Apple, and millions of iPhone and iPad users, disagree too.

But the main reason why I like this app-oriented model is because it just fits greatly into the Security by Isolation philosophy.

Just think about it: if it's possible to get users to consciously select an app, and we now know it is possible thanks to the millions of app-oriented devices sold, then it should be not much more difficult to get them to also consciously select the domain or area, such as “work”, or “personal”, which they wish to use. Just imagine that instead of one “Mail” app, you would have two apps (and two icons): “Mail Work”, and “Mail Personal”.

There are some technicalities here – such as e.g. how to isolate apps between each other? Do we need to build another layer of isolation in a form of VMs to isolate “Mail Work” from “Mail Personal”, or should the (new) OSes and the (new) APIs be designed in such a way, that they were thin and secure, and allow for very good isolation between processes without using virtualization?

In Qubes we must use this additional layer of abstraction (virtualization), because we want to use Linux apps (and in the future also Windows apps), and they require huge POSIX/X API (and Win32 API) to work correctly. And those APIs are not easily isolate-able. So we use VMs as “API providers”. Same with isolating networking drivers and stacks – we need Linux kernel API to get those drivers and stacks running, so that's why we use a Linux-based “NetVM” for isolating networking. For this reason we expect users to explicitly define domains, such as “work”, “personal”, etc. This is because we cannot afford to run every single app in a separate AppVM (more precisely we cannot afford to create a working copy of this huge POSIX/X API for each app).

But we could very well imagine a well constructed API for apps that would just be easily isolate-able (I'm not saying iOS or Android has such an API), and so there would be no need to define domains explicitly. Still, we would need a possibility to define more than one instance of each app – such as the previously mentioned “Mail Work” and “Mail Personal”.

The app-oriented model seems to be the future. And so seems the Security by Isolation philosophy!

Following the White Rabbit: Software Attacks Against Intel VT-d

2011-05-13T19:04:00.001+02:00

Today we publish a new paper which is a result of our several month long in-depth evaluation of Intel VT-d technology. To quote the abstract:

We discuss three software attacks that might allow for escaping from a VT-d-protected driver domain in a virtualization system. We then focus on one of those attacks, and demonstrate practical and reliable code execution exploit against a Xen system. Finally, we discuss how new hardware from Intel offers a potential for protection against our attacks in the form of Interrupt Remapping (for client systems available only on the very latest Sandy Bridge processors). But we also discuss how this protection could be circumvented on a Xen system under certain circumstances...

I think the attack is likely the most complex and surprising out of all the things we have presented so far. Parts of it are even funny (if you share our weird sense of humor), such as the use of ICMP ping to generate MSIs. The paper also covers the vendors' response. You can download the paper here.

The Linux Security Circus: On GUI isolation

2011-04-23T16:52:00.008+02:00

There certainly is one thing that most Linux users don't realize about their Linux systems... this is the lack of GUI-level isolation, and how it essentially nullifies all the desktop security. I wrote about it a few times, I spoke about it a few times, yet I still come across people who don't realize it all the time.

So, let me stress this one more time: if you have two GUI applications, e.g. an OpenOffice Word Processor, and a stupid Tetris game, both of which granted access to your screen (your X server), then there is no isolation between those two apps. Even if they run as different user accounts! Even if they are somehow sandboxed by SELinux or whatever! None, zero, null, nil!

The X server architecture, designed long time ago by some happy hippies who just thought all the ~~people~~ apps are good and non-malicious, simply allows any GUI application to control any other one. No bugs, no exploits, no tricks, are required. This is all by design. One application can sniff or inject keystrokes to another one, can take snapshots of the screen occupied by windows belonging to another one, etc.

If you don't believe me, I suggest you do a simple experiment. Open a terminal window, as normal user, and run xinput list, which is a standard diagnostic program for Xorg (on Fedora you will likely need to install it first: yum install xorg-x11-apps):

$ xinput list

It will show you all the pointer and keyboard devices that your Xorg knows about. Note the ID of the device listed as “AT keyboard” and then run (as normal user!):

$ xinput test id

It should now start displaying the scancodes for all the keys you press on the keyboard. If it doesn't, it means you used a wrong device ID.

Now, for the best, start another terminal window, and switch to root (e.g. using su, or sudo). Notice how the xinput running as user is able to sniff all your keystrokes, including root password (for su), and then all the keystrokes you enter in your root session. Start some GUI app as root, or as different user, again notice how your xinput can sniff all the keystrokes you enter to this other app!

Yes, I can understand what is happening in your mind and heart right now... Don't worry, others have also passed through it. Feel free to hate me, throw out insults at me, etc. I don't mind, really (I just won't moderate them). When you calm down, continue reading.

In Qubes the above problem doesn't exist, because each domain (each AppVM) has it own local, isolated, dummy X server. The main X server, that runs in Dom0 and that handles the real display is never exposed to any of the AppVMs directly (AppVMs cannot connect to it via the X protocol). For details see this technical overview.

You can repeat the same experiment in Qubes. You just need to use the ID of the “qubesdev” device, as shown by xinput list (should be 7). Run the xinput in one of your domains, e.g. in the “red” one. Because we actually use the same device for both mouse and keystrokes, you should now see both the key scancodes, as well as all the mouse events. Notice how your xinput is able to sniff all the events that are destined for other apps belonging to the same domain where you run xinput, and how it is unable to sniff anything targeted to other domains, or Dom0.

BTW, Windows is the only one mainstream OS I'm aware of, that actually attempts to implement some form of GUI-level isolation, starting from Windows Vista. See e.g. this ancient article I wrote in the days when I used Vista at my primary laptop. Of course, it's still easy to bypass this isolation, because of the huge interface that is exposed to each GUI client (that also includes GPU API). Nevertheless, they at least attempt to prevent this at the architecture level.

Why the US "password revolution" won't work

2011-04-16T14:29:00.001+02:00

So, I've been reading this article this morning on how the US "private and public" institutions are going to revolutionize the way we authenticate on the web. The "ground breaking" idea, also illustrated on this NIST animation, is to use 3rd party authorities that would first verify your identity somehow ("Can we see your id?", "What is your Mam's maiden name?", etc), and then would issue you some kind of a token that you would later use for authentication on the web. A token would be e.g. a smart card, or a USB stick (probably they just mean a smart card with USB connector, whatever), or even a "phone application".

The idea is that the user will not have to "remember" all those passwords for all the various websites, which apparently is a problem in practice, because most users never heard about password manager apps, and so they actually try to remember all those passwords, or even try to use the same one all over the place. Using one password for more than one website is obviously wrong and people should be told not to do that. But an easy way to solve this is to just get people to use password managers.

But the key problem that they try to solve, which is identity theft, is just not gonna be solved by this "password revolution". This is because if somebody has compromised my laptop, then it really doesn't matter if I use passwords, or smart cards, or whatever other multi-factor authentication mechanism -- none of them will help if the attacker controls my operating system.

Most people cannot just get it -- this is because they lack understanding of how computers and operating systems work. They don't understand that the operating system can impersonate the user at will! This is because the operating system fully controls the keyboard, the mouse, and the screen.

So, imagine you use your super-secure smart card token for authentication to your bank. So, before you log into your bank account, and perhaps before you make any transaction on the banking website, you must insert your smart card somewhere (e.g. into smart card reader, or into USB port, etc). Before you insert your token, no one can impersonate you on the bank website. So far, so good! But then, once you inserted your token, it's all lost! The compromised OS could have saved your PIN to this card when you used it previously (even if you configured it not to do so!) and now, immediately, it could use the inserted card to authenticate as you to the bank and start issuing transactions on your behalf. And you won't even notice this all, because in the meantime it will show you a faked screen of your banking account. After all, it fully controls the screen.

The bottom line is that we cannot secure our digital lives, if our client operating systems could not be secured first. And today, the operating systems we use on our laptops, such as Windows, or Mac, or Ubuntu, are just trivial to be compromised by the attackers. After all, if that wasn't true we wouldn't have all those problems with identity theft. But introduction of tokens won't make our operating systems any more secure!

What we need instead are technologies that allow to build next-generation trusted operating systems. Technologies such as Intel TXT or VT-d. And we need OS vendors to actually start using them.

You can say I'm biased, because of our work on Qubes OS. But then, consider this -- perhaps we would never invest so much money and resources into this project, if we believed there are other ways to bring security to our digital life.

Qubes Beta 1 has been released!

2011-04-12T08:49:00.001+02:00

I'm very proud to announce that we have just released Qubes Beta 1! Some new features that have come into this release include:

Installer (finally!),

Improved template sharing mechanism: service VMs can now be based on a common template, and you can now easily create many net- and proxy- VMs; template upgrades now don't require shutting down all the VMs;

Standalone VMs, convenient for development, as well as for installing the least trusted software,

Built in, easy to use firewall VM(s),

Seamless integration of virtualized tray icons (check the screen shots!)

Redesigned file-copy between domains (easier, more secure),

Default template based on Fedora 14 (x64)

Reasonably complete User Guide.

... and many other improvements and bug fixes!

To download the installation ISO go to this page.

You can also install Qubes on an external USB disk - this might be a convenient option if you want to just try it out, without the need to "sacrifice" your laptop.

This release is very stable, but we feel that it still requires some more polish, mostly with regards to user interface. We're planning to release at least one more beta, in about 2 months, where we will focus mostly on UI improvements, and also on upgrading Xen and kernel in Dom0 to allow for better hardware support.

The final Qubes 1.0 is planned after the summer holidays. Once we reach this milestone, further work will likely fork into two branches:

The "commercial branch" which will focus on adding various extensions on top of Qubes 1. One specific commercial extension that we think would be a killer is support for Windows-based domains (AppVMs),

The "open source branch" that will continue on implementing even more revolutionary architecture and features, such as untrusted storage domains, safe GPU multiplexing, trusted boot, etc. In the end this should lead to Qubes 2.0 sometime in 2012 or 2013.

Cross your fingers!

Partitioning my digital life into security domains

2011-03-13T12:11:00.007+01:00

The diagram below illustrates how I have decomposed my digital life into security domains. This is a quite sophisticated scheme and most users would probably want something simpler, but I think it's still interesting to discuss it. The domains are implemented as lightweight AppVMs on Qubes OS. The diagram also shows what type of networking connectivity each domain is allowed.

Let's discuss this diagram bit by bit. The three basic domains are work (the green label), personal (the yellow label), and red (for doing all the untrusted, insensitive things) – these are marked on the diagram with bold frames.

A quick digression on domain labels (colors) – in Qubes OS each domain, apart form having a distinct name, is also assigned a label, which basically is one of the several per-defined colors. These colors, which are used for drawing window decorations by the trusted Window Manager (color frames), are supposed to be user friendly, easy noticeable, indicators of how trusted a given window is. It's totally up to the user how he or she interprets these colors. For me, it has been somehow obvious to associate the red color with something that is untrusted and dangerous (the “red light” -- stop! danger!), green with something that is safe and trusted, while yellow and orange with something in the middle. I have also extended this scheme, to also include blue, and black, which I interpret as indicating progressively more trusted domains than green, with black being something ultimately trusted.

Back to my domains: the work domain is where I have access to my work email, where I keep my work PGP keys, where I prepare reports, slides, papers, etc. I also keep various contracts and NDAs here (yes, these are PDFs, but received from trusted parties via encrypted and signed email – otherwise I open them in Disposable VMs). The work domain has only network access to my work email server (SMTP/IMAP4 over SSL), and nothing more.

For other work-related tasks that require some Web access, such as editing Qubes Wiki, or accepting LinkedIn invites, or downloading cool pictures from fotolia.com for my presentations, or specs and manuals from intel.com, for all this I use work-pub domain, which I have assigned the yellow label, meaning I consider it only somehow trusted, and I would certainly never put my PGP keys there, or any work-related confidential information.

The personal domain is where all my non-work related stuff, such as personal email and calendar, holiday photos, videos, etc, are held. It doesn't really have access to the Web, but if I was into social networking I would then probably allow HTTPS to something like Facebook.

Being somehow on the paranoid side, I also have a special very-personal domain, which I use for the communication with my partner when I'm away from home. We use PGP, of course, and I have a separate PGP keys for this purpose. While we don't discuss any secret and sensitive stuff there, we still prefer to keep our intimate conversations private.

I use shopping for accessing all the internet e-commerce sites. Basically what defines this domain is access to my credit card numbers and my personal address (for shipping). Because I don't really have a dedicated “corporate” credit card, I do all the shopping in this domain, from groceries, through sports equipment, on hotel/plane reservations ending. If I had separate business credit cards, then I would probably split my shopping domain into personal-shopping and work-shopping. I also have banking domain, which I use only for managing my bank account (which again combines both my personal and company accounts).

I also have a few specialized work-related domains, that I rarely use. The work-admin domain is used to manage almost all of the ITL servers, such as our webserver, Qubes repo & wiki servers, email server and DNS management, etc. This domain is allowed only SSH traffic to those server, and HTTPS to a few Web-based management servers. The work-blog is used to manage this very blog you're reading now. The reason why it is separate from work-admin or work, is because I'm over paranoid, and because I fear that if somebody compromises the blogger service, and subsequently exploits a bug in my browser that I use for editing my blog, than I don't want this person to be able to also get admin access to all the ITL servers. Similarly, if somebody somehow compromised e.g. the Amazon Web Management Console, and then exploited my browser in work-admin, then I would like at least to retain access to my blog. If I used twitter, I would probably also manage it from this work-blog domain, unless it was a personal twitter account, in which case I would run it from personal.

The qubes-dev domain is used for all the Qubes development, merging other developers' branches (after I verify signatures on their tags, of course!), building RPMs/ISOs (yes, Qubes Beta 1 will ship as a DVD ISO!), and finally signing them. Because the signing keys are there, this domain is very sensitive. This domain is allowed only SSH network access to our Qubes git server. Again, even if somebody compromised this Git server, it still would not be a big problem for us, because we sign and verify all the tags in each others repos (unless somebody could also modify the SSH/Git daemons running there so that they subsequently exploit a hypothetical bug in my git client software when I connect to the server).

I also decided to keep all the accounting-related stuff in a separate domain – whenever I get an invoice I copy it to the accounting domain. The rationale for this is that I trust those PDFs much less than I trust the PDFs I keep in my work domain.

Once a year I move the old stuff from my work domain, such as old email spools, old contracts and NDAs, to the work-archives domain. This is to minimize the potential impact of the potential attack on my work domain (my work domain could be attacked e.g. by exploiting a hypothetical bug in Thunderbird's protocol handshake using a MITM attack, or a hypothetical bug in GPG).

The vault domain is an ultimately trusted one where I generate and keep all my passwords (using keepass) and master GPG keys. Of course, this vault domain has no networking access. Most of those passwords, such as the email server access password is also kept in the specific domains which uses them, such as the work domain, and more specifically in the Thunderbird client (there is absolutely no point in not allowing e.g. Thunderbird to remember the password – if it got compromised it would just steal it the next time I manually enter it)

And finally, there is the previously mentioned red domain (I have tried to call it junk or random in the past, but I think red is still a better name after all). The red domain is totally untrusted – if it gets compromised, I don't care – I would just recreate it within seconds. I don't even back it up! Basically I do there everything that doesn't fit into other domains, and which doesn't require me to provide any sensitive information. I don't differentiate between work-related and personal-related surfing even – I don't care about anonymity for all those tasks I do there. If I was concerned about anonymity, I would create a separate anonymous domain, and proxy all the traffic through a tor proxy from there.

Now, this all looks nice and easy ;) but there is one thing that complicates the above picture...

Data flows between the domains

The diagram below shows the same domains, but additionally with arrows symbolizing typical data flows between them.

You can see that most of the usual data flows are from more trusted domains to less trusted domains – e.g. copy and pasting a URL that I receive via email in my work domain, so that I could open it in my untrusted browser in red, or moving invoices from my work domain (where I receive them via email) to the accounting domain.

But there are, unfortunately, also some transfers from less trusted domains to more trusted ones. One example is copy and pasting an interesting URL that I just stumbled upon when surfing in the red domain, and that I would like to share with a college at work, or a friend, and so I need to copy and paste it to my email client in either work (colleague) or personal (friend) domain.

Now, copying data from less trusted domains to more trusted ones presents a significant problem. While one could think that pasting an URL into Thunderbird email editor is a pretty harmless operation, it's still is an untrusted input – and we don't really know what the red domain really pasted into its local clipboard, and so what we will paste into the work domain's Thunderbird email editor (perhaps 64kB of some junk that will overflow some undo buffer in the editor?). And even more scary is the example with copying the cool-looking graphics file from the Web into my work domain so that I could use it in my presentation slides (e.g. an xkcd . Attacks originating through malicious JPEGs or other graphics format, and exploiting bugs in rendering code have been known for more than a decade.

But this problem – how to handle data flows from less trusted systems to more trusted ones – is not easily solvable in practice, unfortunately...

Some people who design and build high-security systems for use by military and government takes a somehow opposite approach – they say they are not concerned about less-trusted-to-more-trusted data transfers as long as they could assure there is no way to perform a transfer in the opposite direction. So, if we could build a system that guarantees that a more trusted domain can never transfer data to a less trusted domain (even if both of those domains are compromised!), then they are happy to allow one-way “up transfers”. In practice this means we need to eliminate all the covert channels between two cooperating domains. The word cooperating is a key word here, and which makes this whole idea not practical at all, IMHO.

Elimination of the covert channels between cooperating domains is indeed required in this scheme, because the assumption is that the data transfer from the less trusted domain could have indeed compromised the more trusted domain. But this, at least, should not result in any data leak back to the originating domain, and later to the less-classified network, which this less-trusted domain is presumably connected to. One of the assumptions here is that the user of such a system is connected to more than one, isolated networks. Even in that case, elimination of all the covert channels between domains (or at least minimizing their bandwith to something unusable – what is unusable, really?) is a big challenge, and can probably only could be done when we're ready to significantly sacrifice the system's performance (smart scheduling tricks are needed to minimize temporal covert channels).

I would like to make it clear that we are not interested in eliminating cooperative covert channels between domains in Qubes any time in the near future, and perhaps in the long term as well. I just don't believe into such approach, and I also don't like that this approach does nothing to preserve the integrity of the more-trusted domain – it only focuses on the isolation aspect. So, perhaps the attacker might not be able to leak secrets back to the less trusted domain, but he or she can do everything else in this more trusted domain. What good is isolation, if we don't maintain integrity?

An alternative solution to handling the less-trusted-to-more-trusted data transfers, is to have trusted “converters” or “verifiers” that could handle specific file types, such as JPEGs, and ensure we get a non-malicious file in the destination domain. While this might remind the bad-old A/V technology, it is something different. Here, the trusted converters would likely be some programs written in a safe language, running in another trusted domain, rather than a big ugly A/V with a huge database of signatures of “bad” patterns of what might appear in a JPEG file. The obvious problem with such an approach is that somebody must write those converters, and write them for all file types that we wish to allow to be transferred to more trusted domains. Perhaps doable in the longer-term, and perhaps we will do it in some future version of Qubes...

Right now we are ignoring this problem, and we say that all less-trusted-to-more-trusted transfers are to be done on the user's own risk :) You're welcome to submit trusted converters for your favorite file type(s) in the meantime!

Copying files between domains

Speaking of copying files between domains, there is another security catch here. If we imagined two physically separated machines that share no common network resources, the only way to move files between those two air-gaped machines would be via something like a USB stick or a CDROM or DVD disc. But inserting a USB drive or CDROM into a machine triggers a whole lot of actions: from parsing device-provided information, loading required drivers (for USB), parsing the driver's partition table, mounting and finally parsing the filesystem. Each of this stage requires the machine's OS to perform a lot of untrusted input processing, and the potential attack space here is quite large. So, even if we could limit ourselves to copy only harmless files between machines/domains (perhaps they were somehow verified by a trusted party in-between, as discussed above), still there is a huge opportunity that the originating domain could compromise the target domain.

In Qubes Alpha we have been using a similar file copy mechanism, using a virtual stick for file copy between domains. In Qubes Beta 1 we will provide a new scheme based on same shared memory channel that we use for GUI virtualization – the technical details of this solution will be available soon in our wiki. The most sensitive element in this new scheme is the un-cpio-like utility that runs in the target domain and unpacks the incoming blob into the pre-defined directory tree (e.g. /home/user/incoming/from-{domainname}/). We believe we can write pretty safe un-cpio-like utility, in contrast to secure all the previously mentioned elements (USB device parsing, partition parsing, fs parsing). The Qubes Beta 1 is planned to be released at the end of March, BTW.

Partitioning enforcement and easy of use

For any security partitioning scheme to make sense in real life, it is necessary to have some enforcement mechanism that would ensure that the user doesn't mistakenly bypass it. Specifically for this purpose we have come up with special, previously-mentioned firewalling support in Qubes Beta 1, that I will cover in a separate article soon.

Anther thing is to make the partitioning easy to use. For instance, I would like to be able to setup a hint in the policy, that when I click on an URL in an email I received in my work domain that it should be automatically opened in the red domain's default Web browser. Currently we don't do that in Qubes, but we're thinking about doing it in the near future.

Summary

Partitioning one's digital life into security domains is certainly not an easy process and requires some thinking. This process is also very user-specific. The partitioning scheme that I've come up for myself is quite sophisticated, and most people would probably want something much simpler. In case of corporate deployments, the scheme would be designed by CIO or IT admins, and enforced on users automatically. Much bigger problem are home and small business users, who would need to come up with the partitioning themselves. Perhaps in future versions of Qubes we will provide some ready to use templates for select "typical" groups of users.

My documents got lost/stolen [offtopic]

2011-03-08T14:40:00.007+01:00

I just realized yesterday that my wallet has disappeared, with all my credit cards, Polish ID card, and driver's license inside. Most likely somebody stole it. No strange transactions have been observed on my credit card accounts yet, but these are generally not much of a concern, thanks to credit card insurance. What is more troubling, is that perhaps some other woman is currently using my stolen ID and the driver's license doing nasty things on my account.

Apparently there is little one can do in Poland (EU?) in order to invalidate a stolen ID card. While there is an inter-bank Polish-wide database of stolen ID cards, it is being used only by banks, so it can only prevent other people from applying for small loans (for bigger loans, one would need more documents). But there are so many other things one could do, such as renting a car (and then committing a crime with it), signing up a deal with a mobile carrier (and then committing a cyber crime using this phone, or just making a really huge bill), or perhaps buying an SSL cert...

With apparently no better option left, I decided to write this blog post -- hopefully somebody will find it, e.g. before issuing a Class 2 SSL cert to the fake Joanna Rutkowska.

Here are the numbers of my lost/stolen documents:

AFS739530
**********5058

Luckily I have had my ID details written down somewhere, and the driver's license number I extracted from my Hertz profile.

A scene at a police department in Warsaw:

Hi, I would like to report my wallet being lost or stolen...
Madam, was your wallet stolen, or have you lost it?
Officer, how could I possibly know this...? If I lost it, do you really think I would remember the very moment of losing it?
Madam, you must be sure whether it was a crime or not!
...

A scene on the hotline, calling my mobile provider (note that I decided to use the word stolen this time):

Hi, my documents have been stolen -- I would like that you indicate my ID card as invalid in your system (that you hopefully share with other telcom operators)..
You should report such an incident to the police, Madam...
Right, but I guess that neither you, nor any other mobile provides in Poland will consult a Police database before signing up a contract with a strange person who might be using my stolen documents, correct?
Oh, but we will not sign a contract with a strange person who uses your documents! Only with you!
And how would you know it was not me, if that person was similarly aged and looking, and was using my stolen ID and driver's license?
...

Update on Qubes

2010-12-06T18:23:00.004+01:00

It's been a bit quiet on the Qubes development front for the last 2 months. The reason for this was that Rafal and myself got fully engaged in a new commercial research project. After all, we do need to make money somehow, so that we could later spend them on funding Qubes development :)

But this new engagement is actually closely related to what we do with Qubes (i.e. how new hardware technologies allow to build more secure OSes), so it's not like we're abandoning Qubes, as the experience we get with this research project will surely be useful for us when designing and implementing the Qubes 2.0 architecture.

In order to continue with Qubes, we've decided to hire some Linux programmers, while Rafal and I will continue with our research project over the coming months. We've decided to start a cooperation with another Polish computer outfit, TLS Technologies, who specializes in advanced systems design and implementation.

There are a couple of people people from TLS engaged in Qubes, and you will soon "meet" them on qubes-devel, in our wiki, and of course, you will see their contributions in our git repos.

The plan is to have Beta 1 released sometime in January ~~2010~~2011. The two important features that will be implemented first, and that will make it into Beta 1 (apart for the long-awaited installer) are: Firewall VMs, and support for templates for service VMs. Stay tuned for more details soon!

If everything goes smoothly, then we should expect Qubes 1.0 sometime at the end of Q1 2011...

Qubes Alpha 3!

2010-10-06T16:00:00.003+02:00

We have just uploaded the new packages for the Qubes Alpha 3 milestone. A lot of under the hood work went into this release, including:

Support for fast-booting Disposable VM (see also implementation description here)

Dynamic memory balancing between AppVMs

Redesigned networking and NetVM support (for VT-d system)

Reasonably stable S3 sleep support (suspend-to-RAM), that works even with a NetVM!

Improved GUI virtualization (all known bugs fixed finally!)

Disposable VMs are really a killer feature IMO. The screenshot below shows the user's experience:

The user righ-clicks on a PDF file, chooses "Open in Disposable VM", and then waits 1... 2... 3... 4 seconds (assuming a reasonably modern laptop) and the document automagically opens in a fresh new Disposable VM. If you make some changes to the document (e.g. if it was a PDF form, and you edited it), those changes will propagate back to the original file in the original AppVM.

So, within 4-5 seconds, Qubes creates a new VM, boots it up (actually refreshes from a savefile), copies the file in question to the VM, and finally opens the application that is a registered MIME handler for this type of documents, e.g. a PDF viewer. We're pretty confident this time could be further decreased down to some 2 seconds, or maybe even less. This is planned for some later Beta release.

Dynamic memory balancing allows to better utilize system physical memory by moving it between running AppVMs in realtime, according to the VM's real needs. This allows to run more VMs, compared to a scheme with static memory allocation, and also dramatically eliminates system hiccups, that otherwise occur often in a static scheme when one of the VMs is short of memory and initiates swapping.

The screenshot above shows the memory usage on my 6GB laptop when writing this blog post. As you can see I can easily run a dozen of AppVMs (most users will not need that many, but I'm a bit more paranoid I guess ;) and could probably even start a few more if there was such a need (e.g. open some Disposable VMs). Of course, this all depends on the actual type of workload the user runs in each VM - most of my AppVMs run just one or two applications, usually a Web browser (Firefox), but some, e.g. the work, and personal AppVMs run much more memory-hungry applications such as Open Office, or Picasa Photo Browser. I very rarely see more than 1 GB of memory allocated to a single VM, though. Generally speaking, the new memory management in Qubes works pretty nice.

Currently, the biggest slow-down factor for Qubes is somehow poor disk performance, most likely caused by the joint impact of the Xen backend, Linux dm, and kcryptd (we use the simplest possible Xen block backend for security reasons, will move to more sophisticated backends when we introduce untrusted storage domain in Qubes 2.0).

Now, most of the under-the-hood work for Qubes 1.0 seems to be complete, and now it time for all the polishing of the user experience, which will be the main focus of the upcoming Beta development. Just reminding that we're currently looking to hire developers for this effort.

The Installation instructions can be found here. Enjoy!

ITL is hiring!

2010-09-28T13:55:00.007+02:00

We're looking to hire one or two full time developers, who will be working on the open source version of Qubes OS, with the primary task of advancing it from Alpha to Beta stage, and then finally to a production quality version.

We're looking to hire developers, not necessarily security researchers! Specifically we expect the following from candidates:

Many years of experience with Linux/GNU development, including system-level and kernel-level Linux development, documented by the actual projects,
Familiarity with virtualization technologies, and specifically with Xen hypervisor,
Basic understanding of the Qubes architecture and excitement about the project :)
Product-oriented approach (polishing, testing, packaging, understanding of user needs),
Good communication skills in written English

In return we offer the following benefits:

Decent, full-time salary,
Opportunity to be part of a renown security team,
Opportunity to work on an exciting product,
Work on a GPLed project with all the benefits it gives to the developer (visibility, rights to the code)

If you're interested in joining our team, please send a message to joanna at invisiblethingslab.com.

Please do not send typical resumes: don't write about schools you finished, certificates you obtained, driving license, scuba trainings, etc. We are only interested in a short bio (keep it below 100 words please), and links to your past or current projects. Include your geographic location.

While it would be great if you were based in Warsaw (or somewhere in Poland), as it would allow for regular face-to-face meetings, this is not a critical factor. ITL doesn't have a physical office, and everybody work from their apartments, so there is no need to relocate to Warsaw, in case you happened to be based somewhere else.

On Thin Clients Security

2010-09-13T16:35:00.006+02:00

I'm constantly being asked about it, and so I thought I would write a handy blog post, so I could just referrer to it in the future, when yet anther person asks me if I think the use of Thin Clients is a game-changer to desktop security...

It is not! Thin Clients do not improve your desktop security in any way, and that's because:

You still run a regular full-blown OS, such as Widows and all the regular applications, such as those buggy PDF readers, Web browsers, etc - it's just you run them all on some corporate server, rather on your laptop. The fact that you run the OS on the corporate server, doesn't make it any less prone to compromises, compared to if you run it locally on your laptop.

A compromise of your laptop, even if it's just a dump terminal, is still fatal! This is because if your laptop's kernel (or MBR, or BIOS, or some PCI device's firmware, or GPU) is compromised, the attacker can intercept/steal/spoof all the data that you work on remotely, because it is still your laptop that processes the input (keystrokes, mouse events) and output (pixels). So, an Evil Maid attack on your laptop when you use it as a Thin Client, would be just as devastating, as it is otherwise (and don't fool yourselves that crypto tokens can help)

We really need secure end-user systems, even if we just want to use them as dump terminals only! There is really no way we could skip this step (and e.g. focus only on infrastructure, or services security).

(Un)Trusting your GUI Subsystem

2010-09-09T18:21:00.002+02:00

Why do we need secure desktop systems? Why support from hardware is necessary to build secure desktop OSes? Does virtualization make things more, or less complex? Why Dynamic RTM (Intel TXT) is better than Static RTM? Can we have untrusted GUI domain/subsystem?

I tried to cover those questions in my recent keynote at ETISS, and you can grab the slides here.

Particularly, the slide #18 presents the idealistic view of an OS that could be achieved through the use of hardware virtualization and trusted boot technologies. It might look very similar to many other pictures of virtualized systems one can see these days, but what makes it special is that all the dark gray boxes represent untrusted domains (so, their compromise is not security-critical, except for the potential of a denial-of-service).

No OS currently implements this architecture, even Qubes. We still have Storage and GUI subsystem in Dom0 (so they are both trusted), although we already know (we think) how to implement the untrusted storage domain (this is described in detail in the arch spec), and the main reason we don't have it now is that TXT market adoption is so poor, that very few people could make use of it.

The GUI subsystem is, however, a much bigger challenge. When we think about, it should really feel impossible to have an untrusted GUI subsystem, because the GUI subsystem really "sees" all the pixmaps that are to be displayed to the user, so also all the confidential emails, documents, etc. The GUI is different in nature than the networking subsystem, where we can use encrypted protocols to prevent the netvm from sniffing or meaningfully intercepting the application-generated traffic, or the storage subsystem, where we can use fs-encryption and trusted boot technologies to keep the storage domain off from reading or modifying the files used by apps in a meaningful ways. We cannot really encrypt the pixmaps (in the apps, or AppVMs), because for this to work we would need to have graphics cards that would be able to do the decryption and key exchange (note how this is different from the case of an untrusted storage domain, where there is no need for internal hardware encryption!), and the idea of putting, essentially an HTTPS webserver on your GPU is doubtful at best, because it would essentially move the target from the GUI domain to the GPU, and there is really no reason why lots-of-code in the GPU were any harder to attack than lots-of-code in the GUI domain...

So we came out recently with an idea of a Split I/O model that is also presented in my slides, where we separate the user input (keyboard, mouse), and keep it still in dom0 (trusted domain), from the output (GUI, audio), which is moved into an untrusted GUI domain. We obviously need to make sure that the GUI domain cannot "talk" to other domains, to make sure it cannot "leak out" the secrets that it "sees" while processing the various pixmaps. For this we need to have the hypervisor ensure that all the inter-domain shared pages mapped into the GUI domain are read-only for the GUI domain, and this would imply that we need the GUI protocol, exposed by the GUI domain to other AppVMs, to be unidirectional.

There are more challenges though, e.g. how to keep the bandwith of timing covert channels, such as those through the CPU caches, between the GUI domain and other AppVMs on a reasonably low level (please note the distinction between a covert channel, which require cooperation of two domains, and a side-channel, which requires just one domain to be malicious - the latter are much more of a theoretical problem, and are of a concern only in some very high security military systems, while the former are easy to implement in practice usually, and present a practical problem in this very scenario).

Another problem, that was immediately pointed out by the ETISS audience, is that an attacker, who compromised the GUI domain, can manipulate the pixmaps that are being processed in the GUI subsystem to present false picture to the user (remember, the attacker should have no way to send them out anywhere). This includes attacks such as button relabeling ("OK" becomes "Cancel" and the other way around), content manipulation ("$1,000,000" instead of "$100", and vice-versa), security labels spoofing ("red"-labeled windows becoming "green"-labeled), and so on. It's an open question how practical these attacks are, at least when we consider automated attacks, as they require ability to extract some semantics from the pixmaps (where is the button, where is the decoration), as well as understanding the user's actions, intentions, and behavior (just automatically relabeling my Friefox label to "green" would be a poor attack, as I would immediately realize something is going wrong). Nevertheless this is a problem, and I'm not sure how this could be solved with the current hardware architecture.

But do we really need untrusted GUI domain? That depends. Currently in Qubes the GUI subsystem is located in dom0, and thus it is fully trusted, and this also means that a potential compromise of the GUI subsystem is considered fatal. We try to make an attack on GUI as hard as possible, and this is the reason we have designed and implemented special, very simple GUI protocol that is exposed to other AppVMs (instead of e.g. using the X protocol or VNC). But if we wanted to add some more "features", such as 3D hardware acceleration for the apps (3D acceleration is already available to the Window Manager in Qubes, but not for the apps), then we would not be able to keep the GUI protocol so simple anymore, and this might result in introducing exploitable fatal bugs. So, in that case it would be great to have untrusted GUI domain, because we would be able to provide feature-rich GUI protocols, with all the OpenGL-ish like things, without worrying that somebody might exploit the GUI backend. We would also not need to worry about putting all the various 3rd party software in the GUI domain, such as KDE, Xorg, and various 3rd party GPU drivers, like e.g. NVIDIA's closed source ones, and that some of it might be malicious.

So, generally, yes, we would like to have untrusted GUI domain - we can live without it, but then we will not have all the fancy 3D acceleration for games, and also need to carefully choose and verify the GUI-related software (which is lots of software).

But perhaps in the next 5 years everybody will have a computer with a few dozens of cores, and also the CPU-to-DRAM bandwidth will be orders of magnitude faster than today, and so there will be no longer a need to offload graphic intensive work to a specialized GPU, because one of our 64 cores will happily do the work? Wouldn't that be a nicer architecture, also for many other reasons (e.g. better utilization of power/circuit real estate)? In that case nobody will need OpenGL, and so there will be no need for a richer GUI protocol than what is already implemented in Qubes...

It's quite exciting to see what will happen (and what we will come up for Qubes) :)

BTW, some people might confuse X server de-privileging efforts, i.e. making the X server run without root privileges, which is being done in some Linux distros and BSDs, with what had been described in this article, namely making the GUI subsystem untrusted. Please note that a de-priviliged X server doesn't really solve any major security problems related to GUI subsystem, as whoever controls ("0wns") the X server (depriviliged or not) can steal or manipulate all the data that this X server is processing/displaying. Apparently there are some reasons why people want to run Xorg as non-root, but in case of typical desktop OSes this provides little security benefit (unless you want to run a few X servers with different user accounts, and on different vt's, which most people would never do anyway).

Qubes, Qubes Pro, and the Future...

2010-09-02T10:38:00.005+02:00

The work on Qubes OS has been extremely exciting and also very challenging for us. While most of the work we have been doing so far relates to solving various technical, under-the-hood challenges, the more important goals in the long-term are related more to mitigating the so called "human factor", i.e. making the system not only easy to use, but tolerant to user absentmindedness. This includes e.g. ensuring the user uses a correct AppVM (e.g. do the banking in the "banking" AppVM, and not in the "random web browsing" AppVM, and also not the other way around: don't do random surfing in the "banking" AppVM), and generally making the whole isolation between AppVMs as seamless as possible, but without sacrificing the security at the same time.

This is becoming very important, as the technical level of security in Qubes is already very high, and so the "human factor" might easily become a low hanging fruit for the attacker. (In contrast to other OSes)

But for Qubes to become something more than just an interesting OS for Linux geeks and security enthusiasts, it is also critical to have better application support. Right now Qubes lets users run Linux apps, because each AppVM is Linux-based. But, and let's not be afraid to admit this: Linux sucks when it comes to application support! (Take Open Office as an example - it not only looks like MS Office 97, but is also terribly user-unfriendly, especially their presentation program, the Impress. Why is it so difficult to make it look and behave more like Apple Keynote?)

There is only one way to provide better application support to Qubes: make it support Windows-based, or Mac-based, AppVMs. Just imagine that: being able to run most of your Windows (or Mac) applications, but at the same time benefit from the Qubes strong isolation and seamless integration on one common desktop...

In order to implement support for Windows-based AppVMs (or alternatively Mac-based AppVM) we would need to engage significant resources (5+ very skilled developers, working full time for 1+ year), and so we're currently looking for an investor that would be able to provide funding for such an endeavor. The idea is to create a dedicated spin-off company that would focus entirely on Qubes and Qubes Pro, and in the future will make a profit from selling Qubes Pro licenses. Qubes Pro will become a commercial product, still based on the open source Qubes, but adding support for Windows-based or Mac-based AppVMs. I would be happy to discuss the details and business plan via email with interested potential investors.

Speaking about the future of Qubes: next week I will speak at the European Trusted Infrastructure Summer School, where I will talk about some general stuff like why we need secure desktop systems and why trusted computing might be a way to go, but will also dive a little bit into some new things we plan for Qubes 2.0, such as storage domain and split I/O graphics model. The conference features some very reputable speakers in system-level security field, such as David Grawrock (the father of Intel TXT and TPM), and Loic Duflot (our venerable competitor in the filed of offensive system-level research), so I consider a honour to deliver an opening keynote there (Check the agenda here).

I will have my Qubes laptop with me, of course, so if anybody is interested to see Qubes OS live (including Disposable VMs!), I would be happy to do a quick demo on the spot.