Insufficient data from Andrew Fryer

So You want to be an Evangelist

Andrew.Fryer — Wed, 08 May 2013 09:08:00 GMT

With my job title of Evangelist I often get asked about what my roles, both inside and outside of the Microsoft firewall. In the 2 weeks I have presented to the Leeds VM User Group (www.vmug.org.uk), done a careers chat at a Science college, produced and presented at TechDays Online nd attended SQL Bits 11..

So it’s a lot of presenting, blogging and explaining stuff. Of course you can only blog about what you know about, so Simon and I spend a lot of time learning how Microsoft technology works, and talking to IT Professionals about what they are doing and the challenges they face. I hope this gives some credibility both online and in person and given you are reading this that seems to be working. Our knowledge acquisition goes on all the time but occasionally it's good to commit longer periods of time to it and so we get the opportunity to go to things like TechEd in Madrid. For me this checks all the boxes and gives me the chance to hang out with the Microsoft product teams who present at this sort of event.

As an Evangelist I don’t have to pay for my flights, hotel or entry fee, Microsoft pick all of that up for me. In return I will write posts, work up ideas for future presentations and share stories good and bad about how our products are actually being used.

The question is do you fancy being evangelist for a week and come out with our team to Madrid on Microsoft expenses? if you do then you’ll want to enter the TechEd Challenge

From here you can either enter a draw for a place or compete for one of three further places by writing a blog post to show off your evangelist skills. Full details of the prizes are here, good luck and hopefully see you there

Evaluate This – Hyper-V Server 2012

Andrew.Fryer — Wed, 03 Apr 2013 07:19:00 GMT

Hyper-V Server is a free operating system specifically designed to just run Hyper-V so basically a cut down core installation of a paid for edition of Windows Server. The cut down bit refers to the fact that only the roles and features needed to run Hyper-V are there. However Hyper-V itself is in no way cut down; for example you can create clusters for running HA virtual machines (up to 64 nodes hosting 8,000 VMs) and each VM can still have up to 64 logical processes as per the DataCenter edition of Windows Server.

So what’s the catch?

If there is one then it’s that if you want to run Windows Server in a VM it needs to be licensed and the most efficient way to do that once you get to 6-7 VMs per host is to use Windows Server DataCenter edition as this allows any number of guest VMs’ to be licensed for Windows Server as well as the hosts. However if you were going to use Hyper-V to host VDI then your guests need to be licensed for Windows 7/8 and so Hyper-V Server is a good candidate. Another example is if you want to just host Linux VMs which will run really well and are supported (depending on the flavour you are using).

I have made my usual short screencast to show you what it looks like..

Also, you might want to look at the other posts in my Evaluate This series as Hyper-V Server is best managed remotely, and my other screencasts will show you how to do such things as live migrations, VDI, replicate VM’s etc. all of which are possible with Hyper-V Server.

Notes:

To configure Hyper-V Server for remotes access all I did was use the built in SConfig utility to join it to my domain as remote management is turned on by default in Windows Server 2012, and I have group policy setup to allow remote desktop on all of my servers.

NIC teaming is now viable in server core and Hyper-V server because it’s built into the OS where in earlier versions of server you might not have been able to install the hardware vendors NIC teaming software without a user interface.

Hyper-V Server like the server core installation option of Windows Server only needs half the patching of a full installation of Windows Server.

Hyper-V Server 2012 now includes Powershell out of the box.

Finally you can get Hyper-V Server 2012 here and try it yourself and put it into production if needed.

HD Insight

Andrew.Fryer — Fri, 22 Mar 2013 14:34:07 GMT

Despite common misconceptions Microsoft now has extensive interoperability with open source technologies for example you can run a php application on Azure, get support from us to run RedHat, SUSE or CentOs on Hyper-V and manage your applications from System Center. , So extending this approach to the world of big data with Hadoop is a logical step given the pervasiveness of Hadoop in this space.

Hopefully your reading this because you have some idea of what big data is. If not it is basically an order of magnitude bigger than you can store, it changes very quickly and is typically made up of different kinds of data that you can’t handle with the technologies you already have. For example web logs, tweets, photos, and sounds. Traditionally we have discarded this information as having little or no value compared with the investment needed to process it, especially as it often not clear what value is contained in this information. For this reason big data has been filed in the too difficult drawer, unless you are megacorp or a government.

However after some research by Google, an approach to attacking this problem called map reduce was born. Map is where the structure for the data is declared for example pulling out the actual tweet from a twitter massage, the hashtags and other useful fields such as whether this is a retweet. The Reduce phase then pulls out meaning from these structures such as digrams ( the key 2 word phrases) sentiment, and so on.

Hadoop uses map reduce but the key to its power is that it applies the map reduce concept on large clusters of servers by getting each node to run the functions locally, thus taking the code to the data to minimise IO and network traffic using its own file system – HDFS. There are lots of tools in the Hadoop armoury built on top of this, notably Hive which presents HDFS as a data warehouse that you can run SQL against and the PIG (latin) language where you load data and work with your functions.

Here a Map function defines what a word is in a string of character and the reduce function then counts the words. Obviously this a bit sledgehammer/nut, but hopefully you get the idea. Also the clever bit is that each node has part of the data and the algorithm to process and then reports back when it’s done with the answers to a controlling node a bit like High Performance Computing and the SQL Server Parallel Data warehouse.

So where does Microsoft fit into this?

The answer is HDInsight which is now in public beta. This is a toolkit developed in conjunction with Horton Works to add integration to Hadoop to make it more enterprise friendly:

Azure HDInsight is the ability run Hadoop on Azure so you can create clusters and when you need them and use Azure’s massive connectivity to the internet to pull data in there rather than choke bandwidth to your own data centre. There is actually a lot of free and paid for data already in Azure you might want to use as well via the Windows Azure Marketplace including weather data from the Met. Office, UK Crime stats and stats for Greater London.
You can also run HDInsight as a service on Windows Server 2012 via the Web Platform Installer. Like any web service running on server this can of course be managed from System Center 2012; so you can monitor health and performance and if needs be spin up/shut down HDInsight nodes, just like any other web service under your control. I would see this being used to prototype solutions or work on complex data that you have in house and don’t or can’t want to put up in the cloud.
An odbc driver to connect to Hive. With this HDInsight just be caome another data source; for example you can pull the data into SQL Server via it’s built in integration services, and squirt it out if needs be too. The dirver means you can directly build analysis services cubes (another part of SQL Server) or use PowerPivot in Excel to explore the data that way.
The Data Explorer Preview add-in in Excel 2013 to query the HDInsight as well as load of other relational sources
F# programming for Hadoop. F# is a functional programming language that data scientists understand in the same way as I learned Fortran in my distant past as an engineering student. Note these add-ins are free but are NOT Microsoft tools.

Big Data is definitely happening, for example there was even a special meeting at the last G8 meeting on this as it is such a significant technology. However it cannot be solved in one formulaic way by one technology; rather it’s an approach and in the case of Microsoft a set of rich tools to consolidate, store, analyse and consume: The point being to integrate Big Data into your business intelligence project using familiar tools, the only rocket science being the map reduce bit, and that is the specialism of a data scientist. Some of their work is published by academics so you might find the algorithm you need is already out there - for example the map function to interpret a tweet and pull out the bits you need is on twitter.

However research is going all the time to crack such problems as earthquake prediction, emotion recognition from photographs, ,edical research and so on. If you are interested in that sort of thing world then you might want to go along to the Big Data Hackathon 13/14th April in Haymarket, London, and see what other like minded individuals can do with this stuff.

Evaluate This–File Classification

Andrew.Fryer — Tue, 19 Mar 2013 08:00:00 GMT

In my last post & screen cast I showed how Dynamic Access Control (DAC) worked; the business of matching a users claims to the properties of a file (Resource Property in DAC), however the problem then becomes how do I correctly tag my files so that DAC works. You shouldn’t necessarily be doing this; it’s the users data and you are just the curator of that data. The users aren’t going to have the time or inclination to do this even if they are working in a compliance or regulated environment. However they might be able to give you some rules which you could apply to the files and this is what Data Classification does.

File Classification is part of the part of File System resource Manager (FSRM) role service and is new for Windows Server 2012 where before FSRM was just there to only allow certain file type to be uploaded or to grant quotas to users to restrict how much and of what could be stored on your servers. The secret sauce is then to link the resource property you set using the classification rule to a Central Access Rule in DAC

Hopefully this screencast shows how easy this is to do..

Things to note:

As per my previous post you’ll need your domain functional level to be Windows Server 2012.

You’ll need the FSRM role service on your file servers and these also need to be running Windows Server 2012.

The PowerShell is

Add-WindowsFeature –Name FS-Resource-Manager

and you’ll need a copy of Windows Server 2012 Evaluation Edition to try this out

I used a simple expression “Top Secret” in my screen cast but you can write RegEx to look for things like credit card details, NI numbers and appropriately protect those documents automatically using this technique.

File Classification in a production environment would typically run as a scheduled job, so to be clear this does not magically happen on the fly as users save documents onto your file servers.

Evaluate This – Dynamic Access Control

Andrew.Fryer — Thu, 14 Mar 2013 07:31:00 GMT

Managing users access to the right files is a pain on any OS, the best that’s going ot happen is that no one will complain about not having access to a file while none of your sensitive company data gets into the wrong hands. In a traditional hierarchical business life was pretty easy you had a group called finance, a folder with their finance documents in you set up permissions form one to the other and you were done. However in a virtual taming, outsourcing home working organisation all sort of rules are needed to keep third parties at arms length from confidential data and allow users to have different roles on different teams. Also very few of us are good at filing, for example how many of properly tag our holiday photos so that we can track down our friends in all the photos we have?

Windows Server 2012 has several components in it to make this work, but key to this is Dynamic Access Control (DAC) which itself plugs into Active Directory (AD) , Group Policy and File Server Resource Manager (FSRM). The Dynamic in DAC refers to the fact that whenever a user tries to access a file their claim to do so is evaluate at the time of access. There are several parts to DAC to make this work and in my screencast you can see this in action..

However there’s a lot going on here and so I also wanted to describe the moving parts of DAC in more detail.

Claim Types these are the things we know about our users and the devices they are using based on querying what’s in AD for example here I have defined the Country a user in it..

Resource Properties are the things I know about what the user is trying to access such as a file, for example I could setup a tag of Country and tag each file with one or more Countries..

Resource Property Lists are optional groups of Resource Properties that you want to keep together for a purpose, so a subset of the Global Resource Property List that is there by default in DAC. Here’s the Global Property List..

Central Access Rules allow you to define how to evaluate a claim against a Resource Property and assign permissions of the back of this. At the top of this dialog you are asked about which resources (Target Resources) the rule will apply to in my demo I have set this up so that my rules are only applied to objects that have the resource properties I am interested in already set..

Further down the dialog under Current Permissions I can then set the rule that I want to enforce. Here I have said the device the user on must be running Windows 8 Enterprise to get full permissions to the resource. For this to work AD must know the computer I am on and in Windows Server 2012 AD this property is actually only set if I am on a Windows 8 or a Windows Server 2012 machine . So I can’t get in from on an older windows machine or if my machine is not domain joined.

I also have a rule (User-Country-Department) which says that the user’s country and department must match the country and department of the resource being accessed. This is great I don't have to create groups for each user or folders to categorise departments and fiddle with ACLs this one rule makes that work and provided the users data in AD is kept up to date and files are tagged correctly that’s all I have to do.

Central Access Policies. Several Rules can then be combined into a single policy. In my case I have a Central Access Policy I have called Default and this references my two rules..

This is now a policy object that can be applied like any group policy. So If I look at group policy you can see a policy called DA-FileServer-Policy that is filtered to only apply to Server1 ...

If I edit that and expand Computer Configuration –> Windows Settings –> Security Settings –> File System –> Central Access Policy you can see where I have referenced my Default policy..

Things to note:

DAC requires the AD functional level to be at Windows Server 2012.

This can work in concert with traditional ACLs but remember that the principal of least privilege applies so if there’s and explicit deny somewhere in DAC or in an ACL that is what will win. You’ll want to test your scenarios and there’s two tools here to help:

You can set proposed permissions in a Central Access Rule as well as actually set permissions.

For a particular folder or file you can go into properties –> security tabs –> advance security to evaluate security. You can see what policy is applied and what is granting or blocking users’ access to objects. You can also see there’s a classification tab from which you can see and set (depending on permissions ) the resource properties for that file/folder.

I will cover off how to automatically classify files rather then rely on manual tagging them in my next post. In the meantime if you want to try this you’ll need a copy of Windows Server 2012 evaluation edition and use it to make a domain controller.

Evaluate this - Deduplication

Andrew.Fryer — Mon, 11 Feb 2013 07:21:00 GMT

Deduplication is the business of compressing data without loss and this is now built into Windows Server 2012 as a role service. The official marketing from us states that you will save somewhere between 20-70% of the space on your file servers if you implement this. If that’s sounds interesting my screencast shows how to configure and monitor it..

The clever thing about deduplication is that it’s built into NTFS, so you can apply it to any non system volume without the need for specialist storage. There are some caveats:

You can’t use it on the system volume
It works on closed files so it won’t do any good against virtual hard disks or SQL Server database files for example. However you could and in some cases should turn it on inside a virtual machine and get the benefits of deduplication that way.
The deduplication process does not happen when you write a file to disk, it runs in the background and is typically scheduled. The default setting is to only deduplicate files older than 5 days the odds are that a file that hasn’t changed for 5 days won’t change that much again and is worth deduplicating.
It will slow down your initial reads a bit (approx. 3%), but once a file is cached performance will improve a bit (approx. 8%)
You’ll want to have a backup solution that is compatible with this new feature e.g. System Center Data Protection Manager 2012 SP1 as well as VEEAM and Symantec Backup Exec (with more vendors updating their software all the time)

To try this yourself all you’ll need is an Evaluation Copy of Windows Server 2012. Having got the idea you may also want to see how well it will work on your data. To do that install it turn on the deduplication e.g. in Powershell..

Add-WindowsFeature –Name “FS-Data-Deduplication”

and then copy windows\system32\ddpeval.exe and run this against a file share, volume etc. Note that this might put some load on your network but otherwise shouldn’t be too invasive as it will run in the background (possibly for hours on a big volume) before telling you what you would save if you enabled this feature.

Finally thanks to my good friend Simon; he has done most of the legwork in setting up deduplication for our IT camps and I have shamelessly used that for the screencast.

Evaluate This – Storage Spaces

Andrew.Fryer — Thu, 07 Feb 2013 07:46:00 GMT

So far in this series I have used the new storage features of Windows Server 2012 as a place to run VMs from, but there’s more to it than that. Shared storage used to mean presenting SAN storage inside a cluster, and you relied on your SAN experts to provision the storage you needed. However with SAS / JBOD technologies coming along it’s possible to create storage that’s still highly available. However you might still want access to some of the clever things a SAN can do like thin provisioning, where you define storage you plan to use but actually haven’t got yet. So in this short screencast I show how storage spaces in Windows Server meets this need..

To try this out all you’ll need is one virtual machine running on one laptop and an Evaluation Copy of Windows Server 2012

Notes

I used a bunch of SCSI disks in my demo VM to build a storage space and they were all the same size. They don’t have to all be SCSI, they could be attached via USB, SATA etc. and can be of varying size and performance. However if you want to create a storage pool in a cluster then the disks must be SAS (Serial attached SCSI) for that. Also bear in mind that the pool will work down to the slowest disk and not up to the fastest.

I do have a script to build my fileserver, which in turn relies on a configuration file to add in the roles and features I need, and it builds form a sysprep copy of Windows Server 2012 with an answer file to join it to my Contoso domain. It does have a really useful function from Simon to rename the VM in active directory (so it is called FileServer1 in AD as well being the name of the VM in Hyper-V).

Rather than running a virtual machine or using the storage space for ordinary files, in this screencast I used it to host a SQL Server database. SQL Server 2012 has support for storing databases on SMB shares and I have seen 200,000 iops in SQL Server where the database is on a remote share like this. However the UI in Server manager doesn’t seem to allow you to navigate across shares (have I been away form SQL Server too long?) so I did the attach from a simple SQL Server T-SQL script.

Storage spaces often raises a lot of questions at our camps so here’s a good FAQ on TechNet. If you are curious about performance my advice is to test your big idea thoroughly and check this script and whitepaper to ensure you have the optimal setup.

Evaluate This–Remote Desktop Services

Andrew.Fryer — Fri, 01 Feb 2013 07:51:00 GMT

In my last post I showed how easy it is to create virtual desktops in Windows Server 2012, and while that’s now a core part of providing remote desktops to your users there is still the good old fashioned terminal services, or to give its modern name Remote Desktop Services (RDS). RDS also changes quite a lot in Windows Server 2012 and so I have made this short screencast to show how to set it up..

To try this yourself all you’ll need is an Evaluation Copy of Windows Server 2012

Notes:

VDI and RDS are designed to compliment each other:

The client is using the same protocol (RDP - Remote Desktop Protocol) to interact with a remote desktop or a virtual desktop
They share services for example; the remote desktop web access portal for users to access remote desktops and applications, and the remote desktop broker that controls access to the resources on offer.
You can create RD RemoteApp applications from both an RDS Pool or a VDI Pool. If you aren’t familiar with RD RemoteApp it’s a way of delivering an individual application over RDP so it looks like it’s on a users desktop but actually its executing server side. I have seen it being used for providing legacy applications or those that need to run in a secure environment.

So when to use what?

I think this comes down to efficiency and manageability. You can support far more (typically 12x) remote users with RDS than with VDI running on the same server hardware. So if possible use RDS complimented with technologies like App-v to virtualise application delivery to delegated users. That way you’ll just have to maintain the few servers providing RDS and secure the users profile disks.

It may be that some or of all your users can’t use RDS because they applications they want don‘t ‘like’ being run from and RDS server. In that case the next most efficient option is pooled VDI where a virtual desktop is shared rather than being dedicated to a particular user. In this scenario you just have manage one virtual desktop, and then control the deployment of revisions to that (which may just include patches or whole new applications). Your final option is to give your users personal virtual desktops which means that each of these needs to be managed in exactly the same way as if they have real desktops. What’s good about VD/RDS in Windows Server is that the users get a good experience either way with multi touch support, smooth video streaming and USB redirect so they can use webcams, dongle, card readers etc.

Finally if you are planning to do this in your organisation, I would suggest a really thorough trial and to over provision hardware both on the server side to provide a great user experience and also to provide good quality big monitors to win the hearts and minds of your users.

Evaluate This–VDI

Andrew.Fryer — Tue, 29 Jan 2013 07:44:00 GMT

Microsoft is serious about Virtual Desktop Infrastructure, and the first sign of this in Windows server is when you try and add a role or feature..

If you opt for install Remote Desktop Services Installation and select Virtual Desktops as in my short screencast you can see that a lot of work has gone in to making this as simple as possible. However there is more to VDI in Windows Server 2012 than a good installation experience for example:

Hardware Optimisation with RemoteFX. Making use of spare CPU capacity on your hosts to spoof a powerful graphics card into the desktops for multi-touch, and multiple screens
Dynamic memory has start up memory settings as an option so you can quickly start a Windows 7/8 VM with a lot of memory then peg it back to a lower running value until the VM is under memory pressure when it can get more subject to priorities you set.
User Profile Disks. These are differencing disks where each users state and profile are stored. These differences are based on the gold image you use to create your VDI environment for example a Windows 8 machine with Office 2010 on. The clever bit is that if you wanted to revise or patch this gold image, say to put Office 2013 on, then you can do this without users’ losing their profiles and they’ll get the new version once they log off and log in again.

Notes:

In this screencast I put all of the Pooled VDI virtual machines’ storage onto a highly available file server (this post shows you how I built that) and this is where my user profile disks are also stored so that no matter which physical host a user gets their pooled desktop from they will still get their own user settings.

I used a separate VM for each role in my remote desktop infrastructure, however if you elect for a quick setup then you can have all the roles on the one physical host from which the virtual desktops will run as well.

There’s a more details lab guide here, and you can easily navigate to other labs form here for a quick setup as well. Either way you’ll need an Evaluation Copy of Windows Server 2012 and Windows 8

Evaluate This – Hyper-V Replica

Andrew.Fryer — Mon, 21 Jan 2013 07:47:00 GMT

In many of the screencast in this series I have moved a VM around my demo setup, however there has only been the one copy of it whether it was on a scale-out file server, in a cluster or both. In any production environment you would want to augment this with additional disaster recovery techniques including have a backup of the key virtual machines somewhere.

Replica in Windows Server 2012 is a partial answer to this. You setup a process to make an offline copy of a given virtual machine (VM) on another server and continually keep it updated. This replica VM can be updated over “UK speed” (don’t get me started!) broadband and you can also maintain up to 4 roiling snapshots enabling you to go back past a data error you may want to correct. This screencast shows you how to set it up ..

Notes:

The replica is always off and it’s up to you under what conditions you invoke failover and of course you can script this in PowerShell with Start-VMFailover as well as all of the configuration for Replica I did in the screencast.

The principal and replica can either be a cluster or an individual server.

In my demo all the servers belong to the same domain but if that’s not the case then you can use CredSSP to set this up. One use of this is that hosters are planning to offer replica as a service so you’ll be able to set your critical VMs to be replicated (is that English?) over the internet into their data centres as a service.

As I briefly mention in the screencast you can also set your replicated VM to preserve it’s network settings when you fail over to it in its new location.

You are going to need 2 x hosts/physical servers to try this and an Evaluation Copy of Windows Server 2012. It doesn’t matter what OS your virtual machine is running, but do be aware of what applications etc. are supported for replication, e.g. SQL Server , System Center, SharePoint etc.