If you are building a data-driven web or mobile application, then key concerns you need to address is how data will be surfaced to the client and how the client can update the state on the server. This is not a new problem, and one that we have been solving for decades.

GraphQL provides one way to address this problem and uses an approach that provides significant benefits over previous RPC-based approaches. GraphQL was originally created at Facebook several years ago. It has been widely adopted since by Github, Concur, Airbnb and more. We’re even now adopting it at DocuSign. If you are doing React development, GraphQL has become the de-facto way to query from React clients.

What is GraphQL?

At its heart, GraphQL is several things.

• A schema that allows defining a structured view of data that will be surfaced to / queried / updated by a GraphQL client.
• A query language for interacting with a GraphQL endpoint to retrieve, update, and subscribe to notifications.
• Tooling and SDKs for building GraphQL clients and servers, or interacting with GraphQL endpoints such as the GraphiQL editor, clients like Apollo and Relay, servers like Apollo, GraphQL for .NET, and more.

A few things that have stood out for me that I have really liked about GraphQL.

• It provides a standard way to query and update data including non-crud based actions. It supports “graphs” of data allowing queries to pull hierarchical data.
• It puts power in UI teams hands to be able to hand craft queries with the data they need, without having to constantly send custom requests to the API team.
• It has GraphiQL which is an awesome intellisense-driven tool for querying GraphQL endpoints that you can provide to developers for interacting with the endpoint.
• It can work side by side with, and wrap existing “REST” APIs
• It is not opinionated about where data resides, it is not bound to a database. Data  could live in MongoDB, SQL, in documents like S3/Azure Blob, or even in memory. It is really easy to make GraphQL work with any data store
• It allows full control from the server-side as to what data is exposed.

There are also plenty of tradeoffs with GraphQL vs traditional APIs and it is not a silver bullet. Fortunately it is not mutually exclusive and can live side by side with “REST” APIs. Still you should go in with eyes open (as with any other technology or approach).

GraphQL with .NET

Yes you can develop GraphQL solutions with .NET! There are several open source projects that enable this, my favorite one being GraphQL for .NET. GraphQL for .NET lets you create GraphQL schemas and queries in C#, and then surface them via an ASP.NET core endpoint. It also provides a C# client which you can use to issue queries from an application such as as a Xamarin mobile client. All of the libraries are also available on NuGet. If you head to the GraphQL for .NET documentation you can find out more.

I’ve also created an ASP.NET Core example endpoint which can be ran on Windows, Mac or Linux which you can download from here. The readme contains lots of example queries you can run using the GraphiQL editor that is built in.

Check out my LinkedIn Learning Course

If you are interested in learning more about GraphQL for .NET, you can also check out my LinkedIn Learning Course: API Development in .NET with GraphQL. In the course I’ll take you through building the Orders GraphQL endpoint above from scratch in ASP.NET Core. I show you how to use GraphQL for .NET to define your schema, queries, and mutations, and subscriptions and expose them via ASP.NET Core. It’s heavily code based so you can follow along and build it with me. If you check out the course, let me know what you think.

GraphQL is growing

GraphQL is exploding in traction, with more and more companies adopting it as the standard. All signs are that will continue. It is exciting that this powerful ecosystem is available now to you as a .NET developer!

Getting the MVP award every year has become a great side benefit and validation, but the real achievements have been in working with the community and all the things that we’ve accomplished over the last decade!

When I first received the Microsoft MVP award in 2010, it was an achievement I had been working towards and it was something that really validated the work I was doing in the community. It was a goal I had set – to become a Microsoft MVP. When I got it, I was so excited and felt like Microsoft folks were really paying attention to what I was doing. Keep in mind back then that open source wasn’t even a contribution category, but I was doing a lot of talks in the community and working on the Chuck Norris Framework.

After achieving the award for the first time, my focus shifted to primarily doing great things in the community. A lot of that for the last few years has been hands on and I’ve been at the forefront of those efforts. And accordingly, Microsoft continued to validate that what I was doing was important and helpful for developers using Microsoft technologies.

Over the last few years, my role has shifted a bit to building a long term viable business to support the Chocolatey community. This means managing a business and building an amazing team that can help move my vision forward. This last year especially I’ve been focused in that effort of building a great team and that has meant that I’ve had less visible contributions. I’ve been focused enabling my team to do great things, and two of them are MVPs, which is fantastic!

I’m certainly very appreciative of my time as an MVP and have met a lot of amazing folks in the MVP community! I would have loved to have that 10 year blue disk, but Microsoft has rightly saw that my contributions over the last year have not been up to the standard of other folks out there and has made a proper decision on that front.

One thing I will miss is filling out the renewal paperwork every year as it forced me to take some time to reflect on all the great work we were doing in the community, and it put numbers to that work. I think I will look for my own time to do that reflection at some interval, hopefully a bit more often than annually. I always think of this Ferris Bueller quote when I take a moment to reflect. It’s certainly a great quote to apply to your life:

“Life moves pretty fast. If you don’t stop and look around once in awhile, you could miss it.” – Ferris Bueller’s Day Off

Thank you Microsoft for the opportunity and validation over the years – I’m going to continue doing great things in the community and maybe that will bring me back to the MVP award, but maybe it won’t. And that is totally fine.

Fast forward to present, starting soon I will be focused solely on Chocolatey as the Founder of Chocolatey Software, Inc*! It’s an exciting opportunity to really see where we can take this Windows software management thing!

I also could not have had the opportunity to move forward without the support of a tremendous community, who has contributed to Chocolatey’s success in many ways. Your support does not go unnoticed – we will continue to make open source improvements, along with ensuring that organizations can take Chocolatey to the next level with Chocolatey for Business.

It’s a bit bittersweet as I’ve had the opportunity work with a lot of fantastic folks at Puppet and do some really awesome things for furthering automation on Windows. In many ways Puppet has been an amazing place to work (I highly recommend it, they have the remote employee situation handled). However, an opportunity to follow my first love, Chocolatey, is a dream I won’t pass up.

Not everyone gets the opportunity to follow their dreams, so when you get a chance it can be both a thrilling and scary experience! Here’s to the future of Chocolatey and Windows automation!

* – For those keeping track – Chocolatey Software was formed in November 2016 as a spin off of RealDimensions Software, LLC

Read more on what I am doing now here!

Yes it’s true, .NET Fringe is back! We’re gathering back in Portland from July 10th through 12th for 3 days of unadulterated .NET OSS awesomeness. Last year we had a magical event, and we heard you tell us that you couldn’t wait for next year. So this year we’re back and we’re taking it up a notch! You can hear more about why we’re doing .NET Fringe and our plans for this year in our recent Channel 9 Interview.

The Topics

We’ve got a rich and diverse set of topics this year that is very timely and relevant to what is happening in the industry. Just look at the list!

• .NET Core
• DevOps
• Docker
• Hadoop
• IOT
• Kubernetes
• Linked Data
• Microservices
• Machine Learning
• Mobile
• Spark
• Xamarin
• JSON.NET
• IOT
• OSS Project Management

The Speakers

And an excellent lineup of speakers starting with our keynotes. We are thrilled that the list includes Don Syme, the creator of F#!

Lightning Talks

This year back by popular demand we’ll be having about 15 Lightning talks split across the two days. Got an idea you’d like to speak on, we’re open for submissions.

Workshops

Once again we’ll have workshops free for attendees on July 10th. Aaron Stannard is back with more AKKA.NET goodness, and we have several other workshops. Stay tuned to dotnetfringe.org.

Everything else

There’s a ton of other cool stuff, like another awesome venue, the Geek Train from Seattle, a bike ride around Portland, live music, some great food oh and Portland!

Discounts for bloggers

Are you wanting to attend .NET Fringe but are short on cash?

Go write a blog post (and no, not just a one liner) either about your .NET Fringe experience last year, or why you want to attend this year. Then follow @dotnetfringe, tweet with your blog post and include @dotnetfringe or #dotnetfringe. We’ll follow you back and then DM you a 70% DISCOUNT CODE.

Don’t wait

.NET Fringe is around the corner. It’s going to be amazing. Go register and come join us!

Features

Here is a list of the licensed features that will be coming in May. I really think you are going to like what we’ve been cooking up:

• Malware protection / Virus scanning – Automatic protection from software flagged by multiple virus scanners – Read more…
• No more 404s – Alternate permanent download location for Professional customers. Read more…
• Integration with existing Antivirus – Great for businesses that don’t want to reach out to VirusTotal.
• (Business Only) Create packages from software files/installers – Do you keep all the applications you install for your business internally somewhere? Chocolatey can automatically create packages for all the software your organization uses in under 5 minutes! – Shown as a preview in a March webinar (fast forward to 36:45)
• Install Directory Switch – You no longer need to worry about the underlying directives to send to native installers to install software into special locations. You can simply pass one directory switch to Chocolatey and it will handle this for you.
• Support and prioritization of bugs and features for customers.

Sold! But How Do I Buy?

While we are still getting the front end systems setup and ensuring all of the backend systems are in place and working properly, we are limiting availability to the first 500 professional licenses and 20 businesses (Note: we do not expect any issues with our payment processor). Because we are limiting availability, you must register for the Go Live Event at https://chocolatey.eventbrite.com if you are interested.

It bears repeating, the links for purchase will only be sent to folks who have registered for the event, so secure your spot now!

Linux geeks can think of it sort of the inverse of “wine” — Ubuntu binaries running natively in Windows.  Microsoft calls it their “Windows Subsystem for Linux” –Dustin Kirkland

In case you missed the announcement, head to https://channel9.msdn.com/Events/Build/2016/KEY01 and fast forward to 48:15.

Almost immediately folks started asking what this means for Chocolatey. It’s a great question. Here’s the low down. This is fantastic for Chocolatey! You now have a fantastic way to get Unix apps and utilities with dpkg/apt in addition to great Windows apps and software with choco. More developers are going to be using the terminal to do things. It means more users of both apt and choco. More productivity for Windows users and developers. Think about that for a second. On no other platform will you have this ability. It’s an exciting time to be in Windows!

What you can expect to see is more collaboration between choco and apt if they can communicate. Just like you can work with choco install -–source windowsfeatures (back in the latest 0.9.10 betas!), expect to see choco install rsync -–source apt. https://github.com/chocolatey/choco/issues/678

Coming up soon you are going to see what’s coming in the next version of Chocolatey and why it is going to amaze you as another big leap in package management for Windows!

Here’s a preview with PowerShell tab completion and updating path (environment variables) without needing to restart PowerShell (https://raw.githubusercontent.com/wiki/chocolatey/choco/images/gifs/choco_install.gif if the image doesn’t show):

The number of downloads has really increased year over year!

Note: While not a completely accurate representation of usage and popularity, the number of downloads gives a pretty good context. Going up by 7 million in 2014 and then by almost 30 million downloads in one year really shows a trend!

Note: The Chocolatey package has about 1,000 downloads per hour. I shut off the statistics for the install script back in October 2015 due to the extreme load on the site, so the number of Chocolatey package downloads is missing some of the statistics.

History

Let’s take a little stroll through some of the interesting parts of Chocolatey’s history. The history of Chocolatey really starts when I joined the Nubular (Nu) team in summer 2010.

• July 2010 – Nu (Package Management for .NET) is reintroduced and quickly takes off in popularity (I also wrote several posts about it).
• October 6, 2010 – NuGet is introduced. The Nu team had joined up with Microsoft in late August to work on NuPack (as it was called then).
• February 2011 – Joke with other folks on the NuGet team at MVP summit that if we ever introduced a machine package manager, we’d call it Chocolatey NuGet because it wouldn’t be vanilla NuGet packages.
• March 21, 2011 – First lines of Chocolatey are committed. The PowerShell version of Chocolatey represents the first public known use of PowerShell as an application and not just scripts and modules.
• March 23, 2011 – First release of Chocolatey is 0.6.0. I hope you were not using it that early. 😉
• March 28, 2011 – I started talking about Chocolatey on the NuGet list.
• March 29, 2011 – Svein Ackenhausen coins the term “cinst” that would be the main call for choco until 2013.
• April 1, 2011 – Chris Ortman writes the first public blog post about Chocolatey.
• April 26, 2011 – I put up a video showing Chocolatey installing 11 pieces of software. Pardon the music, it was swapped out from my original track.
• September 2011 – Community Repository (https://chocolatey.org) introduced.
• September 7, 2011 – Christiaan Baes writes a post about a side project that is known as ChocolateyGUI.
• October 07, 2011 – A year after NuGet was introduced, I blog about Chocolatey for the first time.
• December 22, 2011 – Anthony Mastrean introduces Pester (a PowerShell BDD testing framework) to Chocolatey.
• March 2012 – Chocolatey is featured in the Pro Nuget Book.
• April 1, 2012 – Matt Wrock commits the first lines of BoxStarter.
• April 23, 2012 – After bouncing ideas off of Keith Dahlby and Aaron Lerch, I figure out how to do mocking for PowerShell testing (which Matt Wrock later puts into Pester proper). I also determine a proper way to structure a PowerShell application (function per file).
• May 13, 2012 – Rich Siegel starts working on Puppet integration.
• September 2012 – Chocolatey is featured on Lifehacker.
• October 4, 2012 – Guilhem Lettron starts working on Chef integration.
• October 19, 2012 – After putting proper mocking support into Pester, Matt Wrock retrofits Chocolatey to use that.
• January 2013 – Automatic packaging is introduced.
• February 13, 2013 – Gary Ewan Park gives ChocolateyGUI a new home and takes over project maintenance.
• June 2013 – Community repo reaches 1,000 stable packages.
• October 2013 – Community repo surpasses 1 million downloads.
• December 13, 2013 – First lines of the C# Chocolatey rewrite are committed.
• February 6, 2014 – Richard Simpson joins Gary on ChocolateyGUI and commits the initial changes for the modern version and look.
• March 2014 – Microsoft validates the idea of Chocolatey.
• July 2014 – The Chocolatey package itself surpasses 1 million downloads.
• Sep 2014 – Introduced Chocolatey Newsletter.
• September 23, 2014 – I show the first C# version of Chocolatey at a conference where my first demo uses choco from non-Windows. There are a lot of murmurs in the crowd.
• Oct 2014 – Kickstarter for Chocolatey started.
• Oct 2014 – Community Feed Moderation turned on. Thomas Walter handles a lion’s share of moderation over 2015 so others can concentrate on the framework.
• November 2014 – Chocolatey version 0.9.8.27 is the first package version alone with more than 1 million downloads. A record that isn’t broken again until October 2015 when 0.9.9.8 has 5.6 million downloads!
• February 2015 – Community repository hits 10 million downloads.
• March 3, 2015 – First C# Chocolatey version is released.
• March 23, 2015 – dtgm modernizes the automatic package process to achieve high quality automatic packages with checksums.
• October 2015 – CloudFlare and caching are introduced to the community repository to reduce pressure.
• November 2015 – Memcached is introduced to the community repository for a single cache across load balancers.
• December 2015 – Indexing is reintroduced to the community repository.
• Q4 2015 – Introduced the validator, the verifier, and the cleaner to the community feed to assist in moderation.
• January 2016 – Moderation backlog is reduced to near zero and is now manageable thanks to the automation.
• February 1, 2016 – First Professional Licenses of Chocolatey are shipped to Kickstarters.
• March 21, 2016 – CloudFlare caching tweaks introduced on the community repository to handle the increased pressure that will come from tab completion for package names.
• March 23, 2016 – Virus scan results shown on the community repository for packages.

This doesn’t represent everything that has happened. I tried to list out and attribute everything I could find and remember. There have been so many amazing package maintainers over the years, there are too many of you to possibly list. You know who you are. You have made the community what it is today and have been instrumental in shaping enhancements in Chocolatey.

Looking to the Future

The community has been amazing in helping Chocolatey grow and showing that there is a need that it fills. Package maintainers have put in countless and sometimes thankless hours to ensure community growth and consumers have really found the framework useful! Thank you so much! The next version of Chocolatey is coming and it is going to be amazing. Here’s to the next 5 years, may we change the world of Windows forever!

I’ve talked on this subject before and it’s something I often have in the back of my mind when reading comments. It’s come to the forefront recently with some conversations we’ve had at Western Devs, which led to our most recent podcast. I wasn’t able to attend so here I am.

There are certain phrases you see in comments that automatically seem to devolve a discussion. They include:

• “Why don’t you just…”
• “Sorry but…”
• “Can’t you just…”
• “It’s amazing that…”

Ultimately, all of these phrases can be summarized as follows:

I’m better than you and here’s why…

In my younger years, I could laugh this off amiably and say “Oh this wacky world we live in”. But I’m turning 44 in a couple of days and it’s time to start practicing my crotchety, even if it means complaining about people being crotchety.

So to that end: I’m asking, nay, begging you to avoid these and similar phrases. This is for your benefit as much as the reader’s. These phrases don’t make you sound smart. Once you use them, it’s very unlikely anyone involved will feel better about themselves, let alone engage in any form of meaningful discussion. Even if you have a valid point, who wants to be talked down to like that? Have you completely forgot what it’s like to learn?

“For fuck’s sake, Mom, why don’t you just type the terms you want to search for in the address bar instead of typing WWW.GOOGLE.COM into Bing?”

Now I know (from experience) it’s hard to fight one’s innate sense of superiority and the overwhelming desire to make it rain down on the unwashed heathen. So take it in steps. After typing your comment, remove all instances of “just” (except when just means “recently” or “fair”, of course). The same probably goes for “simply”. It has more of a condescending tone than a dismissive one. “Actually” is borderline. Rule of thumb: Don’t start a sentence with it.

Once you have that little nervous tic under control, it’s time to remove the negatives. Here’s a handy replacement guide to get you started:

See the difference? Instead of saying Sweet Zombie Jayzus, you must be the stupidest person on the planet for doing it this way, you’ve changed the tone to Have you considered this alternative? In both instances, you’ve made your superior knowledge known but in the second, it’s more likely to get acknowledged. More importantly, you’re less likely to look like an idiot when the response is: I did consider that avenue and here are legitimate reasons why I decided to go a different route.

To be fair, sometimes the author of the work you’re commenting on needs to be knocked down a peg or two themselves. I have yet to meet one of these people who respond well to constructive criticism critique, let alone the destructive type I’m talking about here. Generally, I find they feel the need to cultivate an antagonistic personality but in my experience, they usually don’t have the black turtlenecks to pull it off. Usually, it ends up backfiring and their dismissive comments become too easy to dismiss over time.

Kyle the Inclusive

Originally posted to: http://www.westerndevs.com/communication/Why-can-t-you-just/

Average approval time for moderated packages is currently under 10 hours!

In my last post, I talked about things we were implementing or getting ready to implement to really help out with the process of moderation.  Those things are:

• The validator – checks the quality of the package
• The verifier – tests the package install/uninstall and provides logs
• The cleaner – provides reminders and closes packages under review when they have gone stale.

The Cleanup Service

We’ve created a cleanup service, known as the cleaner that went into production recently.

• It looks for packages under review that have gone stale – defined as 20 or more days since last review and no progress
• Sends a notice/reminder that the package is waiting for the maintainer to fix something and that if another 15 days goes by with no progress, the package will automatically be rejected.
• 15 days later if no progress is made, it automatically rejects packages with a nice message about how to pick things back up later when the maintainer is ready.

Current Backlog

We’ve found that with all of this automation in place, the moderation backlog was quickly reduced and will continue to be manageable.

A visual comparison:

December 18, 2015 – 1630 packages ready

January 16, 2016 – 7 packages ready

Note the improvements all around! The most important numbers to key in on are the first 3, they represent a waiting for reviewer to do something status. With the validator and verifier in place, moderation is much faster and more accurate, and the validator has increased package quality all around with its review!

The waiting for maintainer (927 in the picture above) represents the bulk of the total number of packages under moderation currently. These are packages that require an action on the part of the maintainer to actively move the package to approved. This is also where the clean up service comes in.

The cleaner sent 800+ reminders two days ago. If there is no response by early February on those packages, the waiting for maintainer status will drop significantly as those packages will automatically be rejected. Some of those packages have been waiting for maintainer action for over a year and are likely abandoned. If you are a maintainer and you have not been getting emails from the site, you should log in now and make sure your email address is receiving emails and that the messages are not going to your spam folder. A rejected package version is reversible, the moderators can put it back to submitted at any time when a maintainer is ready to work on moving the package towards approval again.

Statistics

This is where it really starts to get exciting.

Some statistics:

• Around 30 minutes after a package is submitted the validator runs.
• Within 1-2 hours the verifier has finished testing the package and posts results.
• Typical human review wait time after a package is deemed good is less than a day now.

We’re starting to build statistics on average time to approval for packages that go through moderation that will be visible on the site.  Running some statistics by hand, we’ve approved 236 packages that have been created since January 1st, the average final good package (meaning that it was the last time someone submitted fixes to the package) to approval time has been 15 hours. There are some packages that drove that up due to fixing some things in our verifier and rerunning the tests. If I change to only looking at packages since those fixes have went in on the 10th, that is 104 packages with an average approval within 7 hours!

Regardless, here we are and it’s already been a positive move overall, not least because the move to Node means more of us are available to help with the maintenance of the site. But it wasn’t without it’s challenges. So I’m going to outline the major ones we faced here in the hopes that it will help you make your decision more informed than ours was.

To preface this, note that I’m new to Node and in fact, this is my first real project with it. That said, I’m no expert in Ruby either, which is what Jekyll is written in. And the short version of my first impressions is: Jekyll feels more like a real product but I had an easier time customizing Hexo once I dug into it. Here’s the longer version

DOCUMENTATION/RESOURCES

You’ll run into this very quickly. Documentation for Hexo is decent but incomplete. And once you start Googling, you’ll discover many of the resources are in Chinese. I found very quickly that there isposts collection and that each post has a categories collection. But as to what these objects look like, I couldn’t tell. They aren’t arrays. And you can’t JSON.stringify them because they have circular references in them. util.inspect works but it’s not available everywhere.

MULTI-AUTHOR SUPPORT

By default, Hexo doesn’t support multiple authors. Neither does Jekyll, mind you, but we found apretty complete theme that does. In Hexo, there’s a decent package that gets you partway there. It lets you specify an author ID on a post and it will attach a bunch of information to it. But you can’t, for example, get a full list of authors to list on a Who We Are page. So we created a separate data file for the authors. But we also haven’t figured out how to use that file to generate a .json file to use for the Featured section on the home page. So at the moment, we have author information in three places. Our temporary solution is to disallow anyone from joining or leaving Western Devs.

CUSTOMIZATION

If you go with Hexo and choose an existing themes, you won’t run into the same issues we did. Out of the box, it has good support for posts, categories, pagination, even things like tags and aliases with the right plugins.

But we started from a design and were migrating from an existing site with existing URLs and had to make it work. I’ve mentioned the challenge of multiple authors already. Another one: maintaining our URLs. Most of our posts aren’t categorized. In Jekyll, that means they show up at the root of the site. In Hexo, that’s not possible. At least at the moment and I suspect this is a bug. We eventually had to fork Hexo itself to maintain our existing URLs.

Another challenge: excerpts. In Jekyll, excerpts work like this: Check the front matter for an excerpt. If one doesn’t exist, take the first few characters from the post. In Hexo, excerpts are empty by default. If you add a <!--more--> tag in your post, everything before that is considered an excerpt. If you specify an excerpt in your front matter, it’s ignored because there is already an excerptproperty on your posts.

Luckily, there’s a plugin to address the last point. But it still didn’t address the issue of all our posts without an excerpt where we relied solely on the contents of the post.

So if you’re looking to veer from the scripted path, be prepared. More on this later in the “good parts” section.

OVERALL FEELING OF RAWNESS

This is more a culmination of the previous issues. It just feels like Hexo is a work-in-progress whereas Jekyll feels more like a finished product. There’s a strong community behind Jekyll and plenty of help. Hexo still has bugs that suggest it’s just not used much in the wild. Like rendering headers with links in them. It makes the learning process a bit challenging because with Jekyll, if something didn’t work, I’d think I’m obviously doing something wrong. With Hexo, it’s I might be doing something wrong or there might be a bug.

THE GOOD PARTS

I said earlier that the move to Hexo was positive overall and not just because I’m optimistic by nature. There are two key benefits we’ve gained just in the last two weeks.

GENERATION TIME

Hexo is fast, plain and simple. Our old Jekyll site took six seconds to generate. Doesn’t sound like much but when you’re working on a feature or tweaking a post, then saving, then refreshing, then rinsing, then repeating, that six seconds adds up fast. In Hexo, a full site generation takes three seconds. But more importantly, it is smart enough to do incremental updates while you’re working on it. So if you run hexo server, then see a mistake in your post, you can save it, and the change will be reflected almost instantly. In fact, it’s usually done by the time I’ve switched back to the browser.

CONTRIBUTORS

We had logistical challenges with Jekyll. To the point where we had two methods for Windows users that wanted to contribute (i.e. add a post). One involved a Docker image and the other Azure ARM. Neither was ideal as they took between seconds and minutes to refresh if you made changes. Granted, both methods furthered our collective knowledge in both Docker and Azure but they both kinda sucked for productivity.

That meant that realistically, only the Mac users really contributed to the maintenance of the site. And our Docker/Azure ARM processes were largely ignored as we would generally just test in production. I.e. create a post, check it in, wait for the site to deploy, make necessary changes, etc, etc.

With the switch to Hexo, we’ve had no fewer than five contributors to the site’s maintenance already. Hexo just works on Windows. And on Mac. Best of both worlds.

CUSTOMIZATION

This is listed under the challenges but ever the optimist, I’m including it here as well. We’ve had to make some customizations for our site, including forking Hexo itself. And for me personally, once I got past the why isn’t this working the way I want? stage, it’s been a ton of fun. It’s crazy simple to muck around in the node modules to try stuff out. And just as simple to fork something and reference it in your project when the need arises. I mentioned an earlier issue rendering links in headers. No problem, we just swapped out the markdown renderer for another one. And if that doesn’t work, we’ll tweak something until it does.

I want to talk more on specific conversion issues we ran into as a guide to those following in our footsteps. But there are enough of them to warrant a follow up post without all this pre-amble. For now, we’re all feeling the love for Hexo. So much so that no less than three other Western Devs are in the process of converting their personal blogs to it.

Originally posted to: http://www.westerndevs.com/jekyll/hexo/Migrating-from-Jekyll-to-Hexo/

We’ve added a few things to Chocolatey.org (the community feed) to help speed up review times for package maintainers. A little over a year ago we introduced moderation for all new package versions (besides trusted packages) and from the user perspective it has been a fantastic addition. The usage has went up by over 20 million packages installed in one year versus just 5 million the 3 years before it! It’s been an overwhelming response for the user community. Let me say that again for effect: Chocolatey’s usage of community packages has increased 400% in one year over the prior three years combined!

But let’s be honest, we’ve nearly failed in another area. Keeping the moderation backlog low. We introduced moderation as a security measure for Chocolatey’s community feed because it was necessary, but we introduced it too early. We didn’t have the infrastructure automation in place to handle the sheer load of packages that were suddenly thrown at us. And once we put moderation in place, more folks wanted to use Chocolatey so it suddenly became much more popular. And because we have automation surrounding updating and pushing packages (namely automatic packages), we had some folks who would submit 50+ packages at a time. With one particular maintainer submitting 200 packages automatically, and a review of each of them taking somewhere between 2-10 minutes, you don’t have to be a detective to understand how this is going to become a consternation. And from the backlog you can see it really hasn’t worked out well.

The most important number to understand here is the number in the submitted (underlined). This is the number of packages where a moderator has not yet looked at a package. A goal is to keep this well under 100. We want that time from a high quality package getting submitted to approved within 1-2 days.

Moderation has up until recently been a very manual process. Sometimes depending on which moderator that looked at your package determined whether it was going to be held in review for various reasons. We’ve added moderators and we’ve added more guidance around moderation to help bring a more structured review process. But it’s not enough.

Some of you may not know this, but our moderators are volunteers and we currently lack full-time employees to help fix many of the underlying issues. Even considering that we’ve also needed to work towards Kickstarter delivery and the Chocolatey rewrite (making choco better for the long term), it’s still not the greatest news to know that it has taken a long time to fix moderation, but hopefully it brings some understanding. Our goal is to eventually bring on full-time employees but we are not there yet. The Kickstarter was a start, but it was just that. A kick start. A few members of the core team who are also moderators have focused on ensuring the Kickstarter turns into a model that can ensure the longevity of Chocolatey. It may have felt that we have been ignoring the needs of the community, but that has not been our intention at all. It’s just been really busy and we needed to address multiple areas surrounding Chocolatey with a small number of volunteers.

So What Have We Fixed?

All moderation review communication is done on the package page. Now all review is done on the website, which means that there is no email back and forth (the older process) and what looks like one-sided communication on the site. This is a significant improvement.

Package review logging. Now you can see right from the discussion when and who submits package, when statuses change and where the conversation is.

More moderators. A question that comes up quite a bit surrounds the number of moderators that we have and adding more. We have added more moderators. We are up to 12 moderators for the site. Moderators are chosen based on building trust, usually through being extremely familiar with Chocolatey packaging and what is expected of approved packages. Learning what is expected usually comes through having your own packages approved and having a few packages. We’ve written most of this up at https://github.com/chocolatey/choco/wiki/Moderation.

Maintainers can self-reject packages that no longer apply. Say your package has a download url for the software that is always the same. You have some older package versions that could take advantage of being purged out of the queue since they are no longer applicable.

The package validation service (the validator). The validator checks the quality of a package based on requirements, guidelines and suggestions for creating packages for Chocolatey’s community feed. Many of the validation items will automatically roll back into choco and will be displayed when packaging a package. We like to think of the validator as unit testing. It is validating that everything is as it should be and meets the minimum requirements for a package on the community feed.

The package verifier service (the verifier). The verifier checks the correctness (that the package actually works), that it installs and uninstalls correctly, has the right dependencies to ensure it is installed properly and can be installed silently. The verifier runs against both submitted packages and existing packages (checking every two weeks that a package can still install and sending notice when it fails). We like to think of the verifier as integration testing. It’s testing all the parts and ensuring everything is good. On the site, you can see the current status of a package based on a little colored ball next to the title. If the ball is green or red, the ball is a link to the results (only on the package page, not in the list screen).

• Green means good. The ball is a link to the results
• Orange if still pending verification (has not yet run).
• Red means it failed verification for some reason. The ball is a link to the results.
• Grey means unknown or excluded from verification (if excluded, a reason will be listed on the package page).

Coming Soon – Moderators will be automatically be assigned to backlog items. Once a package passes both validation and verification, a moderator is automatically assigned to review the package. Once the backlog is in a manageable state, this will be added.

What About Maintainer Drift?

Many maintainers come in to help out at different times in their lives and they do it nearly always as volunteers. Sometimes it is the tools they are using at the current time and sometimes it has to do with where they work. Over time folks’ preferences/workplaces change and so maintainers drift away from keeping packages up to date because they have no internal incentive to continue to maintain those packages. It’s a natural human response. I’ve been thinking about ways to reduce maintainer drift for the last three years and I keep coming back to the idea that consumers of those packages could come along and provide a one time or weekly tip to the maintainer(s) as a thank you for keeping package(s) updated. We are talking to Gratipay now – https://github.com/gratipay/inside.gratipay.com/issues/441 This, in addition to a reputation system, I feel will go a long way to help reduce maintainer drift.

Final Thoughts

Package moderation review time is down to mere seconds as opposed to minutes like before. This will allow a moderator to review and approve package versions much more quickly and will reduce our backlog and keep it lower.

It’s already working! The number in the unreviewed backlog are down by 1,000 from the month prior. This is because a moderator doesn’t have to wait until a proper time when they can have a machine up and ready for testing and in the right state. Now packages can be reviewed faster. This is only with the verifier in place, sheerly testing package installs. The validator expects to cut that down to near seconds of review time. The total number of packages in the moderation backlog have also been reduced, but honestly I only usually pay attention to the unreviewed backlog number as it is the most important metric for me.

The verifier has rolled through over 6,500 verifications to date! https://gist.github.com/choco-bot/

We sincerely hope that the current maintainers who have been waiting weeks and months to get something reviewed can be understanding that we’ve dug ourselves into a moderation mess and are currently finding our way out of this situation. We may have some required findings and will ask for those things to be fixed, but for anything that doesn’t have required findings, we will approve them as we get to them.

I was asked recently for some advice on generating data for use with integration and UI tests. I already have some ideas but asked the rest of the Western Devs for some elucidation. My tl;dr version is the same as what I mentioned in our discussion on UI testing: it’s hard. But manageable. Probably.

The solution needs to balance a few factors:

• Each test must start from a predictable state
• Creating that predictable state should be fast as possible
• Developers should be able to figure out what is going on by reading the test

The two options we discussed both assume the first factor to be immutable. That means you either clean up after yourself when the test is finished or you wipe out the database and start from scratch with each test. Cleaning up after yourself might be faster but has more moving parts. Cleaning up might mean different things depending on which step you’re in if the test fails.

So given that we will likely re-create the database from scratch before each and every test, there are two options. My current favourite solution is a hybrid of the two.

Maintain a database of known data

In this option, you have a pre-configured database. Maybe it’s a SQL Server .bak file that you restore before each test. Maybe it’s a GenerateDatabase method that you execute. I’ve done the latter on a Google App Engine project, and it works reasonably well from an implementation perspective. We had a class for each domain aggregate and used dependency injection. So adding a new test customer to accommodate a new scenario was fairly simple. There are a number of other ways you can do it, some of which Simon touched on in his post.

We also had it set up so that we could create only the customer we needed for that particular test if we needed to. That way, we could use a step likeGiven I'm logged into 'Christmas Town' and it would set up only that data.

There are some drawbacks to this approach. You still need to create a new class for a new customer if you need to do something out of the ordinary. And if you need to do something only slightly out of the ordinary, there’s a strong tendency to use an existing customer and tweak its data ever so slightly to fit your test’s needs, other tests be damned. With these tests falling firmly in the long-running category, you don’t always find out the effects of this until much later.

Another drawback: it’s not obvious in the test exactly what data you need for that specific test. You can accommodate this somewhat just with a naming convention. For example,Given I'm logged into a company from India, if you’re testing how the app works with rupees. But that’s not always practical. Which leads us to the second option.

Create an API to set up the data the way you want

Here, your API contains steps to fully configure your database exactly the way you want. For example:

Given I have a company named "Christmas Town" owned by "Jack Skellington"
And I have 5 product categories
And I have 30 products
And I have a customer
...

You can probably see the major drawback already. This can become very verbose. But on the other hand, you have the advantage of seeing exactly what data is included which is helpful when debugging. If your test data is wrong, you don’t need to go mucking about in your source code to fix it. Just update the test and you’re done.

Also note the lack of specifics in the steps. Whenever possible, I like to be very vague when setting up my test data. If you have a good framework for generating test data, this isn’t hard to do. And it helps uncover issues you may not account for using hard-coded data (as anyone named D’Arcy O’Toole can probably tell you).

Loading up your data with a granular API isn’t realistic which is why I like the hybrid solution. By default, you pre-load your database with some common data, like lookup tables with lists of countries, currencies, product categories, etc. Stuff that needs to be in place for the majority of your tests.

After that, your API doesn’t need to be that granular. You can use something likeGiven I have a basic company which will create the company, add an owner and maybe some products and use that to test the process for creating an order. Under the hood, it will probably use the specific steps.

One reason I like this approach: it hides only the details you don’t care about. When you sayGiven I have a basic company and I change the name to "Rick's Place", that tells me, “I don’t care how the company is set up but the company name is important”. Very useful to help narrow the focus of the test when you’re reading it.

This approach will understandably lead to a whole bunch of different methods for creating data of various sizes and coarseness. And for that you’ll need to…

Maintain test data

Regardless of your method, maintaining your test data will require constant vigilance. In my experience, there is a tremendous urge to take shortcuts when it comes to test data. You’ll re-use a test company that doesn’t quite fit your scenario. You’ll alter your test to fit the data rather than the other way around. You’ll duplicate a data setup step because your API isn’t discoverable.

Make no mistake, maintaining test data is work. It should be treated with the same respect and care as the rest of your code. Possibly more so since the underlying code (in whatever form it takes) technically won’t be tested. Shortcuts and bad practices should not be tolerated and let go because “it’s just test data”. Fight the urge to let things slide. Call it out as soon as you see it. Refactor mercilessly once you see opportunities to do so.

Don’t be afraid to flip over a table or two to get your point across.

– Kyle the Unmaintainable

Some days/weeks/time ago, I did a presentation at MeasureUP called “Docker For People Who Think Docker Is This Weird Linux Thing That Doesn’t Impact Me”. The slides for that presentation can be found here and the sample application here.

// Uncomment for InMemory Storage
builder.RegisterAssemblyTypes(typeof(Data.InMemory.Repositories.RecipeRepository).Assembly)
       .AsImplementedInterfaces()
       .SingleInstance();

// Uncomment for PostgreSQL storage
builder.RegisterAssemblyTypes(typeof(PancakeProwler.Data.Postgres.IPostgresRepository).Assembly)
    .AsImplementedInterfaces().InstancePerRequest().PropertiesAutowired();

CREATE DATABASE "Pancakes";

CREATE USER "Matt" WITH PASSWORD 'moo';

GRANT ALL PRIVILEGES ON DATABASE "Pancakes" TO "Matt";

psql -U postgres -a -f "C:\path\to\sqlfile.sql"

docker-machine ip default

<add name="Postgres" connectionString="server=192.168.99.100;user id=Matt;password=moo;database=Pancakes" providerName="Npgsql" />

docker run --name my-postgres -e POSTGRES_PASSWORD=moo -p 5432:5432 -d postgres`

psql -h 192.168.99.100 -U postgres -a -f "C:\path\to\sqlfile.sql"

docker run --name elastic -p 9200:9200 -p 9300:9300 -d elasticsearch

It is a new Q&A style show, where folks submit questions on all things HTTP and Darrel and I give answers. Every show is live via Google Hangouts on Air, AND it is recorded and immediately available. In terms of the content, one thing I think that is really nice is we’re getting to dive into some really deep areas of building APIs that are not well covered. For example what level of granularity of media types should you use? Do Microservices impact your API design? And much more!

We’re not always in agreement, we’re not always right. We do always have fun!

Read Darrel’s blog post which goes into more detail of that what and why, then come join us! You can find the previous episodes on our YouTube channel here.

• The networking stack on OSX went downhill.
• I joined Splunk.
• I started using a VPN on my Mac (We use Juniper SSL VPN).
• I started having to deal with this now recurring nightmare of my DNS suddenly failing, generally after using the VPN.

If you use a VPN on a Mac, I am sure you’ve seen it. Suddenly you type “https://github.com” in your browser, and you get a 404. “Is Github down?” you ask your co-workers? “Nope, works perfectly fine for me”. “Is hipchat down?”. “Nope, I am chatting away.”

Meanwhile. your browser looks something like this:

AAARGH!

So you reboot, and then you find out that Github was up all along, the problem was your routing tables got screwed somehow related to the VPN, either that or the DNS demons have taken over your machine!

After dealing with this constantly, you start to seriously lose your sanity! It will of course always happen at the most inopportune time, like when you are about to present to your execs or walk on stage!

But my friends, there is hope, I have a cure! This is a cure I learned from the wise ninjas at my office (Thank you Danielle and Itay!), it is a little bash alias, and it will save you AND your DNS. Drop it in your .bash_profile and open a new terminal.

alias fixvpn="sudo route -n flush && sudo networksetup -setv4off Wi-Fi && sudo networksetup -setdhcp Wi-Fi"

Next time the DNS demons come to get you, run this baby from the shell. It will excommunicate those demons and quick.

Wait a few seconds, and bring up that webpage again.

You DNS and Sanity are restored!

It’s a preview

Like movie previews, this is equal parts exciting and frustrating. Exciting because you get a teaser of things to come. Frustrating because you just want it to work now. And extra frustration points for various technical issues I’ve run into that, I hope, are due to the “technical preview” label.

For example, installing container support into an existing VM is mind-numbingly slow. Kudos to the team for making it easy to install but at the point where you run ContainerSetup.ps1, be prepared to wait for, by my watch, at least 45 minutes without any visual indication that something is happening. The only reason I knew something was happening is because I saw the size of the VM go up (slowly) on my host hard drive. This is on a 70Mbps internet connection so I don’t think this can be attributed to “island problems” either.

I’ve heard tell of issues setting up container support in a Hyper-V VM as well. That’s second-hand info as I’m using Fusion on a Mac rather than Hyper-V. If you run into problems setting it up on Hyper-V, consider switching to the instructions for setting up containers on non-Hyper-V VMs instead.

There’s also the Azure option. Microsoft was gracious enough to provide an Azure image for Windows Server 2016 pre-configured with container support. This works well if you’re on Azure and I was able to run the nginx tutorial on it with no issues. I had less success with the IIS 10 tutorial even locally. I could get it running but was not able to create a new image based on the container I had.

It’s also a start

Technical issues aside, I haven’t been this excited about technology in Windows since…ASP.NET MVC, I guess, if my tag cloud is to be believed. And since this is a technical preview designed to garner feedback, here’s what I want to see in the Windows container world

Docker client and PowerShell support

I love that I can use the Docker client to work with Windows containers. I can leverage what I’ve already learned with Docker in Linux. But I also love that I can spin up containers with PowerShell so I don’t need to mix technologies in a continuous integration/continuous deployment environment if I already have PowerShell scripts set up for other aspects of my process.

Support for legacy .NET applications

I can’t take credit for this. I’ve been talking with Gabriel Schenker about containers a lot lately and it was he who suggested they need to have support for .NET 4, .NET 3.5, and even .NET 2.0. It makes sense though. There are a lot of .NET apps out there and it would be a shame if they couldn’t take advantage of containers.

Smooth local development

Docker Machine is great for getting up and running fast on a local Windows VM. To fully take advantage of containers, devs need to be able to work with them locally with no friction, whether that means a Windows Container version of Docker Machine or the ability to work with containers natively in Windows 10.

ARM support

At Western Devs, we have a PowerShell script that will spin up a new Azure Linux virtual machine, install docker, create a container, and run our website on it. It goes without saying (even though I’m saying it) that I’d like to do the same with Windows containers.

Lots of images out of the gate

I’d like to wean myself off VMs a little. I picture a world where I have one base VM and I use various containers for the different pieces of the app I’m working on. E.g. A SQL Server container, an IIS container, an ElasticSearch container, possibly even a Visual Studio container. I pick and choose which containers I need to build up my dev environment and use just one (or a small handful) of VMs.

In the meantime, I’m excited enough about Windows containers that I hope to incorporate a small demo with them in my talk at MeasureUP in a few scant weeks so if you’re in the Austin area, come on by to see it.

It is a glorious world ahead in this space and it puts a smile on this hillbilly’s face to see it unfold.

Kyle the Barely Contained

Western Devs uses Jekyll. Someone suggested it, I tried it, it worked well, decision was done. Except that it doesn’t work well on Windows. It’s not officially supported on the platform and while there’s a good guide on getting it running, we haven’t been able to do so ourselves. Some issue with a gem we’re using and Nokogiri and lib2xml and some such nonsense.

So in an effort to streamline things, Amir Barylko create a Docker image. It’s based on the Ruby base image (version 2.2). After grabbing the base image, it will:

• Install some packages for building Ruby
• Install the bundler gem
• Clone the source code into the /root/jekyll folder
• Run bundle install
• Expose port 4000, the default port for running Jekyll

With this in place, Windows users can run the website locally without having to install Ruby, Python, or Jekyll. The command to launch the container is:

docker run -t -p 4000:4000 -v //c/path/to/code:/root/jekyll abarylko/western-devs:v1 sh -c 'bundle install && rake serve'

This will:

• create a container based on the abarylko/western-devs:v1 image
• export port 4000 to the host VM
• map the path to the source code on your machine to /root/jekyll in the container
• run bundle install && rake serve to update gems and launch Jekyll in the container

To make this work 100%, you also need to expose port 4000 in VirtualBox so that it’s visible from the VM to the host. Also, I’ve had trouble getting a container working with my local source located anywhere except C:\Users\mysuername. There’s a permission issue somewhere in there where the container appears to successfully map the drive but can’t actually see the contents of the folder. This manifests itself in an error message that says Gemfile not found.

Now, Windows users can navigate to localhost:4000 and see the site running locally. Furthermore, they can add and make changes to their posts, save them, and the changes will get reflected in the browser. Eventually, that is. I’ve noticed a 10-15 second delay between the time you press Save to the time when the changes actually get reflected. Haven’t determined a root cause for this yet. Maybe we just need to soup up the VM.

So far, this has been working reasonably well for us. To the point, where fellow Western Dev, Dylan Smith has automated the deployment of the image to Azure via a Powershell script. That will be the subject of a separate post. Which will give me time to figure out how the thing works.

In the meantime, I have to temper my enthusiasm for the time being because Docker is still a Linux-only concern. Yes, you can run Docker containers on Windows but only Linux-based ones. So no SQL Server and no IIS.

But you can’t stop a hillbilly from dreaming of a world of containers. So with a grand assumption that you know what Docker is roughly all about, here’s what this coder of the earth meditates on, Docker-wise, before going to sleep.

Microservices

Microservices are a hot topic these days. We’ve talked about them at Western Devs already and Donald Belcham has a good and active list of resources. Docker is an eerily natural fit for microservices so much so that one might think it was created specifically to facilitate the architecture. You can package your entire service into a container and deploy it as a single package to your production server.

I don’t think you can understate the importance of a technology like Docker when it comes to microservices. Containers are so lightweight and portable, you just naturally gravitate to the pattern through normal use of containers. I can see a time in the near future where it’s almost negligent not to use microservices with Docker. At least in the Windows world. This might already be the case in Linux.

Works On My Machine

Ah, the crutch of the developer and the bane of DevOps. You set it up so nicely on your machine, with all your undocumented config entries and custom permissions and the fladnoogles and the whaztrubbets and everything else required to get everything perfectly balanced. Then you get your first bug from QA: can’t log in.

But what if you could test your deployment on the exact same image that you deployed to? Furthermore, what if, when a bug came in that you can’t reproduce locally, you could download the exact container where it was occurring? NO MORE EXCUSES, THAT’S WHAT!

Continuous Integration Build Agents

On one project, we had a suite of UI tests which took nigh-on eight hours in TeamCity. We optimized as much as we could and got it down to just over hours. Parallelizing them would have been a lot of effort to set up the appropriate shared resources and configurations. Eventually, we set up multiple virtual machines so that the entire parallel test run could finish in about an hour and a half. But the total test time of all those runs sequentially is now almost ten hours and my working theory is that it’s due to the overhead of the VMs on the host machine.

Offloading services

What I mean here is kind of like microservices applied to the various components of your application. You have an application that needs a database, a queue, a search components, and a cache. You could spin up a VM and install all those pieces. Or you could run a Postgres container, a RabbitMQ container, an ElasticSearch container, and a Redis container and leave your machine solely for the code.

When it comes right down to it, Docker containers are basically practical virtual machines. I’ve used VMs for many years. When I first started out, it was VMWare WorkStation on Windows. People that are smarter than me (including those that would notice that I should have said, “smarter than I”) told me to use them. “One VM per client” they would say. To the point that their host was limited to checking email and Twitter clients.

I tried that and didn’t like it. I didn’t like waiting for the boot process on both the host and each client and I didn’t like not taking full advantage of my host’s hardware on the specific client I happened to be working on at that moment.

But containers are lightweight. Purposefully so. Delightfully so. As I speak, the overworked USB drive that houses my VMs is down to 20 GB of free space. I cringe at the idea of having to spin up another one. But the idea of a dozen containers I can pick and choose from, all under a GB? That’s a development environment I can get behind.

Alas, this is mostly a future world I’m discussing. Docker is Linux only and I’m in the .NET space. So I have to wait until either: a) ASP.NET is ported over to Linux, or b) Docker supports Windows-based containers. And it’s a big part of my excitement that BOTH of those conditions will likely be met within a year.

In the meantime, who’s waiting? Earlier, I mentioned Postgres, Redis, ElasticSearch, and RabbitMQ. Those all work with Windows regardless of where they’re actually running. Furthermore, Azure already has pre-built containers with all of these.

Much of this will be the basis of my talk at the upcoming MeasureUP conference next month. So…uhhh….don’t read this until after that.