2019-07-22T18:50:48+00:00 http://castrojo.github.io 2018-04-23T00:00:00+00:00 http://castrojo.github.io/2018/04/23/video-conferencing <p>While video conferencing has been a real boon to productivity there are still <a href="https://www.youtube.com/watch?v=JMOOG7rWTPg">lots of things that can go wrong</a> during a conference video call.</p> <p>There are some things that are just plain out of your control, but there are some things that you can control. So, after doing these for the past 15 years or so, here are some tips if you’re just getting into remote work and want to do a better job. Of course I have been guilty of all of these. :D</p> <h3 id="stuff-to-have">Stuff to have</h3> <ul> <li> <p>Get a Microphone - Other than my desk, chair, and good monitors, this is the number one upgrade you can do. Sound is one of those things that can immediately change the quality of your call. I use a <a href="https://www.bluedesigns.com/products/yeti/">Blue Yeti</a> due to the simplicity of using USB audio, and having a hardware mute button. This way I know <em>for sure</em> I am muted when there’s a blinking red light in my face. Learn to use your microphone. On my Yeti you speak across the microphone and it has settings for where to pick up the noise from. Adjust these so it sounds correct. Get a <a href="https://en.wikipedia.org/wiki/Pop_filter">pop filter</a>.</p> </li> <li> <p>A Video Camera - Notice I put this second. I can get over a crappy image if the audio is good. The Logitech C-900 series has been my long go to standard for this. It also has dual noise cancelling microphones, which are great for backup (if you’re on a trip), but I will always default to the dedicated microphone.</p> </li> <li> <p>A decent set of headphones - Personal preference. I like open back ones for the home but pack a noise cancelling set for when I am on the road.</p> </li> </ul> <p>What about an integrated headset and microphone? This totally depends on the type. I tend to prefer the full sound of a real microphone but the boom mics on some of these headsets are quite good. If you have awesome heaphones already you can <a href="https://antlionaudio.com/">add a modmic</a> to turn them into headsets. I find that even the most budget dedicated headsets sound better than earbud microphones.</p> <h3 id="stuff-to-get-rid-of">Stuff to get rid of</h3> <ul> <li> <p>Your shitty earbuds - Seriously. If you’re going to be a remote worker <em>invest in respecting your coworker’s time</em>. A full hour long design session with you holding up a junky earbud microphone up to your face is not awesome for anybody. They’re fine if you want to use them for listening, but don’t use the mic.</p> </li> <li> <p>“But this iPhone was $1000, my earbud mic is fine.” Nope. You sound like crap.</p> </li> </ul> <h3 id="garbage-habits-we-all-hate">Garbage Habits we all hate</h3> <p>If you’re just dialing in to listen then most of these won’t apply to you, however …</p> <ul> <li> <p>Always join on muted audio. If the platform you use doesn’t do this by default find this setting and enable it.</p> </li> <li> <p>If you don’t have anything to say at that moment, MUTE. Even if you are just sitting there you’re adding ambient noise to the meeting, and when it gets over 10 people this really, really, sucks. This is why I love having a physical mute button, you can always be sure at a glance without digging into settings. I’ve also used a USB switch pedal for mute with limited success.</p> </li> <li> <p>Jumping in from a coffee shop, your work’s cafeteria, or any other place where there’s noise is not cool. And if you work in an open office all you’re doing is broadcasting to everyone else in the room that your place of employment doesn’t take developer productivity seriously.</p> </li> <li> <p>“Oh I will use my external speakers and built in microphone and adjust the levels and it will sound fine.” - No, it won’t, you sound like a hot mess, put on your headset and use the microphone.</p> </li> <li> <p>If you use your built-in microphone on your laptop and you start typing while you are talking EVERYBODY WILL HATE YOU.</p> </li> <li> <p>If you’re going to dial in from the back on an Uber or from a bus, and you have to talk or present, just don’t come. Ask someone to run the meeting for you or reschedule. You’re just wasting everyone’s time if you think we want to hear you sprinting down a terminal to catch your flight.</p> </li> <li> <p>And if you’re that person sitting on the plane in the meeting and people have to hear whatever thing you’re working on, they will hate you for the entire flight.</p> </li> </ul> <h2 id="treat-video-conferencing-like-you-do-everything-else-at-work">Treat video conferencing like you do everything else at work</h2> <p>We invest in our computers and our developer tools, think seriously about putting your video conferencing footprint in that namespace. There is a good chance no one will notice that you always sound good, but it’s one of those background quality things that just makes everyone more productive. Besides, think of the money you’ve spent on your laptop and everything else to make you better at work, better audio gear is a <em>good investment</em>.</p> <p>In the real world, sometimes you just have to travel and you find yourself stuck on a laptop on hotel wireless in a corner trying to your job, but I strive to make that situation the exception!</p> 2018-04-10T00:00:00+00:00 http://castrojo.github.io/2018/04/10/kubernetes-ama <p>A bunch of Kubernetes developers are doing an <a href="https://redd.it/8b7f0x">Ask Me Anything</a> today on Reddit if you’re interested in asking any questions, hope to see you there!</p> 2018-01-29T00:00:00+00:00 http://castrojo.github.io/2018/01/29/updating-your-cncf-developer-affiliation <p>The <a href="http://cncf.io">Cloud Native Computing Foundation</a> uses <a href="https://github.com/cncf/gitdm">gitdm</a> to figue out who is contributing and from where. This is used to generate reports and so forth.</p> <p>There is a <a href="https://github.com/cncf/gitdm/blob/master/developers_affiliations.txt">huge text file</a> where they are mapping email addresses used and affiliation. It probably doesn’t hurt to check your entry, for example, here’s mine:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Jorge O. Castro*: jorge.castro!gmail.com Lemon Ice Lemon Location City until 2017-05-01 Lemon Travel Smart Vacation Club until 2015-06-01 </code></pre></div></div> <p>Whoa? What? This is what a corrected entry looks like, as you can see it takes into account where you used to work for correctness:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Jorge O. Castro*: jorge!heptio.com, jorge!ubuntu.com, jorge.castro!gmail.com Heptio Canonical until 2017-03-31 </code></pre></div></div> <p>As an aside this also really makes a nice rolodex for looking up people. :D</p> 2017-10-17T16:09:35+00:00 http://castrojo.github.io/2017/10/17/using-kubeadm-to-upgrade-kubernetes <p>I’ve started writing for the Heptio Blog, check out my new article on <a href="https://blog.heptio.com/upgrading-to-kubernetes-1-8-with-kubeadm-797091ad2f9f">Upgrading to Kubernetes 1.8 with Kubeadm</a>.</p> <p>Also if you’re looking for more interactive help with Kubernetes, make sure you check out our brand new <a href="bit.ly/kubernetes-office-hours">Kubernetes Office Hours</a>, where we livestream developers answering user questions about Kubernetes. Starting tomorrow (18 October) at 1pm and 8pm UTC, hope to see you there!</p> <p><span style="font-size: 1rem;"> </span></p> 2017-10-05T15:41:49+00:00 http://castrojo.github.io/2017/10/05/lessons-learned-from-the-first-kubernetes-steering-election <p>The <a href="http://blog.kubernetes.io/2017/10/kubernetes-community-steering-committee-election-results.html">first steering committee election</a> for Kubernetes is now over.  Congratulations to Aaron Crickenberger, Derek Carr, Michelle Noorali, Phillip Wittrock, Quinton Hoole and Timothy St. Clair, who will be joining the newly formed Kubernetes Steering Committe.</p> <p>If you’re unfamiliar with what the SC does, you can check out <a href="https://github.com/kubernetes/steering/blob/master/charter.md">their charter</a> and <a href="https://github.com/kubernetes/steering/blob/master/backlog.md">backlog</a>. I was fortunate to work alongside <a href="https://twitter.com/ParisInBmore">Paris Pittman</a> on executing this election, hopefully the first of many “PB&amp;J Productions”.</p> <p>To give you some backstory on this, the Kubernetes community has been bootstrapping it’s governance over the past few years, and executing a proper election as stated in <a href="https://github.com/kubernetes/steering/blob/master/charter.md">the charter</a> was an important first step. Therefore it was critical for us to run an open election <em>correctly</em>.</p> <p>Thankfully we can stand on the shoulders of giants. <a href="https://governance.openstack.org/election/">OpenStack</a> and <a href="https://www.debian.org/vote/">Debian</a> are just two examples of projects with well formed processes that have stood the test of time. We then produced a <a href="https://github.com/kubernetes/community/tree/master/community/elections/2017">voter’s guide</a> to give people a place where they could find all the information they needed and the candidates a spot to fill in their platform statements.</p> <p>This morning I <a href="https://github.com/kubernetes/community/pull/1154/files">submitted a pull request</a> with our election notes and steps so that we can start building our institutional knowledge on the process, and of course, to share with whomever is interested.</p> <p>Also, a big shout out to Cornell University for providing <a href="http://civs.cs.cornell.edu/">CIVS</a> as a public service.</p> 2017-08-03T04:00:00+00:00 http://castrojo.github.io/2017/08/03/is-a-stateless-home-server-possible <p>I’ve recently embarked on a journey of trying new things in my homelab. I’ve been experimenting with new methods of managing my services at home, and I have tried a few combinations of methods and operating systems.</p> <p>Now that I am more familiar with the cloud native landscape, I was wondering if I could wean myself off the most stateful snowflake in my life, the trusty home server.</p> <p>I’ve been collecting hardware recently and decided to go out of my comfort zone and run some home services via different combinations. Here’s what I found out.</p> <h1 id="what-do-i-need-to-run">What do I need to run?</h1> <p>I have a few things I run in house that I need to host:</p> <ul> <li>A Ubiquiti Unifi controller, which controls my <a href="https://blog.jorgecastro.org/2016/12/29/unifis-new-cheaper-switches-are-great/">network gear</a></li> <li>A Ubiquiti Unifi Video controller, an entirely <a href="https://www.ubnt.com/download/unifi-video/unifi-nvr">different piece of software</a> for recording video from their line of IP cameras</li> <li><a href="https://pi-hole.net/">Pi-hole</a> for network wide privacy and ad blocking</li> <li>A <a href="https://www.plex.tv/">Plex Media Server</a> for my videos</li> </ul> <p>The Unifi stuff depends on Java and MongoDB, and accesses hardware on the network. Pi-hole expects to basically be my DNS server, and Plex is the textbook definition of a stateful app; depending on the size of your video collection it can grow a substantial database that’s all on disk, so moving around means bringing gigs of stuff with it.</p> <p>With this varied set of apps, we shall begin!</p> <h2 id="old-reliable-traditional-ubuntu-1604-with-debs">Old Reliable, traditional Ubuntu 16.04 with .debs</h2> <p>This has been working for me for years up to this point. As with anything involving third party packaging, maintenance can tend to get annoying.</p> <p>For Unifi you need either an external Mongo repo(!) and/or a OpenJDK PPA(!) to get it to work.</p> <p>Pi hole wants me to pipe a script to bash, and Plex just publishes one off debs with no repository, making that update a hassle.</p> <p>There are some benefits here, once I configure <code class="highlighter-rouge">unattended-upgrades</code> things generally run fine. It’s a well understood system, and having the large repository of software is always good. Over time it tends to accumulate crap though.</p> <h2 id="ubuntu-1604-with-docker-containers">Ubuntu 16.04 with Docker containers</h2> <p>The main advantage to this setup is I can crowdsource the maintenance of these apps to people who do a real good job, like the awesome guys at <a href="http://tools.linuxserver.io/dockers">linuxserver.io</a>. I can keep a <em>lean</em> and mostly stock host, toss away broken containers if need be, and keep everything nice and isolated.</p> <p>What ppa do I need for unifi? Is my plex up to date? What java symlink do I need to fix? I don’t need to care anymore!</p> <p><code class="highlighter-rouge">docker-compose</code> was really good for this and relatively simple to grok, especially by stealing other people’s configs from github and modifying them to my needs.</p> <h3 id="why-not-lxd-containers">Why not LXD containers?</h3> <p>I ran with this config for a while, but it had one major issue for me. LXD containers are system containers, that is, they run a copy of the OS. So now instead of maintaining one host I am now maintaining one host OS and many client OSes. Going into each one and installing/configuring the services felt like I was adding complexity. LXD is great, just not for my specific use case.</p> <h2 id="ubuntu-core-1604-with-docker-containers">Ubuntu Core 16.04 with Docker containers</h2> <p>Finally, something totally different. I am pretty sure “home server” doesn’t rank high on the use case here, but I figured I would give it a shot. I took the same <code class="highlighter-rouge">docker-compose</code> files from before, except this time I deploy on top of ubuntu-core.</p> <p>This gives me some nice features over the mutable-buntu. First off, atomic upgrades. Everytime it gets a new kernel it just reboots, and then on boot all the containers update and come back up.</p> <p>This has a few teething issues. First off, if there’s a kernel update it’s just going to reboot, you can’t really control that. Another is, it really is small, so it’s missing tools. It can only install snaps, so no rsync, no git, no curl, no wget. I’m not going to run git out of a docker container. Also I couldn’t figure out how to run docker as the non-root user and I can’t seem to find the documentation on how to do that anywhere.</p> <h2 id="container-linux-with-docker-containers">Container Linux with Docker containers</h2> <p>A slim OS designed to only run containers. This one definately has a more “work related” slant to it. There’s no normal installer, the installer basically takes a cloud-init-like yaml file and then dd’s the disk. Or just fire up your home PXE server. :) Most of the docs don’t even talk about how to configure the OS, the entire “state” is kept in this yaml file in git, it is expected that I can blow it away at any time and drop containers on it.</p> <p>This comes with git, rsync, and curl/wget out of the box, so it’s nice to be able to have these core tools in there instead of totally missing. There’s also a <code class="highlighter-rouge">toolbox</code> command that will automagically fetch a traditional distro container (defaults to fedora) and then mounts the filesystem inside, so you can <code class="highlighter-rouge">nano</code> to your heart’s content.</p> <p>This works really well. Container Linux lets me dictate the update policy as part of the config file, and if I have multiple servers I can cluster them together so that they will take turns rebooting without having a service go down. But as you can see, we quickly venture out of the “home server” use case with this one.</p> <h2 id="container-linux-with-rktsystemd">Container Linux with rkt/systemd</h2> <p>This is the setup I am enjoying the most. So instead of using the docker daemon, I create a systemd service file, like say <code class="highlighter-rouge">/etc/systemd/system/unifi.service</code>:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[Unit] Description=Unifi After=network.target [Service] Slice=machine.slice Type=simple ExecStart=/usr/bin/rkt --insecure-options=image run docker://linuxserver/unifi --volume config,kind=host,source=/home/jorge/config/unifi --mount volume=config,target=/config --net=host --dns=8.8.8.8 KillMode=mixed Restart=always [Install] WantedBy=multi-user.target </code></pre></div></div> <p>Then I <code class="highlighter-rouge">systemctl start unifi</code> to start it, and <code class="highlighter-rouge">systemctl enable unifi</code> to enable it on boot. ContainerOS is set to reboot on Thursdays at 4am, containers update on boot. I can use <code class="highlighter-rouge">journalctl</code> and <code class="highlighter-rouge">machinectl</code> like I can with “normal” services.</p> <p>Note that you can use this config on any OS that has systemd and rkt. Since ContainerOS has a section in it’s yaml file for writing systemd services, I can have one well maintained file that will just enable me to spit out and entire configured server in one shot. Yeah!</p> <p>This one “feels” like the most future proof, as the <a href="https://www.opencontainers.org/">OCI spec</a> is now finalized it feels like over time every tool will just be able to consume these images. I don’t know what that means for rkt and/or docker, but you can similarly <a href="http://container-solutions.com/running-docker-containers-with-systemd/">use docker in this manner as well</a>.</p> <h1 id="what-about-state">What about state?</h1> <p>So far I’ve only really talked about the “installation problem”, I’ve continually left out the hard part, the state of the applications themselves. Reinstalling coreos will get me all the apps back but no data, that doesn’t sound very cloud native!</p> <p>If you look at the systemd service file above, you see I keep the state in <code class="highlighter-rouge">/home/jorge/config/plex</code>. I do this for each of the services. I need to find a way to make sure that that is saved somewhere else than just local disk.</p> <p>Saving this onto an NFS share earned overwhelming NOPE NOPE NOPE from the straw poll I took (and one even threatened to come over and fight me). And that’s kind of cheating by moving the problem.</p> <p>So right now this is still up in the air, I fired up a quick <a href="https://www.duplicati.com/">duplicati</a> instance to keep a copy on S3.</p> <p>Really don’t want to add a ceph cluster to my home administration tasks. Suggestions here would be most welcome.</p> <h1 id="how-have-they-been-running">How have they been running?</h1> <p>I know what you’re thinking. Self rebooting servers and auto updating containers? You’re high.</p> <p>Surprisingly in the last three months I have been running all five of these things side by side and they’ve all been rock solid.</p> <p>I don’t know what to say here, some combination of great OS and container maintainers, or maybe not so much churn. I am going to try to keep these up and running as long as possible just to see what happens.</p> <h1 id="whats-left-to-try">What’s left to try?</h1> <p>The obvious hole here is the Project Atomic stack, which will be next on the list.</p> <p>And of course, it’s only a matter of time until one of these reboots breaks something or a container has a bad day.</p> <p>If you think this blog post isn’t crazy enough, Chuck and I will be <a href="https://gist.github.com/chuckbutler/92164d09fba51b2219b8bb8d4450761f">delving into Kubernetes</a> for this later on as this is all just a warm up.</p> 2017-07-21T11:00:50+00:00 http://castrojo.github.io/2017/07/21/tldring-your-way-to-a-kubernetes-bare-metal-cluster <p>Alex Ellis has an <a href="https://blog.alexellis.io/kubernetes-in-10-minutes/">excellent tutorial</a> on how to install Kubernetes in 10 minutes. It is a summarized version of what you can find in the <a href="https://kubernetes.io/docs/setup/independent/install-kubeadm/">official documentation</a>. Read those first, this is a an even shorter version with my choices mixed in.</p> <p>We’ll install 16.04 on some machines, I’m using three. I just chose to use Weave instead of sending you to a choose your-own-network page as you have other stuff to learn before you dive into an opinion on a networking overlay. We’re also in a lab environment so we assume some things like your machines are on the same network.</p> <h3 id="prep-the-operating-system">Prep the Operating System</h3> <p>First let’s take care of the OS. I set up automatic updates, ensure the latest kernel is installed, and then ensure we’re all up to date, whatever works for you:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo -s dpkg-reconfigure unattended-upgrades apt install linux-generic-hwe-16.04 apt update apt dist-upgrade reboot </code></pre></div></div> <h3 id="prep-each-node-for-kubernetes">Prep each node for Kubernetes:</h3> <p>This is just installing docker and adding the kubernetes repo, we’ll be root for these steps:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo -s curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - cat &lt;&lt;EOF &gt;/etc/apt/sources.list.d/kubernetes.list deb http://apt.kubernetes.io/ kubernetes-xenial main EOF apt update apt install -qy docker.io kubelet kubeadm kubernetes-cni </code></pre></div></div> <h3 id="on-the-master">On the master:</h3> <p>Pick a machine to be a master, then on that one:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubeadm init </code></pre></div></div> <p>And then follow the directions to copy your config file to your user account, we only have a few commands left needed with sudo so you can safely <code class="highlighter-rouge">exit</code> out and continue with your user account:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config </code></pre></div></div> <p>Let’s install the network, and then allow workloads to be scheduled on the master (for a lab we want to use all our hardware for workloads!):</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>kubectl apply -f https://git.io/weave-kube-1.6 kubectl taint nodes --all node-role.kubernetes.io/master- </code></pre></div></div> <h3 id="on-each-worker-node">On each worker node:</h3> <p>On each machine you want to be a worker (yours will be different, the output of <code class="highlighter-rouge">kubeadm init</code> will tell you what to do:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo kubeadm join --token 030b75.21ca2b9818ca75ef 192.168.1.202:6443 </code></pre></div></div> <p>You might need to tack on a <code class="highlighter-rouge">--skip-preflight-checks</code>, see <a href="https://github.com/kubernetes/kubeadm/issues/347">#347</a>, sorry for the inconvenience.</p> <h2 id="ensuring-your-cluster-works">Ensuring your cluster works</h2> <p>It shouldn’t take long for the nodes to come online, just check em out:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ kubectl get nodes NAME STATUS AGE VERSION dahl Ready 45m v1.7.1 hyperion NotReady 16s v1.7.1 tediore Ready 32m v1.7.1 $ kubectl cluster-info Kubernetes master is running at https://192.168.1.202:6443 KubeDNS is running at https://192.168.1.202:6443/api/v1/namespaces/kube-system/services/kube-dns/proxy To further debug and diagnose cluster problems, use 'kubectl cluster-info dump' $ </code></pre></div></div> <p>Ok you’re cluster is rocking, now</p> <h2 id="set-up-your-laptop">Set up your laptop</h2> <p>I don’t like to be ssh’ed into my cluster unless I’m doing maintenance, so now that we know stuff is working let’s copy the kubernetes config from the master node to our local workstation. You should know how to copy files around systems already, but here’s mine for reference:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> sudo scp /etc/kubernetes/admin.conf jorge@ivory.local:/home/jorge/.kube/config </code></pre></div></div> <p>I don’t need the entire Kubernetes repo on my laptop, so we’ll just install the snap for kubectl and check that I can access the server:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> sudo snap install kubectl --classic kubectl get nodes kubectl cluster-info </code></pre></div></div> <p>Don’t forget to <a href="https://blog.heptio.com/kubectl-shell-autocomplete-heptioprotip-48dd023e0bf3">turn on autocompletion</a>!</p> <h2 id="deploy-your-first-application">Deploy your first application</h2> <p>Let’s deploy the Kubernetes dashboard:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> kubectl create -f https://git.io/kube-dashboard kubectl proxy </code></pre></div></div> <p>Then hit up <a href="http://localhost:8001/ui">http://localhost:8001/ui</a>.</p> <p>That’s it, enjoy your new cluster!</p> <h2 id="joining-the-community">Joining the Community</h2> <p><code class="highlighter-rouge">kubeadm</code> is brought to you by SIG Cluster Lifecycle, they have <a href="https://docs.google.com/document/d/1deJYPIF4LmhGjDVaqrswErIrV7mtwJgovtLnPCDxP7U/edit#heading=h.rnqzadju87co">regular meetings</a> that anyone can attend, and you can give feedback on <a href="https://groups.google.com/forum/#!forum/kubernetes-sig-cluster-lifecycle">the mailing list</a>. I’ll see you there!</p> 2017-04-06T14:12:36+00:00 http://castrojo.github.io/2017/04/06/here-s-a-pic-from-ubuntu-down-under <p>While archiving a bunch of documents I found this pic of all of us from Ubuntu Down Under, thought I would share it!</p> <p><img src="/images/UDU_Group_Picture.jpg" alt="" /></p> 2017-03-20T10:16:26+00:00 http://castrojo.github.io/2017/03/20/lessons-learned-from-joining-the-kubernetes-community <p>Software can be complex and daunting, even more so in distributed systems. So when you or your company decide you’re going to give it a shot, it’s easy to get enamored with the technology and not think about the other things you and your team are going to need to learn to make participating rewarding for everyone.</p> <p>When we first launched the <a href="http://ubuntu.com/containers/kubernetes">Canonical Distribution of Kubernetes</a>, our team was new, and while we knew how Kubernetes worked, and what ops expertise we were going to start to bring to market right away, we found the initial huge size of the Kubernetes community to be outright intimidating. Not the people of course, they’re great, it’s the learning curve that can really seem large. So we decided to just dive in head first, and then write about our experiences. While some of the things I mention here work great for individuals, if you have a team on individuals working on Kubernetes at your company then I hope some of these tips will be useful to you. This is by no means an exhaustive list, I’m still finding new things everyday.</p> <h3 id="find-your-sigs">Find your SIGs</h3> <p>Kubernetes is divided into a bunch of Special Interest Groups(SIGs). You can <a href="https://github.com/kubernetes/community/blob/master/sig-list.md">find a list here</a>. Don’t be alarmed! Bookmark this page, I use this as my starting off point anytime we needed to find something out in more detail that we could find in the docs or public list. On this page, you’ll find contact information for the leads, and more importantly, when those SIGs meet. Meetings are open to public and (usually) recorded. Find someone on your team to attend these meetings regularly. This is important for a few reasons:</p> <ul> <li>k8s moves fast, and if there’s an area you care about, you can miss important information about a feature you care about.</li> <li>It’s high bandwidth since SIGs meet regularly, you won’t find long drawn out technical discussions on the mailing lists like you would on a project that only uses lists, these discussions move much faster when people talk face to face.</li> <li>You get to meet people and put faces to names.</li> <li>People get to see you and recognize your name (and optionally, your face). This will help you later on if you’re stuck and need help or if you want to start participating more.</li> <li>Each team has a slack channel and google group (mailing list), so I prefer to sit in those channels as well as they usually have important information announced there, meeting reminders, and links to the important documents for that SIG.</li> </ul> <h3 id="theres-a-sig-just-for-contributor-experience">There’s a SIG just for contributor experience</h3> <p><a href="https://github.com/kubernetes/community/blob/master/sig-contribx/README.md">SIG-contribex</a> - As it ends up there’s an entire SIG who work on improving contributor experience. I found this SIG relatively late when we started, you’ll find that asking quetions here will save you time in the long run. Even if you’re not asking questions yourself you can learn about how the mechanics of project works just by listening in on the conversation.</p> <h3 id="so-many-things-in-community">So many things in /community</h3> <p><a href="https://github.com/kubernetes/community">https://github.com/kubernetes/community</a> - This should be one of your first starting points, if not the starting point. I put the SIGs above this because I’ve found for most people they’re initially interested in one key area, and you can just go to that SIG directly first to get started then come back to this. That doesn’t mean this isn’t important, if I get lost in something this is usually the place I start to look for something. Try to get everyone on your team to have at least a working understanding of the concepts here, and of course don’t forget the CLA and Code of Conduct.</p> <h3 id="theres-a-youtube-channel">There’s a YouTube Channel</h3> <p><a href="https://www.youtube.com/c/KubernetesCommunity">https://www.youtube.com/c/KubernetesCommunity</a> - I found this channel to be very useful for “catching up”. Many SIGs publish their meetings relatively quickly, and tossing in the channel in the background can help you keep track of what’s going on.</p> <p>If you don’t have the time do dig into all the SIG meetings, you can concentrate on the <a href="https://www.youtube.com/playlist?list=PL69nYSiGNLP1pkHsbPjzAewvMgGUpkCnJ">weekly community meeting</a>, which is held weekly and a summary of many of things happening around the different SIGs. The community meetings also have demos, so it’s interesting to see how the ecosystem is building tools around k8s; if you can only make one meeting a week, this is probably the one to go to.</p> <h3 id="the-community-calendar-and-meetings">The Community Calendar and meetings</h3> <p>This sounds like advanced common sense but there’s a <a href="https://calendar.google.com/calendar/embed?src=nt2tcnbtbied3l6gi2h29slvc0%40group.calendar.google.com">community calendar</a> of events.</p> <p>Additionally, I found that adding the SIG meetings to our team calendar helps. We like to rotate people around meetings so that they can get experience in what is happening around the project and to ensure that worst case if someone can’t make a meeting someone is there to take notes. If you’re getting started, do yourself a favor <em>volunteer to take notes at a SIG meeting</em>, you will find that you’ll need to pay closer attention to the material and for me it helps me understand concepts better when I have to write it down in a way that makes sense for others.</p> <p>We also found it useful to not flood one meeting with multiple people. If it’s something important, sure, but if you just want to keep an eye on what’s going on there you can only send one person and then have that person give people a summary at your team standup or whatever. There are so many meetings that you don’t want to fall into the trap of having people sitting in meetings all day instead of getting things done.</p> <h2 id="owners">OWNERS</h2> <p>Whatever area you’re working on, go up the tree and eventually you’ll find an OWNERS file that list who owns/reviews that section of the code or docs or whatever. I use this as a little checklist when I join the SIG meetings to keep track of who is who. When I eventually went to a SIG meeting at Kubecon, it was nice to meet people who will be reviewing your work or you’ll be having a working relationship with.</p> <h2 id="find-a-buddy">Find a buddy</h2> <p>At some point you’ll be sitting in slack and you’ll see some poor person who is asking the same sorts of questions you were. That’s one of the first places you can start to help, just find someone and start talking to them. For me it was “Hey I noticed you sit in SIG-onprem too, you doing bare metal? How’s it going for you?”</p> <h2 id="its-too-big">It’s too big!</h2> <p>This used to worry me because the project is so large I figured I would never understand the entire thing. That’s ok. It’s totally fine to not know every single thing that’s going on, that’s why people have these meetings and summaries in the first place, just concentrate on what’s important to you and the rest will start to fall into place.</p> <h2 id="but-we-only-consume-kubernetes-why-participate">But we only consume Kubernetes, why participate?</h2> <p>One of the biggest benefits of consuming an open source project is taking advantage of the open development process. At some point something you care about will be discussed in the community then you should take advantage of the economies of scale that having so many people working on something gives you. Even if you’re only using k8s on a beautifully set up public cloud from a vendor where you don’t have to worry about the gruesome details, your organization can still learn from all the work that is happening around the ecosystem. I learn about new tools and tips every single day, and even if your participation is “read-only”, you’ll find that there’s value in sharing expertise with peers.</p> <h2 id="ongoing-process">Ongoing process</h2> <p>This post is already too long, so I’ll just have to keep posting more as I keep learning more. If you’ve got any tips to share please leave a comment, or post and send me a link to link to.</p> 2017-03-17T11:06:13+00:00 http://castrojo.github.io/2017/03/17/service-jorge-stop-git-rebase-heptio-master-service-jorge-start <p>After 9 years, 20 Ubuntu releases, <a href="http://askubuntu.com/users/235/jorge-castro?tab=activity&amp;sort=revisions">14,455 edits</a> and 359,186 miles in the air, it’s time for me to move on to something else. I’ve been privileged to work with some of the brightest people in the industry, and for that I’ll always be grateful.</p> <p>But I won’t be going far, I’ll be starting next month at <a href="http://heptio.com">Heptio</a>, where I will get to work with them in supporting and advancing <a href="http://kubernetes.io">Kubernetes</a>. The community is rapidly expanding, and I’m looking forward to contributing to the machinery that helps keep it a great place for people to participate. If you’ve not yet given k8s a spin, you should, it’s great stuff and only getting better every release.</p> <p>There are too many people who I’d like to thank, but you know who you are, and I’ll still see everyone around at conferences. I’ll be around for the next few weeks to tie things up, and then onward and upward!</p> 2017-03-08T08:52:33+00:00 http://castrojo.github.io/2017/03/08/deploying-kubernetes-on-aws-gce-and-bare-metal <p>As part of of the weekly <a href="https://github.com/kubernetes/community/blob/master/community/README.md">Kubernetes Community Meeting</a> Marco Ceppi deploys a fully functional Kubernetes cluster on AWS, GCE, and bare metal:</p> <iframe width="560" height="315" src="https://www.youtube.com/embed/PPDDYUc1iMQ" frameborder="0" allowfullscreen="">&amp;lt;/div&amp;gt;</iframe> <p>If you’re interested in bare metal Kubernetes, we invite you to join us and other contributors in the <a href="https://github.com/kubernetes/community/tree/master/sig-on-prem">sig-onprem</a>.</p> <p>Not sure where to get started? Check out our <a href="https://kubernetes.io/docs/getting-started-guides/ubuntu/">Getting Started documentation</a>.</p> 2017-02-27T08:30:08+00:00 http://castrojo.github.io/2017/02/27/canonical-distribution-of-kubernetes-release-1-5-3 <p>We’re proud to announce support for Kubernetes 1.5.3 in the Canonical Distribution of Kubernetes. This is a pure upstream distribution of Kubernetes, designed to be easily deployable to public clouds, on-premise (ie vsphere, openstack), bare metal, and developer laptops. Kubernetes 1.5.2 is a patch release comprised of mostly bugfixes, and we encourage you to <a href="https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md/#changelog-since-v152">check out the release notes</a>.</p> <h2 id="getting-started">Getting Started:</h2> <p>Here’s the simplest way to get a Kubernetes 1.5.3 cluster up and running on an Ubuntu 16.04 system:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo snap install conjure-up --classic conjure-up kubernetes </code></pre></div></div> <p>During the installation conjure-up will ask you what cloud you want to deploy on and prompt you for the proper credentials. If you’re deploying to local containers (LXD) see <a href="https://kubernetes.io/docs/getting-started-guides/ubuntu/local/">these instructions</a> for localhost-specific considerations.</p> <p>For production grade deployments and cluster lifecycle management it is recommended to read the <a href="https://kubernetes.io/docs/getting-started-guides/ubuntu/">full Canonical Distribution of Kubernetes documentation</a>.</p> <p>Home page: <a href="https://jujucharms.com/canonical-kubernetes/">https://jujucharms.com/canonical-kubernetes/</a></p> <p>Source code: <a href="https://github.com/juju-solutions/bundle-canonical-kubernetes">https://github.com/juju-solutions/bundle-canonical-kubernetes</a></p> <h2 id="how-to-upgrade">How to upgrade</h2> <p>With your kubernetes model selected, you can deploy the bundle to upgrade your cluster if on the 1.5.x series of kubernetes. At this time releases before 1.5.x have not been tested. Depending on which bundle you have previously deployed, run:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>    juju deploy canonical-kubernetes </code></pre></div></div> <p>or</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>    juju deploy kubernetes-core </code></pre></div></div> <p>If you have made tweaks to your deployment bundle, such as deploying additional worker nodes as a different label, you will need to manually upgrade the components. The following command list assumes you have made no tweaks, but can be modified to work for your deployment.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>juju upgrade-charm kubernetes-master juju upgrade-charm kubernetes-worker juju upgrade-charm etcd juju upgrade-charm flannel juju upgrade-charm easyrsa juju upgrade-charm kubeapi-load-balancer </code></pre></div></div> <p>This will upgrade the charm code, and the resources to kubernetes 1.5.3 release of the Canonical Distribution of Kubernetes.</p> <h2 id="new-features">New features:</h2> <ul> <li>Full support for Kubernetes v1.5.3.</li> <li>K8s master charm now properly keeps distributed master files in sync for an HA control plane.</li> </ul> <h2 id="general-fixes">General Fixes</h2> <ul> <li><a href="https://github.com/kubernetes/kubernetes/pull/41251">#41251</a> - Fix UpdateAddonsTactic to use local repo</li> <li><a href="https://github.com/kubernetes/kubernetes/pull/42058">#42058</a> - Fix shebangs in charm actions to use python3</li> <li><a href="https://github.com/kubernetes/kubernetes/pull/41815">#41815</a> - enable DefaultTolerationSeconds admission controller by default</li> <li><a href="https://github.com/kubernetes/kubernetes/pull/41351">#41351</a> - Multi master patch</li> <li><a href="https://github.com/kubernetes/kubernetes/pull/41256">#41256</a> - Lint fixes for the master and worker Python code</li> <li><a href="https://github.com/kubernetes/kubernetes/pull/41919">#41919</a> - Juju: Disable anonymous auth on kubelet (Thanks to community member @raesene for pointing this out!)</li> </ul> <h2 id="etcd-specific-fixes">etcd Specific Fixes</h2> <ul> <li><a href="https://github.com/juju-solutions/layer-etcd/pull/74">#74</a> - Add the ability to attach NRPE to etcd for monitoring with an external Nagios server.</li> <li><a href="https://github.com/juju-solutions/layer-etcd/pull/76">#76</a> - Add the debug action to the etcd charm that makes it easier to collect debug information in the field.</li> </ul> <h2 id="test-results">Test Results</h2> <p>The Canonical Distribution of Kubernetes is a running daily tests to verify it works with the upstream code. As part of the Kubernetes test infrastructructure we upload daily test runs. The test results are available on the dashboard. Follow along with our progress here:</p> <ul> <li><a href="https://k8s-gubernator.appspot.com/builds/canonical-kubernetes-tests/logs/kubernetes-gce-e2e-node/">https://k8s-gubernator.appspot.com/builds/canonical-kubernetes-tests/logs/kubernetes-gce-e2e-node/</a></li> </ul> <h2 id="how-to-contact-us">How to contact us:</h2> <p>We’re normally found in these Slack channels and attend these sig meetings regularly:</p> <ul> <li><a href="https://kubernetes.slack.com/messages/sig-cluster-lifecycle/">sig-cluster-lifecycle</a></li> <li><a href="https://kubernetes.slack.com/messages/sig-cluster-ops/">sig-cluster-ops</a></li> </ul> <p>Operators are an important part of Kubernetes, we encourage you to participate with other members of the Kubernetes community!</p> <p>We also monitor the Kubernetes mailing lists and other <a href="http://kubernetes.io/community/">community channels</a>, feel free to reach out to us. As always, PRs, recommendations, and bug reports are welcome: <a href="https://github.com/juju-solutions/bundle-canonical-kubernetes">https://github.com/juju-solutions/bundle-canonical-kubernetes</a></p> 2017-01-23T08:30:08+00:00 http://castrojo.github.io/2017/01/23/canonical-distribution-of-kubernetes-release-1-5-2 <p>We’re proud to announce support for Kubernetes 1.5.2 in the Canonical Distribution of Kubernetes. This is a pure upstream distribution of Kubernetes, designed to be easily deployable to public clouds, on-premise (ie vsphere, openstack), bare metal, and developer laptops. Kubernetes 1.5.2 is a patch release comprised of mostly bugfixes, and we encourage you to <a href="https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md#changelog-since-v151">check out the release notes</a>.</p> <h2 id="getting-started">Getting Started:</h2> <p>Here’s the simplest way to get a Kubernetes 1.5.2 cluster up and running on an Ubuntu 16.04 system:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo apt-add-repository ppa:juju/stable sudo apt-add-repository ppa:conjure-up/next sudo apt update sudo apt install conjure-up conjure-up kubernetes </code></pre></div></div> <p>During the installation conjure-up will ask you what cloud you want to deploy on and prompt you for the proper credentials. If you’re deploying to local containers (LXD) see <a href="https://kubernetes.io/docs/getting-started-guides/ubuntu/local/">these instructions</a> for localhost-specific considerations.</p> <p>For production grade deployments and cluster lifecycle management it is recommended to read the <a href="https://kubernetes.io/docs/getting-started-guides/ubuntu/">full Canonical Distribution of Kubernetes documentation</a>.</p> <p>Home page: <a href="https://jujucharms.com/canonical-kubernetes/">https://jujucharms.com/canonical-kubernetes/</a></p> <p>Source code: <a href="https://github.com/juju-solutions/bundle-canonical-kubernetes">https://github.com/juju-solutions/bundle-canonical-kubernetes</a></p> <h2 id="how-to-upgrade">How to upgrade</h2> <p>With your kubernetes model selected, you can deploy the bundle to upgrade your cluster if on the 1.5.x series of kubernetes. At this time releases before 1.5.x have not been tested. Depending on which bundle you have previously deployed, run:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>    juju deploy canonical-kubernetes </code></pre></div></div> <p>or</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>    juju deploy kubernetes-core </code></pre></div></div> <p>If you have made tweaks to your deployment bundle, such as deploying additional worker nodes as a different label, you will need to manually upgrade the components. The following command list assumes you have made no tweaks, but can be modified to work for your deployment.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>juju upgrade-charm kubernetes-master juju upgrade-charm kubernetes-worker juju upgrade-charm etcd juju upgrade-charm flannel juju upgrade-charm easyrsa juju upgrade-charm kubeapi-load-balancer </code></pre></div></div> <p>This will upgrade the charm code, and the resources to kubernetes 1.5.2 release of the Canonical Distribution of Kubernetes.</p> <h2 id="new-features">New features:</h2> <ul> <li>Full support for Kubernetes v1.5.2.</li> </ul> <h2 id="general-fixes">General Fixes</h2> <ul> <li> <p><a href="https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/151">#151</a> <a href="https://docs.google.com/document/d/1amyUwQSrDrSc28FZFFqW0rWnKUcm7nYax6_TXBAyirw/edit#">#187</a> It wasn’t very transparent to users that they should be using conjure-up when locally developing, conjure-up is now the defacto default mechanism for deploying CDK.</p> </li> <li> <p><a href="https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/173">#173</a> Resolved permissions on ~/.kube on kubernetes-worker units</p> </li> <li> <p><a href="https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/169">#169</a> Tuned the verbosity of the AddonTacticManager class during charm layer build process</p> </li> <li> <p><a href="https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/162">#162</a> Added NO_PROXY configuration to prevent routing all requests through configured proxy [by <a href="https://github.com/axinojolais">@axinojolais</a>]</p> </li> <li> <p><a href="https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/160">#160</a> Resolved an error by flannel sometimes encountered during cni-relation-changed [by <a href="https://github.com/spikebike">@spikebike</a>]</p> </li> <li> <p><a href="https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/172">#172</a> Resolved sporadic timeout issues between worker and apiserver due to nginx connection buffering [by <a href="https://github.com/axinojolais">@axinojolais</a>]</p> </li> <li> <p><a href="https://github.com/juju-solutions/kubernetes/pull/101">#101</a> Work-around for offline installs attempting to contact pypi to install docker-compose</p> </li> <li> <p><a href="https://github.com/juju-solutions/kubernetes/pull/95">#95 </a>Tuned verbosity of copy operations in the debug script for debugging the debug script.</p> </li> </ul> <h2 id="etcd-layer-specific-changes">Etcd layer-specific changes</h2> <ul> <li><a href="https://github.com/juju-solutions/layer-etcd/issues/72">#72</a> <a href="https://github.com/juju-solutions/layer-etcd/issues/70">#70</a> Resolved a certificate-relation error where etcdctl would attempt to contact the cluster master before services were ready [by <a href="https://github.com/javacruft">@javacruft</a>]</li> </ul> <h2 id="unfiledun-scheduled-fixes">Unfiled/un-scheduled fixes:</h2> <ul> <li><a href="https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/190">#190</a> Removal of assembled bundles from the repository. See bundle author/contributors notice below</li> </ul> <h2 id="additional-features">Additional Feature(s):</h2> <ul> <li>We’ve open sourced our release management process scripts we’re using in a juju deployed jenkins model. These scripts contain the logic we’ve been running by hand, and give users a clear view into how we build, package, test, and release the CDK. You can see these scripts in the <a href="https://github.com/juju-solutions/kubernetes-jenkins">juju-solutions/kubernetes-jenkins</a> repository. This is early work, and will continue to be iterated on / documented as we push towards the Kubernetes 1.6 release.</li> </ul> <h2 id="notice-to-bundle-authors-and-contributors">Notice to bundle authors and contributors:</h2> <p>The fix for <a href="https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/190">#190</a> is a larger change that has landed in the <a href="https://github.com/juju-solutions/bundle-canonical-kubernetes">bundle-canonical-kubernetes</a> repository. Instead of maintaining several copies across several repositories of a single use-case bundle; we are now assembling the CDK based bundles as fragments (un-official nomenclature).</p> <p>This affords us the freedom to rapidly iterate on a CDK based bundle and include partner technologies, such as different SDN vendors, Storage backend components, and other integration points. Keeping our CDK bundle succinct, and allowing the more complex solutions to be assembled easily, reliably, and repeatedly. This does change the contribution guidelines for end users.</p> <p>Any changes to the core bundle should be placed in its respective fragment under the <a href="https://github.com/juju-solutions/bundle-canonical-kubernetes/tree/master/fragments">fragments</a> directory. Once this has been placed/merged, the primary published bundles can be assembled by running <code class="highlighter-rouge">./bundle</code> in the root of the repository. This process has been outlined in the repository <a href="https://github.com/juju-solutions/bundle-canonical-kubernetes#bundle-builder-for-cdk">README.md</a></p> <p>We look forward to any feedback on how opaque/transparent this process is, and if it has any useful applications outside of our own release management process. The <code class="highlighter-rouge">./bundle</code> python script is still very much geared towards our own release process, and how to assemble bundles targeted for the CDK. However we’re open to generalizing them and encourage feedback/contributions to make this more useful to more people.</p> <h2 id="how-to-contact-us">How to contact us:</h2> <p>We’re normally found in these Slack channels and attend these sig meetings regularly:</p> <ul> <li> <p><a href="https://kubernetes.slack.com/messages/sig-cluster-lifecycle/">sig-cluster-lifecycle</a></p> </li> <li> <p><a href="https://kubernetes.slack.com/messages/sig-cluster-ops/">sig-cluster-ops</a></p> </li> </ul> <p>Operators are an important part of Kubernetes, we encourage you to participate with other members of the Kubernetes community!</p> <p>We also monitor the Kubernetes mailing lists and other <a href="http://kubernetes.io/community/">community channels</a>, feel free to reach out to us. As always, PRs, recommendations, and bug reports are welcome: <a href="https://github.com/juju-solutions/bundle-canonical-kubernetes">https://github.com/juju-solutions/bundle-canonical-kubernetes</a></p> 2017-01-10T19:34:12+00:00 http://castrojo.github.io/2017/01/10/fresh-kubernetes-documentation-available-now <p>Over the past few months our team has been working real hard on the <a href="https://www.ubuntu.com/cloud/kubernetes">Canonical Distribution of Kubernetes</a>. This is a pure-upstream distribution of k8s with our community’s operational expertise bundled in.</p> <p>It means that we can use one set of operational code to get the same deployment on GCE, AWS, Azure, Joyent, OpenStack, and Bare Metal.</p> <p>Like most young distributed systems, Kubernetes isn’t exactly famous for it’s ease of use, though there has been tremendous progress over the past 12 months. Our documentation on Kubernetes was nearly non-existent and it became obvious that we had to dive in there and bust it out. I’ve spent some time fixing it up and it’s been recently merged. </p> <p>You can find the <a href="http://kubernetes.io/docs/getting-started-guides/ubuntu/">Official Ubuntu Guides</a> in the “Create a cluster” section. We’re taking what I call a “sig-cluster-lifecycle” approach to this documentation – the pages are organized into lifecycle topics based on what an operator would do. So “Backups”, or “Upgrades” instead one big page with sections. This will allow us to grow each section based on the expertise we learn on k8s for that given task. </p> <p>Over the past few months (and hopefully for Kubernetes 1.6) we will slowly be phasing out the documentation on our individual charm and layer pages to reduce duplication and move to a pure upstream workflow. </p> <p>On behalf of our team we hope you enjoy Kubernetes, and if you’re running into issues please <a href="https://github.com/juju-solutions/bundle-canonical-kubernetes/issues">let us know</a> or you can find us in the Kubernetes slack channels.</p> 2017-01-01T00:00:00+00:00 http://castrojo.github.io/2017/01/01/useful-tool-alert-explainshell-dot-com <p>Ran into this while using the Stack Overflow; <a href="http://explainshell.com">explainshell.com</a>. Basically they take all of Ubuntu’s manpages and then parse them so you can paste in any Linux command and then see right away what each option does. Example:</p> <p><a href="http://explainshell.com/explain?cmd=rsync+-chavzP+--stats+user%40remote.host%3A%2Fpath%2Fto%2Fcopy+%2Fpath%2Fto%2Flocal%2Fstorage">rsync -chavzP –stats user@remote.host:/path/to/copy /path/to/local/storage</a></p> <p>It then takes the command you put in there and breaks down each of the flags. I like this for a few reasons. First of all, I hate reading manpages because I’m a human being. This shows me exactly what each option does without having to parse the entire manpage, it only tells me what I care about. Secondly, I prefer to learn by example instead of just reading manpages in their entirety. I can see this tool being very useful for people who are just starting to learn. Command on the internet confusing you? You can at least paste it in here and figure out what it’s doing before you end up being <a href="https://gist.github.com/KartikTalwar/4393116#gistcomment-1947694">that guy</a>.</p> <p>Another example is <a href="http://bropages.org/rsync">bropages</a>, which lets people submit example commands for each tool, and then users vote on the usefulness of the examples for a nice stackoverflow-like list of example commands. It seems as though they haven’t had a commit for almost a year, so not sure if people are still contributing to that, but it seems like a decent enough idea.</p> 2016-12-29T00:00:00+00:00 http://castrojo.github.io/2016/12/29/unifis-new-cheaper-switches-are-great <p>I started switching to Ubiquiti’s Unifi equipment at home when one of my coworkers, Sean Sosik-Hamor, recommended them for prosumer use. A little while later Lee Hutchinson published <a href="http://arstechnica.com/gadgets/2015/10/review-ubiquiti-unifi-made-me-realize-how-terrible-consumer-wi-fi-gear-is/">Ubiquiti Unifi made me realise how terrible consumer Wi-Fi gear is</a> when they launched their newer (and cheaper) line of 802.11ac access points. I’ve got one of those, as well as the <a href="https://www.ubnt.com/unifi-routing/usg/">USG</a> for routing duties. The USG isn’t something to write home about, but it gets the job done, and in some advanced cases you can always ssh to it, but generally speaking I just use it as intended and mostly setting it up and forgetting about it.</p> <p><img src="/images/unifi/unifi1.png" alt="unifi" /></p> <p>Unlike most routers, you don’t manage Unifi gear through a web UI on the device, you run controller software on a host and then the controller software blasts out the config and updates to the devices. I recommend reading Dustin Kirkland’s <a href="http://blog.dustinkirkland.com/2016/12/unifi-controller-in-lxd.html">blog post for running Unifi in LXD</a> as it currently is 14.04 only, and if you’re like me, you’re finding that it’s becoming much more manageable to keep server software nice and isolated in it’s own container instead of splatting all its dependencies on the host OS. If you prefer things more old school, look for the “EdgeRouter” line of routers and switches.</p> <p>At $99 for an AP and $149 for an access point you can come up with a nice business-grade combo, especially with the latest consumer routers starting to get close the $300(!) and utterly terrible software. The one thing that was always expensive though, was the Unifi line of managed switches. It’s nice, but at $199 for 8 ports, just too much for each port. Here’s a nice review from Lee on the <a href="http://arstechnica.com/gadgets/2016/04/ubiquitis-8-port-poe-switch-is-a-solid-complement-for-a-home-unifi-setup/">Unifi Switch 8</a>. Thanks to the wonder of their beta store, I was able to pick up the newer, slimmed down 8 port, the Unifi US-8:</p> <p><img src="/images/unifi/unifi8.jpg" alt="unifi8" /></p> <p>There it is, with the unmanaged switch it replaced. They dropped the SFP ports, and you can see the port LEDs are on the top instead of in each port, probably for cost? And since it’s Unifi, it plops in nicely with the UI, giving me some nice per-port stats:</p> <p><img src="/images/unifi/unifi3.png" alt="unifi3" /></p> <p>And it gets better, they’ve done a US-24 and US-48 as well. I put a US-24 in my basement. $215 all day for 24 ports, compared to the older model, which would go north of $500!</p> <p><img src="/images/unifi/unifi2.png" alt="unifi2" /></p> <p>I’m in the process of still setting up the homelab VLAN, so I don’t have much to report on that, but having everything managed in one system is a really great feature. I didn’t really need SFP plugs or lots of PoE power for my use, so this new low-end line is perfect for me, if you find yourself wanting cheap-but-good equipment with decent software, then I recommend you check them out, and of course drop by <a href="https://www.reddit.com/r/Ubiquiti/">/r/ubiquiti</a> if you need anything.</p> <p>See also Troy Hunt’s <a href="https://www.troyhunt.com/ubiquiti-all-the-things-how-i-finally-fixed-my-dodgy-wifi/">more indepth blog post</a> for more information.</p> 2016-12-20T00:00:00+00:00 http://castrojo.github.io/2016/12/20/new-blog-and-new-status-updates <p>I’m going to try to blog more in the coming new year, but I figured it would get an early start to get back into the swing of things.</p> <p>Things have been going at breakneck speed lately, so here’s the highlights.</p> <ul> <li>We released <a href="http://insights.ubuntu.com/2016/12/16/announcing-canonical-kubernetes-1-5-1/">The Canonical Distribution of Kubernetes 1.5.1</a>.</li> <li>I’m a new dad! <a href="https://goo.gl/photos/oUk6qBJsk2wVxTh97">Rafael Mateo Castro</a> was born on November 26th 2016. He’s totally healthy and is currently taking up most of my free time.</li> <li>Marco took some time to help launch and run ops for one of <a href="http://blog.silph.io/">the coolest Pokemon Go</a> sites around. Go itself is running on Kubernetes. Marco running this site is a complete coincidence. Make sure you check out that link for some solid gold ops information. All running on Juju of course!</li> <li>Speaking of coincidences, my brother and his wife had a son 6 days early than us, Leonardo Mehta Castro. Again, total coincidence that my son and his son are named after ninja turtles.</li> <li>I’ll be at <a href="http://cfgmgmtcamp.eu/">Config Management Camp</a> and FOSDEM this year, so hope to catch up with everyone there.</li> </ul> <p>It’s cold and wintery now, so here’s a video of Oscar being a super-dog.:</p> <iframe width="560" height="315" src="https://www.youtube.com/embed/o24YHckVOyY?rel=0" frameborder="0" allowfullscreen=""></iframe> 2016-11-16T10:00:00+00:00 http://castrojo.github.io/2016/11/16/kubernetes-the-easy-way <p>If you’re interested in running Kubernetes you’ve probably heard of Kelsey Hightower’s <a href="https://github.com/kelseyhightower/kubernetes-the-hard-way">Kubernetes the Hard Way</a>. Exercises like these are important, they highlight the coordination needed between components in modern stacks, and it highlights how far the world has come when it comes to software automation. Could you imagine if you had to set everything up the hard way every time?</p> <h2 id="learning-is-fun">Learning is fun</h2> <p>Doing things the hard way is fun, once. After that, I’ve got work to do, and soon after I am looking around to see who else has worked on this problem and how I can best leverage the best open source has to offer.</p> <p>It reminds me of the 1990’s when I was learning Linux. Sure, as a professional, you need to know systems and how they work, down to the kernel level if need. Having to do those things without a working keyboard or network makes that process much harder. Give me a working computer, and then I can begin. There’s value in learning how the components work together and understanding the architecture of Kubernetes, I encourage everyone to try the hard way at least one time, if anything it’ll make you appreciate the work people are putting into automating all of this for you in a composable and reusable way.</p> <h2 id="the-easy-way">The easy way</h2> <p>I am starting a new series of videos on how we’re making the Canonical Distribution of Kubernetes easy for anyone to deploy on any cloud. All our code is <a href="https://github.com/juju-solutions/bundle-canonical-kubernetes">open source</a> and we love pull requests. Our goal is to help people get Kubernetes in as many places as quickly and easily as possible. We’ve incorporated lots of the things people tell us they’re looking for in a production-grade Kubernetes, and we’re always looking to codify those best practices.</p> <p>Enjoy:</p> <iframe width="560" height="315" src="https://www.youtube.com/embed/B7nMFVaOOi8" frameborder="0" allowfullscreen=""></iframe> <p>Following these steps will get you a working cluster, in this example I’m deploying to <code class="highlighter-rouge">us-east-2</code>, the shiny new AWS region. Subsequent videos will cover how to interact with the cluster and do more things with it.</p> 2016-07-29T10:28:00+00:00 http://castrojo.github.io/2016/07/29/ubuntu-kubernetes-v1-dot-3-3-ready-for-testing <p>We’ve been trailing the <a href="http://blog.kubernetes.io/2016/07/kubernetes-1.3-bridging-cloud-native-and-enterprise-workloads.html">Kubernetes 1.3</a> release for the past few weeks, mostly to ensure that etcd data migrations are preserved from 1.2 to 1.3. We’re also in the process of adding TLS between all the nodes for security reasons, and that has led to use being a bit behind on getting Kubernetes 1.3 out to you, but don’t worry, we’re in the process of testing the upgrade path and this post will outline how to set up a Kubernetes 1.2 cluster and upgrade it to v1.3.3. Once we get good feedback from the community on how this is working out for you, we’ll go ahead and set v1.3.3 (or subsequent version) as the new default for Ubuntu Kubernetes.</p> <p>Our bundle, which we call “observable-kubernetes” features the following model:</p> <ul> <li>Kubernetes (automating deployment, operations, and scaling containers) <ul> <li>Three node Kubernetes cluster with one master and two worker nodes.</li> <li>TLS used for communication between nodes for security.</li> </ul> </li> <li>Etcd (distributed key value store) <ul> <li>Three node cluster for reliability.</li> </ul> </li> <li>Elastic stack <ul> <li>Two nodes for ElasticSearch</li> <li>One node for a Kibana dashboard</li> <li>Beats on every Kubernetes and Etcd node: <ul> <li>Filebeat for forwarding logs to ElasticSearch</li> <li>Topbeat for inserting server monitoring data to ElasticSearch</li> </ul> </li> </ul> </li> </ul> <p>As usual, you get pure Kubernetes direct from upstream and of course it’s cross-cloud, making it easy for you to use your own bare metal for deployment.</p> <h2 id="your-first-kubernetes-cluster">Your First Kubernetes Cluster</h2> <p>After <a href="https://jujucharms.com/docs/stable/getting-started">configuring Juju</a> to use the cloud you prefer we can start the cluster deployment.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>juju deploy observable-kubernetes </code></pre></div></div> <p>This will deploy the bundle with default constraints. This is great for testing out Kubernets but most clouds won’t give you enough CPU and memory to use the cluster in anger, so I recommend <a href="https://github.com/juju-solutions/bundle-observable-kubernetes">checking out the documentation</a> on how to modify the bundle to more accurately reflect either the hardware you have on hand, or the instance size you prefer.</p> <p>We can watch the cluster coming up with a <code class="highlighter-rouge">watch juju status</code>, this will give us a near-realtime view of the cluster as it comes up:</p> <p><img src="/images/kubes.png" alt="kubes" /></p> <h2 id="making-sure-your-cluster-works">Making sure your cluster works</h2> <p>We just wait for things to come up and hit an idle state before moving on. Once we’re up we can manage the cluster with <code class="highlighter-rouge">kubectl</code>. We’ve provided this tool for you on the master node with a config file prepoluated for you, first let’s find the master node.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>juju run --application kubernetes is-leader </code></pre></div></div> <p>The output will show you which node is the master node, you can then copy the tools to your local machine:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>juju scp kubernetes/0:kubectl_package.tar.gz . </code></pre></div></div> <p>Untar that wherever you’d like and cd to that directory, you should have a kubectl binary and a <code class="highlighter-rouge">kubeconfig</code> file for you to use along with <code class="highlighter-rouge">kubectl</code>. You can now check the status of your cluster with:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./kubectl cluster-info --kubeconfig ./kubeconfig Kubernetes master is running at https://104.196.123.155:6443 KubeDNS is running at https://104.196.123.155:6443/api/v1/proxy/namespaces/kube-system/services/kube-dns </code></pre></div></div> <p>Now let’s check the version of Kubernetes we’re running, note how it responds with both the client and server version.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./kubectl version --kubeconfig ./kubeconfig Client Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.3", GitCommit:"882d296a99218da8f6b2a340eb0e81c69e66ecc7", GitTreeState:"clean"} Server Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.3", GitCommit:"882d296a99218da8f6b2a340eb0e81c69e66ecc7", GitTreeState:"clean"} </code></pre></div></div> <h2 id="upgrading-to-a-new-version">Upgrading to a new version</h2> <p>So far we’ve done the usual bits on getting Kubernetes running on Ubuntu, now we’re ready to test the latest stuff from upstream.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>juju set-config kubernetes version=v1.3.1 </code></pre></div></div> <p>And then check <code class="highlighter-rouge">juju status</code> again while the model mutates. Now let’s see the version:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./kubectl version --kubeconfig ./kubeconfig Client Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.3", GitCommit:"882d296a99218da8f6b2a340eb0e81c69e66ecc7", GitTreeState:"clean"} Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.1", GitCommit:"fe4aa01af2e1ce3d464e11bc465237e38dbcff27", GitTreeState:"clean"} </code></pre></div></div> <p>Aha! As you see here, the cluster has upgraded to v1.3.1, but my local tools are obviously still v1.2.3. Obviously I can just recopy the kubectl tarball from the master node again, but I appear to have made a mistake, as the latest upstream version of Kubernetes is actually v1.3.3. No worries man:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> juju set-config kubernetes version=v1.3.3 </code></pre></div></div> <p>And let’s check our version:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./kubectl version --kubeconfig ./kubeconfig Client Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.3", GitCommit:"882d296a99218da8f6b2a340eb0e81c69e66ecc7", GitTreeState:"clean"} Server Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.3", GitCommit:"c6411395e09da356c608896d3d9725acab821418", GitTreeState:"clean"} </code></pre></div></div> <p>Now that my cluster is up to date, don’t forget to copy a new version of <code class="highlighter-rouge">kubectl</code> from the master so that your client is also up to date. Now we’re ballin’ and on the latest upstream, now we can go ahead and <a href="http://kubernetes.io/docs/hellonode/">dive into the Kubernetes docs</a> to get started deploying a real workload inside your cluster.</p> <h2 id="future-goals">Future Goals</h2> <p>So why isn’t v1.3.3 the default? Well we’d like to see some real feedback from people first and we’re still in the process of validating that your data will get migrated without issues as you upgrade. As you can see upgrading an empty cluster is trivial, and we’d like to make sure we’ve dotted all the t’s and i’s before moving on.</p> <p>We’d also like to take the next few weeks to prep the charms for the upcoming v1.4 release, as well as revving the etcd and Elastic stacks to their latest upstream versions, as well as revving the OS itself to xenial so that the ZFS backed storage is more robust and has a better out-of-the-box experience. We should also make it so that getting <code class="highlighter-rouge">kubectl</code> from the master isn’t so annoying.</p> <p>Got any feedback for us? You can find us on the <a href="https://lists.ubuntu.com/mailman/listinfo/juju">Juju mailing list</a> and #sig-cluster-ops and #sig-cluster-lifecycle on <a href="kubernetes.slack.com">kubernetes.slack.com</a>. Hope to see you there!</p> 2016-06-01T08:12:00+00:00 http://castrojo.github.io/2016/06/01/nvidia-ppa-download-statistics-for-may <p>Welcome back to another exciting update of Nvidia driver downloads!</p> <p>The biggest change is 364.15 is the new most popular version of the driver, and of course, we’ve added xenial as a new series. Here are the download statistics for the graphics-drivers PPA:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Version summaries 346.72: 57 346.96: 292 352.21: 219 352.79: 249 355.06: 3291 355.11: 11483 358.09: 16949 358.16: 22317 361.18: 4475 361.28: 20638 364.12: 12170 364.15: 37125 Series summaries precise: 985 trusty: 51066 vivid: 11307 wily: 36171 xenial: 17996 yakkety: 11740 Arch summaries amd64: 123560 armhf: 55 i386: 5650 </code></pre></div></div> <p>Want to help? <a href="http://store.steampowered.com/browse/linux/">buy a game</a> and check out <a href="https://launchpad.net/~graphics-drivers/+archive/ubuntu/ppa">ppa:graphics-drivers</a>.</p> 2016-05-31T13:19:00+00:00 http://castrojo.github.io/2016/05/31/zeppelin-is-now-a-top-level-apache-project <p>Apache Zeppelin has just graduated to become a <a href="https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces92">top-level project</a> at the Apache Foundation.</p> <p>As always, our Big Data team has you covered, you can find all the goodness here:</p> <ul> <li><a href="https://jujucharms.com/apache-zeppelin">Apache Zeppelin charm</a> - use this if you’re looking to build a solution using Zeppelin.</li> </ul> <p>But for most people you likely just want to be able to consume Zeppelin as part of your Spark cluster, check out these links below for some out-of-the-box clusters:</p> <ul> <li><a href="https://jujucharms.com/apache-hadoop-spark-zeppelin/">6 node Spark Cluster</a></li> </ul> <script async="" src="https://assets.ubuntu.com/v1/juju-cards-v1.3.0.js"></script> <div class="juju-card" data-id="apache-hadoop-spark-zeppelin"></div> <p>Happy Big-data-ing, and as always, you can join other big data enthusiasts on the mailing list: <a href="https://lists.ubuntu.com/mailman/listinfo/bigdata">bigdata@lists.ubuntu.com</a></p> 2016-04-19T15:58:00+00:00 http://castrojo.github.io/2016/04/19/a-quick-intro-to-juju-resources <p>For all the fancy bits of technology in your infrastructure there are still plenty of simple things that are useful and will probably never go away. One of these is blobs. Blobs are useful, they can be workload payloads, binaries for software, or whatever bits you need to deploy and manage.</p> <p>Juju never had a way of knowing about blobs. Sure, you could plop something on an http server and your charm could snag it. But then we’re not really solving any problems for you, you still need to deal with managing that blob, versioning it, using a server to serve it to clients, etc.</p> <p>Ideally, these blobs are accounted for, just like anything else in your infrastructure, so it makes sense that as of Juju 2.0 we can <em>model blobs as part of a model</em>; we call it Juju Resources. That way we can track them, cache them, acl them, and so on, just like everything else.</p> <h3 id="resources">Resources</h3> <p>A new concept has been introduced into Charms called “resources”. Resources are binary blobs that the charm can utilize, and are declared in the metadata for the Charm. All resources declared will have a version stored in the Charm store, however updates to these can be uploaded from an admin’s local machine to the controller.</p> <h4 id="change-to-metadata">Change to Metadata</h4> <p>A new clause has been added to <code class="highlighter-rouge">metadata.yaml</code> for resources. Resources can be declared as follows:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>resources: name: type: file # the only type initially filename: filename.tgz description: "One line that is useful when operators need to push it." </code></pre></div></div> <h4 id="new-user-commands">New User Commands</h4> <p>Three new commands have been introduced:</p> <ol> <li> <p><code class="highlighter-rouge">juju list-resources</code></p> <p>Pretty obvious, this command shows the resources required by and those in use by an existing service or unit in your model.</p> </li> <li> <p><code class="highlighter-rouge">juju push-resource</code></p> <p>This command uploads a file from your local disk to the juju controller to be used as a resource for a service.</p> </li> <li> <p><code class="highlighter-rouge">juju charm list-resources</code></p> <p><code class="highlighter-rouge">juju charm</code> is the the juju CLI equivalent of the “charm” command used by charm authors, though only applicable functionality is mirrored.</p> </li> </ol> <p>In addition, resources may be uploaded when deploying or upgrading charms by specifying the resource option to the deploy command. Following the resource option should be name=filepath pair. This option may be repeated more than once to upload more than one resource.</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>juju deploy foo --resource bar=/some/file.tgz --resource baz=./docs/cfg.xml </code></pre></div></div> <p>or</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>juju upgrade-charm foo --resource bar=/some/file.tgz --resource baz=./docs/cfg.xml </code></pre></div></div> <p>Where bar and baz are resources named in the metadata for the foo charm.</p> <h3 id="conclusion">Conclusion</h3> <p>It’s pretty simple. Put stuff in a place and be able to snag it later. People can use resources for all sorts of things.</p> <ul> <li>Payloads for the service you’re deploying.</li> <li>Software. Let’s face it, when you look at the amount of enterprise software in the wild, you’re not going to be able to <code class="highlighter-rouge">apt</code> eveything; you can now gate trusted binaries into resources to be used by charms.</li> </ul> <p>Hope you enjoy it!</p> 2016-04-05T12:19:00+00:00 http://castrojo.github.io/2016/04/05/new-wiki-bundles-for-you-to-play-with-dot-dot-dot <p>I’ve pushed new sample bundles to the <a href="http://jujucharms.com">Juju Charm Store</a>. The first is a simple mediawiki with mysql:</p> <script async="" src="https://assets.ubuntu.com/v1/juju-cards-v1.2.0.js"></script> <div class="juju-card" style="width:600px" data-id="bundle/wiki-simple-0"></div> <p>For a more scalable approach I’ve also pushed up a version with MariaDB, haproxy, and memcached. This allows you to add more wiki units to horizontally scale:</p> <div class="juju-card" style="width:600px" data-id="~jorge/bundle/wiki-scalable-0"></div> <p>I’ll be working on a “smoosh everything onto one machine” bundle next, so stay tuned!</p> 2016-03-18T10:16:00+00:00 http://castrojo.github.io/2016/03/18/solarized-theme-in-ubuntu <p><img src="http://www.jorgecastro.org/images/solarized.png" alt="terminal" /></p> 2016-03-03T16:02:00+00:00 http://castrojo.github.io/2016/03/03/servers-at-your-fingertips-dot-dot-dot <p>One of the nice things about system containers is that it makes the cost of creating a new machine essentially zero. With <a href="https://linuxcontainers.org/lxd/news/">LXD 2.0</a> around the corner this is now easier than ever. LXD now ships with native simplestreams support, which means we can now list and cache all sorts of interesting OS cloud images.</p> <p>You can see every image provided by doing a <code class="highlighter-rouge">lxc image list images:</code>, but the nice thing is the syntax is easy to remember. You can now just launch all sorts of Linux Server cloud images from all sorts of vendors and projects:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ lxc launch images:centos/7/amd64 test-centos Creating test-centos Retrieving image: 100% Starting test-centos $ lxc exec test-centos /bin/bash [root@test-centos ~]# yum update Loaded plugins: fastestmirror </code></pre></div></div> <p>… and so on. Give em a try:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>lxc launch images:debian/sid/amd64 lxc launch images:gentoo/current/amd64 lxc launch images:oracle/6.5/i386 </code></pre></div></div> <p>And of course ubuntu is supported:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>lxc launch ubuntu:14.04 </code></pre></div></div> <p>So if you’re sick of manually snagging ISOs for things and keeping those up to date, then you’ll dig 16.04, just install LXD and you can launch any Linux almost instantly. We’ll keep ‘em cached and updated for you too. I can <code class="highlighter-rouge">lxc launch ubuntu:12.04</code> and do <code class="highlighter-rouge">python --version</code> faster than I can look it up on packages.ubuntu.com. That’s pretty slick.</p> 2016-02-17T00:00:00+00:00 http://castrojo.github.io/2016/02/17/70k-nvidia-driver-downloads-in-6-months <p>It’s been about six months since ricotz and mamarley have <a href="http://www.omgubuntu.co.uk/2015/08/ubuntu-nvidia-graphics-drivers-ppa-is-ready-for-action">brought fresh Nvidia drivers</a> to Ubuntu users. I covered some of this in my SCaLE 14x talk on <a href="https://www.socallinuxexpo.org/scale/14x/presentations/gaming-ubuntu">Ubuntu Gaming</a>. If you missed SCaLE then feel free to check out <a href="http://princessleia.com/journal/?p=11266">Liz’s summary</a>.</p> <p>In that talk I summarized some of the challenges (and victories!) to making our platform be better for gamers. One of the obvious pain points is of course, delivering fresh drivers for people, which can be difficult (especially when it comes to managing regressions). First let’s look at the version breakdown:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>346.72: 57 346.96: 292 352.21: 219 352.79: 62 355.06: 3291 355.11: 11483 358.09: 16949 358.16: 22317 361.18: 4076 361.28: 10219 </code></pre></div></div> <p>And then by Ubuntu release:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>precise: 630 trusty: 30665 vivid: 10908 wily: 21537 xenial: 5225 </code></pre></div></div> <p>Trusty wins, no surprise there. Wow, way more people on Xenial than I expected, good to see people are testing our upcoming LTS! And finally, by architecture:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>amd64: 65661 armhf: 31 i386: 3273 </code></pre></div></div> <p>So when you add them all up, it’s about 70,000 downloads of Nvidia drivers from the PPA in the last six months. This is great, as it means there’s certainly a demand for fresh drivers, but it’s also frightening when you consider the end-user experience of traditional packaging and all the wonderful face punching that entails. But hey, that’s XCOM. Snappy can’t come fast enough!</p> <p>Want to help? <a href="http://store.steampowered.com/browse/linux/">Buy a game</a> and check out <a href="https://launchpad.net/~graphics-drivers/+archive/ubuntu/ppa">ppa:graphics-drivers</a>.</p> <p>Oh and happy belated Vulkan day!</p> 2014-08-21T00:00:00+00:00 http://castrojo.github.io/2014/08/21/the-ubuntu-steam-box <p>It’s been about a year since <a href="http://www.jorgecastro.org/2013/09/25/the-faux-steam-machine/">I started building my own Steam console</a> for my living room. A ton has changed since then. SteamOS has been released, In Home Streaming is out of beta and generally speaking the living room experience has gotten a ton better.</p> <p>This blog post will be a summary of what’s changed in the past year, in the hopes that it will help someone who might be interested in building their own “next-gen console” for about the same price, and take advantage of nicer hardware and all the things that PC gaming has to offer.</p> <ul> <li>My <a href="http://www.jorgecastro.org/2013/09/25/the-faux-steam-machine">first blog post</a> on building your own box.</li> <li>A talk I gave at Penguicon on <a href="https://www.youtube.com/watch?v=lXF5VebhhAc">SteamOS</a> for background.</li> </ul> <h2 id="step-1-choosing-the-hardware">Step 1: Choosing the hardware</h2> <ul> <li> <p>I consider the <a href="https://www.youtube.com/watch?v=zcrFzOpTHkw">NVIDIA GTX 750Ti</a> to be the best thing to happen in hardware for this sort of project. It’s based on their newest Maxwell technology so it runs cool, it does not need a special power supply plug, and it’s pretty small. It’s also between $120-$150 – which means nearly any computer is now capable of becoming a game console. And a competent one at that.</p> </li> <li> <p>I have settled on the Cooler Master 110 case, which is one of the least obnoxious PC case you can find that won’t look too bad in the living room. Unfortunately Valve’s <a href="http://www.washingtonpost.com/blogs/the-switch/files/2013/11/med_STEAM_M_console__hero.jpg">slick-looking case</a> did not kick the case makers into making awesome-looking living room style cases. The closest you can find is the <a href="http://www.silverstonetek.com/raven/products/index.php?model=RVZ01">Silverstone RVZ01</a>, which has the right insides, but they ruined the outside with crazy plastic ribs. The <a href="http://www.pcworld.com/article/2465160/digital-storm-bolt-ii-review-rise-of-the-un-steam-machines.html">Digital Storm Bolt II</a> looks great, but you can’t buy the case seperately. Both cases have CD drives for some reason, boo!</p> </li> </ul> <p><img src="http://www.jorgecastro.org/images/coolermaster.jpg" alt="" /></p> <ul> <li> <p>Nvidia has a <a href="http://www.geforce.com/whats-new/guides/geforce-gtx-750-ti-mini-itx-pc-build-guide">great guide</a> on building a PC within the console-price range if you want to look around. I also recommend checking out <a href="http://www.reddit.com/r/buildapc">r/buildapc</a>, which has tons of Mini-ITX/750Ti builds.</p> </li> <li> <p>Another alternative is the excellent <a href="http://www.intel.com/content/www/us/en/nuc/overview.html">Intel NUC</a> and <a href="http://www.gigabyte.us/products/list.aspx?s=47&amp;ck=104">Gigabyte Brix</a>. These make for great portable machines, but for the upcoming AAA titles for Linux like Metro Redux, Star Citizen, and so on I decided to go with a dedicated graphics card. Gigabyte makes a <a href="http://www.gigabyte.us/products/list.aspx?s=47&amp;ck=104">very interesting</a> model that is the size of a NUC, but with a GTX 760(!). This looks to be ideal, but unfortunately when <a href="https://www.youtube.com/watch?v=hGI8iYjk0rc">Linus reviewed it</a> he found heat/throttling issues. When they make a Maxwell based one of these it will likely be awesome.</p> </li> <li> <p>Don’t forget <a href="http://smile.amazon.com/Microsoft-Xbox-Wireless-Controller-Windows/dp/B004QRKWKQ/ref=sr_1_3?ie=UTF8&amp;qid=1408660327&amp;sr=8-3&amp;keywords=xbox+controller+for+PC">the controller</a>. The Xbox wireless ones will work out of the box. I recommend avoiding the off-brand dongles you see on Amazon, they can be hit or miss.</p> </li> </ul> <h2 id="step-2-choosing-the-software">Step 2: Choosing the software</h2> <p>I’ve been using SteamOS since it came out. The genious about SteamOS is that fundamentally it does only 2 things. It boots, and then runs Steam Big Picture (BPM) mode. This means for a dedicated console, the OS is really not important. I have 2 drives in the box, one with SteamOS, and one with Ubuntu running BPM. After running both I prefer Ubuntu/Steam to SteamOS:</p> <ul> <li>Faster boot (Upstart v. SysV)</li> <li>PPAs enable fresh access to new Nvidia drivers and Plex Home Theater</li> <li>Newer kernels and access to HWE kernels over the next 5 years</li> </ul> <p>I tend to alternate between the two, but since I am more familiar with Ubuntu it makes it easier to use for, so the rest of this post will cover how to build a dedicated Steam Box using Ubuntu.</p> <p>This isn’t to say SteamOS is bad, in fact, setting it up is actually easier than doing the next few steps; remember that the entire point is to not care about the OS underneath, and get you into Steam. So build whatever is most comfortable for you!</p> <p><img src="http://www.jorgecastro.org/images/stats.jpg" alt="" /></p> <h2 id="step-3-installation">Step 3: Installation</h2> <p>These are the steps I am currently doing. It’s not for beginners, you should be comfortable admining an Ubuntu system.</p> <ul> <li>Install Ubuntu 14.04.</li> <li>(Optional) - Install <code class="highlighter-rouge">openssh-server</code>. I don’t know about you but lugging a keyboard/mouse back and forth to my living room is not my idea of a good time. I prefer to sit on the couch, and ssh into the box from my laptop.</li> <li>Add the <a href="https://launchpad.net/~xorg-edgers/+archive/ubuntu/ppa">xorg-edgers PPA</a>. You don’t need this per se, but let’s go all in!</li> <li>Install the latest Nvidia drivers: As of this writing, nvidia-graphics-drivers-343.</li> </ul> <p>After you’ve installed the drivers and all the security updates you should reboot to get to your nice new clean desktop system. Now it’s time to make it a console:</p> <ul> <li>Log in, and install Steam. Log into steam, make sure it works.</li> <li>Add the Marc Deslaurier’s <a href="https://launchpad.net/~mdeslaur/+archive/ubuntu/steamos">SteamOS packages</a> PPA. These are rebuilt for Ubuntu and he does a great job keeping them up to date.</li> <li><code class="highlighter-rouge">sudo apt-get install steamos-compositor steamos-modeswitch-inhibitor steamos-xpad-dkms</code></li> <li>Log out, and in the login screen, click on the Ubuntu symbol by the username and select the Steam session. This will get you the dedicated Steam session. Make sure that works. Exit out of that and now let’s make it so we can boot into that new session by default</li> <li><a href="http://askubuntu.com/q/51086/235">Enable autologin in LightDM</a> after the fact so that when your machine boots it goes right into Steam’s Big Picture mode.</li> </ul> <p>We’re using Valve’s xpad module instead of xboxdrv because that’s what they use in SteamOS and I don’t want to deviate too much. But if you prefer about xboxdrv then <a href="http://www.omgubuntu.co.uk/2014/06/ubuntu-xbox-controller-support-xboxdrv-driver">follow this guide</a>.</p> <ul> <li>Steam updates itself at the client level so there’s no need to worry about that, the final step for a console-like experience is to <a href="http://askubuntu.com/q/9/235">enable automatic updates</a>. Remember you’re using PPAs, so if you’re not confident that you can fix things, just leave it and do maintenance by hand every once in a while.</li> </ul> <h3 id="step-4-home-theater-bling">Step 4: Home Theater Bling</h3> <p>If you’re going to have a nice living room box, then let’s use it for other things. I have a dedicated server with media that I share out with Plex Media Server, so in this step I’ll install the client side.</p> <p>Plex Home Theater:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo add-apt-repository ppa:plexapp/plexht sudo add-apt-repository ppa:ppa:pulse-eight/libcec sudo apt-get update &amp;&amp; sudo apt-get install plexhometheater </code></pre></div></div> <p>In Steam you can then click on the + symbol, Add a non-steam game, and then add Plex. Use the gamepad (not the stick) to navigate the UI once you launch it. If you prefer XBMC/Kodi you can <a href="http://wiki.xbmc.org/index.php?title=Installing_XBMC_for_Linux">install that instead</a>. I found that the controller also works out of the box there, so it’s a nice experience no matter which one you choose.</p> <h3 id="step-5-in-home-streaming">Step 5: In Home Streaming</h3> <p>This is a killer Steam feature, that allows you to stream your Windows games to your new console. It’s very straight forward, just have both machines on and logged into Steam on the same network, they will autodiscover each other, and your Windows games will show up in your Ubuntu/Steam UI, and you can stream them. Though it works suprisingly well over wireless, you’ll definately want to ensure you’ve got gigabit ethernet if you want to stream games 1080p at 60 frames per second.</p> <h2 id="conclusion">Conclusion</h2> <p>And that’s basically it! There’s tons of stuff I’ve glossed over, but these are the basic steps. There’s lots of little things you can do, like remove a bunch of desktop packages you won’t need (so you don’t need to download and update them) and other tips and tricks, I’ll try to keep everyone up to date on how it’s going.</p> <p>Enjoy your new next-gen gaming console!</p> <p><img src="http://www.jorgecastro.org/images/xcom.jpg" alt="" /></p> <p>TODO:</p> <ul> <li>You can change out the plymouth theme to use <a href="http://repo.steampowered.com/steamos/pool/main/p/plymouth-themes-steamos/">the one for SteamOS</a> - but I have an SSD in the box and combined with the fast boot it never comes up for me anyway.</li> <li>It’d be cool to make a prototype of Ubuntu Core and then provide Steam in an LXC container on top of that so we don’t have to use a full blown desktop ISO.</li> </ul>