Warning: gzdecode(): data error in /customers/c/9/b/feedsapi.com/httpd.www/advance/libraries/humble-http-agent/HumbleHttpAgent.php on line 560 Warning: Cannot modify header information - headers already sent by (output started at /customers/c/9/b/feedsapi.com/httpd.www/advance/libraries/humble-http-agent/HumbleHttpAgent.php:560) in /customers/c/9/b/feedsapi.com/httpd.www/advance/libraries/feedwriter/FeedWriter.php on line 89 Hacker News https://news.ycombinator.com/ Links for the intellectually curious, ranked by readers. Alphabet Backs GitLab's Quest to Surpass Microsoft's GitHub https://www.bloomberg.com/news/articles/2018-09-19/alphabet-backs-gitlab-s-quest-to-surpass-microsoft-s-github https://www.bloomberg.com/news/articles/2018-09-19/alphabet-backs-gitlab-s-quest-to-surpass-microsoft-s-github <head><title>Terms of Service Violation</title></head><body id="readabilityBody" readability="24"> <div class="container" readability="9.4285714285714"> <img src="https://www.bloomberg.com/graphics/assets/img/BB-Logo-2line.svg" width="310"/> <p>Your usage has been flagged as a violation of our <a href="http://www.bloomberg.com/tos" target="_blank" rel="noopener noreferrer">terms of service</a>. </p> <p> For inquiries related to this message please <a href="http://www.bloomberg.com/feedback">contact support</a>. For sales inquiries, please visit <a href="http://www.bloomberg.com/professional/request-demo">http://www.bloomberg.com/professional/request-demo</a> </p> <h3> If you believe this to be in error, please confirm below that you are not a robot by clicking "I'm not a robot" below.</h3> <br/><h3>Please make sure your browser supports JavaScript and cookies and that you are not blocking them from loading. For more information you can review the Terms of Service and Cookie Policy.</h3> <br/><h3 id="block_uuid">Block reference ID: </h3> </div> </body> Thu, 20 Sep 2018 08:50:20 +0000 https://www.bloomberg.com/tosv2.html?vid=&uuid=b8bf8380-bcc8-11e8-b8e7-dfd79ed9b39b&url=L25ld3MvYXJ0aWNsZXMvMjAxOC0wOS0xOS9hbHBoYWJldC1iYWNrcy1naXRsYWItcy1xdWVzdC10by1zdXJwYXNzLW1pY3Jvc29mdC1zLWdpdGh1Yg== Idle Until Urgent https://philipwalton.com/articles/idle-until-urgent/ https://philipwalton.com/articles/idle-until-urgent/ <p>A few weeks ago I was looking at some of the performance metrics for my site. Specifically, I wanted to see how I was doing on our newest metric, <a href="https://developers.google.com/web/updates/2018/05/first-input-delay">first input delay</a> (FID). My site is just a blog (and doesn’t run much JavaScript), so I expected to see pretty good results.</p><p>Input delay that’s less than 100 milliseconds is typically <a href="https://developers.google.com/web/fundamentals/performance/rail#ux">perceived as instant</a> by users, so the performance goal we recommend (and the numbers I was hoping to see in my analytics) is FID &lt; 100ms for 99% of page loads.</p><p>To my surprise, my site’s FID was 254ms at the 99th percentile. And while that’s not terrible, the perfectionist in me just couldn’t let that slide. I had to fix it!</p><p>To make a long story short, without removing any functionality from my site, I was able to get my FID under 100ms at the 99th percentile. But what I’m sure is more interesting to you readers is:</p><ul><li><em>How</em> I approached diagnosing the problem.</li><li><em>What</em> specific strategies and techniques I used to fix it.</li></ul><p>To that second point above, while I was trying to solve my issue I stumbled upon a pretty interesting performance strategy that I want to share (it’s the primary reason I’m writing this article).</p><p>I’m calling the strategy: <em>idle until urgent</em>.</p><h2 id="my-performance-problem">My performance problem</h2><p>First input delay (FID) is a metric that measures the time between when a user <a href="https://developers.google.com/web/updates/2018/05/first-input-delay#what_counts_as_a_first_input">first interacts</a> with your site (for a blog like mine, that’s most likely them clicking a link) and the time when the browser is able to respond to that interaction (make a request to load the next page).</p><p>The reason there might be a delay is if the browser’s main thread is busy doing something else (usually executing JavaScript code). So to diagnose a higher-than-expected FID, you should start by creating a performance trace of your site as it’s loading (with CPU and network throttling enabled) and look for individual tasks on the main thread that take a long time to execute. Then once you’ve identified those long tasks, you can try to break them up into smaller tasks.</p><p>Here’s what I found when doing a performance trace of my site:</p><figure><a href="https://philipwalton.com/static/idle-until-urget-before-1400w-efc9f3a53c.png"><img srcset="/static/idle-until-urget-before-1400w-efc9f3a53c.png, /static/idle-until-urget-before-9bc2ecd0b0.png 700w" src="https://philipwalton.com/static/idle-until-urget-before-9bc2ecd0b0.png" alt="A performance trace of my site's JavaScript while loading (with network/CPU throttling enabled)"/></a><figcaption>A performance trace of my site's JavaScript while loading (with network/CPU throttling enabled).</figcaption></figure><p>Notice, when the main script bundle is evaluated, it’s run as a single task that takes 233 milliseconds to complete.</p><figure><a href="https://philipwalton.com/static/idle-until-urget-before-eval-1400w-7a455de908.png"><img srcset="/static/idle-until-urget-before-eval-1400w-7a455de908.png, /static/idle-until-urget-before-eval-1d68f2dff6.png 700w" src="https://philipwalton.com/static/idle-until-urget-before-eval-1d68f2dff6.png" alt="Evaluating my site's main bundle takes 233ms"/></a><figcaption>Evaluating my site's main bundle takes 233ms.</figcaption></figure><p>Some of this code is webpack boilerplate and babel polyfills, but the majority of it is from my script’s <code>main()</code> entry function, which itself takes 183ms to complete:</p><figure><a href="https://philipwalton.com/static/idle-until-urget-before-main-1400w-08fe4dd1c5.png"><img srcset="/static/idle-until-urget-before-main-1400w-08fe4dd1c5.png, /static/idle-until-urget-before-main-59f7c95e33.png 700w" src="https://philipwalton.com/static/idle-until-urget-before-main-59f7c95e33.png" alt="Executing my site's main() entry function takes 183ms."/></a><figcaption>Executing my site's <code>main()</code> entry function takes 183ms.</figcaption></figure><p>And it’s not like I’m doing anything ridiculous in my <code>main()</code> function. I’m initializing my UI components and then running my analytics:</p><pre><code class="language-js"><span class="hljs-keyword">const</span> main = <span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> { drawer.init(); contentLoader.init(); breakpoints.init(); alerts.init(); analytics.init(); }; main(); </code></pre><p>So what’s taking so long to run?</p><p>Well, if you look at the tails of this flame chart, you won’t see any single functions that are clearly taking up the bulk of the time. Most individual functions are run in less than 1ms, but when you add them all up, it’s taking more than 100ms to run them in a single, synchronous call stack.</p><p>This is the JavaScript equivalent of <em>death by a thousand cuts.</em></p><p>Since the problem is all these functions are being run as part of a single task, the browser has to wait until this task finishes to respond to user interaction. So clearly the solution is to break up this code into multiple tasks, but that’s a lot easier said than done.</p><p>At first glance, it might seem like the obvious solution is to prioritize each of the components in my <code>main()</code> function (they’re actually already in priority order), initialize the highest priority components right away, and then defer other component initialization to a subsequent task.</p><p>While this may help some, it’s not a solution that everyone could implement, nor does it scale well to a really large site. Here’s why:</p><ul><li>Deferring UI component initialization only helps if the component isn’t yet rendered. If it’s already rendered than deferring initialization runs the risk that the user tries to interact with it and it’s not yet ready.</li><li>In many cases all UI components are either equally important or they depend on each other, so they all need to be initialized at the same time.</li><li>Sometimes individual components take long enough to initialize that they’ll block the main thread even if they’re run in their own tasks.</li></ul><p>The reality is that initializing each component in its own task is usually not sufficient and oftentimes not even possible. What’s usually needed is breaking up tasks <em>within</em> each component being initialized.</p><h3 id="greedy-components">Greedy components</h3><p>A perfect example of a component that really needs to have its initialization code broken up can be illustrated by zooming closer down into this performance trace. Mid-way through the <code>main()</code> function, you’ll see one of my components uses the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DateTimeFormat">Intl.DateTimeFormat</a> API:</p><figure><a href="https://philipwalton.com/static/idle-until-urget-before-date-time-format-1400w-c67615763f.png"><img srcset="/static/idle-until-urget-before-date-time-format-1400w-c67615763f.png, /static/idle-until-urget-before-date-time-format-252558f2ab.png 700w" src="https://philipwalton.com/static/idle-until-urget-before-date-time-format-252558f2ab.png" alt="Creating an Intl.DateTimeFormat instance took 13.47ms!"/></a><figcaption>Creating an <code>Intl.DateTimeFormat</code> instance took 13.47ms!</figcaption></figure><p>Creating this object took 13.47 milliseconds!</p><p>The thing is, the <code>Intl.DateTimeFormat</code> instance is created in the component’s constructor, but it’s <em>not actually used</em> until it’s needed by other components that reference it to format dates. However, this component doesn’t know when it’s going to be referenced, so it’s playing it safe and instantiating the <code>Int.DateTimeFormat</code> object right away.</p><p>But is this the right code evaluation strategy? And if not, what is?</p><h2 id="code-evaluation-strategies">Code evaluation strategies</h2><p>When choosing an <a href="https://en.wikibooks.org/wiki/Introduction_to_Programming_Languages/Evaluation_Strategies">evaluation strategy</a> for potentially expensive code, most developers select one of the following:</p><ul><li><strong><a href="https://en.wikipedia.org/wiki/Eager_evaluation">Eager evaluation</a>:</strong> where you run your expensive code right away.</li><li><strong><a href="https://en.wikipedia.org/wiki/Lazy_evaluation">Lazy evaluation</a>:</strong> where you wait until another part of your program needs the result of that expensive code, and you run it then.</li></ul><p>These are probably the two most popular evaluation strategies, but after my experience refactoring my site, I now think these are probably your two worst options.</p><h3 id="the-downsides-of-eager-evaluation">The downsides of eager evaluation</h3><p>As the performance problem on my site illustrates pretty well, eager evaluation has the downside that, if a user tries to interact with your page while the code is evaluating, the browser must wait until the code is done evaluating to respond.</p><p>This is especially problematic if your page <em>looks</em> like it’s ready to respond to user input, but then it can’t. Users will perceive your page as sluggish or maybe even completely broken.</p><p>The more code you evaluate up front, the longer it will take for your page to become interactive.</p><h3 id="the-downsides-of-lazy-evaluation">The downsides of lazy evaluation</h3><p>If it’s bad to run all your code right away, the next most obvious solution is to wait to run it until it’s actually needed. This way you don’t run code unnecessarily, especially if it’s never actually needed by the user.</p><p>Of course, the problem with waiting until the user needs the result of running that code is now you’re <em>guaranteeing</em> that your expensive code will block user input.</p><p>For some things (like loading additional content from the network), it makes sense to defer it until it’s requested by the user. But for most code you’re evaluating (e.g. reading from localStorage, processing large datasets, etc.) you definitely want it to happen <em>before</em> the user interaction that needs it.</p><h3 id="other-options">Other options</h3><p>The other evaluation strategies you can choose from all take an approach somewhere in between eager and lazy. I’m not sure if the following two strategies have official names, but I’m going to call them deferred evaluation and idle evaluation:</p><ul><li><strong>Deferred evaluation:</strong> where you schedule your code to be run in a future task, using something like <code>setTimeout</code>.</li><li><strong>Idle evaluation:</strong> a type of deferred evaluation where you use an API like <a href="https://developers.google.com/web/updates/2015/08/using-requestidlecallback">requestIdleCallback</a> to schedule your code to run.</li></ul><p>Both of these options are usually better than eager or lazy evaluation because they’re far less likely to lead to individual long tasks that block input. This is because, while browsers cannot interrupt any single task to respond to user input (doing so would very likely break sites), they <em>can</em> run a task in between a queue of scheduled tasks, and most browsers <em>do</em> when that task is caused by user input. This is known as <a href="https://blogs.windows.com/msedgedev/2017/06/01/input-responsiveness-event-loop-microsoft-edge/">input prioritization</a>.</p><p>To put that another way: if you ensure all your code is run in short, distinct tasks (preferably <a href="https://developers.google.com/web/fundamentals/performance/user-centric-performance-metrics#long_tasks">less than 50ms</a>), your code will never block user input.</p><aside class="Info Info--warning"><p><strong>Important!</strong> While browsers can run input callbacks ahead of queued tasks, they <em>cannot</em> run input callbacks ahead of queued microtasks. And since promises and <code>async</code> functions run as microtasks, converting your sync code to promise-based code will not prevent it from blocking user input!</p><p>If you’re not familiar with the difference between tasks and microtasks, I highly recommend watching my colleague Jake’s excellent <a href="https://youtu.be/cCOL7MC4Pl0">talk on the event loop</a>.</p></aside><p>Given what I just said, I could refactor my <code>main()</code> function to use <code>setTimeout()</code> and <code>requestIdleCallback()</code> to break up my initialization code into separate tasks:</p><pre><code class="language-js"><span class="hljs-keyword">const</span> main = <span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> { <mark>setTimeout</mark><span class="hljs-function">(<span class="hljs-params">(</span>) =&gt;</span> drawer.init(), <span class="hljs-number">0</span>); <mark>setTimeout</mark><span class="hljs-function">(<span class="hljs-params">(</span>) =&gt;</span> contentLoader.init(), <span class="hljs-number">0</span>); <mark>setTimeout</mark><span class="hljs-function">(<span class="hljs-params">(</span>) =&gt;</span> breakpoints.init(), <span class="hljs-number">0</span>); <mark>setTimeout</mark><span class="hljs-function">(<span class="hljs-params">(</span>) =&gt;</span> alerts.init(), <span class="hljs-number">0</span>); <mark>requestIdleCallback</mark><span class="hljs-function">(<span class="hljs-params">(</span>) =&gt;</span> analytics.init()); }; main(); </code></pre><p>However, while this is better than before (many small tasks vs. one long task), as I explained above it’s likely still not good enough. For example, if I defer the initialization of my UI components (specifically <code>contentLoader</code> and <code>drawer</code>) they’ll be less likely to block user input, but they also run the risk of not being ready when the user tries to interact with them!</p><p>And while delaying my analytics with <code>requestIdleCallback()</code> is probably a good idea, any interactions I care about before the next idle period will be missed. And if there’s not an idle period before the user leaves the page, these callbacks may never run at all!</p><p>So if all evaluations strategies have downsides, which one should you pick?</p><h2 id="idle-until-urgent">Idle Until Urgent</h2><p>After spending a lot of time thinking about this problem, I realized that the evaluation strategy I <em>really</em> wanted was one where my code would initially be deferred to idle periods but then run immediately as soon as it’s needed. In other words: <em>idle-until-urgent</em>.</p><p><em>Idle-until-urgent</em> sidesteps most of the downsides I described in the previous section. In the worst case, it has the exact same performance characteristics as lazy evaluation, and in the best case it doesn’t block interactivity at all because execution happens during idle periods.</p><p>I should also mention that this strategy works both for single tasks (computing values idly) as well as multiple tasks (an ordered queue of tasks to be run idly). I’ll explain the single-task (idle value) variant first because it’s a bit easier to understand.</p><h3 id="idle-values">Idle values</h3><p>I showed above that <code>Int.DateTimeFormat</code> objects can be pretty expensive to initialize, so if an instance isn’t needed right away, it’s better to initialize it during an idle period. Of course, as soon as it <em>is</em> needed, you want it to exist, so this is a perfect candidate for <em>idle-until-urgent</em> evaluation.</p><p>Consider the following simplified component example that we want to refactor to use this new strategy:</p><pre><code class="language-js"><span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">MyComponent</span> </span>{ <span class="hljs-keyword">constructor</span>() { addEventListener(<span class="hljs-string">'click'</span>, () =&gt; <span class="hljs-keyword">this</span>.handleUserClick()); <mark><span class="hljs-keyword">this</span>.formatter</mark> = <span class="hljs-keyword">new</span> <span class="hljs-built_in">Intl</span>.DateTimeFormat(<span class="hljs-string">'en-US'</span>, { <span class="hljs-attr">timeZone</span>: <span class="hljs-string">'America/Los_Angeles'</span>, }); } handleUserClick() { <span class="hljs-built_in">console</span>.log(<mark><span class="hljs-keyword">this</span>.formatter.format</mark>(<span class="hljs-keyword">new</span> <span class="hljs-built_in">Date</span>())); } } </code></pre><p>Instances of <code>MyComponent</code> above do two things in their constructor:</p><ul><li>Add an event listener for user interactions.</li><li>Create an <code>Intl.DateTimeFormat</code> object.</li></ul><p>This component perfectly illustrates why you often need to split up tasks <em>within</em> an individual component (rather than just at the component level).</p><p>In this case it’s really important that the event listeners run right away, but it’s not important that the <code>Intl.DateTimeFormat</code> instance is created until it’s needed by the event handler. Of course we don’t want to create the <code>Intl.DateTimeFormat</code> object in the event handler because then its slowness will delay that event from running.</p><p>So here’s how we could update this code to use the <em>idle-until-urgent</em> strategy. Note, I’m making use of an <code>IdleValue</code> helper class, which I’ll explain next:</p><pre><code class="language-js"><span class="hljs-keyword">import</span> {IdleValue} <span class="hljs-keyword">from</span> <span class="hljs-string">'./path/to/IdleValue.mjs'</span>; <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">MyComponent</span> </span>{ <span class="hljs-keyword">constructor</span>() { addEventListener(<span class="hljs-string">'click'</span>, () =&gt; <span class="hljs-keyword">this</span>.handleUserClick()); <span class="hljs-keyword">this</span>.formatter = <mark><span class="hljs-keyword">new</span> IdleValue</mark><span class="hljs-function">(<span class="hljs-params">(</span>) =&gt;</span> { <span class="hljs-keyword">return</span> <span class="hljs-keyword">new</span> <span class="hljs-built_in">Intl</span>.DateTimeFormat(<span class="hljs-string">'en-US'</span>, { <span class="hljs-attr">timeZone</span>: <span class="hljs-string">'America/Los_Angeles'</span>, }); }); } handleUserClick() { <span class="hljs-built_in">console</span>.log(<span class="hljs-keyword">this</span>.formatter<mark>.getValue()</mark>.format(<span class="hljs-keyword">new</span> <span class="hljs-built_in">Date</span>())); } } </code></pre><p>As you can see, this code doesn’t look much different from the previous version, but instead of assigning <code>this.formatter</code> to a new <code>Intl.DateTimeFormat</code> object, I’m assigning <code>this.formatter</code> to an <code>IdleValue</code> object, which I pass an initialization function.</p><p>The way this <code>IdleValue</code> class works is it schedules the initialization function to be run during the next idle period. If the idle period occurs before the IdleValue instance is referenced, then no blocking occurs and the value can be returned immediately when requested. But if, on the other hand, the value is referenced <em>before</em> the next idle period, then the scheduled idle callback is canceled and the initialization function is run immediately.</p><p>Here’s the gist of how the <code>IdleValue</code> class is implemented (note: I’ve also released this code as part of the <a href="https://github.com/GoogleChromeLabs/idlize"><code>idlize</code> package</a>, which includes all the helpers shown in this article):</p><pre><code class="language-js"><span class="hljs-keyword">export</span> <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">IdleValue</span> </span>{ <span class="hljs-keyword">constructor</span>(init) { <span class="hljs-keyword">this</span>._init = init; <span class="hljs-keyword">this</span>._value; <span class="hljs-keyword">this</span>._idleHandle = requestIdleCallback(<span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> { <span class="hljs-keyword">this</span>._value = <span class="hljs-keyword">this</span>._init(); }); } getValue() { <span class="hljs-keyword">if</span> (<span class="hljs-keyword">this</span>._value === <span class="hljs-literal">undefined</span>) { cancelIdleCallback(<span class="hljs-keyword">this</span>._idleHandle); <span class="hljs-keyword">this</span>._value = <span class="hljs-keyword">this</span>._init(); } <span class="hljs-keyword">return</span> <span class="hljs-keyword">this</span>._value; } <span class="hljs-comment">// ...</span> } </code></pre><p>While including the <code>IdleValue</code> class in my example above didn’t require many changes, it did technically change the public API (<code>this.formatter</code> vs. <code>this.formatter.getValue()</code>).</p><p>If you’re in a situation where you want to use the <code>IdleValue</code> class but you can’t change your public API, you can use the <code>IdleValue</code> class with ES2015 getters:</p><pre><code class="language-js"><span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">MyComponent</span> </span>{ <span class="hljs-keyword">constructor</span>() { addEventListener(<span class="hljs-string">'click'</span>, () =&gt; <span class="hljs-keyword">this</span>.handleUserClick()); <span class="hljs-keyword">this</span>._formatter = <span class="hljs-keyword">new</span> IdleValue(<span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> { <span class="hljs-keyword">return</span> <span class="hljs-keyword">new</span> <span class="hljs-built_in">Intl</span>.DateTimeFormat(<span class="hljs-string">'en-US'</span>, { <span class="hljs-attr">timeZone</span>: <span class="hljs-string">'America/Los_Angeles'</span>, }); }); } <mark>get formatter() {</mark><mark><span class="hljs-keyword">return</span> <span class="hljs-keyword">this</span>._formatter.getValue();</mark><mark>}</mark><span class="hljs-comment">// ...</span> } </code></pre><p>Or, if you don’t mind a little abstraction, you can use the <a href="https://github.com/GoogleChromeLabs/idlize/blob/master/docs/defineIdleProperty.md"><code>defineIdleProperty()</code></a> helper (which uses <code>Object.defineProperty()</code> under the hood):</p><pre><code class="language-js"><span class="hljs-keyword">import</span> {defineIdleProperty} <span class="hljs-keyword">from</span> <span class="hljs-string">'./path/to/defineIdleProperty.mjs'</span>; <span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">MyComponent</span> </span>{ <span class="hljs-keyword">constructor</span>() { addEventListener(<span class="hljs-string">'click'</span>, () =&gt; <span class="hljs-keyword">this</span>.handleUserClick()); <mark>defineIdleProperty</mark><span class="hljs-function">(<span class="hljs-params"><span class="hljs-keyword">this</span>, <span class="hljs-string">'formatter'</span>, (</span>) =&gt;</span> { <span class="hljs-keyword">return</span> <span class="hljs-keyword">new</span> <span class="hljs-built_in">Intl</span>.DateTimeFormat(<span class="hljs-string">'en-US'</span>, { <span class="hljs-attr">timeZone</span>: <span class="hljs-string">'America/Los_Angeles'</span>, }); }); } <span class="hljs-comment">// ...</span> } </code></pre><p>For individual property values that may be expensive to compute, there’s really no reason not to use this strategy, especially since you can employ it without changing your API!</p><p>While this example used the <code>Intl.DateTimeFormat</code> object, it’s also probably a good candidate for any of the following:</p><ul><li>Processing large sets of values.</li><li>Getting a value from localStorage (or a cookie).</li><li>Running <code>getComputedStyle()</code>, <code>getBoundingClientRect()</code>, or any other API that may require recalculating style or layout on the main thread.</li></ul><h3 id="idle-task-queues">Idle task queues</h3><p>The above technique works pretty well for individual properties whose values can be computed with a single function, but in some cases your logic doesn’t fit into a single function, or, even if it technically could, you’d still want to break it up into smaller functions because otherwise you’d risk blocking the main thread for too long.</p><p>In such cases what you really need is a queue where you can schedule multiple tasks (functions) to run when the browser has idle time. The queue will run tasks when it can, and it will pause execution of tasks when it needs to yield back to the browser (e.g. if the user is interacting).</p><p>To handle this, I built an <a href="https://github.com/GoogleChromeLabs/idlize"><code>IdleQueue</code></a> class, and you can use it like this:</p><pre><code class="language-js"><span class="hljs-keyword">import</span> {IdleQueue} <span class="hljs-keyword">from</span> <span class="hljs-string">'./path/to/IdleQueue.mjs'</span>; <span class="hljs-keyword">const</span> queue = <span class="hljs-keyword">new</span> IdleQueue(); queue.pushTask(<span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> { <span class="hljs-comment">// Some expensive function that can run idly...</span> }); queue.pushTask(<span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> { <span class="hljs-comment">// Some other task that depends on the above</span> <span class="hljs-comment">// expensive function having already run...</span> }); </code></pre><aside class="Info"><p><strong>Note:</strong> breaking up your synchronous JavaScript code into separate tasks that can run asynchronously as part of a task queue is different from <a href="https://developers.google.com/web/fundamentals/performance/optimizing-javascript/code-splitting/">code splitting</a>, which is about breaking up large JavaScript bundles into smaller files (and is also important for improving performance).</p></aside><p>As with the idly-initialized property strategy shown above, idle tasks queues also have a way to run immediately in cases where the result of their execution is needed right away (the “urgent” case).</p><p>Again, this last bit is really important; not just because sometimes you need to compute something as soon as possible, but often you’re integrating with a third-party API that’s synchronous, so you need the ability to run your tasks synchronously as well if you want to be compatible.</p><p>In a perfect world, all JavaScript APIs would be non-blocking, asynchronous, and composed of small chunks of code that can yield at will back to the main thread. But in the real world, we often have no choice but to be synchronous due to a legacy codebase or integrations with third-party libraries we don’t control.</p><p>As I said before, this is one of the great strengths of the <em>idle-until-urgent</em> pattern. It can be easily applied to most programs without requiring a large-scale rewrite of the architecture.</p><h3 id="guaranteeing-the-urgent">Guaranteeing the urgent</h3><p>I mentioned above that <code>requestIdleCallback()</code> doesn’t come with any guarantees that the callback will ever run. And when talking to developers about <code>requestIdleCallback()</code>, this is the primary explanation I hear for why they don’t use it. In many cases the possibility that code might not run is enough of a reason not to use it—to play it safe and keep their code synchronous (and therefore blocking).</p><p>A perfect example of this is analytics code. The problem with analytics code is there are many cases where it needs to run when the page is unloading (e.g. tracking outbound link clicks, etc.), and in such cases <code>requestIdleCallback()</code> is simply not an option because the callback would never run. And since analytics libraries don’t know when in the page lifecycle their users will call their APIs, they also tend to play it safe and run all their code synchronously (which is unfortunate since analytics code is definitely not critical to the user experience).</p><p>But with the <em>idle-until-urgent</em> pattern, there’s a simple solution to this. All we have to do is ensure the queue is run immediately whenever the page is in a state where it might soon be unloaded.</p><p>If you’re familiar with the advice I give in my recent article on the <a href="https://developers.google.com/web/updates/2018/07/page-lifecycle-api">Page Lifecycle API</a>, you’ll know that <a href="https://developers.google.com/web/updates/2018/07/page-lifecycle-api#advice-hidden">the last reliable callback</a> developers have before a page gets terminated or discarded is the <code>visibilitychange</code> event (as the page’s <code>visibilityState</code> changes to hidden). And since in the hidden state the user cannot be interacting with the page, it’s a perfect time to run any queued idle tasks.</p><p>In fact, if you use the <code>IdleQueue</code> class, you can enable this ability with a simple configuration option passed to the constructor.</p><pre><code class="language-js"><span class="hljs-keyword">const</span> queue = <span class="hljs-keyword">new</span> IdleQueue(<mark>{<span class="hljs-attr">ensureTasksRun</span>: <span class="hljs-literal">true</span>}</mark>); </code></pre><p>For tasks like rendering, there’s no need to ensure tasks run before the page unloads, but for tasks like saving user state and sending end-of-session analytics, you’ll likely want to set this option to true.</p><aside class="Info"><p><strong>Note:</strong> listening for the <code>visibilitychange</code> event should be sufficient to ensure tasks run before the page is unloaded, but due to Safari bugs where <a href="https://github.com/GoogleChromeLabs/page-lifecycle/issues/2">the pagehide and visibilitychange events don’t always fire</a> when users close a tab, you have to implement a small workaround just for Safari. This workaround <a href="https://github.com/GoogleChromeLabs/idlize/blob/master/IdleQueue.mjs#L60-L69">is implemented for you</a> in the <code>IdleQueue</code> class, but if you’re implementing this yourself, you’ll need to be aware of it.</p></aside><aside class="Info Info--warning"><p><strong>Warning!</strong> Do not listen for the <code>unload</code> event as a way to run the queue before the page is unloaded. The unload event is not reliable and it can hurt performance in some cases. See my <a href="https://developers.google.com/web/updates/2018/07/page-lifecycle-api#the-unload-event">Page Lifecycle API article</a> for more details.</p></aside><h2 id="use-cases-for-idle-until-urgent">Use cases for idle-until-urgent</h2><p>Any time you have potentially-expensive code you need to run, you should try to break it up into smaller tasks. And if that code isn’t needed right away but may be needed at some point in the future, it’s a perfect use case for <em>idle-until-urgent</em>.</p><p>In your own code, the first thing I’d suggest to do is look at all your constructor functions, and if any of them run potentially-expensive operations, refactor them to use an <a href="https://github.com/GoogleChromeLabs/idlize/blob/master/docs/IdleValue.md"><code>IdleValue</code></a> object instead.</p><p>For other bits of logic that are essential but not necessarily critical to immediate user interactions, consider adding that logic to an <a href="https://github.com/GoogleChromeLabs/idlize/blob/master/docs/IdleQueue.md"><code>IdleQueue</code></a>. Don’t worry, if at any time you need to run that code immediately, you can.</p><p>Two specific examples that are particularly amenable to this technique (and are relevant to a large percentage of websites out there) are persisting application state (e.g. with something like Redux) and analytics.</p><aside class="Info"><p><strong>Note:</strong> these are all use cases where the <em>intention</em> is that tasks should run during idle periods, so it’s not a problem if they don’t run right away. If you need to handle high-priority tasks where the <em>intention</em> is they should run as soon as possible (yet still yielding to input), then <code>requestIdleCallback()</code> may not solve your problem.</p><p>Fortunately, some of my colleagues have proposals for new web platform APIs (<a href="https://discourse.wicg.io/t/shouldyield-enabling-script-to-yield-to-user-input/2881/17"><code>shouldYield()</code></a>, and a native <a href="https://github.com/spanicker/main-thread-scheduling/blob/master/README.md">Scheduling API</a>) that should help.</p></aside><h3 id="persisting-application-state">Persisting application state</h3><p>Consider a Redux app that stores application state in memory but also needs to store it in persistent storage (like localStorage) so it can be reloaded the next time the user visits the page.</p><p>Most Redux apps that store state in localStorage use a debounce technique roughly equivalent to this:</p><pre><code class="language-js"><span class="hljs-keyword">let</span> debounceTimeout; <span class="hljs-comment">// Persist state changes to localStorage using a 1000ms debounce.</span> store.subscribe(<span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> { <span class="hljs-comment">// Clear pending writes since there are new changes to save.</span> clearTimeout(debounceTimeout); <span class="hljs-comment">// Schedule the save with a 1000ms timeout (debounce),</span> <span class="hljs-comment">// so frequent changes aren't saved unnecessarily.</span> debounceTimeout = setTimeout(<span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> { <span class="hljs-keyword">const</span> jsonData = <span class="hljs-built_in">JSON</span>.stringify(store.getState()); localStorage.setItem(<span class="hljs-string">'redux-data'</span>, jsonData); }, <span class="hljs-number">1000</span>); }); </code></pre><p>While using a debounce technique is definitely better than nothing, it’s not a perfect solution. The problem is there’s no guarantee that when the debounced function does run, it won’t block the main thread at a time critical to the user.</p><p>It’s much better to schedule the localStorage write for an idle time. You can convert the above code from a debounce strategy to an <em>idle-until-urgent</em> strategy as follows:</p><pre><code class="language-js"><span class="hljs-keyword">const</span> queue = <span class="hljs-keyword">new</span> IdleQueue(); <span class="hljs-comment">// Persist state changes when the browser is idle, and</span> <span class="hljs-comment">// only persist the most recent changes to avoid extra work.</span> store.subscribe(<span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> { <span class="hljs-comment">// Clear pending writes since there are new changes to save.</span> queue.clearPendingTasks(); <span class="hljs-comment">// Schedule the save to run when idle.</span> queue.pushTask(<span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> { <span class="hljs-keyword">const</span> jsonData = <span class="hljs-built_in">JSON</span>.stringify(store.getState()); localStorage.setItem(<span class="hljs-string">'redux-data'</span>, jsonData); }); }); </code></pre><p>And note that this strategy is definitely better than using debounce because it guarantees the state gets saved even if the user is navigating away from the page. With the debounce example, the write would likely fail in such a situation.</p><h3 id="analytics">Analytics</h3><p>Another perfect use case for <em>idle-until-urgent</em> is analytics code. Here’s an example of how you can use the <code>IdleQueue</code> class to schedule sending your analytics data in a way that ensures it <em>will be sent</em> even if the user closes the tab or navigates away before the next idle period.</p><pre><code class="language-js"><span class="hljs-keyword">const</span> queue = <span class="hljs-keyword">new</span> IdleQueue(); <span class="hljs-keyword">const</span> signupBtn = <span class="hljs-built_in">document</span>.getElementById(<span class="hljs-string">'signup'</span>); signupBtn.addEventListener(<span class="hljs-string">'click'</span>, () =&gt; { <span class="hljs-comment">// Instead of sending the event immediately, add it to the idle queue.</span> <span class="hljs-comment">// The idle queue will ensure the event is sent even if the user</span> <span class="hljs-comment">// closes the tab or navigates away.</span> queue.pushTask(<span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> { ga(<span class="hljs-string">'send'</span>, <span class="hljs-string">'event'</span>, { <span class="hljs-attr">eventCategory</span>: <span class="hljs-string">'Signup Button'</span>, <span class="hljs-attr">eventAction</span>: <span class="hljs-string">'click'</span>, }); }); }); </code></pre><p>In addition to ensuring the urgent, adding this task to the idle queue also ensures it won’t block any other code that’s needed to respond to the user’s click.</p><p>In fact, it’s generally a good idea to run <em>all</em> your analytics code idly, including your initialization code. And for libraries like analytics.js whose <a href="https://developers.google.com/analytics/devguides/collection/analyticsjs/how-analyticsjs-works#the_ga_command_queue">API is already effectively a queue</a>, it’s easy to just add these commands to our <code>IdleQueue</code> instance.</p><p>For example, you can convert the last part of the <a href="https://developers.google.com/analytics/devguides/collection/analyticsjs/#the_javascript_tracking_snippet">default analytics.js installation snippet</a> from this:</p><pre><code class="language-js">ga(<span class="hljs-string">'create'</span>, <span class="hljs-string">'UA-XXXXX-Y'</span>, <span class="hljs-string">'auto'</span>); ga(<span class="hljs-string">'send'</span>, <span class="hljs-string">'pageview'</span>); </code></pre><p>Into this:</p><pre><code class="language-js"><span class="hljs-keyword">const</span> queue = <span class="hljs-keyword">new</span> IdleQueue(); queue.pushTask(<span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> ga(<span class="hljs-string">'create'</span>, <span class="hljs-string">'UA-XXXXX-Y'</span>, <span class="hljs-string">'auto'</span>)); queue.pushTask(<span class="hljs-function"><span class="hljs-params">()</span> =&gt;</span> ga(<span class="hljs-string">'send'</span>, <span class="hljs-string">'pageview'</span>)); </code></pre><p>(You could also just create a wrapper around the <code>ga()</code> function that automatically queues commands, which is <a href="https://github.com/philipwalton/blog/blob/0670d46/assets/javascript/analytics.js#L114-L127">what I did</a>).</p><h2 id="browser-support-for-requestidlecallback">Browser support for requestIdleCallback</h2><p>As of this writing, only Chrome and Firefox support <code>requestIdleCallback()</code>. And while a true polyfill isn’t really possible (only the browser can know when it’s idle), it’s quite easy to write a fallback to <code>setTimeout</code> (all the helper classes and methods mentioned here <a href="https://github.com/GoogleChromeLabs/idlize/blob/master/docs/idle-callback-polyfills.md">use this fallback</a>).</p><p>And even in browsers that don’t support <code>requestIdleCallback()</code> natively, the fallback to <code>setTimeout</code> is definitely still better than not using this strategy because browsers can still do input prioritization ahead of tasks queued via <code>setTimeout()</code>.</p><h2 id="how-much-does-this-actually-improve-performance%3F">How much does this actually improve performance?</h2><p>At the beginning of this article I mentioned I came up with this strategy as I was trying to improve my website’s FID value. I was trying to split up all the code that ran as soon as my main bundle was loaded, but I also needed to ensure my site continued to work with some third-party libraries that only have synchronous APIs (e.g. analytics.js).</p><p>The trace I showed before implementing <em>idle-until-urgent</em> had a single, 233ms task that contained all my initialization code. After implementing the techniques I described here, you can see I have multiple, much shorter tasks. In fact, the longest one is now only 37ms!</p><figure><a href="https://philipwalton.com/static/idle-until-urget-after-1400w-d526f6cca8.png"><img srcset="/static/idle-until-urget-after-1400w-d526f6cca8.png, /static/idle-until-urget-after-4789aca119.png 700w" src="https://philipwalton.com/static/idle-until-urget-after-4789aca119.png" alt="A performance trace of my site's JavaScript showing many short tasks"/></a><figcaption>A performance trace of my site's JavaScript showing many short tasks.</figcaption></figure><p>A really important point to emphasize here is that the same amount of work is being done as before, it’s just now spread out over multiple tasks and run during idle periods.</p><p>And since no single task is greater than 50ms, none of them affect my time to interactive (TTI), which is great for my lighthouse score:</p><figure><a href="https://philipwalton.com/static/lighthouse-report-1400w-1136c250ac.png"><img srcset="/static/lighthouse-report-1400w-1136c250ac.png, /static/lighthouse-report-4721b091da.png 700w" src="https://philipwalton.com/static/lighthouse-report-4721b091da.png" alt="My lighthouse report after implementing idle-until-urget - 100s across the board!"/></a><figcaption>My lighthouse report after implementing <em>idle-until-urget</em>.</figcaption></figure><p>Lastly, since the point of all this work was to improve my FID, after releasing these changes to production and looking at the results, I was thrilled to discover <em>a 67% reduction in FID values at the 99th percentile!</em></p><table><tr><th>Code version</th><th>FID (p99)</th><th>FID (p95)</th><th>FID (p50)</th></tr><tr><td>Before <em>idle-until-urgent</em></td><td><strong>254ms</strong></td><td>20ms</td><td>3ms</td></tr><tr><td>After <em>Idle-until-urgent</em></td><td><strong>85ms</strong></td><td>16ms</td><td>3ms</td></tr></table><h2 id="conclusions">Conclusions</h2><p>In a perfect world, none of our sites would ever block the main thread unnecessarily. We’d all be using web workers to do our non-UI work, and we’d have <a href="https://discourse.wicg.io/t/shouldyield-enabling-script-to-yield-to-user-input/2881/17"><code>shouldYield()</code></a> and a native <a href="https://github.com/spanicker/main-thread-scheduling/blob/master/README.md">Scheduling API</a>) built into the browser.</p><p>But in our current world, we web developers often have no choice but to run non-UI code on the main thread, which leads to unresponsiveness and jank.</p><p>Hopefully this article has convinced you of the need to break up our long-running JavaScript tasks. And since <em>idle-until-urgent</em> can turn a synchronous-looking API into something that actually evaluates code in idle periods, it’s a great solution that works with the libraries we all know and use today.</p> Thu, 20 Sep 2018 09:16:43 +0000 https://philipwalton.com/articles/idle-until-urgent/ The Making of Underrun – A WebGL Shooter in 13kb of JavaScript https://phoboslab.org/log/2018/09/underrun-making-of https://phoboslab.org/log/2018/09/underrun-making-of <p>I participated in this year's <a href="https://js13kgames.com/">js13kGames</a>, a JavaScript game development competition with a file size limit of 13kb, including code, assets and everything else. My entry was <a href="https://phoboslab.org/underrun/">Underrun</a>, a twin stick shooter using WebGL.</p> <p><a href="https://phoboslab.org/underrun/"><img src="https://phoboslab.org/content/assets/underrun.png" alt="Underrun" /></a> <em>Play Underrun – A WebGL shooter in 13kb of JavaScript</em></p> <p>For this competition I set out to produce something with a dense atmosphere – which is inherently difficult to do with so little space. In my experience a dense atmosphere is most easily achieved with two things: moody lighting and rich sound &amp; music. The assets for sound &amp; music usually take up a lot of space and lighting for 2D games requires more and bigger graphic assets. With only 13kb to space I had to look for different solutions for these things.</p> <h2>Graphic Assets</h2> <p>Since I was very limited with the amount of graphic assets I could fit in this game the decision to implement a 3D perspective came naturally: atmospheric lighting is easier to do in 3D than in 2D and requires less assets to look neat. In contrast, to produce interesting light effects in 2D typically requires you to implement a third dimension anyway. This can be done through normal maps <a href="https://www.gamasutra.com/view/news/312977/Adding_depth_to_2D_with_handdrawn_normal_maps_in_The_Siege_and_the_Sandfox.php">as some 2D Pixel Art games do</a> or by explicitly separating your environment into different layers as <a href="https://store.steampowered.com/app/234390/Teleglitch_Die_More_Edition/">Teleglitch</a> does for example.</p> <p>My approach for the graphics was to render a simple 2D tilemap with a bunch of textured quads. All walls are rendered as cubes and entities (the player, enemies, projectiles, particles and health pickups) are just simple sprites. All textures for the game fit in a single tile sheet, 2.12kb in size.</p> <p><img src="https://phoboslab.org/content/assets/underrun-assets-scaled.png" alt="Underrun Assets" /><em>All graphic assets for the game, carefully tuned to 16 colors</em></p> <p>I opted to render the player character as a sprite as well, even though it needs to be freely rotated. It was far easier to produce 8 different sprites – one for each direction – than to implement a loader and renderer for complex 3D models. This allowed me to omit <em>any</em> 3D vector or matrix math operations. The game never needs to rotate any geometry. Everything is rendered with axis aligned quads.</p> <p>To make the player character rotation look convincing I build a 3D model and simply took screenshots for each of the different perspectives. I'm a total doofus when it comes to 3D modeling software, but that doesn't stop me from using the ingenious <a href="http://www.wings3d.com/">Wings3D</a>. The result doesn't look like much, but scaled down to 16px, it doesn't matter.</p> <p><img src="https://phoboslab.org/content/assets/underrun-guy-wings3d.png" alt="title" /><em>The player character model built in Wings3D</em></p> <p>The final game features 3 different levels, each 64×64 tiles in size. When I started to work on the game I considered to use Run Length Encoding to compress the level maps. However, even as simple as RLE is, a decompressor for it would have taken up some valuable space. So my solution was to just let the browser handle the decompression by using PNG images.</p> <p><img src="https://phoboslab.org/content/assets/underrun-levels-scaled.png" alt="title" /><em>Underrun's levels are stored as PNG images</em></p> <p>With the PNG compression, each level image is just about 300bytes in size. A naive approach would have needed 64×64 bytes = 4kb per level (assuming storage as raw binary, 1byte per tile).</p> <p>When loading the level a random number generator (RNG) is used to determine exactly which tile graphic to use for each floor and wall. The RNG also decides where to place enemies and powerups, as these are not encoded in the level images. To make this consistent between playthroughs, so that each player plays exactly the same game, I implemented a tiny seedable RNG (<a href="https://github.com/phoboslab/underrun/blob/master/source/random.js">source code</a>) and re-seeded it with the same constant before loading each level.</p> <p>I also hardcoded some of the wall tiles to be rendered with twice the normal height to make it look more interesting. You can see the complete level loading routines in the <a href="https://github.com/phoboslab/underrun/blob/master/source/game.js#L45">load_level() function</a>.</p> <h2>Rendering</h2> <p>The renderer follows an extremely simple design: a single buffer is used to store all vertices, texture coordinates and normals. A second buffer is used to store all light sources (position, color, attenuation). The game can write to these buffers using the <code>push_quad()</code> and <code>push_light()</code> functions. The renderer clears these buffers at the beginning of each frame and draws the (filled) buffers at the end of the frame.</p> <p>There's one small optimization to this. During level load, since all the level geometry is static, the game sets a <em>reset mark</em> for the geometry buffer after all the level geometry has been pushed. Before each frame the renderer will then reset the write position for the geometry buffer to this <em>reset mark</em> instead of to <code>0</code>. This clears all the entities, but we don't we don't have to re-push the level data.</p> <p>Since the game's graphics are quite simple, there's no need for any occlusion culling or any other optimizations to reduce the amount of geometry being drawn. Underrun draws the whole level – all floor an ceiling tiles and all sprites – for every single frame in a single draw call.</p> <p>The gist of the renderer looks like this:</p> <pre><code class="language-js"><span class="keyword">var</span> num_verts <span class="punct">=</span> <span class="number">0</span><span class="punct">,</span> level_num_verts <span class="punct">=</span> <span class="number">0</span><span class="punct">,</span> max_verts <span class="punct">=</span> <span class="number">1024</span> <span class="punct">*</span> <span class="number">64</span><span class="punct">,</span> buffer_data <span class="punct">=</span> <span class="keyword">new</span> Float32Array<span class="punct">(</span>max_verts <span class="punct">*</span> <span class="number">8</span><span class="punct">);</span> <span class="keyword">function</span> push_quad<span class="punct">(</span>x1<span class="punct">,</span> y1<span class="punct">,</span> z1<span class="punct">,</span> <span class="punct">...)</span> <span class="punct">{</span> buffer_data<span class="punct">.</span>set<span class="punct">([</span>x1<span class="punct">,</span> y1<span class="punct">,</span> z1<span class="punct">,</span> <span class="punct">...],</span> num_verts <span class="punct">*</span> <span class="number">8</span><span class="punct">);</span> num_verts <span class="punct">+=</span> <span class="number">6</span><span class="punct">;</span> <span class="punct">}</span> <span class="keyword">function</span> load_level<span class="punct">(</span>data<span class="punct">)</span> <span class="punct">{</span> num_verts <span class="punct">=</span> <span class="number">0</span><span class="punct">;</span> <span class="keyword">for</span> <span class="punct">(</span><span class="keyword">var</span> y <span class="punct">=</span> <span class="number">0</span><span class="punct">;</span> y <span class="punct">&lt;</span> <span class="number">64</span><span class="punct">;</span> y<span class="punct">++)</span> <span class="punct">{</span> <span class="keyword">for</span> <span class="punct">(</span><span class="keyword">var</span> x <span class="punct">=</span> <span class="number">0</span><span class="punct">;</span> x <span class="punct">&lt;</span> <span class="number">64</span><span class="punct">;</span> x<span class="punct">++)</span> <span class="punct">{</span> <span class="punct">}</span> <span class="punct">}</span> level_num_verts <span class="punct">=</span> num_verts<span class="punct">;</span> <span class="punct">}</span> <span class="keyword">function</span> renderer_prepare_frame<span class="punct">()</span> <span class="punct">{</span> num_verts <span class="punct">=</span> level_num_verts<span class="punct">;</span> <span class="punct">}</span> <span class="keyword">function</span> renderer_end_frame<span class="punct">()</span> <span class="punct">{</span> gl<span class="punct">.</span>bufferData<span class="punct">(</span>gl<span class="punct">.</span>ARRAY_BUFFER<span class="punct">,</span> buffer_data<span class="punct">,</span> gl<span class="punct">.</span>DYNAMIC_DRAW<span class="punct">);</span> gl<span class="punct">.</span>drawArrays<span class="punct">(</span>gl<span class="punct">.</span>TRIANGLES<span class="punct">,</span> <span class="number">0</span><span class="punct">,</span> num_verts<span class="punct">);</span> <span class="punct">};</span> <span class="keyword">function</span> tick<span class="punct">()</span> <span class="punct">{</span> renderer_prepare_frame<span class="punct">();</span> <span class="keyword">for</span> <span class="punct">(</span><span class="keyword">var</span> i <span class="punct">=</span> <span class="number">0</span><span class="punct">;</span> i <span class="punct">&lt;</span> entities<span class="punct">.</span>length<span class="punct">;</span> i<span class="punct">++)</span> <span class="punct">{</span> entities<span class="punct">[</span>i<span class="punct">].</span>update<span class="punct">();</span> entities<span class="punct">[</span>i<span class="punct">].</span>draw<span class="punct">();</span> <span class="punct">}</span> renderer_end_frame<span class="punct">();</span> requestAnimationFrame<span class="punct">(</span>tick<span class="punct">);</span> <span class="punct">}</span></code></pre> <p>For the lighting I opted for some very simple vertex lights. Since the game only renders relatively small quads, it doesn't matter much that the light computation is handled in the vertex shader instead of per-pixel in the fragment shader. This also allowed for many more light sources. The game currently allows for 32 lights, each of which considered for every single vertex. Current GPUs are so stupidly fast that no optimizations are needed for such simple cases.</p> <p>The light calculation in Underrun's Vertex Shader looks somewhat like this:</p> <pre><code class="language-js">uniform <span class="keyword">float</span> lights<span class="punct">[</span><span class="number">7</span> <span class="punct">*</span> max_lights<span class="punct">];</span> varying vec3 vertex_light<span class="punct">;</span> <span class="keyword">void</span> main<span class="punct">(</span><span class="keyword">void</span><span class="punct">){</span> vertex_light <span class="punct">=</span> vec3<span class="punct">(</span><span class="number">0</span><span class="punct">.</span><span class="number">3</span><span class="punct">,</span> <span class="number">0</span><span class="punct">.</span><span class="number">3</span><span class="punct">,</span> <span class="number">0</span><span class="punct">.</span><span class="number">6</span><span class="punct">);</span> <span class="keyword">for</span><span class="punct">(</span><span class="keyword">int</span> i <span class="punct">=</span> <span class="number">0</span><span class="punct">;</span> i <span class="punct">&lt;</span> max_lights<span class="punct">;</span> i<span class="punct">++</span> <span class="punct">)</span> <span class="punct">{</span> vec3 light_position <span class="punct">=</span> vec3<span class="punct">(</span>lights<span class="punct">[</span>i<span class="punct">*</span><span class="number">7</span><span class="punct">],</span> lights<span class="punct">[</span>i<span class="punct">*</span><span class="number">7</span><span class="punct">+</span><span class="number">1</span><span class="punct">],</span> lights<span class="punct">[</span>i<span class="punct">*</span><span class="number">7</span><span class="punct">+</span><span class="number">2</span><span class="punct">]);</span> vec3 light_color <span class="punct">=</span> vec3<span class="punct">(</span>lights<span class="punct">[</span>i<span class="punct">*</span><span class="number">7</span><span class="punct">+</span><span class="number">3</span><span class="punct">],</span> lights<span class="punct">[</span>i<span class="punct">*</span><span class="number">7</span><span class="punct">+</span><span class="number">4</span><span class="punct">],</span> lights<span class="punct">[</span>i<span class="punct">*</span><span class="number">7</span><span class="punct">+</span><span class="number">5</span><span class="punct">]);</span> vec3 distance <span class="punct">=</span> length<span class="punct">(</span>light_position <span class="punct">-</span> vertex_position<span class="punct">);</span> <span class="keyword">float</span> attenuation<span class="punct">=</span> <span class="punct">(</span><span class="number">1</span><span class="punct">.</span><span class="number">0</span><span class="punct">/(</span>lights<span class="punct">[</span>i<span class="punct">*</span><span class="number">7</span><span class="punct">+</span><span class="number">6</span><span class="punct">]</span> <span class="punct">*</span> distance<span class="punct">);</span> vertex_light <span class="punct">+=</span> light_color <span class="punct">*</span> max<span class="punct">(</span>dot<span class="punct">(</span>normal<span class="punct">,</span> normalize<span class="punct">(</span>dist<span class="punct">)),</span> <span class="number">0</span><span class="punct">.</span><span class="number">0</span><span class="punct">)</span> <span class="punct">*</span> attenuation<span class="punct">;</span> <span class="punct">}</span> <span class="punct">}</span></code></pre> <p>For some extra scary atmosphere the fragment shader calculates a simple black fog based on the distance to the camera. An exception is made to the enemies' eyes: all full red texture pixels are always rendered unchanged – no light, no fog. This makes them essentially glow in the dark.</p> <p>The fragment shader also reduces the final image to 256 colors. I experimented with some color tinting but ultimately settled for a uniform rgb palette. Rendering in 256 colors and a screen size of just 320×180 pixels not only gives the game an old-school vibe, but also hides a lot of the graphical shortcomings.</p> <p>Have a look at the source code for the <a href="https://github.com/phoboslab/underrun/blob/master/source/renderer.js">renderer.js</a> and <a href="https://github.com/phoboslab/underrun/blob/master/source/game.js">game.js</a> - there's really not that much to it.</p> <h2>Music &amp; Sounds</h2> <p>The sound and music for all my previous games made up the bulk of the data size. E.g. <a href="https://phoboslab.org/xibalba/">Xibalba</a> has about 800kb of graphic assets (tile sheets, sprites, title screen etc.) but more than 4mb of music and 2.5mb of sound effects (actually 8mb and 5mb respectively, since all assets need to be provided as .ogg and .mp3). This was of course infeasible when you only have 13kb total to work with.</p> <p>So for Underrun all music and sound effects are generated in JavaScript when the game starts. The game uses the brilliant <a href="https://github.com/nicolas-van/sonant-x">Sonant-X</a> library to render <em>instruments</em> from a bunch of parameters into sound waves.</p> <p>My dear friend Andreas Lösch of <a href="http://no-fate.net">no-fate.net</a> produced the instruments and sequences for the music in the tracker <a href="https://nicolas-van.github.io/sonant-x-live/#N4IgTg9g7gMgpgOxALgKyoIwGYA0JEAmACgIYAuZcYSyATKngM4QIDmAIuSSgNqgSMAxhgD6EQWRQB2PAOEiCcScgAMsoaMVkArgjgo1IOaIAeiAG4H188xAA2KegyMaRUEubgAzCGAC2jta0YhIoABxBCkpWLoLBWrr6yBgRscFmCJaqkbYOdOiR7p4%2B%2Fii4IAgQAJaMcCJeJIpgMRYi5GQkggDWjhgqUgAseK2M2owdVTSoAGxYs8OZImBwdnAktSh9KtuGrX7rlM3JAJzHeF4m9VV2h4EgF%2FXLAI4oxwOpD8vMCCQIgknHWjnS6KOwkACeIjIVT8SRk9xBKwhbT8ymmhgeAAdfo84C9kOUsTiSKjNts8HYfGJXF5njFKRB6pdaXjNhSqSz8fCGSjlNhUjyit5fAECXhMbwgbghgwpTgZTg5QqlThZThparFer5aqALp4QS8UA0HhqM04c2Wi3Wq22m32u2Om26gC%2BOGNvAwtCBTt9DutXow%2Fr9vtd7oqnowQxDwdjMfjFrDHuQPAwGDOCZjgbjOcdSYjKbT0dzJczAYwMhDrvzxhCynhta0MUbSkSzdcGSyhlruU2gOsoiFJVF5TkwXEyiGaSiym7Qnirb07biIk7y%2BCvbQDfnbg8wtK2Qq1Vq9UaVBai3anR6yXJ%2BEWo3GJEmm1Qu0Wy1W6yS2Fm0wW5giPs4znicQYIlcNygUCEGcigqBhFgZwQV8LC%2FP8jioFODygsi0Kwig2GImCkIknytDgUSCC4vi%2F4Qdi1FkZsAzclStZwYePIPBxhhcXSyB0TyTEnM4gp7sOMQSimPpamoMnydaClyYpin6iAhopsmpqljpZbxvmJrYEGahGQGWDGfa2C4CZWDFtgxZ6bp1ZuvwrgTtIA4zncLY6Euh61muySeZu6Z0bWQ4iuudYedOTb%2BTuCR%2BXOK6BeBY4iJuTiFOJkWHpUNR1A0TQXoBV7dGS76AY%2BEw0JVSwrGsGwEn%2BAFAQc0FYcCkG3HQXUcdMXpdahPx%2FEk9BEVEJFQjCSR0ThSKkaSdDwlRNExKtwm8WxNL8VtjLcbt7L7YdIBCUte27sUuWGFJqY4EGD33U9j0vfdakaXwBapgMCZRr9P06X9pbOeGtbucg27yHFyWaIuSQw6uFjLqIm5SJDg45QeaU7uDMHpdDkSJT%2BYSpOlgXJRu9gxelEUHoY%2BUnkV0Hga0ZU3qgWBSFgMEjGMNUlfVX5NVGWHOHs7VHNgfXXD1GLMidnxwN86FjQDEG4ZC%2BFjV1Gu8o43pdQxa1ivRxJLSxR3UvIPGWwdrKcRyJ1nbOlu06KN28HaQJDHK3targfu%2BzqQc%2B%2Bq71Gl9jlR7pDoGZ65lmRZplWlZZl2bZMdR3HhYJzZSe55Zcxp2Z1mZ5n2epgX0fV8DLo1m5oRoJ5BOxLDvnw55qXBVTdAI27UXg5DC4u7FcNRV306o9lV103gDOFWektSKkrMUNeAvVc%2BNCzFgK8fg136vgMnX3oBwE9Vs0tQUchLy%2FbAxYKgMGK8ro2OGr81TVrhE6wtevIKcQ2OIbam0YktWgF12JOw5HffE4E%2BL21YoyYS0wpxiRnu7cUnt%2FSvVwW9A0EcTS%2FU5mXP02BKy5hBq5eQ4NSauBbj5Ns8V5ATx7D3MIeNXD916tOWhhNojMOHkwimiNMhRU3CI7h9NjwL2KskegrU2YVVapvF8h5WifkagCFQxw6rn2gtzK%2BPVb7GyfneF%2BaE34CRMbrb%2ByAJq62EqZUBxs5YiCNptS2UD7YXTtviXx0DkHnVdpjTBIBbo1wsuaKJAZw6aS%2BlGB6AwkkpNIWk%2FSdc65MBYKweANA%2BgYBdEAA">Sonant-X Live</a>. The sound effects were produced in Sonant-X Live as well; each effect being a single instrument.</p> <p><img src="https://phoboslab.org/content/assets/underrun-sonant-x-live.png" alt="Sonant-X" /><em>Underrun's Music, produced in the Sonant-X Live tracker</em></p> <p>To save some more space I began to remove all the parts from Sonant-X that were not needed for my game, leaving only the WebAudio generator. In doing so I noticed that the library was unnecessary slow, taking half a second to generate an <em>empty</em> buffer object. The culprit was the use of a single, interleaved Uint8 Buffer for the left and right audio channel, storing unsigned 16bit values with the implicit zero amplitude at 32767.</p> <p>The library was spending a lot of time loading from and storing 16bit values into this 8bit buffer and later converting it to a signed float suitable for WebAudio. I believe this is an artifact from the original use case of this library: producing a WAV data URI.</p> <p>I refactored the library to use two Float32Arrays (one for each channel) that can be directly used by the WebAudio API in the browser. This not only simplified the code and reduced the file size by 30% but also made the music generation twice as fast.</p> <p>As an example, have a look at the <a href="https://github.com/nicolas-van/sonant-x/blob/master/sonantx.js#L105-L139">original applyDelay() function</a> and my <a href="https://github.com/phoboslab/underrun/blob/68d831dbcdbdb044161eae1c2c0918986704a973/source/sonantx-reduced.js#L85-L97">modified one</a> - granted, I also removed the pausing/re-scheduling functions here that originally prevented blocking the event loop with a long running computation, but it wasn't really needed anymore.</p> <p>All in all, minified and gzipped, the music and sound for Underrun along with the Sonant-X library now occupy only about 2kb. That's a pretty big deal, considering all my previous games' biggest asset were sound and music. So even if your game is not size restricted as much, it may make sense to generate audio directly in your game, instead of adding big files to the download.</p> <h2>Minification</h2> <p>When I started this project I took great care to write as little code as possible and to make sure all my code could be minified effectively by UglifyJS. The source has quite an unusual style compared to my other JS projects. It's using a lot of global variables and functions, instead of abstracting it more neatly into classes. I also wrote the code in the more C-like <code>snake_case</code> style to force my brain into this different mode.</p> <p>In hindsight, this was totally unnecessarily. Minifying and zipping would have gotten rid of most of the overhead of using more abstractions.</p> <p>One trick that made a small difference is the "inlining" of all WebGL constants at build-time. E.g. <code>gl.ONE_MINUS_SRC_ALPHA</code> is replaced with just <code>771</code> - the constant's value. I also shortened all the WebGL function calls by producing aliases that just contained the first two letters and all subsequent upper-case letters of the function's original name:</p> <pre><code class="language-js"><span class="keyword">for</span> <span class="punct">(</span><span class="keyword">var</span> name in gl<span class="punct">)</span> <span class="punct">{</span> <span class="keyword">if</span> <span class="punct">(</span>gl<span class="punct">[</span>name<span class="punct">].</span>length <span class="punct">!==</span> undefined<span class="punct">)</span> <span class="punct">{</span> gl<span class="punct">[</span>name<span class="punct">.</span>match<span class="punct">(</span><span class="regexp">/(^..|[A-Z]|\d.|v$)/g</span><span class="punct">).</span>join<span class="punct">(</span><span class="string">''</span><span class="punct">)]</span> <span class="punct">=</span> gl<span class="punct">[</span>name<span class="punct">];</span> <span class="punct">}</span> <span class="punct">}</span></code></pre> <p>This allowed me to use <code>gl.getUniformLocation()</code> as just <code>gl.geUL()</code>. Note though that this approach is quite hack-ish and produces some name collisions. But for my game it worked out nicely.</p> <p>I also took a lot of shortcuts with the game logic and especially collision detection. Collision detection between entities is just using two nested loops, checking each entity against all other entities. This quadratic function quickly explodes with a lot of entities in the level (e.g. 100 entities = 10.000 comparisons) - it's just amazing with what you can get away with these days.</p> <p>Other than that I didn't do anything particularly clever or exciting to further minify the game.</p> <p>The full source for Underrun is on github: <a href="http://github.com/phoboslab/underrun/">github.com/phoboslab/underrun</a></p> <p>Be sure to have a look at some of the other <a href="https://js13kgames.com/entries">js13Games entries</a> this year. Some of my favorites are <a href="https://js13kgames.com/entries/the-chroma-incident">The Chroma Incident</a>, <a href="https://js13kgames.com/entries/1024-moves">1024 Moves</a>, <a href="https://js13kgames.com/entries/isp">ISP</a>, <a href="https://js13kgames.com/entries/wander">Wander</a> and <a href="https://js13kgames.com/entries/envisionator">Envisonator</a>.</p> Thu, 20 Sep 2018 10:01:33 +0000 https://phoboslab.org/log/2018/09/underrun-making-of Show HN: Sublime Merge – A Git client from the makers of Sublime Text https://www.sublimemerge.com https://www.sublimemerge.com <div class="feature odd" readability="19"> <h3>Integrated Merge Tool</h3> <p>The <span class="feature">Integrated Merge Tool</span> allows you to resolve any merge conflicts directly in Sublime Merge, rather than having to open up your editor of choice.</p> <p>Conflicts are presented with a 3-pane view. On the left are your changes and on the right are theirs. In the center pane is the resolved text, with buttons to choose between your changes or theirs. The same text editing functionality as Sublime Text is also available for more complicated merges.</p> <p>Clicking on the header in the middle pane will switch between the editable merge results and the base file.</p> <p>Use <span class="pc keys"><b>Ctrl</b><i>+</i><b>S</b></span><span class="mac keys"><b>⌘</b><i>+</i><b>S</b></span> or the button to save your merged file.</p> </div> <div class="feature even" readability="20"> <h3>Powerful Search</h3> <p>Use find-as-you-type search to dig up the exact commit you're looking for.</p> <p>Search for commit messages, commit authors, file names, and wildcard patterns. Complex search queries can be constructed using <kbd>and</kbd>, <kbd>or</kbd> and <kbd>()</kbd> symbols.</p> <p>For example <kbd>min-parents:2 author:jskinner path:tests/* (bug fix or test)</kbd> will search for a merge commit from jskinner inside the tests directory that contain the words "bug" and "fix" or the word "test".</p> <p>You can open search with <span class="pc keys"><b>Ctrl</b><i>+</i><b>F</b></span><span class="mac keys"><b>⌘</b><i>+</i><b>F</b></span> or through the menu: .</p> </div> <div class="feature odd" readability="14"> <h3>Unmatched Performance</h3> <p>Sublime Merge is built on the same custom platform as Sublime Text, providing unmatched responsiveness. With a powerful, cross-platform UI toolkit, an unmatched syntax highlighting engine, and a custom high-performance Git reading library, Sublime Merge sets the bar for performance.</p> </div> <div class="feature even" readability="18"> <h3>Blame</h3> <p>Open the <span class="feature">Blame</span> of a file through the <span class="feature">Command Palette</span> or when viewing a commit to see exactly which lines of the file were added by which commits.</p> <p>Easily view the age, author, and commit hash of every line of code and see which lines come from the same commit with our commit colour coding. Click a line in the gutter to highlight all other lines from the same commit.</p> <p>The blame tool will also detect when a piece of code was moved from another place in your repository, so you can truly follow the history of your code.</p> </div> <div class="feature odd" readability="15"> <h3>Advanced Diffs</h3> <p>Where it makes sense we will show you exactly which individual characters have been changed for a commit.</p> <p>This includes when you rename or move a file, when you're resolving conflicts or just viewing commit history.</p> <p> Select any two commits in Sublime Merge with <span class="pc keys"><b>Ctrl</b><i>+</i><b>Left Mouse</b></span><span class="mac keys"><b>⌘</b><i>+</i><b>Left Mouse</b></span> to show the diff between them.</p> </div> <div class="feature even" readability="8"> <h3>File and Hunk History</h3> <p>From any file or hunk in the history use or the button to view the full history of that source code. This will also follow any file moves or renames throughout the repository.</p> </div> <div class="feature odd" readability="15"> <h3>Command Palette and Key Bindings</h3> <p>Keyboard usage is important to us. Use <span class="keys"><b>Tab</b></span> to navigate through parts of the application, <span class="keys"><b>Space</b></span> to toggle expansion, and <span class="keys pc"><b>Enter</b></span><span class="keys mac"><b>Return</b></span> to stage/unstage hunks. When writing a commit message, use <span class="keys pc"><b>Ctrl</b><i>+</i><b>Enter</b></span><span class="keys mac"><b>⌘</b><i>+</i><b>Return</b></span> to commit.</p> <p>The <span class="feature">Command Palette</span> is triggered by <span class="keys pc"><b>Ctrl</b><i>+</i><b>P</b></span><span class="keys mac"><b>⌘</b><i>+</i><b>P</b></span> and allows quick access to a large set of Git commands as well as other Sublime Merge functionality.</p> </div> <div class="feature even" readability="11"> <h3>Light and Dark Themes</h3> <p>Choose between a light and a dark theme to suit your tastes. You can preview these themes above.</p> <p>Open the menu to change theme.</p> <p>The dark theme is only available with a valid license key.</p> </div> <div class="feature odd" readability="12"> <h3>Extensibility</h3> <p>Just like Sublime Text, just about everything in Sublime Merge is extensible. Key bindings, menus, theming, and the command palette are all customizable with simple JSON files.</p> </div> <div class="feature even" readability="11"> <h3>Syntax Highlighting</h3> <p>Sublime Merge performs full syntax highlighting identically to Sublime Text for every line of code you see.</p> <p>We'll even use any extra syntax definitions that we find in your Sublime Text installation for syntax highlighting!</p> </div> <div class="feature odd" readability="13"> <h3>Command Line Integration</h3> <p>Sublime Merge is designed to work hand in hand with the command line. All changes to the repository are updated live and everything works the same way whether it's triggered from the command line or the UI. Use the terminal where it makes sense and use the GUI where it works best.</p> <p>Use the <tt>smerge</tt> tool that comes with Sublime Merge to interact with it from the command line: open repositories, blame files or search for commits</p> </div> <div class="feature even" readability="13"> <h3>Real Git</h3> <p>Sublime Merge uses the same terminology as Git, and doesn't make use of any state beyond Git itself. When you're working with Sublime Merge, you're working with Real Git, not a simplified version</p> <p>You can also hover over most buttons to see which git command it will run.</p> </div> Thu, 20 Sep 2018 07:36:57 +0000 https://www.sublimemerge.com/ New York inmate's golf drawings lead to exoneration in murder https://www.bbc.co.uk/news/world-us-canada-45582610 https://www.bbc.co.uk/news/world-us-canada-45582610 <div class="with-extracted-share-icons"> <div class="story-body__mini-info-list-and-share"> <div class="story-body__mini-info-list-and-share-row"> <div class="mini-info-list-wrap"> <ul class="mini-info-list"><li class="mini-info-list__item"><div class="date date--v2" data-seconds="1537400612" data-datetime="20 September 2018">20 September 2018</div></li> </ul></div> <div class="share-tools--event-tag"> <div id="comp-pattern-library-5" class="distinct-component-group container-twite"> <ul class="sharetools"><li class="twite__channel-out twite__channel-out--desktop twite__channel-out--facebook twite__channel-click-extracted--facebook twite__list-elements" aria-hidden="true"> <a class="extracted__channel-link extracted__channel-link--facebook" onclick="window.open('http://www.facebook.com/dialog/feed?app_id=58567469885&amp;redirect_uri=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fspecial%2Fshared%2Fvj_sharetools%2Ffb_red_uri.html&amp;link=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610%3FSThisFB&amp;display=popup', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=555,height=615').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610" tabindex="-1"> <span class="extracted__icon extracted__icon--facebook" data-platform="facebook"> <svg class="extracted-svg ex-facebook" viewbox="-17 -13 44 44" enable-background="new 0 0 44 44" width="44px" height="44px" aria-hidden="true" focusable="false"><g><path d="M5.73,17 L5.73,9.246 L8.333,9.246 L8.723,6.223 L5.73,6.223 L5.73,4.294 C5.73,3.419 5.973,2.823 7.228,2.823 L8.828,2.822 L8.828,0.119 C8.551,0.082 7.601,0 6.496,0 C4.189,0 2.609,1.408 2.609,3.995 L2.609,6.223 L0,6.223 L0,9.246 L2.609,9.246 L2.609,17 L5.73,17 Z"/></g></svg></span> <span class="off-screen">Share this with Facebook</span> </a> </li> <li class="twite__channel-out twite__channel-out--desktop twite__channel-out--messengerdesktop twite__channel-click-extracted--messengerdesktop twite__list-elements" aria-hidden="true"> <a class="extracted__channel-link extracted__channel-link--messengerdesktop" onclick="window.open('http://www.facebook.com/dialog/send?app_id=58567469885&amp;redirect_uri=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fspecial%2Fshared%2Fvj_sharetools%2Ffb_red_uri.html&amp;link=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610%3FSThisFB&amp;display=popup', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=645,height=580').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610" tabindex="-1"> <span class="extracted__icon extracted__icon--messengerdesktop" data-platform="messengerdesktop"> <svg class="extracted-svg ex-messengerdesktop" viewbox="-14 -13 44 44" enable-background="new 0 0 44 44" width="44px" height="44px" aria-hidden="true" focusable="false"><g><path d="M9.84804801,11.1721834 L7.54389655,8.77893955 L3.1059828,11.2323207 L7.97153994,6.06941781 L10.2756914,8.46377529 L14.7136051,6.00928046 L9.84804801,11.1721834 Z M8.90923715,0 C3.98911093,0 0,3.73074306 0,8.33125039 C0,10.9494525 1.29183939,13.2847862 3.3097816,14.8116068 L3.3097816,18 L6.35117243,16.3139269 C7.16079936,16.5399988 8.02054074,16.6625008 8.90923715,16.6625008 C13.830477,16.6625008 17.8184743,12.9328714 17.8184743,8.33125039 C17.8184743,3.73074306 13.830477,0 8.90923715,0 L8.90923715,0 Z"/></g></svg></span> <span class="off-screen">Share this with Messenger</span> </a> </li> <li class="twite__channel-out twite__channel-out--desktop twite__channel-out--twitter twite__channel-click-extracted--twitter twite__list-elements" aria-hidden="true"> <a class="extracted__channel-link extracted__channel-link--twitter" onclick="window.open('https://twitter.com/intent/tweet?text=BBC%20News%20-%20New%20York%20inmate%27s%20golf%20drawings%20lead%20to%20exoneration%20in%20murder&amp;url=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=550,height=250').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610" data-social-url="https://twitter.com/intent/tweet?text=BBC+News+-+New+York+inmate%27s+golf+drawings+lead+to+exoneration+in+murder&amp;amp;url=" data-target-url="https://www.bbc.co.uk/news/world-us-canada-45582610" tabindex="-1"> <span class="extracted__icon extracted__icon--twitter" data-platform="twitter"> <svg class="extracted-svg ex-twitter" viewbox="-13 -15 44 44" enable-background="new 0 0 44 44" width="44px" height="44px" aria-hidden="true" focusable="false"><g><path d="M5.80573373,15 C12.7721527,15 16.581877,9.22887915 16.581877,4.22385671 C16.581877,4.06002242 16.581877,3.89618812 16.5714931,3.73466135 C17.3122088,3.19816171 17.9525471,2.53359441 18.4602026,1.77326482 C17.7690988,2.08016568 17.0364595,2.28092039 16.28536,2.36976011 C17.0756874,1.89671742 17.6675677,1.15138674 17.9502395,0.274527115 C17.2072164,0.715264453 16.3938137,1.02678037 15.5457981,1.19407596 C14.1105174,-0.331198284 11.7118448,-0.405039095 10.1865706,1.0290879 C9.20241101,1.95440555 8.78590269,3.33315194 9.09049603,4.64844138 C6.04571636,4.4961447 3.20861397,3.05740266 1.28529161,0.691035437 C0.280364327,2.42167943 0.793788713,4.63574999 2.45751448,5.74682343 C1.85525036,5.72951699 1.26567764,5.56683646 0.738408105,5.27262698 L0.738408105,5.32108501 C0.739561868,7.12441605 2.00985456,8.67622684 3.77741896,9.03389326 C3.2201516,9.18618993 2.63519393,9.20811142 2.06754269,9.09850397 C2.56366064,10.6410847 3.98509624,11.6979313 5.60613279,11.7290828 C4.26430681,12.7824682 2.60750362,13.3547344 0.902242404,13.3535807 C0.601110348,13.3524269 0.299978293,13.3339667 7.10542736e-15,13.2982001 C1.73295152,14.4104273 3.74742113,15 5.80573373,14.9965387"/></g></svg></span> <span class="off-screen">Share this with Twitter</span> </a> </li> <li class="twite__channel-out twite__channel-out--desktop twite__channel-out--email twite__channel-click-extracted--email twite__list-elements" aria-hidden="true"> <a class="extracted__channel-link extracted__channel-link--email" href="mailto:?subject=Shared%20from%20BBC%20News&amp;body=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610" tabindex="-1"> <span class="extracted__icon extracted__icon--email" data-platform="email"> <svg class="extracted-svg ex-email" viewbox="-7 1 27 9" enable-background="new 0 0 44 44" width="44px" height="44px" aria-hidden="true" focusable="false"><g><path d="M11,4.9V9H2V2.1l4.6,5L13.1,0H0.2H0v11h13V2.8L11,4.9z M10.2,2L6.7,5.8L3.2,2H10.2z"/></g></svg></span> <span class="off-screen">Share this with Email</span> </a> </li> <li class="twite__channel-out twite__channel-out--mobile twite__channel-out--facebook twite__channel-click-extracted--facebook twite__list-elements" aria-hidden="true"> <a class="extracted__channel-link extracted__channel-link--facebook" onclick="window.open('http://www.facebook.com/dialog/feed?app_id=58567469885&amp;redirect_uri=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fspecial%2Fshared%2Fvj_sharetools%2Ffb_red_uri.html&amp;link=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610%3FSThisFB&amp;display=popup', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=555,height=615').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610" tabindex="-1"> <span class="extracted__icon extracted__icon--facebook" data-platform="facebook"> <svg class="extracted-svg ex-facebook" viewbox="-17 -13 44 44" enable-background="new 0 0 44 44" width="44px" height="44px" aria-hidden="true" focusable="false"><g><path d="M5.73,17 L5.73,9.246 L8.333,9.246 L8.723,6.223 L5.73,6.223 L5.73,4.294 C5.73,3.419 5.973,2.823 7.228,2.823 L8.828,2.822 L8.828,0.119 C8.551,0.082 7.601,0 6.496,0 C4.189,0 2.609,1.408 2.609,3.995 L2.609,6.223 L0,6.223 L0,9.246 L2.609,9.246 L2.609,17 L5.73,17 Z"/></g></svg></span> <span class="off-screen">Share this with Facebook</span> </a> </li> <li class="twite__channel-out twite__channel-out--mobile twite__channel-out--whatsapp twite__channel-click-extracted--whatsapp twite__list-elements" aria-hidden="true"> <a class="extracted__channel-link extracted__channel-link--whatsapp" onclick="window.open('whatsapp://send?text=BBC%20News%20%7C%20New%20York%20inmate%27s%20golf%20drawings%20lead%20to%20exoneration%20in%20murder%20-%20https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610%3Focid%3Dwsnews.chat-apps.in-app-msg.whatsapp.trial.link1_.auin', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=600,height=600').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610" tabindex="-1"> <span class="extracted__icon extracted__icon--whatsapp" data-platform="whatsapp"> <svg class="extracted-svg ex-whatsapp" viewbox="-11 -11 44 44" enable-background="new 0 0 44 44" width="44px" height="44px" aria-hidden="true" focusable="false"><g><path d="M0.6 19.4L1.9 14.7C1 13.3 0.6 11.7 0.6 10 0.6 4.8 4.8 0.5 10 0.5 15.2 0.5 19.4 4.8 19.4 10 19.4 15.2 15.2 19.4 10 19.4 8.4 19.4 6.8 19 5.4 18.2L0.6 19.4ZM5.6 16.5L5.9 16.7C7.2 17.4 8.6 17.8 10 17.8 14.3 17.8 17.8 14.3 17.8 10 17.8 5.7 14.3 2.1 10 2.1 5.7 2.1 2.2 5.7 2.2 10 2.2 11.5 2.6 12.9 3.4 14.2L3.6 14.5 2.8 17.2 5.6 16.5Z M7.4 5.6L6.7 5.5C6.6 5.5 6.4 5.6 6.2 5.7 5.9 6 5.5 6.5 5.3 7.1 5.1 8.1 5.4 9.3 6.3 10.5 7.2 11.6 8.8 13.5 11.6 14.4 12.6 14.6 13.3 14.4 13.8 14.1 14.3 13.8 14.6 13.3 14.7 12.8L14.8 12.4C14.8 12.2 14.8 12.1 14.6 12L12.6 11.1C12.4 11 12.3 11 12.2 11.2L11.4 12.2C11.3 12.3 11.2 12.3 11.1 12.3 10.5 12.1 8.7 11.3 7.7 9.3 7.6 9.3 7.6 9.2 7.7 9.1L8.5 8.2C8.5 8.1 8.6 8 8.5 7.9L7.6 5.8C7.6 5.7 7.5 5.6 7.4 5.6Z"/></g></svg></span> <span class="off-screen">Share this with WhatsApp</span> </a> </li> <li class="twite__channel-out twite__channel-out--mobile twite__channel-out--messengermobile twite__channel-click-extracted--messengermobile twite__list-elements" aria-hidden="true"> <a class="extracted__channel-link extracted__channel-link--messengermobile" href="fb-messenger://share?app_id=58567469885&amp;redirect_uri=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610&amp;link=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610%3FCMP%3Dshare_btn_me" target="_blank" rel="noopener" tabindex="-1"> <span class="extracted__icon extracted__icon--messengermobile" data-platform="messengermobile"> <svg class="extracted-svg ex-messengermobile" viewbox="-14 -13 44 44" enable-background="new 0 0 44 44" width="44px" height="44px" aria-hidden="true" focusable="false"><g><path d="M9.84804801,11.1721834 L7.54389655,8.77893955 L3.1059828,11.2323207 L7.97153994,6.06941781 L10.2756914,8.46377529 L14.7136051,6.00928046 L9.84804801,11.1721834 Z M8.90923715,0 C3.98911093,0 0,3.73074306 0,8.33125039 C0,10.9494525 1.29183939,13.2847862 3.3097816,14.8116068 L3.3097816,18 L6.35117243,16.3139269 C7.16079936,16.5399988 8.02054074,16.6625008 8.90923715,16.6625008 C13.830477,16.6625008 17.8184743,12.9328714 17.8184743,8.33125039 C17.8184743,3.73074306 13.830477,0 8.90923715,0 L8.90923715,0 Z"/></g></svg></span> <span class="off-screen">Share this with Messenger</span> </a> </li> <li class="twite__channel-out twite__channel-out--mobile twite__channel-out--twitter twite__channel-click-extracted--twitter twite__list-elements" aria-hidden="true"> <a class="extracted__channel-link extracted__channel-link--twitter" onclick="window.open('https://twitter.com/intent/tweet?text=BBC%20News%20-%20New%20York%20inmate%27s%20golf%20drawings%20lead%20to%20exoneration%20in%20murder&amp;url=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=550,height=250').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610" data-social-url="https://twitter.com/intent/tweet?text=BBC+News+-+New+York+inmate%27s+golf+drawings+lead+to+exoneration+in+murder&amp;amp;url=" data-target-url="https://www.bbc.co.uk/news/world-us-canada-45582610" tabindex="-1"> <span class="extracted__icon extracted__icon--twitter" data-platform="twitter"> <svg class="extracted-svg ex-twitter" viewbox="-13 -15 44 44" enable-background="new 0 0 44 44" width="44px" height="44px" aria-hidden="true" focusable="false"><g><path d="M5.80573373,15 C12.7721527,15 16.581877,9.22887915 16.581877,4.22385671 C16.581877,4.06002242 16.581877,3.89618812 16.5714931,3.73466135 C17.3122088,3.19816171 17.9525471,2.53359441 18.4602026,1.77326482 C17.7690988,2.08016568 17.0364595,2.28092039 16.28536,2.36976011 C17.0756874,1.89671742 17.6675677,1.15138674 17.9502395,0.274527115 C17.2072164,0.715264453 16.3938137,1.02678037 15.5457981,1.19407596 C14.1105174,-0.331198284 11.7118448,-0.405039095 10.1865706,1.0290879 C9.20241101,1.95440555 8.78590269,3.33315194 9.09049603,4.64844138 C6.04571636,4.4961447 3.20861397,3.05740266 1.28529161,0.691035437 C0.280364327,2.42167943 0.793788713,4.63574999 2.45751448,5.74682343 C1.85525036,5.72951699 1.26567764,5.56683646 0.738408105,5.27262698 L0.738408105,5.32108501 C0.739561868,7.12441605 2.00985456,8.67622684 3.77741896,9.03389326 C3.2201516,9.18618993 2.63519393,9.20811142 2.06754269,9.09850397 C2.56366064,10.6410847 3.98509624,11.6979313 5.60613279,11.7290828 C4.26430681,12.7824682 2.60750362,13.3547344 0.902242404,13.3535807 C0.601110348,13.3524269 0.299978293,13.3339667 7.10542736e-15,13.2982001 C1.73295152,14.4104273 3.74742113,15 5.80573373,14.9965387"/></g></svg></span> <span class="off-screen">Share this with Twitter</span> </a> </li> <li class="twite twite__list-elements"> <a href="https://www.bbc.co.uk/news/world-us-canada-45582610#share-tools" class="twite__share-button" aria-label="Open share panel" data-origin="page" aria-expanded="false" aria-haspopup="true"> <svg class="twite__share-icon" width="17px" aria-hidden="true" focusable="false" viewbox="0 0 29.266 32"><path d="M5.473 22.153c1.586 0 3.01-.684 4.012-1.762l9 4.845c-.102.412-.16.85-.16 1.297 0 3.02 2.452 5.468 5.472 5.468 3.017 0 5.47-2.446 5.47-5.468 0-3.023-2.453-5.47-5.47-5.47-1.587 0-3.02.68-4.015 1.757l-9.457-5.175-.074-2.792 9.74-5.456c.99.953 2.327 1.543 3.807 1.543 3.017 0 5.47-2.45 5.47-5.474 0-3.022-2.453-5.467-5.47-5.467-3.02 0-5.473 2.444-5.473 5.466 0 .554.08 1.09.243 1.597L9.27 12.75c-.988-.95-2.326-1.537-3.797-1.537C2.447 11.213 0 13.657 0 16.68c0 3.03 2.447 5.473 5.473 5.473"/></svg><span class="twite__share-text">Share</span> </a> <div class="twite__panel arrow-top" data-share-uri=""> <p class="twite__title" aria-hidden="true">Share this with</p> <span class="off-screen">These are external links and will open in a new window</span> <ul class="twite__channels"><li class="twite__channel twite__channel--email twite__channel-click--email"> <a class="twite__channel-link" href="mailto:?subject=Shared%20from%20BBC%20News&amp;body=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610"> <span class="twite__icon twite__icon--email" data-platform="email"/> <p class="twite__channel-text" aria-hidden="true">Email</p> <span class="off-screen">Share this with Email</span> </a> </li> <li class="twite__channel twite__channel--facebook twite__channel-click--facebook"> <a class="twite__channel-link" onclick="window.open('http://www.facebook.com/dialog/feed?app_id=58567469885&amp;redirect_uri=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fspecial%2Fshared%2Fvj_sharetools%2Ffb_red_uri.html&amp;link=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610%3FSThisFB&amp;display=popup', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=555,height=615').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610"> <span class="twite__icon twite__icon--facebook" data-platform="facebook"/> <p class="twite__channel-text" aria-hidden="true">Facebook</p> <span class="off-screen">Share this with Facebook</span> </a> </li> <li class="twite__channel twite__channel--messengerdesktop twite__channel-click--messengerdesktop"> <a class="twite__channel-link" onclick="window.open('http://www.facebook.com/dialog/send?app_id=58567469885&amp;redirect_uri=http%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fspecial%2Fshared%2Fvj_sharetools%2Ffb_red_uri.html&amp;link=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610%3FSThisFB&amp;display=popup', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=645,height=580').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610"> <span class="twite__icon twite__icon--messengerdesktop" data-platform="messengerdesktop"/> <p class="twite__channel-text" aria-hidden="true">Messenger</p> <span class="off-screen">Share this with Messenger</span> </a> </li> <li class="twite__channel twite__channel--messengermobile twite__channel-click--messengermobile"> <a class="twite__channel-link" href="fb-messenger://share?app_id=58567469885&amp;redirect_uri=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610&amp;link=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610%3FCMP%3Dshare_btn_me" target="_blank" rel="noopener"> <span class="twite__icon twite__icon--messengermobile" data-platform="messengermobile"/> <p class="twite__channel-text" aria-hidden="true">Messenger</p> <span class="off-screen">Share this with Messenger</span> </a> </li> <li class="twite__channel twite__channel--twitter twite__channel-click--twitter"> <a class="twite__channel-link" onclick="window.open('https://twitter.com/intent/tweet?text=BBC%20News%20-%20New%20York%20inmate%27s%20golf%20drawings%20lead%20to%20exoneration%20in%20murder&amp;url=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=550,height=250').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610" data-social-url="https://twitter.com/intent/tweet?text=BBC+News+-+New+York+inmate%27s+golf+drawings+lead+to+exoneration+in+murder&amp;amp;url=" data-target-url="https://www.bbc.co.uk/news/world-us-canada-45582610"> <span class="twite__icon twite__icon--twitter" data-platform="twitter"/> <p class="twite__channel-text" aria-hidden="true">Twitter</p> <span class="off-screen">Share this with Twitter</span> </a> </li> <li class="twite__channel twite__channel--pinterest twite__channel-click--pinterest"> <a class="twite__channel-link" onclick="window.open('https://uk.pinterest.com/pin/create/bookmarklet/?url=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610&amp;description=Valentino%20Dixon%2C%20whose%20art%20featured%20in%20a%20golf%20magazine%2C%20is%20a%20free%20man%20after%2027%20years%20behind%20bars.&amp;title=New%20York%20inmate%27s%20golf%20drawings%20lead%20to%20exoneration%20in%20murder&amp;media=https%3A%2F%2Fc.files.bbci.co.uk%2FE38A%2Fproduction%2F_103505285_capturhwe565473e.jpg', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=750,height=675').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610"> <span class="twite__icon twite__icon--pinterest" data-platform="pinterest"/> <p class="twite__channel-text" aria-hidden="true">Pinterest</p> <span class="off-screen">Share this with Pinterest</span> </a> </li> <li class="twite__channel twite__channel--whatsapp twite__channel-click--whatsapp"> <a class="twite__channel-link" onclick="window.open('whatsapp://send?text=BBC%20News%20%7C%20New%20York%20inmate%27s%20golf%20drawings%20lead%20to%20exoneration%20in%20murder%20-%20https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610%3Focid%3Dwsnews.chat-apps.in-app-msg.whatsapp.trial.link1_.auin', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=600,height=600').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610"> <span class="twite__icon twite__icon--whatsapp" data-platform="whatsapp"/> <p class="twite__channel-text" aria-hidden="true">WhatsApp</p> <span class="off-screen">Share this with WhatsApp</span> </a> </li> <li class="twite__channel twite__channel--linkedin twite__channel-click--linkedin"> <a class="twite__channel-link" onclick="window.open('https://www.linkedin.com/shareArticle?mini=true&amp;url=https%3A%2F%2Fwww.bbc.co.uk%2Fnews%2Fworld-us-canada-45582610&amp;title=New%20York%20inmate%27s%20golf%20drawings%20lead%20to%20exoneration%20in%20murder&amp;summary=Valentino%20Dixon%2C%20whose%20art%20featured%20in%20a%20golf%20magazine%2C%20is%20a%20free%20man%20after%2027%20years%20behind%20bars.&amp;source=BBC', '_blank', 'toolbar=no,scrollbars=yes,resizable=no,fullscreen=no,top=50,left=50,width=550,height=500').opener = null" href="https://www.bbc.co.uk/news/world-us-canada-45582610"> <span class="twite__icon twite__icon--linkedin" data-platform="linkedin"/> <p class="twite__channel-text" aria-hidden="true">LinkedIn</p> <span class="off-screen">Share this with LinkedIn</span> </a> </li> </ul><p class="twite__copy-text">Copy this link</p> <div class="twite__copy-input"> <a class="twite__share-link" href="https://www.bbc.co.uk/news/world-us-canada-45582610" tabindex="-1" contenteditable="true">https://www.bbc.co.uk/news/world-us-canada-45582610</a> </div> <a class="twite__read-more" href="https://www.bbc.co.uk/faqs/questions/bbc_online/sharing">Read more about sharing.</a> <p class="twite__new-window" aria-hidden="true">These are external links and will open in a new window</p> <button class="twite__close-button"> <span class="off-screen">Close share panel</span> <div class="twite__close-button-graphic" aria-hidden="true"/> </button> </div> </li> </ul></div> </div> </div> <div id="topic-tags"><div id="u33446952933445573"><div/></div></div> </div> </div> <div class="story-body__inner" property="articleBody"> <figure class="media-landscape has-caption full-width lead"><span class="image-and-copyright-container"> <img class="js-image-replace" alt="Dixon and his artwork" src="https://ichef.bbci.co.uk/news/320/cpsprodpb/F564/production/_103502826_capturhwe565473e.jpg" width="976" height="549"/><span class="off-screen">Image copyright</span> <span class="story-image-copyright">Georgetown University</span> </span> <figcaption class="media-caption"><span class="off-screen">Image caption</span> <span class="media-caption__text"> Valentino Dixon served most of his sentence in New York's notorious Attica Correctional Facility </span> </figcaption></figure><p class="story-body__introduction">A US convict whose talent for art caught the eye of a prison warden, ultimately leading to a review of his case, has been exonerated of murder.</p><p>Valentino Dixon's colour pencil drawings of golf courses were featured in Golf Digest - prompting advocates to uncover flawed evidence in the case.</p><p>Mr Dixon, 48, had maintained his innocence during 27 years behind bars for a shooting in Buffalo, New York.</p><p>He walked free after another man formally confessed to the murder.</p><h2 class="story-body__crosshead">The golf drawings</h2><p>Mr Dixon had served nearly two decades at a notorious prison in upstate New York when his artistic flair attracted the notice of correctional authorities.</p><div id="bbccom_mpu_1_2_3" class="bbccom_slot mpu-ad" aria-hidden="true"> <div class="bbccom_advert"> </div> </div><p>The warden gave him a photograph of the famed Augusta National's 12th hole in the US state of Georgia, and asked if he would draw it for him.</p><figure class="media-portrait no-caption full-width"><span class="image-and-copyright-container"> <div class="js-delayed-image-load" data-alt="Golf course drawing by Valentine Dixon" data-src="https://ichef.bbci.co.uk/news/320/cpsprodpb/16A94/production/_103502829_captasdf234236ure.jpg" data-width="549" data-height="589"/> <span class="off-screen">Image copyright</span> <span class="story-image-copyright">Golf Digest</span> </span> </figure><p>"After 19 years in Attica Correctional Facility, the look of a golf hole spoke to me," said Mr Dixon.</p><p>"It seemed peaceful. I imagine playing it would be a lot like fishing."</p><p>Using colouring pencils, he began to create meticulous, lush drawings of various links and fairways.</p><p>"I didn't know anything about golf. I'm from the 'hood," he told local media.</p><figure class="media-landscape no-caption full-width"><span class="image-and-copyright-container"> <div class="js-delayed-image-load" data-alt="Several paintings by Valentine Dixon" data-src="https://ichef.bbci.co.uk/news/320/cpsprodpb/35FC/production/_103502831_capdsf234ture.jpg" data-width="976" data-height="549"/> <span class="off-screen">Image copyright</span> <span class="story-image-copyright">Golf Digest</span> </span> </figure><p>His sketches impressed the editors at Golf Digest, which featured his artwork and <a href="https://www.golfdigest.com/story/golf-saved-my-life-valentino-dixon" class="story-body__link-external">a profile of the prisoner in 2012</a>.</p><p>"Maybe one day I'll get to play the game I've only imagined," Mr Dixon said in the article, describing how he drew landscapes he has never seen.</p><p>The magazine's coverage spurred wrongful-conviction advocates to look into the matter. Georgetown University law students championed his case.</p><h2 class="story-body__crosshead">The conviction</h2><p>Mr Dixon was handed a minimum 38-year-to-life sentence for killing 17-year-old Torriano Jackson one night in August 1991, after an argument over a girl.</p><p>He acknowledged being at the crime scene, but said he was at a nearby shop buying beer when the gunshots rang out.</p><p>Mr Dixon said multiple witnesses could have testified he did not fire the gun.</p><p>But his trial lawyer did not call any of them as several had been accused of perjury.</p><p>Unusually, the investigating homicide detective did not testify during the trial either.</p><p>But a more serious flaw in the case was uncovered by <a href="https://prisonsandjustice.georgetown.edu/" class="story-body__link-external">Georgetown University's Prisons and Justice Initiative</a>.</p><p>Prosecutors had omitted to reveal to Mr Dixon's defence attorney that a gunpowder test on his client's clothes had come back negative.</p><h2 class="story-body__crosshead">The real killer</h2><p>Perhaps even more egregiously, another man, Lamarr Scott, admitted to local media only days after the murder that he shot Torriano Jackson.</p><p><a href="https://www.wgrz.com/article/news/local/wrongfully-convicted-valentino-dixon-exonerated-after-27-years-in-prison/71-595938252" class="story-body__link-external">Scott told a WGRZ-TV reporter</a>: "I don't want my friend [Mr Dixon] to take the rap for something that I did."</p><figure class="media-landscape has-caption full-width"><span class="image-and-copyright-container"> <div class="js-delayed-image-load" data-alt="Georgetown University students greeted him upon his release" data-src="https://ichef.bbci.co.uk/news/320/cpsprodpb/FAE6/production/_103503246_capturedsfgwe4634.jpg" data-width="976" data-height="549"/> <span class="off-screen">Image copyright</span> <span class="story-image-copyright">Georgetown University</span> </span> <figcaption class="media-caption"><span class="off-screen">Image caption</span> <span class="media-caption__text"> Georgetown University students who worked on the case greeted Mr Dixon upon his release </span> </figcaption></figure><p>But he was never arrested. The victim's brother said he saw Mr Dixon open fire.</p><p><a href="https://buffalonews.com/2018/09/19/man-who-repeatedly-confessed-faces-charges-in-1991-buffalo-killing/" class="story-body__link-external">According to the Buffalo News</a>, prosecutors conceded that Scott had been admitting his guilt in the case for a long time.</p><p>"Mr Scott has been confessing to this crime since 12 August 1991," Assistant District Attorney Sara Dee told the court. </p><p>"He has confessed to this crime in excess of 10 times."</p><h2 class="story-body__crosshead">Exoneration</h2><p>Lamarr Scott - who is currently in jail for a separate attack - finally had the chance to formally confess to the crime on Wednesday.</p><p>Hours later, Mr Dixon was released. </p><p>"I grabbed the gun," Scott, now 46, told the courtroom in Erie, New York. </p><p>"I pulled the trigger and all the bullets came out. Unfortunately, Torriano ended up dying."</p><p>It was Erie County District Attorney John Flynn - in the job less than a year - who ordered a review of the case. </p><p>But despite Mr Dixon's exoneration, prosecutors say he did provide the murder weapon, which they described as a machine gun.</p><p>They also said he was an "up-and-coming drug dealer" in Buffalo at the time of his arrest.</p><p>"Mr Dixon is innocent of the shooting and of the murder for what he was found guilty of, but Mr Dixon brought the gun to the fight," said the district attorney.</p><h2 class="story-body__crosshead">What next for Valentino Dixon?</h2><p>"It's the greatest feeling in the world," he said as he walked a free man from court in Buffalo, New York, on Wednesday.</p><p>He was greeted outside by his daughter, who was a baby when he was jailed. </p><p>The 27-year-old brought along her own 14-month-old twins.</p><p>Mr Dixon says he hopes to keep up his illustrations, and even visit a golf course in real life someday.</p><p>"With his mind and body intact, Dixon hopefully has some good years ahead," <a href="https://www.golfdigest.com/story/for-valentino-dixon-a-wrong-righted-murder-charge-vacated-by-court-after-serving-27-years-in-prison" class="story-body__link-external">Golf Digest wrote on Wednesday</a>. </p><p>"Maybe he'll even take up golf."</p> </div> Thu, 20 Sep 2018 09:11:42 +0000 https://www.bbc.co.uk/news/world-us-canada-45582610 Quit Genius (YC W18) is looking for a remote front-end web developer – Contract https://news.ycombinator.com/item?id=18031384 https://news.ycombinator.com/item?id=18031384 Quit Genius is looking for a senior front-end web developer who can work remotely and part-time. We are a multi award-winning quit smoking app, build natively for iOS and Android.<p>Since launching in June 2017 we’ve experienced incredible user growth and are already the world leaders in smoking cessation. The app works – we’ve published 3 peer-reviewed papers and have already helped over 28,000 people quit smoking, for good. What we do saves lives and is incredibly rewarding</p><p>We’re looking for someone who can lay the foundation for our web products. You will need to exhibit strong attention to detail and be able to think several steps beyond the initial requirements. We also expect you to communicate effectively with designers and product managers using tools like Slack, Trello etc.</p><p>You will be able to work on a remote project basis.</p><p><i/>Key skills:<i/></p><p>- 4+ years of software development experience, with at least two years using AngularJS - Thorough understanding of browser technologies like HTML, CSS and JavaScript</p><p><i/>Bonus:<i/></p><p>- Design chops; experience with the Adobe suite (or other similar tools)</p><p><i/>Apply:<i/></p><p>Fill out the form here: <a href="https://goo.gl/forms/2VA1YT9GITGTbmER2" rel="nofollow">https://goo.gl/forms/2VA1YT9GITGTbmER2</a>. We will review all applicants and get back to you within 48 hours.</p> Thu, 20 Sep 2018 11:25:05 +0000 https://news.ycombinator.com/item?id=18031384 Ikarus electric “rocket” – Thrust-vectored flying ducted fan [video] https://www.youtube.com/watch?v=RMeEh5OUaDs https://www.youtube.com/watch?v=RMeEh5OUaDs <p><strong class="watch-time-text">Offentliggjort den 27. aug. 2018</strong></p><div id="watch-description-text" readability="13.259423503326"><p id="eow-description" class="">Ikarus is a 3D-printed, Electric Ducted Fan (EDF) "rocket" that takes off and lands vertically, sort of like the Falcon 9 first stage. Based on a 90mm EDF, this entire project can be built for about $250.</p><p>0-100M speed record<br/><a href="https://www.youtube.com/watch?v=yCQ7F_TI6FM" class="yt-uix-sessionlink spf-link " data-sessionlink="itct=CDUQ6TgYACITCIux0-u9yd0CFQKtmwodUlcE1yj4HQ">https://www.youtube.com/watch?v=yCQ7F...</a></p><p>Design info<br/><a href="https://www.youtube.com/redirect?redir_token=31SQkfatmP8fFAvRNkiZYjQOtyF8MTUzNzUyOTQ5M0AxNTM3NDQzMDkz&amp;q=http%3A%2F%2Fomeganaught.com%2F2018%2F08%2Fikarus-electric-rocket%2F&amp;v=RMeEh5OUaDs&amp;event=video_description" class="yt-uix-sessionlink " data-url="/redirect?redir_token=31SQkfatmP8fFAvRNkiZYjQOtyF8MTUzNzUyOTQ5M0AxNTM3NDQzMDkz&amp;q=http%3A%2F%2Fomeganaught.com%2F2018%2F08%2Fikarus-electric-rocket%2F&amp;v=RMeEh5OUaDs&amp;event=video_description" data-target-new-window="True" data-sessionlink="itct=CDUQ6TgYACITCIux0-u9yd0CFQKtmwodUlcE1yj4HUi70NGc-ZDh40Q" target="_blank" rel="nofollow noopener">http://omeganaught.com/2018/08/ikarus...</a></p><p>Pelican Case Contest<br/><a href="https://www.youtube.com/redirect?redir_token=31SQkfatmP8fFAvRNkiZYjQOtyF8MTUzNzUyOTQ5M0AxNTM3NDQzMDkz&amp;q=https%3A%2F%2Fwww.krontech.ca%2Fnews-blog%2Fpelican-case-contest&amp;v=RMeEh5OUaDs&amp;event=video_description" class="yt-uix-sessionlink " data-url="/redirect?redir_token=31SQkfatmP8fFAvRNkiZYjQOtyF8MTUzNzUyOTQ5M0AxNTM3NDQzMDkz&amp;q=https%3A%2F%2Fwww.krontech.ca%2Fnews-blog%2Fpelican-case-contest&amp;v=RMeEh5OUaDs&amp;event=video_description" data-target-new-window="True" data-sessionlink="itct=CDUQ6TgYACITCIux0-u9yd0CFQKtmwodUlcE1yj4HUi70NGc-ZDh40Q" target="_blank" rel="nofollow noopener">https://www.krontech.ca/news-blog/pel...</a></p><p>Support me on Patreon for more cool projects!<br/><a href="https://www.youtube.com/redirect?redir_token=31SQkfatmP8fFAvRNkiZYjQOtyF8MTUzNzUyOTQ5M0AxNTM3NDQzMDkz&amp;q=https%3A%2F%2Fwww.patreon.com%2Ftesla500&amp;v=RMeEh5OUaDs&amp;event=video_description" class="yt-uix-sessionlink " data-url="/redirect?redir_token=31SQkfatmP8fFAvRNkiZYjQOtyF8MTUzNzUyOTQ5M0AxNTM3NDQzMDkz&amp;q=https%3A%2F%2Fwww.patreon.com%2Ftesla500&amp;v=RMeEh5OUaDs&amp;event=video_description" data-target-new-window="True" data-sessionlink="itct=CDUQ6TgYACITCIux0-u9yd0CFQKtmwodUlcE1yj4HUi70NGc-ZDh40Q" target="_blank" rel="nofollow noopener">https://www.patreon.com/tesla500</a></p></div> Thu, 20 Sep 2018 03:29:54 +0000 https://www.youtube.com/watch?v=RMeEh5OUaDs Crippling DDoS vulnerability put the entire Bitcoin market at risk https://thenextweb.com/hardfork/2018/09/20/bitcoin-core-vulnerability-blockchain-ddos/ https://thenextweb.com/hardfork/2018/09/20/bitcoin-core-vulnerability-blockchain-ddos/ <p>The entire <a href="https://thenextweb.com/topic/bitcoin/" target="_blank" title="Read more about Bitcoin.">Bitcoin</a> infrastructure has been issued with a stern warning: update Bitcoin <a title="Trade on eToro" target="_blank" rel="nofollow noopener" href="https://partners.etoro.com/B9157_A72407_TClick.aspx" class="c-etoroTrade" data-short="BTC">BTC</a> Core software or risk having the whole thing collapse.</p> <p>Until now, Bitcoin miners could have brought down the entire blockchain by flooding full node operators with traffic, via a Distributed Denial-of-Service (DDoS) attack.</p> <p>“A denial-of-service vulnerability (CVE-2018-17144) exploitable by miners has been discovered in Bitcoin Core versions 0.14.0 up to 0.16.2.” the <a href="https://github.com/bitcoin/bitcoin/commit/4b8a3f5d235f40be8102506ab26caad005cc40d6" target="_blank" rel="nofollow noopener">patch notes</a> state. “It is recommended to upgrade any of the vulnerable versions to 0.16.3 as soon as possible.”</p> <p>Developers have issued a patch for anyone running nodes, along with an appeal to update the software immediately.</p> <p>As far as the attack vector in question goes, there’s a catch: anyone ballsy enough to try to bring down Bitcoin would have to sacrifice almost $80,000 worth of Bitcoin in order do it.</p> <p>The bug relates to its consensus code. It meant that some miners had the option to send transaction data twice, causing the Bitcoin network to crash when attempting to validate them.</p> <p>As such invalid blocks need to be mined anyway, only those willing to disregard block reward of 12.5BTC ($80,000) could actually do any real damage.</p> <p>While this certainly seems unlikely (barring any digital Tyler Durden-types wanting to destroy something beautiful), it does raise eyebrows. The great defence of Bitcoin is that it’s far too decentralized to be brought down by any single entity.</p> <p>Prolific speaker and cryptocurrency advocate Andreas Antonopolous weighed in on the vulnerability in a tweetstorm. He defended the quality of Bitcoin’s development, considering its open source nature.</p> <p>He was particularly enamoured by the community’s rigorous dedication to checking code quality. While the situation was surely dangerous, it could have been way worse – especially if new, buggy cryptocurrencies had decided to <a href="https://thenextweb.com/contributors/2018/04/28/comprehensive-guide-to-hard-forks-a-look-into-the-different-types-of-bitcoin-forks/" target="_blank" rel="noopener">fork</a> the Bitcoin Core version susceptible to DDoS.</p> <p>It’s worth pointing out that Bitcoin is hardly the only cryptocurrency <a href="https://thenextweb.com/hardfork/2018/09/19/cryptocurrency-bounty-hackers/" target="_blank" rel="noopener">researchers have found kinks in</a> recently. Indeed, a Bitcoin Core developer recently discovered a crippling flaw in Bitcoin Cash – a forked version of Bitcoin.</p> <p>While never convenient, responding appropriately to such potential dangers is crucial to maintaining the integrity of blockchain tech – especially when reversing transactions is not an option.</p> <p>But in the meantime, go ahead and mark this as yet another day of discovering just how close we were to a Bitcoin collapse: crisis averted.</p> <p><em>If you’re interested in everything blockchain, chances are you’ll love Hard Fork Decentralized. Our blockchain and cryptocurrency event is coming up soon – join us to hear from experts about the industry’s future. <a href="https://thenextweb.com/hardfork-decentralized/" target="_blank" rel="noopener">Check it out</a>!</em></p> <p class="c-post-pubDate"> Published September 20, 2018 — 09:22 UTC </p> Thu, 20 Sep 2018 10:03:41 +0000 https://thenextweb.com/hardfork/2018/09/20/bitcoin-core-vulnerability-blockchain-ddos/ Adding Mercurial support to Gitlab https://gitlab.com/gitlab-org/gitlab-ce/issues/31600#note_96793407 https://gitlab.com/gitlab-org/gitlab-ce/issues/31600#note_96793407 <h3 dir="auto"> <a id="user-content-description" class="anchor" href="https://gitlab.com/gitlab-org/gitlab-ce/issues/31600#description" aria-hidden="true"/>Description</h3> <p dir="auto">Can't host projects using mercurial. I find mercurial orders of magnitudes more friendly than git, for example I can remember the commands!</p> <h3 dir="auto"> <a id="user-content-proposal" class="anchor" href="https://gitlab.com/gitlab-org/gitlab-ce/issues/31600#proposal" aria-hidden="true"/>Proposal</h3> <p dir="auto">Add mercurial as an option for the CVS</p> <h3 dir="auto"> <a id="user-content-links-references" class="anchor" href="https://gitlab.com/gitlab-org/gitlab-ce/issues/31600#links-references" aria-hidden="true"/>Links / references</h3> <p dir="auto"><a href="https://confluence.atlassian.com/bitbucket/create-a-git-or-mercurial-repository-760120240.html" rel="nofollow noreferrer noopener" target="_blank">https://confluence.atlassian.com/bitbucket/create-a-git-or-mercurial-repository-760120240.html</a></p> Thu, 20 Sep 2018 03:15:57 +0000 https://gitlab.com/gitlab-org/gitlab-ce/issues/31600 China’s all-seeing surveillance network http://mobile.abc.net.au/news/2018-09-18/china-social-credit-a-model-citizen-in-a-digital-dictatorship/10200278 http://mobile.abc.net.au/news/2018-09-18/china-social-credit-a-model-citizen-in-a-digital-dictatorship/10200278 <a name="headersupplantdark"> </a><a name="video10251196"> </a><p>China is building a digital dictatorship to exert control over its 1.4 billion citizens. For some, "social credit" will bring privileges — for others, punishment.</p><a name="endheader"> </a><a name="blockpiecemealleft"> </a><a name="video10249796"> </a><p>Dandan Fan is very much the modern Chinese woman.</p><p>A marketing professional, she's diligent and prosperous — in many ways she's a model Chinese citizen.</p><p>But Dandan is being watched 24 hours a day. </p><a name="endblock"> </a><a name="remove"> </a><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-18/dandan-fan-walking.jpg/10255964"><h3> <strong>Photo</strong> Dandan Fan welcomes China's coming 'social credit' system.</h3> <img src="http://www.abc.net.au/news/image/10255964-3x2-220x147.jpg" alt="Dandan Fan welcomes China's coming 'social credit' system." title="Dandan Fan walks in Soho" width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><a name="endremove"> </a><a name="blockpiecemealleft"> </a><a name="video10251394"> </a><p>A vast network of 200 million CCTV cameras across China ensures there's no dark corner in which to hide. </p><p>Every step she takes, every one of her actions big or small — even what she thinks — can be tracked and judged.</p><p>And Dandan says that's fine with her.</p><a name="endblock"> </a><a name="hr"> </a><p>What may sound like a dystopian vision of the future is already happening in China. And it's making and breaking lives.</p><p>The Communist Party calls it "social credit" and says it will be fully operational by 2020.</p><p>Within years, an official Party outline claims, it will "allow the trustworthy to roam freely under heaven while making it hard for the discredited to take a single step".</p><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-18/traffic-cams-overhead-in-beijing.jpg/10251260"><h3> <strong>Photo</strong> Cameras monitor a busy street in Beijing, part of a network of 200 million CCTV cameras across China.</h3> <img src="http://www.abc.net.au/news/image/10251260-3x2-220x147.jpg" alt="Overhead cameras film cars on a busy Beijing street." title="Cameras monitor Beijing traffic" width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><p>Social credit is like a personal scorecard for each of China's 1.4 billion citizens.</p><p>In one pilot program already in place, each citizen has been assigned a score out of 800. In other programs it's 900.</p><p>Those, like Dandan, with top "citizen scores" get VIP treatment at hotels and airports, cheap loans and a fast track to the best universities and jobs.</p><a name="configright"> </a><blockquote class="quote--pullquote" readability="6"><p>"It will allow the trustworthy to roam freely under heaven while making it hard for the discredited to take a single step."</p></blockquote><p>Those at the bottom can be locked out of society and banned from travel, or barred from getting credit or government jobs.</p><p>The system will be enforced by the latest in high-tech surveillance systems as China pushes to become the world leader in artificial intelligence.</p><p>Surveillance cameras will be equipped with facial recognition, body scanning and geo-tracking to cast a constant gaze over every citizen.</p><a name="configambient"> </a><a name="video10251378"> </a><p>Smartphone apps will also be used to collect data and monitor online behaviour on a day-to-day basis. </p><p>Then, big data from more traditional sources like government records, including educational and medical, state security assessments and financial records, will be fed into individual scores.</p><p>Trial social credit systems are now in various stages of development in at least a dozen cities across China.</p><p>Several companies are working with the state to nationalise the system, co-ordinate and configure the technology, and finalise the algorithms that will determine the national citizen score.</p><p>It's probably the largest social engineering project ever attempted, a way to control and coerce more than a billion people. </p><p>If successful, it will be the world's first digital dictatorship.</p><a name="remove"> </a><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-17/dandan-fan-pushes-a-trolley-at-the-supermarket/10252982"><h3> <strong>Photo</strong> </h3> <img src="http://www.abc.net.au/news/image/10252982-3x2-220x147.jpg" alt="Dandan Fan pushes a trolley at the supermarket." title="Dandan Fan pushes a trolley at the supermarket." width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><a name="endremove"> </a><a name="blockpiecemeal"> </a><a name="video10253084"> </a><p>At the supermarket, Dandan is browsing the aisles. Even this everyday task will not escape the Party's penetrating gaze.</p><p>When social credit is fully implemented, what she puts into the trolley could impact her social score.</p><a name="endblock"> </a><a name="blockpiecemeal"> </a><a name="video10251288"> </a><p>Buying too much alcohol might suggest dependence; she'll lose a couple of points.</p><p>But buying a pack of nappies might suggest responsibility; she'll gain a few points.</p><p>The system will be "live" so her score will update in real time.</p><a name="endblock"> </a><p>Dandan doesn't object to the prospect of life under the state's all-seeing surveillance network.</p><p>The 36-year-old knows social credit is not a perfect system but believes it's the best way to manage a complex country with the world's biggest population.</p><p>"I think people in every country want a stable and safe society," she says.</p><p>"If, as our government says, every corner of public space is installed with cameras, I'll feel safe."</p><p>She's also likely to benefit from the system.</p><p>Dandan's financial behaviour will be an important measure for the national social credit score.</p><p>Under an existing financial credit scheme called Sesame Credit, Dandan has a very high score of 770 out of 800 — she is very much the loyal Chinese citizen.</p><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-18/dandans-credit-score-on-app.jpg/10251546"><h3> <strong>Photo</strong> A financial app shows Dandan's score is 773 out of 800. The perks of such a high score are only a few taps away.</h3> <img src="http://www.abc.net.au/news/image/10251546-3x2-220x147.jpg" alt="A financial app shows Dandan's score is 773 out of 800. The perks of such a high score are only a few taps away." title="Dandan's credit score on app.jpg" width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><p>Thanks to her rating, Dandan is already able to partake in many of the rewards of China's rapid development.</p><p>An app on her phone gives access to special privileges like renting a car, hotel room or a house without a deposit.</p><a name="configright"> </a><blockquote class="quote--pullquote" readability="9"><p>"If, as our government says, every corner of public space is installed with cameras, I'll feel safe."</p></blockquote><p>But social credit will be affected by more than just internet browsing and shopping decisions.</p><p>Who your friends and family are will affect your score. If your best friend or your dad says something negative about the government, you'll lose points too.</p><p>Who you date and ultimately partner with will also affect social credit.</p><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-18/dandan-in-beijing-park.jpg/10256380"><h3> <strong>Photo</strong> Dandan says surveillance in China's public spaces makes her feel safe, even if the government uses it to keep an eye on citizens.</h3> <img src="http://www.abc.net.au/news/image/10256380-3x2-220x147.jpg" alt="Dandan Fan and her family walk through a Beijing park." title="Park picture.jpg" width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><p>Dandan married for love but she chose the right husband — Xiaojing Zhang is likely to have an even higher score than her.</p><p>He's a civil servant in the justice department, a loyal cadre to the party.</p><p>"We need a social credit system," says Xiaojing.</p><p>"In the Chinese nation, we hope we can help each other, love each other, and help everyone become prosperous.</p><p>"As President Xi said, we will be rich and democratic, cultural, harmonious and beautiful.</p><p>"It is Xi's hope for the country's future. It is also the hope of the whole Chinese nation."</p><a name="configsm3x2"> </a><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-18/dandan-xiaoping-and-ruibao.jpg/10247520"><h3> <strong>Photo</strong> As Dandan and Xiaojing are model parents, their son Ruibao, 2, will benefit from their social credit too.</h3> <img src="http://www.abc.net.au/news/image/10247520-3x2-220x147.jpg" alt="Danda and Xiaoping play with their son Ruibao at a Beijing park." title="Dandan Xiaoping and Ruibao" width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><p>China has long been a surveillance state, so the citizenry is accustomed to the government taking a determining role in personal affairs.</p><p>For many in China, privacy doesn't have the same premium as it does in the West.</p><p>The Chinese place a higher value on community good versus individual rights, so most feel that, if social credit will bring a safer, more secure, more stable society, then bring it on.</p><p>But most don't seem to comprehend the all-encompassing control social credit is likely to have, and there's been no public debate about implementing the system inside China.</p><p>In private, there's been some disquiet in the educated middle classes about the citizen score being the only criterion for character assessment.</p><p>But that's not going to stop the rollout.</p><p>The Party is using the system to win back some of the control it lost when China opened up to the world in the 1980s and rapid development followed.</p><p>It's a way to silence dissent and ensure the Party's absolute dominance.</p><p>Already, about 10 million people have been punished in the trial areas of social credit.</p><p>Liu Hu is just one of them. </p><a name="remove"> </a><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-18/hu-monitoring-still.jpg/10256062"><h3> <strong>Photo</strong> Liu Hu believes the Chinese government sees him as an enemy.</h3> <img src="http://www.abc.net.au/news/image/10256062-3x2-220x147.jpg" alt="Liu Hu steps into an elevator." title="Hu monitoring still.jpg" width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><a name="endremove"> </a><a name="blockpiecemeal"> </a><a name="video10253076"> </a><p>In many societies, he would be celebrated. Not in China.</p><p>Liu Hu is an investigative journalist who has uncovered corruption at the top levels of the Party and solved serial murder cases. </p><p>He says the government considers him an enemy.</p><a name="endblock"> </a><p>Hu lost his social credit when he was charged with a speech crime and now finds himself locked out of society due to his low score.</p><p>In 2015, Hu lost a defamation case after he accused an official of extortion.</p><p>He was made to publish an apology and pay a fine but when the court demanded an additional fee, he refused.</p><p>Last year, the 43-year-old found himself blacklisted as "dishonest" under a pilot social credit scheme.</p><p>"There are a lot of people who are on the blacklist wrongly, but they can't get off it," says Hu.</p><a name="mosaic3"> </a><a name="configanon"> </a><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-18/a-mock-surveillance-image4.jpg/10249962"><h3> <strong>Photo</strong> </h3> <img src="http://www.abc.net.au/news/image/10249962-3x2-220x147.jpg" alt="Liu Hu is effectively under house arrest.jpg" title="Liu Hu is effectively under house arrest.jpg" width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-18/liu-hu-surveillance.jpg/10249964"><h3> <strong>Photo</strong> </h3> <img src="http://www.abc.net.au/news/image/10249964-3x2-220x147.jpg" alt="Hu's low social credit means he can't even book high-speed rail tickets." title="Hu's low social credit means he can't even book high-speed rail tickets." width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-18/liu-hu-surveillance-2/10249966"><h3> <strong>Photo</strong> </h3> <img src="http://www.abc.net.au/news/image/10249966-3x2-220x147.jpg" alt="Hu says the Party considers him an enemy." title="Hu says the Party considers him an enemy." width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><a name="endmosaic"> </a><p>It's destroyed his career and isolated him, and he now fears for his family's future.</p><p>The social credit system has closed down his travel options and kept him under effective house arrest in his hometown of Chongqing. </p><a name="configright"> </a><blockquote class="quote--pullquote" readability="6"><p>"Their eyes are blinded and their ears are blocked. They know little about the world and live in an illusion."</p></blockquote><p>In an apartment above the streets of Chongqing city, Hu tries to use a phone app to book train tickets to Xi'an. The attempt is rejected.</p><p>"[The app] says it fails to make a booking and my access to high-speed rail is legally restricted," he explains.</p><p>Hu's social media accounts, where he published much of his investigative journalism, have also been shut down.</p><p>Hu claims his combined Wechat and Weibo accounts had two million followers at their peak but are now censored.</p><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-18/chinese-journalist-liu-hu/10247532"><h3> <strong>Photo</strong> Chinese journalist Liu Hu lost his social credit and is effectively confined to house arrest.</h3> <img src="http://www.abc.net.au/news/image/10247532-3x2-220x147.jpg" alt="Chinese journalist Liu Hu lost his social credit and is effectively confined to house arrest." title="Chinese journalist Liu Hu has lost his social credit" width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><p>Hu believes his blacklisting is political and has tried to appeal to authorities. So far he has been met with silence.</p><p>Hu wants to warn the world of the nightmare of social credit. </p><p>Doing so could put his friends and family at risk of reprisals from the state, but Hu believes most Chinese don't yet understand what's to come under the digital totalitarian state.</p><p>"You can see from the Chinese people's mental state," says Hu. </p><p>"Their eyes are blinded and their ears are blocked. They know little about the world and live in an illusion."</p><a name="remove"> </a><figure class="embed-content"><article class="type-photo"><a href="http://mobile.abc.net.au/news/2018-09-18/dandan-fan-and-her-family-feed-the-fish-at-a-beijing-park/10253118"><h3> <strong>Photo</strong> </h3> <img src="http://www.abc.net.au/news/image/10253118-3x2-220x147.jpg" alt="Dandan Fan and her family feed the fish at a Beijing park." title="Dandan Fan and her family feed the fish at a Beijing park." width="220" height="147"/><span class="attribution">Foreign Correspondent: Brant Cumming</span> </a></article></figure><a name="endremove"> </a><a name="blockpiecemealleft"> </a><a name="video10255330"> </a><p>Dandan sees blue skies in her digital future. And for her, there's another incentive to be optimistic about social credit. </p><p>It's a way to ensure a happy and healthy future for her two-year-old son, Ruibao.</p><p>Thanks to his parents' high citizen scores, Ruibao will get the best possible start in life — the best housing, schools and healthcare.</p><p>The provisions and protections of the Party will be bestowed upon him.</p><p>So long as mum and dad keep their credit up.</p><a name="endblock"> </a><p><strong><em>Watch Foreign Correspondent's 'Leave no dark corner' on <a href="https://www.youtube.com/watch?v=eViswN602_k" target="_self" title="">YouTube</a>.</em></strong></p><ul><li>Reporter: Matthew Carney</li><li>Video and photography: Brant Cumming</li><li>Digital producer: Matthew Henry</li><li>Producer: Alex Barry</li><li>Graphics: Andres Gomez Isaza</li></ul> Thu, 20 Sep 2018 08:28:59 +0000 http://mobile.abc.net.au/news/2018-09-18/china-social-credit-a-model-citizen-in-a-digital-dictatorship/10200278 Ask HN: Did any Show HN posts turn into successful startups? https://news.ycombinator.com/item?id=18030355 https://news.ycombinator.com/item?id=18030355 <tr class="athing comtr " id="18030381" readability="0.32352941176471"><td> </td></tr><tr class="athing comtr " id="18030541" readability="5.4929577464789"><td> </td></tr><tr class="athing comtr " id="18030672" readability="5.5086206896552"><td> </td></tr><tr class="athing comtr " id="18030803" readability="4.3661971830986"><td> </td></tr><tr class="athing comtr " id="18030897" readability="8.6493506493506"><td> <table border="0" readability="4.3246753246753"><tr readability="8.6493506493506"><td class="ind"><img src="https://news.ycombinator.com/s.gif" height="1" width="160"/></td><td valign="top" class="votelinks"><center><a id="up_18030897" href="https://news.ycombinator.com/vote?id=18030897&amp;how=up&amp;goto=item%3Fid%3D18030355"/></center></td><td class="default"><br/><div class="comment" readability="11"> <span class="commtext c00" readability="12">I’m leaving because of this. And I give them the ~$100/year for the pro plan. Screw ‘em.<p>Also it really shits me that every time I log in I get a little banner that says “Almost out of space? Try Dropbox Business!”.</p><p>One, no, I am not almost out of space. I’m at 15% and if you don’t know that, there’s something terribly wrong.</p><p>Two, I’m not a business. I’m just me. And I already pay for Pro. Get out of my goddamned face.</p><p>/rant</p></span> </div></td></tr></table></td></tr><tr class="athing comtr " id="18031351" readability="9.3967517401392"><td> </td></tr><tr class="athing comtr " id="18030454" readability="2.5233644859813"><td> </td></tr><tr class="athing comtr " id="18031255" readability="12.291487532244"><td> <table border="0" readability="6.1457437661221"><tr readability="12.291487532244"><td class="ind"><img src="https://news.ycombinator.com/s.gif" height="1" width="0"/></td><td valign="top" class="votelinks"><center><a id="up_18031255" href="https://news.ycombinator.com/vote?id=18031255&amp;how=up&amp;goto=item%3Fid%3D18030355"/></center></td><td class="default"><br/><div class="comment" readability="15.405866900175"> <span class="commtext c00" readability="22.484238178634">I saw this post more than eight years ago and commented on it: <a href="https://news.ycombinator.com/item?id=1261786" rel="nofollow">https://news.ycombinator.com/item?id=1261786</a><p>I then started helping Anton out and we managed to hit the front page a few more times: <a href="https://hn.algolia.com/?query=akshell&amp;sort=byDate&amp;prefix&amp;page=0&amp;dateRange=all&amp;type=story" rel="nofollow">https://hn.algolia.com/?query=akshell&amp;sort=byDate&amp;prefix&amp;pag...</a></p><p>To get feedback on the platform, we paid developers to use it to make apps for our clients, which brought in some revenue. However, we failed to get people to pay for the service itself, even though we had a couple of thousand developers signed up and a few hundred apps in production. So we put Akshell on hold, but stayed in touch.</p><p>Three years ago, we reconnected and started Toughbyte: <a href="https://toughbyte.com" rel="nofollow">https://toughbyte.com</a>. We initially focused on the thing that worked at Akshell, i.e. writing software for clients. Almost by accident, in the end we ended up doing tech recruitment instead. More on that here: <a href="https://blog.toughbyte.com/whats-next-for-toughbyte-aed3cf54c085" rel="nofollow">https://blog.toughbyte.com/whats-next-for-toughbyte-aed3cf54...</a></p><p>Now we're building a tech recruitment platform. We've been profitable since day one and have been growing 50% each year. So, in a somewhat roundabout way, that first Show HN post did result in a startup that's still around.</p></span> </div></td></tr></table></td></tr><tr class="athing comtr " id="18031125" readability="0.37333333333333"><td> </td></tr><tr class="athing comtr " id="18030700" readability="2.063829787234"><td> </td></tr><tr class="athing comtr " id="18030556" readability="1.7206703910615"><td> </td></tr><tr class="athing comtr " id="18030474" readability="5.3676092544987"><td> </td></tr><tr class="athing comtr " id="18030479" readability="4.1150442477876"><td> </td></tr><tr class="athing comtr " id="18030893" readability="2.504854368932"><td> </td></tr><tr class="athing comtr " id="18031003" readability="2.5826086956522"><td> </td></tr><tr class="athing comtr " id="18030437" readability="3.4181818181818"><td> </td></tr><tr class="athing comtr " id="18030749" readability="1.32"><td> </td></tr><tr class="athing comtr " id="18030483" readability="3.5395683453237"><td> </td></tr><tr class="athing comtr " id="18030537" readability="2.7090909090909"><td> </td></tr><tr class="athing comtr " id="18030973" readability="0.28169014084507"><td> </td></tr><tr class="athing comtr " id="18030549" readability="7.7340659340659"><td> <table border="0" readability="3.867032967033"><tr readability="7.7340659340659"><td class="ind"><img src="https://news.ycombinator.com/s.gif" height="1" width="0"/></td><td valign="top" class="votelinks"><center><a id="up_18030549" href="https://news.ycombinator.com/vote?id=18030549&amp;how=up&amp;goto=item%3Fid%3D18030355"/></center></td><td class="default"><br/><div class="comment" readability="8.0854503464203"> <span class="commtext c00" readability="7.1870669745958">We started BitGym nearly 6 years ago(<a href="https://news.ycombinator.com/item?id=4519256" rel="nofollow">https://news.ycombinator.com/item?id=4519256</a>) and had pretty low traction on here,<p>By most definitions we've pivoted from tech startup to "profitable small tech business", staying at 4-6 people. We're no dropbox, but at a solid cohort paying customers for our reasonably well-loved consumer product we've turned into a successful (by our personal definitions) company, if not a successful startup.</p></span> </div></td></tr></table></td></tr><tr class="athing comtr " id="18030409" readability="0.77551020408163"><td> </td></tr><tr class="athing comtr " id="18030380" readability="7.649667405765"><td> </td></tr><tr class="athing comtr " id="18030723" readability="2.5769230769231"><td> </td></tr><tr class="athing comtr " id="18030468" readability="4.7993311036789"><td> </td></tr><tr class="athing comtr " id="18031186" readability="1.4871794871795"><td> </td></tr><tr class="athing comtr " id="18030971" readability="4.8235294117647"><td> <table border="0" readability="2.4117647058824"><tr readability="4.8235294117647"><td class="ind"><img src="https://news.ycombinator.com/s.gif" height="1" width="0"/></td><td valign="top" class="votelinks"><center><a id="up_18030971" href="https://news.ycombinator.com/vote?id=18030971&amp;how=up&amp;goto=item%3Fid%3D18030355"/></center></td><td class="default"><br/><div class="comment" readability="5.9305555555556"> <span class="commtext c00" readability="3.3888888888889">Almost 4.5 years ago we launched Instabug (www.instabug.com) on HN (<a href="https://news.ycombinator.com/item?id=5526949" rel="nofollow">https://news.ycombinator.com/item?id=5526949</a>)<p>Successful is a broad term, but we have tens of thousands of apps on our platform and our SDK is running on over a billion devices. We did YC and raised multiple rounds after.</p></span> </div></td></tr></table></td></tr><tr class="athing comtr " id="18030574" readability="1.2475247524752"><td> </td></tr><tr class="athing comtr " id="18030647" readability="3.5495495495495"><td> </td></tr><tr class="athing comtr " id="18030671" readability="5.596837944664"><td> </td></tr><tr class="athing comtr " id="18030463" readability="2.6292134831461"><td> </td></tr><tr class="athing comtr " id="18030448" readability="2.4429530201342"><td> </td></tr><tr class="athing comtr " id="18031198" readability="0.2962962962963"><td> </td></tr><tr class="athing comtr " id="18030631" readability="4.7033898305085"><td> </td></tr><tr class="athing comtr " id="18031288" readability="2.5818181818182"><td> <table border="0" readability="1.2909090909091"><tr readability="2.5818181818182"><td class="ind"><img src="https://news.ycombinator.com/s.gif" height="1" width="40"/></td><td valign="top" class="votelinks"><center><a id="up_18031288" href="https://news.ycombinator.com/vote?id=18031288&amp;how=up&amp;goto=item%3Fid%3D18030355"/></center></td><td class="default"><br/><div class="comment" readability="6"> <span class="commtext c00" readability="2">I remember seeing the launch post few years back. I tried your demo and thought it was an excellent product.<p>Good to hear that. Best of luck.</p></span> </div></td></tr></table></td></tr><tr class="athing comtr " id="18030798" readability="1.4137931034483"><td> </td></tr><tr class="athing comtr " id="18031216" readability="4.3386243386243"><td> </td></tr><tr class="athing comtr " id="18030870" readability="7.4236311239193"><td> </td></tr><tr class="athing comtr " id="18030813" readability="1.2527472527473"><td> </td></tr><tr class="athing comtr " id="18030881" readability="1.8421052631579"><td> </td></tr><tr class="athing comtr " id="18030469" readability="1.2115384615385"><td> </td></tr><tr class="athing comtr " id="18030565" readability="6.5"><td> </td></tr><tr class="athing comtr " id="18030484" readability="0.35897435897436"><td> </td></tr><tr class="athing comtr " id="18030440" readability="3.8167938931298"><td> </td></tr><tr class="athing comtr " id="18030785" readability="4.0151515151515"><td> <table border="0" readability="2.0075757575758"><tr readability="4.0151515151515"><td class="ind"><img src="https://news.ycombinator.com/s.gif" height="1" width="40"/></td><td valign="top" class="votelinks"><center><a id="up_18030785" href="https://news.ycombinator.com/vote?id=18030785&amp;how=up&amp;goto=item%3Fid%3D18030355"/></center></td><td class="default"><br/><div class="comment" readability="6.8758169934641"> <span class="commtext c00" readability="5.156862745098">Thanks for the mention. The name of the company is Wingify.<p>Here's the Ask HN for Visual Website Optimizer that got 0 comments: <a href="https://news.ycombinator.com/item?id=991252" rel="nofollow">https://news.ycombinator.com/item?id=991252</a></p><p>This product has gone to help Wingify bootstrap to $20mn in annual recurring revenue. So there's hope even if an 'Ask HN' doesn't fly :)</p></span> </div></td></tr></table></td></tr><tr class="athing comtr " id="18030577" readability="3.0877192982456"><td> </td></tr><tr class="athing comtr " id="18030704" readability="1.9173553719008"><td> </td></tr><tr class="athing comtr " id="18030475" readability="2.2380952380952"><td> </td></tr><tr class="athing comtr " id="18030374" readability="1.2363636363636"><td> </td></tr><tr class="athing comtr " id="18030530" readability="2.7467105263158"><td> </td></tr> Thu, 20 Sep 2018 07:11:56 +0000 https://news.ycombinator.com/item?id=18030355 Vapid: an intentionally simple CMS https://www.vapid.com https://www.vapid.com <div readability="13">Static site builders like Jekyll and Middleman offer a modern development environment (e.g. live editing, SASS, Webpack, etc), but are difficult to share with folks who would like a dashboard to edit content.</div><p><strong><em>Vapid takes the best of static site builders and introduces the ability to have a dashboard with almost no extra effort</em></strong>.</p> Thu, 20 Sep 2018 03:51:05 +0000 https://www.vapid.com/ Former Columnist Jailed for Facebook Criticism of Myanmar’s Aung San Suu Kyi https://www.mmtimes.com/news/ngar-min-swe-given-7-years-facebook-post.html https://www.mmtimes.com/news/ngar-min-swe-given-7-years-facebook-post.html <p> Ngar Min Swe, also known as Sar Min Swe, was found guilty in Yangon Western District Court on Monday of inciting hatred against the government in his Facebook post of January 24, in violation of section 124A of the penal code.  </p> <p> The court ruled against Sar Min Swe after 11 hearings. He did not have a lawyer but represented himself in court.</p> <p> During the Tatmadaw (military) government, U Min Swe was a weekly columnist in state-owned newspapers who wrote articles critical of Daw Aung San Suu Kyi, who is now head of government. His pro-government articles at the time earned him the nickname “Phar (Toady) Min Swe”. </p> <p> He distributed his articles from vehicles in many cities across the country. He reported in 2017 that the public hugging and kissing of US President Barack Obama and Daw Aung San Suu Kyi during visits in 2012 and 2014 were shameful and an insult to the dignity of the Myanmar people.</p> <p> Last year, another Facebook user was sentenced to six months in prison with hard labour for defaming the State Counsellor.</p> Thu, 20 Sep 2018 07:13:58 +0000 https://www.mmtimes.com/news/ngar-min-swe-given-7-years-facebook-post.html Putting This Blog on IPFS https://ipfs.io/ipns/Qme48wyZ7LaF9gC5693DZyJBtehgaFhaKycESroemD5fNX/post/putting_this_blog_on_ipfs/ https://ipfs.io/ipns/Qme48wyZ7LaF9gC5693DZyJBtehgaFhaKycESroemD5fNX/post/putting_this_blog_on_ipfs/ <p><a href="https://ipfs.io/">IPFS</a>, the Interplanetary File System, is a global distributed immutable datastore, an effort to decentralize and distribute the load of hosting websites, which I <a href="https://ipfs.io/ipns/Qme48wyZ7LaF9gC5693DZyJBtehgaFhaKycESroemD5fNX/post/ipfs-the-interplanetary-file-system/">first wrote about</a> back in 2016. It’s a great technology, and of course that means that <a href="https://blog.cloudflare.com/distributed-web-gateway/">Cloudflare wants to run a monkey-in-the-middle attack on it</a>.</p> <h2 id="the-gateway">The Gateway</h2> <p>This website now runs an IPFS node, which you can use to access the network by prefixing an IPFS or IPNS (the DNS equivalent that allows “mutable” content on the immutable web) URL with <code>https://ipfs.leotindall.com</code>. For instance, the url <code>/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki/</code> links to an immutable snapshot of the English Wikipedia; to access it through my IPFS gateway, you could use <a href="https://ipfs.leotindall.com//ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki/">https://ipfs.leotindall.com/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki/</a>.</p> <p>Setting this up was immensely easy. I installed it using <code>snap</code>:</p> <pre><code>sudo snap install ipfs </code></pre> <p>I then added a simple <code>systemd</code> unit:</p> <pre><code>[Unit] Description=Interplanetary File System Daemon After=network.target Requires=snapd.service [Service] User=leo ExecStart=/snap/bin/ipfs daemon Restart=on-failure [Install] WantedBy=default.target </code></pre> <p>The final piece was to configure nginx to forward traffic to the daemon (which runs on port 8080). I got a LetsEncrypt cert and added the following file to my nginx config:</p> <div class="highlight" readability="10"><pre><code class="language-nginx" data-lang="nginx"><span>server</span> { <span>listen</span> <span>443</span> <span>ssl</span> <span>http2</span>; <span>listen</span> <span>[::]:443</span> <span>ssl</span> <span>http2</span>; <span>server_name</span> <span>ipfs.leotindall.com</span>; <span>access_log</span> <span>/var/log/nginx/ipfs.leotindall.com.access.log</span>; <span>error_log</span> <span>/var/log/nginx/ipfs.leotindall.com.error.log</span>; <span>location</span> <span>/</span> { <span>proxy_pass</span> <span>http://localhost:8080/</span>; <span>proxy_set_header</span> <span>Host</span> <span>$host</span>; <span>proxy_buffering</span> <span>off</span>; <span>proxy_pass_request_headers</span> <span>on</span>; } <span>ssl_certificate</span> <span>/etc/letsencrypt/live/ipfs.leotindall.com/fullchain.pem</span>; <span># managed by Certbot </span><span/> <span>ssl_certificate_key</span> <span>/etc/letsencrypt/live/ipfs.leotindall.com/privkey.pem</span>; <span># managed by Certbot </span><span/>}</code></pre></div> <p>This is great for a couple reasons, but the primary one is that it terminates the connection using HTTP2 and SSL, so my interactions with my IPFS gateway are fast and encrypted.</p> <h2 id="the-blog">The Blog</h2> <p>I wanted to make my blog available on IPFS, automatically and rapidly. I already have a script that syncs my blog from its Git repository:</p> <div class="highlight" readability="13"><pre><code class="language-bash" data-lang="bash"><span>#!/bin/bash </span><span/>set -e <span># Enter the correct directory </span><span/>cd ~/leotindall.com <span># Update the Git repo </span><span/><span>OLD_ID</span>=<span>$(</span>git rev-parse HEAD<span>)</span> git pull origin master <span>NEW_ID</span>=<span>$(</span>git rev-parse HEAD<span>)</span> <span># If there's no change, abort with success </span><span/><span>if</span> [ <span>$OLD_ID</span> = <span>$NEW_ID</span> ]; <span>then</span> echo <span>"No change, execution finished."</span> logger <span>"</span><span>$0</span><span> - leotindall.com VCS update found no changes."</span> exit <span>0</span> <span>fi</span> <span># If there was a change, rebuild the site </span><span/>mkdir -p public_new/ hugo --destination ./public_new/ <span># Copy the files to their new destination </span><span/>rm -rf public mv public_new public chmod <span>777</span> public logger <span>"</span><span>$0</span><span> - leotindall.com VCS update found changes and rebuilt site."</span></code></pre></div> <p>This is pretty simple: grab the latest version from Git, compare the hashes, and rebuild the website if the hashes don’t match, then swap in the new version only if that succeeded.</p> <p>After a bit of experimentation, I figured out how to automatically update the site on IPFS, as well:</p> <div class="highlight" readability="11"><pre><code class="language-bash" data-lang="bash"><span># Get the peer ID, which is what IPNS will name our files </span><span/><span>peerid</span>=<span>$(</span>/snap/bin/ipfs id -f<span>"&lt;id&gt;"</span><span>)</span> <span># Add the files to IPFS and grab the hash of the directory root </span><span/><span>dirhash</span>=<span>$(</span>/snap/bin/ipfs add -r public/ | grep public$ | cut -d<span>" "</span> -f2<span>)</span> <span># Publish the directory to our node's IPNS entry </span><span/>/snap/bin/ipfs name publish <span>$dirhash</span> <span># Log to syslog </span><span/>logger <span>"</span><span>$0</span><span> - leotindall.com (</span><span>$dirhash</span><span>) republished to IPFS and IPNS as (</span><span>$peerid</span><span>)"</span></code></pre></div> <p>Unfortunately, this wasn’t the end of my efforts. I’d been using root-relative link URLs within the blog, things like <code>/posts/whatever</code>, which break when used through an IPFS gateway that prefixes my blog URLs with the gateway’s domain and the IPFS URL. Hugo’s <code>relativeURLs</code> options fixes that.</p> <p>Finally, my custom font broke, becuase it’s in static CSS and there’s no way to change the URL per page without using JavaScript, which I don’t really want to do.</p> <p>So, I used a rather ugly hack to “fix” it for nested directories down to 3 levels:</p> <div class="highlight" readability="18"><pre><code class="language-css" data-lang="css">@<span>font-face</span> { <span>font-family</span>: <span>BitterRegular</span>; <span>src</span>: <span>url</span>(/<span>fonts</span>/<span>bitter</span>.woff); } @<span>font-face</span> { <span>font-family</span>: <span>BitterRegularFB1</span>; <span>src</span>: <span>url</span>(<span>fonts</span>/<span>bitter</span>.woff); } @<span>font-face</span> { <span>font-family</span>: <span>BitterRegularFB2</span>; <span>src</span>: <span>url</span>(../<span>fonts</span>/<span>bitter</span>.woff); } @<span>font-face</span> { <span>font-family</span>: <span>BitterRegularFB3</span>; <span>src</span>: <span>url</span>(../../<span>fonts</span>/<span>bitter</span>.woff); } <span>body</span> { <span>font-family</span>: BitterRegular, BitterRegularFB1, BitterRegularFB2, BitterRegularFB3, <span>serif</span>, <span>'Apple Color Emoji'</span>, <span>'Segoe UI Emoji'</span>, <span>'Segoe UI Symbol'</span>, <span>'Symbola'</span>; }</code></pre></div> <p>I also realized that, since these immutable pages will be available forever (in theory), I should add some way for people to know what version they’re looking at, so I used a Hugo template to add a “Last update” at the bottom of each page:</p> <pre><code>Last updated {{ now.Format "Jan 01 2006" }} </code></pre> <h2 id="the-results">The Results</h2> <p>All is now ready, and this blog is fully available on IPFS, at <code>/ipns/Qme48wyZ7LaF9gC5693DZyJBtehgaFhaKycESroemD5fNX/</code> (which you can <a href="https://ipfs.leotindall.com/ipns/Qme48wyZ7LaF9gC5693DZyJBtehgaFhaKycESroemD5fNX/">access via my gateway</a>). Even if my server goes down, as long as these files are pinned <em>somewhere</em>, anyone should be able to use that IPNS name at another gateway and see the blog.</p> Wed, 19 Sep 2018 21:46:41 +0000 https://ipfs.io/ipns/Qme48wyZ7LaF9gC5693DZyJBtehgaFhaKycESroemD5fNX/post/putting_this_blog_on_ipfs/ China appears to be accelerating development of a super-heavy lift rocket https://arstechnica.com/science/2018/09/china-appears-to-be-accelerating-development-of-a-super-heavy-lift-rocket/ https://arstechnica.com/science/2018/09/china-appears-to-be-accelerating-development-of-a-super-heavy-lift-rocket/ <figure class="intro-image intro-left" readability="0.97107438016529"><img src="https://cdn.arstechnica.net/wp-content/uploads/2018/09/GettyImages-996594594-800x534.jpg" alt="Article intro image"/><figcaption class="caption" readability="4.3698347107438"><div class="caption-text" readability="33.816425120773"><a href="https://cdn.arstechnica.net/wp-content/uploads/2018/09/GettyImages-996594594.jpg" class="enlarge-link" data-height="683" data-width="1024">Enlarge</a> <span class="sep">/</span> A Long March-2C rocket carrying two satellites is launched at the Jiuquan Satellite Launch Center on July 9, 2018 in Jiuquan, Gansu Province of China. (Photo by Wang Jiangbo/China News Service/VCG)</div><p>Wang Jiangbo/China News Service/VCG</p></figcaption></figure><aside id="social-left"> </aside><!-- cache hit 130:single/related:2757e6109f3b5dad6f146c2331e3c43a --><!-- empty --><p>As part of its long-term planning, Chinese rocket officials have talked for some time about a super-heavy lift rocket that will enable a human lunar program. For this rocket, called the Long March 9, officials have <a href="https://spacenews.com/china-reveals-details-for-super-heavy-lift-long-march-9-and-reusable-long-march-8-rockets/">generally cited</a> the 2030 time frame for its maiden launch.</p> <p>However, at the World Conference on Science Literacy 2018 this week, an engineer with the China National Space Administration, Li Guoping, said the country planned to launch the Long March 9 booster in 2028. This comes as China has successfully ramped up its launch cadence in 2018—it should launch about three dozen orbital rockets this year, more than any other country. The <a href="http://www.xinhuanet.com/english/2018-09/19/c_137477227.htm">report</a> in the Chinese news service <em>Xinhua</em> did not specify why this larger rocket was now expected to launch two years earlier than previously announced.</p> <h2>A huge rocket</h2> <p>The Long March 9 is an extremely ambitious booster, with a diameter of 10 meters, length of 90 meters, and a proposed lift capacity of 140 tons to low-Earth orbit. Those numbers are on par with the Saturn V rocket that NASA designed and built during the 1960s to carry out the Apollo lunar landing program. It would be roughly equivalent, in terms of capability, to SpaceX's proposed Big Falcon Rocket, although there has been no word from China on whether any part of the Long March 9 might be reusable.</p> <p>NASA is further along in its development of its own big booster, the large Space Launch System rocket, which could make its maiden flight in 2020 or 2021. This version of the SLS rocket will have a launch capability of up to 95 tons to low-Earth orbit, according to a <a href="https://www.nasa.gov/exploration/systems/sls/to-the-moon.html">recent NASA update</a>. Eventually, the space agency plans to upgrade the SLS rocket into a Block 2 configuration with a more powerful second stage as well as advanced side boosters, and this rocket would have an estimated capability of 130 tons to low-Earth orbit. However, it seems unlikely that the Block 2 rocket would launch before 2028.</p> <p>This means that if SpaceX <a href="https://arstechnica.com/science/2018/09/nasa-isnt-going-to-pay-for-the-bfr-so-musk-charts-a-new-course/">fails to secure funding</a> for the Big Falcon Rocket and NASA continues on its slow development pace of the SLS rocket, China could have the world's most powerful rocket about a decade from now.</p> <h2>The purpose</h2> <p>China <a href="http://www.planetary.org/blogs/guest-blogs/2017/20171219-china-mars-sample-return-plans.html">reportedly</a> wants to use the Long March 9 rocket for an ambitious Mars sample return mission, and that could be the payload for the vehicle's first flight. However, given the scale of the proposed rocket, its overarching purpose seems likely to be geared toward taikonaut visits to the surface of the Moon.</p> <p>Chinese officials have previously said their long-term goal is a series of lunar surface missions beginning in the 2030 time frame. This approach could prove more attractive to some partners, such as the European Space Agency, than NASA's plan to build a <a href="https://arstechnica.com/science/2018/09/nasa-says-its-building-a-gateway-to-the-moon-critics-say-its-just-a-gate/">Deep Space Gateway</a> in a distant lunar orbit rather than going directly to the surface.</p> <p>By accelerating development of the Long March 9 rocket, Chinese officials could be offering a viable alternative to NASA's plans to other potential partners sooner. As ever with the Chinese space program, however, the most consequential decisions and planning occur out of the public view, so definitive answers are few.</p> Thu, 20 Sep 2018 07:23:48 +0000 https://arstechnica.com/science/2018/09/china-appears-to-be-accelerating-development-of-a-super-heavy-lift-rocket/ Introducing New Cloud Source Repositories https://cloud.google.com/blog/products/application-development/introducing-new-cloud-source-repositories https://cloud.google.com/blog/products/application-development/introducing-new-cloud-source-repositories <p>At Google, we spend a lot of time each day working with code. As Google has grown and the code base has increased in complexity, Google engineers have built a set of code tools to help our developers stay happy and productive each day. One particularly essential tool is code search, which is well-loved by Google engineers, and used by most engineers here multiple times a day to improve their productivity.</p><p>We’re pleased to bring code search to you with the newly revamped <a href="https://source.cloud.google.com">Cloud Source Repositories</a> in beta availability. It features an entirely new user interface and semantic code search capabilities. Cloud Source Repositories is powered by the same underlying code search infrastructure that Google engineers perform their code searches on every day. You get super fast results and save a lot of time finding that piece of code that you know you’ve seen before. The code search in Cloud Source Repositories uses the same document indexing and retrieval technologies that are used for Google Search, making it work very quickly. Using this code search can improve developer productivity, whether you host your code in Cloud Source Repositories or mirror your code from the cloud versions of GitHub or Bitbucket.</p><h2>Speeding up code search for better development</h2><p>As developers ourselves, we know how frequently we need to search code. One <a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/43835.pdf">developer case study</a> found that programmers conduct an average of five search sessions with 12 total queries each workday.</p><p>These search queries are often targeted at a particular code location, and programmers are typically looking for code with which they are somewhat familiar. Programmers are generally seeking answers to questions about:</p><ul readability="2.5"><li><p><b>what</b> code does, </p></li><li readability="0"><p><b>where</b> is code instantiated, </p></li><li readability="0"><p><b>why</b> code is behaving a certain way, </p></li><li readability="0"><p><b>who</b> was responsible for an edit and when it happened, </p></li><li readability="-1"><p>and <b>how</b> to perform a task.</p></li></ul><p>There are some <a href="https://storage.googleapis.com/pub-tools-public-publication-data/pdf/43835.pdf">common code search challenges</a> we’ve encountered. Here are a few of those, along with how Cloud Source Repositories handles them.</p><ul readability="1.5"><li readability="6"><p>You want to search across all the code at your company, but there are a lot of repositories with only a few stored locally—which aren’t up-to-date with the versions on the server. Storing code locally isn’t a great option, and your computer’s search tools aren’t very powerful. When you’re using Cloud Source Repositories, the default branches of all repositories are always indexed and up-to-date. It’s simpler and faster to search across all the code you have access to for a particular file/class/function, rather than hunting for the code you have stored locally.</p></li></ul><ul readability="1"><li readability="5"><p>You’re looking for code that performs a common operation that is used by other people in the company. With Cloud Source Repositories, you can do a quick search and find that code. By discovering and using the existing solution rather than reinventing a new solution, you’ll save time, avoid introducing bugs and keep a healthier code base by not adding unnecessary code that has to be maintained over time.</p></li></ul><ul readability="0"><li readability="3"><p>You don’t remember the right way to use a common code component like an event handler. With Cloud Source Repositories, you can enter a query and search across all of your company’s code for examples of how that event handler has been used successfully by other developers. You can write the code correctly the first time.</p></li></ul><ul readability="0.5"><li readability="4"><p>You discover an issue with your production application. It reports a specific error message to the server logs that reads “User ID 2503295 not found in PaymentDatabase.” You can perform a regular expression search for “User ID .* not found in PaymentDatabase” and instantly find the location in the code where this error was triggered. Then you can get a fix deployed to production for users. Rich regular expression matching allows an easy way to find usages, definitions and sample code, and helps for refactoring.</p></li></ul><h2>Search across all your code</h2><p>Bringing Google’s code search capabilities to Cloud Source Repositories means you’ll get the benefits of Google Search technology for your code base searches.</p><p>One key benefit is that now all owned repositories that are either mirrored or added to Cloud Source Repositories can be searched in a single query. This works whether you have a small weekend project or a code base the size of Google’s. And it’s fast: You’ll get the answers you need super quickly—much faster than previous functionality—so you can get back to writing code. And indexing is super fast, too, so the time between new code being added and being available means you’re always searching up-to-date code. Cloud Source Repositories respects all identity and access management (IAM) permissions, so users won’t see any code in search that they shouldn’t have access to, and there are no additional permissions to set up.</p><p>In addition, Cloud Source Repositories can search across thousands of different repositories with a single query. You can search for either files or code within files in the default branch of your repository. Cloud Source Repositories also has a semantic understanding of the code, which means that the search index identifies which parts of your code are entities such as classes, functions, and fields. Since the search index has classified these entities, your queries can include filters to limit the search to classes or functions. It also allows for improved search relevance by ranking important parts of code like classes, functions, and fields higher. By default, Cloud Source Repositories allows powerful search patterns using RE2 regular expressions so you can find the answers you need for even very complex questions. (You can work with individual special characters by using a backslash or an entire string by enclosing it in quotation marks.)</p><p>As you type in the search box, you’ll get suggestions for matching results (shown below). For Java, JavaScript, Go, C++, Python, TypeScript and Proto files, you’ll see result suggestions indicating whether the match is an entity such as a class, method, enum, or field.</p> Thu, 20 Sep 2018 10:51:10 +0000 https://cloud.google.com/blog/products/application-development/introducing-new-cloud-source-repositories Versity open-sources ScoutFS http://www.versity.com/blog/versity-open-sources-scoutfs http://www.versity.com/blog/versity-open-sources-scoutfs <p><span><br/>Versity has big news - today we open sourced our next generation archiving filesystem, <!--HubSpot Call-to-Action Code --><span class="hs-cta-wrapper" id="hs-cta-wrapper-66817e69-b37c-4dac-a3a2-66de3de56333"><span class="hs-cta-node hs-cta-66817e69-b37c-4dac-a3a2-66de3de56333" id="hs-cta-66817e69-b37c-4dac-a3a2-66de3de56333"><!--[if lte IE 8]><div id="hs-cta-ie-element"></div><![endif]--><a href="https://cta-redirect.hubspot.com/cta/redirect/3446843/66817e69-b37c-4dac-a3a2-66de3de56333"><img class="hs-cta-img" id="hs-cta-img-66817e69-b37c-4dac-a3a2-66de3de56333" src="https://no-cache.hubspot.com/cta/default/3446843/66817e69-b37c-4dac-a3a2-66de3de56333.png" alt="ScoutFS"/></a></span></span><!-- end HubSpot Call-to-Action Code -->.</span></p> <!--more--><p><strong>Background</strong></p> <p><span>As large archive sites around the world plan their future data storage systems, one of the key areas of concern is the scalability of the namespace.  Data collections with over one billion files become difficult to manage with current technology that typically relies on a central server for metadata management.</span></p> <p><span>In 2016, Versity started working on a new archiving filesystem specifically designed to manage ultra large namespaces by using a cluster of nodes to scale out metadata handling.  The scale out architecture led us to the name “scale out filesystem” or ScoutFS.</span></p> <p><span>Today, we have reached an important milestone in the development of ScoutFS.  We are releaing the technology under the GPLv2 open source license.</span></p> <p><span>ScoutFS is a POSIX compliant scalable clustered file system designed and implemented for archiving large data sets to low cost external storage resources such as tape, disk, object, and cloud. Key areas of innovation within the ScoutFS project include increasing the capacity of POSIX namespaces while being performant, eliminating the need for file system scans, and harnessing the power of multiple nodes to reach extremely high file creation rates.</span></p> <p><span>If you would like to go straight to the project to learn more, here are the links:</span></p> <p><span>Community site:<br/><!--HubSpot Call-to-Action Code --><span class="hs-cta-wrapper" id="hs-cta-wrapper-bc29623c-37cb-4e0d-984f-9b2f783c6079"><span class="hs-cta-node hs-cta-bc29623c-37cb-4e0d-984f-9b2f783c6079" id="hs-cta-bc29623c-37cb-4e0d-984f-9b2f783c6079"><!--[if lte IE 8]><div id="hs-cta-ie-element"></div><![endif]--><a href="https://cta-redirect.hubspot.com/cta/redirect/3446843/bc29623c-37cb-4e0d-984f-9b2f783c6079"><img class="hs-cta-img" id="hs-cta-img-bc29623c-37cb-4e0d-984f-9b2f783c6079" src="https://no-cache.hubspot.com/cta/default/3446843/bc29623c-37cb-4e0d-984f-9b2f783c6079.png" alt="ScoutFS.org"/></a></span></span><!-- end HubSpot Call-to-Action Code --><br/></span></p> <p><span>Github site: </span><a href="https://github.com/versity/scoutfs-kmod-dev" rel=" noopener"><span>https://github.com/versity/scoutfs-kmod-dev,</span></a> <a href="https://github.com/versity/scoutfs-utils-dev"><span>https://github.com/versity/scoutfs-utils-dev</span></a><span> patches or contributions welcome.</span></p> <p><span>Versity site:<br/><!--HubSpot Call-to-Action Code --><span class="hs-cta-wrapper" id="hs-cta-wrapper-66817e69-b37c-4dac-a3a2-66de3de56333"><span class="hs-cta-node hs-cta-66817e69-b37c-4dac-a3a2-66de3de56333" id="hs-cta-66817e69-b37c-4dac-a3a2-66de3de56333"><!--[if lte IE 8]><div id="hs-cta-ie-element"></div><![endif]--><a href="https://cta-redirect.hubspot.com/cta/redirect/3446843/66817e69-b37c-4dac-a3a2-66de3de56333"><img class="hs-cta-img" id="hs-cta-img-66817e69-b37c-4dac-a3a2-66de3de56333" src="https://no-cache.hubspot.com/cta/default/3446843/66817e69-b37c-4dac-a3a2-66de3de56333.png" alt="ScoutFS"/></a></span></span><!-- end HubSpot Call-to-Action Code --><br/></span></p> <p><span>To learn more about the next generation scalable archive product Versity is building that utilizes ScoutFS, see:<br/></span><!--HubSpot Call-to-Action Code --><span class="hs-cta-wrapper" id="hs-cta-wrapper-131391ad-bdb0-4918-84cf-24c66d4cebab"><span class="hs-cta-node hs-cta-131391ad-bdb0-4918-84cf-24c66d4cebab" id="hs-cta-131391ad-bdb0-4918-84cf-24c66d4cebab"><!--[if lte IE 8]><div id="hs-cta-ie-element"></div><![endif]--><a href="https://cta-redirect.hubspot.com/cta/redirect/3446843/131391ad-bdb0-4918-84cf-24c66d4cebab"><img class="hs-cta-img" id="hs-cta-img-131391ad-bdb0-4918-84cf-24c66d4cebab" src="https://no-cache.hubspot.com/cta/default/3446843/131391ad-bdb0-4918-84cf-24c66d4cebab.png" alt="VSM2"/></a></span></span><!-- end HubSpot Call-to-Action Code --></p> Share this blog post: <a href="https://twitter.com/share?ref_src=twsrc%5Etfw" class="twitter-share-button" data-via="versitysoftware" data-hashtags="data #archive" data-show-count="false">Tweet</a> <p><strong>Motivation and requirements</strong></p> <p><span>One might be wondering why we decided tackle the development of a new filesystem. The short answer is that no existing filesystem met our needs and no existing archiving file system was GPL.</span></p> <ul><li><span>An open source GPL archiving file system is an inherently safer and more user friendly long term solution for storing archival data where accessibility over very large time scales is a key consideration.  Placing archival data in proprietary file systems is costly and subjects the owner of the data to many vendor specific risks including discontinuation of the product. As in other technology verticals, it is likely that open source archival file system software will come to dominate the landscape. </span></li> <li><span>Versity customers need us to support massive namespaces - billions of files today, and tens to hundreds of billions in the near future. With exascale workloads quickly approaching, we set a goal to efficiently operate on at least one trillion files in a single namespace. We did not see an archiving filesystem technology on the horizon that would meet our namespace goals.</span></li> <li><span>The Versity archival user space application must efficiently find and apply archive policy to any added or modified files in the filesystem namespace. Additionally, there might be service level constraints that require any new or modified files to be archived within a set amount of time. The traditional namespace sweep using readdir() and stat() grows proportionally with the number of files and directories in the namespace. Using this method would grow beyond the service level requirements at a certain scale depending on hardware and filesystem implementation. While some filesystems might do better than others with this workload, it was clear that no filesystem could scale this workload to one trillion files while maintaining the service level requirements.</span></li> <li><span>Enterprise archive systems must be highly available and resilient to individual component failures to service the need of enterprise data protection. A clustered filesystem has both the benefit of remaining available during individual node failures as well as scaling out I/O performance across multiple hosts. This allows the use of commodity servers maximizing the price per performance and price per capacity of the system while remaining highly available.</span></li> <li><span>No two archive deployments are the same. Every site has different requirements to meet their individual business needs. Versity needs a flexible filesystem that can run on various hardware and storage devices depending on the specific deployment requirements of the site. This means that we need a filesystem that works well on the full scale of slower rotational drives to the latest solid state technologies.</span></li> <li><span>Versity supports multiple archive targets include tape, object, and disk archives. These all have demanding streaming performance requirements, but tape has even more constraints than the others. Tape devices must be written to with a single data stream, and have a fixed size buffer to maintain tape performance. The performance of a tape archive is strictly limited to the peak performance of the number of tape drives deployed. The economics of large tape archive systems demand that the tape performance be as close to peak at any given time as possible to maximize the value of the system. If each drive performance is not maximized, then a higher number of costly tape drives must be deployed to maintain the overall performance requirements of the system. Therefore Versity requires a filesystem that is capable of single stream performance in excess of the tape drive speeds while multiple streams to multiple tape devices are in flight.</span></li> <li><span>For an archive system, data correctness is paramount. <a href="https://en.wikipedia.org/wiki/Data_corruption" rel=" noopener" target="_blank">The wikipedia article on data corruption</a> outlines several cases where underlying storage devices have been found to return corrupted data without and notification of error. Because we cannot rely on the block devices to return correct data 100% of the time, an archive filesystem must have the ability to detect when the underlying data or metadata has been corrupted and either correct the data or return an error back to the user.</span></li> <li><span>The typical archive capacity far exceeds the reasonable capacity of a block storage filesystem. The Versity archive application copies file data to the archive storage tiers for long term storage. To make space for new incoming data, the file data for inactive files needs to be removed from the filesystem. The POSIX metadata, however, must remain online and accessible for immediate access. This requires an interface that allows for the removal of file data without changing any of the POSIX attributes of the file. When the inactive files become active again, the Versity archive application must retrieve the data from the archive tier and have an interface to be able to reconstitute the file data payload within the filesystem.</span></li> <li><span>Modern storage systems are comprised mostly of small files. A few large files take most of the capacity, but the small files take the bulk of the metadata space and arguably are used the most. Distributed filesystems typically don’t handle small files well, in fact they are known as their ‘achilles heel’. Single node filesystems aren’t capable of modern scale workloads. We needed a filesystem that handles both small and large files quickly and efficiently. </span></li> </ul><p><strong>Our Solution:</strong></p> <p><span>Given the motivations for our extreme scale archiving product, we decided to invest in building new technology that meets our needs. Two years ago we embarked on a journey to build a filesystem that is scalable, POSIX compliant, supports large scale archiving workloads, and most importantly keeps data safe and correct. </span></p> <p><span><!--HubSpot Call-to-Action Code --><span class="hs-cta-wrapper" id="hs-cta-wrapper-66817e69-b37c-4dac-a3a2-66de3de56333"><span class="hs-cta-node hs-cta-66817e69-b37c-4dac-a3a2-66de3de56333" id="hs-cta-66817e69-b37c-4dac-a3a2-66de3de56333"><!--[if lte IE 8]><div id="hs-cta-ie-element"></div><![endif]--><a href="https://cta-redirect.hubspot.com/cta/redirect/3446843/66817e69-b37c-4dac-a3a2-66de3de56333"><img class="hs-cta-img" id="hs-cta-img-66817e69-b37c-4dac-a3a2-66de3de56333" src="https://no-cache.hubspot.com/cta/default/3446843/66817e69-b37c-4dac-a3a2-66de3de56333.png" alt="ScoutFS"/></a></span></span><!-- end HubSpot Call-to-Action Code --> is built to address all the needs previously discussed. It is architected to support 1 trillion files in a single namespace, which supports the at scale large archive workloads that Versity currently supports and anticipates in the future. ScoutFS runs on a cluster of commodity Linux hardware. Some of the key features and design decisions follow.</span></p> <ul><li><span>ScoutFS maintains a metadata and data transaction sequence number index, which allows an application to retrieve the files roughly in order of modification time starting at any arbitrary index. This allows the application to retrieve all files that have changed since a given index. These queries are accomplished through the <a href="http://www.versity.com/scoutfs#AQI" rel=" noopener" target="_blank">Accelerated Query Interface (AQI)</a>. Knowing what has changed since the previous query allows our archiving application to apply archive policy to newly created and modified files without scanning the filesystem. <a href="http://www.versity.com/scoutfs#QP">This graph</a> shows AQI performance and how it stays flat even as the number of files increase. There are other application workflows that will be able to take advantage of the query interface.</span></li> <li><span>The shared-block filesystem is designed to minimize any synchronization or messaging the cluster may need, enabling POSIX compliance while still being performant. Each node having access to the same storage has a distinct advantage: whatever one node writes, another node can see. So if a node dies or is unreachable for any reason, another node simply picks up the work where the first node left off. The journaled and atomic design ensures that work is written to storage where other nodes can read it. </span></li> <li><span>The shared block design enables ‘local’ workloads and it does preallocated metadata work, which makes small file operations much faster while still being able to handle large files.</span></li> <li><span>Checksums are computed at the block level and include both data and metadata, ensuring that data are written and read correctly.</span></li> </ul><p><span>For more information on how ScoutFS is architected and implemented, download our white paper or see the community or github site:</span></p> <p><span>Community site: <br/></span><!--HubSpot Call-to-Action Code --><span class="hs-cta-wrapper" id="hs-cta-wrapper-bc29623c-37cb-4e0d-984f-9b2f783c6079"><span class="hs-cta-node hs-cta-bc29623c-37cb-4e0d-984f-9b2f783c6079" id="hs-cta-bc29623c-37cb-4e0d-984f-9b2f783c6079"><!--[if lte IE 8]><div id="hs-cta-ie-element"></div><![endif]--><a href="https://cta-redirect.hubspot.com/cta/redirect/3446843/bc29623c-37cb-4e0d-984f-9b2f783c6079"><img class="hs-cta-img" id="hs-cta-img-bc29623c-37cb-4e0d-984f-9b2f783c6079" src="https://no-cache.hubspot.com/cta/default/3446843/bc29623c-37cb-4e0d-984f-9b2f783c6079.png" alt="ScoutFS.org"/></a></span></span><!-- end HubSpot Call-to-Action Code --></p> <p><span>Github site: </span><a href="https://github.com/versity/scoutfs-kmod-dev" rel=" noopener"><span>https://github.com/versity/scoutfs-kmod-dev,</span></a> <a href="https://github.com/versity/scoutfs-utils-dev"><span>https://github.com/versity/scoutfs-utils-dev</span></a><span> patches or contributions welcome.</span></p> <p><span><span>Versity site:</span><br/><!--HubSpot Call-to-Action Code --><span class="hs-cta-wrapper" id="hs-cta-wrapper-66817e69-b37c-4dac-a3a2-66de3de56333"><span class="hs-cta-node hs-cta-66817e69-b37c-4dac-a3a2-66de3de56333" id="hs-cta-66817e69-b37c-4dac-a3a2-66de3de56333"><!--[if lte IE 8]><div id="hs-cta-ie-element"></div><![endif]--><a href="https://cta-redirect.hubspot.com/cta/redirect/3446843/66817e69-b37c-4dac-a3a2-66de3de56333"><img class="hs-cta-img" id="hs-cta-img-66817e69-b37c-4dac-a3a2-66de3de56333" src="https://no-cache.hubspot.com/cta/default/3446843/66817e69-b37c-4dac-a3a2-66de3de56333.png" alt="ScoutFS"/></a></span></span><!-- end HubSpot Call-to-Action Code --><br/></span></p> <p><span>To learn more about the next generation scalable archive product Versity is building that utilizes ScoutFS, see:<br/><!--HubSpot Call-to-Action Code --><span class="hs-cta-wrapper" id="hs-cta-wrapper-131391ad-bdb0-4918-84cf-24c66d4cebab"><span class="hs-cta-node hs-cta-131391ad-bdb0-4918-84cf-24c66d4cebab" id="hs-cta-131391ad-bdb0-4918-84cf-24c66d4cebab"><!--[if lte IE 8]><div id="hs-cta-ie-element"></div><![endif]--><a href="https://cta-redirect.hubspot.com/cta/redirect/3446843/131391ad-bdb0-4918-84cf-24c66d4cebab"><img class="hs-cta-img" id="hs-cta-img-131391ad-bdb0-4918-84cf-24c66d4cebab" src="https://no-cache.hubspot.com/cta/default/3446843/131391ad-bdb0-4918-84cf-24c66d4cebab.png" alt="VSM2"/></a></span></span><!-- end HubSpot Call-to-Action Code --></span></p> Share this blog post: <a href="https://twitter.com/share?ref_src=twsrc%5Etfw" class="twitter-share-button" data-via="versitysoftware" data-hashtags="data #archive" data-show-count="false">Tweet</a> Wed, 19 Sep 2018 08:45:04 +0000 http://www.versity.com/blog/versity-open-sources-scoutfs X11 Pause/Unpause Any Application https://vermaden.wordpress.com/2018/09/19/freebsd-desktop-part-16-configuration-pause-any-application/ https://vermaden.wordpress.com/2018/09/19/freebsd-desktop-part-16-configuration-pause-any-application/ <p>Many desktop oriented operating systems try to provide various usability improvements and features, like quite useful <em>Expose</em> or <em>Dashboard</em> in <em>Mac OS X</em> or useless <em>Tiles</em> concept in recent editions of <em>Microsoft Windows</em> systems.</p> <p><img data-attachment-id="1036" data-permalink="https://vermaden.wordpress.com/2018/09/19/freebsd-desktop-part-16-configuration-pause-any-application/pause-key/" data-orig-file="https://vermaden.files.wordpress.com/2018/09/pause-key.jpg?w=960" data-orig-size="435,335" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="pause.key" data-image-description="" data-medium-file="https://vermaden.files.wordpress.com/2018/09/pause-key.jpg?w=960?w=300" data-large-file="https://vermaden.files.wordpress.com/2018/09/pause-key.jpg?w=960?w=435" src="https://vermaden.files.wordpress.com/2018/09/pause-key.jpg?w=960" alt="pause.key.jpg" class="alignnone size-full wp-image-1036" srcset="https://vermaden.files.wordpress.com/2018/09/pause-key.jpg 435w, https://vermaden.files.wordpress.com/2018/09/pause-key.jpg?w=150 150w, https://vermaden.files.wordpress.com/2018/09/pause-key.jpg?w=300 300w" sizes="(max-width: 435px) 100vw, 435px"/></p> <p>After using UNIX for so many years I knew that I could freeze (or pause) any process in the system with <tt><strong>kill -17</strong></tt> (<tt>SIGSTOP</tt>) signal and then unfreeze it with with <tt><tt>kill -19</tt></tt> (<tt>SIGCONT</tt>) signal as I described in the <strong>Process Management</strong> section of the <strong><a href="https://vermaden.wordpress.com/2018/07/08/ghost-in-the-shell-part-2/" target="_blank">Ghost in the Shell – Part 2</a></strong> article. Doing it that way for the desktop applications is PITA to say the least. Can you imagine opening <tt><strong>xterm(1)</strong></tt> terminal and searching for all <em>Chromium</em> or <em>Firefox</em> processes and then freezing them one by one every time you need it? Me neither.</p> <p>Fortunately with introduction of so called X11 helper utilities – like <tt><strong>xdotool(1)</strong></tt> – it is now possible to implement it in more usable manner.</p> <p><strong>Today I will show you how to freeze any X11 application with single keyboard shortcut or mouse gesture if you utilize them in any way with small simple script.</strong></p> <p>When such feature can be useful (or what for)?</p> <p>Lets say you have <em>Firefox</em> started with many tabs open (50+) and you know that it drains battery life from your laptop. You can close it but when You will need information from any of those tabs, then You will have to start <em>Firefox</em> again (even more battery usage) and load all needed tabs (battery …). The alternative is to pause all <em>Firefox</em> processes when You do not use them. This will freeze all its processes and subprocesses and it will not use any CPU (or battery) power. When you will need it, then you will unpause it without the need to load all tabs again.</p> <p>Other example may be some heavy processing. For example you started <em>RawTherapee</em> or <em>Darktable</em> processing of large amount of photos and you are not able to smoothly watch a video. Just pause it, watch the video and unpause it again to finish its work.</p> <p>Its also usable in single player gaming when You can REALLY pause the game, literally 🙂</p> <p>You may also check earlier articles of the <em>FreeBSD Desktop</em> series:</p> <p>First we need to install the so called X11 helpers. Do that with this <tt><strong>pkg(8)</strong></tt> command.</p> <pre> # <strong>pkg install xprop xdotool zenity xbindkeys</strong> </pre> <p>Now for the script that would make all this magic happen. The <a href="https://github.com/vermaden/scripts/blob/master/desktop-pause.sh" target="_blank"><tt><strong>desktop-pause.sh</strong></tt></a> script is available on <em>GitHub</em> as its syntax is nicely colored there. Save it in some place where its searchable through <tt><strong>${PATH}</strong></tt> variable like <tt><strong>~/bin</strong></tt> or <tt><strong>~/script</strong></tt> directory and make it executable.</p> <pre> % <strong>fetch -O ~/scripts/desktop-pause.sh https://raw.githubusercontent.com/vermaden/scripts/master/desktop-pause.sh</strong> % <strong>chmod +x ~/scripts/desktop-pause.sh</strong> % <strong>echo $PATH | grep scripts</strong> /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/home/vermaden/scripts </pre> <p>It has three ways of usage.</p> <pre> % <strong>desktop-pause.sh</strong> usage: desktop-pause.sh OPTION [ARGUMENT] OPTIONS: -a - Do pause/resume active window. -s - Do pause/resume interactively selected window. -p - Do pause/resume specified PID. -l - Do list paused processes/windows. -L - Do list paused processes/windows with PIDs. ARGUMENT: PID for '-p' option. </pre> <p>If started with <tt><strong>-a</strong></tt> option, then it would pause/unpause the currently active window. This option is best used with keyboard shortcut or mouse gesture. It you start <a href="https://github.com/vermaden/scripts/blob/master/desktop-pause.sh" target="_blank"><tt><strong>desktop-pause.sh</strong></tt></a> script with <tt><strong>-s</strong></tt> argument, then the cursor will change and you will be able to select which window to freeze (or unfreeze). The <tt><strong>-p</strong></tt> option is usable in terminal directly as you may want to freeze/unfreeze a process without X11 environment or for some debugging purposes for example. The last <tt><strong>-l</strong></tt> option will list applications that are currently paused.</p> <p><img data-attachment-id="1035" data-permalink="https://vermaden.wordpress.com/2018/09/19/freebsd-desktop-part-16-configuration-pause-any-application/pause-key-thinkpad/" data-orig-file="https://vermaden.files.wordpress.com/2018/09/pause-key-thinkpad.jpg?w=960" data-orig-size="500,310" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="pause.key.thinkpad" data-image-description="" data-medium-file="https://vermaden.files.wordpress.com/2018/09/pause-key-thinkpad.jpg?w=960?w=300" data-large-file="https://vermaden.files.wordpress.com/2018/09/pause-key-thinkpad.jpg?w=960?w=500" src="https://vermaden.files.wordpress.com/2018/09/pause-key-thinkpad.jpg?w=960" alt="pause.key.thinkpad" class="alignnone size-full wp-image-1035" srcset="https://vermaden.files.wordpress.com/2018/09/pause-key-thinkpad.jpg 500w, https://vermaden.files.wordpress.com/2018/09/pause-key-thinkpad.jpg?w=150 150w, https://vermaden.files.wordpress.com/2018/09/pause-key-thinkpad.jpg?w=300 300w" sizes="(max-width: 500px) 100vw, 500px"/></p> <p>Most present-day generation laptops have island type limited keyboards so you will have to choose for yourself which keyboard shortcut to use. As I still use 2011 <em>ThinkPad T420s</em> laptop with 7-row keyboard I have little more options. The <a href="https://en.wikipedia.org/wiki/Break_key" target="_blank"><tt>[Pause Break]</tt></a> key seems to be the best candidate for such feature 🙂 I will use it for the ‘active window freeze/unfreeze’ with <tt><strong>-a</strong></tt> option and <tt>[SHIFT]-[Pause Break]</tt> key for the more interactive <tt><strong>-s</strong></tt> option.</p> <p>To create such new keyboard shortcut we will use handy <tt><strong>xbindkeys(1)</strong></tt> tool.</p> <p>Lets see what code we will have to put into the <tt><strong>~/.xbindkeysrc</strong></tt> configuration file.</p> <pre> % <strong>xbindkeys --help</strong> xbindkeys 1.8.6 by Philippe Brochard usage: xbindkeys [options] where options are: -V, --version Print version and exit -d, --defaults Print a default rc file -f, --file Use an alternative rc file -p, --poll-rc Poll the rc/guile configs for updates -h, --help This help! -X, --display Set X display to use -v, --verbose More information on xbindkeys when it run -s, --show Show the actual keybinding -k, --key Identify one key pressed -mk, --multikey Identify multi key pressed -g, --geometry size and position of window open with -k|-mk option -n, --nodaemon don't start as daemon </pre> <p>As its single key we will need <tt><strong>--key</strong></tt> option. Lets do it then.</p> <pre> % <strong>xbindkeys --key</strong> Press combination of keys or/and click under the window. You can use one of the two lines after "NoCommand" in $HOME/.xbindkeysrc to bind a key. "(Scheme function)" m:0x0 + c:110 Pause </pre> <p>Now lets read the <tt>[SHIFT]-[Pause Break]</tt> sequence.</p> <pre> % <strong>xbindkeys --key</strong> Press combination of keys or/and click under the window. You can use one of the two lines after "NoCommand" in $HOME/.xbindkeysrc to bind a key. "(Scheme function)" m:0x1 + c:110 Shift + Pause </pre> <p>We now have all needed information for the <tt><strong>~/.xbindkeysrc</strong></tt> configuration file. Here is how it looks configured.</p> <pre> % <strong>cat ~/.xbindkeysrc</strong> # [Pause Break] FOR ACTIVE WINDOW "~/scripts/desktop-pause.sh -a" Pause # [Shift]-[Pause Break] FOR INTERACTIVE WINDOW "~/scripts/desktop-pause.sh -s" Shift + Pause </pre> <p>Now lets start <tt><strong>xbindkeys(1)</strong></tt> and verify that it works.</p> <pre> % <strong>xbindkeys</strong> </pre> <p>Press the <tt>[Pause Break]</tt> key when you are in the terminal where you started <tt><strong>xbindkeys(1)</strong></tt> utility. Now hit <tt>[ENTER]</tt> several times, the terminal should be freezed. Now hit <tt>[Pause Break]</tt> key again. The etnered <tt>[ENTER]</tt> keys have been passed to it as it was unfreezed.</p> <p>Lets check the <em>Firefox</em> example.</p> <p>When processes run like usual they have on of the <tt>I*</tt>/<tt>S*</tt>/<tt>R*</tt> state like shown below.</p> <pre> % <strong>ps ax | grep firefox | grep -v grep</strong> 67981 - S 3:28.66 /usr/local/lib/firefox/firefox -contentproc -childID 58 -isForBrowser -prefsLen 31209 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab 41124 0- S 68:44.94 firefox 43940 0- S 25:52.43 /usr/local/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 27620 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab </pre> <p>When you will now freeze <em>Firefox</em> with <tt>[Pause Break]</tt> key its processes will have <tt>T</tt> state.</p> <pre> % <strong>ps ax | grep firefox | grep -v grep</strong> 67981 - T 3:28.66 /usr/local/lib/firefox/firefox -contentproc -childID 58 -isForBrowser -prefsLen 31209 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab 41124 0- T 68:45.17 firefox 43940 0- T 25:52.85 /usr/local/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 27620 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab </pre> <p>After you unfreeze them again with <tt>[Pause Break]</tt> key they will get back to normal <tt>I*</tt>/<tt>S*</tt>/<tt>R*</tt> state.</p> <pre> % <strong>ps ax | grep firefox | grep -v grep</strong> 67981 - S 3:28.67 /usr/local/lib/firefox/firefox -contentproc -childID 58 -isForBrowser -prefsLen 31209 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab 41124 0- S 68:45.54 firefox 43940 0- S 25:53.01 /usr/local/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 27620 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab </pre> <p>You may of course specify by hand the <em>Firefox</em> PID which is <tt>41124</tt> in current state.</p> <pre> % <strong>desktop-pause.sh -p 41124</strong> INFO: kill -17 41124 INFO: kill -17 67981 INFO: kill -17 43940 </pre> <p>The <em>Firefox</em> browser will be paused again.</p> <pre> % <strong>ps ax | grep firefox | grep -v grep</strong> 67981 - T 3:28.68 /usr/local/lib/firefox/firefox -contentproc -childID 58 -isForBrowser -prefsLen 31209 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab 41124 0- T 68:46.68 firefox 43940 0- T 25:56.22 /usr/local/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 27620 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab </pre> <p>Use it again to unpause it.</p> <pre> % <strong>desktop-pause.sh -p 41124</strong> INFO: kill -19 41124 INFO: kill -19 67981 INFO: kill -19 43940 </pre> <p>And viola! <em>Firefox</em> runs again.</p> <pre> % <strong>ps ax | grep firefox | grep -v grep</strong> 67981 - S 3:28.68 /usr/local/lib/firefox/firefox -contentproc -childID 58 -isForBrowser -prefsLen 31209 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab 41124 0- S 68:46.72 firefox 43940 0- S 25:56.28 /usr/local/lib/firefox/firefox -contentproc -childID 1 -isForBrowser -prefsLen 27620 -schedulerPrefs 0001,2 -appdir /usr/local/lib/firefox/browser 41124 tab </pre> <p>There are no downsides to this feature but one has to remember paused applications will not refresh themselves as their processes are freezed. Below you can see frozen <em>Epiphany</em> browser upon which the <tt><strong>xterm(1)</strong></tt> window was moved. Pretty <em>Windows</em> like effect.</p> <p><img data-attachment-id="1037" data-permalink="https://vermaden.wordpress.com/2018/09/19/freebsd-desktop-part-16-configuration-pause-any-application/epiphany-paused/" data-orig-file="https://vermaden.files.wordpress.com/2018/09/epiphany-paused.png?w=960" data-orig-size="1020,675" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="epiphany.paused" data-image-description="" data-medium-file="https://vermaden.files.wordpress.com/2018/09/epiphany-paused.png?w=960?w=300" data-large-file="https://vermaden.files.wordpress.com/2018/09/epiphany-paused.png?w=960?w=960" src="https://vermaden.files.wordpress.com/2018/09/epiphany-paused.png?w=960" alt="epiphany.paused" class="alignnone size-full wp-image-1037" srcset="https://vermaden.files.wordpress.com/2018/09/epiphany-paused.png?w=960 960w, https://vermaden.files.wordpress.com/2018/09/epiphany-paused.png?w=150 150w, https://vermaden.files.wordpress.com/2018/09/epiphany-paused.png?w=300 300w, https://vermaden.files.wordpress.com/2018/09/epiphany-paused.png?w=768 768w, https://vermaden.files.wordpress.com/2018/09/epiphany-paused.png 1020w" sizes="(max-width: 960px) 100vw, 960px"/></p> <p>After you unpause the <em>Epiphany</em> it gets back to normal as shown below.</p> <p><img data-attachment-id="1038" data-permalink="https://vermaden.wordpress.com/2018/09/19/freebsd-desktop-part-16-configuration-pause-any-application/epiphany-unpaused/" data-orig-file="https://vermaden.files.wordpress.com/2018/09/epiphany-unpaused.png?w=960" data-orig-size="1017,672" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}" data-image-title="epiphany.unpaused" data-image-description="" data-medium-file="https://vermaden.files.wordpress.com/2018/09/epiphany-unpaused.png?w=960?w=300" data-large-file="https://vermaden.files.wordpress.com/2018/09/epiphany-unpaused.png?w=960?w=960" src="https://vermaden.files.wordpress.com/2018/09/epiphany-unpaused.png?w=960" alt="epiphany.unpaused.png" class="alignnone size-full wp-image-1038" srcset="https://vermaden.files.wordpress.com/2018/09/epiphany-unpaused.png?w=960 960w, https://vermaden.files.wordpress.com/2018/09/epiphany-unpaused.png?w=150 150w, https://vermaden.files.wordpress.com/2018/09/epiphany-unpaused.png?w=300 300w, https://vermaden.files.wordpress.com/2018/09/epiphany-unpaused.png?w=768 768w, https://vermaden.files.wordpress.com/2018/09/epiphany-unpaused.png 1017w" sizes="(max-width: 960px) 100vw, 960px"/></p> <p>Remember to add <tt><strong>xbindkeys(1)</strong></tt> command to your <tt><strong>~/.xinitrc</strong></tt> (or <tt><strong>~/.xsession</strong></tt> file) to make it permanent.</p> <h2>UPDATE 1</h2> <p>One of the <em>Hacker News</em> users named <a href="https://news.ycombinator.com/user?id=rhn_mk1" target="_blank">rhn_mk1</a> explained the lack of window contents refresh while application is freezed. I will just cite his comment below.</p> <blockquote><p> That depends on the window manager. The application state is not really affected, it just stops updating (redrawing its area). When another window moves away, the window manager asks the “underlying” application to update that area of the screen. It’s dead, so the WM keeps displaying the last thing that was there, until something else happens in that spot.</p> <p>On the other hand, compositing window managers will dedicate a separate buffer to each application, where they have exclusive access. That kind of a window manager would not have to ask the application to update anything – it would just take the image from the dedicated application’s buffer and update the screen with it. Since the application’s buffer can’t be modified by anything else, it would have the last state of the application in it. That would in turn find its way to the screen. No glitches. </p></blockquote> <h2>UPDATE 2</h2> <p>One of the <em>Reddit</em> users <a href="https://www.reddit.com/user/89luca89" target="_blank">89luca89</a> pointed me to the <a href="https://gitlab.com/mkourim/browser-suspender" target="_blank">browser-suspender</a> solution that <em>‘simply suspends the browser when not in focus using STOP/CONT’</em> signals.</p> <h2>UPDATE 3</h2> <p>The <em>Lobsters</em> user <a href="https://lobste.rs/u/seschwar" target="_blank">seschwar</a> pointed out that there is <a href="https://hackage.haskell.org/package/xmonad-contrib-0.14/docs/XMonad-Layout-Stoppable.html" target="_blank">Stoppable Layout</a> functionality for <a href="https://xmonad.org/" target="_blank">XMonad</a> which automatically pauses the processes of all windows except for the active one and it also uses <tt>SIGCONT</tt> and <tt>SIGSTOP</tt> signals.</p> Wed, 19 Sep 2018 07:50:12 +0000 vermaden https://vermaden.wordpress.com/2018/09/19/freebsd-desktop-part-16-configuration-pause-any-application/ Kit Programming Language https://www.kitlang.org/ https://www.kitlang.org/ <tr readability="10.484949832776"><td>Modern language features: type inference, <a href="https://www.kitlang.org/examples.html#enumsalgebraic-data-types">algebraic data types</a>, <a href="https://www.kitlang.org/examples.html#match">pattern matching</a>, explicit function <a href="https://www.kitlang.org/examples.html#inline">inlining</a>, automatic <a href="https://www.kitlang.org/examples.html#pointers">pointer dereferencing</a>, <a href="https://www.kitlang.org/examples.html#generics">generics</a>, <a href="https://www.kitlang.org/examples.html#implicits">implicits</a>.</td> <td>Low-level control to optimize performance: <a href="https://www.kitlang.org/examples.html#pointers">pointers</a>, manual memory management, no GC (unless you introduce it yourself, which is easy!)</td> </tr><tr readability="8.0212201591512"><td>A more expressive type system, including <a href="https://www.kitlang.org/examples.html#traits">traits</a> for polymorphism, and <a href="https://www.kitlang.org/examples.html#abstracts">abstract types</a>, which provide custom compile-time behavioral and type checking semantics to existing types with no runtime cost.</td> <td>Metaprogramming via a typed <a href="https://www.kitlang.org/examples.html#term-rewriting">term rewriting system</a>; use rules to transform arbitrary expressions at compile time based on their type information. Create your own interface or DSL.</td> </tr><tr readability="12.427872860636"><td>A sane, easy to use build system. Kit features modules, imports, and standard package structure, plus a simple but powerful build tool: manage your project via a simple YAML configuration file and <code class="highlighter-rouge">kit build</code>, <code class="highlighter-rouge">kit test</code>, or <code class="highlighter-rouge">kit run</code>. (coming soon…)</td> <td>Zero-overhead <a href="https://www.kitlang.org/examples.html#c-interoperability">C interoperability</a>. Take advantage of existing C libraries without any wrappers; just include the header and directly use types/functions/variables.</td> </tr> Wed, 19 Sep 2018 12:13:46 +0000 https://www.kitlang.org/ National Museum of Computing to hold live Enigma code-breaking demo with a Bombe https://www.theregister.co.uk/2018/09/19/tnmoc_bombe_enigma_code_breaking_demo/ https://www.theregister.co.uk/2018/09/19/tnmoc_bombe_enigma_code_breaking_demo/ <p>The National Museum of Computing (TNMOC) is going to fire up its replica Enigma code-breaker to decrypt encoded messages sent from Poland – with an original wartime Bombe operator supervising the process.</p> <p>The British museum's Bombe replica, recently moved into the original Block H building that housed the wartime Bombes, is a fully functional reproduction of the machines that broke Nazi Germany's Enigma ciphers during the Second World War.</p> <p>The electro-mechanical computers were used to read military messages sent between high-ranking German commanders, giving Britain and its allies a vital insight into what the Germans were up to and what they were planning to do next.</p> <p>As part of the World Computer Congress being held in Poland this Friday (21 September), the Bombe will be fired up and set to work decrypting messages sent from the eastern European country. While the machine does its work (this is 1940s technology, remember), participants in the conference will hear papers about cryptography, the Bombe and master codebreaker Alan Turing, who, along with Gordon Welchman, was one of the leading minds behind the creation of the original Bombes.</p> <p>Ruth Bourne, a wartime Bombe operator, will be present at TNMOC on the day to verify the process as the operators attempt to find the key to decrypt the message traffic.</p> <p>The Bombe was built on codebreaking principles developed by Polish cryptanalysts in the very early stages of the Second World War, before that country was overrun by the invading German forces. Fleeing codebreakers took their knowledge to Britain, where what we now call GCHQ set to work adding to the Poles' efforts.</p> <p>The Enigma machine was the main cipher machine used by German signallers. It worked through three – later four or five – rotor wheels that determined how plain-text messages were enciphered ready for transmissions. At the heart of the German signals process was the sending of the enciphered key setting twice at the beginning of each message. Repetition of letters in that key allowed British cryptanalysts to analyse the text and start building machines to decipher German messages, figuring out the wheel settings for each day.</p> <p>Interested folk can watch the challenge live from around 0830 BST on Friday on the World Computing Congress <a target="_blank" href="http://wcc2018.org/Enigma-live">website</a>. ®</p> <p class="wptl btm"> <span>Sponsored: </span> <a href="https://go.theregister.co.uk/tl/1790/-6625/following-bottomlines-journey-to-the-hybrid-cloud?td=wptl1790">Following Bottomline’s journey to the Hybrid Cloud</a> </p> Thu, 20 Sep 2018 08:40:33 +0000 https://www.theregister.co.uk/2018/09/19/tnmoc_bombe_enigma_code_breaking_demo/