Deprecated: The each() function is deprecated. This message will be suppressed on further calls in /customers/c/9/b/feedsapi.com/httpd.www/advance/libraries/Zend/Cache/Backend.php on line 66 Warning: Cannot modify header information - headers already sent by (output started at /customers/c/9/b/feedsapi.com/httpd.www/advance/libraries/Zend/Cache/Backend.php:66) in /customers/c/9/b/feedsapi.com/httpd.www/advance/makefulltextfeed.php on line 316 Warning: Use of undefined constant SIMPLEPIE_FILE_SOURCE_NONE - assumed 'SIMPLEPIE_FILE_SOURCE_NONE' (this will throw an Error in a future version of PHP) in /customers/c/9/b/feedsapi.com/httpd.www/advance/libraries/humble-http-agent/SimplePie_HumbleHttpAgent.php on line 28 Warning: Use of undefined constant SIMPLEPIE_FILE_SOURCE_NONE - assumed 'SIMPLEPIE_FILE_SOURCE_NONE' (this will throw an Error in a future version of PHP) in /customers/c/9/b/feedsapi.com/httpd.www/advance/libraries/humble-http-agent/SimplePie_HumbleHttpAgent.php on line 28 Warning: Cannot modify header information - headers already sent by (output started at /customers/c/9/b/feedsapi.com/httpd.www/advance/libraries/Zend/Cache/Backend.php:66) in /customers/c/9/b/feedsapi.com/httpd.www/advance/libraries/feedwriter/FeedWriter.php on line 89 Hacker News https://news.ycombinator.com/ Links for the intellectually curious, ranked by readers. Chess World Championship: Stockfish live-analyzing game 7 http://analysis.sesse.net/ http://analysis.sesse.net/ <head><meta charset="utf-8"/><meta http-equiv="X-UA-Compatible" content="IE=edge"/><title>analysis.sesse.net</title><link rel="stylesheet" href="css/chessboard-0.3.0.min.css"/><link rel="stylesheet" href="css/remoteglot.css"/><meta name="viewport" content="width=device-width, initial-scale=1"/></head><body id="readabilityBody" readability="29.549663928305"> <audio id="ding" preload="none"><source src="ding.opus" type="audio/ogg; codecs=opus"/><source src="ding.mp3" type="audio/mp3"/></audio> <ul><li><strong>Score:</strong> 1.00 is the value of one pawn (in a balanced endgame). Positive values are better for white.</li> <li><strong>PV:</strong> Principal Variation, the series of moves the engine thinks is the best.</li> <li><strong>Thick red line:</strong> Marks the best move (in the view of the engine). Multiple chained arrows means that the PV starts with multiple successive moves with the same piece, ie., the engine thinks that the piece will execute a maneuver.</li> <li><strong>Thin red lines:</strong> Other good moves, maximum two. Note that even though these are also quality checked, these are less thoroughly analyzed by the engine, and should be taken with a grain of salt.</li> <li><strong>Thick blue line:</strong> Marks the best <em>response</em> move. Note that this is only rarely shown, since usually, the best response move depends on what the first move is. A typical case is when the current move is forced or nearly so.</li> </ul><p id="credits"><a href="http://git.sesse.net/?p=remoteglot;a=summary">remoteglot</a> © 2007–2018 <a href="http://www.sesse.net/">Steinar H. Gunderson</a>. Chess analysis by <a href="http://stockfishchess.org/" id="engineid">Stockfish</a><span id="enginedetails"/>. <span id="movesource"/> Hosting and additional analysis hardware by <a href="http://www.samfundet.no/">Studentersamfundet i Trondhjem</a> and Berge Schwebs Bjørlo. JavaScript chessboard powered by <a href="http://chessboardjs.com/">chessboard.js</a> and <a href="https://github.com/jhlywa/chess.js">chess.js</a>. Ding sound by <a href="https://www.freesound.org/people/Aiwha/sounds/196106/">Aiwha</a> (CC-BY-3.0). <span id="lomonosov">7-man Lomonosov tablebase lookup by <a href="http://tb7.chessok.com/">ChessOK</a>.</span></p> <!-- For faster development --> <!-- <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script> <script type="text/javascript" src="js/chessboard-0.3.0.min.js"></script> <script type="text/javascript" src="js/chess.js"></script> <script type="text/javascript" src="js/json_delta.js"></script> <script type="text/javascript" src="js/jquery.sparkline.js"></script> <script type="text/javascript" src="js/remoteglot.js"></script> --> <!-- Minified version of the previous six, compiled together --> <!-- --></body> Sun, 18 Nov 2018 17:55:58 +0000 http://analysis.sesse.net/ Show HN: Don't code your UI, draw it https://github.com/karanchahal/DoodleMaster https://github.com/karanchahal/DoodleMaster <div class="Box-body p-6"> <article class="markdown-body entry-content" itemprop="text"> <p>The Doodle Master seeks to turn your UI mockups into real code. Currently this repository just serves to demonstrate a Proof Of Concept of Artificially Intelligent Design Tools.</p> <p>The demo supports the detection of 4 classes of UI components</p> <ol><li>TextViews</li> <li>Headers</li> <li>ImageViews</li> <li>Buttons</li> </ol><p>This type of tool can be useful in a big organisation which adheres to a consistent design guide and has several reusable components on variety of platforms such as mobile and the web. For example a button on the Airbnb websote follows the same styling accross all platforms. This tool can simultaneously generate the same UI for Android and the Web.</p> <h2><a id="user-content-demo" class="anchor" aria-hidden="true" href="https://github.com/karanchahal/DoodleMaster#demo"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Demo</h2> <p><a target="_blank" rel="noopener noreferrer" href="https://github.com/karanchahal/DoodleMaster/blob/master/assets/new.gif"><img src="https://github.com/karanchahal/DoodleMaster/raw/master/assets/new.gif" alt="alt text"/></a></p> <h2><a id="user-content-technologies" class="anchor" aria-hidden="true" href="https://github.com/karanchahal/DoodleMaster#technologies"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Technologies</h2> <p>This demo uses a simple CNN for the classification of the doodles made. The software is able to localise the UI element through a combintion of geometric techniques and simple mouse/ finger position detction. Frameworks used are:</p> <ol><li>PyTorch - Deep Learning Model</li> <li>React- UI</li> </ol><p>This demo was inspired by AIrbnb's work on SKetching INterfaces. This demo was made while I was a frontend developer. I was inspired to make this as I saw a lot of developers spending time on coding the UI from the mockups, this time could be better spent in coding the functionality behing the UI. Doodle master seeks to build a painless and natural way of prototyping interfaces. This tool can be used by designers and developers alike to showcase ideas quickly in the form of real code.</p> <h5><a id="user-content-note" class="anchor" aria-hidden="true" href="https://github.com/karanchahal/DoodleMaster#note"><svg class="octicon octicon-link" viewbox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"/></svg></a>Note</h5> <p>This is not a prodution worthy piece of software,it is only meant for demo purposes. Please don't hesistate to contact me for more details on this project.</p> </article></div> Sun, 18 Nov 2018 12:48:24 +0000 https://github.com/karanchahal/DoodleMaster The Remarkable Brain Waves of High Level Meditators [video] https://kottke.org/18/11/the-remarkable-brain-waves-of-high-level-meditators https://kottke.org/18/11/the-remarkable-brain-waves-of-high-level-meditators <p>What’s going on in the brains of people who meditate? Anecdotal evidence suggests that meditation does something to people’s minds and bodies…quiets and calms them. In this video, Daniel Goleman reports on research done by his colleague Richard Davidson, a neuroscientist at the University of Wisconsin–Madison. Davidson brought a number of “Olympic level meditators” into his lab and hooked them up to a brain scanner. He found that the brains of these expert meditators have different brain wave patterns than the rest of us.</p> <blockquote readability="26"><p>Perhaps the most remarkable findings in the Olympic level meditators has to do with what’s called a gamma wave. All of us get gamma for a very short period when we solve a problem we’ve been grappling with, even if it’s something that’s vexed us for months. We get about half second of gamma; it’s the strongest wave in the EEG spectrum. We get it when we bite into an apple or imagine biting into an apple, and for a brief period, a split-second, inputs from taste, sound, smell, vision, all of that come together in that imagined bite into the apple. But that lasts very short period in an ordinary EEG.</p> <p>What was stunning was that the Olympic level meditators, these are people who have done up to 62,000 lifetime hours of meditation, their brainwave shows gamma very strong all the time as a lasting trait just no matter what they’re doing. It’s not a state effect, it’s not during their meditation alone, but it’s just their every day state of mind. We actually have no idea what that means experientially. Science has never seen it before.</p></blockquote> <p>Goleman and Davidson have written more about how meditation affects the mind and body in their book, <a href="https://www.amazon.com/exec/obidos/ASIN/0399184392/ref=nosim/0sil8">Altered Traits</a>.</p> <blockquote readability="14"><p>Sweeping away common misconceptions and neuromythology to open readers’ eyes to the ways data has been distorted to sell mind-training methods, the authors demonstrate that beyond the pleasant states mental exercises can produce, the real payoffs are the lasting personality traits that can result. But short daily doses will not get us to the highest level of lasting positive change — even if we continue for years — without specific additions. More than sheer hours, we need smart practice, including crucial ingredients such as targeted feedback from a master teacher and a more spacious, less attached view of the self, all of which are missing in widespread versions of mind training.</p></blockquote> <p>There’s that pesky <a href="https://kottke.org/06/05/creating-talent">deliberate practice</a> popping up again.</p> <p>More about...</p> Sun, 18 Nov 2018 12:37:55 +0000 https://kottke.org/18/11/the-remarkable-brain-waves-of-high-level-meditators Prac­ti­cal Ty­pog­ra­phy https://practicaltypography.com/ https://practicaltypography.com/ <div id="doc" readability="4.2151898734177"><style type="text/css"><![CDATA[ div#content { padding-top: 0; padding-bottom: 0; } div#content { margin-left: 13.5rem; } .mobile-only { display: none; } @media all and (max-width:520px) { div#content { margin: 1.5rem; margin-top: 0; padding: 0; } .mobile-only { display: block; } } #front-cover { position: absolute; top: -5rem; left : 2rem; font-family: "advocate-c41"; text-transform: uppercase; font-size: 1.7rem; line-height: 120%; mix-blend-mode: darken; } #front-cover-inner { position: fixed; } @media all and (max-width:520px) { #front-cover { display: none; } } #front-cover p { margin-bottom: 0.5rem;} a:after { content: none; } h1.toc-topic { font-size: 110%; font-style: normal; font-weight: bolder; } h1.toc-topic a { display: inline-block; width: 100%; } h1.toc-topic + h1.toc-topic { margin-top: 1.25rem; } ul.toc { font-size: 105%; column-count: 2; column-gap: 2em; margin-left: 0.5rem; margin-top: 0.5rem; margin-bottom: 1rem; } ul.toc + h1.toc-topic { margin-top: 1.75rem; } @media all and (max-width:520px) { ul.toc { column-count: 1; column-gap: 2em; font-size: 120%; } } ul.toc li { margin: 0; padding: 0; margin-bottom: 0.6em; line-height: 1.2; list-style-type: none; } ul.toc li:before { content: none; } .toc li a{ display: block; } .toc:after { content: none; } #toc-marker:hover { border-radius: inherit; } a#toc-marker:after { content: none; } ]]></style><p class="mobile-only"><a href="https://practicaltypography.com/how-to-pay-for-this-book.html">But­t­er­ick’s Prac­ti­cal <span class="no-hyphens">Typography</span></a></p><style type="text/css"><![CDATA[ /* form styles */ form { display: flex; flex-direction: column; margin-top: 1em; margin-bottom: 1em; } form .control_row { display: flex; flex-wrap: nowrap; } form, form * { font-family: "triplicate"; font-size: 0.7rem; } form > * { margin-top: 0.25em; margin-bottom: 0.25em; color: #222; } form input[name="email"] { margin-right: 0.22em; } form input[name="submit"] { margin-left: 0.22em; cursor: pointer; } input { background: white; padding: 0.25em; } input[type="radio"] { clip: rect(0 0 0 0); position: absolute; width: 0px; height: 0px; } input[type="radio"] + label { padding: 0.3em; border: 1px solid rgba(0, 0, 0, 0); } form:hover input[type="submit"] { background: #fbf3f3; } input[type="submit"] { border: 1px solid #d2d8ea; } form label { text-align: center; } form label .icon { font-family: feather; margin-right: 0.5em; } form textarea, form input[name="email"] { border: 1px solid #ddd; background: #fcfcfd; padding: 0.5em; } ]]></style><p><a href="https://practicaltypography.com/how-to-pay-for-this-book.html">Reader-sup­ported &amp; ad-free since <span class="no-hyphens">2013</span></a></p></div><a id="links"/> Sat, 17 Nov 2018 20:32:06 +0000 https://practicaltypography.com/ The Coming Crisis in Home Computers (1983) https://www.nytimes.com/1983/06/19/business/the-coming-crisis-in-home-computers.html https://www.nytimes.com/1983/06/19/business/the-coming-crisis-in-home-computers.html <p class="story-body-text story-content" data-para-count="438" data-total-count="2224" id="story-continues-3">T.I.'s crisis called to mind the sudden drop in video game sales announced by Atari last December. The video game business, which is now overlapping the home computer business, has been beset by oversupply - and in some cases big losses - ever since. And, it's possible a similar situation could develop in home computers, especially if T.I. decides to unload its excess inventory at prices that would make today's $99 bargains seem high.</p><p class="story-body-text story-content" data-para-count="403" data-total-count="2627">What is most surprising about the home computer business, said Mr. Purdy and others, is that such price cutting should occur so quickly after the market was born. Before 1982, there really were no ''home'' computers for sale in general retail channels under $1,000. Personal computers, such as the Apple and I.B.M., were generally sold through computer specialty stores and were often used in business.</p><p class="story-body-text story-content" data-para-count="298" data-total-count="2925">It was only last year that the home market started to develop as the price of computers dropped to little more than video games, as department stores and discount houses started carrying the machines and as the average homeowner developed a fear of his family being left behind in the computer age.</p><p class="story-body-text story-content" data-para-count="394" data-total-count="3319">The home computer business is still healthy in the sense that sales are continuing to increase, although there is disagreement over just how strong demand will be. While there were perhaps two million home computers sold last year, some analysts estimate that about five million will be sold this year. Thus, other companies are quickly moving to try to divorce themselves from T.I.'s problems.</p><p class="story-body-text story-content" data-para-count="368" data-total-count="3687">''Basically, I don't think we have a disaster in progress in the home computer business,'' said John V. Roach, chairman and chief executive of the Tandy Corporation, which sells its home computer through its Radio Shack chain. Sales are continuing healthy growth, he and other industry executives said. But T.I. produced even more than the growing market could handle.</p><p class="story-body-text story-content" data-para-count="508" data-total-count="4195">THE T.I. fiasco, however, might mark a shift in the way computers a remarketed and used. Fewer suppliers might survive at the low end, where computers sell for $100 or less and can be used for little more than playing video games or learning to program. Timex Corporation's sales of the first $100 computer have slowed despite a price reduction to below $50, and Mattel, a new entrant, is being stalled at the starting gate. Atari, one of the leaders of the market, has never made a profit in home computers.</p><p class="story-body-text story-content" data-para-count="382" data-total-count="4577">More attention might be paid to more expensive machines, selling for a few hundred dollars, which can be used for more than the $100 machines that are mainly designed to play games. Coleco Industries' new $600 machine, Adam, is seen as the start of that trend. I.B.M. is also likely to enter the field later this year with a machine geared toward business-oriented uses in the home.</p> <p class="story-body-text story-content" data-para-count="498" data-total-count="5075" id="story-continues-4">A factor propelling that trend, in addition to the price war at the lower end, is a concern that home computer growth will slow unless the machine is made more useful. ''You'll never get to sell 10 million units a year on the basis of current products,'' said Clive Smith, an analyst with the Yankee Group, management consultants who say that improvements are needed in the amount of software and its ease of use. ''The whole home computer market is a market racing to live up to its own promise.''</p><p class="story-body-text story-content" data-para-count="265" data-total-count="5340">FOR Texas Instruments, the loss was a serious blow and raised questions about whether the company will abandon the business as it did digital watches. Even the fear that T.I. might pull out could deter retailers from placing orders, further damaging T.I.'s chances.</p><p class="story-body-text story-content" data-para-count="201" data-total-count="5541">''You can't play a defensive game in consumer electronics,'' said Adam F. Cuhney of Salomon Brothers, who thinks T.I. will not recover in the business. ''They are not in control of their own destiny.''</p><p class="story-body-text story-content" data-para-count="256" data-total-count="5797">The company said the loss was caused by an unexpected slowdown in orders, particulary for software, that was brought about by a glut of home computers and video games on the market. T.I. said it was scaling down production and writing off excess inventory.</p><p class="story-body-text story-content" data-para-count="250" data-total-count="6047">But the magnitude and the suddenness of the loss surprised and puzzled analysts. An after-tax loss of $100 million in the second quarter for the company as a whole translates into a pretax loss on home computers alone of $200 million to $250 million.</p><p class="story-body-text story-content" data-para-count="363" data-total-count="6410">Texas Instruments officials last week declined to be interviewed. ''Not a chance. Absolutely not a chance,'' said company spokesman Norman Neureiter. Indeed, even the home computer division's public relations manager, whose job it is to talk to the press about that specific T.I. activity, has not been allowed to talk to some publications in the last few weeks.</p> <p class="story-body-text story-content" data-para-count="305" data-total-count="6715">Nevertheless, it is evident that the loss was the result of a combination of factors, all of which are evidence of a consistent problem that T.I. has had in the consumer electronics business. In digital watches and calculators, T.I. helped drive prices so low that it ruined the business, even for itself.</p><p class="story-body-text story-content" data-para-count="473" data-total-count="7188">One of the other companies severely hurt in the calculator wars was Commodore International, which was dependent on other companies for the chips it used and was therefore limited in how much it could cut costs. Since then, Commodore has vertically integrated. When Commodore and T.I. faced off again in home computers, Commodore was the winner. Mistake number one for T.I., analysts and competitors say, was getting into a price war with a more agile, lower-cost producer.</p><p class="story-body-text story-content" data-para-count="671" data-total-count="7859">T.I.'s home computer was first introduced in 1979 for $1,150 and was an embarrassing failure. T.I. discovered in 1982 that it could sell large volumes by cutting the price and offering rebates. But the T.I. 99/4A, which was little different from the machine that was originally designed to sell for $1,150, was no match cost-wise for Commodore's VIC-20. As Commodore and T.I. started their downhill race, it was inevitable that prices would get so low that T.I. would start losing money while Commodore stayed profitable. T.I. ''got suckered by Jack,'' said Harry Fox, president of Spectravideo, a new home computer company, referring to Commodore president Jack Tramiel.</p> <p class="story-body-text story-content" data-para-count="265" data-total-count="8124" id="story-continues-5">Even that would not have been too bad had T.I. been able to make up for low-priced computers by making large profits on software and peripheral equipment as it had hoped. But there is a flaw in such a razor-and-blade approach to profits in the home computer market.</p><p class="story-body-text story-content" data-para-count="310" data-total-count="8434">The history of the personal computer business, brief as it is, has shown that the successful machines are the ones that have the most and best software available for them. It has also shown that no single company can write all the software itself. It must take advantage of the cottage industry of programmers.</p><p class="story-body-text story-content" data-para-count="533" data-total-count="8967">Yet if T.I. was to make its profits in software, the software that sold had to be its own, not that of others. That became clear in December, when Atari's profits took a plunge because other companies began making games for its video game machine that were better than the games Atari itself was selling. T.I., trying to balance those contradictory influences, decided to try to get other companies to write software for its machine, but insisted on marketing the software itself. But many software companies would have none of that.</p><p class="story-body-text story-content" data-para-count="533" data-total-count="9500">The result was that T.I. discouraged software companies, putting its computer at a further disadvantage relative to the VIC-20 and other machines. And without other companies competing with it, T.I. did not put enough effort into developing its own software, particularly games, according to competitors and former employees. It also did not license popular arcade games like Zaxxon and Frogger, which are still the best selling programs for home computers. Its peripheral equipment, such as disk drives, were also overly expensive.</p><p class="story-body-text story-content" data-para-count="515" data-total-count="10015">Two other mistakes helped administer the final blow. One was that T.I. became over-optimistic and produced too many machines. While most industry analysts expect about five million home computers to be sold this year, Texas Instruments, at its annual meeting in April, predicted industry sales would total 6.7 million. T.I. geared up to produce three million computers this year, and now might be able to sell only two million, according to Michael Krasko of L.F. Rothschild, Unterberg, Towbin. Inventory piled up.</p><p class="story-body-text story-content" data-para-count="366" data-total-count="10381">T.I. also had to withdraw its computer from the market for about a month earlier this year to correct a potential electric shock hazard, for which T.I. took an unusually large $50 million charge against earnings in the first quarter. Inventories swelled because production continued while sales halted, and the product never regained its momentum when it reappeared.</p><p class="story-body-text story-content" data-para-count="194" data-total-count="10575">With all that excess inventory, T.I. was stung when it reduced prices and had to make up the difference to retailers and distributors who were holding inventory purchased at the previous prices.</p><p class="story-body-text story-content" data-para-count="337" data-total-count="10912">Future options are slim. The company said in its announcement that it plans to stay in the business. But it needs new products. One, a $100 computer introduced in January that was designed to be used to learn programming never got to market because prices for the more versatile 99/4A had already dropped almost that low by last spring.</p> <p class="story-body-text story-content" data-para-count="517" data-total-count="11429" id="story-continues-6">The company was expected to introduce a more expensive machine, the 99/8 at the Consumer Electronics Show in Chicago two weeks ago. But the machine, code-named the Armadillo, was not displayed publicly, partly because it would have been upstaged by Coleco's Adam, as well as by new products from Commodore and Atari. Its status is somewhat unclear. Mr. Purdy of Service Merchandise said that when he expressed interest in putting the 99/8 into the company's next catalog, T.I. officials were ''very vague about it.''</p><p class="story-body-text story-content" data-para-count="704" data-total-count="12133">It is also believed T.I. is working on a bundled system, with software, printer and data storage device combined in a package. Indeed, bundling appears to be a new trend. Rather than competing on price for the basic unit, and hoping to earn a profit on purchases of software and peripherals later, manufacturers are starting to package everything together in an effort to recover their profit up front. Coleco's Adam, which will sell for $600, includes 80,000 characters of internal memory, a daisy-wheel printer, a tape storage device, game joysticks, a word-processing program and a game. The package will allow people at home to use the computer in place of a typewriter as well as for playing games.</p><p class="story-body-text story-content" data-para-count="417" data-total-count="12550">Not everyone is convinced, however, that bundling is the way to go. Mr. Tramiel of Commodore said that bundling makes the entry price for many consumers too high. Bundling also limits the consumers choice. ''They've made up your mind for you on what kind of printer you should have with your machine,'' said William G. Kelley, president of the American Home Video Corporation, a Denver-based electronics store chain.</p><p class="story-body-text story-content" data-para-count="631" data-total-count="13181">One thing that seems certain is that movement will be away from game playing toward more powerful computers with greater capabilities. Those more versatile machines will drop in price to the point where they slow the sales of computers like the T.I. 99/4A and the VIC-20, just as those two computers dropped in price to the point where they slowed the sales of even less versatile video game machines and of computers like the Timex Sinclair 1000, which is used mostly to learn programming. Already Commodore has reduced the price of the Commodore 64 to about $200 wholesale and has started to deemphasize the less powerful VIC-20.</p><p class="story-body-text story-content" data-para-count="707" data-total-count="13888">I.B.M.'S entry into the home computer market, expected this fall with the introduction of Peanut, will accelerate that trend. The Peanut is expected to sell for aout $600 and will be compatible with I.B.M.'s more expensive personal computer. It would have a powerful 16-bit microprocessor, compared to the 8-bit chips in most home computers now. Mr. Smith of the Yankee Group predicts that, within a year, major home computer manufacturers will come out with new computers that can run their existing programs as well as programs written for the new I.B.M. machine. Japanese companies, who last week agreed on standard specifications for home computers, might also make an entrance into the American market.</p><p class="story-body-text story-content" data-para-count="288" data-total-count="14176">Ironically, the move toward more powerful systems might actually restore in consumers' minds the computer's perceived value that had been damaged in the fierce price cutting. Said Mr. Kelley of American Home Video, ''When they went to $99, people started asking 'What's wrong with it?'' '</p><aside class="story-content subscriptions-module hidden" id="subscriptions-module"/><a class="visually-hidden skip-to-text-link" href="https://www.nytimes.com/1983/06/19/business/the-coming-crisis-in-home-computers.html#whats-next">Continue reading the main story</a> Sun, 18 Nov 2018 11:54:58 +0000 https://www.nytimes.com/1983/06/19/business/the-coming-crisis-in-home-computers.html Building your own shell using Rust https://www.joshmcguigan.com/blog/build-your-own-shell-rust/ https://www.joshmcguigan.com/blog/build-your-own-shell-rust/ <p>This is a tutorial on building your own shell using Rust, in the spirit of the <a href="https://github.com/danistefanovic/build-your-own-x">build-your-own-x</a> list. Creating a shell is a great way to understand how the shell, terminal emulator, and OS work together.</p> <h2>What is a shell?</h2> <p>A shell is a program which allows you to control your computer. It does this largely by making it easy to launch other applications. But a shell on it’s own isn’t an interactive application. </p> <p>Most users interact with the shell through a terminal emulator. A concise description of a terminal emulator by user <a href="https://askubuntu.com/a/111149">geirha</a> follows:</p> <blockquote readability="12"> <p>The terminal emulator (often just called terminal) is “just the window”, yes. It runs a text based program, which by default is your login shell (which is bash in Ubuntu). When you type characters in the window, the terminal draws these characters in the window in addition to sending it to the shell’s (or other program’s) stdin. The characters the shell outputs to stdout and stderr get sent to the terminal, which in turn draws these characters in the window. </p> </blockquote> <p>In this tutorial, we’ll write our own shell and run it inside our normal terminal emulator (wherever you’d typically <code class="language-text">cargo run</code>). </p> <h2>A Starting Point</h2> <p>The simplest possible shell requires only a handful of lines of Rust code. Here we create a new string, which is used to hold the user input. The <code class="language-text">stdin().read_line</code> function blocks until the user presses the enter key, then it writes the entire user input (including the newline from pressing enter) into our string. After stripping the newline character with <code class="language-text">input.trim()</code> we attempt to run the command. </p> <div class="gatsby-highlight" readability="8"> <pre class="language-rust"><code class="language-rust"><span class="token keyword">fn</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> input <span class="token operator">=</span> String<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">stdin</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">read_line</span><span class="token punctuation">(</span><span class="token operator">&amp;</span><span class="token keyword">mut</span> input<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> command <span class="token operator">=</span> input<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> Command<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span>command<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">spawn</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span></code></pre> </div> <p>After <code class="language-text">cargo run</code>ing this, you should see a flashing cursor in your terminal which is waiting for input. Try typing <code class="language-text">ls</code> and pressing <code class="language-text">enter</code>, and you will see the <code class="language-text">ls</code> command print the contents of the current directory, and then the shell will exit.</p> <p>Note: These examples cannot be run in the <a href="https://play.rust-lang.org/">Rust Playground</a> because the playground does not currently support stdin nor long running processes. </p> <h2>Accept Multiple Commands</h2> <p>We don’t want our shell to exit after the user enters a single command. Supporting multiple commands is mostly a matter of wrapping the code above in a <code class="language-text">loop</code>, and adding a call to <code class="language-text">wait</code> on each child process to ensure we don’t prompt the user for additional input before the current process finishes. I’ve also added a couple lines to print the <code class="language-text">&gt;</code> character in order to make it easier for the user to distinguish their input from the output of the processes they spawn. </p> <div class="gatsby-highlight" readability="9"> <pre class="language-rust"><code class="language-rust"><span class="token keyword">fn</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span> <span class="token keyword">loop</span> <span class="token punctuation">{</span> <span class="token function">print!</span><span class="token punctuation">(</span><span class="token string">"&gt; "</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">stdout</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">flush</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> input <span class="token operator">=</span> String<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">stdin</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">read_line</span><span class="token punctuation">(</span><span class="token operator">&amp;</span><span class="token keyword">mut</span> input<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> command <span class="token operator">=</span> input<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> child <span class="token operator">=</span> Command<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span>command<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">spawn</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> child<span class="token punctuation">.</span><span class="token function">wait</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre> </div> <p>Running this code you’ll see that after running a first command, the prompt comes back so you can enter a second command. Try it out with, for example, the <code class="language-text">ls</code> and <code class="language-text">pwd</code> commands.</p> <h2>Handling Args</h2> <p>If you try running the command <code class="language-text">ls -a</code> on the shell above, it will crash. Since it is not aware of arguments, it tries to run a command called <code class="language-text">ls -a</code>, but the proper behavior is running a command called <code class="language-text">ls</code> with the argument <code class="language-text">-a</code>. </p> <p>This is fixed below by splitting the user input on whitespace characters, and treating anything before the first whitespace as the name of the command (e.g. <code class="language-text">ls</code>), while anything after the first whitespace is passed to that command as args (e.g. <code class="language-text">-a</code>). </p> <div class="gatsby-highlight" readability="10"> <pre class="language-rust"><code class="language-rust"><span class="token keyword">fn</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span> <span class="token keyword">loop</span> <span class="token punctuation">{</span> <span class="token function">print!</span><span class="token punctuation">(</span><span class="token string">"&gt; "</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">stdout</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">flush</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> input <span class="token operator">=</span> String<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">stdin</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">read_line</span><span class="token punctuation">(</span><span class="token operator">&amp;</span><span class="token keyword">mut</span> input<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> parts <span class="token operator">=</span> input<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">split_whitespace</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> command <span class="token operator">=</span> parts<span class="token punctuation">.</span><span class="token function">next</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> args <span class="token operator">=</span> parts<span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> child <span class="token operator">=</span> Command<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span>command<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">args</span><span class="token punctuation">(</span>args<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">spawn</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> child<span class="token punctuation">.</span><span class="token function">wait</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre> </div> <h2>Shell Built-ins</h2> <p>It turns out there are certain commands that the shell cannot simply dispatch to another process. These are things which affect something internal to the shell, and thus must be implemented by the shell itself. </p> <p>Probably the most common example of this is the <code class="language-text">cd</code> command. For an explanation of why <code class="language-text">cd</code> must be a shell built-in, check out <a href="https://unix.stackexchange.com/a/38809">this link</a>. In addition to the shell built-in, there actually is a program called <code class="language-text">cd</code>. The reasons for this duality is explained <a href="https://unix.stackexchange.com/a/38819">here</a>.</p> <p>Below we add support to our shell for the <code class="language-text">cd</code> built-in. </p> <div class="gatsby-highlight" readability="13"> <pre class="language-rust"><code class="language-rust"><span class="token keyword">fn</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span> <span class="token keyword">loop</span> <span class="token punctuation">{</span> <span class="token function">print!</span><span class="token punctuation">(</span><span class="token string">"&gt; "</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">stdout</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">flush</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> input <span class="token operator">=</span> String<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">stdin</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">read_line</span><span class="token punctuation">(</span><span class="token operator">&amp;</span><span class="token keyword">mut</span> input<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> parts <span class="token operator">=</span> input<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">split_whitespace</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> command <span class="token operator">=</span> parts<span class="token punctuation">.</span><span class="token function">next</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> args <span class="token operator">=</span> parts<span class="token punctuation">;</span> <span class="token keyword">match</span> command <span class="token punctuation">{</span> <span class="token string">"cd"</span> <span class="token operator">=&gt;</span> <span class="token punctuation">{</span> <span class="token keyword">let</span> new_dir <span class="token operator">=</span> args<span class="token punctuation">.</span><span class="token function">peekable</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">peek</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">map_or</span><span class="token punctuation">(</span><span class="token string">"/"</span><span class="token punctuation">,</span> <span class="token operator">|</span>x<span class="token operator">|</span> <span class="token operator">*</span>x<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> root <span class="token operator">=</span> Path<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span>new_dir<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token function">Err</span><span class="token punctuation">(</span>e<span class="token punctuation">)</span> <span class="token operator">=</span> env<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">set_current_dir</span><span class="token punctuation">(</span><span class="token operator">&amp;</span>root<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">eprintln!</span><span class="token punctuation">(</span><span class="token string">"{}"</span><span class="token punctuation">,</span> e<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> command <span class="token operator">=&gt;</span> <span class="token punctuation">{</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> child <span class="token operator">=</span> Command<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span>command<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">args</span><span class="token punctuation">(</span>args<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">spawn</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> child<span class="token punctuation">.</span><span class="token function">wait</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre> </div> <h2>Error Handling</h2> <p>If you’ve been following along, you’ve probably already noticed that the shells above will crash if you input a command which does not exist. In the version below, that is handled gracefully, by printing an error to the user and then allowing them to enter another command. </p> <p>Since entering a bad command was acting as an easy way to quit the shell, I’ve also implemented another shell built-in, the <code class="language-text">exit</code> command. </p> <div class="gatsby-highlight" readability="17"> <pre class="language-rust"><code class="language-rust"><span class="token keyword">fn</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span> <span class="token keyword">loop</span> <span class="token punctuation">{</span> <span class="token function">print!</span><span class="token punctuation">(</span><span class="token string">"&gt; "</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">stdout</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">flush</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> input <span class="token operator">=</span> String<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">stdin</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">read_line</span><span class="token punctuation">(</span><span class="token operator">&amp;</span><span class="token keyword">mut</span> input<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> parts <span class="token operator">=</span> input<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">split_whitespace</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> command <span class="token operator">=</span> parts<span class="token punctuation">.</span><span class="token function">next</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> args <span class="token operator">=</span> parts<span class="token punctuation">;</span> <span class="token keyword">match</span> command <span class="token punctuation">{</span> <span class="token string">"cd"</span> <span class="token operator">=&gt;</span> <span class="token punctuation">{</span> <span class="token keyword">let</span> new_dir <span class="token operator">=</span> args<span class="token punctuation">.</span><span class="token function">peekable</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">peek</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">map_or</span><span class="token punctuation">(</span><span class="token string">"/"</span><span class="token punctuation">,</span> <span class="token operator">|</span>x<span class="token operator">|</span> <span class="token operator">*</span>x<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> root <span class="token operator">=</span> Path<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span>new_dir<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token function">Err</span><span class="token punctuation">(</span>e<span class="token punctuation">)</span> <span class="token operator">=</span> env<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">set_current_dir</span><span class="token punctuation">(</span><span class="token operator">&amp;</span>root<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">eprintln!</span><span class="token punctuation">(</span><span class="token string">"{}"</span><span class="token punctuation">,</span> e<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"exit"</span> <span class="token operator">=&gt;</span> <span class="token keyword">return</span><span class="token punctuation">,</span> command <span class="token operator">=&gt;</span> <span class="token punctuation">{</span> <span class="token keyword">let</span> child <span class="token operator">=</span> Command<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span>command<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">args</span><span class="token punctuation">(</span>args<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">spawn</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">match</span> child <span class="token punctuation">{</span> <span class="token function">Ok</span><span class="token punctuation">(</span><span class="token keyword">mut</span> child<span class="token punctuation">)</span> <span class="token operator">=&gt;</span> <span class="token punctuation">{</span> child<span class="token punctuation">.</span><span class="token function">wait</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token function">Err</span><span class="token punctuation">(</span>e<span class="token punctuation">)</span> <span class="token operator">=&gt;</span> <span class="token function">eprintln!</span><span class="token punctuation">(</span><span class="token string">"{}"</span><span class="token punctuation">,</span> e<span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre> </div> <h2>Pipes</h2> <p>It would be difficult to be productive in a shell which didn’t include pipes. If you aren’t familiar with this feature, the <code class="language-text">|</code> character is used to tell the shell to redirect the output of the first command into the input of the second command. For example, running the command <code class="language-text">ls | grep Cargo</code> triggers the following set of actions:</p> <ol><li><code class="language-text">ls</code> will list all files in the current directory</li> <li>The shell will pipe the above list of files to <code class="language-text">grep</code></li> <li><code class="language-text">grep</code> will filter the list and output only files which contain the string <code class="language-text">Cargo</code></li> </ol><p>This final iteration of our shell includes very basic support for pipes. For an introduction to many other things pipes and IO redirection can do, check out <a href="https://robots.thoughtbot.com/input-output-redirection-in-the-shell">this article</a>.</p> <div class="gatsby-highlight" readability="18"> <pre class="language-rust"><code class="language-rust"><span class="token keyword">fn</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">{</span> <span class="token keyword">loop</span> <span class="token punctuation">{</span> <span class="token function">print!</span><span class="token punctuation">(</span><span class="token string">"&gt; "</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">stdout</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">flush</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> input <span class="token operator">=</span> String<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token function">stdin</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">read_line</span><span class="token punctuation">(</span><span class="token operator">&amp;</span><span class="token keyword">mut</span> input<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> commands <span class="token operator">=</span> input<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">split</span><span class="token punctuation">(</span><span class="token string">" | "</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">peekable</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> previous_command <span class="token operator">=</span> None<span class="token punctuation">;</span> <span class="token keyword">while</span> <span class="token keyword">let</span> <span class="token function">Some</span><span class="token punctuation">(</span>command<span class="token punctuation">)</span> <span class="token operator">=</span> commands<span class="token punctuation">.</span><span class="token function">next</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token keyword">let</span> <span class="token keyword">mut</span> parts <span class="token operator">=</span> command<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">split_whitespace</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> command <span class="token operator">=</span> parts<span class="token punctuation">.</span><span class="token function">next</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> args <span class="token operator">=</span> parts<span class="token punctuation">;</span> <span class="token keyword">match</span> command <span class="token punctuation">{</span> <span class="token string">"cd"</span> <span class="token operator">=&gt;</span> <span class="token punctuation">{</span> <span class="token keyword">let</span> new_dir <span class="token operator">=</span> args<span class="token punctuation">.</span><span class="token function">peekable</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">peek</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">map_or</span><span class="token punctuation">(</span><span class="token string">"/"</span><span class="token punctuation">,</span> <span class="token operator">|</span>x<span class="token operator">|</span> <span class="token operator">*</span>x<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> root <span class="token operator">=</span> Path<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span>new_dir<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token function">Err</span><span class="token punctuation">(</span>e<span class="token punctuation">)</span> <span class="token operator">=</span> env<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">set_current_dir</span><span class="token punctuation">(</span><span class="token operator">&amp;</span>root<span class="token punctuation">)</span> <span class="token punctuation">{</span> <span class="token function">eprintln!</span><span class="token punctuation">(</span><span class="token string">"{}"</span><span class="token punctuation">,</span> e<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> previous_command <span class="token operator">=</span> None<span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token string">"exit"</span> <span class="token operator">=&gt;</span> <span class="token keyword">return</span><span class="token punctuation">,</span> command <span class="token operator">=&gt;</span> <span class="token punctuation">{</span> <span class="token keyword">let</span> stdin <span class="token operator">=</span> previous_command <span class="token punctuation">.</span><span class="token function">map_or</span><span class="token punctuation">(</span> Stdio<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">inherit</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token operator">|</span>output<span class="token punctuation">:</span> Child<span class="token operator">|</span> Stdio<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">from</span><span class="token punctuation">(</span>output<span class="token punctuation">.</span>stdout<span class="token punctuation">.</span><span class="token function">unwrap</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">let</span> stdout <span class="token operator">=</span> <span class="token keyword">if</span> commands<span class="token punctuation">.</span><span class="token function">peek</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">is_some</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span> Stdio<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">piped</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span> Stdio<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">inherit</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token keyword">let</span> output <span class="token operator">=</span> Command<span class="token punctuation">:</span><span class="token punctuation">:</span><span class="token function">new</span><span class="token punctuation">(</span>command<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">args</span><span class="token punctuation">(</span>args<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">stdin</span><span class="token punctuation">(</span>stdin<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">stdout</span><span class="token punctuation">(</span>stdout<span class="token punctuation">)</span> <span class="token punctuation">.</span><span class="token function">spawn</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token keyword">match</span> output <span class="token punctuation">{</span> <span class="token function">Ok</span><span class="token punctuation">(</span>output<span class="token punctuation">)</span> <span class="token operator">=&gt;</span> <span class="token punctuation">{</span> previous_command <span class="token operator">=</span> <span class="token function">Some</span><span class="token punctuation">(</span>output<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token function">Err</span><span class="token punctuation">(</span>e<span class="token punctuation">)</span> <span class="token operator">=&gt;</span> <span class="token punctuation">{</span> previous_command <span class="token operator">=</span> None<span class="token punctuation">;</span> <span class="token function">eprintln!</span><span class="token punctuation">(</span><span class="token string">"{}"</span><span class="token punctuation">,</span> e<span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token punctuation">}</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token keyword">if</span> <span class="token keyword">let</span> <span class="token function">Some</span><span class="token punctuation">(</span><span class="token keyword">mut</span> final_command<span class="token punctuation">)</span> <span class="token operator">=</span> previous_command <span class="token punctuation">{</span> final_command<span class="token punctuation">.</span><span class="token function">wait</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre> </div> <h2>Conclusion</h2> <p>In less than 100 lines of Rust we’ve created a shell which would be usable for many day to day tasks, but a real shell has many more features. The GNU website has an online manual for the bash shell, including this list of <a href="https://www.gnu.org/software/bash/manual/html_node/Basic-Shell-Features.html#Basic-Shell-Features">shell features</a> which is a great place to start looking into the more advanced functionality. </p> <p>Please note that this was a learning project for me, and in cases where there was a trade-off between simplicity and robustness I most often chose simplicity. </p> <p>This shell is <a href="https://github.com/JoshMcguigan/bubble-shell">available on GitHub</a>. The latest commit as of this writing is <a href="https://github.com/JoshMcguigan/bubble-shell/tree/a6b81d837e4f5e68cf0b72a4d55e95fb08a47640">a47640</a>. Another Rust shell learning project that you may be interested in is <a href="https://github.com/psinghal20/rush">Rush</a>.</p> Sun, 18 Nov 2018 02:59:26 +0000 https://www.joshmcguigan.com/blog/build-your-own-shell-rust/ Rust and SPARK: Software Reliability for Everyone (2017) https://www.electronicdesign.com/industrial/rust-and-spark-software-reliability-everyone https://www.electronicdesign.com/industrial/rust-and-spark-software-reliability-everyone <p><a href="http://electronicdesign.com/datasheet/rust-and-spark-software-reliability-everyone" target="_blank"><em><strong>Download this article in PDF format.</strong></em></a></p> <p>Looking at programming languages, it seems that for a long time, safety or reliability was considered an afterthought, usually covered later in tools such as testing and static analysis, rather than in the language itself. However, over the past few years, it seems there’s been a growing realization that much higher levels of reliability could be achieved for a fraction of the cost if the programming language were designed with reliability in mind. Two names come to mind here: Rust and SPARK.</p> <p>Rust is probably the most notable indication of the growing need for languages with safety in mind. Sponsored by Mozilla and used in the experimental Servo browser, the language was developed as a response to the inability to identify a suitable language for safe software development. Since then, several other projects have adopted this language, mostly in the IT domain.</p> <p>Coming from an A&amp;D perspective, it is intriguing to see that the Ada language wasn’t considered as a suitable starting point. In particular, looking at the Rust mission statement as expressed today in its <a href="https://doc.rust-lang.org/book/README.html" target="_blank">documentation</a>:</p> <p>“Rust is a systems programming language focused on three goals: safety, speed, and concurrency. [...], making it a useful language for a number of use cases other languages aren’t good at: embedding in other languages, programs with specific space and time requirements, and writing low-level code, like device drivers and operating systems. It improves on current languages targeting this space by having a number of compile-time safety checks that produce no runtime overhead, while eliminating all data races. [...]”</p> <p>One can find some troubling similarities with the <a href="http://www.iment.com/maida/computer/requirements/ironman.htm" target="_blank">Ironman requirements</a> from the DoD that led to Ada:</p> <p>“The language shall provide generality only to the extent necessary to satisfy the needs of embedded computer applications. Such applications require real time control, self-diagnostics, input-output to nonstandard peripheral devices, parallel processing, numeric computation, and file processing. [...] The language should aid the design and development of reliable programs. The language shall be designed to avoid error prone features and to maximize automatic detection of programming errors. [...] The language design should aid the production of efficient object programs. [...]”</p> <p>These two extracts, separated by more than 30 years, seem to be targeting the same set of needs. Emphasis is on safety and targeting embedded environments, with efficiency and real-time responsiveness in mind. It’s not as if these are core criteria of all languages. Other languages may put the emphasis on developer productivity, integration with scientific computations, or dynamic capabilities.</p> <p><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/CODEpromo_web.jpg" class="old-inline-image"/></p> <p>And as it turns out, while Rust was being developed, another effort was undertaken by a different community, also trying to raise the bar in terms of safety. The SPARK language is an evolution of the Ada language that aims at providing a technology suitable for automatic analysis and verification.</p> <p>It’s extremely interesting to see that different communities recognized the need to improve the state-of-the-art technology related to software reliability. Rust and SPARK are definitely two initiatives in the forefront of this trend. Let’s look at what each has to say.</p> <p>As a disclaimer, this article is written with an Ada/C++/C#/Java background, and probably more familiarity with SPARK patterns than with Rust’s—though fascinated with what Rust brings to the table.</p> <p><strong>What Brings Them Together</strong></p> <p>To simplify the comparison, we’re going to disregard syntactic differences. SPARK is based on Ada, inspired by Pascal, while Rust is loosely closer to C. That’s that. They’re both imperative languages, compiled directly into object code, and both manage memory directly (i.e., no garbage collection). Each provides abstractions for the usual programming paradigms (procedural and object-oriented). In addition, they both offer advanced concurrency models, providing assurance of properties such as absence of race conditions.</p> <p>Both languages implement a number of static and dynamic checks directly in the language definition. For example, they implement strict-type safety; that is, objects can’t be implicitly converted from one type to the next. The following doesn’t compile in SPARK:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_a.gif" class="old-inline-image"/></strong></p> <p>But will with a conversion:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_b.gif" class="old-inline-image"/></strong></p> <p>Similarly, in Rust:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_c.gif" class="old-inline-image"/></strong></p> <p>will not work, but can be fixed with a conversion:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_d.gif" class="old-inline-image"/></strong></p> <p>Arrays are treated as first-class citizens in both languages. In particular, equality between arrays is an equality between values, and arrays are provided with high-level initialization (aggregates) and syntax to refer to subsets of their elements (slices).</p> <p>Each language enforces strict safety-related rules, but allows the developer to relax them by using explicit “unsafe” operations. In both cases, this helps to make these operations visible (and thus prime suspects in case of problems) while still allowing the right level of flexibility.</p> <p><strong>Rust and SPARK Memory Models</strong></p> <p>SPARK and Rust treat dynamic memory in two ways: the “safe” way and the “unsafe” way. The objectives of both memory models, however, are different. While both consider safety, Rust focuses on memory integrity, providing a model that allows the use of dynamic memory without risk of memory corruption. SPARK focuses on analyzability and provability of program properties and doesn’t permit direct use of dynamic memory or even pointers. Instead, it provides support for the most common cases where pointers are usually needed, specifically for containers and objects of dynamic size, and leaves other uses to the unsafe portions of the code. Both languages forbid memory aliasing in their safe subsets, and neither provides garbage-collection capability.</p> <p>These models are unusual enough to deserve a couple of examples to illustrate them. Let’s start with Rust. To clarify up front the issue of containers (we’ll discuss them more for SPARK), Rust provides a native extensible vector type with checks:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_e.gif" class="old-inline-image"/></strong></p> <p>The above will lead to the value 41 in v[0]. Access to an element that’s not available (for example, v[1]), will lead to a clear error, a panic. There’s no risk of accessing an unallocated piece of memory. Note that the vector doesn’t need to be manually deallocated. Regardless of the underlying implementation (most probably will have some dynamic memory and pointers lying around), it will be freed when reaching the end of its scope.</p> <p>The Rust memory model shines when looking at how it handles arbitrary references and dynamic memory. The core idea is that a piece of data is always owned by a variable (a binding, in Rust lingo) and only one. This protects against two problems, the first of which is aliasing. A common problem arises when manipulation of a variable modifies another variable at the same time because they point to the same piece of data. A typical example of this kind of problem is double deallocation, although aliasing can also cause other subtle bugs. This situation can’t arise in Rust.</p> <p>The other common problem relates to memory leaks or, more generally, the overall deallocation policy. Because a piece of data is always owned by a binding, when that binding goes out of scope, we know it’ll never be accessed by anything else (there are no other references or aliases). It can thus be safely freed.</p> <p>To fully appreciate the consequences of this, let’s look at a short example:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_f.gif" class="old-inline-image"/></strong></p> <p>v2 is a reference (or pointer) to v. Because of the assignment, v2 borrows the value of v and owns it for the rest of the scope. As a result, it’s no longer possible to directly modify v. This is checked statically, and the compiler will refuse to compile the above code. Borrowing can be done on a smaller scope as well. For example:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_g.gif" class="old-inline-image"/></strong></p> <p>When the above code works correctly, v2 releases the ownership of v at the end of the scope. This is particularly useful in parameter passing.</p> <p>With SPARK, large chunks of memory have to be managed through dedicated containers. You can use either real-time embedded containers—typically of fixed capacity—or unbounded containers. The key idea is that containers not only provide safe access to a pool of objects from a memory perspective, but they allow ways to reason about them and statically verify certain run-time properties. Let’s take one example, one with a fixed capacity container:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_h.gif" class="old-inline-image"/></strong></p> <p>The example creates a vector with a maximum capacity of 10 elements; then adds two elements to it (10 and 20). Subsequently, we attempt to replace the element at index 3. By default, vectors are indexed starting at 1 in SPARK (although other indexing is possible). Therefore, after adding two elements, only items 1 and 2 are present. The normal behavior of this code would be to raise an exception at run-time, which it will.</p> <p>However, the SPARK formal prover is able to verify that the condition “the index of the element must exist before a replacement” cannot be verified, and will actually issue an error statically. In this example, the mistake is quite obvious to the careful reader.</p> <p>SPARK unleashes its full power when working on larger pieces of code, making sure that certain categories of errors don’t happen, such as buffer overflow or division by zero. In other words, as soon as there’s a possibility of failure, SPARK will detect it and allow you to fix it prior to testing, giving 100% confidence that any places it doesn’t complain about are free from these kind of errors.</p> <p><strong>A Word on Functional Safety</strong></p> <p>The previous section is interesting because it highlights a fundamental difference between Rust and Ada in terms of how they consider safety. Rust, like many other languages for the IT world, such as Java, C# and C++11, look at software safety through the prism of memory corruption.</p> <p>The goal of Rust’s memory model, which was the same goal of the Java/C# garbage collectors or C++11 lvalue references, is to provide a mechanism to reduce or avoid memory corruption. This makes a lot of sense, as these languages are used extensively on desktop applications and servers. In such applications, a lot of memory is being employed dynamically and memory corruption is the number one offender in terms of software vulnerability.</p> <p>On the other hand, SPARK originated from the Ada language and has the roots of its history in high-integrity embedded software. Although memory corruption also plays a role there, the objective of safety is to verify that a given implementation is correct with regard to a given specification.</p> <p>Looking at the above example, we can consider that “the index exists before calling Replace_Element” is part of the specification of Replace_Element and should be respected by any caller. SPARK allows one to specify custom requirements on user subprograms in the forms of preconditions and postconditions. For example, one could write:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_i.gif" class="old-inline-image"/></strong></p> <p>This states that the procedure Fold_Lasts expects a vector of at least two elements and will update the vector so that the length of the new value is one less than the length of the old value. The implementation of the procedure will therefore verify that, given the precondition, the postcondition holds. For example:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_j.gif" class="old-inline-image"/></strong></p> <p>This subprogram accesses the last two elements of the vector (which is possible and correct, as we established there are at least two elements) and removes the last one (hence ensuring the length of the returned object is equal to the length of the old object minus one).</p> <p>This can then be used in an actual piece of code:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_k.gif" class="old-inline-image"/></strong></p> <p>In the above example, the second Fold_Last will issue a proof error, as it should: We’re adding two elements to the vector and folding one leaves only one, which is inconsistent and can be detected statically.</p> <p>It’s useful to note at this stage that these functional properties, or contracts, have hybrid static and dynamic semantics. In other words, they can be used by the prover, but they may also generate assertions in the final executables to test their validity at run-time. This is particularly useful in cases of contracts too complex to be proven, or while developing contracts to allow for debugging of their behavior at run-time. These contracts can also be used in the non-SPARK parts of the application, which would typically be written in Ada or in C.</p> <p><strong>Object Orientation</strong></p> <p>Both Rust and SPARK provide support for object orientation. Beyond syntactic oddities inherited from Ada 95, the SPARK object-orientation model is relatively close to the usual paradigms implemented with C++ or Java. The interesting aspect of the SPARK OO model is that it supports Liskov substitutability analysis. That is, verifying that a class is substitutable by its subclasses in the case of dynamic dispatch. In other words, the child classes must be consistent with their parent.</p> <p>Let’s take a simple example—a class hierarchy that manipulates images. In this design, there’s a root class that can handle an RGB image. We decide to derive from this class to create a specialization that handles grayscale images. Here’s how it might look in SPARK:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_l.gif" class="old-inline-image"/></strong></p> <p>This may initially look like a decent OO design However, looking further at the Set method, the “Image” parameter requires a pixel to be an array of three float values, while “Image_Gray” requires a pixel to be an array of one element. On a typical dispatching call from “Image,” the user doesn’t know if Image or Image_Gray is going to be the actual type, and e doesn’t know if the length of a pixel is 1 or 3.</p> <p>The above is typical of awkward OO designs. SPARK will verify consistency of behaviors declared with methods (expressed in the form of contracts; i.e., preconditions and postconditions) and flag any error. In particular, in the case above, it will complain on the inconsistency of the Set method. One of the general rules is that all inputs accepted by the parent class must also be accepted by the child (and possibly more). For instance, it would be okay for the Set method of “Image_Gray” to accept only a pixel that has one or more elements, which can be expressed as follows in its precondition: P’Length &gt;= 1.</p> <p>As a side note, SPARK additionally provides static polymorphism, also known as templates or generics, in a model very close to that of C++. The same goes for Rust.</p> <p>Rust doesn’t provide any means to specify behavior or verify class consistency. However, it offers an OO model that’s much more modern through the notion of “traits.” In Rust, declaring a type Image like the above can’t be done directly in a type, but rather in a trait (a bit like an interface in Java and Ada if you will):</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_m.gif" class="old-inline-image"/></strong></p> <p>There’s no notion of deriving a type in Rust, so to create an architecture comparable to the above, we would need to implement two different types. Of course, nothing prevents us from having some common type that holds services common to both Image and Image_Gray. But, nonetheless, there are essentially two distinct types implementing the same trait. Such code would look like:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_n.gif" class="old-inline-image"/></strong></p> <p>As you can see from the example, a structure isn’t inherently derived from a trait, as it would be in the case of a Java/Ada interface derivation. A given trait is implemented for a given type in a separate block. This gives a nice solution to the known problem of the tyranny of the dominant decomposition that forces a hierarchy of types to be decomposed on one axis only. This also allows third-party trait implementation.</p> <p>I can use a library in Rust and decide to implement a trait for a type even if that hasn’t been initially planned by the implementer (in particular if the trait is mine!). This is also a nice answer to the problem that was at the origin of aspect-based programming—the separation of concerns—without drifting into overly permissive models such as AspectJ.</p> <p><strong>What Sets Them Apart</strong></p> <p>It would take too much time to go over each feature one by one and highlight every single axis that can be used to compare these two languages. We talked a lot about what brings them together and how they look at different angles of similar problems coming up with different solutions. There are, however, interesting elements that also make them very different.</p> <p>The most visible one relates to specification. Rust doesn’t provide a clear separation between a specification file and an implementation file. As with C or Java, this can be done through language features (you can decide to have traits in a “specification-only” file), but this is left to the discretion of the developer. SPARK enforces a clear distinction between these two notions and provides language-level verification of consistency between specification and implementation.</p> <p>Another aspect is the tradeoff between writability and readability. In Rust, when typing can be statically inferred, it becomes optional. So I can write:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_o.gif" class="old-inline-image"/></strong></p> <p>The compiler will know that x is an i32. This becomes particularly useful when the type is actually a complex generic instance, reducing the amount of text to be written. On the other hand, the downside is that it may be more difficult to figure out what the type actually is when reading the code (to the extent that, to date, the Rust documentation mentions types in comments next to variables so that newcomers understand what they refer to).</p> <p>SPARK takes the opposite stance. It limits to the absolute minimum what’s to be inferred by the compiler, forcing the developer to specify every type precisely at the cost of sometimes pedantic constraints.</p> <p>Both languages provide strong static typing features. However, in Rust, native types are just machine types, such as i32, f64, etc. In Ada, a type is a semantic entity associated with constraints that may or may not be defined, such as the range of values or memory size. Moreover, a type is associated with semantics that make it clear that even if it comes down to the same implementation, it should be considered as a different entity. For example:</p> <p><strong><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/0117_CTE_Adacore_Code_list_p.gif" class="old-inline-image"/></strong></p> <p>The compiler will know that V1 assignment into V2 doesn’t make sense because they’re different types. Indeed, miles has nothing to do with percentage, even though they may both end up being implemented as 32-bit-machine floating point. Moreover, extra constraints—here, a range of values—can be verified either statically (ensuring that a value can’t be outside of the range) or dynamically via testing.</p> <p>Rust provides an extremely powerful and structured macro language that allows one to effectively create expansions of pieces of code based on patterns. There’s nothing similar with SPARK. Rust also provides a so-called “matching” control structure much more powerful than the SPARK equivalent, closer to C/Java switch statements.</p> <p>Differences do exist in the error recovery model: SPARK has some limited support for exceptions (complete support if you go to the unsafe code mode), while Rust uses a much more constraining notion of “panic” when something is unexpected. Rust supports lambdas and closures, which don’t really have an equivalent in SPARK.</p> <p>And the list goes on. Determining which set of features is the best is a tradeoff that depends on many different things external to the language, but are instead specific to a given project, team, market, etc. But it’s useful to have some understanding of these differences in order to make an educated choice.</p> <p><strong>A Word on Ecosystems</strong></p> <p>Choosing to use one language rather than the other isn’t tied solely to its intrinsic technical merits. A language is the basis of a software ecosystem that includes the availability of additional tools, resources, and support. In this regard, Rust and SPARK are both very similar and very different. They’re similar in the sense that they each rely on an open-source community, though at different levels.</p> <p>The Rust compiler is LLVM-based, supported by the Mozilla foundation and a large community of hobbyists. The SPARK compiler is GCC-based, and while an open-source community does exist around it, it’s more restricted. The compiler element of the tool is mostly maintained by a software vendor called AdaCore (of which I happen to be part of) and the formal proof engines underneath are maintained and developed in collaboration with various universities.</p> <p>Both technologies are available for a large set of targets, in particular native and embedded environments (specifically, the obvious: ARM). They’re two open-source strategies, if you will, but shaped very differently.</p> <p>It would be wrong, however, to consider this picture to be static. With Rust being deployed on a growing number of projects, there’s no reason why commercial entities would not start to provide official support on it. There’s also no reason why professional tools wouldn’t appear for static analysis, code coverage, requirement traceability, code quality, and so on.</p> <p>On the SPARK side, these tools exist from various vendors, inherited from the Ada history. However, AdaCore has a history of scarce interactions with the communities, limiting its contributions to the production of a few GPL binaries and commits of its compiler sources to the FSF trunk. That facade is also breaking down, with lots of development now done on GitHub and several other community-friendly initiatives being started.</p> <p>As to industrial usage, the first users of Rust are coming from the IT world (Mozilla needs are definitely IT-related), while SPARK users come from high-integrity embedded A&amp;D applications. Both technologies target embedded application in the more general sense, thinking about things such as medical devices, industrial automation, automotive, and, of course, IoT. So, in a way, they’re two options to address the same needs, perhaps at different levels, perhaps in competition, but perhaps also in symbiosis.</p> <p>Last but not least, the question of available developers must be considered. There won’t be many people who include Rust in their résumé when getting out of university this year. There won’t be lots of people who include SPARK either. But that’s not really an issue, is it? The language shouldn’t be an obstacle for a decent embedded software engineer, and the internet is filled with tutorials and training resources for both languages.</p> <p>While this message is sometimes difficult to get through to management, the real question is more about finding these decent embedded software engineers than finding people who happen to claim knowledge in a particular technology. That, however, is an issue on its own, whether you develop embedded application in C, Rust, SPARK, C++, Java, or anything else.</p> <p><strong>Conclusion</strong></p> <p>There’s little doubt that Rust has a bright future ahead of it. It provides a unique approach to software reliability and safety, and is being adopted by a growing number of projects. The language seems to address reliability concerns in a much better way than C or C++, without the overhead of VM-based languages like C# or Java. It’s a young language, and some features or tools may still be missing, but there’s no reason why that shouldn't be overcome in the future.</p> <p>There’s little doubt SPARK has a bright future ahead of it, too. It provides a unique approach to software reliability and safety, and is being adopted by a growing number of projects. The language leverages its Ada foundation to target a large set of use cases in the embedded domain. SPARK is also a relatively young language (at least in its current form), and the language is rapidly evolving. At the time of this writing, for example, new prover capabilities are being developed and new containers are being written.</p> <p>It seems that the two languages can coexist. SPARK remains very well-suited for safety-critical embedded applications, while Rust looks like a good fit for the IT domain. Generic embedded applications may lean on one side or the other, depending on various factors. Both languages bring interesting ideas to the table—and both suffer from shortcomings. Perhaps there’s room for cross-fertilization.</p> <p>Above all, it’s very nice to see new languages considering safety and reliability at their core. The market can only be enriched by a larger offer of technologies, which will undoubtedly push better practices in industrial settings. As a matter of fact, I’m not sure if I care so much whether you’re using SPARK, Rust, or a super-constrained MISRA-C—as long as my car, my money, and my blood pressure are handled with safe and robust pieces of software!</p> <p><a href="http://sourceesb.com" target="_blank"><img alt="" src="https://www.electronicdesign.com/sites/electronicdesign.com/files/uploads/2016/10/11/SourceESB_Lookin_For_Parts_Banner_0.jpg" class="old-inline-image"/></a></p> Sun, 18 Nov 2018 11:22:08 +0000 https://www.electronicdesign.com/industrial/rust-and-spark-software-reliability-everyone The World’s Loudest Sound (2014) https://kottke.org/14/10/the-worlds-loudest-sound https://kottke.org/14/10/the-worlds-loudest-sound <p><img src="https://kottke.org/plus/misc/images/krakatoa-sound-01.jpg" width="1000" height="564" border="0" alt="Krakatoa Sound 01" /></p> <p>The sound made by the Krakatoa volcanic eruption in 1883 was so loud it ruptured eardrums of people 40 miles away, travelled around the world four times, and <a href="http://nautil.us/blog/the-sound-so-loud-that-it-circled-the-earth-four-times">was clearly heard 3,000 miles away</a>.</p> <blockquote readability="14"><p>Think, for a moment, just how crazy this is. If you’re in Boston and someone tells you that they heard a sound coming from New York City, you’re probably going to give them a funny look. But Boston is a mere 200 miles from New York. What we’re talking about here is like being in Boston and clearly hearing a noise coming from Dublin, Ireland. Travelling at the speed of sound (766 miles or 1,233 kilometers per hour), it takes a noise about 4 hours to cover that distance. This is the most distant sound that has ever been heard in recorded history.</p></blockquote> <p>A much much smaller eruption occurred recently in Papua New Guinea. From the video, you can get a tiny sense of the sonic damage unleashed by Krakatoa:</p> <p>Holy smoking Toledos indeed. <a href="https://www.reddit.com/r/space/comments/2h9y9g#ckr6mev">On Reddit</a>, a user details how loud a Saturn V rocket is and what the effects would be at different distances. At very close range, the sound from the Saturn V measures an incredible 220 db, loud enough to melt concrete <em>just from the sound</em>.</p> <blockquote readability="17"><p>At 500 meters, 155 db you would experience painful, violent shaking in your entire body, you would feel compressed, as though deep underwater. Your vision would blur, breathing would be very difficult, your eardrums are obviously a lost cause, even with advanced active noise cancelling protection you could experience permanent damage. This is the sort of sound level aircraft mechanics sometimes experience for short periods of time. Almost twice as “loud” as putting your ear up to the exhaust of a formula 1 car. The air temperature would drop significantly, perhaps 10-25 degrees F, becoming suddenly cold because of the air being so violently stretched and moved.</p></blockquote> <p>Even at three miles away, the sound is loud enough to cause permanent hearing damage. But that’s <em>nothing</em> compared to the Krakatoa sound. The Saturn V sound is ~170 db at 100 meters away while the Krakatoa explosion was that loud 100 miles away! What happens at 170 db?</p> <blockquote readability="13"><p>…you would be unable to breathe or likely see at all from the sound pressure, glass would shatter, fog would be generated as the water in the air dropped out of suspension in the pressure waves, your house at this distance would have a roughly 50% chance of being torn apart from sound pressure alone. Military stun grenades reach this volume for a split second… if they are placed up to your face. Survival chance from sound alone, minimal, you would certainly experience permanent deafness but probably also organ damage.</p></blockquote> <p>The word “loud” is inadequate to describe how loud that is. (thx, david)</p> <p><strong>Update:</strong> <a href="http://fivethirtyeight.com/features/the-loudest-sound-in-the-world-would-kill-you-on-the-spot/">Sperm whales are loud!</a> 200 decibels in water and 174 in air.</p> <blockquote readability="15"><p>But the whale is not really as loud as the rocket, she told me. Because water is denser than air, sound in water is measured on a different decibel scale. In air, the sperm whale would still be extremely loud, but significantly less so - 174 decibels. That’s roughly equivalent to the decibel levels measured at the closest barometer, 100 miles away from the Krakatoa eruption, and is loud enough to rupture people’s ear drums. Suffice to say, you probably don’t want to spend a lot of time swimming with the sperm whales.</p></blockquote> <p>More about...</p> Sun, 18 Nov 2018 13:51:45 +0000 https://kottke.org/14/10/the-worlds-loudest-sound Why affordable housing is scarce in progressive cities https://www.curbed.com/2018/11/16/18098432/rent-housing-affordable-generation-priced-out https://www.curbed.com/2018/11/16/18098432/rent-housing-affordable-generation-priced-out <p id="zEtH18">Plenty of factors can explain the crisis of housing affordability plaguing U.S. cities—a shortage of new construction, a lack of tenant protection, greedy developers and speculators, or a lack of upzoning. But according to San Francisco-based housing activist Randy Shaw, author of <a href="https://www.ucpress.edu/book/9780520299122/generation-priced-out">Generation Priced Out: Who Gets to Live in the New Urban America</a>, one of the true challenges is the entrenched power and privilege of an older generation of homeowners. </p> <p id="GYy5ee">“No progressive city posts “Priced Out: Only the Affluent Allowed” signs in it neighborhoods,” he writes in the book’s introduction.“But that’s what’s happened.”</p> <p id="XbU1FZ">Shaw, who runs the Tenderloin Housing Clinic in San Francisco, has a first-hand understanding of the causes of today’s crisis, having witnessed his city’s transformation over the last few decades into a poster child for extreme housing costs. He also knows many of the traditional solutions, having pushed for the tenant protections that make San Francisco a progressive beacon for renter’s rights.</p> <p id="Uz1BVR">But as he bounces from city to city in the book, showing how similar problems have manifested themselves in Austin, Los Angeles, Seattle, Denver, and elsewhere, it’s clear there’s a generational battle being fought. The entrenched power of homeowners to restrict the housing supply, even as the nation sees <a href="https://www.curbed.com/2018/11/9/18075392/california-real-estate-home-cost-million-dollar">more expensive housing </a>and increased wealth segregation across the country, shocked even a seasoned San Francisco activist. </p> <p id="sYA1yH">“I sometimes feel akin to an emergency room physician as the victims of rising unaffordability and civil strife come to our office seeking help,” Shaw wrote, describing his experience in how home city. What shocked him, as someone who witnessed how that shift can change a city, is how widespread it has become. </p> <h4 id="vGfezY"><strong>A challenge so-called progressive cities aren’t meeting</strong></h4> <p id="PL7BPT">Shaw says the housing crisis has taken on a different tone over the last few years because its impact is so widespread. What’s changed in many cities is how the middle class is being affected; now, low-income households and young adults aren’t the only ones facing long commutes or makeshift living situations due to a dearth of accessible and affordable options near jobs. </p> <p id="GGOFPZ">“If you’d spoken to me in 2011 and asked how Seattle is doing, I may have said it seems pretty affordable,” Shaw told Curbed. “Now, cities like Seattle, Portland, Minneapolis, and Austin all face similar problems as San Francisco. The nation perceived this as a problem that was only in certain cities, and now it’s spread.”</p> <p id="yo5jSq">As soon as the housing market began recovering in earnest after the Great Recession, and prices really started rising in 2011 and 2012, says Shaw, cities that had restricted growth or favored single-family homes faced a reckoning. <em>Generation Priced Out </em>does a great job of telling the personal stories of families and tenants who can barely hang on; what’s even more damning is imaging all the other stories of those who missed opportunity because they couldn’t afford to move to big cities. </p> <p id="L3zN6u">The fealty to policies that have restricted supply and raise prices challenge the self-image of many deep blue, Democratic cities; how can you be progressive and welcoming to all if low income, people of color, students, and the middle class can’t afford a place to live? </p> <p id="1n6olR">“So many cities are seeing long-standing pro-ownership policies come face to face with growth,” he says. “It’s challenging their progressive cred. If you drive a Prius and recycle, yet don’t allow apartments to be built, and then force people to drive 50 miles each way to work and cause all that pollution, you can’t say ‘out of sight, out of mind.’”</p> <figure class="e-image"><span class="e-image__inner"> <span class="e-image__image " data-original="https://cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg"> <picture class="c-picture" data-cid="site/picture_element-1542554282_3720_13003" data-cdata="{&quot;asset_id&quot;:13448006,&quot;ratio&quot;:&quot;*&quot;}"><source srcset="https://cdn.vox-cdn.com/thumbor/xsuW7nC57OTn7wXjJRENlkREpyk=/0x0:3992x2992/320x0/filters:focal(0x0:3992x2992):format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 320w, https://cdn.vox-cdn.com/thumbor/Z86rc4tFaOzJOCOfkV29E-YswKA=/0x0:3992x2992/520x0/filters:focal(0x0:3992x2992):format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 520w, https://cdn.vox-cdn.com/thumbor/WzWA5-vSptu30O4J461DTF6JfmA=/0x0:3992x2992/720x0/filters:focal(0x0:3992x2992):format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 720w, https://cdn.vox-cdn.com/thumbor/I4WY-ClKD-3wYbhft-kdYs7ZIqU=/0x0:3992x2992/920x0/filters:focal(0x0:3992x2992):format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 920w, https://cdn.vox-cdn.com/thumbor/jV0ods7u7yN_Axe9amftT8_4_1c=/0x0:3992x2992/1120x0/filters:focal(0x0:3992x2992):format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 1120w, https://cdn.vox-cdn.com/thumbor/YjW8lKtPppzJXqjKy8H7KJuOgAU=/0x0:3992x2992/1320x0/filters:focal(0x0:3992x2992):format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 1320w, https://cdn.vox-cdn.com/thumbor/ynUSFwa7hZTJPfwxTDq6c5xWh3E=/0x0:3992x2992/1520x0/filters:focal(0x0:3992x2992):format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 1520w, https://cdn.vox-cdn.com/thumbor/e6u3MYNg-Evude7T17f8MeC4Rj4=/0x0:3992x2992/1720x0/filters:focal(0x0:3992x2992):format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 1720w, https://cdn.vox-cdn.com/thumbor/h0eSn8jwfHQWgzy4sugy80JQAL8=/0x0:3992x2992/1920x0/filters:focal(0x0:3992x2992):format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 1920w" sizes="(min-width: 1221px) 846px, (min-width: 880px) calc(100vw - 334px), 100vw" type="image/webp"><img srcset="https://cdn.vox-cdn.com/thumbor/AzZE6ecBCDvKhNhiK9x4r1LDCI8=/0x0:3992x2992/320x0/filters:focal(0x0:3992x2992):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 320w, https://cdn.vox-cdn.com/thumbor/AFGw0iOthrjIEWnIOHalxDZnlNg=/0x0:3992x2992/520x0/filters:focal(0x0:3992x2992):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 520w, https://cdn.vox-cdn.com/thumbor/pJiymSxcHL2qc2eWNlTypJ-38Vk=/0x0:3992x2992/720x0/filters:focal(0x0:3992x2992):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 720w, https://cdn.vox-cdn.com/thumbor/FC1evStfyGCF3NrUoBe-LD7AsSA=/0x0:3992x2992/920x0/filters:focal(0x0:3992x2992):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 920w, https://cdn.vox-cdn.com/thumbor/5qjw8pAemTHaJ85pnXh2a2IJcZU=/0x0:3992x2992/1120x0/filters:focal(0x0:3992x2992):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 1120w, https://cdn.vox-cdn.com/thumbor/dbIl1ZcvUX6RRE8lrpK07UD5AJw=/0x0:3992x2992/1320x0/filters:focal(0x0:3992x2992):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 1320w, https://cdn.vox-cdn.com/thumbor/yaDiE0_2qCEH11ZIWgvD14Pn7WU=/0x0:3992x2992/1520x0/filters:focal(0x0:3992x2992):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 1520w, https://cdn.vox-cdn.com/thumbor/BD0ggMdXLt1OuJbKMeQrXnTcggE=/0x0:3992x2992/1720x0/filters:focal(0x0:3992x2992):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 1720w, https://cdn.vox-cdn.com/thumbor/mJhnU58vrhyfVl5j9NazUu0-Uts=/0x0:3992x2992/1920x0/filters:focal(0x0:3992x2992):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg 1920w" sizes="(min-width: 1221px) 846px, (min-width: 880px) calc(100vw - 334px), 100vw" alt="" data-upload-width="3992" src="https://cdn.vox-cdn.com/thumbor/HPujBrj7LNDUFsUiVb1sq9TzrT4=/0x0:3992x2992/1200x0/filters:focal(0x0:3992x2992):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/13448006/shutterstock_1033232824.jpg"/></source></picture></span> </span> <span class="e-image__meta"> <figcaption>Venice, California</figcaption><cite>Shutterstock</cite> </span> </figure><h4 id="H9DX32"><strong>Los Angeles as a cautionary tale</strong></h4> <p id="4Fi4v7">Shaw works in San Francisco, but he grew up in the Westside of Los Angeles, so his descriptions of how many sections of that city went from working class to unaffordable offer some additional personal and historical perspective. </p> <p id="TY1Vtd">Like San Francisco, the city’s downtown development boom, economic growth, and increased population all occurred without the city building much housing. During the recent boom, working-class neighborhoods like Boyle Heights and Highland Park were targeted by speculators, and working class renters, who lacked tenant and housing protections, found themselves pushed out. </p> <p id="Pd7mXe">Like many other cities profiled in the book, generations of bad policy caught up during the last decade’s downtown building boom. Between 1980 to 2010, Los Angeles’s population growth exceeded housing growth by 42 percent. When demand and prices suddenly shifted, a city ripe for affordability issues faced disaster. </p> <p id="66ab2B">What Shaw found most eye-opening during writing and research was the widespread, entrenched power of homeowner groups. Between neighborhood groups and councils that held power over zoning and land-use decisions and the spread of NIMBYism, these older groups have often controlled the dialogue around adding housing supply, all while reaping an oversized amount of the gains from rising land values. As author Mike Davis once wrote, this “Sunset Bolshevism,” the collective political power of homeowners, has proven itself to be California’s strongest social movement. </p> <p id="UnGWkA">“The most surprising thing was how widespread these homeowner groups was,” Shaw says. “The most powerful voters are homeowner voters. </p> <p id="Oucq4a">Shaw highlighted Venice, the once-Bohemian beach community in Los Angeles, as a prime example. From 1960 to 2010, as the LA metro area added two million housing units, Venice added virtually none. Venice actually lost 700 housing units between 2000 and 2015, and the population decreased. It’s created incredible homeowner value, as well as an acute affordability and homelessness crisis. </p> <h4 id="AWAuvo"><strong>Cities can change the narrative </strong></h4> <p id="ESWfy6">While so many cities lament a steady shift toward a San Francisco-style housing situation, Shaw says he’s seeing hope, especially at a local level. Cities such as Seattle and Denver, while far from solving the issue, have shown the benefits of pro-growth policies. Portland, San Francisco, and Austin have also passed housing bonds. </p> <p id="yNhatH">There have been flickers of hope at the national level lately. Senators including Kamala Harris, Cory Booker, and Elizabeth Warren have all issued <a href="https://www.curbed.com/2018/7/20/17595698/rent-relief-act-tax-credit-kamala-harris">housing policy platforms</a>. Shaw agrees that housing has been a larger national issue than its been in decades. But the solution, he says, can be found at the local level today. </p> <p id="wLy0iu">“There’s been a wakeup call around America,” he says. “This book is based on the idea that cities can make a change and make this work.” </p> Sun, 18 Nov 2018 17:17:58 +0000 https://www.curbed.com/2018/11/16/18098432/rent-housing-affordable-generation-priced-out Overlooked No More: Pandita Ramabai, Indian Scholar, Feminist and Educator https://www.nytimes.com/2018/11/14/obituaries/pandita-ramabai-overlooked.html https://www.nytimes.com/2018/11/14/obituaries/pandita-ramabai-overlooked.html <div readability="47.125689519307"><div class="css-4w7y5l" readability="39.684791174153"><p class="css-1ebnwsw e2kc3sl0"><em class="css-2fg4z9 ehxkw330">Since 1851, obituaries in The New York Times have been dominated by white men. With </em><a class="css-1g7m0tk" href="https://nytimes.com/overlooked" title=""><em class="css-2fg4z9 ehxkw330">Overlooked</em></a><em class="css-2fg4z9 ehxkw330">, we’re adding the stories of remarkable people whose deaths went unreported in The Times.</em></p><p class="css-1ebnwsw e2kc3sl0">Pandita Ramabai Saraswati, a scholar, feminist and educator, broke nearly every rule and tradition that confined the life of an upper-caste Hindu woman in 19th-century India.</p><p class="css-1ebnwsw e2kc3sl0">She was the rare woman who had learned Sanskrit, the ancient Hindu liturgical language reserved for Brahmin men; the rare Brahmin to marry out of caste; the rare widow who remained in public view, defying customs; and the rare Indian woman to decide, on her own, to convert to Christianity.</p><p class="css-1ebnwsw e2kc3sl0">At a time when<span class="css-8l6xbc e1tp43kk0"> </span>women were expected<span class="css-8l6xbc e1tp43kk0"> </span>to be little seen and never heard, Ramabai was an outspoken advocate of women’s education and participation in public affairs. She traveled across India giving lectures on women’s rights. She studied in Britain and the United States, gave lectures in Japan and Australia, and taught Sanskrit as well as her mother tongue, Marathi.</p><p class="css-1ebnwsw e2kc3sl0">Most remarkably for her time, Ramabai charted these paths as a single woman and mother;<span class="css-8l6xbc e1tp43kk0"> </span>the few Indian women who were active in social reform then did so only with the encouragement — or, at least, the permission — of their husbands.</p></div><aside class="css-1m2ozyi"><span/></aside></div><div readability="31.355555555556"><div class="css-4w7y5l" readability="11.988888888889"><p class="css-1ebnwsw e2kc3sl0">“Her background, her life choices, her personality and her career catapulted her into the public gaze, making her the most controversial Indian woman of her times,” <a class="css-1g7m0tk" href="http://zubaanbooks.com/bookauthor/uma-chakravarti/" title="" rel="noopener noreferrer" target="_blank">Prof. Uma Chakravarti</a> wrote in “Rewriting History: The Life and Times of Pandita Ramabai” (1998).</p></div><aside class="css-1m2ozyi"><span/></aside></div><div readability="36.044362292052"><div class="css-4w7y5l" readability="21.256931608133"><p class="css-1ebnwsw e2kc3sl0">She was born Rama Dongre on April 23, 1858,<span class="css-8l6xbc e1tp43kk0"> </span>into a Brahmin family. Brahmins, who were generally priests and scholars, were at the top of the caste system that governed Hindu society.</p><p class="css-1ebnwsw e2kc3sl0">Though it was standard for learning to be limited to men and for women to be married off at a young age, Rama’s father, Anant Shastri Dongre, kept her at home and taught her Sanskrit.</p><p class="css-1ebnwsw e2kc3sl0">This “enabled her to escape the rigid gender code,” said Meera Kosambi, an Indian academic who wrote a biography of Ramabai and <a class="css-1g7m0tk" href="https://books.google.com/books?isbn=1317334019" title="" rel="noopener noreferrer" target="_blank">translated many of her works from Marathi</a>.</p></div><aside class="css-1m2ozyi"><span/></aside></div><div readability="50"><div class="css-4w7y5l" readability="45"><p class="css-1ebnwsw e2kc3sl0">Ramabai was only 16 when she lost both her parents to famine. Her father’s “last loving command to me,” she wrote in a letter quoted by Kosambi, “was to live an honourable life,<span class="css-8l6xbc e1tp43kk0"> </span>if I lived at all, and serve God all my life.”</p><p class="css-1ebnwsw e2kc3sl0">She and her older brother made a livelihood reciting Sanskrit scripture. They moved to Calcutta in 1878, where word spread of Rama’s mastery of the Hindu holy books. Sanskrit scholars at the University of Calcutta gave her the titles Pandita (scholar) and Saraswati (for the goddess of learning), and she became involved in social reform and education circles in Bengal. Bai was added to her first name as a term of respect.</p><p class="css-1ebnwsw e2kc3sl0">After her brother’s death in 1880, she married Bipin Behari Medhavi, a lawyer who was of a lower caste.</p><p class="css-1ebnwsw e2kc3sl0">Rama was only 23 when Medhavi died of an illness, leaving her alone with their 1-year-old daughter, Manorama. She moved to Poona (now Pune), in western India, and formed the Arya Women’s Association to promote education and empowerment.</p><p class="css-1ebnwsw e2kc3sl0">She left for England with her daughter in 1883 to study medicine but was told she could not become a doctor because of her increasing deafness. Instead she enrolled in a teaching program at the Cheltenham Ladies’ College and taught Marathi and Sanskrit.</p><p class="css-1ebnwsw e2kc3sl0">From there she straddled two worlds. She converted to Christianity and took the name Mary, angering her Indian supporters. At the same time, she clashed with church officials, having<span class="css-8l6xbc e1tp43kk0"> </span>chafed at their colonial attitudes, and she continued to wear Indian dress and remained a vegetarian.</p></div><aside class="css-1m2ozyi"><span/></aside></div><div readability="27.247386759582"><div class="css-4w7y5l" readability="10.418118466899"><p class="css-1ebnwsw e2kc3sl0">In 1886, she sailed to the United States to attend the graduation of a relative, Anandibai Joshi, the <a class="css-1g7m0tk" href="https://www.indiatoday.in/education-today/gk-current-affairs/story/anandibai-joshi-311180-2016-03-31" title="" rel="noopener noreferrer" target="_blank">first Indian woman known to have completed medical </a><a class="css-1g7m0tk" href="https://www.indiatoday.in/education-today/gk-current-affairs/story/anandibai-joshi-311180-2016-03-31" title="" rel="noopener noreferrer" target="_blank">college</a>. Ramabai was touched deeply by the optimism of American girls, and she viewed the United States as a model for a modern India.</p></div><aside class="css-1m2ozyi"><span/></aside></div><div readability="50.97796883396"><div class="css-4w7y5l" readability="49.05427189683"><p class="css-1ebnwsw e2kc3sl0">Her most important published work, “<a class="css-1g7m0tk" href="http://digital.library.upenn.edu/women/ramabai/woman/woman.html" title="" rel="noopener noreferrer" target="_blank">The High Caste Hindu Woman</a>,” was written in English in the United States in 1887, when she was 29. It focused on the plight of Hindu widows — she called widowhood “the worst and most dreaded period of a high-caste woman’s life.”</p><p class="css-1ebnwsw e2kc3sl0">Brahmin customs prohibited widows from remarrying. Considered cursed, they<span class="css-8l6xbc e1tp43kk0"> </span>were required to shave their heads, wear drab, coarse clothes and subsist on meager food. Widows were also subject to physical and sexual abuse. The common practice of child marriage meant that some widows were still<span class="css-8l6xbc e1tp43kk0"> </span>girls when they were doomed to a lifetime on the margins.</p><p class="css-1ebnwsw e2kc3sl0">Readers were moved by Ramabai’s <a class="css-1g7m0tk" href="https://timesmachine.nytimes.com/timesmachine/1901/03/23/101071375.pdf" title="">account of life in India</a>, and women’s groups formed the American Ramabai Association, with dozens of chapters to support Ramabai financially in her mission.</p><p class="css-1ebnwsw e2kc3sl0">“Here was a woman who circumnavigated the globe in the 19th century, built community in foreign countries and overcame the visceral challenges of diet, dress and language,” said <a class="css-1g7m0tk" href="https://history.artsci.wustl.edu/chandra" title="" rel="noopener noreferrer" target="_blank">Shefali Chandra</a>, associate professor of history at Washington University in St. Louis.</p><p class="css-1ebnwsw e2kc3sl0">Ramabai wrote wry, thoughtful accounts of her travels that were well received in India. She described how an American host had been<span class="css-8l6xbc e1tp43kk0"> </span>horrified to see her barefoot in the house, how Europeans had avoided her small party of Indians on a ship, and how she had stood out when she pulled woolen sweaters over her Indian clothing to stay warm.</p><p class="css-1ebnwsw e2kc3sl0">Using proceeds from her book and lectures, she raised funds to open the Sharada Sadan (Home of Learning) center in 1889 in Bombay, offering widowed girls a refuge where they could study and learn skills like gardening, carpentry and sewing. The shelter grew, at one point serving more than 700 girls and women. Many became teachers and nurses while others stayed, running a dairy farm and their own printing press. The home is still <a class="css-1g7m0tk" href="http://www.helpindiakids.org/" title="" rel="noopener noreferrer" target="_blank">active</a>.</p></div><aside class="css-1m2ozyi"><span/></aside></div><div readability="49.657423971377"><div class="css-4w7y5l" readability="46.282647584973"><p class="css-1ebnwsw e2kc3sl0">“The chief means of happiness is complete independence,” Ramabai urged in her writings, and the means for that is education, which she called “indestructible wealth.”</p><p class="css-1ebnwsw e2kc3sl0">She identified with Native Americans and African Americans. In a letter to her daughter, she described meeting the escaped slave and abolitionist Harriet Tubman and urged Manorama to be “as helpful to her own dear countrywomen as Harriet was and is to her own people.”</p><p class="css-1ebnwsw e2kc3sl0">Over time, her shelter, which started as a strictly secular mission, became unabashedly religious. Ramabai built a church and established the <a class="css-1g7m0tk" href="http://www.helpindiakids.org/" title="" rel="noopener noreferrer" target="_blank">Mukti (Salvation) Mission</a>. She made contacts with Christian groups in Australia to help finance its expansion and received dozens of volunteers.</p><p class="css-1ebnwsw e2kc3sl0">Her work drew opposition from conservatives and others in India suspicious of her conversion to Christianity. One newspaper accused her of trying to “set afire the ancient religion of her compatriots with the help of foreigners.” But she found allies in <a class="css-1g7m0tk" href="http://www.dalitweb.org/?p=972" title="" rel="noopener noreferrer" target="_blank">Jyotiba Phule and his wife, Savitri</a>, two anti-caste reformers.</p><p class="css-1ebnwsw e2kc3sl0">“It was not just her feminism that was remarkable but the way that she understood and revealed the Hindu caste system” while also breaching the divisions the system created,<span class="css-8l6xbc e1tp43kk0"> </span>Chandra said.</p><p class="css-1ebnwsw e2kc3sl0">Manorama worked as Ramabai’s partner, helping run the schools and mission. But her health was poor, possibly from overwork, and she died at the age of 40. Ramabai died soon after, on April 5, 1922. She was 63.</p><p class="css-1ebnwsw e2kc3sl0">Ramabai’s critique of the Hindu patriarchy and embrace of Christianity extracted a heavy price, according to her biographers: It led to her marginalization in India and her ultimate omission from mainstream history books.</p></div><aside class="css-1m2ozyi"><span/></aside></div> Sun, 18 Nov 2018 10:43:13 +0000 https://www.nytimes.com/2018/11/14/obituaries/pandita-ramabai-overlooked.html Extending optionals in Swift https://www.swiftbysundell.com/posts/extending-optionals-in-swift https://www.swiftbysundell.com/posts/extending-optionals-in-swift <p>One thing that most modern programming languages have in common is that they provide some form of language-level support for expressing optional values. Instead of risking crashes or other runtime failures when a value turns out to be missing, languages like Swift let us leverage the compiler to verify that the right checks have been made, and that we have correctly <em>unwrapped</em> any value that was defined as optional.</p> <p>A really elegant aspect of Swift's implementation of optionals, is that a large part of this feature is implemented using the type system - since all optional values are actually represented using the enum <code>Optional&lt;Wrapped&gt;</code> under the hood. That gives us some interesting capabilities, since we can extend that enum - just like any other type - to add our own convenience APIs and other kinds of functionality.</p> <p>This week, let's take a look at how to do just that, and how doing so can let us deal with certain optional values in a really nice way.</p> <h2 id="converting-nil-into-errors">Converting nil into errors</h2> <p>When working with optional values, it's very common to want to convert a <code>nil</code> value into a proper Swift error, that can then be propagated and displayed to the user. For example, here we're preparing an image to be uploaded to our server through a series of operations. Since each operation might return <code>nil</code>, we're unwrapping the result of each step, throwing an error if <code>nil</code> was encountered - like this:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">func</span> prepareImageForUpload(<span class="keyword">_</span> image: <span class="type">UIImage</span>) <span class="keyword">throws</span> -&gt; <span class="type">UIImage</span> { <span class="keyword">guard</span> <span class="keyword">let</span> watermarked = <span class="call">watermark</span>(image) <span class="keyword">else</span> { <span class="keyword">throw</span> <span class="type">Error</span>.<span class="property">preparationFailed</span> } <span class="keyword">guard</span> <span class="keyword">let</span> encrypted = <span class="call">encrypt</span>(watermarked) <span class="keyword">else</span> { <span class="keyword">throw</span> <span class="type">Error</span>.<span class="property">preparationFailed</span> } <span class="keyword">return</span> encrypted }</code></pre> <p>The above code works, but let's see if we can use the power of extensions to make our code a bit more concise. First, let's create an extension on the <code>Optional</code> enum type that lets us either return its <em>wrapped</em> value or throw an error in case it contained <code>nil</code>, like this:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">extension</span> <span class="type">Optional</span> { <span class="keyword">func</span> orThrow( <span class="keyword">_</span> errorExpression: <span class="keyword">@autoclosure</span> () -&gt; <span class="type">Error</span> ) <span class="keyword">throws</span> -&gt; <span class="type">Wrapped</span> { <span class="keyword">guard</span> <span class="keyword">let</span> value = <span class="keyword">self</span> <span class="keyword">else</span> { <span class="keyword">throw</span> <span class="call">errorExpression</span>() } <span class="keyword">return</span> value } }</code></pre> <p><em>Above we use <a href="https://www.swiftbysundell.com/posts/using-autoclosure-when-designing-swift-apis"><code>@autoclosure</code></a> so that we only have to evaluate the error expression if needed, as to not do any unnecessary work - without requiring the caller of our new function to use any additional syntax.</em></p> <p>Using the above, along with a bit of optional chaining using <code>flatMap</code>, we can now construct a very nice <em>"chain of operations"</em> that lets us deal with any errors resulting from <code>nil</code> values at the end of the chain, and easily convert any such missing value into a proper error - like this:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">func</span> prepareImageForUpload(<span class="keyword">_</span> image: <span class="type">UIImage</span>) <span class="keyword">throws</span> -&gt; <span class="type">UIImage</span> { <span class="keyword">return</span> <span class="keyword">try</span> <span class="call">watermark</span>(image) .<span class="call">flatMap</span>(encrypt) .<span class="call">orThrow</span>(<span class="type">Error</span>.<span class="property">preparationFailed</span>) }</code></pre> <p><em>The above <code>flatMap</code> call works thanks to Swift's first class function capabilities. For more on that, check out <a href="https://www.swiftbysundell.com/posts/first-class-functions-in-swift">"First class functions in Swift"</a>.</em></p> <p>Doing something like the above might seem like a purely cosmetic change, but it enables us to increase the predictability of our code by modelling it as a sequence of operations - rather than having to keep track of multiple local variables. Of course it doesn't hurt that it looks nice as well 😉.</p> <h2 id="expressive-checks">Expressive checks</h2> <p>Another common scenario when dealing with optionals is wanting to perform some kind of check on the unwrapped value. For example, when implementing a form UI, we might want to change the border color of each text field whenever its text changed - depending on whether that string is currently empty:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">extension</span> <span class="type">FormViewController</span> { <span class="keyword">@objc</span> <span class="keyword">func</span> textFieldDidChange(<span class="keyword">_</span> textField: <span class="type">UITextField</span>) { <span class="keyword">if</span> textField.<span class="property">text</span>?.<span class="property">isEmpty</span> ?? <span class="keyword">true</span> { textField.<span class="property">layer</span>.<span class="property">borderColor</span> = <span class="type">UIColor</span>.<span class="property">red</span>.<span class="property">cgColor</span> } <span class="keyword">else</span> { textField.<span class="property">layer</span>.<span class="property">borderColor</span> = <span class="type">UIColor</span>.<span class="property">green</span>.<span class="property">cgColor</span> } } }</code></pre> <p>Using a default value when an optional contains <code>nil</code>, like we do above, is often a nice solution when used in a clear context - but it can also result in code that becomes harder to read due to the extra syntax required. Since checking whether a string (or any other kind of collection) is empty is quite a common operation - let's again see if we can improve the readability of our code by using an extension on <code>Optional</code>.</p> <p>This time we'll add a computed property called <code>isNilOrEmpty</code> to all optionals that contain a type conforming to <code>Collection</code>, in which we essentially perform the same check as we did above:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">extension</span> <span class="type">Optional</span> <span class="keyword">where</span> <span class="type">Wrapped</span>: <span class="type">Collection</span> { <span class="keyword">var</span> isNilOrEmpty: <span class="type">Bool</span> { <span class="keyword">return</span> <span class="keyword">self</span>?.<span class="property">isEmpty</span> ?? <span class="keyword">true</span> } }</code></pre> <p>If we now update our text field handling code from before to use our new property, we end up with a control flow that is a bit nicer and easier to read:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">extension</span> <span class="type">FormViewController</span> { <span class="keyword">@objc</span> <span class="keyword">func</span> textFieldDidChange(<span class="keyword">_</span> textField: <span class="type">UITextField</span>) { <span class="keyword">if</span> textField.<span class="property">text</span>.<span class="property">isNilOrEmpty</span> { textField.<span class="property">layer</span>.<span class="property">borderColor</span> = <span class="type">UIColor</span>.<span class="property">red</span>.<span class="property">cgColor</span> } <span class="keyword">else</span> { textField.<span class="property">layer</span>.<span class="property">borderColor</span> = <span class="type">UIColor</span>.<span class="property">green</span>.<span class="property">cgColor</span> } } }</code></pre> <p>The beauty of using extensions on <code>Optional</code> to implement convenience APIs is that we can easily reuse those same APIs in many different contexts. For example, we might want to check if an optional set of completed tutorial steps is <code>nil</code> or empty to determine what initial screen to show to the user, or to use the same API to check if the user has added any friends in our app:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">let</span> showInitialTutorial = completedTutorialSteps.<span class="property">isNilOrEmpty</span> <span class="keyword">let</span> hasAddedFriends = !user.<span class="property">friends</span>.<span class="property">isNilOrEmpty</span></code></pre> <p>Using extensions to wrap certain common expressions can also be a great way to make our code more self-documenting, since the intent of our expression becomes crystal clear. For more tips and techniques for more self-documenting code, check out <em><a href="https://www.swiftbysundell.com/posts/writing-self-documenting-swift-code">"Writing self-documenting Swift code"</a></em>.</p> <h2 id="matching-against-a-predicate">Matching against a predicate</h2> <p>Next, let's take a look at how we can add matching capabilities to optionals. Just like how we previously checked whether collections were empty, it's also common to want to match optional values against a more custom expression while the optional is being unwrapped.</p> <p>For example, here we're unwrapping a search bar's <code>text</code>, and then verifying that it contains at least 3 characters before performing a search:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">guard</span> <span class="keyword">let</span> query = searchBar.<span class="property">text</span>, query.<span class="property">length</span> &gt; <span class="number">2</span> <span class="keyword">else</span> { <span class="keyword">return</span> } <span class="call">performSearch</span>(with: query)</code></pre> <p>Again, the above code works, but let's see if we can make it a bit more elegant by enabling any optional to be matched against a given predicate. To do that, let's add another <code>Optional</code> extension. This one adds a function called <code>matching</code>, which takes a predicate in the form of a closure that returns a <code>Bool</code> after being passed an unwrapped optional value:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">extension</span> <span class="type">Optional</span> { <span class="keyword">func</span> matching(<span class="keyword">_</span> predicate: (<span class="type">Wrapped</span>) -&gt; <span class="type">Bool</span>) -&gt; <span class="type">Wrapped</span>? { <span class="keyword">guard</span> <span class="keyword">let</span> value = <span class="keyword">self</span> <span class="keyword">else</span> { <span class="keyword">return</span> <span class="keyword">nil</span> } <span class="keyword">guard</span> predicate(value) <span class="keyword">else</span> { <span class="keyword">return</span> <span class="keyword">nil</span> } <span class="keyword">return</span> value } }</code></pre> <p>Using the above, we can now make our search bar handling code very nice and expressive, by constructing yet another chain of optional expressions. First we use our new <code>matching</code> API to check that the search bar's text matches our length requirement, and then we map the result of that operation directly into our <code>performSearch</code> method - like this:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true">searchBar.<span class="property">text</span>.<span class="call">matching</span> { $0.<span class="property">count</span> &gt; <span class="number">2</span> } .<span class="call">map</span>(performSearch)</code></pre> <p>While the above is pretty cool, where this approach really shines is when we want to match a given optional against a series of different predicates. Using a <code>guard</code> statement with a longer list of conditions can quickly become messy, but with our new <code>matching</code> function we now have an easy way to chain multiple predicates together - like in this example where we're verifying that a database record matches two different requirements:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">let</span> activeFriend = database.<span class="call">userRecord</span>(withID: id) .<span class="call">matching</span> { $0.<span class="property">isFriend</span> } .<span class="call">matching</span> { $0.<span class="property">isActive</span> }</code></pre> <p>We can now treat our optionals a bit like queryable values. Pretty cool! 😎</p> <h2 id="assigning-reusable-views">Assigning reusable views</h2> <p>Finally, let's take a look at how we can extend the <code>Optional</code> type to make working with reusable views a bit nicer. A common pattern in Apple's UI frameworks is for views to provide certain <em>"slots"</em> where we as the API user can insert our own custom subviews. For example, <code>UITableViewCell</code> provides an <code>accessoryView</code> property that lets us place any view we want at the trailing edge of a cell - which is super convenient when building custom lists.</p> <p>However, since those slots need to support any kind of view, the type we're dealing with is most often <code>Optional&lt;UIView&gt;</code> - which means that we almost always have to do typecasting to convert the value of such a property into our own view type, leading to many <code>if let</code> dances that look something like this:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"> <span class="keyword">if</span> <span class="keyword">let</span> statusView = cell.<span class="property">accessoryView</span> <span class="keyword">as</span>? <span class="type">TodoItemStatusView</span> { statusView.<span class="property">status</span> = item.<span class="property">status</span> } <span class="keyword">else</span> { <span class="keyword">let</span> statusView = <span class="type">TodoItemStatusView</span>() statusView.<span class="property">status</span> = item.<span class="property">status</span> cell.<span class="property">accessoryView</span> = statusView }</code></pre> <p>This is yet another situation in which an extension on <code>Optional</code> can come very much in handy. Let's write an extension on all optionals that wrap a <code>UIView</code>, and again use the power of <code>@autoclosure</code> to enable us to pass an expression that creates a new view if needed, which is only used in case we don't have an existing one:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">extension</span> <span class="type">Optional</span> <span class="keyword">where</span> <span class="type">Wrapped</span> == <span class="type">UIView</span> { <span class="keyword">mutating</span> <span class="keyword">func</span> get&lt;<span class="type">T</span>: <span class="type">UIView</span>&gt;( orSet expression: <span class="keyword">@autoclosure</span> () -&gt; <span class="type">T</span> ) -&gt; <span class="type">T</span> { <span class="keyword">guard</span> <span class="keyword">let</span> view = <span class="keyword">self</span> <span class="keyword">as</span>? <span class="type">T</span> <span class="keyword">else</span> { <span class="keyword">let</span> newView = <span class="call">expression</span>() <span class="keyword">self</span> = newView <span class="keyword">return</span> newView } <span class="keyword">return</span> view } }</code></pre> <p>Using the above extension we can now make our cell configuration code from before a lot nicer - transforming it into two simple lines of code:</p> <pre data-preserve-html-node="true" class="splash"><code data-preserve-html-node="true"><span class="keyword">let</span> statusView = cell.<span class="property">accessoryView</span>.<span class="call">get</span>(orSet: <span class="type">TodoItemStatusView</span>()) statusView.<span class="property">status</span> = item.<span class="property">status</span></code></pre> <p>Not only have we eliminated boilerplate by getting rid of the need to manually unwrap and assign an accessory view, we've also made our code more declarative by replacing <code>if</code> and <code>else</code> conditions with a clearly defined expression. Big win! 😀</p> <h2 id="conclusion">Conclusion</h2> <p>The way Swift's type system was designed to support custom extensions on any type - and the fact that optionals are implemented using a standard enum - makes it possible to do some really interesting things. By wrapping common expressions and operations in well-targeted <code>Optional</code> extensions, we can both reduce boilerplate and make our code dealing with optionals more clear and expressive.</p> <p>However, it's also important to carefully decide what type of extensions that we wish to introduce into our code base, since a large number of extensions can make our project's learning curve a lot steeper. In general, avoiding premature optimization becomes key here, and only introducing an extension once we see a pattern repeated in multiple places might be a good policy to have.</p> <p>What do you think? Do you have any extensions on <code>Optional</code> that you really like, or could any of the ones from this article be useful in your code base? Let me know - along with any questions, comments or feedback that you might have - <a href="https://twitter.com/johnsundell">on Twitter @johnsundell</a>.</p> <p>Thanks for reading! 🚀</p> Sun, 18 Nov 2018 16:45:02 +0000 https://www.swiftbysundell.com/posts/extending-optionals-in-swift The New York Hustle of Amazon’s Second Headquarters https://www.newyorker.com/tech/annals-of-technology/the-new-york-hustle-of-amazons-second-headquarters https://www.newyorker.com/tech/annals-of-technology/the-new-york-hustle-of-amazons-second-headquarters <p>The <span data-page="page_1"/>city of hustlers got hustled. Jeff Bezos, the founder of Amazon and the world’s richest man, is wealthy enough to write a ten-thousand-dollar check to every apartment nabber, Wall Street shouter, what-time-is-it-showtime subway dancer, part-time-student pension chaser, asylum-seeking stage actor, author-cum-barista, wrong-way delivery guy, business-call-conducting taxi driver—to each of the more than eight million people currently living in New York City. This week, though, when Bezos bestowed upon the city the dubious honor of becoming Amazon’s second “headquarters”—an honor it shares with two other cities, including its existing headquarters, in Seattle—it was New Yorkers who paid him. The city and state offered Amazon at least 1.5 billion dollars in tax breaks and other grants to settle in a place that has not, historically, struggled to attract newcomers. (“I really think this could be the thing that finally puts New York on the map,” <a class="ArticleBody__link___1FS03" href="https://www.nytimes.com/2018/11/14/arts/television/corden-amazon-headquarters-new-york-washington.html" target="_blank">James Corden joked</a>.) When combined with existing incentives, Amazon might receive three billion dollars in breaks in New York alone, the equivalent of every city resident Venmoing $348 to Bezos.</p><p>Throughout a bidding process that saw dozens of cities vie to be the next location of a proposed hydra-headquarters, there were murmurs that Amazon might really just be looking for a regular office, and rebranding it a “headquarters” to corner those tax breaks. Those suspicions seemed validated when Amazon announced this week that its second headquarters would actually be two additional headquarters, one in Queens and one in Northern Virginia. (The company also announced plans to build a new Operations Center of Excellence in Nashville, which would employ five thousand people.)</p><p>In New York, the 1.2-billion-dollar tax break is based on Amazon’s commitment to generate twenty-five-thousand jobs, at an average annual salary of more than a hundred and fifty thousand dollars, which would net the company a forty-eight-thousand-dollar tax credit per job created. As part of a scheme called <em class="small">PILOT</em>, which stands for Payment In Lieu Of Tax, Amazon has also committed to fund community infrastructure and to donate space “for a tech startup incubator and for use by artists and industrial businesses” as well as for a newly built public school. “I’ll change my name to Amazon Cuomo if that’s what it takes,” Andrew Cuomo, the Democratic governor, said during the bidding process. After the announcement, Mayor Bill de Blasio, a progressive Democrat whom one might expect to have gripes with Amazon’s brand of <a class="ArticleBody__link___1FS03" href="https://nypost.com/2018/04/16/amazon-warehouse-workers-pee-into-bottles-to-avoid-wasting-time-undercover-investigator/" target="_blank">peeing-in-a-bottle capitalism</a>, declared Amazon’s move “a giant step on our path to building an economy in New York City that leaves no one behind.”</p><p>New York’s leaders were not alone in their enthusiasm for Amazon. Some <a class="ArticleBody__link___1FS03" href="https://qz.com/1119945/a-nearly-complete-list-of-the-238-places-that-bid-for-amazons-next-headquarters/" target="_blank">238 entities</a>—cities, states, two dozen different towns in Massachusetts, and a cluster of three businesswomen in Anchorage, Alaska—sent proposals to host Amazon’s second headquarters. Many of the bidders are understood to have shared proprietary data about their areas and plans, in addition to dangling tax incentives. “A key concept for understanding how Amazon operates is leverage,” Lina Khan, an academic fellow at Columbia Law School who has written extensively about Amazon, told me. “By running the search as a nationwide competition and receiving proposals from hundreds of cities, Amazon now has a database of information that gives it a further competitive advantage over rivals, as it’ll use this research to inform future expansion, and Amazon extracted the best deal through exercising its bargaining power over cities. The end outcome was to further enhance its dominance.”</p><p>It was the game-show quality of this bidding, the spectre of cash-starved governments begging to give money to a billionaire, that left some critics fuming. Richard Florida, the urban-studies theorist, told me that Amazon’s HQ2 competition “captures the zeitgeist of early 21st century American late capitalism.” He added, “The very idea that a trillion-dollar company run by the world’s richest man could run an <em class="">American Idol</em> auction on more than two hundred thirty cities across the United States (and Canada and Mexico) to extract data on sites and on incentives, and pick up a handy three billion dollars of taxpayer money in the process, is a sad statement of extreme corporate power in our time.”</p><p>Even as many of New York’s leading Democrats congratulated Amazon—and themselves—Alexandria Ocasio-Cortez, the representative-elect of New York’s Fourteenth Congressional District, which spans parts of the Bronx and Queens, criticized the deal on Twitter. “The idea that [Amazon] will receive hundreds of millions of dollars in tax breaks at a time when our subway is crumbling and our communities need MORE investment, not less, is extremely concerning to residents here,” she wrote. If American politics weren’t already confusing enough, her statement caused the conservative <em class="">National Review</em> to run a headline that supported the view of a Democratic Socialist: “Alexandria Ocasio-Cortez Is Right about Amazon’s Corporate Welfare.”</p><p>Reached by telephone on Thursday, Ocasio-Cortez called the Amazon deal “dressed-up trickle-down economics.” “What we’re seeing here is a complete public cost for a private corporate benefit,” she told me. “When you give a three-billion-dollar tax break to the richest company in the world, that means that you’re giving up our schools. You’re giving up our infrastructure. You’re giving up our community development.” In other words, there is an opportunity cost to luring the world’s richest man by letting him free-ride on the public services that other New Yorkers must pay for—whether it’s the failing subway system, the troubled and segregated school system, or, as Ocasio-Cortez noted, critical renovations<span data-page="page_final"/> at public-housing complexes like Queensbridge, the largest in the United States, which will soon be down the street from Amazon’s New York headquarters. Last year, Ocasio-Cortez said, residents there “went without heat and hot water in the dead of winter.” Residents have experienced similar outages this year.</p><p>One could argue that critics of the deal are being unfair. New York State and New York City have every reason to anticipate that the deal will put them in the black—netting them more revenue from Amazon than they are foregoing in tax breaks and grants. Although, for a company in the publishing and media businesses, New York has some allure beyond tax breaks, and a tougher set of demands still might have caused Amazon to walk away. As Cuomo said in a press conference, extending a tax break “costs us nothing, nada, niente.” But losing a guaranteed job creator to a state like Texas or Florida, where Amazon would have paid little to no income tax, would have cost the city dearly. Which is fair enough. But the larger question is: Why should it be this way? Why should states engage in races to the bottom and effectively penalize existing businesses whose only sin is having been in the state for a longer period of time?</p><p>There is also the particular question of why Bezos, of all people, needs to play this way. After the announcement, David Heinemeier Hansson, the founder of the software firm Basecamp, published an open letter to Bezos, who is one of his investors. “It doesn’t have to be like this,” he wrote. “If you tilt your perspective a little, I think you’ll be able to catch the optics that the richest man in the world asking for tribute like this is an ugly one.” He urged Bezos to consider shaping his legacy “into something more than the man who killed retail, extracted the greatest loot from its HQ cities, and who expanded the most monopoly holdings the fastest.”</p><p>What makes Bezos’s approach especially confounding is that it occurs at the precise moment that he is reinventing himself as a philanthropist. He and his wife announced earlier this fall that they will start by donating two billion dollars toward the education of poor children and the care of the homeless. It is a noble impulse, and it generates good press, but it also raises a more general question about American life today. Why do we tolerate, even celebrate, the particular arc of hyper-capitalism that Amazon represents? Build a company from the ground up. Do whatever it takes to survive and grow, regardless of the consequences for your customers and workers. Consolidate a monopoly if you can, first in one arena, then in multiple. Use that power, that leverage, to exact concessions from governments, so that you pay even fewer taxes and grow even faster, even bigger. And then, with the wealth that was accumulated by underpaying workers, by avoiding taxes, by lobbying against regulations, by amassing uncompetitive levels of market power—then, with that wealth, you give back. You make a difference. You become a philanthropist, a lover of mankind. You salve by philanthropic moonlight the wounds you may have cut by operational daylight. You solve the very problems you have helped to cause.</p> Sun, 18 Nov 2018 15:18:24 +0000 https://www.newyorker.com/tech/annals-of-technology/the-new-york-hustle-of-amazons-second-headquarters Dynamic DNS and Falco: detecting unexpected network activity https://sysdig.com/blog/unexpected-domain-connection/ https://sysdig.com/blog/unexpected-domain-connection/ <div class="o-container"> <div class="u-alignwidth-narrow"> <p><span> Since the </span><a href="https://sysdig.com/blog/sysdig-falco/"><span>inception of Falco</span></a><span>, we’ve seen users write custom rules covering a number of different use cases. Because Falco is behavioral monitoring with a syntax that leverages system calls, you can write a rule for just about anything: opening a file, becoming root, or making a network connection.</span></p><p><span>Today I’m going to talk about using Falco in a more traditional manner – using a rule to detect unexpected network activity. A Sysdig Secure customer wanted a policy to trigger if a connection was established to an unknown (and thus untrusted) domain names.</span></p><p><span>Out of the box Falco has a macro to detect outbound network activity:</span></p><pre><span>- macro: outbound</span><p><span> condition: &gt;</span></p><p><span> (((evt.type = connect and evt.dir=&lt;)) or</span></p><p><span> (fd.typechar = 4 or fd.typechar = 6) and</span></p><p><span> (fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and</span></p><p><span> (evt.rawres &gt;= 0 or evt.res = EINPROGRESS))</span><span><br/></span></p></pre><p><span>Let’s break this down:<br/><code>evt.type = connect and evt.dir=&lt;</code></span><span> is looking for a “connect” exit event</span></p><p><span><code>fd.typechar = 4 or fd.typechar = 6 </code></span><span>is matching a IPv4 or v6 file descriptor </span></p><p><span><code>fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8" </code></span><span>is excluding local connections</span></p><p><span><code>evt.rawres &gt;= 0 or evt.res = EINPROGRESS </code></span><span>ensures we only look for successful (or in progress) socket binds</span></p><p><span>With this macro we can easily write a rule matching against a connection to an IP address. This rule will trigger upon establishing connection with sysdig.com</span></p><pre><span>$ host sysdig.com</span><p><span>sysdig.com has address 35.184.21.208</span></p><p><span>- rule: Connection to sysdig.com</span></p><p><span> desc: Detect attempts to connect to sysdig.com (35.184.21.208)</span></p><p><span> condition: outbound and fd.sip="35.184.21.208" </span></p><p><span> output: Outbound connection to sysdig.com (command=%proc.cmdline connection=%fd.name %container.info image=%container.image)</span></p><p><span> priority: NOTICE</span></p><p><span> tags: [network]</span></p></pre><p><span>This gets us part way there, but what if sysdig.com has dynamic DNS resulting in the IP address changing? Our rule would be useless, as we’ve hardcoded in a static IP 35.184.21.208. For example, s3.us-east-2.amazonaws.com can resolve to different IP addresses on each lookup:</span></p><pre><span>harry-mbp:~ $ host s3.us-east-2.amazonaws.com</span><p><span>s3.us-east-2.amazonaws.com has address 52.219.96.58</span></p><p><span>harry-mbp:~ $ host s3.us-east-2.amazonaws.com</span></p><p><span>s3.us-east-2.amazonaws.com has address 52.219.104.82</span></p></pre><p><span>We could keep a number of different IP addresses in a Falco list:</span></p><pre><span>- list: s3_ips</span><p><span> items: ['"52.219.96.58"', '"52.219.104.82"']</span></p><p><span>- rule: Connection to S3</span></p><p><span> desc: Detect attempts to connect to S3</span></p><p><span> condition: outbound and fd.sip in (s3_ips)</span></p><p><span> output: Outbound connection to S3 URL(command=%proc.cmdline connection=%fd.name %container.info image=%container.image)</span></p><p><span> priority: NOTICE</span></p><p><span> tags: [network]</span></p></pre><p><span>…but it wouldn’t be easy to ensure this list is fresh and up to date, resulting in expected network connections to trigger a policy.</span></p><h2><span>Introducing fd.sip.name</span></h2><p><span>As a member of our customer success team, I’m well aware how receptive Sysdig’s product teams are to customer requests. Realising the awkwardness of dynamic DNS and (not wanting) hard coded IP addresses within Falco rules, our engineering team worked on a better way to solve this problem. In version 0.24.0 of sysdig open source, we added a number of new filter checks:</span></p><pre><span>fd.cip.name Domain name associated with the client IP address.</span><p><span>fd.sip.name Domain name associated with the server IP address.</span></p><p><span>fd.lip.name Domain name associated with the local IP address.</span></p><p><span>fd.rip.name Domain name associated with the remote IP address.</span></p></pre><p><span>These new filterchecks allow specifying a domain name, and Sysdig will resolve the IP address, intelligently maintaining the IP addresses it resolves to. This means we can now easily pass Falco domain names, and not have to worry about unreliably resolving the IP addresses and keeping them up to date.</span></p><p><span>In the example below, I have defined a list of trusted domain names (sysdig.com, github.com &amp; google.com). Any network connection to an IP address that isn’t resolved to by any of these domain names will trigger the policy. Great for telling Falco what’s allowed and being told about everything else.</span></p><pre><span>- list: trusted_domains</span><p><span> items: [sysdig.com, github.com, google.com]</span></p><p><span>- rule: Unexpected outbound network connection</span></p><p><span> desc: Detect outbound connections with destinations not on allowed list</span></p><p><span> condition: &gt;</span></p><p><span> outbound</span></p><p><span> and not (fd.sip.name in (trusted_domains))</span></p><p><span> output: Unexpected Outbound Connection</span></p><p><span> (container=%container.name</span></p><p><span> command=%proc.cmdline</span></p><p><span> procpname=%proc.pname</span></p><p><span> connection=%fd.name</span></p><p><span> servername=%fd.sip.name</span></p><p><span> serverip=%fd.sip</span></p><p><span> type=%fd.type</span></p><p><span> typechar=%fd.typechar</span></p><p><span> fdlocal=%fd.lip</span></p><p><span> fdremote=%fd.rip)<br/></span><span>priority: NOTICE</span></p></pre><h2><span>But what about Kubernetes?</span></h2><p><span>Once a Falco rule has been written it can be easily added to Sysdig Secure. Rules can be associated with a policy, scoped to a specific portion of your infrastructure, and tied to remediation actions like killing or pausing a container.</span></p><p><span>In the example below the above Falco rule has been applied to the scope of kubernetes.deployment.name = javaapp meaning if any container with the metadata in that scoping makes a connection to an IP address outside the IPs our domains resolve to, then an alert will be fired and in this case the container will also be killed.</span></p><p><a href="https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1.png"><img src="https://sysdig.com/blog/unexpected-domain-connection/" data-lazy-src="https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1-350x388.png" alt="" width="350" height="388" class="img-lightbox alignnone wp-image-12126 size-medium" data-lazy-srcset="https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1-350x388.png 350w, https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1-768x851.png 768w, https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1-1170x1296.png 1170w, https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1-175x194.png 175w, https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1.png 1210w, / 1w" data-lazy-sizes="(max-width: 350px) 100vw, 350px"/><noscript><img src="https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1-350x388.png" alt="" width="350" height="388" class="img-lightbox alignnone wp-image-12126 size-medium" srcset="https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1-350x388.png 350w, https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1-768x851.png 768w, https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1-1170x1296.png 1170w, https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1-175x194.png 175w, https://478h5m1yrfsa3bbe262u7muv-wpengine.netdna-ssl.com/wp-content/uploads/2018/11/falco-name-resolution-blog-1.png 1210w, / 1w" sizes="(max-width: 350px) 100vw, 350px"/></noscript></a><br/></p><p><span>Using the labels from Kubernetes or your cloud provider can be a powerful way to make sure your policies are tailored to your specific clusters, namespaces, or VPC’s. Sysdig Secure does all of this tagging automatically and manages the distribution of falco rules to all your endpoints. Even if they’re across thousands of nodes.</span></p><h2><span>Conclusion</span></h2><p><span>Hopefully this blog gives you an idea of how powerful system calls can be as a data source to use for intrusion detection, auditing, and behavioral monitoring. We’d love to hear more from you about what you’re doing with Falco in the wild. Reach out to us on <a href="https://twitter.com/sysdig">twitter</a> or join our <a href="http://slack.sysdig.com">slack</a>.</span></p><p><span>Check out some of these blogs to learn more:</span></p> </div><!-- .entry-content --> </div> Sun, 18 Nov 2018 17:21:09 +0000 Harry Perks https://sysdig.com/blog/unexpected-domain-connection/ OpenZFS vs. Btrfs and other file systems (2017) https://www.ixsystems.com/blog/open-zfs-vs-btrfs/ https://www.ixsystems.com/blog/open-zfs-vs-btrfs/ &#13; <!-- iX custom-->&#13; <div class="et_pb_section et_pb_section_0 et_section_regular">&#13; <div class=" et_pb_row et_pb_row_0">&#13; <div class="et_pb_column et_pb_column_2_3 et_pb_column_0">&#13; <div class="et_pb_posts et_pb_module et_pb_bg_layout_light blog_layout et_pb_blog_0">&#13; &#13; &#13; &#13; <article id="post-57392" class="et_pb_post post-57392 post type-post status-publish format-standard has-post-thumbnail hentry category-blog tag-open-source tag-openzfs tag-storage tag-zfs">&#13; &#13; <p class="post-meta"><span class="published">Aug 4, 2017</span> | <a href="https://www.ixsystems.com/blog/category/blog/" rel="category tag">Blog</a> | <span class="comments-number "><a href="https://www.ixsystems.com/blog/open-zfs-vs-btrfs/#comments">11 comments</a></span></p> &#13; &#13; &#13; &#13; <p><strong>Our Senior Analyst’s take on this week’s Btrfs news from Red Hat</strong></p> <p>I don’t know who said it first but hats off to them: “The only thing worse than competition is no competition.” This adage applies equally to market making where no competition can mean no customers, and to monopolies and monocultures. Beyond the balance of freedom and control that Open Source provides, the sheer choice found in the Open Source ecosystem is one of its greatest strengths. Name any category of software from complete operating systems on up and you have a plethora of choices with drastically-different philosophies, licenses, countries of origin, programming languages, and user experiences. I personally have invested my volunteer time and career in Open Source hypervisors and file systems and I am saddened to hear that a fledgling alternative to OpenZFS suffered a setback this week with Red Hat’s announcement that it is <a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/7.4_Release_Notes/chap-Red_Hat_Enterprise_Linux-7.4_Release_Notes-Deprecated_Functionality.html">deprecating Btrfs</a> as a “Preview” file system. SUSE continues to support Btrfs <em><a href="https://www.suse.com/documentation/sles-12/singlehtml/stor_admin/stor_admin.html#sec.filesystems.major.btrfs">in only RAID 10 equivalent configurations</a></em>, and only time will tell if <a href="http://bcachefs.org/">bcachefs</a> proves to be a compelling alternative to OpenZFS. This vote of no confidence from Red Hat leaves OpenZFS as the only proven Open Source data-validating enterprise file system and with that role comes great responsibility.</p> <p>“In their FACES, right?” Wrong. Monocultures risk becoming vulnerable monopolies which is why virus writers target Microsoft Windows and we may face an “<a href="https://lwn.net/Articles/681616/">Impending Crypto Monoculture</a>“. My colleagues with the OpenBSD project are flattered by the popularity of OpenSSH but insist that they don’t want it to be the only game in town. Monoculturalism has long been a driving factor in computing and is often self-perpetuating: Why not use and standardize on a good technology? OpenSSH was the right solution at the right time and remains the de facto remote login tool on Internet-connected systems, open source and proprietary. The same is becoming true of OpenZFS, the community branch of Sun Microsystems’ revolutionary, and eventually open sourced enterprise file system.</p> <p>Fortunately, like OpenSSH, OpenZFS <em>really is as good as people say it is</em>. OpenZFS goes to unrivaled lengths to protect your data and is highly flexible and scalable. I have <a href="https://www.ixsystems.com/blog/openzfs-vs-the-competition/"><span>addressed the merits of OpenZFS</span></a> at length in various ways and I welcome you, in fact urge you to verify those merits on your own. I invite you to start that journey with a simple question: “Can you verify without a doubt that your data has not suffered from bit rot?” I look forward to your answer. In the meantime, I personally am confident that OpenZFS truly addresses the shortcomings of other file systems and does so in a way that is extremely accessible to me:</p> <ul><li>OpenZFS has been my primary store under macOS for over three years and root file system under FreeBSD</li> <li>I have moved OpenZFS-formatted multi-terabyte USB drives from my FreeNAS system to a<em> Raspberry Pi 3</em> running FreeBSD and run my backup routine without issue</li> <li>I have helped clients configure, maintain and optimize OpenZFS-based systems ranging from one to 500 terabytes in size</li> <li>I have watched the <a href="http://open-zfs.org/wiki/Main_Page"><span>OpenZFS community</span></a> grow to include amazing volunteers and vendors who do what was impossible with storage at any price only a few years ago</li> </ul><p>It is an honor to work with the OpenZFS community and iXsystems in particular who, thanks to FreeNAS, TrueNAS and TrueOS, <em>has put OpenZFS in more hands than any other project or product on Earth</em>. Both are just now accelerating from a trot to a gallop and I am very glad that they have been cautious and calculating. Drama is not something you want to associate with file systems or the hardware they run on. Thanks to Illumos, FreeBSD and FreeNAS, no one is stopping you from building a petabyte of storage with whatever hardware you can afford. You really want to get the right hardware but no artificial barriers stand in your way. As you can imagine, iXsystems is an excellent source of the right hardware for OpenZFS, but that too is something I invite you to verify on your own. I am after all, a geek, not a salesperson.</p> <p><strong>If it’s so good, why isn’t OpenZFS as popular as GNU/Linux?</strong></p> <p>Short answer: The OpenZFS and Linux kernel licenses are incompatible, but for a reason. It took time, but I accept Bryan Cantrill’s assertion that the Sun CDDL was essential to <a href="https://www.youtube.com/watch?v=Zpnncakrelk"><span>keeping Sun and later Oracle from doing evil things with ZFS</span></a>. This pains me because I am not a believer in software patents and believe that <a href="http://callfortesting.org/curse/"><span>permissively-licensed software is the way forward</span></a>, even if paradoxically at times. I also believe in the <a href="http://gondwanaland.com/mlog/2015/03/05/gpl-lovers-haters-exploiters-enforce/"><span>6 reasons for GPL lovers, haters, exploiters, and others to enjoy and support GPL enforcement</span></a> because <em>all</em> free software licenses need to be enforced to remain meaningful. In the case of GNU/Linux, OpenZFS’ CDDL license is incompatible with the Linux kernel’s General Public License according to the <a href="https://www.fsf.org/licensing/zfs-and-linux"><span>Free Software Foundation</span></a> and <a href="https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/"><span>Software Freedom Conservancy</span></a>. This is presumably why OpenZFS is not even a “Preview” file system in Red Hat Enterprise Linux as Btrfs was. To comply with each license, the end user must manually build OpenZFS for Linux and for what it’s worth, <em>this sounds like a great way to stay true to GNU/Linux’s DIY community roots</em>. Embrace the license diversity and obligations, or agree with me that the permissive licensing of each project would resolve this incompatibility without consequences.</p> <p>To that point, I welcome the bcachefs project to consider a permissive license to allow its incorporation into FreeBSD, OpenBSD, NetBSD, macOS and even Windows to allow its merits to shine on equal footing and <em>in the hands of as many users as possible</em>. Until that happens though, the Illumos distributions, FreeBSD, TrueOS and FreeNAS remain the only tier-one OpenZFS operating systems and thus <em>places you want to keep your valuable data for the foreseeable future.</em></p> <p>Michael Dexter<br/>Independent Analyst</p> &#13; <!-- You can start editing here. --><section id="comment-wrap"> <ol class="commentlist clearfix"><li class="comment even thread-even depth-1" id="li-comment-1011"> <article id="comment-1011" class="comment-body clearfix"> <div class="comment_postinfo"> <span class="fn">Pedro</span> <span class="comment_date"> on August 6, 2017 at 12:51 pm </span> </div> <!-- .comment_postinfo --> <!-- end comment_area--> </article><!-- .comment-body --><ul class="children"><li class="comment odd alt depth-2" id="li-comment-1016"> <article id="comment-1016" class="comment-body clearfix"> <div class="comment_postinfo"> <span class="fn">Chris</span> <span class="comment_date"> on August 15, 2017 at 6:42 pm </span> </div> <!-- .comment_postinfo --> <!-- end comment_area--> </article><!-- .comment-body --></li><!-- #comment-## --> </ul><!-- .children --></li><!-- #comment-## --> <li class="comment even thread-odd thread-alt depth-1" id="li-comment-1015"> <article id="comment-1015" class="comment-body clearfix"> <!-- .comment_postinfo --> <!-- end comment_area--> </article><!-- .comment-body --></li><!-- #comment-## --> <li class="comment odd alt thread-even depth-1" id="li-comment-1031"> <article id="comment-1031" class="comment-body clearfix"> <div class="comment_postinfo"> <span class="fn">MB</span> <span class="comment_date"> on September 28, 2017 at 12:58 am </span> </div> <!-- .comment_postinfo --> <!-- end comment_area--> </article><!-- .comment-body --></li><!-- #comment-## --> <li class="comment even thread-odd thread-alt depth-1" id="li-comment-1095"> <article id="comment-1095" class="comment-body clearfix"> <div class="comment_postinfo"> <span class="fn">Evi1M4chine</span> <span class="comment_date"> on October 17, 2017 at 10:18 am </span> </div> <!-- .comment_postinfo --> <!-- end comment_area--> </article><!-- .comment-body --><ul class="children"><li class="comment odd alt depth-2" id="li-comment-1123"> <article id="comment-1123" class="comment-body clearfix"> <div class="comment_postinfo"> <span class="fn">Corrodias</span> <span class="comment_date"> on November 10, 2017 at 3:37 pm </span> </div> <!-- .comment_postinfo --> <!-- end comment_area--> </article><!-- .comment-body --></li><!-- #comment-## --> </ul><!-- .children --></li><!-- #comment-## --> <li class="comment even thread-even depth-1" id="li-comment-1122"> <article id="comment-1122" class="comment-body clearfix"> <div class="comment_postinfo"> <span class="fn">GreyGeek</span> <span class="comment_date"> on November 8, 2017 at 9:05 pm </span> </div> <!-- .comment_postinfo --> <!-- end comment_area--> </article><!-- .comment-body --></li><!-- #comment-## --> <li class="comment odd alt thread-odd thread-alt depth-1" id="li-comment-1365"> <article id="comment-1365" class="comment-body clearfix"> <div class="comment_postinfo"> <span class="fn">John Judenrein</span> <span class="comment_date"> on May 9, 2018 at 4:33 am </span> </div> <!-- .comment_postinfo --> <!-- end comment_area--> </article><!-- .comment-body --></li><!-- #comment-## --> <li class="comment even thread-even depth-1" id="li-comment-1372"> <article id="comment-1372" class="comment-body clearfix"> <div class="comment_postinfo"> <span class="fn"><a href="http://c3d.github.io" rel="external nofollow" class="url">c3d</a></span> <span class="comment_date"> on May 15, 2018 at 9:32 am </span> </div> <!-- .comment_postinfo --> <!-- end comment_area--> </article><!-- .comment-body --></li><!-- #comment-## --> <li class="comment odd alt thread-odd thread-alt depth-1" id="li-comment-1384"> <article id="comment-1384" class="comment-body clearfix"> <div class="comment_postinfo"> <span class="fn">kamtaot</span> <span class="comment_date"> on May 29, 2018 at 3:27 am </span> </div> <!-- .comment_postinfo --> <!-- end comment_area--> </article><!-- .comment-body --></li><!-- #comment-## --> <li class="comment even thread-even depth-1" id="li-comment-1419"> <article id="comment-1419" class="comment-body clearfix"> <!-- .comment_postinfo --> <!-- end comment_area--> </article><!-- .comment-body --></li><!-- #comment-## --> </ol><!-- #respond --> </section></article><!-- .et_pb_post --></div>&#13; </div> <!-- .et_pb_column -->&#13; <!-- .et_pb_column -->&#13; </div> <!-- .et_pb_row -->&#13; <!-- .et_pb_row -->&#13; </div> <!-- .et_pb_section -->&#13; Sun, 18 Nov 2018 09:39:07 +0000 https://www.ixsystems.com/blog/open-zfs-vs-btrfs/ Here are the 'cyber 9/11' scenarios that really worry the experts https://www.cnbc.com/2018/11/18/cyber-911-scenarios-power-outages-bank-runs-changed-data.html https://www.cnbc.com/2018/11/18/cyber-911-scenarios-power-outages-bank-runs-changed-data.html <p>Cyberattacks that cause major disruption to public services have happened many times in the real world.</p> <p>Some of them are very old news, in fact. But it's easy to imagine how a similar attack could shut down basic services, like electricity or water, that affect millions of people.</p> <p><a class="inline_asset" href="https://www.computerworld.com/article/2561484/security0/utility-hack-led-to-security-overhaul.html">In 2000</a>, a disgruntled sewage treatment plant worker in Queensland, Australia hacked into his employer's industrial control system to unleash torrents of raw sewage onto public grounds, flooding the city's local <a class="inline_quotes" data-gdsid="69490" data-inline-quote-symbol="H" href="https://www.cnbc.com/quotes/?symbol=H">Hyatt</a> hotel. The perpetrator was sentenced to two years for the attack.</p> <p>In 2007, <a class="inline_asset" href="http://www.nbcnews.com/id/18723187/ns/technology_and_science-security/t/estonia-says-russia-may-back-web-attacks/">the country of Estonia</a> was subject to widespread outages in its entire telecommunications network, following a cyberattack stemming from a dispute with Russia over a military statue. The incident was so damaging, it led to a decision to place the North Atlantic Treaty Organization's Cyber Security organization in Tallinn, the country's capital.</p> <p>In 2015, Ukraine's power grid had massive outages after a cyberattack — <a class="inline_asset" href="https://www.cnn.com/2016/02/11/politics/ukraine-power-grid-attack-russia-us/index.html">which some officials have attributed to Russia</a> — two days before Christmas, during a cold snap. Around a quarter-million residents were left without power, but the outages only lasted a few hours before government agencies were able to restore service.</p> <p>Major cyberattacks aimed at taking down official services don't need to be strictly nation-state sponsored or terrorist-backed. They can be strictly criminal in nature, or come from a malevolent backer under the guise of a criminal attack.</p> <p><a href="https://www.cnbc.com/2018/02/15/uk-blames-russia-for-cyberattack-says-wont-tolerate-disruption.html">The NotPetya cyberattacks of June 2017</a>, known by the name of the criminal ransomware-inspired computer virus behind it, were notorious for the real-world harm they caused to companies. In Germany, consumer goods-maker <a class="inline_quotes" data-gdsid="29945" data-inline-quote-symbol="RB.-GB" href="https://www.cnbc.com/quotes/?symbol=RB.-GB">Reckitt Benckiser</a> halted shipments of numerous products. Ships belonging to logistics giant<a class="inline_quotes" data-gdsid="24074" data-inline-quote-symbol="MAERSK.B-DK" href="https://www.cnbc.com/quotes/?symbol=MAERSK.B-DK"> Maersk</a> were at a standstill, and the company later said it took a <a href="https://www.cnbc.com/2017/08/16/maersk-says-notpetya-cyberattack-could-cost-300-million.html">$300 million hit from the attack</a>. In the U.S., a facility owned by <a class="inline_quotes" data-gdsid="25524" data-inline-quote-symbol="MRK" href="https://www.cnbc.com/quotes/?symbol=MRK">Merck</a> that makes the HPV vaccine Gardasil was shut down to such a big extent, the company had to borrow hundreds of millions of dollars worth of back-up vaccines stockpiled by the Center for Disease Control.</p> <p>Power outages or water supply corruption are the most worrisome to Peter Beshar, general counsel for risk management firm <a class="inline_quotes" data-gdsid="25132" data-inline-quote-symbol="MMC" href="https://www.cnbc.com/quotes/?symbol=MMC">Marsh &amp; McLennan</a>. Loss of electricity, he said, is just one piece of the greater risk for physical security stemming from a cyberattack.</p> <p>"Utilities are one vital resource. But it's not just power, water is another type of utility. If all of a sudden, the quality of drinking water is called into question, and then manufacturers who rely on using untainted water for making drugs or food is called into question. That is a potential crisis," he said.</p> Sun, 18 Nov 2018 16:03:25 +0000 Kate Fazzini https://www.cnbc.com/2018/11/18/cyber-911-scenarios-power-outages-bank-runs-changed-data.html Genetic Programming in Clojure https://sulami.github.io/posts/genetic-programming-in-clojure/ https://sulami.github.io/posts/genetic-programming-in-clojure/ <h2 id="the-theory">The Theory</h2> <p>Like most programmers I have always had a vague interest in AI, and one of its branches that requires less complicated maths than recurrent neural networks which are the most well known one, is genetic programming. The idea of genetic programming is quite simple<span><label for="sn-1" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-1" class="margin-toggle" /><span class="sidenote">If you are into more visual examples, this I believe is a very good practical example: <a href="http://rednuht.org/genetic_cars_2/" class="uri">http://rednuht.org/genetic_cars_2/</a></span></span></p><p>:</p> <ol><li>You build something that is parameterised in key places</li> <li>You build a scoring function to assess the performance of your something with a set of parameters</li> <li>You randomly adjust (mutate) your parameters in some way a couple of times and compare the score of each set</li> <li>You take the best one or ones as a base to start a new round of mutations and scoring</li> <li>Basically just repeat steps 3 &amp; 4 for a while and your parameters will tend towards a maximum score</li> </ol><p>Depending on a variety of meta-parameters that control for example the size of each generation or the nature of the mutations you might just find a local maximum, but often times this can yield pretty good results for a variety of problems.</p> <h2 id="the-practice">The Practice</h2> <p>I have toyed around with this over the last couple of days and built an very simple abstract implementation in Clojure, which I am going to share here (and eventually will be somewhere on Github as well). Let us explore it from the inside out.</p> <p>First of all we need to be able to generate some mutations of our specimen. Because we do not assume anything about the specimen, this ends up being quite simple because a lot of the heavy lifting is done outside of this implementation as it is specific to the problem in question.</p> <p>This returns a potentially infinite list containing first the specimen passed in, and then as many mutations of it as we want.</p> <div class="sourceCode" readability="9"><pre class="sourceCode clojure"><code class="sourceCode clojure">(<span class="bu">defn</span><span class="fu"> mutate</span> <span class="st">"Generator that mutates a base, the first element being the base."</span> [base mutator] (<span class="kw">concat</span> [base] (<span class="kw">repeatedly</span> #(mutator base))))</code></pre></div> <p>Next we also need to be able to score it. In this case we would like to attach the scores to the specimens so that we can use them to sort and select specimens without losing the specimens themselves.</p> <div class="sourceCode" readability="8"><pre class="sourceCode clojure"><code class="sourceCode clojure">(<span class="bu">defn</span><span class="fu"> attach-score</span> <span class="st">"Attaches the score to a specimen."</span> [score-fn specimen] [specimen (score-fn specimen)])</code></pre></div> <p>Now let us begin to tie these together. A single generation should take a base specimen, mutate it a couple of times, score each of them, and then select the “fittest” based on the scores<span><label for="sn-2" class="margin-toggle sidenote-number"></label><input type="checkbox" id="sn-2" class="margin-toggle" /><span class="sidenote">This also only keeps the best specimen in every generation, which makes the code much simpler. For actual real world usage it might be beneficial to keep the best <code>n</code> specimens in every generation to avoid running into local maxima. This would make the mutation slightly more complex though because there would be several base specimens which need to be mutated, so I decided to leave out this feature for the purposes of explanation.</span></span></p><p>. Note that in this implementation a lower score is better. To change this just reverse the sorting.</p> <div class="sourceCode" readability="9"><pre class="sourceCode clojure"><code class="sourceCode clojure">(<span class="bu">defn</span><span class="fu"> generation</span> <span class="st">"Picks out the best one from a generation."</span> [base mutator score-fn gen-size] (<span class="kw">-&gt;&gt;</span> (mutate base mutator) (<span class="kw">take</span> gen-size) (<span class="kw">map</span> (<span class="kw">partial</span> attach-score score-fn)) (<span class="kw">sort-by</span> <span class="kw">second</span>) <span class="kw">first</span> <span class="kw">first</span>))</code></pre></div> <p>And to finish off, we just need to run a number of generations, each based on the previous one’s winner.</p> <div class="sourceCode" readability="8"><pre class="sourceCode clojure"><code class="sourceCode clojure">(<span class="bu">defn</span><span class="fu"> evolution</span> <span class="st">"Generator for generations."</span> [base mutator score-fn gen-size] (<span class="kw">iterate</span> #(generation % mutator score-fn gen-size) base))</code></pre></div> <p>The lazy nature of this implementation is allows us to inspect intermediate results easily, as we can see the path evolution has taken in the form of each generation’s winner.</p> <h2 id="the-actual-practice">The Actual Practice</h2> <p>Now this above is actually not that much code, and it is very abstract in its nature, so let us have a look at what it looks like when we actually use it. A simple example would be approximating a single number that is hard to approximate, like √2.</p> <p>Our specimen is just a float, and any will do as the initial seed. It is itself the only parameter.</p> <p>To mutate it, we just adjust it by a random amount within <code>0.5</code> in either direction.</p> <div class="sourceCode" readability="7"><pre class="sourceCode clojure"><code class="sourceCode clojure">(<span class="bu">defn</span><span class="fu"> mutator </span>[base] (<span class="kw">-&gt;</span> (<span class="kw">rand</span>) (<span class="kw">-</span> <span class="fl">0.5</span>) (<span class="kw">+</span> base)))</code></pre></div> <p>Our scoring function is cheating a little, because we already know the target, we can just compare against it and use the distance as the score.</p> <div class="sourceCode" readability="7"><pre class="sourceCode clojure"><code class="sourceCode clojure">(<span class="bu">defn</span><span class="fu"> score-fn </span>[x] (<span class="kw">-&gt;</span> x (<span class="kw">-</span> (Math/sqrt <span class="dv">2</span>)) Math/abs))</code></pre></div> <p>Now when we run this, we can see how it approximates the target value over time (<code>√2 ≈ 1.4142</code>).</p> <div class="sourceCode" readability="8"><pre class="sourceCode clojure"><code class="sourceCode clojure">(<span class="kw">take</span> <span class="dv">6</span> (evolution base mutator score-fn <span class="dv">25</span>)) <span class="co">;; =&gt; (0.0</span> <span class="co">;; 0.33079046010191426</span> <span class="co">;; 0.7509224756253191</span> <span class="co">;; 1.2164225056336746</span> <span class="co">;; 1.3768753691848903</span> <span class="co">;; 1.4125030676422798)</span></code></pre></div> <p>Because <code>evolution</code> returns an infinite sequence, we can just use <code>nth</code> on it to get the winner after a certain number of generations.</p> <p>While this is a very simple example, I am currently working on a way of using this to build and mutate a Clojure <a href="https://en.wikipedia.org/wiki/S-expression">S-expression</a> and score it by running a series of unit tests against the generated code. If this works out I might write about it here soon.</p> Sat, 17 Nov 2018 16:32:43 +0000 https://sulami.github.io/posts/genetic-programming-in-clojure/ Shaved Heads, Snipped Tubes, Imperial Marines and Dope Fiends: On Synanon (2013) http://www.cabinetmagazine.org/issues/48/pendle.php http://www.cabinetmagazine.org/issues/48/pendle.php <p>Winter 2012–2013</p><div class="bg" readability="13"><div class="padding" readability="21"><p>Can a false god deliver real miracles? Take the case of Charles E. Dederich, better known as Chuck. A more unlikely figure of divinity would be hard to imagine. Born in Toledo, Ohio, in 1913, Dederich’s early years were spent staggering through the American wilderness in a drunken haze. He flunked out of Notre Dame, was fired from his job at Gulf Oil, married and divorced twice, lost touch with his children, and slipped into what he termed “a holocaust of boredom.” By the mid-1950s, he was a wino stumbling along the beach in Santa Monica, California, without friends or family to help him.
</p></div></div> <div class="bg" readability="9"><div class="padding" readability="13"><p>Somehow he was shepherded into an Alcoholics Anonymous meeting, and it was here that his special nature began to reveal itself. Dederich studiously worked his way through the Twelve-Step Program, admitting that he was “powerless over alcohol” and reluctantly accepting that a “higher power” was all that could restore him to sobriety. Although he was wary of AA’s woozy spirituality, he was enamored of and energized by the family-like dynamic of mutual support.</p></div></div> <div class="bg" readability="11"><div class="padding" readability="17"><p>A large man with pale white skin seemingly untouched by the California sun, Dederich’s face had been partially paralyzed by a bout of meningitis, leaving him with a tic and a sneer. He spoke in a raspy, bullfrog croak. But the rough eloquence that burst forth from him at meetings belied his intimidating appearance. He was insightful yet funny, philosophical yet colloquial, and, above all, he was brutally honest with both others and himself. He quickly became a favorite speaker within the Santa Monica group.
</p></div></div> <p>Chuck Dederich lecturing at a Synanon meeting, 1962.</p> <div class="bg" readability="12.5"><div class="padding" readability="20"><p>Embracing AA whole-heartedly, Dederich began helping others like himself, talking them sober and bullying them into giving up the bottle. He was good at it too, and began inviting AA members back to his apartment after meetings to eat soup, drink coffee, and engage in informal “bull sessions.” He may have been living on welfare, sleeping in fleabag boarding houses, and barely able to afford a meal a day, but he was sober and content. Fate, however, had greater things in store for him. </p></div></div> <div class="bg" readability="10.5"><div class="padding" readability="16"><p>In August 1957, the University of California was searching for therapeutic uses for the new wonder drug, LSD. The researchers who had tried the drug had been amazed by the spiritual vistas it opened to them and the profound change in outlook it afforded. With this in mind, it was thought that alcoholics, especially those drawn from AA, could receive the greatest therapeutic benefits since they had already given themselves over to “a higher power” and would thus be more open to the beneficent effects of an outside force. In need of money, Dederich volunteered. </p></div></div> <div class="bg" readability="8"><div class="padding" readability="11"><p>Yet while the other volunteers endured sensory enhancements and hallucinations, Dederich experienced none of these. Instead, he recalled having only “feelings of omnipotence and omniscience.” Dederich had finally found a higher power he could be comfortable accepting. It was himself. </p></div></div> <div class="bg" readability="9"><div class="padding" readability="13"><p>• • •</p> <p>Soon the number of people wanting to join Dederich’s after-hours sessions grew too big for his living quarters. This was largely due to an influx of drug addicts who had heard of Dederich’s ability to keep people straight. For the addicts, Dederich offered their only chance of salvation. AA didn’t want them and the state offered only hospitalization or prison. Sympathetic to their need, Dederich scraped together some cash and rented an old store on whose front he painted the letters TLC, short for “Tender Loving Care.”
</p></div></div> <div class="bg" readability="14"><div class="padding" readability="23"><p> The store was a safe place in which drugs, alcohol, and violence were forbidden. But the reason for going there was Dederich himself. Following the LSD experiment, he had become an awesome presence. He held seminars now in which he would talk for hours on end, weaving psychological and philosophical insights together and ridiculing, cajoling, teasing, and harrying the addicts who surrounded him. And most amazingly, it seemed to be working. When one addict slurred the words <em>seminar</em> and <em>symposium</em> together, Dederich suddenly had a name for his project—Synanon. It was a word redolent of “sin,” “Zion,” and, of course, “Alcoholics Anonymous.”
</p></div></div> <div class="bg" readability="9.5"><div class="padding" readability="14"><p>“I knew something,” remembered Dederich, “and I wanted to transmit this to other people. I had the feeling I could really make people more comfortable.” He would choose to do this by making them profoundly uncomfortable. Combining AA’s teachings with the cursory knowledge he had of psychiatry and a heavy dose of Ralph Waldo Emerson’s 1841 essay “Self-Reliance,” Dederich took his natural loquacity and love of rhetorical combat and created a sort of moonshine therapy, a form of treatment that would live on long after Synanon was destroyed and Dederich was disgraced. It was known as the Game. 
</p></div></div> <div class="bg" readability="13.5"><div class="padding" readability="22"><p>The Game consisted of a dozen or so addicts sitting in a circle. One player would start talking about the appearance or behavior of another, picking out their defects and criticizing their character. But as soon as the subject of the attack tried to defend him- or herself, other players would join the barrage, unleashing a no-holds-barred verbal onslaught. Vulgarity was encouraged—“talk dirty and live clean,” said Dederich—and so the other members would accuse the defendant of real and imagined crimes, of being selfish, unthinking, of being a no-good, ugly, diseased cocksucker who was too weak to go straight and was too much of an asshole, junkie, cry-baby motherfucker to admit it. Faced with this unrelenting group assault, the recipient would eventually have little choice but to admit their wrongdoing and promise to mend their ways. Then the group would turn to the next person and begin all over again. 
</p></div></div> <p>The Game, ca. 1962. Here, a Synanon member is verbally attacked and accused of being “a lousy dope fiend mother.”</p> <p>Infuriated, she fights back, shouting down one of the other “players.”</p> <div class="bg" readability="12.5"><div class="padding" readability="20"><p>“The first time it hits you, it absolutely destroys you,” remembered a former Game player. “No matter how loud you scream, they can scream louder,” recalled another, “and no matter how long you talk, when you run out of breath they’re there to start raving again at you. And laughing.” Emotional catharsis was the aim. There were only two rules: no drugs, and no physical violence. It was vicious, but it actually seemed to work. “One cannot get up,” remarked Dederich, “until he’s knocked down.”
</p></div></div> <div class="bg" readability="10.5"><div class="padding" readability="16"><p>• • •</p> <p>In the late 1950s, the perils of drug addiction had become a national preoccupation. Everyday the newspapers spoke of the havoc the “dope monster” was causing the United States. The newly elected governor of New York, Nelson Rockefeller, declared that 50 percent of all crimes in his state could be linked to drugs. Thus, when Dederich moved Synanon to a huge red-brick building on the Santa Monica beachfront and began to populate it with drug addicts, it was unsurprising that it should raise the ire of his neighbors. The local government tried to shut Synanon down repeatedly. Dederich even went to prison for a month for a host of zoning violations, but this only increased public awareness of his “tough love” campaign. Donations started to flood in.
</p></div></div> <div class="bg" readability="13"><div class="padding" readability="21"><p>By now Syanon had a routine. New members quit cold turkey and were then slowly ushered into the building’s life. There was hard physical labor, constant mutual support, and the Game, played three times a week. Coffee, peanut butter sandwiches, and cigarettes were always available. There were classes in public speaking, art, drama, and music to keep the members entertained. (Some of the great jazz musicians of the age ended up at Synanon, cutting an album entitled <em>Sounds of Synanon</em>. The first song was entitled “C. E. D.” after Dederich’s initials.) Equally revolutionary was the interracial nature of the place, with Dederich leading the way through his marriage to Betty, a former heroin addict and prostitute who was African-American. </p></div></div> <div class="bg" readability="11"><div class="padding" readability="17"><p>As long as people worked—washing dishes, waxing floors, ironing laundry, painting walls, picking up food donations—they never had to leave. Synanon was also self-policed. You were expected to report those breaking the rules. Those who slacked off, or failed to tell on someone else, were taken to task in the Game. Those who smuggled in contraband were given a “haircut,” a private dressing down from a senior member. Repeated infractions led to banishment.
</p></div></div> <div class="bg" readability="10.5"><div class="padding" readability="16"><p>But perhaps Synanon’s greatest innovation was realizing that addicts knew more about addiction than medical specialists. The dope fiend, as Dederich insisted they be known, was painfully familiar with the tactics of denial and evasion that their colleagues used. What’s more, they shared the same language. There was no “we–they” in Synanon. Here, if you spoke about caps and bennies, turp and horse, everyone knew what you were talking about.
</p></div></div> <div class="bg" readability="11.5"><div class="padding" readability="18"><p>As for Dederich, he was never coy about his role. “I am considered a megalomaniac nut,” he declared. “Of course, this is true, but I’m not so crazy.” He freely admitted to populating Synanon’s board of directors with recovering addicts whom he could control. But no one doubted that this was wise and canny thinking; after all, these were dope fiends and Dederich was entering uncharted territory. Dederich predicted that within three to five years at Synanon, a dope fiend would be ready to “graduate” back to the outside world. No one doubted Dederich’s sincerity, nor worried about the ambiguous undertones to his most famous maxim, the one that he told to each new arrival at Synanon: “Today is the first day of the rest of your life.” 
</p></div></div> <div class="bg" readability="9"><div class="padding" readability="13"><p>• • •</p> <p>By 1969, Synanon had become a phenomenon. A Hollywood film had been made about it (one tagline read: “Dope fiends scream the truth ... about the house where they live together ... love together ... while they fight their way <em>back</em>!”). It had been referred to in the House of Representatives as “the Miracle on The Beach.” The psychologist Abraham Maslow had praised it, lauding its “no-crap therapy,” and it had become an accepted part of the rehabilitative landscape with courts now routinely sending addicts to Synanon as a condition of their parole.
</p></div></div> <p>Promotional poster for 1965 film about the “coolest corporation in 
the world.”</p> <div class="bg" readability="11"><div class="padding" readability="17"><p>Such was its fame, and such was the 1960s fascination with communal living, that it had started attracting not just addicts but fee-paying “squares,” people who wanted to live at Synanon simply to be a part of its self-actualization community. With a membership now close to one thousand, Synanon was buying and developing properties in Tomales Bay, north of San Francisco, and in the isolated town of Badger in Northern California. Yet as a reaction to the influx of new recruits, Synanon was growing increasingly proscriptive. 
</p></div></div> <div class="bg" readability="9.5"><div class="padding" readability="14"><p>When members stepped out of line now, the “haircuts” they received were literal ones, with men having their heads shaved for bad behavior and women being forced to wear stocking caps. Whereas sex was rampant in Synanon’s early days, now members had to ask a Synanon “elder” for permission to date and were forced to follow a strict and celibate courting ritual. “Glut raids” were routinely run on residents’ rooms to confiscate excessive personal possessions, and Dederich and his elders would instigate arbitrary new rules such as the “twenty-four-hour day” in which half of Synanon would go to work at night while the other half worked during the day. A Synanon police force patrolled the nearby streets looking for members who might be breaking the rules. 
</p></div></div> <div class="bg" readability="10"><div class="padding" readability="15"><p>The jazz saxophonist Art Pepper, who had spent years in San Quentin on drug charges, checked himself into Synanon in 1969. Suspicious of the culture of self-policing, not to mention the intentions of Dederich (whom he termed “the Old Wino”), he nevertheless understood the need for constant fluctuations in schedule. “Dopefiends and nuts can’t stand routine and when they get bored they have to do something crazy, so Synanon made the insanity. Themselves. The people that ran it caused the insanity.” 
</p></div></div> <p>Synanon’s house band playing for members and guests, 1962. Over the years, the group included numerous celebrated musicians, including guitarist Joe Pass.</p> <div class="bg" readability="10"><div class="padding" readability="15"><p>No matter how much changed at Synanon, the Game remained at its center. Indeed, even the cynical Pepper quickly became addicted to its verbal volleys. What’s more, Dederich had created variations on it. He now ordered groups to occasionally engage in a forty-eight-hour-long Game called the Trip. The lack of sleep, combined with endless hectoring and supplemented by the use of a Ouija board, led exhausted members to break down and experience hallucinations. Some even declared Dederich their god and savior.</p></div></div> <div class="bg" readability="10"><div class="padding" readability="15"><p>But Dederich was a troubled god. He was issuing ever more “containment” orders, forbidding residents to leave Synanon buildings and growing increasingly fevered in his demonization of those who quit the program, labeling them “splittees.” The fact was that he was seeing many of Synanon’s recent graduates relapse into drug use. Addicts would only stay clean, it seemed, if they remained under his direct control within the walls of the complex. Rather than encourage his members to graduate, Dederich was wondering if the best thing might be to make them stay within Synanon forever.</p> <p>• • •</p></div></div> <div class="bg" readability="11"><div class="padding" readability="17"><p>As his isolationist sentiments grew, Synanon increasingly began to act as a macrocosm of Dederich himself. When health problems forced him to give up sugar and refined grains, the offending foodstuffs were quickly removed from the Synanon kitchens. When he started running in place to try and lose weight, it quickly became compulsory for everyone to join him in this activity. When Dederich shaved his head, within hours everyone had shaved their head (or had it forcibly shaved) as a sign of solidarity. Most controversially, when Dederich’s doctors advised he give up smoking, an order was issued banning cigarettes. The gravity of this ruling should not be underestimated. Ever since the organization’s founding, coffee and cigarettes had been the only chemical buffer between recovering addicts and the abyss of addiction. The smoking ban led to 150 members quitting on the spot.</p></div></div> <div class="bg" readability="14"><div class="padding" readability="23"><p>But Dederich didn’t care. Synanon was growing rich. By 1972, there were 1,700 live-in residents, who were either paying monthly dues or working for almost nothing. Dederich was receiving bequests. A wealthy woman resident donated $1 million to the cause. Another member gave Synanon control of his mortgage company. Added to this was Synanon’s hugely profitable business manufacturing corporate merchandising gimmicks—ballpoint pens, wallets, T-shirts. It was the second largest such firm in the United States. Founded to get people off junk, Synanon was now creating it. Soon, Synanon owned properties across America; at Tomales Bay, it had a fleet of ships, and at the Badger property hundreds of motorbikes and trucks, not to mention an airstrip and a private plane. It was also there that a luxurious complex known as the Homeplace was being constructed. Boasting hot tubs and riding stables, this was where Dederich, his wife Betty, and some of Synanon’s elders could most often be found. The rank and file lived half a mile away in army-style barracks.</p></div></div> <div class="bg" readability="13"><div class="padding" readability="21"><p>There were battles along the way, but each one seemed to strengthen Synanon, and increase Dederich’s suspicion of the outside world. In 1972, the <em>San Francisco Examiner</em> published a series of critical articles about the organization’s increasingly eccentric character. Dederich sued. By this time, his group had a legal staff of forty-eight drawn from its residents, which included class-topping, Harvard-educated lawyers. Hearst Newspapers settled out of court for $2.6 million. Similarly, when the IRS challenged Synanon’s tax-exempt status, arguing that it was no longer a drug rehabilitation center but an experimental society, Dederich one-upped them and declared it a religion. This exempted the organization from all income tax and sidestepped the increasingly hostile attempts by various state legislatures to license it. Still more conveniently, it allowed Dederich to shrug off the difficult questions he was being asked by members as to when they might be “graduating” from Synanon. “Nobody ‘graduates’ from a religion,” wrote one of Synanon’s lawyers. Dederich’s apotheosis was complete.</p></div></div> <div class="bg" readability="10"><div class="padding" readability="15"><p>• • •</p> <p>In the summer of 1973, an incident would occur that would profoundly change the Synanon community forever. Dederich himself was taking part in a Game, but one female member was showing him no respect and kept interrupting his gnomic utterances. Infuriated, Dederich stood up, walked over to the woman, and poured a can of root beer over her head. 
</p></div></div> <p>Female Synanon members dance at an Oakland, California, press conference, 1975. Shaved heads became <em>de rigueur</em> after Dederich adopted the style.</p> <div class="bg" readability="10"><div class="padding" readability="15"><p>It was a small gesture of frustration, but the effect within Synanon was earth-shattering. No matter the other changes that had taken place, the rules of the Game had always been sacrosanct—no drugs, no violence. Now Dederich himself had broken one of them. Some wondered whether he’d gone crazy, but his more devoted followers preferred to see it as a sign, as a call to arms.</p></div></div> <div class="bg" readability="14"><div class="padding" readability="23"><p>Just as the outside world had been slow to embrace Synanon, it was now slow to let go of it, despite the turn away from non-violence. In fact, soon after the root beer incident, juvenile courts started sending troubled children to Synanon. Unlike the other residents, many of these children had no wish to change their ways, and in the past, this “Punk Squad,” as they became known, would have proved impossible to control. But unfortunately for them, Dederich had shown that the gloves were now off. When any member of the Punk Squad rebelled, they were brutally beaten. Some older members of Synanon balked at this, but they were forced to accept it or be Gamed out of the organization altogether. The shift to violence led to a spate of internal purges within the group. As Dederich’s wife Betty wrote, “We’re beginning to find some creeps amongst the squares.”</p></div></div> <div class="bg" readability="10.5"><div class="padding" readability="16"><p>The question of children at Synanon had been troubling Dederich even before the Punk Squad arrived. It had been usual for children born to families at Synanon to be separated from their parents and raised by the community as a whole. But Dederich now began to wonder whether this was too much effort. In a speech he gave on the Wire, an FM radio station he had installed at the Badger complex, he announced, “There’s no profit to this community in raising our own children … every baby that we indulge a Synanon female with takes up a bed and somewhere between $100,000 and $200,000 worth of energy.” To those who claimed they wanted to have a baby, he explained the experience was greatly overrated: “I understand it’s more like crapping a football than anything else.” It wasn’t long before Dederich came up with a practicable solution: all male members would receive vasectomies. Pregnant females were ordered to have abortions.</p></div></div> <div class="bg" readability="10.5"><div class="padding" readability="16"><p>Some agreed immediately, rushing to Synanon’s hospital. Others needed to be Gamed into it. Regarding the baby ban, Dederich opined, “Nothing is sacred just because it’s been done for a million years.” Curiously enough, only Dederich himself failed to receive the snip, but then he was having his own problems. In April 1977, Betty died and Dederich found himself alone. He immediately announced that he would accept applications from any women who wanted to marry him. Six applied and he eventually chose a thirty-one-year old. He was so delighted with the experiment that he ordered all married couples to take separation vows and pick a new mate every three years.</p></div></div> <div class="bg" readability="10"><div class="padding" readability="15"><p>Even with the sterilizations and the decimation of the family unit, Synanon continued to have trouble with its youngest members. Encouraged to play the Game from the age of four, many children chose to run away from the constant verbal and physical abuse. Farmers who lived on ranches near the Badger complex would often be awakened in the middle of the night by sobbing runaways seeking help. Nevertheless, locals’ accusations of child abuse and kidnapping failed to cause Dederich much anxiety, largely because the sheriff was firmly in his pocket, and both his deputies were loyal Synanon members.</p></div></div> <div class="bg" readability="10.5"><div class="padding" readability="16"><p>This sense of security notwithstanding, a doomsday philosophy was growing within Synanon. Dederich was increasingly espousing a “we–they” philosophy with the outside world, and his paranoia was infusing the group. Brawls over alleged slights were increasingly common between members and locals, leading Dederich to create a specially trained force to protect Synanon’s people and property. He termed them the “Imperial Marines” and watched on with glee as they practiced a special form of karate they had named “Syno-do.” But the Imperial Marines were only the tip of the iceberg. When you saw Synanon’s residents standing in line in their uniform of dungarees, with their clipped hair, you saw less a utopian community than an angry militia. This comparison was made even stronger when it was revealed that Synanon had bought over $300,000 worth of guns and ammunition.</p></div></div> <div class="bg" readability="11"><div class="padding" readability="17"><p>Despite Synanon’s intimidating stature, a few sought to shed light on its abuses. Paul Morantz, a crusading Los Angeles attorney, had sought to free members whose families believed had been kept against their will. In September 1977, he won a $300,000 judgment against Synanon for abducting and brainwashing a member. The news sent Dederich into a rage. Soon afterwards, he spoke on the Wire, announcing the organization’s “New Religious Posture.”</p></div></div> <div class="bg" readability="9"><div class="padding" readability="13"><p>“We’re not going to mess with the old-time, turn-the-other-cheek religious postures. Our religious posture is: Don’t mess with us—you can get killed dead, literally dead! ... I am quite willing to break some lawyer’s legs and next break his wife’s legs and threaten to cut their child’s arm off. That is the end of that lawyer. That is a very satisfactory, humane way of transmitting information. … I really do want an ear in a glass of alcohol on my desk. Yes indeed.” The next month Paul Morantz was bitten on the hand by a four-and-a-half-foot-long rattlesnake that had been placed in his mailbox. He barely survived. Two members of the Imperial Marines were accused of conspiracy to commit murder. </p></div></div> <div class="bg" readability="14.5"><div class="padding" readability="24"><p>However, thanks to the work of Morantz and that of the tiny local newspaper, <em>The Point Reyes Light</em>, the wider world started to take notice. It was not an easy task though. When <em>Time</em> magazine and NBC ran critical profiles on Synanon, their executives were bombarded with death threats. A former board director of the group who spoke out against Dederich found his guard dog hanged outside his house. Another former member who wanted to visit one of the Synanon complexes was tied to a post and pistol-whipped for being a “splittee.” Tension was building, it seemed an explosion of extreme violence could happen at any time, and then on 17 November 1978, over nine hundred people were killed and an American congressman assassinated in what became known as the Jonestown Massacre in Guyana. Suddenly, the authorities could no longer look the other way at what, in the cold light of day, was undoubtedly a cult. Within days, Synanon was raided by the police. Incriminating material was found, including a recording of Dederich’s “New Religious Posture.” Dederich himself was nowhere to be found, although he was later tracked down to a Synanon property in Arizona. When over thirty police officers burst into the house, they found the sixty-five-year-old Dederich sitting by himself in a stupor. In front of him was an empty bottle of whiskey.
</p></div></div> <cite readability="15.161882893226"><div class="padding" readability="35.214695752009"><p><span class="text-orange">Bibliography</span><br /><span class="sources">Barbara Leslie Austin, <em>Sad Nun at Synanon</em> (New York: Holt, Rinehart, and Winston, 1970).</span><br /><span class="sources">Daniel Casriel, <em>So Fair a House: The Story of Synanon</em> (Englewood Cliffs, NJ: Prentice-Hall, 1963).</span><br /><span class="sources">Betty Dederich, <em>No Time for “Yeah But”</em> (Marshall, CA: Synanon, 1978).</span><br /><span class="sources">S. Guy Endore, <em>Synanon</em> (Garden City, NY: Doubleday, 1968).</span><br /><span class="sources">Dave Mitchell, Cathy Mitchell, &amp; Richard Ofshe, <em>Light on Synanon: How a Country Weekly Exposed a Corporate Cult—and Won the Pulitzer Prize</em> (New York: Seaview, 1982).</span><br /><span class="sources">Paul Morantz, <em>Escape: My Lifelong War Against Cults</em> (Los Angeles: Figuroa Press, 2012). </span><br /><span class="sources"><a href="http://www.paulmorantz.com" target="_blank">www.paulmorantz.com</a>.</span><br /><span class="sources">William F. Olin, <em>Escape from Utopia: My Ten Years in Synanon</em> (Santa Cruz, CA: Unity Press, 1980).</span><br /><span class="sources">Art Pepper &amp; Laurie Pepper, <em>Straight Life: The Story of Art Pepper</em> (New York: Schirmer Books, 1979).</span><br /><span class="sources">Lewis Yablonsky, <em>The Tunnel Back: Synanon</em> (New York: Macmillan, 1965).</span></p> </div></cite> <div class="bio" readability="12.453074433657"><p><a href="http://www.cabinetmagazine.org/contributors/pendle_george.php">George Pendle</a> is a writer based in Washington, DC. His books include <em>Strange Angel: The Otherworldly Life of Rocket Scientist John Whiteside Parsons</em> (Harcourt, 2005) and <em>Death: A Life</em> (Three Rivers, 2008). A collection of his essays entitled <em>Happy Failure</em> (An Art Service) will be released later this year.
</p></div> Fri, 16 Nov 2018 22:03:13 +0000 http://www.cabinetmagazine.org/issues/48/pendle.php AMD Discloses Initial Zen 2 Details https://fuse.wikichip.org/news/1815/amd-discloses-initial-zen-2-details/ https://fuse.wikichip.org/news/1815/amd-discloses-initial-zen-2-details/ &#13; <p>Earlier this month AMD made their first disclosure of <a href="https://en.wikichip.org/wiki/amd/microarchitectures/zen_2">Zen 2</a>, their next-generation core microarchitecture for desktop and server chips. Along with Zen 2, AMD also unveiled initial details of their next-generation server chips, codename <a href="https://en.wikichip.org/wiki/amd/cores/rome">Rome</a>.</p> <h2>Zen 2</h2> <p><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen2-logo.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen2-logo-1024x236.png" alt="" width="640" height="148" class="aligncenter size-large wp-image-1816" srcset="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen2-logo-1024x236.png 1024w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen2-logo-300x69.png 300w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen2-logo-768x177.png 768w" sizes="(max-width: 640px) 100vw, 640px"/></a><br/><a href="https://en.wikichip.org/wiki/amd/microarchitectures/zen">Zen 2</a> succeeds <a href="https://en.wikichip.org/wiki/amd/microarchitectures/zen">Zen</a>/<a href="https://en.wikichip.org/wiki/amd/microarchitectures/zen%2B">Zen+</a>. The design targeted <a href="https://en.wikichip.org/wiki/7_nm_lithography_process">TSMC 7 nm process node</a>. AMD evaluted both <a href="https://en.wikichip.org/wiki/10_nm_lithography_process">10 nm</a> and 7 nm. The choice to go with 7 nm boiled down to the much lower power and higher density they were able get. AMD claims 7-nanometers delivers 2x the density and offers 0.5x the power at the same performance or &gt;1.25x the performance at the same power (note that at Computex AMD’s slide said “1.35x”). Zen 2-based chips are currently sampling and are on track to be delivered to market in 2019.</p> <p>AMD has made a large set of enhancements to Zen 2. To feed the widened execution units which were improved in throughput, the front-end had to be adjusted. For that reason, the branch prediction unit has been reworked. This includes improvements to the prefetcher and various undisclosed optimizations to the instruction cache. The µOP cache was also tweaked including changes to the µOP cache tags and the µOP cache itself which has been enlarged to improve the instruction stream throughput. The size of the cache on Zen was 2,048 entries. The exact details of Zen 2 changes were not disclosed at this time.</p> <p>The majority of the changes to the back-end involve the floating-point units. The most major change is the widening of the data path which has been doubled in width for the floating-point execution units. This includes the load/store operations as well as the FPUs. In Zen, AVX2 is fully supported through the use of two 128-bit micro-ops per instruction. Likewise, the load and store data paths were 128-bit wide. Every cycle, the FPU is capable of receiving 2 loads from the load/store unit, each up to 128 bits. In Zen 2, the data path is now 256 bits. Additionally, the execution units are now 256-bit wide as well, meaning 256-bit AVX operations no longer need to be cracked into two 128-bit micro-ops per instruction. With 2 256-bit FMAs, Zen 2 is capable of 16 FLOPS/cycle, matching that of Intel’s <a href="https://en.wikichip.org/wiki/intel/microarchitectures/skylake_(client)">Skylake client core</a>.</p> <figure id="attachment_1819" class="wp-caption aligncenter"><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/zen_zen2_changes_initial_disclosure.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/zen_zen2_changes_initial_disclosure-1024x556.png" alt="" width="640" height="348" class="size-large wp-image-1819"/></a><figcaption class="wp-caption-text">Initial changes disclosed by AMD. (WikiChip)</figcaption></figure><p>AMD stated that Zen 2 IPC has been improved along with an increase in both the dispatch and retire bandwidth, however, the fine details were not disclosed. On the security side, Zen 2 introduces in-silicon enhanced Spectre mitigations that were originally offered in firmware and software in Zen.</p> <h2>Rome</h2> <blockquote><p>Some people called it [chiplets] gluing things together; we called it the next-generation of system design. – Dr. Lisa Su, AMD President and CEO</p></blockquote> <p><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-rome.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-rome-1024x237.png" alt="" width="640" height="148" class="aligncenter size-large wp-image-1818" srcset="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-rome-1024x237.png 1024w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-rome-300x69.png 300w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-rome-768x177.png 768w" sizes="(max-width: 640px) 100vw, 640px"/></a></p> <p>AMD’s second-generation <a href="https://en.wikichip.org/wiki/epyc">EPYC</a> is codename <a href="https://en.wikichip.org/wiki/amd/cores/rome">Rome</a>, the successor to <a href="https://en.wikichip.org/wiki/amd/cores/naples">Naples</a>. The two are socket and platform compatible. Note that Milan, Rome’s sucessor, is also socket compatible. Rome still uses a multi-chip approach to scale up the core count but the system design itself has changed quite radically from the prior generation. In Naples, AMD scales up the 8-core design, called a <a href="https://en.wikichip.org/wiki/amd/zeppelin">Zeppelin</a>, to 32 cores by stiching together four of those SoCs through their proprietary interconnect called the <a href="https://en.wikichip.org/wiki/amd/infinity_fabric">Infinity Fabric</a>. This method provided eight memory channels and 128 PCIe lanes distributed accross all the dies.</p> <figure id="attachment_1820" class="wp-caption aligncenter"><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/amd_zeppelin_4-die.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/amd_zeppelin_4-die-1024x901.png" alt="" width="500px" class="size-large wp-image-1820" srcset="https://fuse.wikichip.org/wp-content/uploads/2018/11/amd_zeppelin_4-die-1024x901.png 1024w, https://fuse.wikichip.org/wp-content/uploads/2018/11/amd_zeppelin_4-die-300x264.png 300w, https://fuse.wikichip.org/wp-content/uploads/2018/11/amd_zeppelin_4-die-768x676.png 768w, https://fuse.wikichip.org/wp-content/uploads/2018/11/amd_zeppelin_4-die.png 1200w" sizes="(max-width: 1024px) 100vw, 1024px"/></a><figcaption class="wp-caption-text">An AMD Zen-based Epyc chip uses four dies. (WikiChip)</figcaption></figure><p>With Rome, AMD is taking the idea of chiplets further. Similar to what they <a href="https://fuse.wikichip.org/news/1569/amd-announces-threadripper-2-chiplets-aid-core-scaling/">initially started</a> with Threadripper 2, Rome has compute dies and an I/O die. However, this time, AMD took out the core execution blocks and moved them to new compute dies, leveraging TSMC’s 7 nm process and taking advantage of the lower power and higher density. The compute dies are then connected to a centralized I/O die that manages the I/O and the memory. The much bigger I/O die is manufactured on GlobalFoundries mature 14 nm process where most the power and density cannot be realized.</p> <p><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen2-rome.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen2-rome-1024x240.png" alt="" width="640" height="150" class="aligncenter size-large wp-image-1821" srcset="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen2-rome-1024x240.png 1024w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen2-rome-300x70.png 300w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen2-rome-768x180.png 768w" sizes="(max-width: 640px) 100vw, 640px"/></a></p> <p>In total, there are nine dies. One I/O die and eight compute dies – each with 8 Zen 2 cores. Neither the details of the individual compute dies nor the I/O die were disclosed. There are a fair bit of challenges involved in this kind of design and it would be interesting to see how they were addressed. The I/O die creates deterministic and unified latencies across the entire chip, but it would potentially affect best-case/sensitive scenarios. The package is organized in four pairs of compute dies similar to our diagram below. It’s worth noting that each pair of compute dies are packaged tightly together on the organic substrate, indicating there might be very short traces going between them.</p> <figure id="attachment_1822" class="wp-caption aligncenter"><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/naples_to_rome_initial_changes_mcp.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/naples_to_rome_initial_changes_mcp-1024x232.png" alt="" width="640" height="145" class="size-large wp-image-1822"/></a><figcaption class="wp-caption-text">Rome chiplet design based on initial details (WikiChip)</figcaption></figure><p><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-rome-chip.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-rome-chip-1024x576.png" alt="" width="400px" class="aligncenter size-large wp-image-1827" srcset="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-rome-chip-1024x576.png 1024w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-rome-chip-300x169.png 300w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-rome-chip-768x432.png 768w" sizes="(max-width: 1024px) 100vw, 1024px"/></a></p> <p>With eight octa-core compute dies, Rome can offer up to 64 cores and 128 threads, effectively doubling/quadrupling (AVX2) the throughput of first-generation EPYC. Although Rome stays with 128 PCIe lanes, it brings new supports for PCIe Gen 4, doubling the transfer rate from 8 GT/s to 16 GT/s. There are eight DDR4 memory channels supporting up to four terabytes of DRAM per socket. One interesting detail AMD disclosed with their GPU announcement is that the infinity fabric now supports 100 GB/s (BiDir) per link. If we assume the <a href="https://en.wikichip.org/wiki/amd/infinity_fabric">Infinity Fabric</a> 2 still uses 16 differential pairs as with first-generation IF, it would mean the IF 2 now operates at 25 GT/s, identical to <a href="https://en.wikichip.org/wiki/nvidia/nvlink">NVLink 2.0</a> data rate. However, since AMD’s IF is twice as wide, it provides twice the bandwidth per link over Nvidia’s NVLink.</p> <p><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-if-links.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-if-links-1024x239.png" alt="" width="640" height="149" class="aligncenter size-large wp-image-1825" srcset="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-if-links-1024x239.png 1024w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-if-links-300x70.png 300w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-preview-if-links-768x179.png 768w" sizes="(max-width: 640px) 100vw, 640px"/></a></p> <p>One of the things <a href="https://en.wikichip.org/wiki/epyc">EPYC</a> brought is <a href="https://en.wikichip.org/wiki/x86/sme">SME</a> and on top of that <a href="https://en.wikichip.org/wiki/x86/sme#Secure_Encrypted_Virtualization">SVE</a> which extended SME to AMD-V, allowing individual VMs to run SME using their own secure keys. With Rome, AMD says the number of keys (and thus VMs) support has also been increased.</p> <figure id="attachment_1828" class="wp-caption aligncenter"><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/zen-2-rome.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/zen-2-rome-1024x576.png" alt="" width="450px" class="size-large wp-image-1828" srcset="https://fuse.wikichip.org/wp-content/uploads/2018/11/zen-2-rome-1024x576.png 1024w, https://fuse.wikichip.org/wp-content/uploads/2018/11/zen-2-rome-300x169.png 300w, https://fuse.wikichip.org/wp-content/uploads/2018/11/zen-2-rome-768x432.png 768w" sizes="(max-width: 1024px) 100vw, 1024px"/></a><figcaption class="wp-caption-text">Rome server on display at SuperComputing 18 (WikiChip)</figcaption></figure><h3>I/O Die</h3> <p>There is a lot of mystery surrounding the capabilities of the I/O die and AMD’s plan for the future. By moving all the “redundant components”, such as the I/O and southbridge, from the compute die to the I/O die, AMD has opened up their design to some intriguing possibilities. Since all the controls can be found in the centralized I/O die, it becomes possible to swap out the compute dies with other types of logic such an FPGA (e.g., from Xilinx) or a GPU. In Naples, this would have meant sacrificing some of the I/O or memory but with Rome, this is no longer the case. AMD has not announced any such plans, but the option is there.</p> <figure id="attachment_1826" class="wp-caption aligncenter"><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/naples_to_rome_mpc-potential-ideas.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/naples_to_rome_mpc-potential-ideas-658x1024.png" alt="" width="300px" class="size-large wp-image-1826" srcset="https://fuse.wikichip.org/wp-content/uploads/2018/11/naples_to_rome_mpc-potential-ideas-658x1024.png 658w, https://fuse.wikichip.org/wp-content/uploads/2018/11/naples_to_rome_mpc-potential-ideas-193x300.png 193w, https://fuse.wikichip.org/wp-content/uploads/2018/11/naples_to_rome_mpc-potential-ideas-768x1195.png 768w, https://fuse.wikichip.org/wp-content/uploads/2018/11/naples_to_rome_mpc-potential-ideas.png 1020w" sizes="(max-width: 658px) 100vw, 658px"/></a><figcaption class="wp-caption-text">Potential designs (WikiChip)</figcaption></figure><h2>Roadmap</h2> <p><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen3-4-roadmap.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen3-4-roadmap-1024x242.png" alt="" width="640" height="151" class="aligncenter size-large wp-image-1823" srcset="https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen3-4-roadmap-1024x242.png 1024w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen3-4-roadmap-300x71.png 300w, https://fuse.wikichip.org/wp-content/uploads/2018/11/next-horizon-zen3-4-roadmap-768x181.png 768w" sizes="(max-width: 640px) 100vw, 640px"/></a><br/>The key takeaway from AMD’s event is their roadmap. A predictable roadmap helps improve customers confidence in the platform. AMD wanted to show that they are capable of laying out a roadmap and execute on it. To that end, AMD expects Zen 2 to launch in 2019. Zen 3 is on track and Zen 4 is at the design completion phase.</p> <figure id="attachment_1829" class="wp-caption aligncenter"><a href="https://fuse.wikichip.org/wp-content/uploads/2018/11/shasta-naples-blade.png"><img src="https://fuse.wikichip.org/wp-content/uploads/2018/11/shasta-naples-blade-1024x768.png" alt="" width="500px" class="size-large wp-image-1829" srcset="https://fuse.wikichip.org/wp-content/uploads/2018/11/shasta-naples-blade-1024x768.png 1024w, https://fuse.wikichip.org/wp-content/uploads/2018/11/shasta-naples-blade-300x225.png 300w, https://fuse.wikichip.org/wp-content/uploads/2018/11/shasta-naples-blade-768x576.png 768w, https://fuse.wikichip.org/wp-content/uploads/2018/11/shasta-naples-blade-134x100.png 134w" sizes="(max-width: 1024px) 100vw, 1024px"/></a><figcaption class="wp-caption-text">Cray Shasta blade with AMD EPYC on display at SuperComputing 18. (WikiChip)</figcaption></figure><div class="mistape_caption"><span class="mistape-link-wrap"><a href="http://mistape.com" target="_blank" rel="nofollow" class="mistape-link mistape-logo"/></span> <p>Spotted an error? Help us fix it! Simply select the problematic text and press Ctrl+Enter to notify us.</p> </div> &#13; &#13; Sun, 18 Nov 2018 16:55:44 +0000 David Schor https://fuse.wikichip.org/news/1815/amd-discloses-initial-zen-2-details/ The Future of Aging Just Might Be in Margaritaville https://www.nytimes.com/interactive/2018/11/14/magazine/tech-design-longevity-margaritaville.html https://www.nytimes.com/interactive/2018/11/14/magazine/tech-design-longevity-margaritaville.html <p class="paragraph"><strong>Off a vacant</strong> stretch of highway in Daytona Beach, Fla., a line began to form outside the sales center for the first Latitude Margaritaville “55 and better” community. Those waiting dragged folding chairs, coolers, tents and dog-eared brochures featuring numbered sites that, in just over 24 hours, they could stake a claim to for a $10,000 deposit. The mood, shortly after 8 a.m. one Sunday last November, was festive, ecstatic even. Drinks flowed, pizza appeared, a steel-drum band played into the balmy night. Some neighbors in the 300-person queue liked each other so much that they decided to become actual neighbors, switching their site choices to live closer together. A sense of destiny seemed to guide many of their decisions. Karen Goodwin, 55, a homemaker, had won the exact amount of the down payment a few weeks earlier in a Domino’s sweepstakes. Matt Kelly, 62, a retired firefighter, had been chipping ice off his shingles in Orange County, N.Y., when a chunk broke loose in the shape of the Sunshine State, which he took as a sign.</p> <p class="paragraph">“I never thought I’d be in a 55-plus community,” Ruth Kelly, 61, a former real estate agent and Matt’s wife, said the following September. The three of us were sitting at a table in the dining room of the home the Kellys had secured on Cool Breeze Drive, a single-family unit with the L-shaped lanai that Ruth had had her eye on. “Being in real estate, I didn’t think I would do what we did, wait in line for 11 hours. I always told my customers: ‘Never buy in Phase 1. Never buy sight-unseen.’ I did all of that. But I never once had doubt. Not once did I feel that way. It was meant to be, I really believe that.”</p> <p class="paragraph">Outside, under an endless blue sky, a parade of trucks bore the trappings of former homes from as far away as Hawaii, Canada and El Salvador to sorbet-colored dwellings with emerald green lawns. At the entrance to the gated enclave, past a “Barkaritaville” dog park, beeping excavators moved dirt around what would soon be a town-square for concerts and dancing, surrounded by a state-of-the-art workout center, a restaurant and a walk-in pool with cabanas and a bar. It was impossible to stand on their cement foundations — which I had, in fact, done that morning — and not see a frontier settlement being carved into an expanse of subtropical wetland. The real frontier here, though, was not the surrounding wilderness but a hitherto uncolonized stretch of time: the multiple decades that more and more Americans can expect to live in better and better health after they retire. What will these pioneers do? Who will they become? And how will that, in turn, alter the course of human history?</p> <p class="paragraph">To be sure, Margaritaville is not representative of how most of us will spend our retirement years. Fewer than 14 percent of Americans 75 and older occupy some form of senior housing today. Three-quarters of those over 50 say they would prefer not to move at all. And untold numbers of seniors who might need or want to enter an age-restricted or assisted-living community won’t be able to afford to do so; 30 percent of those 65 and older have an annual income below $23,000, according to a study by the Kaiser Family Foundation. The least-expensive homes in Margaritaville are more than 10 times that, before the monthly association fee of roughly $200 — and those sums don’t include meals or care. (For statewide comparison, a private room in a skilled-nursing facility has a median cost of $9,000 per month, and in an assisted-living residence, $3,500 per month, according to LeadingAge Florida, an association of elder-care organizations. Continuing-care communities that guarantee all levels of lifetime care on-site have charges that range from $2,500 to $5,400 per month, plus substantial entry fees.)</p> <figure class="media photo large" data-id="100000006209726" data-slug="18mag-longevity-slide-GAJ9"> </figure><p class="paragraph">Like all pioneer settlements, however, Margaritaville is not just a place but an idea — an imagined utopia, in this case inspired by a Jimmy Buffett song’s reference to a frozen cocktail. Historically, such extreme aspirational-lifestyle experiments have had an outsize influence on our cultural imagination: It would be hard to call Jamestown, one cradle of our democracy, a practical project; same goes for the squalid boomtowns of the Gold Rush, which helped define the American dream. What intrigued me about Margaritaville was the specificity of its promise — a retirement based on music Buffett has described as “drunken Caribbean rock ‘n’ roll” — and the fact that it is still under construction. I could actually meet the early colonists as they went about pursuing their vision.</p> <p class="paragraph">Gaining access to Margaritaville as a reporter had not been easy; several feet away, politely pretending not to listen to my conversation with the Kellys, was a Margaritaville brand ambassador and two public relations specialists for Minto Communities, the developer (Latitude Margaritaville is their collaboration). My visit had been approved at the highest corporate levels and my subjects preselected and scheduled in 30-minute time slots. All brands cultivate their image, but building communities for seniors typically requires a special degree of staging. “Architects are basically set designers,” David Dillard, the president of D2 Architecture, a senior-housing firm, told me. “We fight to diminish any icons of this being a senior place. We literally hide things. Inside the community — this is part of our struggle — you don’t want to see your future.”</p> <p class="paragraph">Or at least not the part that includes inevitable death, possibly preceded by mental or bodily deterioration. The problem is that in order to avoid considering those eventualities, we tend to avert our collective gaze from the future altogether — and in doing so, risk missing our opportunity to stage a more enjoyable and meaningful one. Consequently, our concept of senior housing is often dystopic: a quarantining of those who can no longer care for themselves and are of no “use” to society. To purchase a home in Margaritaville, on the other hand, is to aggressively reimagine the aging process as a ticket to an island paradise, which may prove to be willfully naïve or ingeniously farsighted — or both.</p> <p class="paragraph"><strong>If you make</strong> it to your 65th birthday in the U.S. today, you can expect about 20 more years and have a good chance at 30. The fastest-growing segment of the population, in fact, is between the ages of 85 and 94. But longevity alone will not transform senior housing, an industry that experts warn is about to experience a revolution for which it is ill prepared. The truly transformative trend is about proportions. The Census Bureau projects that in 2034, for the first time ever, people 65 and older will outnumber those under 18. Americans are living longer and having fewer children, and fewer immigrants are showing up.</p> <figure class="media photo large" data-id="100000006209727" data-slug="18mag-longevity-slide-0RQO"> </figure><p class="paragraph">A similar demographic shift is underway around the globe, and no one seems to have a solid plan for addressing it. “It really comes down to two questions: Where are we going to live, and who is going to take care of us?” says Andrew Carle, founding director of the program in senior-housing administration at George Mason University. “The rapidly aging worldwide population will affect us more than global warming, I think. It’s the seminal event of the 21st century.” As Carle and many others see it, there are only two possible answers, which will need to be deployed in tandem to avoid a social and economic collapse: first, develop technology that will help seniors live independently longer, and second, persuade them to congregate so that they can be cared for more effectively, by humans or robots or both. The latter will be a more difficult sell than it was for those in the “silent generation,” who have been loath to complain as long as they feel reasonably safe and comfortable. Baby boomers came of age in the consumer culture of the 1950s and are comparison shoppers who expect to have agency and choice. “We’re the same generation that invented 14 flavors of Coca-Cola,” Carle says. “So we want more flavors. We’ve blown up the portfolio of every product we’ve touched.”</p> <p class="paragraph">In addition to senior communities organized around faith, military service and civic fraternities, there are now retirement cruise ships being planned for the superwealthy and floating condos for presale that will go up and down the Mississippi. The Villages, outside Orlando, has 52 golf courses and a new enrichment program that offers courses ranging from scuba diving to current affairs. Increasingly developers are partnering with schools like Duke and the University of Florida; Arizona State University recently broke ground on a 20-story senior-living facility it calls “the world’s coolest dorm.” “There’s a lot of experimentation with models,” Bob Kramer, the founder of the National Investment Center for Seniors Housing &amp; Care, told me. “There’s no model that’s really caught on yet.” (A proposed nudist community has fallen through.) But existing facilities have been notoriously slow to innovate. “Change things!” David Dillard says. “Senior housing are sluggards in that respect.”</p> <p class="paragraph">What needs remodeling can be depressingly obvious: Cafeterias and set mealtimes are out; multiple restaurants, from fine dining to deli takeout, are in. But the bigger question is whether redesigning the physical environment where seniors live can redefine the way we experience aging itself. Biologically speaking, we know very little about what aging is or how it works. It isn’t fatal, technically, but in developed countries it is the greatest risk factor for cancer, dementia and heart disease. It is a universal experience across the animal kingdom, one that unfolds uniquely in each individual. And while we think of other life phases as having a particular purpose — childhood is about growing and learning; adulthood is about procreating and producing — we have not defined old age as clearly. (In fact, apart from humans, only a few species of cetacean live beyond menopause.) “We have no shared collective articulation for what later life is for, what the value of living longer is, except not dying,” says Elana Buch, an anthropologist at the University of Iowa who studies aging and labor inequality in the U.S. “We don’t have any conversations about what the point of it is, so we reproduce it from other parts of life — doing things you didn’t get to do before.”</p> <p class="paragraph">There’s only so much that polling can tell us about how older people envision their coming decades. In a recent survey, the AARP found that, when it comes to growing old, people 55 and older looked most forward to having free time for family, friends and activities they enjoy; they most feared dementia and losing independence. What older people value in a place to live is much the same as it is for people in other life stages: well-maintained streets and sidewalks, safe parks. All of which seems to suggest a desire to stay connected to their current community — but says little about what their changing role within that community might be.</p> <figure class="media photo large" data-id="100000006209719" data-slug="18mag-longevity-slide-OX1W"> </figure><p class="paragraph"><strong>When Jimmy Buffett </strong>rang the doorbell of the home on Island Breeze Avenue, he was greeted by Al Murdock, an energetic 89-year-old with a firm handshake.</p> <p class="paragraph">“I’m Jimmy Buffett,” Jimmy Buffett said.</p> <p class="paragraph">“No, you’re not!” said Al’s wife, Fran, who assumed this was a practical joke.</p> <p class="paragraph">Despite being the second household to arrive in Latitude Margaritaville, the Murdocks, like many residents, are not Parrotheads, as superfans of Buffett’s music are known. And yet, as the performer’s appearance at their door illustrated, the man and his brand are embedded in the fiber of the place, from the Miami Beach scent floating through the models to the Margaritaville® Key West™ Frozen Concoction Makers® that the sales team sets up in every finished home. In fact, Buffett, 71, bought a Bimini model (two beds, 2.5 baths) and was ostensibly calling on his new neighbors (P.R. team in tow). More than anything, I was curious how residents who were not connoisseurs of Buffett’s music had decided to embrace — quite possibly for the rest of their lives — such a particular atmosphere. “We had to downsize,” Fran said. “Didn’t know if it would need to be assisted living. Because I’m 92. People were saying that’s what I need. We looked at some of the places, and you’d see people playing bingo and we thought, That’s not what we want to do.” They happened to drive by the Margaritaville sales center and stopped in on a whim; a greeter showed them around. “Well, he had a sales pitch that you wouldn’t believe,” Fran went on. “And the next thing we know, Al’s in line at 3 a.m. waiting to get us a model.”</p> <p class="paragraph">Al added, “We wanted to meet more people. In most subdivisions, you know the names of people on both sides and maybe across the street. We know people all over the block and where they came from and the jobs they had. There’s a feeling of neighborhood that I haven’t experienced since I was a kid.”</p> <figure class="media photo large" data-id="100000006209715" data-slug="18mag-longevity-slide-IFX4"> </figure><p class="paragraph">More than half of Americans — young people more so than older people, in fact — report feeling that “no one knows them well,” a recent Cigna survey suggests. But for baby boomers, according to a new report by the Joint Center for Housing Studies at Harvard, the problem of social isolation has become alarming, thanks to the generation’s inclination to “age in place,” especially in the suburbs. “In these low-density areas, it’s harder to meet your neighbors by accident, by walking in the neighborhood or bumping into them at a coffee shop,” Jennifer Molinsky, the study’s lead author, told me. And the evidence that social ties improve health for older people is significant: A recent study by the AARP Public Policy Institute and Harvard and Stanford Universities estimated that isolation among older adults costs Medicare an additional $6.7 billion a year.</p> <p class="paragraph">All the residents I met shared Al’s desire for a “feeling of neighborhood” their previous homes lacked. Lana Byer, 71, moved in in April. “Everyone looks out for everyone else,” Byer said. “That was missing in the place I was living. Frankly, it’s exceeded my expectations. It’s so much better. It’s a joy to get up in the morning.” Byer had owned a court-reporting firm in Pittsburgh and is divorced. In other communities, she felt excluded by married couples, but that has not happened here. “There were places I’ve lived where, if my daughter didn’t call me, no one would know I was alive,” she said. “Here, they know I’m alive. It’s like living at a resort.” That morning, she’d come from a nutrition group. Later, on her way to paint wine glasses, she planned to stop by happy hour.</p> <p class="paragraph">I wondered if there was a chance that the feeling of being on a perpetual vacation would get old after a while? “Only if you do,” she said.</p> <p class="paragraph">Byer is far from the only single woman in Margaritaville. Rhonda Harvey, 62, lost her husband in an accident on Interstate 4 last year. Afterward, her mother, Ethel, 85, moved in. “I had 12 acres,” Harvey said. “Now there’s almost 12 feet between here and the next house.” Harvey works as the chief operating officer of a behavioral health care agency with nearly 900 employees. Ethel, who is home alone during the day, feels safer with a guardhouse and nearby neighbors. Both are from Kentucky, where one of Harvey’s daughters still lives. “In some ways I think you could say the community that’s intentional here is reminiscent of small towns there where that’s just the way it was,” she said.</p> <figure class="media photo large" data-id="100000006209722" data-slug="18mag-longevity-slide-A4FG"> </figure><p class="paragraph">Harvey’s career requires her to understand basic human needs. “In social work, there are three things for good mental health,” she told me. “Something to do, something to love, something to hope for.” Harvey plainly had the first two: she worked full time; she loved her family. But what did she hope for?</p> <p class="paragraph">“This community fully developing,” she said. “I think that’s one thing we all hope for.”</p> <p class="paragraph"><strong>What does a</strong> successful community look like? Are its features universal or situational? In “The Republic,” Plato sketches an ideal city, ruled by a gentle philosopher-king with scant interest in power. It occurred to me that Margaritaville, too, is ruled by a benevolent philosopher. His decrees are painted on driftwood and hung everywhere: “It’s 5 O’Clock Somewhere,” “Livin’ for the Weekend.” Al Murdock, it turned out, had thought a lot about this iconography. At the turn-in, a Margaritaville sign is affixed to an enormous ladderlike structure — “a lifeguard tower,” Al said. “In Jimmy Buffett’s mind, this is an island. It’s a state of mind surrounded by land.”</p> <p class="paragraph">“Al bought his autobiography,” Fran said. “That’s where he’s getting all this information.”</p> <p class="paragraph">On the surface, the state of mind Buffett advertises seems purposefully mindless: grab a drink, kick back, relax. But Al read the invitation differently. By way of explanation he gave me two poems he’d written about “latitude” as freedom from expectation and opportunity for change; “paradise” was simply “where neighbors help their neighbors/And their differences forgot.”</p> <figure class="media photo large" data-id="100000006209723" data-slug="18mag-longevity-slide-48I9"> </figure><p class="paragraph">For all their paradisal appeal, islands are also isolated. Many researchers and designers I spoke with described a future in which seniors will want to be more integrated with other age groups, not less, and the challenge will be making senior-housing communities, which exclude younger people to create an inclusive place for older people, more porous.</p> <p class="paragraph">“I go to so many zoning meetings where people will say on the public record, ‘I don’t want seniors in the center of my community,’ ” Max Winters, a planner and designer at Perkins Eastman, said. Winters is in his late 20s; the co-leader of the firm’s senior-housing group, Dan Cinelli, is in his mid-60s. They are encouraging their clients to share square footage with companies like day spas, restaurants and coffee shops that appeal to a wider audience. The physical spaces could be adjusted to accommodate seniors — the lighting tweaked to reduce glare on thicker glasses, the acoustics tuned for the hard of hearing — while remaining inviting to millennials, too. “We’re seeing more adult children wanting to visit their parents in those settings,” Cinelli said. “Whereas 20 years ago, it was, ‘Oh, I’ll come get you and take you to a nice restaurant.’ Kids and grandkids want to come because it’s got a good vibe.”</p> <p class="paragraph">Technology, meanwhile, is poised to shift how and where care is delivered. At some point, automated cars will let people who are no longer able to drive remain mobile longer. Nicole Werner, an industrial and systems engineer at the University of Wisconsin-Madison, is preparing to pilot a social platform that would help people caring for a relative at home to organize assistance from a network of friends and relatives without having to ask for it. The software would learn routines and assign tasks automatically. “If you live in a city and you’re rich, it’s easy,” Werner, who plans to test the platform with people living with dementia in low-income rural and urban households, told me. “Disadvantaged groups have community. You can harness the people around you.”</p> <p class="paragraph">Still, it will be hard to change how we value old age without changing how we value the labor needed to support it — which, right now, is being done primarily by immigrant women and women of color for starkly low pay. “The challenge is we’ve mostly managed longer life spans by inventing low-wage jobs to support older adults, and that’s already failing,” Elana Buch, the anthropologist, says. A partial solution could be personal robots, which at most are a decade away from being able to assist older adults with tasks as delicate and varied as using the bathroom and making tea, says Vivek Wadhwa, a fellow with Carnegie Mellon University’s Integrated Innovation Institute. “The beauty of A.I. is it’s affordable by all,” he says. Robots, though, are likely to reveal the degree to which we’ve taken for granted the emotional connection human caregivers provide — and force us to put a price tag on it.</p> <figure class="media photo large" data-id="100000006209718" data-slug="18mag-longevity-slide-RAPE"> </figure><p class="paragraph">Retailers, too, remain largely unprepared to tap this rapidly expanding market of older adults. The AgeLab at M.I.T. estimates that just 5 to 10 percent of advertising dollars are spent to reach people over 50. “We never speak of ‘fun’ with the older consumer,” Joe Coughlin, the lab’s director, says. “But retirement is a third of adult life.”</p> <p class="paragraph">In seeking to understand the desires, not just needs, of older consumers, Latitude Margaritaville seems ahead of the curve. In 2010, Minto, the developer, purchased the remaining buildable land in Sun City Center, Fla., one of the nation’s first age-restricted retirement communities. “There were residents that had been there for 20-plus years. There are 17,000 households there,” William Bullock, Minto’s Latitude Margaritaville Division president, told me. “It gave us insight into where the market was headed.” Once the companies announced the Latitude Margaritaville partnership, their database of interested buyers exploded to 150,000, whom they could target with digital surveys, rather than convening a handful of seniors for a focus group behind a two-way mirror. Bullock said: “We got as granular as asking, ‘How many concerts do you want a week? Five by the pool and six by the community green.’ ” As Stuart Schultz, director of residential community relations for the Daytona site, put it, “Our aim is to give them exactly what they want.”</p> <p class="paragraph">But the paradox persists: No one wants to see themselves as old, because of the stigma attached to aging, so products and communities are designed to disguise or ignore the real struggles that accompany aging, thereby reinforcing the idea that those struggles are aberrant or shameful. The residents of Margaritaville often compared their living experience to college, to the amusement of their children. But were they pretending to be young, or refusing to conform to ageist notions about who gets to enjoy what? “Honey, we don’t sit around and drink mint tea,” Byer told her daughter, “We go to each other’s homes, we dance, we bring hors d’oeuvres.”</p> <p class="paragraph">The challenge will most likely be keeping these activities accessible to residents who become less mobile. “No one wants to die and no one wants to believe they won’t be able to do the things they do now,” Cinelli told me. “If you’re in senior living and you see somebody who needs help being fed, you want them to go somewhere more appropriate. Seeing someone I used to play cards with needing help, that is fundamentally disturbing.</p> <figure class="media photo large" data-id="100000006209721" data-slug="18mag-longevity-slide-ZPDH"> </figure><p class="paragraph">“How do you create a world where everyone knows how to deal with a person with dementia?” he went on. “If a waitstaff knew you had dementia, they don’t hand you a menu. They say, ‘Hey, we’ve got some great salmon and it has that sauce you like.’ Rather than handing you a menu, and your spouse says, ‘He can’t read a menu,’ and that embarrasses you.”</p> <p class="paragraph">“It’s a feedback loop,” Winters added. “We’re not comfortable with aging because we’ve relegated it to these other places. As more places do it, it does get integrated into our cultural consciousness in a way. I think it’s going to transform the whole thing.”</p> <p class="paragraph"><strong>If senior housing,</strong> with its careful stagecraft, is theater, who are the playwrights and who the audience members? Who’s in charge? I found myself increasingly preoccupied by this question as I toured the model homes in Margaritaville: cheerful open-floor-plans with discreet accommodations for aging bodies — raised counters, walk-in showers, buttons that summon emergency aid and flash a light outside your house so E.M.T.s can find you. They were brand-designed sets, down to the countertops styled with fake tequila shots and plastic cheeseburgers. What had people seen — or not seen — here that made them so sure this was where they wanted their third act to happen? What did these homes say about who they aspired to be?</p> <p class="paragraph">The mystery deepened as the residents themselves opened their doors to personalized interiors that bore no resemblance to the showrooms. Lana Byer favored cherry wood and floral upholstery; Karen Goodwin and her 55-year-old husband, Norm, had modular furniture and purple walls. Fran Murdock, despite her intention to downsize, couldn’t part with her curio cabinets, which held among other treasures a childhood tea set with Mickey and Minnie painted on it.</p> <figure class="media photo large" data-id="100000006209716" data-slug="18mag-longevity-slide-V02U"> </figure><p class="paragraph">As we spoke in their living rooms and at their kitchen tables, they were frank about watching their own parents, grandparents and even children struggle with poor health as they got older. But until that happened to them, they said, they wanted to have fun — more than anything, that’s what Margaritaville represented.</p> <p class="paragraph">When I pressed them to explain what “fun” meant, however, nearly all of them told me about a tiki hut that the Latitude sales team stands beneath from 9 to 5, handing out maps and bottled water; after a protracted negotiation, they had secured its use one night a week for a potluck. “It’s B.Y.O.B.,” Fran said. “Every time we go we meet someone new.” While their carefully curated stage set was being finished, they had essentially gone rogue.</p> <p class="paragraph">In “The Death and Life of Great American Cities,” Jane Jacobs dismisses planned urban utopias of the 19th century as “very nice” — if “you were docile and had no plans of your own.” Instead of designing to some imagined ideal, or to statistics, she writes, planners and citizens ought to study small, existing successes — a popular tiki hut, say — to understand what is “most vital” to a place and its people, and build around that.</p> <p class="paragraph">In trying to imagine how today’s residents might feel about Margaritaville in 20 or 30 years, I kept returning to the tiki hut, where everyone was welcome: What created that feeling, and could it last? I asked Bill and Kelley Sarantis, both 55, who live across the street from the Murdocks, why they thought the residents got along so well, when the country felt so divided outside. Did Margaritaville attract a particular type of person, or did the atmosphere of the place change those who settled there?</p> <p class="paragraph">It wasn’t always harmonious, Bill said. There were people on the neighborhood Facebook page who complained — about the lack of a nearby Costco, or a scratch in their granite countertop. “And people have said, ‘Maybe this is not the community for you.’</p> <p class="paragraph">“You focus on the lifestyle of living here, which I think is pretty incredible,” he went on. “Sometimes in the morning I have to pinch myself. We’re in a place where people all over the country are waiting in line to come. When you get older, you think about your mortality a little more than when you were younger. And I think that helps people here say, ‘We’re along for the ride, we’re here to share our love and experiences.’ ”</p> <p class="paragraph">I put Rhonda Harvey’s question to them: What did they hope for? “I hope not to be bored,” Kelley said.</p> <p class="paragraph">“Not to be alone,” Bill added. So far, both wishes had come true.</p> Sun, 18 Nov 2018 16:40:52 +0000 https://www.nytimes.com/interactive/2018/11/14/magazine/tech-design-longevity-margaritaville.html