Presenting the 2021-era guide to pretty easy DIY video stabilization!
ffmpeg
and vidstab
from homebrew:1 2 |
|
On linux, you can sudo make install
.
There are plenty of options for libvidstab, like shakiness
, accuracy
, smoothing
. The defaults are good, but you may want to experiment. There’s even a visual diagnostic mode.
Assuming the source video is named clip.mkv
…
1 2 3 4 5 6 |
|
You now have a clip-stabilized.mkv
!
Use the vstack
or hstack
filter, depending on if you want them stacked vertically or side-by-side:
1 2 3 4 5 |
|
1 2 |
|
Understand how rAF callbacks are scheduled and why its very reasonable to have multiple callbacks execute within the same frame.
The canonical guide to using the Chrome DevTools UI for debugging Node.js. It definitely beats console.log. ;)
Aside from that, I’ve been busy working on Lighthouse, performance metrics, tooling, and DevTools.
]]>I’ve written up case study documents for each, incorporating all the findings:
➜ Perf Audits: CNet, Time, Google Play
In this doc, we look at the scrolling of CNET, input latency on CNET, some very interesting challenges on the responsive Time.com, and infinite scroll on Google Play’s desktop site.
The intended audience is browser engineers and performance-minded frontend developers. It’s fairly advanced, but I’m spelunking deep to identify how the sites butt heads with the browser APIs and architecture.
Lastly, we’re using this research to improve Chrome DevTools and what you hear from Chrome.
(BTW, use this link to view the same doc but with comments enabled)
1 - More on this performance model later. Stay tuned.
]]>Feb 2015: A lot’s happened since I wrote this post two years ago. Chrome forked WebKit and started Blink, Opera adopted Chromium, and node-webkit became nw.js. This post describes a complexity of defining WebKit that doesn’t exist much anymore; with Chrome’s departure the WebKit world is more simple and clear.
WebKit is deployed through iOS Safari and Mac Safari, and the active GTK community leverages WebKit inside the GNOME Platform. Some smaller mobile browsers use WebKit, some Chromium, some use forks of either, and many just use the system WebViews that are both powered by up-to-date version of iOS WebKit and Android Chromium.
The post below is kept intact and represents a snapshot of history in early 2013, rather than the modern WebKit landscape.
For many of us developers, WebKit is a black box. We throw HTML, CSS, JS and a bunch of assets at it, and WebKit, somehow.. magically, gives us a webpage that looks and works well. But in fact, as my colleague Ilya Grigorik puts it…
WebKit isn’t a black box. It’s a white box. And not just that, but an open, white box.
So let’s take a moment to understand some things:
Now, especially with the news that Opera has moved to WebKit, we have a lot of WebKit browsers out there, but its pretty hard to know what they share and where they part ways. Below we’ll hopefully shine some light on this. As a result you’ll be able to diagnose browser differences better, report bugs at the right tracker, and understand how to develop against specific browsers more effectively.
Let’s lay out a few components of the modern day web browser:
Which of those are shared in WebKit-based browsers? Pretty much only the first two.
The others are handled by individual WebKit ports. Let’s review what that means…
There are different “ports” of WebKit, but allow me to let Ariya Hidayat, WebKit hacker and eng director at Sencha to explain:
What is the popular reference to WebKit is usually Apple’s own flavor of WebKit which runs on Mac OS X (the first and the original WebKit library). As you can guess, the various interfaces are implemented using different native libraries on Mac OS X, mostly centered around CoreFoundation. For example, if you specify a flat colored button with specific border radius, well WebKit knows where and how to draw that button. However, the final actual responsibility of drawing the button (as pixels on the user’s monitor) falls into CoreGraphics.
As mentioned above, using CG is unique to the Mac port. Chrome on Mac uses Skia here.
With time, WebKit was “ported” into different platform, both desktop and mobile. Such flavor is often called “a WebKit port”. For Safari Windows, Apple themselves also ported WebKit to run on Windows, using the Windows version of its (limited implementation of) CoreFoundation library.
… though Safari for Windows is now dead.
Beside that, there were many other “ports” as well (see the full list). Google has created and continues to maintain its Chromium port. There is also WebKitGtk which is based on Gtk+. Nokia (through Trolltech, which it acquired) maintains the Qt port of WebKit, popular as its QtWebKit module.
Different ports can have different focuses. The Mac port’s focus is split between Browser and OS, and introduces Obj-C and C++ bindings to embed the renderer into native applications. Chromium’s focus is purely on the browser. QtWebKit offers its port for applications to use as a runtime or rendering engine within its cross-platform GUI application architecture.
First, let’s review the commonalities shared by all WebKit ports.
window
object and document
object for everyone. True, though the properties and constructors it exposes can be conditional on the feature flags enabled.-webkit-
prefix whereas Apple and other ports accept legacy prefixes like -khtml-
and -apple-
.So, it’s complicated.
Just like how Flickr and Github implement features behind flags, WebKit does the same. This allows ports to enable/disable all sorts of functionality with WebKit’s compile-time Feature Flags. Features can also be exposed as run-time flags either through command line switches (Chromium’s here) or configuration like about:flags.
All right, well let’s try again to codify what’s the same in WebKit land…
window
, document
It gets a little murky in those areas. Let’s try some differences that are much less murky.
<video>
& <audio>
element behavior (and codec support)Let’s take one of these: 2D graphics Depending on the port, we’re using completely different libraries to handle drawing to screen:
Or to go a little more micro… a recently landed feature: CSS.supports()
was enabled for all ports except win and wincairo, which don’t have css3 conditional features enabled.
Now that we’ve gotten technical.. time to get pedantic. Even the above isn’t correct. It’s actually WebCore that’s shared. WebCore is a layout, rendering, and Document Object Model (DOM) library for HTML and SVG, and is generally what people think of when they say WebKit. In actuality “WebKit” is technically the binding layer between WebCore and the ports, though in casual conversation this distinction is mostly unimportant.
A diagram should help:
Many of the components within WebKit are swappable (shown above in gray).
As an example, WebKit’s JavaScript engine, JavaScriptCore, is a default in WebKit. (It is based originally on KJS (from KDE) back when WebKit started as a fork of KHTML). Meanwhile, the Chromium port swaps in V8 here, and uses unique DOM bindings to map things over.
Fonts and Text rendering is a huge part of platform. There are 2 separate text paths in WebKit: Fast and Complex. Both require platform-specific (port-side) support, but Fast just needs to know how to blit glyphs (which WebKit caches for the platform) and complex actually hands the whole string off to the platform layer and says “draw this, plz”.
“WebKit is like a Sandwich. Though in Chromium’s case it’s more like a taco. A delicious web platform taco.” Dimitri Glazkov, Chrome WebKit hacker. Champion of Web Components and Shadow DOM
Now, let’s widen the lens and look at a few ports and a few subsystems. Below are five ports of WebKit; consider how varied their stacks are, despite sharing much of WebCore.
Chrome (OS X) | Safari (OS X) | QtWebKit | Android Browser | Chrome for iOS | |
---|---|---|---|---|---|
Rendering | Skia | CoreGraphics | QtGui | Android stack/Skia | CoreGraphics |
Networking | Chromium network stack | CFNetwork | QtNetwork | Fork of Chromium’s network stack | Chromium stack |
Fonts | CoreText via Skia | CoreText | Qt internals | Android stack | CoreText |
JavaScript | V8 | JavaScriptCore | JSC (V8 is used elsewhere in Qt) | V8 | JavaScriptCore (without JITting) * |
* A side note on Chrome for IOS. It uses UIWebView, as you likely know. Due to the capabilities of UIWebView that means it can only use the same rendering layer as Mobile Safari, JavaScriptCore (instead of V8) and a single-process model. Still, considerable Chromium-side code is leveraged, such as the network layer, the sync and bookmarks infrastructure, omnibox, metrics and crash reporting. (Also, for what it’s worth, JavaScript is so rarely the bottleneck on mobile that the lack of JITting compiler has minimal impact.)
Don’t be! The layoutTest coverage in WebKit is enormous (28,000 layoutTests at last count), not only for existing features but for any found regressions. In fact, whenever you’re exploring some new or esoteric DOM/CSS/HTML5-y feature, the layoutTests often have fantastic minimal demos of the entire web platform.
In addition, the W3C is ramping up its effort for conformance suite testing. This means we can expect both different WebKit ports and all browsers to be testing against the same suite of tests, leading to fewer quirks and a more interoperable web. For all those who have assisted this effort by going to a Test The Web Forward event… thank you!
Robert Nyman and Rob Hawkes touched on this, too, but I’ll add that one of the significant parts of Opera’s announcement was that Opera adopted Chromium. This means the WebGL, Canvas, HTML5 forms, 2D graphics implementations–all that stuff will be the same on Chrome and Opera now. Same APIs, and same backend implementation. Since Opera is Chromium-based, you can feel confident that your cutting-edge work will be compatible with Chrome and Opera simultaneously.
I should also point out that all Opera browsers will be adopting Chromium. So Opera for Windows, Mac and Linux and Opera Mobile (the fully fledged mobile browser). Even Opera Mini, the thin client, will be swapping out the current server-side rendering farm based on Presto with one based on Chromium.
It’s the mac port of WebKit, running inside of the same binary that Safari uses (though with a few underlying libraries swapped out). Apple mostly calls the shots in it, so its behavior and feature set is congruent with what you’ll find in Safari. In many cases Apple takes a conservative approach when enabling features that other ports may be implementing or experimenting with. Anyway, if you want to go back to middle-school analogies, think of it as… WebKit Nightly is to Safari what Chromium is to Chrome.
Chrome Canary also has the latest WebKit source within a day or so.
You got it, bucko.
Hopefully this article described a bit of the internals of WebKit browsers and gave some insight on where the WebKit ends and the ports begin.
Reviewed by Peter Beverloo (Chrome), Eric Seidel (WebKit).
I’ll update with any corrections or modifications.
Nokia (through Trolltech, which it acquired) maintains the Qt port of WebKit, popular as its QtWebKit module. It might be valuable to mentioned that Digia acquired the Trolltech/Qt division of Nokia and now maintains QtWebKit. Also, not to be too nit-picker, in your subtitles “Some of the ports of WebKit” I would have said “Some of the Products running on top of specific ports of WebKit” because strictly speaking Safari is not a port, but a “Browser that makes use of the Apple WebKit port on Mac”.True.
transform
& opacity
; never top
/left
!
In modern days we have two primary options for moving an element across the screen:
translate()
position:absolute
and top
/left
Chris Coyier was asked why you should use translate. Go read his response which covers well why it’s more logical to move elements for design purposes (with transform
) independent of your element layout (with position
).
I wanted to answer this with Chrome and get some good evidence on what’s going on. I ended up recording a video on the experience:
It’s a good watch and dives into Chrome DevTools’ Timeline, compositing, paint cost, accelerated layers, and more… but if you want the abbreviated text version, read on:
First thing, Chris made some simple demos to try things out:
This sorta works, but there is such low complexity in this situation that it’s all going to look pretty great. We need something closer to a complex website to properly evaluate the two (thx joshua for the macbook!):
Now we’re starting to get closer, but immediately I get distracted by something.
If you run the demo above you might notice the top edge of the MacBook looks a little bit better in the top/left one. (And here I am writing a post about why translate is better! Preposterous!) So this is due to the absolute positioned macbook sticks to pixel positions, whereas the translate()
‘d one can interpolate at sub-pixel positions.
One of the Chrome GPU engineers, James Robinson, called this the “dubstep effect”, as these pixels appear to be getting pounded by bass. Here’s a closeup of the effect… watch the top white edge of the macbook in each:
On the left, you see stair-stepping down three pixels and back up again. This pixel snapping may result in a less distracting transition in this case, though this wouldn’t be so noticeable if you were moving objects with less of a high-contrast situation.
If you run DevTool’s Timeline Frames mode on these two examples here, they start to tell a very different story:
The top/left has very large time to paint each frame, which results in a choppier transition. All the CSS including some big box shadows, are computed on the CPU and composited against that gradient backdrop every frame. The translate
version, on the other hand, gets the laptop element elevated onto it’s own layer on the GPU (called a RenderLayer). Now that it sits on its own layer, any 2D transform, 3D transform, or opacity changes can happen purely on the GPU which will stay extremely fast and still get us quick frame rates.
Watch the video above towards the end to see more on how to diagnose paint costs, see what areas are being repainted, and evaluate what elements are on the GPU.
Click through below, try adding more and see how the the framerate reacts. Feel free to open up DevTools, too and explore Timeline:
requestAnimationFrame
. Avoid setTimeout
, setInterval
.animate()
-style) if you can, declarative animations in CSS can be optimized by the browser way more.If you do front-end web development and already use Chrome as your development browser, I encourage you to use Chrome Canary.
A new Chrome Canary build is available daily (we cut at 2am PST and take the best of last 40 revisions, to be specific). Running fresh builds gives you great goodies to look forward:
$_
(for the Console’s most recently evaluated result)about:flags
)
The other reason why Canary is great is because it can run side-by-side with your other Chrome install. In fact this is why the URL for Canary’s download page is “sxs”.
I recommend running Chrome Stable and Canary. That’s how most of the Chrome Developer Relations team does it. You never need to update your browser, you can watch the new stuff coming at you (and file bugs if it breaks) and still see what your users see.
Fun fact: when a feature lands in Canary, it’s only a short 11 week trip down to shipping to all 300+ million Chrome users on stable. That is, as long as everything is on schedule and the feature doesn’t need to bake a little longer on dev channel before making its way down. Given that, it’s a smart move as a dev team to watch as features (and sometimes bugs) trickle down to have time to adapt. When a new stable Chrome is ready, all users will have it within just a few days.
Canary is available for Mac and Windows, but sadly not Linux. :( But luckily there are packages that track the dev channel Chromium that’ll keep you up to date on a weekly basis, so you can have a pseudo-canary. If you need a one-off build of the freshest Chromium available, just go to download-chromium.appspot.com; it’ll detect your OS and serve up a build that’s less than a few hours old. Hotchachacha!
Protip: you can set Canary as your default browser : Canary itself doesn’t have a setting to let you do this, as its an edge release. (Read about how Chrome ensures the stability of the Canary build). However… if you open Safari’s preferences, Canary will be listed in the Default web browser choices, so you can choose it if you want all new links to open there.
Another small tip: sometimes your Chrome extensions affect your page inspection behavior is weird ways. Jump into incognito where extensions don’t run to get a clean look.
Running Canary regularly rocks, but so do edge versions of other browsers:
Oh right, call to action:
A few highlights:
I’ve also updated the interview listing on my About page. <3z
]]>I absolutely love developing for the clientside of the web. Delivering the actual interactions and UI that all users feel is an absolutely gratifying experience. However… it’s not a piece of cake to develop for all desktop browsers, mobile browsers, and all the device/OS combinations within.
It’s hard to track down accurate info, but hey we get by with an unruly combination of Mozilla’s MDN, StackOverflow, HTML5 Please, W3C/WHATWG specs, Wikipedia, and a mental inventory of blog posts, tweets and articles strewn over the internet.
Today’s announcement of Web Platform Docs, an initiative that’s been well over a year in the making, is huge. A few reasons why I’m pumped:
This won’t be perfect overnight and will take a while to completely supplant existing distributed documentation sources… but I’m very excited about this first, shared small step. Three cheers to making a better experience for everyone building for the web.
]]>These days, our performance bottlenecks are oftentimes DOM, <canvas>
API methods, SVG. Those are our priorities. So then why do we see all browser vendors competing on raw JS when we are hacking our way around to avoid the actual performance pain points? So we wanted to solve that with RoboHornet.
Today, a cross-vendor collection of developers, led by Google, is open-sourcing RoboHornet. As project lead Alex Komoroske put it, “it’s a living, dynamic benchmark that aims to use the collective efforts of the web development community and ultimately get browser vendors to fix real-world performance pain points.” Let’s look at how RoboHornet will hopefully flip the above situation on its head.
The current items that RoboHornet tests captures the major pain points of jQuery, Google Apps, Google Maps, Ember, Handlebars and Cappuccino:
You can vote on and review incoming tests in the issue tracker.
The project is still in alpha and we’d love to get your involvement to better represent everyone. We’re using Benchmark.js internally, as its experience on jsPerf has proven it a trustworthy and reliable benchmark harness.
While you’re at it, go check out the full review of RoboHornet at Tom’s Hardware
]]>Early last year, something clicked in my head when I saw this video: Crossing the Chasm: Pitching Security Research to Mainstream Browser Vendors.
You mean.. there are academics… who are doing research on browsers.. in universities? My my! You mean bright minds think about the same things as us and don’t publish them on Octopress? ;)
Turns out it’s quite an active area of research. So, I’ve put together a list of academic papers, talks, and researchers all centered around frontend development and browsers (Please chime in on the comments if you know of more):
Gibraltar: Exposing Hardware Devices to Web Pages Using AJAX UC San Diego, Microsoft Research, Univ. of Singapore, 2012
Modeling and Reasoning about DOM Events Brown University, 2012
Don’t Repeat Yourself: Automatically Synthesizing Client-side Validation Code for Web Applications University of Illinois at Chicago, 2012
Who Killed My Battery: Analyzing Mobile Browser Energy Consumption Stanford, 2012
How far can client-only solutions go for mobile browser speed? ACM Digital Library, 2012
Race Detection for Web Applications Sofia University & ETH Zurich & IBM T.J. Watson Research Center, 2012
Leo Meyerovich - Berkeley Rsearch UC Berkeley, 2009 (Parallel Browser Execution, Security)
James Mickens - Microsoft Research Microsoft, 2004- (Performance, localStorage, Mashups, Speculative Execution
Timelapse - Interactive record/reply with Webkit inspector Univ. of Washington, 2012
Cleanroom: Edit-Time Error Detection with the Uniqueness Heuristic Univ. of Washington, 2010
Firebug Research - Academic papers related to Firebug Various, 2012
Realtime Performance Visualizations using Node.js How to NODE, 2011
How Programmers Debug, Revisited: An Information Foraging Theory Perspective Wentworth, OSU, & IBM TJ Watson Research Center, 2010
Crossing the Chasm: Pitching Security Research to Mainstream Browser Vendors Carnegie Mellon, 2011
Benjamin Livshits - Microsoft Research Microsoft, 2004-2012 (Malware, Privacy, JS Benchmarking, JS Compression, JS Static Analysis, Security)
Dynamics of JavaScript Purdue, 2011
Adam Barth - Research Various, 2007- (XSS, CRSF, Security…)
Highlights of the Infovis Conference, VisWeek 2011 2011
Gregor Richards - JS Research at Purdue Purdue, 2009-2011
Cleanroom University of British Columbia, 2011-2012
Anders Møller - Research Aarhus University, 2002-
Whyline for Java Natural Programming, 2009
Stacks Plorer Media Computing Group, 2012
Omniscient debugging with TOD Pleiad, 2007
The Elm Programming Language Elm Lang, 2011-2012 (a type-safe, functional reactive language that compiles to HTML, CSS, and JavaScript)
]]>So, here is a list of people we admire:
Ryan Seddon: @ryanseddon aka the CSS Ninja develops Modernizr, writes polyfills, and explores the edge of DOM APIs. His recent exploration of source maps lays a lot of groundwork for future web debugging potentials.
Hans Christian Renl: @drublic started contributing extensively to HTML5 Boilerplate and has been actively engaged in other projects we have worked on. He also maintains a blog where he shares his learning experiences and explorations of advanced CSS features.
Alexis Deveria: @fyrd maintains caniuse.com, and contributes to svg-edit, the open source svg editor. He also created & maintains the canonical timeline of web browsers which has been the basis of other browser history graphs since. He also upstreams all the data from caniuse.com to its API, is used to power sites such as api.html5please.com and html5rocks.com
Jason Kiss: @jkiss’s notes on accessibility are like a breath of fresh air into the dense field of accessibility. He tests in as many accessibility technology tools as available and bases his opinions on findings from his research. His articles are extremely valuable to web developers who have long had to base how to do the right thing on superstition and fact-free opinions.
Kit Cambridge: @kitcambridge develops some of the best-in-class javascript libraries and assists many open source projects on Github. Take a look around bestiejs to see his fine work and friendly demeanor.
Eric Dahlström: @erikdahlstrom is not just the editor of SVG specs, but also helps out often in mailing lists and Stack Overflow with SVG questions. Not only that, he is an unbelievably skilled (obviously!) at hand-coding lightweight SVG, and maintains a repository of SVG images that are worthy of learning from on how to do SVG right.
Masataka Yakura: @myakura watches HTML/etc. specification changes, browser bug trackers and mailing lists, and reports the best stuff on his G+
Matt Seeley: @innerhtml is a JS hacker at Netflix, who measures constantly and talks about the science of cross-device hardware accelleration
Tim Branyen: @tbranyen created Backbone Boilerplate and Backbone Layout Manager to make it easy for everyone to get started with Backbone. He also writes extensively on his blog on dealing with problems of creating web apps.
Mathieu Henri: @p01 has an in-depth understanding of JavaScript and shares amazing experiments that are possible within less than 1K of JavaScript such as canvas with audio (warning audio) & Speech Synthesizer. He also frequently contributes to @140bytes on what is possible with 140 bytes of JS.
Roman Cortes: @romancortes is one of few who can explore some interesting innovative uses of CSS. His explanation of a 1kb chrismas tree is a whirlwind of frontend knowledge.
Peter Beverloo: @beverloo started off a series on his blog documenting what’s new in WebKit/Chromium. He’s since been hired by Google and works on Chrome for Android. His work in distributing edge browser development news has helped thousands of people track the quickly expanding web platform.
Erik J Moller: @erikjmoller is to WebGL what @dahlstrom is to SVG. His skill and ability to hand-code webGL is unparalleled. He created a 10-part video on learning webGL (webGL 101) that is archived on youtube not to mention is the main contributor to EmberWind an open source webGL game that is highly performant.
Irene Ros: @ireneros dabbles in info-viz tools and has published several of them on GitHub. She is active in contributing back to open-source projects and also maintaining a few of them including Miso Project and underscore.nest.
Mike Taylor: @miketaylr is developer relations extraordinare for Opera. He has been a behind-the-scenes force getting a lot of cross-browser compatibility and feature detection complete and distributed.
Zoe Mickley Gillenwater: @zomigi wrote an excellent book about CSS3 and publishes informative articles on CSS on her website that are a good resource for anyone who is working with CSS3.
Niklas von Hertzen: @Niklasvh created html2canvas, essentially a rendering engine in JavaScript. He’s also had a number of exciting experiements like a Sudoku Solver in CSS.
Zoltan Hawryluk: @zoltandulac painstakingly researches making CSS3 features backwards-compatible with IE, in addition to blogging about several discoveries in the process.
There are clearly far more people that deserve to have their praises sung. Plenty of people whose names you already know, but also folks like @ZachLeat, @Alex_Gibson, @mbostock, @RobFlaherty, @Raynos2, @bricss and more.
We are grateful towards everyone who has helped us along the way to shape the web to be as friendly as possible to web developers, by driving the work on frameworks, libraries, articles and outreach. They do it on their own time, because of their love of and fascination with the web.
All of them are well worth your time to pay attention to and follow and hear their opinions of the web.
We’ve left out some of the bigger names in this list, so hopefully we can all discover someone new who covers a relevant field extremely well. This is our list, but feel free to leave a comment with anyone you’d put on yours. Or better yet, blog it yourself and link up your list here.
I wanted to take a moment to explain what this IE update policy means for us web developers.
The big change is how IE now interacts with Windows Update: Before you had to opt-in to have “Recommended” updates installed automatically. It’s likely few folks did. Now, IE upgrades are shipping as “Important” and this class of updates are defaulted to install automatically. A user can still opt-out and immediately after the announcement of this feature they also published details on how enterprises can opt-out. But still, this is a good move, it helps.
The policy manifests as:
A very smart policy call was that this update policy also includes all of China, which is important because most of their Windows installs are not genuine. Microsoft decided they should get the IE bump anyway, which is great because China has been dominated by IE6 and IE8 for ages.
This update procedure was scheduled to start in Australia and Brazil in January 2012. If we look at what happened in Brazil, we have a good shift of users but it’s flattened out with a nontrivial amount of left behind users: (see the two blue lines in the chart below)
(Source: StatCounter Global Stats - Browser Version (Partially Combined) Market Share)
Australia has a similar swap of users from IE8 to IE9 but IE8 still remains steady above 10% overall share there.
Microsoft is in a tough spot in that they have enterprise customers who have developed their intranet applications in an extremely poor manner and they break outside of old browsers. But we’re going to need some more pushes from them and us, the developer community, to get in a better state. We want to develop for the web platform of now, not the platform of four years ago.
TL;DR: There is now a policy in place, but evidence indicates that it’s not as effective in eradicating these zombie browsers as what we need. Personal opinion: we need to do better.
Hopefully this clarifies a bit about the mechanics of the update procedure. Please do correct me if anything is wrong and I’ll update the post.
(Consider the two blue lines)
Source: StatCounter Global Stats - Browser Version (Partially Combined) Market Share
]]>For a very long time, developers have been told about making websites ‘accessible’. Like it is a faucet that should be turned on or off. Till Victor Tsaran’s demonstration of using a screenreader (which was in 2007 - a very long time since accessibility evangelism began) - very few web developers had any idea of the real immediate impact of implementing accessibility features.
During a Nicholas Zakas & Victor Tsaran talk years ago I finally grokked the easiest rule for a first step towards accessibility. For such a long time, we conflated functionality while JavaScript-disabled with “being accessible”. It took me years to learn that making it keyboard-navigable was the top priority.
Just as a datapoint, The WebAIM survey results (published yesterday) reveal 98.6% of screenreader users have javascript enabled.
People like Jason Kiss and Steve Faulkner who detail the behavior and support in browsers and screenreaders are providing the developer community with invaluable knowledge, stuff that is far more worthwhile than publishing a yet another list of you-should-do-these-things. (See also my resources list in the semantics post).
Most developers have no idea what effect adding ARIA markup to our documents has on people using screenreaders. We need that. Actual before-and-after video stuff makes this topic real. See zomigi’s recent post: videos of screenreaders using ARIA, for great examples.
Most developers have no idea how to implement ARIA. The W3C Primer that looks like a spec is not at all appropriate for a web developer audience.
Just like it’s valuable to witness an a11y usability session where you see how disabled people use your site, I think it’d be useful for the accessibility community to see how developers build, because thus far there has been a disconnect in communicating effectively. I don’t think we need more on the ground evangelism yet. There is a dearth in online resources at the moment, developers need to know these practical ways of using and having users benefit from accessibility techniques:
Also, I’m not quite sure how everyone else feels, but from here it feels like yelling at a fucking brick wall with JAWS and Window Eyes. We just kinda hope and pray their support for things improve. As a pragmatic approach, I tell developers that if it works in NVDA, good enough. We have a mess of browsers we need to support already, dealing with ATs that lag years behind is not at all feasible.
A lot of developers do not understand that accessibility first comes from implementing the platform accessibility API by browsers such that screen readers can take advantage of the rendering. Most operating systems expose an API that allows browsers to map what is rendered on the screen to one of the interfaces that the operating system’s accessibility API exposes.
Unfortunately, there has been no specification, so each browser has been left to do whatever it wants in terms of defining the relationship between a HTML element and the accessibility interface. Hopefully this will change with the HTML to Platform Accessibility API implementation guide and ensure each browser exposes each element consistently to the Platform Accessibility API.
Which shores up a greater concern: Better accessibility starts with the browser (and the platform). Browsers should be able to handle content loaded dynamically rather than expecting developers to do it for them. If content on a page that is displayed is altered, it needs to be noted to the accessibility api transparently instead of having developers bolt on extra markup to do so.
Accessibility should not be opt-in but opt-out.
This post is a elongated version of the comment I left on Karl Grove’s Barriers to improving the accessibility game plan. It was probably the best post on a11y I’ve ever read. I really liked his approach in evolving the a11y strategy and left my thoughts.
Thanks to Divya Manian for her thoughts and words on this post. Thanks to Jason Kiss for reviewing drafts.
Because there are so many tools around, I wanted to break them down somehow. I think contextualizing the tools as to what phase of development that assist with works well:
(You could think of the Y-axis here as the amount of code in a project..)
Looking at our tools this way we end up with:
I’ve been thinking a lot about workflow and integrating these tools together. Screencasts like Andrey Tarantsov’s Sublime Text Workflow really excite me and I’m eager to see more people exploring a robust development setup.
There’s a lot of conversation about making prefixes work (by changing policy), but I believe they already are at odds with the developer workflow. In this proposal I hope to show that:
If I’m using a feature that’s prefixed, I have a choice of using a tool to help me manage outputting the various prefixes I need or I’ll do it myself. Based on my informal poll last week it looks like 40% of developers are not using tools. Without tool usage, authors are not including -ms-, -o- or the unprefixed variants. Page 4 of Mozilla’s prefix report verifies this; developers don’t write necessary prefixed properties about half the time (e.g. -o-transform) and in fact, they already write out the unprefixed variant well before the spec goes to CR (and do this well before adding in -ms- or -o- compatibility).
This matches my experience as well. I don’t want to return to this code to maintain it, so I’m going to add the unprefixed version. There is less of a cost to me leaving the unprefixed version and assuming I won’t need to change it vs. returning to my CSS of shipped-to-production work to add unprefixed variants each time a feature hits CR.
Any successful evangelization plan for prefixes relies on tools, but all tools prioritize the unprefixed state. prefix-free, compass, and css3 please (I’m a tool vendor, too) all heavily suggest starting with the unprefixed state. So any successful evangelization push will just result in the unprefixed version being 100% deployed from the get-go, which ties implementors hands anyway and defeats the purpose of the prefix. I don’t have data to show that developers are not returning to their deployed, production code to update prefixes, but anecdotal evidence suggest so. Opera’s Web Opener team, for example, reports a <10% success rate when directly contacting sites to request updates to their prefixes.
PPK wrote “If standards make web developers’ lives much harder, web developers ignore standards. This was true back in 1998, it’s still true today.” Without prefixes, implementors have an opportunity to empower developers to do the right thing while still allowing room for innovation.
PPK covered this ground well last week, but in short the policy of binding prefix-drop to CR has failed plenty. Developers have no expectation of when this will happen, nor should they. Mozilla’s roc has confirmed developers are using prefixes and they won’t stop regardless of their supposed “experimental” status.
So without prefixes, vendors hands would be tied and the first implementation to ship would be the one to stick, right? Nah. I think we can walk through a few examples from recent history too see what it would have been like if there had been no vendor prefixes in play.
While plenty of implementation bugs were fixed, the developer would never have updated their syntax and would have complete browser support.
No problems. No changes.
Gradients has been the most tumultuous css spec, but if they had been implemented without prefixes, the feature would be in a better situation today.
Browsers discard properties and values they don’t parse successfully, so developers would have just included a cascade of the iterations of the spec. This would have successfully targeted all shipped versions of gradients:
1 2 3 4 5 |
|
Mozilla started Flexbox v1, WebKit followed with a slightly stronger implementation. Both Alex Russell (post) and I (article) recommended boilerplate code that assumed no change for unprefixed state and blocked out IE10’s upcoming implementation.
Flexbox v2 debuted and has an implementation in Chrome. Syntax is already different enough that the fallback story is happy. Without prefixing, the resulting browser support of shipped author code would be better. Again, I see no drawbacks in this example had these properties not been prefixed.
Soon after this feature debuted, shims were written to manage all prefixed implementations including the unprefixed state. (Sorry). Combined with quick adoption, browser have as much ability to change the feature as they would if it were unprefixed to start (that is, not much ability).
In CSS, new breaking changes have to introduce syntax changes that would break previous parsing attempts, as seen in the gradients example. However, if the semantics of a feature changed without syntax changes, we’d have a situation where developers are forced to detect each behavior and write compatibility code.
border-image introduced a similar situation recently: WebKit changed its semantics and required a `fill` keyword to match previous behavior. Sounds like trouble, but this change happened when the unprefixed variant was exposed, matching implementor best practices of changing a feature. The result: Greg Rewis of Adobe blogging that Chrome broke his site. This is truly how it is supposed to work, but still expectations are broken. Developers have plenty of motivation to leave in the unprefixed state at the beginning, but this means that any change will always break them. With prefixes or without prefixes, we would have gotten the same resolution here.
In JavaScript/DOM we have a more secure safety net, as behavior can be detected. Take for example querySelectorAll(); it was shipped without prefix and while implementation differences exist, JavaScript libraries detect and route around them. Of course, semantics can change and the prefix → unprefix move provides an opportunity, but in my experience it’s more common to see these changes happen early on (like blob.slice()’s change).
Without prefixes there will be no mechanism to notify developers “beware, thar be dragons!”. Prefixes don’t do this now, as their removal isn’t prioritized so developers accept them as a fact of life rather than an indication of instability. The solution: getting accurate metrics of feature usage and adoption can inform implementors of if it’s too late to change a feature or still early enough to modify.
As vendor prefixing is a well-worn debate, allows me to deflect common challenges to this proposal to:
-webkit-
styles.-webkit-
prefixes on their mobile browsers. I don’t see a way around this nor do I think it’s bad.In summary, the ideal developer experience is 1) using tools to manage prefixes and 2) always using the unprefixed state to reduce maintenance cost. These two facts eviscerate the benefits that prefixes were introduced for, leaving only the drawbacks.
]]>Ugh. If I say the width is 200px, gosh darn it, it’s gonna be a 200px wide box even if I have 20px of padding. So as you know, this is NOT how the box model has worked for the past ten years. Wikipedia has a great history of this box model. Jeff Kaufman also dove into the history
Anyway, I have a recommendation for your CSS going forward:
1 2 3 4 5 6 7 |
|
Update August 2014: This code was updated to match new box-sizing best practices. Also prefixes are pretty much dead.
This gives you the box model you want. Applies it to all elements. Turns out many browsers already use border-box for a lot of form elements (which is why inputs and textareas look diff at width:100%;) But applying this to all elements is safe and wise.
Due to browser support, this recommendation is only for projects that support IE8 and up. (Full browser compat at MDN) Firefox <= 28 still needs the -moz-
prefix, and <= iOS4, Android <= 2.3 need the -webkit-
, but everyone else uses the unprefixed. You can find more info about a box-sizing polyfill for IE6 & 7 at html5please.com/#box-sizing (which was developed with * { box-sizing: border-box
!).
Totally. jQuery works pretty great with it (except this). As mentioned, browser support is excellent. And a number of projects use this layout model by default, including the WebKit Web Inspector (aka Chrome DevTools). I heard from Dutch front-end developer Yathura Thorn on his experience:
We’ve been using
*{box-sizing: border-box;}
in one of my projects (deployed in production, averaging over 1mln visits a month) at work for about a year now, and it seems to be holding up just fine. The project has been tested in IE8 & 9 and the latests Firefox and Chrome versions and we’ve had no problems. All jQuery (+UI) offset calculations and animations run fine, even in any element we’ve displayed as inline-block. As of late we’ve started testing the project on mobile devices (iPhone, iPad and Android) and we’ve had no issues regarding box-sizing with any of them yet, so it seems to work just fine.
One of my favorite use-cases that border-box solves well is columns. I might want to divide up my grid with 50% or 20% columns, but want to add padding
via px or em. Without CSS’s upcoming calc()
this is impossible… unless you use border-box. So easy. :) Another good one is applying 100% width and then wanting to add a padding to that element.
Lastly on @miketaylr’s inquiry, I also tested perf overhead. Anecdotal evidence suggests border-box isn’t significant.
You might get up in arms about the universal *
selector.
Apparently you’ve heard its slow. Firstly, it’s not. It is as fast as h1
as a selector. It can be slow when you specifically use it like .foo > *
, so don’t do that. Aside from that, you are not allowed to care about the performance of * unless you concatenate all your javascript, have it at the bottom, minify your css and js, gzip all your assets, and losslessly compress all your images. If you aren’t getting 90+ Page Speed scores, it’s way too early to be thinking about selector optimization. See also: CSS Selector Performance has changed! (For the better) by Nicole Sullivan.
So… enjoy and hope you’ll find this a far more natural layout model.
Any third party widgets add content directly into the page may need extra attention. Any widgets inside an iframe (like Disqus’s new theme) are naturally insulated from the parent page’s layout styles. If you need to make adjustments to the third party content you can apply box-sizing: content-box;
to the widget and its descendants. This may not be trivial as form controls in particular are border-box by default, so you’ll also have to account for that.
This Wednesday, me and a ragtag team of like-minded developers launched Move the Web Forward, an initiative to direct people’s passions into productive efforts that assist not only the web community but the entire web platform. People often feel the need to “give back”, but when it comes to contributing it’s a challenge to find the right place. This is something I’ve seen happen firsthand in the jQuery and HTML5 Boilerplate communities.
The MWF effort was inspired by a tweet by WebKit extraordinaire Dimitri Glazkov.
Me and Divya started brainstorming on an etherpad. We rounded up some folks on Google+ and IRC and started collecting a lot of excellent resources. Soon after a design was in the making and we started working in an open repo on github…
At this point, I want to call out Mat Marquis aka @wilto, in particular, for really taking early ownership of this project and spearheading the design and frontend development. His efforts really made the site communicate its content effectively.
Naturally the site is on github if you’d like to contribute. We have a few issues open that we’ll be improving the site with. (Also, let me just say for projects like this, a cron
job to pull latest from Github makes for such a relaxing workflow.)
Thanks to Divya, Mat, Aaron Forsander, Connor Montgomery, Stephanie Rewis (great writeup!), Nicolas Gallagher, Addy, and all these awesome folk. Truly a socially developed site; everyone collaborating on IRC, etherpad and github (with as many having commit rights as possible). Fun project to work on and I hope to see everyone find the right place for them to get more involved in actively making the web as awesome as you want it to be.
Beyond all the people who signed up (with the avatars) on the site to commit to things.. We’ve seen some moves in the community already:
\o\ \o/ /o/
Frontend development exploration, techniques, tips. Lots of JavaScript.
275 feeds. |
Standards development and browser dev news.
56 feeds. |
Blogs focusing on building large webapps (new!)
22 feeds. |
Also, a short commentary on blogging vs tweeting.. while a lot of people have changed to twitter+jsfiddle only, tweets are not available via search after 30 days, so if you expect anything you share to last, blog it for real. :)
Also please subscribe to these guys which do a great job of highlighting the top stories of the week:
The Primitives of the HTML5 Foundation - Slides
I do discuss optional start and end tags as well as not quoting your attributes. I concede that these ideas scare the shit out of people. In fact, when I first heard Jens Meiert propose the idea in 2009 my brain rejected it.
Still, I think many of the markup parsing curiosities that can be frightening only need to be understood by HTML minifiers. Always leave it up to tools to make your frontend payload smaller; you just want the most comfortable and maintainable authoring environment possible.
I shared them in this talk because 1) it can lead to writing more beautiful HTML that is easier to maintain. Leaving off </li>
’s is a good example. 2) I just want people to consider browsers less of a black box of uncertainty. There are rules and specs that define behavior, so once understood you can feel confident relying on a consistent behavior. We certainly don’t have that consistency across all the web platform, but for these examples you do.
In summary, I’m not advocating you write more spartan markup if you don’t want to. But if it makes you feel good, by all means, go at it; you know now how the browsers work.
ps. the monkey HTML shirt I wore is from a store in tokyo; solo location. but they also sell on this delightfully overwhelming site
]]>Thanks Jeremy for raising a practical example. This discussion is a tough one as much of HTML5 has clear semantics (nav/header/footer), but then parts have underdeveloped semantic meaning or add confusion to authors.
Jeremy’s gist is a great example, in fact, of a poor time investment in semantics. It ascribes value to the new method of document outlining, which recently sees different styling for h1’s depending on section nesting. Recent browsers do indeed style the h1’s different; but if you actually structure your document as such, you do it to the detriment of your screenreader users: they will either get these h1’s all as top level headlines or a mishmash of heading nesting levels that don’t match any expected behavior (As an aside, this is completely unrelated but if anyone’s curious what CSS it takes to make that new h1 styling work across browsers… feast your eyes on this beauty)
Additionally, <hgroup>
is on the chopping block and is not supported by JAWS. Suffice it to say, Now is not a smart time to invest your time understanding the unwieldy document outlining algorithm. Luke’s comment digs in deeper to the messy semantic state of the outlining algorithm.
The practicalities of making accessible web content are messy, but important. The fact that we seem to spend more time on div
vs article
vs. section
than on learning ARIA is a crime. (Furthermore, learning ARIA isn’t complete unless you’re listening to the results in a screenreader.)
My recommendation: follow the work of Steve Faulkner and Jason Kiss. These two guys are publishing the only data that illuminates what’s actually happening at the AT level in response to our work. Steve’s comment on this post does a solid job of unraveling this complex topic by showing all the many layers involved: specs, editors, authoring experience, aria, browser impl, screenreader compliance.
Also if you’d like to investigate what’s happening under the covers, I recommend:
In summary, I think it’s best to ground our efforts in pursuing semantic value by identifying precisely what the results will be. A fair rule of thumb: when it comes to semantics, if it’s confusing enough for you to ask a question about it, chances are the answer won’t make a realistic difference. Let’s build incredible stuff on this impressive platform and avoid getting mired in semantic inconsequentialities. There’s also room for more author engagement and screen-reader involvement in the standards process regarding these issues; but that’s a large topic I’ll have to save for another day.
]]>