Despite the changes that were unveiled to iOS today, I still expected quite a bit more based on published patent applications and statements made earlier by executives. I am left wondering how much new functionality was actually presented today--all the more so, because iOS7 represents the culmination of years of work and is clearly the main development branch. We finally have a good sense of the true state of Apple’s iOS development progress, although some functionality may still be hidden for unreleased devices.
I must say the new font makes it easier to distinguish characters for reading text on a small screen. I have yet to review the beta of iOS7, so I may uncover more unannounced goodies. My gut reaction was that iOS7 has a new coat of paint – stylistically different, but not much so under the hood. Some reviews have indicated numerous incremental features across the system.
The upcoming iPhone and iPad could see significant changes as well as unlocked functionality from iOS7. There were rumors that the two devices would be announced or launch on the anniversary of the original iPhone in an event entitles, “Original Passion. New Ideas.”
There are some hints of 3D functionality with iOS7 UI motion tracking and this could be expanded further with the faster, more battery-efficient processors in newer hardware. One thing of note is that the user interface reportedly feels layered across applications, which gives it an added sense of dimensionality with physical realism and depth.
Otherwise, there’s 2014, which Tim Cook has promised to be full of innovation. I really like the company’s vision as revealed from patent applications, but I am getting real tired of waiting for it to come to fruition. For now, I will experiment with iOS7.
I have a contrarian view on the software and hardware technologies being introduced this year by Apple.
I don’t think iOS is truly going flat. I would be surprised if iOS7 does not go full 3D, though the 3D mode might only be available in next iPhone. Apple’s patents form an edifice in which later patents, each with several inventors, are built on top of earlier 3D patents stretching the entirety of the interface, both hardware and software. This is not a research project but a heavy investment into rebuilding iOS on top of a new user interface paradigm.
I also think that the display is an important thing to watch for. Apple won’t be releasing an iPhone 5S but an iPhone 6 within the same form factor but with an enhanced integrated display. Apple has patents which integrate cameras, speakers, solar panels and optical sensors directly onto the display. The display can still grow to 4.5 inches edge-to-edge with the aspect ratio unchanged.
The rumored movement towards flat interfaces is probably a move towards vector graphics, which avoids requiring multiple image assets to account for different display resolutions. Vector graphics also do not appear distorted inside 3D transformations. While the iPhone will eliminate many aspects of skeuomorphism, I think just the gratuitous textures and elements will be discarded such as in the GameCenter and PodCast apps. GarageBand will likely maintain its musical keyboard. The iBooks user interface might not change.
The physicality of the interface will be preserved. The software and hardware will move towards a unitary design in which software no longer feels separate from the hardware as the device will appear to morph dynamically over the course of use. The display will occupy the entirety of the front surface, removing the seam that exists between the bezel and the display. Various buttons and other user interface elements protruding from the surface; the next phone is rumored to use a flexible film-based touch sensor. Other sensors and home button will come in and out of view. The home button may also appear on the back of the device, where invisible microperforations in the metal back frame emit light to reveal a button. Apple has a unique layered 3D display design that tries to more accurately reproduce the light field entering the user’s eyes without the need for special glasses. The operating system will track the user’s eyes and the orientation of the device to dynamically change the screen’s perspective. Drop shadows and lighting effects will be automatically rendered on top a scene graph using the new SceneKit API based on the lighting conditions of the room as detected by the ambient sensor. Quite possibly, the touch sensations on the display will simulate real world textures on software-based user interface elements.
I have been thinking a lot about the purpose of my blogging and the creation of workflow that will allow me to blog more. There are a few things that I am interested in:
Online Courses
I am a strong believer in continual learning and I have taken about fifty university courses last year through Coursera and other MOOC sites, which is more than the number of courses typically required for a bachelor’s degree. I have received over thirty certificates. I also have annual subscriptions to Pluralsight.com, which specializes in Windows development, and Lynda.com, which specializes in design. The coursework took considerable amount of time that detracted from software development but allowed me to become up-to-date on the latest technologies and theories.
As I stated earlier, I am trying to create a new blogging workflow. One possibility is writing up a blog post covering some of the material that I learned. There aren’t many courses left though as I have exhausted the courses that I have an interest in. I am currently setting up a program of study to shore up my mathematical and computer science background that relies on free electronic textbooks and lecture notes. Many advanced/graduate courses are not available online, so I have to develop my own. This could lead me to contributing lecture videos online via Udemy or YouTube.
C#
My programming language of choice is C# used across a number of different platforms and also for scripting in place of Perl or Python. Unfortunately, C# is not very portable even among the various .NET and Mono platforms (WinRT, Silverlight, MonoTouch, WPF, ASP.NET, and WinForms) as the namespaces and interfaces differ substantially.
C# hasn’t progress very much in the past few years, but there are indications from the Microsoft MVP Summit that .NET 5.0 will be a substantial advance and will probably be announced at the Build conference later this month; also, channel 9 videos indicate that major changes to the type system will be arriving.
I see myself writing more posts on C# in the future based on these developments as well as from other works of my own such as static analysis and semantic code conversion. I am looking to port a number of popular open source libraries to C# from Python, Objective C and C++ this year and next. These include libraries in numerical computing, natural language and computer vision.
Software Entrepreneurship
Software entrepreneurship is another area of interest for me to write about. I would like to talk more about unreleased products, but I am worried about intellectual property issues, which is why I have stop communicating about them until they are released.
This year’s Mac development conference should usher in the next wave of Apple innovation that has been in development for several years as evidenced by Apple patents and leaks. Apple’s previous keynotes have always been characterized by years of incremental improvements punctuated by large developments.
Apple apparently pursued parallel branches of its mobile operating system in order to redesign the user interface according to new interface paradigms such as 3D, motion gestures, object/face recognition, and eye/head tracking. Many of these technologies, originally destined for the Mac, predated the iPhone launch but appear to have been retargeted for initial or simultaneous debut on the iPhone.
Some general predictions:
Regarding new hardware this year:
User interface advancements:
Sources: PatentlyApple, PatentBolt, AppleInsider, MacRumor, USPTO, blogs and rumor sites.
[Pardon my love of Apple. I was originally a Mac developer before Windows 3.1 came out. I switched over to the PC after Windows garnered a 90% marketshare in just six months. One of my first college papers focused on the real world metaphors and the introduction of the graphical user interface. I also wrote about Harvard Computer Review article on the launch of the NeXT in 1990. I followed Apple keynotes since the late 1990s and regularly read up on Apple patents and developments. I was completely floored by Ars Technica technical reviews on the Mac OS X advances in graphical technologies ten years ago. I purchased a Mac Mini in 2009 to learn iOS application development using MonoTouch and regularly follow Apple patents and developments.]
Microsoft released a preliminary version of immutable collections with mutable performance. It uses many of the same performance tricks that I used to build functional collections such as supporting freezable binary tree data structures. The algorithm used is similar to AVL trees in that it minimizes the difference in heights of two child trees. Red-black trees are not used probably because the number of rotations will still cause the O(lg(n)) allocations in an immutable setting and these allocations are a function of the height of the whole tree..
However, one major difference is the lack of support for constant equality comparison and fast set operations, each of which will limit the usage scenarios for the collection. Set operations in the Microsoft library are O(m lg(m+n) ), but can be implemented in O(m lg(n/m)) and even faster with constant-time equality comparisons O(d lg(n/d)), where d is the minimum of the number of differences between the two sets and m. A common problem which this helps is quickly determining all the changes between two different versions of the same collections, but I often use another solution with O(d). This is important for using immutable data structures pervasively such in a layout engine, where there are potentially many items and the interface needs to remain responsive.
At some point in the future, I plan to open-source my functional collections and other data structures as they are typically the fastest implementations available. In addition, multiple different strategies can be used to optimized for different types of usages such as full persistence, back-tracking, and ephemerality.
UPDATE: I just pulled the performance numbers from my head from a while back, but I think I made a mistake. They look wrong to me. I wrote quickly, and, on reflection, the Microsoft libraries should probably be O(m lg (n+m )), not O(n+m), for the union, intersection, etc operations. I also need to verify the bounds for mine. I’ll probably post a correction with a detailed analysis in the near future.
2013 will be the year of real hardware and software mobile innovation following many years of incremental spec bumps in smartphone technology. A number of smartphone manufacturers, Apple, Samsung, Nokia, and Motorola, are set to release this year highly differentiated smartphones that have been in been development for many years now.
The new technologies we should expect this year:
Samsung is launching the new flagship Galaxy S4 smartphone with technology breakthroughs that “rival color television.” Leaked phones of a new Samsung device appear to be an updated Galaxy S3 to ship alongside the S4. Strong evidence indicates that the Galaxy S4 will include 3D technology, hand gestures, eye-tracking technology and a flexible display. A flexible display is a necessary prerequisite for a tactile touchscreen. The best indications are from Samsung TV, which already ship with most of these technologies. Both Samsung and Nokia representatives in 2011 previously stated that 3D phones would be emerge in 2012; and many rumors even predicted Galaxy S3 sporting a 3D display. Samsung has a few 3D smartphone patents; their announcement teaser also hints with a three-dimensional 4 logo.
Apple has been recently criticized for lack of innovations, somewhat rightly, since many of the improvements in the OS and hardware since the original iPhone have been lackluster. Apple will likely release a major hardware and software innovations that will roll up a multiyear development effort. Based on patents, Apple has apparently branched off its OS efforts to support a complete redesign based on 3D and other advanced input technologies, of which there are many patents. Almost all new features of the past six iterations lie outside the core user interface as Apple has chosen not to invest in a moribund interface. Steve Jobs mentioned in the launch of the iPhone.
There's an old Wayne Gretzky quote that I love. 'I skate to where the puck is going to be, not where it has been.' And we've always tried to do that at Apple. Since the very very beginning. And we always will.
Game-changing technologies require a long incubation period to develop, which is not helped by the annual shutdown and re-launch of a yearly product cycle. According to Apple’s designer Ive, a proper design involves multiple iterations to correct flaws and incorporate feedback from the whole process, so that new features are not simply tacked up but fit into a whole gestalt. For instance, eye-tracking, hover gestures and tactile touchscreens make the whole 3D experience work, as objects angled away from the screen are confusing to touch. Eye-tracking is a high-speed, maximum-accuracy input mechanism that works seamlessly on a 3D display.
Apple’s patents reveal interest in a featureless front phone surface, almost indistinguishable from the back. The display becomes the device and subsumes the functionality previously of discrete components such as cameras, speakers, optical sensors, solar panels, multi-tiered haptics, and touch. One patent allows the home button to disappear and reappear with specialized hardware. In this way, the front panel becomes the whole display.
Recent leaks indicate two new iPhones are at or near production for an upcoming release in the next couple months. A new iPhone 5S in the iPhone 5 form factor but with an “advanced Retina+ display” and a large-screen “budget” iPhone with a plastic case minus the home button. The advanced display is likely the revolutionary new integrated display mentioned earlier. The plastic casing may be a placeholder for an all new case made from LiquidMetal, a metallic glass, via an Apple-patented process.
Motorola and Nokia will also introduce revolutionary technologies, but they will likely be thwarted in market share gains by new technologies from Apple and Samsung. Motorola is working on a new X-phone that will feature a longer battery life, an unbreakable screen, and an unprecedented new technology. My research indicates early work on solar panels. The use of a flexible screen indicates a tactile display. Nokia is likely to introduce new 3D technology, a bendable phone, a hybrid tablet-phone.
I started out on my projects using imperative techniques, but have since rewritten all my code to based on the functional paradigm. It was the only way that I could eliminate the escalating complexity in achieving my ambitious goals.
Although the first programming language I was formally taught in a university was Lisp, much of my early programming background was in C++. I used to be an early advocate of object-oriented programming, but I soon noticed its limitation in programming in a more mathematical way, particularly those involving functions or manipulating symbolic expressions algebraically. This meant that translating ideas into code often resulted in a program that are overly complex with little resemblance to the underlying ideas. Also, the types of problems that I have solved in functional languages were considerably more higher level than in traditional languages using techniques such as pattern matching and unification. I have seen programs in natural language and AI written in C++, where the programmers recreated the Lisp runtime complete with garbage collection and linked data structures.
While there are some formalisms such as sigma calculus, which feels a bit contrived, OO programming does not seem to have a strong mathematical basis. The various principles and practices that accompany object-oriented training resemble a discipline still in the “art” phase rather than “science.” My general feeling is that object-oriented programming is the current “fad” of our times and won’t be as relevant in the middle of the century. FP has a stronger mathematical basis and scales big and small to different types of computing such as over a network.
I am reminded of Wes Dyer, Microsoft C# developer, post on the conceptual simplicity of functional programming to its alternative.
Imperative programming is sometimes reminiscent of a Rube Goldberg machine. Both require meticulous thought to ensure that a process works correctly despite a myriad of state transitions and interdependencies. It is amazing these complicated programs work at all.
Dijkstra pointed out that too many programmers rely on executing a program in order to understand it. The reason is imperative programs lack sufficient underlying formalisms to make guarantees about any but the most trivial of programs. As much as I love a debugger, it is disheartening to need to use it to understand my code.
Languages like Clojure and Scala have incorporated functional programming at the core. Rob Hickey, inventor of Clojure, gave a presentation “Are We There Yet?” in which he identified the sources of complexity as deriving from mutation from his 20 years of software experience. He also has another good presentation “Simple Made Easy.”
CMU revised its computer science curriculum to emphasize functional programming over object-oriented programming. Object-oriented programming was eliminated from the introductory curriculum, because it is “anti-modular and anti-parallel by its very nature, and hence unsuitable for a modern CS curriculum.” Also, the “new data structures course emphasizes parallel algorithms as the general case, and places equal emphasis on persistent, as well as ephemeral, data structures.” When I first learned about functional data structures, I felt that they were just as important as any other data structures taught in my Algorithms course but that the university unwittingly steered us into an imperative programming paradigm.
One comment from the post elaborates that object-oriented programming is anti-concurrent because it’s about state which is shared liberally. It’s also anti-modular because of dependencies. Functional data structures don’t have these characteristics because they are immutable and acyclic. “Clean code” thinking like SOLID principles and others have emerged to compensate for deficiencies in OO programming though “to not much avail.” I would add that mutation also loses information from prior versions of data structures, which can result in unnecessary and awkward limitations in software.
It’s good to see some activity at Microsoft, which in my mind used to be the epitome of stateful programming. Joe Duffy, Microsoft architect and concurrency expert, wrote in his blog on the benefits of immutability.
What about concurrency? Immutable data structures facilitate sharing data amongst otherwise isolated tasks in an efficient zero-copy manner. No synchronization necessary. This is the real payoff.
For example, say we’ve got a document-editor and would like to launch a background task that does spellchecking in parallel. How will the spellchecker concurrently access the document, given that the user may continue editing it simultaneously? Likely we will use an immutable data structure to hold some interesting document state, such as storing text in a piece-table. OneNote, Visual Studio, and many other document-editors use this technique. This is zero-cost snapshot isolation.
Not having immutability in this particular scenario is immensely painful. Isolation won’t work very well. You could model the document as a task, and require the spellchecker to interact with it using messages.... Those kinds of message-passing races are non-trivial to deal with. Synchronization won’t work well either. Clearly we don’t want to lock the user out of editing his or her document just because spellchecking is occurring. Such a boneheaded design is what leads to spinning donuts, bleached-white screens, and “(Not Responding)” title bars. But clearly we don’t want to acquire a lock and then make a full copy of the entire document. Perhaps we’d try to copy just what is visible on the screen. This is a dangerous game to play.
Immutability does not solve all of the problems in this scenario, however. Snapshots of any kind lead to a subtle issue that is familiar to those with experience doing multimaster, in which multiple parties have conflicting views on what “the” data ought to be, and in which these views must be reconciled.
In this particular case, the spellchecker sends the results back to the task which spawned it, and presumably owns the document, when it has finished checking some portion of the document. Because the spellchecker was working with an immutable snapshot, however, its answer may now be out-of-date. We have turned the need to deal with message-level interleaving – as described above – into the need to deal with all of the messages that may have interleaved within a window of time. This is where multimaster techniques, such as diffing and merging come into play. Other techniques can be used, of course, like cancelling and ignoring out-of-date results. But it is clear something intentional must be done.
Joe seems to be working on an experimental operating system based on functional programming ideas.
What fascinates me is this post is how similar much of my thinking is to Microsoft’s regarding background tasks concurrently with editing and the use of multimaster synchronization techniques.
With Apple’s outright win in the trial Apple vs Samsung over trade dress (“cloning”) and patent issues, various people have argued the importance of copying in the development of user interface, citing Apple’s copying of the Xerox Alto user interface.
I came across two articles, explaining that there were substantial differences between Xerox and Apple’s graphical interfaces. The story of Apple’s copying appears to be in many ways a well-known myth.
Apple employees had visited the research labs twice, which is a very limited amount of exposure time for substantial copying to take place. Xerox accept shares of Apple in exchange for the visits. Apple was already at work on bitmapped user interfaces including multiple fonts and graphical capabilities. The Mac and Lisa projects actually predated the visits. Engineers Bill Atkinson and Jef Raskin had known of work at Xerox or interned there.
In addition, Apple’s graphical interfaces sported a number of improvements.
Some inventions such as the mouse were already known prior to the visit and underwent substantial improvements before commercialization.