deeje.com/musings

tappr.tv roadmap, Spring 2013

2013-04-09T07:09:28-07:00

So far, I've released 6 major versions of tappr.tv, with each release having a specific theme:

Wands = the basic input-output architecture
Loops = recording and playing back tracks
Dances = recording and playing back in time with a song
Community 1.0 = publish dances and subscribe to favorite artists
Effects and Inputs = symmetry mode and audio in
Live 1.0 = dynamic BPM-based loops, network IO for real-time multi-device input

I'm working on a 6.2 release that modernizes the code, fixes some critical bugs, and addresses a few minor UI needs, but otherwise doesn't offer any significant new theme.

After that, I've got several big themes to tackle. In no particular order beyond the first:

Remix
Playlists
Wands
Community 2.0
iRadio
Apple TV
3D
Web
Live 2.0

Below are mini specs for each of these. Beyond Remix, I'm open to suggestions on the order.

Remix (v7)

Allows a user to dynamically "join" a dance in the Theater, and auto publish a remix

Allows a user to manually remix a dance = make a new dance by including tracks from other dances (to the same song)

When remixing a dance and user is original publisher, user is asked whether to prepare a new version or a remix

Before publish, check that all the tracks are NOT by just one dancer

On activity (publish/rating/sharing), send a push notification to each included dancer

Theater grid cell must be updated to expose the dancers

Theater dance (grid cell and/or stage) must have a details view

Dance versions = user (or admin) can publish a new version of a dance with changes to song or track list

When updating subscriptions, handle updated dance versions

Add minimum app version info to published dances

Playlists

Need a new UI that allows for many lists

Lists can be manual or smart

Manual list = user can add and remove specific dances

Smart list = user can define predicates that query the server, can remember last date to automatically get deltas

Lists can be published and subscribed

All new users create and publish a list, e.g. name = "deeje's 4 star ratings" dance, where user = user, rating >= 4 stars

All new users get subscribed to "deeje's 4 star ratings".

Published lists generate smart push notifications when going into background, e.g. "deeje's 4 star ratings list just got 4 new dances"

All the lists in Theater become pre-defined playlists

Studio is also a set of lists. Can Studio and Theater also be combined into one?

Wands

wand maker UI = expose many of the properties of wands for user editing

Wand single-touch behavior property = point, line, circle

User Wands Playlist = each user gets a

Non pro users only allowed 3 new wands per week, 12 total. All pro users can create as many as they want.

Non pro users get local push notifications for 4 weeks.

Community 2.0

Builds on Playlists and Wands

Tags = allow user to add tags to a dance

Followers?

ADN?

Newsletter? need to add UX to first launch and settings

Twitter Cards?

iRadio

Add support for iRadio, Rdio, Deezer, Grooveshark

Incorporate into artist subscriptions

Rework the first launch UX for this

Add support for whatever Apple does

Fall back to these services for song thumbnails

AppleTV

A version of the tappr.tv UX that works within the AppleTV SDK.

It wouldn't have the studio interface, but it would have the theater auto-remix feature

And it would have the network IO so that users could pull out an iPhone, put it into Wand mode, and dynamically join the dance

And it would hopefully have iRadio

3D

Wands now include a z parameter, normalized just like x and y

Inputs can be configured to route their inputs to specific wand attributes.

Composite touch pads = combine 2 or more inputs, e.g. two touch pads to control 3D, or several touch pads for a drum kit.

Tracks can also bounded and oriented within the 3D space

Web

Implement the playback engine using WebGL

Use RESTful versions of Rdio, Deezer, Grooveshark, et al for the dance song

Allow visitors to rate and share, optionally becoming members

Home page would have a radio-like experience

Live 2.0

Allow Live dance stages to be persisted and published

OSC input = allow external OSC masters to trigger specific loops

WIST support = allow master to synchronize BPM and timing

Extreme Core Data - 61 NSFetchedResultsControllers in one view

2013-02-12T11:10:49-08:00

Ever since I got into app development (4 years ago now), I've had the mindset that "no app is an island", and that apps should be in constant contact with a cloud. This invariably means designing and developing clients using Core Data and domain-specific synchronization engines. One of my most sophisticated examples so far is Thirst.

Thirst is a different kind of news reader, one that marries natural language processing (NLP) and machine learning (ML) to quickly and seamlessly present the most compelling topical news to an individual reader at any given moment.

In this post, I want to focus on the power of NSFetchedResultsControllers in presenting rich data views.

When it comes to news readers, one of the key features for me is the ability to quickly and easily understand what's new. In Thirst, we do this with the subtle concept called item freshness, much like RSS aggregators of old, and Mail.app. This is what a typical Thirst newspaper view looks like:

The newspaper consists of 20 topics, arranged in a newspaper-y grid, where each topic shows an image, title, and a series of thumbnails. Each thumbnail represents a fresh comment, tweet, or news article related to that topic.

Touching on a topic brings up a topic detail view like this:

Now as the user peruses and reads the various items in the topic, they are marked as "stale": the numbers dynamically go down in the topic detail view, and the thumbnails dynamically disappear from the topic in the newspaper view.

This is, of course, accomplished using NSFetchedResultsControllers from Core Data. I won't go into the basic of using NSFetchedResultsControllers, as there are plenty of gifted writers who have covered this, such as Ray Wenderlich and Marcus Zarra. Instead, I'd like to articulate the power of Core Data thru numbers:

Model classes represent 100s of comments, chatter, and news articles, with relationships to 1 or more topics.
The newspaper uses an NSManagedObjectContent (MOC) and NSFetchedResultsController (FRC) to find all the "active" topics, as defined by our NLP server.
Each found topic is represented by a topic box object. There are always 20 topics in the newspaper.
Each topic box has its own MOC as well as 3 FRCs to find all the related model entities for that topic.

So, that's a total of 21 MOCs and 61 FRCs for the one newspaper view. And the performance is buttery smooth.

The take-away here is the MOCs and FRCs are fairly cheap constructs, and you shouldn't be afraid to use them liberally in your code, if the use case warrants it.

There are few more details to achieving this performance:

Each topic box MOC is a private queue context, so that fetching the results and detecting changes all happen in the background.
The thumbnail display in each topic box is assembled via an NSOperation on a shared NSOperationQueue.
None of this would have been possible if it weren't for Michael Heyeck's write up of faulting model objects during MOC merging.
In the topic detail view, the various tabs indicate how many fresh items are available. Rather than reconstruct the same set of FRCs for each relationship, the tabs use KVO on the topic boxes.

How tappr.tv Is Made

2012-10-28T13:16:35-07:00

tappr.tv is all about community, and ultimately about building beautiful experiences together. In that spirit, tappr.tv itself is built using a wide assortment of open source projects, kits, and frameworks, without which I couldn't do what I do.

Version 5 was built in part using:

QuickTIme	No, there's no QuickTime inside tappr.tv. But ever since I learned about this fantastic temporal framework, I've been fascinated with the idea of tools and communities for music visuals. tappr.tv is the culmination of my thoughts spanning almost 15 years. Inspired by the filmstrip concepts of QT, a "dance" in tappr.tv ultimately boils down to a long list of events. "Inputs" normalize data from various hardware services, "recorders" capture and play back those event streams, and "outputs" render. All the parts are loosely coupled, making it easier to leverage into new feature directions.
UIKit	I'm aiming the UI as be as invisible as possible. One thing Photoshop did right in their UX is try to disappear. The real genius of UIKit is all the input capabilities: touch, accelerometers, and audio are all enabled in v5, and more are already being built.
OpenGL Cocos2D	Particles-based wands are just the first feature when it comes to visuals. By leveraging OpenGL, tappr.tv is prepared to add more wand types, more depth, and interactive spatial experiences. I'm still on OpenGL and Cocos2D v1.x though, unsure if I can still render UIKit at the same time as the CCDirector in v2.
CoreData	The structure of dances is rather complex, by design. Each dance is comprised of 1 or more tracks of normalized input data, which is recorded by a user using a wand. By separating all these elements out, I'm laying the foundation for a future release of tappr.tv that includes a variety of Remix features. The granularity also enables sophisticated MVC architectures for various areas of the app. For instance, the Studio has a list of tracks on the left, and a "stage" view on the right, where the various tracks can be touch recorded and played back. Both views are actually using NSFetchedResultsControllers to dynamically detect changes. So, changing the input or mirror attributes on the left get saved and propagated to the view on the right, which actually handles the implementation of those two features. Creating a new track on the left similarly propagates to the right, where an appropriate CCLayer is instantiated and managed.
Parse.framework	I've written several Heroku/RoR servers to work with iOS apps, and its laborious. Creating synchronization engines requires replicated schemas on both sides, and there are few general frameworks for creating truly asynchronous CoreData-Social model layers. When I finally got to the social release of tappr.tv (v4, Nov 2011), I probably spent a month staring at the task at hand, while also wondering if the brand new Parse team had the chops to deliver a solid platform. I took a chance on them, and have been thrilled ever since. Stupid simple schema development, support for synchronous APIs (YEAH!), server-side logic, REST/JSON for websites, third-party social authentication… good good stuff. I'm particularly thrilled with their support for APNS and their "channels" feature. In tappr.tv, each artist has a channel, and as tappr.tv scans your music, subscribes you to the top 100 artists in your music collection. Then when someone else publishes a dance to a song, a push notification is broadcast on that artist's channel. Similarly, each user in tappr.tv has her own push channel, so when others watch, rate, and share dances, tappr.tv sends a push notification back to that user.
NSOperations	Because of the complexity of a dance structure, I needed to write rather complex synchronization logic, which was best accomplished using NSOperations as wrappers for long strings of synchronous calls to Parse. For instance, publishing a dance involves up to 3 network saves for song and artist info, up to 6 network saves per track, and 2 additional network saves for the dance record itself. I just can't imagine how complex this 11-step pseudo-transactional logic would be if I had to string together a set of asynchronous calls with delegate+selector or block-based callbacks.
AQGridView	This class enabled so many of us to jump into iPad with experiences like Apple's updated apps demonstrated. Of course, now that Apple has released UICollectionView, AQ and I must bid adieu soon. Big kudos to Alan Quatermain for releasing this grid implementation.
Spotify CocoaLibSpotify AirPlay	In tappr.tv, Spotify premium members can watch just about any dance in the Theater. tappr.tv uses song information in each published dance to find a matching song in Spotify, and choreographs the two. This, along with continuous playback and AirPlay support, brings a dance party to any room in the house. (One aside, sometimes songs disappear from Spotify for reasons I can probably understand, and then the match results may only contain covers or parodies, which can make dances either somewhat or entirely off. C'est la vie.)
David Cairns Demetri Miller	These two gents provided source online that enabled the tappr.tv audio input feature. In the Studio, I convert songs from iTunes into a local audio file, then set up an AU chain in order to play and detect at the same time.
MGSplitViewController	I was drawn to this class because it would work inside a UINavigationViewController, thus enabling me to have different UXs in certain areas of the app. I am, however, interested in developing a column-view navigation view controller, where the columns can adjust as they move from primary to indexed layers.
ASIHTTP	This class provided a solid S3 API. But, as you probably know, its been superseded by the likes of AFNetworking. I was so saddened to see the abuse the originator of this repository received. I hope AllSeeing-i lives long and prospers.
iVersion	stupid simple implementation of release notices with notes. The +load class method does all the work, so you literally just add the code to your project and you're done.
JSONKit	Man, this JSON lib screamed in its day. Like so many things though, Apple needed and did provide a native API, which will eventually take over.
Mugunth	Some nice blocks-based categories for various Apple foundation and UI classes.
InAppSettingsKit QuickDialog	Used in the Settings area of the app.
ShareKit	Kudos to all who contributed to this open source project, clearly a need and Apple seemed to agree with the overall approach to the API in iOS 6.
CMPopTipView	This is used in all of the bubbles that present themselves during the first launch UX.
Airship	This is what manages the store for Wand Packs. Unfortunately, I am considering an entirely different approach to the freemium store UX.
Touchpose	For when I start giving demos on stage ;-)
Trevor Harmon	Thanks for sharing with us the best way to resize images.

Coming in v6:

AyncNetwork.framework

Links for 2012-09-14 [del.icio.us]

2012-09-15T00:00:00-07:00

Links for 2012-09-12 [del.icio.us]

2012-09-13T00:00:00-07:00

Links for 2012-09-10 [del.icio.us]

2012-09-11T00:00:00-07:00

Is Gift-Giving Bad For The Economy? : NPR

Links for 2012-09-07 [del.icio.us]

2012-09-08T00:00:00-07:00

CITI Home Page

Links for 2012-09-05 [del.icio.us]

2012-09-06T00:00:00-07:00

Rise of the Social Art Apps

2011-11-10T10:47:35-08:00

There's a new class of apps emerging, what I call Social Art apps. These are apps that let you create something, then see and rate creations by others.

Besides offering the tools to create, all of these apps allow you to see and rate content by others from within the app itself and see leader boards of content and creators. Many also allow you to comment and post links on social networks like Facebook and Twitter.

Here are some of the more interesting ones:

Doodley
medium: simple painting
audience: kid-centric
Spray Can
medium: graffiti
audience: youth
Sketch Club
medium: drawings
audience: serious painters
Mixel
medium: clip art
audience: casual
tappr.tv
medium: visuals for music
audience: music fans

tappr.tv Roadmap

2011-10-08T23:25:27-07:00

I have a grand vision for tappr.tv, and I've been steadily working my way thru a roadmap to get there. Below are the various features of the roadmap. I'm thinking of incorporating a "Roadmap" panel inside of tappr.tv, so that you can vote on order of the features.

Wands (tappr.tv 1.0)

Wands are the core of tappr.tv, something that responds to your taps and tilts by putting stuff on the screen. The first generation of wands are based on a particle emitter engine with over a dozen parameters, leading a near-infinite variety of visual styles.

Tracks (tappr.tv 2.0)

Tracks allow you to record and layer multiple wand motions on top of each other.

Dances (tappr.tv 3.0)

Create tracks of wand motion recordings to your favorite songs.

Community (tappr.tv 4.0 - Coming Soon!)

Join the tappr.tv community to publish your dances and subscribe to dances to your favorite musical artists.

Leaderboards

Compete for the top spot on weekly, monthly, and all-time leader boards based on views, votes, and shares by the tappr.tv community.

Timeline

See and edit your dances on a timeline for more precise creation.

Comments

Add comments to dances, both yours and others, in the timeline.

Remix

Like that bass track by another tappr.tv member, but want to do your own vocals? Create a remix that incorporates tracks by others along with tracks of your own.

Visual Effects

Apply visual effects to tracks, such as echo, chorus, symmetry.

3D

Moving beyond wands in a single plane, be able to position and orient your wand recordings in a 3D space, and moving around that space while the dance is being played.

Wand Editor

Ability to create and customize new particle wands, and dynamically change wands in tracks (e.g. alpha, size, color).

Strings

Wands that require two touch points, that react to your inputs in interesting ways.

Geometries

Wands that are 3D shapes (spheres, cubes, etc), that react to your inputs in interesting ways.

Audio Input

Create tracks of wand motions by singing along to your favorite songs.

Drummer Input

Create tracks of wand motions by tapping the sides of your iDevice.

Gyro Input

Wave your iDevice around in the air to generate 3D wand motions

Network Input

Capture and present input from multiple paired iDevices, each using their own input methods.

Magic Backgrounds

Generate and customize background tracks of wand motion automatically.

VJ

A mode in tappr.tv, comparable to Studio and Theater, that is customized for live and collaborate performances.

OSC

Integrate tappr.tv into your actual music studio to systematically animate your music.

WIST support

Korg's open source library for synchronizing several apps via BlueTooth.

Desktop Playback

A tappr.tv music visualizer plugin that integrates with iTunes on your desktop.

Web Playback

Ability to watch a dance via a web browser.

Programmable Wands

Wands that can be defined using a scripting language such as Lua.

Links for 2011-10-04 [del.icio.us]

2011-10-05T00:00:00-07:00

None

Seeking RoR engineer in SF

2011-06-15T13:32:46-07:00

About Me

I've been making software experiences for over 20 years, recently moved back to coding and into mobile. I've worked on numerous projects and have many of my own in the app store. All of mine have a mobile-cloud architecture to enable social experiences, and I'm fluent in iOS <-> RoR. The goal of my latest project, http://tappr.tv, is to digitize our impulse to dance together to our favorite music, and I'm starting the next phase of adding a cloud layer. I could do it myself, literally, but I now have motive enough to accelerate things significantly. You can learn more about me starting here http://deeje.com

About You

Ideally… you've already built several mobile-cloud apps that are available in the app stores. You have experience with Heroku, RoR, AWS, SendGrid, and APNS APIs. You code fast and you code well. You can estimate, communicate, and deliver. You live in SF and can commit to up to 2 months of full time work, the first 2 weeks of which will be dedicated to finalizing UX, architecture, and specs.

If this sounds interesting, ping me deeje@tappr.tv and let's meet in person to discuss.

Introducing tappr.tv

2010-12-02T22:26:59-08:00

tappr.tv is a new iOS app for visual scratching. What's visual scratching? Well, DJs play music and scratch records, but VJs only seem to play videos… I wanted a more interactive visual experience.

Here are some samples of visual scratching:

In tappr.tv, you have a selection of visual "instruments" that you can choose from, and then multi-touch and gesture to play the instrument on your screen. On the iPad, the interface looks like this:

And on the iPhone and iPod Touch, the interface looks like this:

You can make all the controls go away by tapping the Fullscreen button in the lower right corner.

There are 20 instruments included with tappr.tv, and here is a gallery of some of my favorites:

If that wasn't enough, you have two options for even more instruments!

First, you can tap the Download button (at the top of the instrument list), and browse the available instrument packs. Packs include streaks, circles, sticks, etc.

Alternatively, you can generate random instruments by tapping the Random button (also at the top of the instrument list).

If you find an instrument you like, tap the Star button and give your new instrument a witty name!

As you play with various instruments, you can take snapshots of them by touching the Camera button (in the upper right corner on iPad and lower left corner on iPhone/iPod). This will start a time-series of 5 snapshots, 1 per second.

After the snapshots are captured, you'll be taken to the snapshots view, where you can browse and view the various snapshots for the current instrument. If you see one you like, share it by tapping the Action button in the lower-right corner. You can share your snapshot to Facebook, to Twitter, via Email, and more.

One last feature, for iPad only right now… video out! This requires a video out cable, of course, which come in two varieties. A composite or component cable will support a 480p resolution, while a VGA cable will 1024x768. And of course, it just works… plug in the cable, and the instrument plays on the TV while you still have all your controls on the iPad.

This is just the beginning for tappr.tv, too. If you're curious about our roadmap, tap the Home button at the top of the instruments list, then drop into the Settings area to see what features we're planning next!

tappr.tv has been submitted to Apple for approval, and should be in the app store by mid-December 2010.

I hope you love tappr.tv as much as I do. Tweet us your thoughts @tapprtv

Bansai Tree 3

2010-07-22T21:18:08-07:00

Bansai Tree 3
Originally uploaded by deeje

ommmmmmmm

iPad First Impressions

2010-04-05T10:34:04-07:00

Overall, I think the iPad is great, and I've already got a list of features I want to see soon…

iPod and Airport Express - would love to play songs directly from my iPad over my home stereo, without having to plug in a wire.
Users and/or guest accounts - want to be able to hand my iPad to my kids, spouse, or friends who are visiting, and not worry about them playing my games (in progress), sending emails or tweeting from my account, or buying things with my credit card.
iMagazine - not sure any of the periodicals have the use case right for iPad… what I want is to subscribe, and have new content delivered automatically, all of which speaks to a background process. What I don't want is each periodical with its own background process. Podcast content has the same issues.
Springboard 2.0 - As it is, the home screen only scales to a few pages of apps, and as with the iPhone, doesn't scale well to hundreds of apps. This is further compounded on iPad because sometimes all I want to see are iPad-specific apps… personally, I want to tag all my apps and then pivot on those tags.
Family iPhoto Cloud - my wife and I both take pictures, and store them on different computers… how can my iPad pull all those photos in, complete with tags, faces, event info, etc?

Media, Media, Everywhere!

2010-01-27T20:41:16-08:00

Here's something I've noticed about the iPod/iPad user experience… my media is scattered all over the place!

iTunes - buy songs
iPod - play music, videos (movies and TV shows), audiobooks, podcasts
Videos - play TV shows and movies
App Store - buy apps
Home screen - run apps
Photos - view images and play movies
Camera - take and view images and videos
MobileMe - view photos from the web
iBooks - buy and read books

That's a lot of places to go to get to my media. What's more, sometimes the store for a kind of media is a feature of its related player (iBooks), and sometimes its a product in and of itself (iTunes/iPod, AppStore/Home). I think that lack of consistency is a problem that will only get worse.

Links for 2009-10-05 [del.icio.us]

2009-10-06T00:00:00-07:00

See It, Follow It