With Father’s Day upon us, I thought I’d share a story about a running joke in our family for the better part of our daughter’s childhood.

One value I’ve tried to instill in my daughter over the years is to never understimate the importance of silliness in everyday life, nor to overlook it in one’s daily interactions with friends, family, and co-workers. You never know what rituals might be born at any given time.

And similar to a new product feature, remember to document, document, document — photos, videos, diaries, blog posts, whatever suits your style. Your older self will thank you for reminding them of all the good stuff that happened along the way.

Happy Father’s Day!

Providing auth in your platforms can get really expensive really quickly, as your 3rd-party vendor costs grow as your user base grows. I had a chance to write about a great way to implement authentication, authorization, and SSO in a scalable and practical manner. I hope you like it.

https://betterprogramming.pub/low-cost-auth-identity-and-single-sign-on-f8adad300025

Hello, hello. It’s been a while but I’m finally getting a chance to document some of my favorite engineering design patterns and tools adopted over the years, and how to leverage them in your own systems.

First up is a great lever to pull when addressing issues of scalability and domain ownership in your growing platform: GraphQL Federation. I hope you like it.

GraphQL Federation for Cross-domain Bliss

The premise was to able to script these behaviors dynamically and make them available for users to apply to their videos to create experiences like this:

For our needs, it wasn’t going to be possible to extract video frames in real-time in the order needed to convey reverse movement. The only possible solution involved the creation of a companion video file that contained a reverse ordering of the video frames of the original. Also, since BeatCam processes input audio sources using a completely different pipeline from the video, we could conveniently ignore any audio in the original and simply output a silent, reversed version of the original.

A nice solution to this problem is outlined by Andy Hin in which an AVAssetReader can be used to read the input frames and their corresponding timestamps. One can keep an array of the frame buffers, and when it’s time to write them to file, you use the same original timestamps in order, but use frame data starting at the end of the frame array and append to the output in time descending order.

However, Andy’s actual implementation is geared more toward reversing short videos of 30s or less. Longer videos will cause memory usage to blow up since one can’t keep a large number of frames around in memory before starting to output to file. And in BeatCam, any video in your library or that you shoot with the camera is fair game, so any implementation would need to function independently of input video duration.

In addition, this processing within BeatCam needed to occur behind the scenes since the user could be occupied with other aspects of the UI, so it was imperative that it run in its own thread. And if this call was now asynchronous, then some kind of completion block or delegate callback would need to be implemented.

One possible solution to the memory issue is to perform an initial reconnaissance of the entire file with respect to timestamps and frames, to organize this data into an array of passes where each pass contains a maximum number of frames (in this case, 100), and to then process each pass by extracting just the frames in the given pass and writing to output.

The Recon Stage

// main array to hold presentation times
NSMutableArray *revSampleTimes = [[NSMutableArray alloc] init];

// now go through the reader output to get some recon on frame presentation times
CMSampleBufferRef sample;
int localCount = 0;
while((sample = [assetReaderOutput copyNextSampleBuffer])) {
  CMTime presentationTime = CMSampleBufferGetPresentationTimeStamp(sample);
  NSValue *presentationValue = [NSValue valueWithBytes:&presentationTime objCType:@encode(CMTime)];
  [revSampleTimes addObject:presentationValue];
  CFRelease(sample);
  
  localCount++;
}

This method works only because AVAssetReader has the ability to read from input file for a given time range with little cost in initial seek time to the beginning of the range. Thus we can use the array of input frame timestamps to create the main array of dictionaries that will define the passes.

Creating the Array of Pass Info

// each pass is defined by a time range which we can specify each time we re-init the asset reader
    
// array that holds the pass info
NSMutableArray *passDicts = [[NSMutableArray alloc] init];

NSValue *initEventValue = [revSampleTimes objectAtIndex:0];
CMTime initEventTime = [initEventValue CMTimeValue];

CMTime passStartTime = [initEventValue CMTimeValue];
CMTime passEndTime = [initEventValue CMTimeValue];

int timeStartIndex = -1;
int timeEndIndex = -1;
int frameStartIndex = -1;
int frameEndIndex = -1;

NSValue *timeEventValue, *frameEventValue;
NSValue *passStartValue, *passEndValue;
CMTime timeEventTime, frameEventTime;

int totalPasses = (int)ceil((float)revSampleTimes.count / (float)numSamplesInPass);

BOOL initNewPass = NO;
for (NSInteger i=0; i0) {
      passStartValue = [NSValue valueWithBytes:&passStartTime objCType:@encode(CMTime)];
      passEndValue = [NSValue valueWithBytes:&passEndTime objCType:@encode(CMTime)];
      NSDictionary *dict = @{
        @"passStartTime": passStartValue,
        @"passEndTime": passEndValue,
        @"timeStartIndex" : [NSNumber numberWithLong:timeStartIndex],
        @"timeEndIndex": [NSNumber numberWithLong:timeEndIndex],
        @"frameStartIndex" : [NSNumber numberWithLong:frameStartIndex],
        @"frameEndIndex": [NSNumber numberWithLong:frameEndIndex]
      };
      [passDicts addObject:dict];
    }
    initNewPass = YES;
  }
  
  // if new pass then init the main vars
  if (initNewPass) {
    passStartTime = timeEventTime;
    timeStartIndex = (int)i;
    frameStartIndex = (int)(revSampleTimes.count - 1 - i);
    initNewPass = NO;
  }
}

// handle last pass
if ((passDicts.count < totalPasses) || revSampleTimes.count%numSamplesInPass != 0) {
  passStartValue = [NSValue valueWithBytes:&passStartTime objCType:@encode(CMTime)];
  passEndValue = [NSValue valueWithBytes:&passEndTime objCType:@encode(CMTime)];
  NSDictionary *dict = @{
    @"passStartTime": passStartValue,
    @"passEndTime": passEndValue,
    @"timeStartIndex" : [NSNumber numberWithLong:timeStartIndex],
    @"timeEndIndex": [NSNumber numberWithLong:timeEndIndex],
    @"frameStartIndex" : [NSNumber numberWithLong:frameStartIndex],
    @"frameEndIndex": [NSNumber numberWithLong:frameEndIndex]
  };
  [passDicts addObject:dict];
}

For each pass, a new AVAssetReader is initialized and the time range to be read is set to the subset of frames relating to the current pass.

Passes are read in reverse order, e.g. last pass to first, and in each pass, frames are written from the end of the pass to the start, which effectively writes all frames in the file in reverse order with capped memory usage (the number of frames in a pass) regardless of the length of the input video. This means the last pass will contain the total number of frames modulo the number of frames per pass. For example, if the input video has 456 total frames, and we are processing 100 frames per pass, then we will have 5 total passes, with the last pass containing 56 frames. This last pass will be the first one written to output, followed by 4 more of 100 frames a piece.

Using the Pass Array to Write the Output

int frameCount = 0; // master frame counter
int fpsInt = (int)(fps + 0.5);

// now go through the read passes and write to output
for (NSInteger z=passDicts.count-1; z>=0; z--) {
  NSDictionary *dict = [passDicts objectAtIndex:z];
  
  passStartValue = dict[@"passStartTime"];
  passStartTime = [passStartValue CMTimeValue];
  
  passEndValue = dict[@"passEndTime"];
  passEndTime = [passEndValue CMTimeValue];
  
  CMTime passDuration = CMTimeSubtract(passEndTime, passStartTime);
  
  int timeStartIx = (int)[dict[@"timeStartIndex"] longValue];
  int timeEndIx = (int)[dict[@"timeEndIndex"] longValue];
  
  int frameStartIx = (int)[dict[@"frameStartIndex"] longValue];
  int frameEndIx = (int)[dict[@"frameEndIndex"] longValue];
  
  CMTimeRange localRange = CMTimeRangeMake(passStartTime,passDuration);
  NSValue *localRangeValue = [NSValue valueWithBytes:&localRange objCType:@encode(CMTimeRange)];
  NSMutableArray *localRanges = [[NSMutableArray alloc] init];
  [localRanges addObject:localRangeValue];
  
  // reset the reader to the range of the pass
  [assetReaderOutput resetForReadingTimeRanges:localRanges];
  
  // read in the samples of the pass
  NSMutableArray *samples = [[NSMutableArray alloc] init];
  while((sample = [assetReaderOutput copyNextSampleBuffer])) {
    [samples addObject:(__bridge id)sample];
    CFRelease(sample);
  }
  
  // append samples to output using the recorded frame times
  for (NSInteger i=0; i= revSampleTimes.count) {
      NSLog(@"%s pass %ld: more samples than recorded frames! %d >= %lu ", __FUNCTION__, (long)z, frameCount, (unsigned long)revSampleTimes.count);
      break;
    }
    
    // get the orig presentation time (from start to end)
    NSValue *eventValue = [revSampleTimes objectAtIndex:frameCount];
    CMTime eventTime = [eventValue CMTimeValue];
    
    // take the image/pixel buffer from tail end of the array
    CVPixelBufferRef imageBufferRef = CMSampleBufferGetImageBuffer((__bridge CMSampleBufferRef)samples[samples.count - i - 1]);
    
    // append frames to output
    BOOL append_ok = NO;
    int j = 0;
    while (!append_ok && j < fpsInt) {
      
      if (adaptor.assetWriterInput.readyForMoreMediaData) {
        append_ok = [adaptor appendPixelBuffer:imageBufferRef withPresentationTime:eventTime];
        if (!append_ok)
          NSLog(@"%s Problem appending frame at time: %lld", __FUNCTION__, eventTime.value);
      }
      else {
        // adaptor not ready
        [NSThread sleepForTimeInterval:0.05];
      }
      
      j++;
    }
    
    if (!append_ok)
      NSLog(@"%s error appending frame %d; times %d", __FUNCTION__, frameCount, j);
    
    frameCount++;
  }
  
  // release the samples array for this pass
  samples = nil;
}

You can see this in action in the app’s video filters as exemplified by some of the posts on BeatCam’s Instagram page

The class used for creating reverse video, along with a sample XCode project is available on Github. I hope you find it useful.

—

One aside: if you do find that you need to include a reversed version of the audio, then you will need to extract the audio from the video, reverse the order of the audio samples (make sure you take mono vs stereo into account), and then export a new video using the newly created reversed video and audio tracks.

The export, in this case, is achieved by creating a new AVMutableComposition, defining the tracks, length, and other details, and then exporting to file using AVAssetExportSession. Consult Apple’s documentation for a full explanation of how this is achieved.

My daughter Alana is 5, and is just starting to discover professional sports and the differences between them. So last week, I thought she might have an interest in a 4pm game between the Eagles and the Ravens. The dialogue went something like this:

Me: Alana, who do you want to win – the Eagles or the Ravens?
Alana: The Eagles or who? The Raisins …?
Me: No, the Ravens
Alana: (thinking) … I want the Raisins to win
Me: No, no, it’s the Ravens, not the Raisins
Alana: (spots closeup of Haloti Ngota) Wow! Look at that giant Raisin!

I’m now hoping that the Ravens continue to be the Raisins for a few years to come, as it is now standard practice to find out who the Raisins are playing this week. Turns out it was the Patriots in a hard-fought contest last night, and now the Browns this coming Thurs.

“Knicks vs. Heat, April 15: LeBron James says he doesn’t want to play the “villain” role anymore, but he won’t be able to avoid it on this night. James will be making his third trip to the Garden since spurning the Knicks [and others] to form a big three with Dwyane Wade and Chris Bosh. This matchup is Game No. 60 of the season, so it could have serious playoff implications.”

via Rapid Reaction: Knicks’ 2011-12 schedule – Knicks Blog – ESPN New York.

This will be a good one to revisit in April …

The one caveat of this process is that I’ve only been successful in setup and sync with an existing remote directory, and then importing it locally, not the other way around, e.g. exporting local to remote (though I have not tried very hard in this area — for whatever reason, the target has always existed whenever I’ve set up sync). So a remote target folder of some kind must already exist before you sync it to a local one, if you’re using these instructions.

To equate a local directory with a pre-existing remote one:

1. Help > Software Updates
2. Choose Available Software Tab
3. Click Add Site…, and then put http://eclipse.jcraft.com/ for the URL field.
4. Choose SFTP Plug-in, and then click Install …, and then restart after the install
5. Using Project Explorer view, make a new local project
6. Create a folder within that project that will map to a remote folder
7. Right click on the local folder in Project Explorer and choose Import
8. In the Select dialog, choose Other > SFTP
9. Choose the local folder as the one to receive the import
10. Create a new SFTP site and give your host and login info
11. If successfully connected to remote site, you should get the remote directory tree
12. Choose the remote folder to import
13. The remote files should now be in the local folder

Now to keep sync between these two:

1. Go to Window > Show View > Other… > Team > Synchronize
2. Click the first upper right icon in the Synchronize tab
3. Choose SFTP from the Synchronize dialog
4. The previously mapped folder should appear with all files selected
5. Click Finish
6. Eclipse will ask you if you want to use Team Synch perspective. Personally I don’t use this. I just keep the Synchronize view as part of my workbench

To test:

1. Make a change in a file within the local folder
2. It should then appear in the Synchronize view in Outgoing mode
3. Using the buttons at the top of the Synchronize tab, or the right-click menus on any of the items that have changed, you can upload or check the diff between the local and remote counterpart

So there you have it. For projects that involve multiple people, I would recommend using a version control system like Subversion, but for quick one-person, one code-base jobs, this works nicely …

ActiveMusician.com is winding down their Classic Vibe Strat Giveaway this Friday. How does one enter? Glad you asked.

Simply give a like to ActiveMusician’s Facebook page and you’re entered. Right now, this is at ~1500 likes, so that’s your chances. Not bad for a guitar with a street value of $350.

Guitar will ship from our warehouse in NJ same or next biz day — depending on how long it takes to hear back from the winner (my guess is that it won’t take long).

When I first starting learning to play guitar, and after the initial absorption of chords, scales, strumming, and the like, the focus was on playing all the classic guitar riffs from the bands of the day. Throw in an electric guitar copy and a cheap practice amp from my folks one birthday and all you could hear from my room was many out-of-tune distorted licks all day long.

Eventually, the guitar came into tune (better ears, plus a Boss tuner), the distortion a bit better, but the premise was the same. Learn all the most complicated guitar music note-for-note, rhythm and lead parts both, until I could play along with the recordings with ease.

This was not a strategy that I alone cultivated — in my town, this was the standard. Everyone wanted to be the local hotshot guitarist, and those in the mix were practicing many hours daily to make this happen. It was like our local version of the Guitar Olympics.

There are a few situations in which these skills really shine, most notably onstage with a band, and with all your own gear. However, more often than not, life is not like that. It might be a friend whose spirits need to be picked up and there’s an acoustic guitar laying around. Or at a house party and people want to hear you play a song alone with a roommate’s beater guitar.

Being able to shred every Metallica lick is a valuable skill, but in these situations, knowing, playing, and singing (I’ll get to this) is infinitely better. No one wants to hear the Best Guitar Licks from Metallica — they want to hear a Metallica song. A good musician should be able to play the music that they care about in any situation, any environment, with whatever tools are available.

This is where the song comes in. It was written for a reason and is the part with which the listener ultimately connects. When you can replicate as much of that as possible, using whatever means are available, the connection becomes that much stronger.

And because the listener’s bond with the song is so strong, more often than not, they’ll forgive your singing if it turns out to be poor, as long as it’s heartfelt. If you can tune a guitar, then you know what “in tune” and “out of tune” sounds like, so you should be able to correct your vocal pitch to the point where you can at least carry the tune. If you can’t, you just haven’t spent enough time doing it. Practice it with as much gusto as your playing and you’ll see the difference.

Now, presumably if you’ve learned riffs for various songs, then you know how those songs go. It’s just a matter of remembering the chords, the song structure, and the lyrics. To aid in this, you can look up the chords and melody on tab sites like Ultimate Guitar, or if you want the official stuff, check out any online seller of music songbooks.

For me personally, I never knew many complete songs until my daughter was born. When she was one year old, my wife and I could barely get her to eat while sitting in her high chair. However, we eventually found out that if I sang and played guitar while food was in front of her, she’d become absorbed in the music and start to eat without noticing.

Thus, playing for her during mealtime became standard practice and I needed to start learning songs in a hurry. So I pulled out my Beatles Complete songbook and started memorizing a new tune daily until enough I had enough repertoire to last a few meals.

In the end, it isn’t about being a jukebox or a karaoke machine. It’s simply tapping the love of music shared by yourself you and the listener via your performance.

And that’s a very powerful thing. Can you think of a better reason to be learning & playing music?

Even with an artist as innovative as James Taylor, I assumed that most aspects of his guitar style and technique would be relatively orthodox. This turned out to be anything but.

Though there are many great musical devices that he employs throughout his wide ranging repertoire of folk-rock hits, I find the most fascinating to be the way he approaches fingering chords. Specifically, his method for fingering the standard A major and D major chords deserves special attention.

Anyone who’s spent time with the guitar has probably delved into the classic CAGED guitar chords. These are your most commonly played major chords for C, A, G, E, and D using open strings. E, A, and D are especially interesting because these three are all variants of the same fingering, just starting on a different root or string.

In any event, all theory aside (though I’ll come back to this), most guitarists would finger a D chord as shown at below left. However, James fingers a D chord as shown below at right.

Standard D major guitar chord

James Taylor’s D major guitar chord

This is not a hard substitution to make, and in some ways, feels just as natural or even more natural than the commonly accepted fingering.

However, when you start to look at the A major chord (below left), James uses a similar approach — an approach that in my estimation requires magic fingers. The standard fingering for an A major chord is shown as left, but James uses a fingering shown at below right that is the exact inverse.

Standard A major guitar chord

James Taylor’s A major guitar chord

I do not have the biggest hands, and I can barely get my fingers to fit on or below the 2nd fret using this configuration. To make matters worse, James is 6’3″ with much larger hands, so either he has the world’s tiniest fingertips (this could be part of the equation), or he has magic fingers that can move in and out of this type of fingering with ease.

Taking this issue one step further, James often capos up to the 3rd fret for songs like “Fire and Rain” and “Something In The Way She Moves”, so that his A chord is actually a C chord (for vocal purposes). If you look at a guitar, you can see that as you go higher up the neck, the space between frets becomes narrower and narrower. But he still employs the same fingering technique.

From a theory standpoint (unfortunately, I did say I’d come back to this), James’ approach to fingering these chords makes sense because, you can easily move from major and minor variants of the same chord simply by moving your 1st finger one fret lower (see below).

James Taylor’s A major guitar chord

James Taylor’s A minor guitar chord

However, the main advantage of the conventional approach is that you can throw in suspensions with ease. With James’ approach, you need to be pretty nimble with your pinky to make it happen (a whole different topic for another time), but if you have magic fingers, of course, well, no problem.

The only other variable in this equation is that the guitars he uses are somehow of a larger scale. Custom made by James A. Olson Guitars, no mention is made of anything in the design that would favor this technique.

Thus, I propose that James Taylor has magic fingers.