John Crosby

Aptana SDOCML & Code Hinting Support

John Crosby — Fri, 06 Jan 2012 18:43:03 +0000

Prompted by this post, I’ve been working on a tool that uses jsduck and Adobe AIR to create the SDOCML necessary for Aptana code hinting. Specifically I’ve been playing with EXTJS 4.0 becuaseof some interest in Sencha.

In the near future I’ll post the SDOCML files that I’ve created for the code hinting and once I get the tool looking presentable and working efficiently, I’ll post the source of that to GitHub.

Here is the current version of the file. I’m not completely sure is has everything it needs. So, it is a work in progress and I’ll post updates as I create them.
extjs4.0-v0.1.sdocml

If you’d like the SDOCML for EXTJS 4.1 Beta
extjs4.1.beta-v0.1.sdocml

Take the PIA out of load testing AMF services

John Crosby — Wed, 02 Nov 2011 17:01:00 +0000

Testing service APIs is a big pain. More so when you are using a format that isn’t widely supported by testing tools. Load testing AMF services…ugh.

Recently I was tasked with finding a way to load test some AMF services built using PHP. In the past I had used jMeter for load testing. So, that is where I started. jMeter is a good tool, but not for AMF. A few extensions for jMeter have been built for AMF, but they are a PIA to setup.

I found a couple of tools that have made load testing AMF serivices a snap. Both of the tools are from SmartBear, and both are open source and free to use. Bonus!

Tool one: soapUI
Tool two: loadUI

The following are the basic steps to get a simple load test set up:

Setting up the project

Download and install soapUI (you’ll need Java too).
Start up soapUI.
Create a new soapUI project – File -> ‘New soapUI Project’
Name your project and click ‘OK’.
Add a new TestSuite – right-click your new project -> ‘New TestSuite’
Name your TestSuite and click ‘OK’
Create a new TestCase – right-click the new TestSuite -> ‘New TestCase’
Name your TestCase and click ‘OK’
Expand the Test Case
Right-click ‘Test Steps’, select ‘Add Step’ -> AMF Request.
Your project is set up and ready-to-roll. Save it.

Configuring the AMF Request

Now that we have a request set up we need to specify the arguments for the call. The is where I had troubles with jMeter – setting up the data required proxies and additional tools. There was no where to easily create & edit AMF object to pass along in the AMF request.

Enter soapUI. Lets say we have an API method called getMonkeys(). getMonkeys() requires an array of ids that specifies what monkeys we want in the list. The name of this parameter is ‘monkeyIds’.

In soapUI, right-click the AMF Request object and select ‘Open Editor’. You should see a window similar to the following:
In the text field for ‘Endpoint’ enter your service end point. For example: http://www.thekuroko.com/Monkey/amfphp/gateway.php
Enter the name of your call in the text field for the AMF Call setting: For example Monkey.getMonkeys
Just under the entry for Endpoint add property to the property list for the call by clicking the ‘Add Property’ button.
Enter ‘monkeyIds’ as the name of the property. If the value for this property were a simple value we could enter it into the value column. We need an array though.
To set the value for the property we’ll use the script window just under the property list.
In the script window, enter the following to create an Array that contains the id values 1,2 & 3 and assigns that Array to the monkeyIds parameter.
parameters["monkeyIds"] = [1,2,3];
That is it. The call for getMonkeys() is set up.
To test the call click the little green arrow in the top left of the AMF Request editor.
If your paths and data are set up correctly, you should see an XML formatted response in the window to the right of the Script editor.

Creating and Running a Load Test

So now we have a test for a service, but we wanted to get some load testing done. If you’re looking for quick and simple load testing, you don’t have to go much further than soapUI itself. To create a load test in soapUI:

Right-click the ‘Load Tests’ node under ‘Test Steps’ -> ‘New Load Test’
Name the new load test and click ‘OK’
2 steps. That’s it, the load test is set up. You can run the test “as-is”.

Now, this is a very simple load test and there are a ton of things you can add to the test to improve it to build more useful load tests within soapUI.

Running Load Test with loadUI

The other tool I mentioned, loadUI, is built to integrate with soapUI and make load testing “easier and more efficient”.

Once loadUI is installed can you execute the test case that you set up in soapUI in loadUI.

Right-click the test case, then select ‘Run with loadUI’.
You will be prompted to save the project, do so.
Select ‘Fixed Rate for the ‘Default Generator’ selection – this will determine how “clients” are generated for the load test.
Select ‘Statistics’ for the ‘Default Statistics’ selection – this will display a graph for the load test metrics.
Click ‘OK’.
loadUI will launch.
Click the ‘Play/Stop’ button to start the load test.

You can play around with the Generator settings to change the rate at which clients are created to see changes in the load test results while the load test is running.

To view a report of the results you can click the ‘Summary Report’ button in the top right of the loadUI interface.

This is just a simple load test and there are plenty of additional settings, assertions and analysis tools that can be added, adjusted and tweaked to improve the validity of the load tests.

Next Steps

Our next step is to integrate the tests into our Continuous Integration (CI) system. We use Jenkins and I saw this post about Automating loadUI and Jenkins (formerly Hudson). So, in theory it can be done. I’ll let you know what we get worked out on that end when we get there.

So far, I’m pretty excited about the two tools. They are very useful, and free to boot. Hey SmartBear – you really are smart, thank you – you rule.

Resources:

Multi-screen is Sexy, even at Adobe Max

John Crosby — Wed, 28 Sep 2011 17:21:21 +0000

There are too many devices to consider when developing video players now-a-days.

Don’t you wish you didn’t have to think about each and every device that Bob & Sally were using to play video?

We’ve got it covered with our Adobe Max 2011 BYOD Lab

“Video Player Development for Multiple Devices” .
(boring name right? I know, but still…)

WTF is BYOD? Stands for Bring Your Own Device – it just means you’ll need to bring your own devices to follow along. Here is what we’re looking at for setup:

Install Flash Builder 4.5.1
Install Flash Media Server 4.5
Oh, and grab the lab files

We’ll go over concepts and code with you to take the tediousness out of developing video players.

Android? Yep.

iOS? Sure-nuff

Blackberry? Um….yeah, this is Flash baby – we got it.

We’ll show you how to AUTO-MAGICALLY detect the platform and screen size and let the player do the work for you.

If you’re attending our session, AWESOME. Thanks!

I’m sure you’d like to get going early right…I thought you might.

Go download the BYOD Lab Files

Hey, even if you’re not going to the session or Max, you’ll get some good sample files right?

While you’re at it, check out Jun Heider’s “Multiscreen Project Best Practices” session too.

UPDATE: F4Fragment Extractor App & Adobe Marketplace Problems

John Crosby — Fri, 22 Jul 2011 23:19:22 +0000

I just received an email stating that Adobe will be closing down Adobe InMarket this includes the AIR Marketplace. So, I’ll start working on a way to distribute the F4Fragment Extractor. As soon as I get the app updates uploaded and in a place to mange it, I’ll let everyone know. Thanks for your patience.

HTTP Dynamic Streaming & Live Events

John Crosby — Thu, 14 Jul 2011 19:52:35 +0000

Flash Media Server (FMS) allows for the streaming of live video over HTTP. The following is a list of concepts that apply to HTTP Dynamic Streaming & live streams:

To stream live video over HTTP, the FMS uses the concept of live events.
A live event is configured as part of a FMS application.
An application configured for live events and can have multiple events.
As with normal FMS applications the name of the directory is the name of the event.
Live events can be configured for multi-bitrate streaming.
The streams are packaged as fragments and written to disk on the fly. Meaning a player can access the content after the live stream has stopped.
Live events support DVR and DRM.

Creating a live event

The steps involved in creating a live event are very similar to creating a normal FMS application with the addition of the Event.xml file. They are:

Create the FMS application {FMS_INSTALL}/applications/{APPLICATION_NAME}
In the application directory create the following directory sturcture: events/_defInst_/{EVENT_NAME}
Create an Event.xml file in the ‘events/_defInst_/{EVENT_NAME}’ directory and at a minimum specify the fragment & segment duration:
```
{EVENT_NAME}

	4000
	10000
```

When you install the Flash Media Server a preconfigured live event is installed as well. The application is called ‘livepkgr’ with a live event called ‘liveevent’. The configuration exists in ‘{FMS_INSTALL}/applications/livepkgr/events/_defInst_/liveevent’

Packaging for Live Events

The Event.xml contains the configuration information for the live event. This includes:

Segment and fragment settings
Content protection (Flash Access) information

A sample Event.xml file with segment/fragment settings as well as content protection configuration:


	liveevent
	
		4000
		10000
		
			FlashAccessV2
			
				foo
				common-key.bin
				http://dill.corp.adobe.com:8090
				production_transport.der
				license_server.der
				production_packager.pfx
				hbXX5omIhzI=
				policy01.pol

The parts to pay attention to in the file Event.xml file above are:

SegmentDuration: The length of the segments in milliseconds. Each F4F file contains one segment.
FragmentDuration: The length of the fragments in milliseconds. Each segment can contain multiple fragments.
ContentProtection: Specifies if the content is protected as well as the details necessary for Flash Access content protection.

DVR & Multi-bitrate Live Events

To create a DVR or multi-bitrate (MBR) live event you will also need to create a Manifest.xml file in the live event folder (events/_defInst_/{EVENT_NAME}/Manifest.xml). This file will contain the DVR and MBR settings for the live event. Sample Manifest.xml file:

The node:

The node contains 2 attributes and controls the DVR functionality for the live event:

beginOffset: The value (in seconds) is where the client players will begin viewing the stream. The default is 0 & negative values are treated as 0.
endOffset: This value in seconds specifies how many seconds behind the current duration of the stream clients can view. The default is 0 & negative values are treated as 0.

Do not include this node if you do not want to use DVR functionality.

The node:

Thenodes are used to specify the multi-bitrate (MBR) settings for the live event. The file is parsed and used by FMS to create the streams for the live event based on the settings specified in the media nodes. The manifest file is then updated with the nodes and data necessary (id, duration, bootstrap and metadata) for a client player to play the stream. The initial file should contain the following attributes for eachnode:

streamId: The name of the publishing stream
bitrate: The bitrate the stream was encoded at

If you do not want to use MBR do not include the media node(s).

Playing Back Live Events

Setting up an FMS application to publish a live event is as simple as creating the application (directory), and the Event.xml file. Of course adding MBR, DVR and content protection will add to the setup and configuration, but it is still a pretty straight forward process. Once the FMS application is configured we’ll need to do a couple of things:

Publish a stream to the application and associate the stream with the live event
Play back the stream in a player. We’ll use the Flash Media Playback to keep things simple.

Publish a Stream and Associate it with a Live Event

To publish a stream we’ll use the Flash Media Live Encoder. We’ll also need to associate the stream to the live event. This will be accomplished with a main.asc file and a bit of server side code. First we’ll cover the server side code, then we’ll set up the Flash Media Live Encoder.

Associating a Stream with a Live Event

I used the main.asc found in the ‘livepkgr’ application installed with Flash Media Server as a base (I’ve change a couple of things). You can download the main.asc I use here. The main.asc file just needs to go in the application directory ({FMS_INSTALL}/applications/{APPLICATION_NAME}). A little explanation of what is going on in the main.asc: The stream is associated to the live event in the onPublish() method (ln. 60 in main.asc):

s.liveEvent = liveEventName;

The variable ‘liveEventName’ is set to the stream name (default):

if( nv.localeCompare( "event" ) == 0 )
{
  liveEventName = nval;
  break;
}

or to the value passed in as the URL variable ‘event’ (ln. 34 – 44 in main.asc).

var nvpairs = new LoadVars();
nvpairs.decode(queryString);
for( var nv in nvpairs )
{
  var nval = nvpairs[nv];
  if( nv.localeCompare( "event" ) == 0 )
  {
    liveEventName = nval;
    break;
  }
}

Now to publish the stream.

Publishing the Stream Using Flash Media Live Encoder

To publish the stream we’ll need to connect to the FMS application and then pass the name of the event that we want to stream to. This is accomplished by adding URL variables to the stream name. For example: Make sure to replace {APPLICATION_NAME} with the actual name of your live event application and {EVENT_NAME} with the actual name of your event. You will also need to make sure you are using H.264 and set up any MBR streams: Click the start button, and the stream should start publishing to your FMS server.

Playing Back the Live Event

To play back the stream, we’ll use the Flash Media Playback. So go to http://www.osmf.org/configurator/fmpto setup the player.

Set the ‘Video Source’ to: http://{FMS_SERVER}:{FMS_HTTP_PORT}/live/events/{APPLICATION_NAME}/events/_defInst_/{EVENT_NAME}.f4m
Select Yes for ‘Are you using HTTP Sstreaming or Flash Access 2.0?’
Remove the value for ‘Poster frame file location’
Select Yes for ‘Autoplay Content’
Click the ‘Preview’ button.

The live event should start playing: There you go – live Events streamed over HTTP that supports multi-bitrate, DVR & content protection using Flash Access. In the next few articles I’ll dive into Flash Access and what is required to get your content protected and secured. Resources:

Flash Media Server Developer’s Guide: http://help.adobe.com/en_US/flashmediaserver/devguide/WSd391de4d9c7bd609-52e437a812a3725dfa0-8000.html

F4Fragment Extractor App & Adobe Marketplace Problems

John Crosby — Mon, 11 Jul 2011 20:27:08 +0000

I want to let anyone interested in the F4Fragment Extractor know that I’m working with Adobe to resolve some issues with my Marketplace account. For now I can’t access the account to make updates and changes to the offering. My plan is to add some requested features like command line arguments and the ability to handle multi-bitrate files auto-magically. But, I can’t until Adobe helps with fixing my access.

I’ll make sure to post an update as soon as I get access to the site.

HTTP Dynamic Streaming Presentation Summary

John Crosby — Thu, 07 Jul 2011 23:46:09 +0000

Okay, after a crazy week buried under bugs and project management tasks I’ve finally carved out some time to post the slides and resources from my HTTP Dynamic Streaming presentation I gave last month.

You can grab the slides on Slide Share, although they won’t be too useful.

The presentation was recorded – which will be more useful than the slides alone. You can view the Connect session here: http://bit.ly/isfJmo.

Resources:

F4V File Format Spec: http://www.adobe.com/devnet/f4v.html
More detailed information on the F4M File Format: http://osmf.org/dev/osmf/specpdfs/FlashMediaManifestFileFormatSpecification.pdf
Flash Media Playback: http://www.osmf.org/configurator/fmp/#
Strobe Media Playback: http://osmf.org/strobe_mediaplayback.html

Next on the list of stuff I’d like to get up on the site is a write-up about creating Live events. So look for that in the next few days now that I have some room to breath.

HTTP Dynamic Streaming: Simple HDS Playback

John Crosby — Mon, 27 Jun 2011 21:03:43 +0000

In my last few posts i’ve covered:

What is HDS? – Describing what HTTP Dynamic Streaming is and where it is useful
Getting Started with HDS – The pieces and parts that you need to be familiar with to start using HDS video content

Now let’s get to the fun part. Let’s play back our HTTP Dynamic Streaming videos. First we’ll use the Flash Media Playback to play our HDS content, then we’ll build a simple OSMF player that plays back the same HDS content. Both of the examples that I’ll walk though use the Open Source Media Framework. OSMF provides the code necessary for the heavy lifting parts of HDS playback. This includes:

Loading the F4M file
Determining if miulti-bitrate (MBR) is available
Determining the best MBR content for playback
Extracting and inspecting the bootstrap data contained in the F4M file
Using the bootstrap data to determine what segment and fragment to call based on the current time of the video playback
Assembling the video data for playback as the fragments are loaded
Playing back the video stream

There are a couple of things that we’ll need to make sure we have in place to get the players working:

Hopefully you have some HDS video files to play with. If you don’t, you can download some that I’ve packaged here:

download: HTTP Dynamic Streaming Sample Files ()
added: 27/06/2011
clicks: 256
description: A sample video packaged for HTTP Dynamic Streaming

If you haven’t already, install and configure the HTTP Origin module on your Apache 2.2 web server, or install the Flash Media Server and its pre-configured webserver.

Deploying the video content

Depending on how you’ve configured the HTTP Origin Module, the directory you will need to deploy your files to will differ from mine, so we’ll use ‘{HTTP_STREAMING_DIR}‘ to represent the path to the directory configured to deliver HDS content using the HTTP Origin Module. On my windows server my path is:
C:\Program Files\Adoeb\FMS\wwwroot\vod

Here are the 2 simple steps to deploying your files:

Upload: Copy your packaged HDS files to the {HTTP_STREAMING_DIR}
Verify: Check to make sure that the files are accessible by requesting the F4M file in a browser

* Note: you’ll also need to have a crossdomain.xml file if you are loading across domains. You can put this in the {HTTP_STREAMING_DIR} as well.

Playing the content back using Strobe Media Playback

Now that we have our content online, we can see if we can load it. I use the Flash Media Playback to test this. Flash Media Playback is a hosted OSMF player built by Adobe. If you want something that you host, check out the Strobe Media Player – another OSMF player provided by Adobe that you can download and deploy to your own server.

To use the the Flash Media Playback go to http://www.osmf.org/configurator/fmp/

You should see a page that looks similar to the following:

We’ll need to update a couple of things to get our HDS content playing:

Change the video source URL to point to the F4M file that you deployed your Apache server. Mine is ‘http://office.realeyes.com:8134/vod/hds-sample/sample.f4m’
Change the selected value for ‘Are you using HTTP Streaming or Flash Access 2.0?’ to ‘Yes’
You can remove the ‘Poster frame image’ and set your content to autoplay, but this is not required.

The settings should resemble the following:

Click the preview button and the video should begin playing. I’ve embedded the Flash Media Playback here with the above settings (except autoplay is false):

Now that we know the our HDS content is working correctly, let’s build a simple OSMF player that plays back the same content.

Create a new ActionScript 3 project in Flash Builder and name it ‘SimplePlayer’.
In the constructor of the class that is created for you ‘SimplePlayer.as’ create a local variable named mediaFactory and typed as a MediaFactory. Set mediaFactory equal to a new DefaultMediaFactory()
```
var mediaFactory:MediaFactory = new DefaultMediaFactory();
```
Create a local URLResource variable named resource and set it equal to a new URLResource(). Pass the URLResource constructor the URL to your F4M file. Mine is ‘http://office.realeyes.com:8134/vod/hds-sample/sample.f4m’
```
var resource:URLResource = new URLResource( "http://office.realeyes.com:8134/vod/hds-sample/sample.f4m" );
```
Create an MediaElement local variable named element and set it equal to the result of calling the createMediaElement() method on the mediaFactory object. Make sure to pass ‘resoure’ as the only argument to the createMediaElement() method.
```
var element:MediaElement = mediaFactory.createMediaElement( resource );
```
Create a MediaPlayer local variable named mediaPlayer and set it equal to a new MediaPlayer, passing the constructor the element variable as the only argument.
```
var mediaPlayer:MediaPlayer = new MediaPlayer( element );
```
Create a local variable named mediaContainer typed as a MediaContainer. Set it equal to a new MediaContainer.
```
var mediaContainer:MediaContainer = new MediaContainer();
```
Call the addMediaElement() method on the mediaContainer object, passing the element object in as the only argument.
```
mediaContainer.addMediaElement( element );
```
Add the mediaContainer to the display list by calling the addChild() method making sure to pass in the mediaContainer object as the only argument.
```
addChild( mediaContainer );
```

The completed class should resemble the following:

package
{
  import flash.display.Sprite;import org.osmf.containers.MediaContainer;
  import org.osmf.elements.VideoElement;
  import org.osmf.media.DefaultMediaFactory;
  import org.osmf.media.MediaElement;
  import org.osmf.media.MediaFactory;
  import org.osmf.media.MediaPlayer;
  import org.osmf.media.URLResource;
  import org.osmf.utils.URL;

  public class SimplePlayer extends Sprite
  {
    public function SimplePlayer()
    {
      var mediaFactory:MediaFactory = new DefaultMediaFactory();

      var resource:URLResource = new URLResource( "http://office.realeyes.com:8134/vod/hds-sample/sample.f4m" );

      var element:MediaElement = mediaFactory.createMediaElement( resource );

      var mediaPlayer:MediaPlayer = new MediaPlayer( element );

      var mediaContainer:MediaContainer = new MediaContainer();

      mediaContainer.addMediaElement( element );
      addChild( mediaContainer );
    }
  }
}

When you run this the video should play. There aren’t any controls, but the important thing to note here is that there wasn’t anything special we needed to do to playback our HDS content. We relied on OSMF’s DefaultMediaFactory to determine the type of media and play it back. You can download the project source here:

download: SimplePlayer.zip (19.14KB)
added: 27/06/2011
clicks: 190
description: Flash Builder 4.5 project archive of simple OSMF player.

Getting Started with HTTP Dynamic Streaming

John Crosby — Mon, 20 Jun 2011 19:30:39 +0000

As described in the first post in this series: What is Dynamic Streaming?, HTTP Dynamic Streaming (HDS) allows for the delivery of streaming content over the HTTP protocol. Streaming video over HTTP requires some knowledge and preparation. The following is a rundown and description for each of the components involved with HTTP streaming.

The files: The F4F and F4m file formats

The F4F file format specification (http://www.adobe.com/devnet/f4v.html) describes how to divide media content into segments and fragments. These segments and fragments are what make ‘streaming’ content over HTTP possible. Basically, the file is broken up into multiple pieces and parts when it is prepared for for HTTP streaming.

A sample segment and its fragments

These way the segments and each segments fragments are created is based on multiple factors such as the length of the video content, the the number of keyframes and the length specified for each segment. The result of packaging your video content for HDS delivery is a set of files that look similar to the following for a simple, single bit-rate file:

The set of files created by the f4fpackager for a short video clip.

From the image above:

sample.f4m: This is an XML file created by the f4fpackager that contains the information necessary for a client player to playback the video file. There will be only one of these for each source video file.
sampleSeg1.f4f: This segment file contains the fragments that the client player will request for playback. There can be multiple of these files for each source video file.
sampleSeg1.f4x: This is an index file that contains specific information about the fragments inside the segment files. There will be one of these types of files for each segment file. The HTTP Origin Module uses the data in this file to determine the fragment to send back to the client after a request.

Each F4F file is a single segment and the segment can contain multiple fragments. So, if you inspect the HTTP requests as a media item is being played back you will see files being requested that map to a fragment within a segment. For example:

Segments and fragments being requested for HDS video

This request specifies to the HTTP Origin Module what segment’s fragment to pull out of the F4F file and deliver to the client for playback. These requests are based on where the time code of where the media item is during playback. So if you scrub to later in the video, the Requests might look something like the following:

Segments and fragment requests while scrubbing HDS video content

This allows the client to request any piece of the video content and start playing back almost immediately. You can control how the files are segmented and fragmented when preparing content with the f4fpackager. The basic concept is to balance the size of the fragment being delivered and the number of HTTP requests that a client needs to make for playback. Please note, the fragments may not be sequential, so you cannot rely on this when requesting content. The the fragments are based on the settings passed into the packager and can skip fragment numbers. So , you can expect to see fragment sequences like the following (no scrubbing involved):

sampleSeg1-Frag1
sampleSeg1-Frag2
sampleSeg1-Frag3
sampleSeg1-Frag5
sampleSeg1-Frag6

This doesn’t mean that the file is incomplete, it is just how the fragments were created by the packager.

The Details

The F4M FIle Format

The F4M or Flash Media Manifest file format contains the information about the package of files created when video content is packaged for HDS. The information included in the manifest file can include some or all of the following:

Media location
Media type
Media bootstrap
Multi-bitrate (MBR) availability
Codecs
Resolutions
Digital Rights Management (DRM) authentication
DVR information
moov atom, metadata block and XMP metadata block

When playing back HDS video content, the F4M file is expected to be loaded as the ‘media file’. The client is responsible for inspecting the data included in the F4M file to authenticate (if DRM authentication is required), determine if MBR content is available and select the best MBR version of the content and then request the media from the server.

Sample of a simple F4M file with a single media item:



	Sample
	67

For a more detailed explanation of the F4M file format specification you can refer to OSMF’s FlashMediaManifestFileFormatSpecification.pdf

When packaging content for HDS, an F4M file is created for you or you can pass the packager the file path of an existing F4M file for multi-bitrate video content. The packager will add the necessary information to the F4M file for client side playback and is the file that should be requested by the client for a specific piece of HDS content.

Sample of a not-so-simple multi-bitrate F4M file:



	
		sample_500kbps
	
	
		recorded
	
	
		114.61450000000001
	
	
		AAABq2Fic3QAAAAAAAAAFAAAAAPoAAAAAAABv7UAAAAA...
	
	bitrate="500" bootstrapInfoId="bootstrap1291">
		
			AgAKb25NZXRhRGF0YQgAAAAAAAhkdXJhdGl...
		
	
	
		
			AgAKb25NZXRhRGF0YQgAAAAAAAhkdXJhdG...

Preparing content: The f4fpackager

As mentioned previously, video content must be ‘prepared’ for HDS delivery. Adobe has provided a tool to do this – the f4fpackager. The f4fpackager is installed with the Flash Media server and can be found at {FMS_INSTALL_ROOT}/tools/f4fpackager or you can download the packager and necessary libraries from the Adobe website: http://www.adobe.com/go/httpdynamicstreaming_bits

The packager is a command line (Windows/Linux) tool that allow you to package video content into the F4F format. The f4fpackager also creates the F4M file for your video content. Packaging video is a straightforward process that can grow in complexity with the addition of live media, multi-bitrate content, DRM and custom packaging settings. I won’t try to cover all of the details now as the Adobe web site has some great info about using the packager. Here are a few examples though:

Single bitrate file

Preparing a single bitrate file is the simplest type you will have to deal with when packaging HDS content. For example if we have a single file, sample.flv, the following command would package the content using the default settings and output the resulting packaged content to a sub-folder named ‘SampleOutput’:

f4fackager.exe --input-file=sample.flv --output-path=./SampleOutput

Multi-bitrate file

Packaging multi-bitrate (MBR) files requires a little more preparation. To package MBR files you will need to pass the F4M file each time you package one of the video files. This lets the f4fpackager utility add the necessary data to the F4M file for each bitrate. For example, if we have the following set of files sample_100kbps.flv, sample_200kbps.flv, sample_300kbps.flv you would package these files with the following 3 commands:

f4fackager.exe --input-file=sample_100kbps.flv --bitrate=100 --output-path=./SampleOutput
f4fackager.exe --manifest-file=sample_100kbps.f4m --bitrate=200 --input-file=sample_200kbps.flv --output-path=./SampleOutput
f4fackager.exe --manifest-file=sample_100kbps.f4m --bitrate=300 --input-file=sample_300kbps.flv --output-path=./SampleOutput

Notice we have added the ‘–bitrate’ argument to all of the commands and the ‘–manifest-file’ arguments to the second and third commands. The –manifest-file aregument passes in the f4m file created when we packaged the first 100 kbps file. This allows the f4fpackager to append the data for each successive packaged file to the data created for the previous file(s). This means you will pass the sample_100kbps.f4m file to the client player to play the MBR content. The player will then use the data contained in this F4M file for each of the 3 bitrates to determine the optimal bitrate to request.

HDS Delivery: Delivering Your HDS Video Content

What is the HTTP Origin Module and what does it do?

Delivering HDS content requires an Apache server with the HTTP Origin Module (kinda of, but we’ll get to that in a minute). The Apache HTTP Origin Module makes delivering HDS content very straight forward. Just install and configure it on your Apache web server, upload your files and your ready to go. basically the client player and the Origin Module work together to find and playback the fragment for a specific time for playback. The flow would going something like:

The player loads and parses the data provided in the F4M file.
Based on that data the player figures out what segment/fragment pair to request from the server.
The HTTP Origin Module is passed the request from the Apache web server.
The origin module then pulls segment fragment pair from the files on the server and delivers it to the player.
The player can then handle the response by assembling the video data for playback.

*NOTE: OSMF provides the code to handle playback of fragments as well as determining which segment/fragment pair to request based on the current time of playback. This code exists in the package org.osmf.net.httpstreaming.

Do I have to use the HTTP Origin Module?

What if you want to deliver your content from an IIS web server or from Amazon S3? You don’t have to use the Origin module, but additional work will need to be done to remove the origin module from the playback loop. There is a solution – the F4F Fragment Extrator. This is a tool that I built to extract the fragments from the segments and creates files that you can deploy anywhere. In effect it does the work of the HTTP Origin Module up front so when a client requests the fragment, sampleSeg1-Frag9 for example, that file exists and can be delivered to the client for playback.

Getting Started with HDS Summary

Now you should have some knowledge about the main components for preparing and deliverying HDS content. In the next article, I’ll go over playback of HDS content using the Open Source Media Framework (OSMF). Until the next article, please don’t hesitate to ask questions or let me know if have questions about what HDS is, how to get started with it or if there is something else you’d like to see.

Reference:

Adobe HTTP Dynamic Streaming Help: http://help.adobe.com/en_US/HTTPStreaming/1.0/Using/WS9463dbe8dbe45c4c-c126f3b1260533756d-7ffc.html
F4M Specification: http://osmf.org/dev/osmf/specpdfs/FlashMediaManifestFileFormatSpecification.pdf
Packaging On-demand Video Content: http://help.adobe.com/en_US/HTTPStreaming/1.0/Using/WS9463dbe8dbe45c4c-c126f3b1260533756d-7ffc.html
Installing and configuring the Apache HTTP Origin Module: http://help.adobe.com/en_US/HTTPStreaming/1.0/Using/WS7b362c044b7dd076-735e76121260080a90e-8000.html

What is HTTP Dynamic Streaming?

John Crosby — Fri, 03 Jun 2011 15:10:26 +0000

Until now video content delivery over HTTP has been delivered in a progressive manner, meaning to view part of or seek to a specific location in the video you have to wait for that part to download. RTMP allows for the ability to seek to any point in the video content via streaming, but requires a server technology such as the Flash Media Server to do this. HTTP Dynamic Streaming or HDS combines HTTP (progressive download) and RTMP (streaming download) allow for the ability to deliver video content in a steaming manner over HTTP. This means:

A streaming server technology is not required
Clients can access and begin playing content ‘instantly’
Clients can seek to points in the video content that have not yet downloaded.

…and all the above over the HTTP protocol.

There are a of couple things that you’ll need to consider when getting started with HTTP Dynamic Streaming.

First, a media server to stream the content is not required, but the Apache Web server with the HTTP Origin Module installed is. The HTTP Origin Module is a free to use Apache module provided by Adobe, and comes pre-installed and configured with the Flash Media Server when you install the bundled web server. We’ll cover the installation, set up and data flow for the HTTP Origin Module in a subsequent post.

Second, the video content will need to be ‘prepared’ for HTTP Dynamic Streaming before being deployed to your server. This means the workflow for content creation will need to be adjusted to accommodate the packaging of your video content into the F4F file format. There is a tool, the f4fpackager, that Adobe has created to do this for you and we will discuss the details of this tools and how you can use it later as well.

At Realeyes, we’ve done some some work with AIR and the F4F file format spec to create the The F4Fragment Extractor. The F4FragmentExtractor is a utility for extracting the F4V fragments from the F4F files created by Adobe’s file packager for HTTP streaming. This means that you can deploy the fragments produced by this tool to any Web server or cloud services like Amazon S3 and reap the benefits of HDS even without the HTTP Origin Module.

In the next few articles, we’ll look at:

Getting Started with HDS
Use cases for HDS
Integrating HDS into your content creation workflow

Between now and when I get to the next post let me know if you have any questions or would like to see something specific about HTTP Dynamic Streaming in the comments.

The Cooking Channel’s Eat Street in our Parking Lot

John Crosby — Mon, 30 May 2011 17:27:14 +0000

A long time friend of mine, Ian Kleinman is going to be featured on the Cooking Channel’s Eat Street TV show. If you want some awesomely-crazy-wacked-out snacks, to show your support for Ian, or just want to get on TV, come hang with the Inventing Room and the Realeyes crew. Here the menu that Ian has creaeted for the cart tomorrow:

Apple pie “Koolaid”
Hot fudge ice cream, espresso fudge cookie
Homemade whipped marshmallow
Strawberry yuzu sorbet with frozen olive oil
Popcorn space foam
Miracle fruit tasting
Creme brulee with banana foster
Coconut ice cream with brown sugar
Braised pineapple and raspberry bubbles

Ian & The Eat Street crew will be in the parking lot (behind our offices at 940 Logan) filming Ian and his inventive cart Tuesday March 31st from 8:30 am until 3:30 pm.

Mobile Flex: View Data

John Crosby — Mon, 16 May 2011 16:06:35 +0000

From the previous post you should know how to navigate from 1 view to the next using the ViewNavigator. Now, you want some data in that view right? No problem, this is where the View object’s ‘data‘ property comes into play. Setting the data property is accomplished by passing the data object, in addition to the View’s class name, into the pushView() method on the navigator object.

Example:

navigator.pushView(MyNewView, dataObject);

This effectively calls the setting for the data property of the new View (MyNewView) object that is created.

Managing View Data

You could work with the data property on the View object directly. For instance, if the data object passed into the View via the pushView() method was a simple user object that contained a name property, you could bind the name property to a label control.

Example:

Overriding the Data Property Setter

Usually though, you’d want to override the setter for the data property. Then you can type your object and work with it in a better manner.

Example:

protected var user:User;
override public function set data(value:Object):void
{
 super.data = value;
 user = value as User;
}

So now we’ve got the data in the view. The next step is to manage the state of each view. With mobile apps you can’t count on the view staying around, so we’ll need to keep a tight control on the state of each view. That way we can bring the user right back where they expect to be when they come back to the app after a call for example. In the next post we’ll look into how to do this. Stay tuned.

This article has also been posted on the Realeyes website.

Mobile Flex: ViewNavigator Basics

John Crosby — Tue, 15 Mar 2011 20:19:54 +0000

Flex 4.5 provides some pretty slick updates and enhancements, the least of which is the of Mobile components and the ability to slam out some pretty nice mobile apps easily. The first thing I’d like to talk about is a new concept, the ViewNavigator. The ViewNavigator provides some pretty intense functionality such as view management.

What is the ViewNavigator?

The ViewNavigator keeps track of your views. It does this by keeping your views in a list. To add a new view you ‘push’ the view into the list, to remove a view you can ‘pop’ a view out of the list. You can think of it as a stack – first in, last out – and the last view in is the visible view.

Pushing a View into ViewNavigator's 'stack'

Popping a view out of the ViewNavigator's 'stack'

Using the ViewNavigator

Using the view navigator is a pretty straight forward process of capturing a user interaction, such as a button click, then pushing the new View into the ViewNavigator’s stack.

For example, let’s pretend that you have a new Flex mobile project. The default view of that project contains a button, that when clicked should display another view named MyNewView. MyNewView also contains a button, that when clicked returns you to the the home view.

Home View Component

In the Home View component all you really need to worry about the the click handler on the button:

The click handler calls the pushView() method on ‘navigator‘, a property available from the View class, passing it the class name of the View that you want to display. We’ll cover getting data into that view and transitions in other posts. The creation of the new View & default transition are all handled by the framework.

MyNewView Component

The MyNewView View component is basically the same thing:

You call popView() on the ‘navigator‘ property which removes the view from the stack displaying the Home view again.

Here is a quick screen cast of an application using similar code:

This article has also been posted on the Realeyes website.

OSMF 1 Day Course: What do you want?

John Crosby — Tue, 07 Dec 2010 19:27:41 +0000

We are working on a 1 day Open Source Media Framework (OSMF) course at Realeyes and wanted to know what should be included in the training. Here is the initial course description:

This intense one day hands-on course is a serious dive into the Open Source Media Framework (OSMF). Starting with an introduction of the OSMF the course quickly picks up pace and covers the basics of simple implementations that can be done in less 4 lines of code or less for powerful media player applications to the creation, understanding, and use of the more robust features for dynamic media management, from progressive playback to HTTP Dynamic Streaming. The course also will cover creating and using HTML embeddable players, implementing brandable media player user interfaces, advanced stream and connection management, as well as a solid exploration of the advanced plugin architecture available with OSMF.

And here is the initial outline:

Unit 1: Getting Started
- Introducing OSMF
- Using the Strobe Media Player
- The basics of OSMF
  - BuildingThe Simplest Player
  - Breaking down and understanding the parts of OSMF
    - MediaElement
    - MediaPlayer
    - MediaContainer
    - MediaFactory
  - Handling different types of media (Images/SWF/Video/Audio)
  - Understanding Traits
  - Utilizing MetaData
  - Making an Embeddable Media Player
    - Using FlashVars to control the player
    - Using the HTMLMediaContainer
Unit 2: Enhanced Media Experiences
- Creating and implementing User Interface control with OSMF
- Media Compositions & base level custom advertising/monetization
  - Serial
    - Poster art, playlists, pre/post/in-play injection
  - Parallel
    - Overlay/PiP, outside synchronized advertising
      - Basics of Layout controls
Unit 3: Extensibility
- Advanced low level control techniques
  - Advanced NetConnection Management & Control
  - Advanced NetStream Management & Control
  - Advanced Buffer Management
- Plugins
  - Understanding the different types:
    - Static and Dynamic
      - Reference and Proxy
  - CDN Plugins
    - Implementing a CDN plugin
      - Akamai, LimeLight
  - Analytics Plugins
    - Building a simple plugin with JS integration/reporting
    - Implementing the GTrack Plugin
  - Advertising Plugins
    - VAST/MAST – pre-roll and overlay

Do you see anything missing?

Something you’d like to have in the 1 day course?

Would you be interested in sitting in on a test run of the course in the future?

Let me know.

OSMF Custom Media Elements

John Crosby — Tue, 23 Nov 2010 21:34:01 +0000

A good argument for using a framework is the ability to extend the built in capabilities of the framework. For example, there was a comment on the ‘Getting Sstarted with OSMF Plugins‘ post that asked about using embedded images in theWatermarkPlugin sample.

Here are the steps that I took to get an embedded asset (instead of a ‘loadable’ asset) to show as a watermark:

1. Create a new class that extends MediaElement (this is a simple element, but you could extend any existing element depending on your needs). I named mine StaticImageElement.

package com.realeyes.osmf.plugin.element
{
  public class StaticImageElement extends MediaElement
  {
    ...
  }
}

2. Add a private Bitmap property with a getter and setter to the class – I named mine _bitmap.

private var _bitmap:Bitmap;

...

public function get bitmap():Bitmap
{
  return _bitmap;
}

public function set bitmap( value:Bitmap ):void
{
  if( value != _bitmap )
  {
    bitmap = value;
  }
}

3. In the setter for the bitmap property add the DisplayObjectTrait to the StaticImageElement

addTrait( MediaTraitType.DISPLAY_OBJECT, new DisplayObjectTrait( _bitmap as DisplayObject, bitmap.width, bitmap.height ) );

4. The completed class is pretty simple because we get to use everything already created for OSMF.

package com.realeyes.osmf.plugin.element
{
  import flash.display.Bitmap;
  import flash.display.BitmapData;
  import flash.display.DisplayObject;

  import org.osmf.media.MediaElement;
  import org.osmf.traits.DisplayObjectTrait;
  import org.osmf.traits.MediaTraitType;

  public class StaticImageElement extends MediaElement
  {
    private var _bitmap:Bitmap;

    public function StaticImageElement()
    {
      super();
    }

    public function get bitmap():Bitmap
    {
      return _bitmap;
    }

    public function set bitmap( value:Bitmap ):void
    {
      if( value != _bitmap )
      {
        _bitmap = value;

        addTrait( MediaTraitType.DISPLAY_OBJECT, new DisplayObjectTrait( _bitmap as DisplayObject, bitmap.width, bitmap.height ) );
      }
    }
  }
}

5. Create the embedded asset in the WatermarkPluginElement class

[Embed( "/assets/osmf_logo.png" )]
protected static const OSMF_LOGO:Class;

6. Now all we need to do in the WatermarkProxyElement set the bitmap property on a new instance of the StaticImageElement instead of creating an ImageElement with the watermark URL and ImageLoader.
Before:

var watermark:ImageElement = new ImageElement( new URLResource( watermarkURL ), new ImageLoader() );

After:

var watermark:StaticImageElement = new StaticImageElement();
watermark.bitmap = new OSMF_LOGO();

Bonus points for developing with a framework – more specifically OSMF! The embedded watermark shows up.

Download the original sample code:

download: OSMF Plugin Samples (722.43KB)
added: 25/08/2010
clicks: 933
description: Samples from the RMAUG presentation on OSMF plugins.

UPDATE: I’ve created an additional custom MediaElement called InteractiveImageElement.as. Thanks for the idea @cucu_adrian! The new element handles rollover and rollout by adjusting the image’s alpha property and setting the cursor to a button cursor. It also navigates to a url specified in the class – this would be any easy thing to make configurable though.

download: InteractiveImageElement.zip (774B)
added: 24/11/2010
clicks: 382
description: Custom OSMF MediaElement that adjusts the cursor & alpha on rollover and goes to a URL when clicked. See the following articles for details and usage: http://www.thekuroko.com/2010/08/articles/presentations/getting-started-osmf-plugins/ http://www.thekuroko.com/2010/11/articles/code/osmf-custom-media-elements/

John Crosby

Aptana SDOCML & Code Hinting Support

Related Posts:

Take the PIA out of load testing AMF services

Setting up the project

Configuring the AMF Request

Creating and Running a Load Test

Running Load Test with loadUI

Next Steps

Resources:

Related Posts:

Multi-screen is Sexy, even at Adobe Max

Go download the BYOD Lab Files

Related Posts:

UPDATE: F4Fragment Extractor App & Adobe Marketplace Problems

Related Posts:

HTTP Dynamic Streaming & Live Events

Creating a live event

Packaging for Live Events

DVR & Multi-bitrate Live Events

The node:

The node:

Playing Back Live Events

Publish a Stream and Associate it with a Live Event

Associating a Stream with a Live Event

Publishing the Stream Using Flash Media Live Encoder

Playing Back the Live Event

Related Posts:

F4Fragment Extractor App & Adobe Marketplace Problems

Related Posts:

HTTP Dynamic Streaming Presentation Summary

Related Posts:

HTTP Dynamic Streaming: Simple HDS Playback

Deploying the video content

Playing the content back using Strobe Media Playback

Related Posts:

Getting Started with HTTP Dynamic Streaming

The files: The F4F and F4m file formats

The Details

The F4M FIle Format

Preparing content: The f4fpackager

Single bitrate file

Multi-bitrate file

HDS Delivery: Delivering Your HDS Video Content

What is the HTTP Origin Module and what does it do?

Do I have to use the HTTP Origin Module?

Getting Started with HDS Summary

Reference:

Related Posts:

What is HTTP Dynamic Streaming?

Related Posts:

The Cooking Channel’s Eat Street in our Parking Lot

Related Posts:

Mobile Flex: View Data

Managing View Data

Overriding the Data Property Setter

Related Posts:

Mobile Flex: ViewNavigator Basics

What is the ViewNavigator?

Using the ViewNavigator

Home View Component

MyNewView Component

Related Posts:

OSMF 1 Day Course: What do you want?

Related Posts:

OSMF Custom Media Elements

Related Posts: