Last week I too k the time to map the different options of video coding chip solutions. There is one thing I left out, although it can be deduced from that post, and that’s the trends that exist in the video coding chip market.

As a quick recap, there are two main groups of solutions for video coding chips - those that are “pure” software, and those that rely on hardware. If we were to compare these different solutions by their price and flexibility, the resulting scatter graph would look similar to the one below:

Video Coding Chip Comparison: Price vs. Flexibility

Put simply:

• Hardware-based solutions are cheaper in their cost for the same amount of performance
• Software-based solutions are more flexible (with faster time to market)

I’d say that today, for a company to come out with a new video telephony client product, there are several viable options:

1. Software Based Solution

The only time a software solution should be selected is when the chip already exists in your planning for other purposes. In any other case - you should not choose a software only solution. Why? Because the price you will pay will increase the price of your product to a point where exiting solutions will be way cheaper - everyone else has already taken the leap towards a hardware solution.

What do I mean by having the chip there already? If you plan on doing a videophone for a laptop, then the Intel x86 chip that exist there will be suitable (a bit stressed to its limits though). This is what we do ourselves in our SCOPIA Desktop product.

2. Hardware Based Solution

This is the place where most companies should be focusing today: find a solid chip that can do hardware encoding - be it using acceleration, an FPGA or an ASIC, and then build the rest of your hardware design around it.

Your selection of the chip should take into consideration the following parameters:

• Do you need to interoperate with others? If you do, then make sure to find a flexible solution. An ASIC in this case may be risky.
• Is there a good video codec available for that chip? Has this video codec been used for video conferencing already or only for other purposes?
• What additional functionality will you be needing that can’t be addressed by this chip? How will they affect your design and cost structure?
• What interfaces does the chip offer? Do these map well to your requirements?

3. Hybrid Solution

In the case of the hybrid solution, software is used for decoding and hardware is used for encoding or vice versa. This allows reducing the cost of a pure software solution a bit by offloading part of the work it needs done to hardware.

While this solution exists and has been used in the past by video conferencing vendors, taking that route is risky and complex. My recommendation would be to stick with either a software solution or a hardware solution for your video coding chips selection - you’ll have more than enough on your hands to deal with in the hardware design so adding hybrids here will only complicate things further.

What Are Major Video Conferencing Vendors Doing These Days?

If you analyze the way that the major players in the video conferencing market selected the chips for their terminals, you will notice an interesting trend: most of them started with software-based solutions and later on migrated to hardware-based ones when it came to HD. The move was done at the point in time when HD started to show up - the cost of processing it with software-only solutions was higher than what the market was willing to pay, forcing companies to move to hardware solutions instead.

Video Telephony Migrates From Software to Hardware Based Chip Solutions

Today, most of the video conferencing companies base their solutions on the acceleration technology provided in the TI DM6467 chip (DaVinci HD) - and for good reason.

In my next post on this series, I’ll provide the case study of using the TI DM6467 for video coding.

Related posts

• How to Select the Best Chip for Your Video Coding? (0)
• Why Is Designing HD Video Endpoint Hardware So Hard? (0)
• What Makes our BEEHD Video Codec Different? (6)
• Multi-Core, Optimization and Atwood’s Law (0)
• How to Develop Your Own Hardware For an HD Video Endpoint? (0)

This week, a distinguished group of operators and vendors publicized an initiative called “One Voice” (here’s the PDF spec). These companies are AT&T Inc., Telefonica, TeliaSonera, Verizon, Vodafone, Orange, Alcatel-Lucent, Ericsson, Nokia Siemens Networks, Nokia, Samsung and Sony Ericsson. And their plan is to come up with a specific technical profile for voice and SMS services over LTE based on IMS.

Simply put, there is now serious work being done to take IMS into the next level and make sure that once LTE networks are deployed (=all IP cellular networks), their voice and SMS services will be provided robustly by the mobile operators.

What does this mean to all the downloadable VoIP clients out there? A death blow. Instead of utilizing LTE’s higher bandwidths and lower latencies for their own service, they will see the operators provide robust alternatives that don’t suffer from the same drawbacks that they do (being a downloadable application).

What does it mean to our industry? That IMS is taking center stage on LTE networks.

There are others who have written their opinion on One Voice over the past several days. Here are some of the best writings on this topic:

• Martin Sauter provides his insights on One Voice from a political and a technical viewpoints.
• Caroline Gabriel covers the dynamics that brought about the One Voice initiative.
• Stacey Higginbotham points at the missing players in One Voice.
• UPDATE: My other post on the subject at NoJitter.

Related posts

• Mobile VoIP Movement Might Become Irrelevant Due to IMS (3)
• Mobile VoIP is Still Far Ahead (3)
• Want Mobile VoIP? Wait Another Decade (5)
• 5 Reasons Why Mobile VoIP = #FAIL (4)
• WiFi VoIP Instead of 3G Calling? Give Me Seamless WiFi First (0)

Why am I telling you that about myself? Because some people think WiFi VoIP is going to replace 3G calling.

Michael Graves, for instance, thinks DECT is better than WiFi, when it comes to VoIP:

In everything experiment that I’ve ever tried DECT/CATiq products soundly beat any dedicated SIP Wifi handset.

While Michael Graces probably takes the comparison to the realm of latency and packet loss (e.g. call quality), I’d like to take it to another place, which poses a lot more of an issue, especially when people think WiFi VoIP is mobile VoIP. Well, it’s not.

I have my share of business travel, and I need to come up with ways to connect to the office to read my precious emails. It usually requires finding a WiFi hotspot and praying that it has the iPass roaming service (or at least that nice web page, popular in hotel chains, where I can pay $20-25 for a single day use). It’s not always easy, and a lot of the time requires working my way over stupid web forms.

Sign-in procedure for a coffee shop in Israel (via HolesInTheNet)

For mobile e-mail, for me - as I do not own a smartphone - there is no real alternative. But if I’d have to go through that arduous effort of trying to connect to WiFi every time I had to make a mobile phone call, I’d simply give up on the whole thing.

So please don’t pitch WiFi VoIP to me, just because you are looking for a few more reasons why Mobile VoIP isn’t here yet.

3G networks might be expensive, but they sure work - and seamlessly. Whenever I am abroad, my mobile phone connects automatically and simply does what it has to - it’s magic, especially compared to WiFi.

Wake me up when WiFi gets to that level of robustness and seamlessness. Then, and only then, will it (maybe) be a candidate for mobile VoIP.

Related posts

• 5 Reasons Why Mobile VoIP = #FAIL (4)
• Video conferencing on an iPhone? In what protocol? (7)
• Consumers HD Voice – Where is the Incentive? (3)
• Mobile VoIP is Still Far Ahead (3)
• Ask an Expert: For Mobile Video Telephony in the US, Should I Use SIP or 3G-324M? (5)

One of the most burning issues involved with designing a video telephony device is where the video codec is going to reside. The reason is that this is by far the most CPU intensive task of the whole system, especially when going up to high definition. I’d like to discuss the various options that exist today on the market, with their relative advantages and challenges.

There is a wide variety of solutions that can accommodate a video codec successfully. At their very basic level, they can be split into software-based and hardware-based solutions.

Solution options for HD video-processing chips

Software-based Solutions

In software-based solutions the video codec is treated as another piece of code that gets compiled and executed on the chip. These solutions can be further divided into single core chips, multi core chips and many core chips.

Single core chips are the simplest to program, but these are hard pressed to meet the demands of HD video processing. The Intel Atom chip, for instance, falls in this category.

Multi-core chips are chips that have a relatively small number of cores )up to 8 today), which are viewed by the operating system as a single unit. The cores execute various tasks of the same program (or multiple programs). Usually, the operating system selects which core will run which of the tasks at hand at any given time. You can find here GPP (General Purpose Processor) chips, such as the Intel Core i7, or DSPs such as the Freescale MSC8144E.

Multi-core chips are considered today hard to program, due to the need to deal with multi-threaded applications. That said, splitting a video codec into multiple threads is a relatively easy task. I’d say that from a programmer’s point of view, using a GPP, or even a DSP would be the easiest solution.

Many core systems will usually run a larger number of cores, up to a hundred today and going to the tens or hundreds of cores in the near future, each running independently of the rest with its own operating system. The Tilera TILE family of processors is a good example.

Many core systems are harder to deal with, as the synchronization mechanisms that exist between cores are a lot harder to program and manage. But, what a many core system lacks in programming it returns ten fold in power usage, with a much lower watt requirement for the same processing task.

Software based video codecs provide a high level of flexibility - since the codec runs as “pure” software, changes to it are relatively easy to make. It comes at a cost - the price for the same processing power will be relatively high.

Hardware-based Solutions

In Hardware-based solutions, the codec is not a single piece of software that is executed on the chip - parts of the codec (or all of it) will run as part of the dedicated hardware circuitry itself. As with software solutions, there is a broad spectrum of options here as well.

Acceleration is a kind of a hybrid solution, where the codec’s processing is divided between software that manages the higher level of logic of the codec and a hardware acceleration unit, which will tend to take on the menial tasks of video processing, such as motion estimation and DCT calculations. The TI DM6467 belongs to this family.

Next in the spectrum are the FPGA-based solutions. In the case of FPGAs (Field-Programmable Gate Array), the solution will come from third parties. This is a good comparison of HD encoders on FPGAs that can be a good starting point for companies who wish to go down this path.

An FPGA chip will be harder to program and will be by far more complex than other solutions. This is why the relevant video codecs come from third parties, who are experienced in such development.

The last family in this category is ASIC solutions, where the chip itself has the video codec hard-wired into its chip. Notable solutions here are Cavium Networks PureVu^TM CNW22XX or Maxim MG3500.

ASIC solutions can be treated as black boxes - they do the job intended - nothing more and nothing less. This comes with a great risk of running into interoperability issues that might require changes in the codec or simply the need to improve the codec, which are quite impossible.

How to Choose?

There are a lot of parameters that you need to consider when choosing your processor:

• Price - easy. If you are price sensitive in your product, then you will need to select a solution that isn’t expensive from both a BOM point of view and a development point of view
• Flexibility - does the chip offer you a way to change the codec? To control it properly? How much time does it take to make such changes? Will you be dependent on a third party for such changes?
• Connectivity - does the chip integrate other functions that you will need? Audio coding, application, GUI, video ports, etc.
• Experience - is your team experienced with such a platform? Will you be able to recruit the team necessary to design and develop the end product easily?
• Design wins - have you seen any design wins of the processor for the target market you are going to? Do you need such guarantees before you start your project?

Decide what is important to you - there might be other aspects not listed here that make sense to you. Check the various alternatives that exist on the market and select the one that fits best.

Related posts

• Trends in Video Coding Chips (0)
• Why Is Designing HD Video Endpoint Hardware So Hard? (0)
• Multi-Core, Optimization and Atwood’s Law (0)
• Video Coding For Dummies (1)
• What Makes our BEEHD Video Codec Different? (6)

I’ve written here before about mobile video telephony and health care, but this is a bit different. This time the idea is to mesh up one service with another. And there’s no better way to do that than using VoIP.

The Internet has brought us the ability to link services together in ways impossible to do before (or at least too hard to be worth doing). And while Houston Neal gives 7 examples of such mesh-ups, I am sure that there are more to be found - in health care and elsewhere.

It is also why I think that innovation in video conferencing (a VoIP thingy) won’t come from new features but from new applications - from meshing-up video conferencing with other services for specific needs - from the current mundane web conference with added video conferencing to playing an online game in the living room between multiple families around the world.

Related posts

• Why NOW is THE Time for Visual Communications? (0)
• My 50+ non-Top VoIP Blogs (3)
• What Does Skype Have to do with iPhone and User Interfaces? (1)
• Welcome, My Son, to the Virtual Machine (0)
• The Carnival is Back in Town (Carnival of the Mobilists #158) (14)

I came to this conclusion reading Om Malik’s post about the wireless bandwidth boom, which has (of course) been driven by the iPhone:

But the popularity of new 3G devices such as the iPhone and BlackBerry 3G has increased the use of data, putting the backend networks under strain. And from that perspective, today’s 3G networks are like glittering skyscrapers built on a foundation of matchsticks.

So what does a blogger do when such a realization hits him? He starts looking at other areas, where the iPhone has been a game changer. Here are a few that I’ve been able to collect.

i-Prefix

The “i” prefix is used by Apple for all of its products: the iPod, iTunes and iPhone are making their way to every possible corner. Ben Robinson from IntoMobile even reported on the iBore phenomena:

Warning: if you have an iPhone, you might be an iBore!

What happened next is that every other service today gets an “i” prefix attached to it to increase its coolness factor.

While not a game changer on its own, this phenomenon shows the strength of the Apple brand. I wonder when will they patent the letter “”" and start sewing others for using it.

Touch Technology

Ever since the first iPhone came out, people want to touch the screen. For some arcane reason, it has now become the most coveted input technology. And it’s not just touch - it’s MULTI-touch.

Windows 7 touch support

Be it mobile handset screens, eBooks, netbooks, laptops or even large digital whiteboards - touch as an input technology got into user input interface heaven thanks to the iPhone.

Application Stores

The Apple AppStore is the biggest cloud computing store for apps out there. There were a few before Apple, but none as successful, and none in a way that created a huge influx in the amount of them out there - and not only for mobile. Need a few examples?

• Ondeego - an app store for enterprise-focused mobile apps
• An app store for US agencies?
• A conference dedicated for mobile app stores
• An app store for twitter apps
• A VoIP App Store from BroadSoft

Application stores also raise a central question - will we consume our future services through the browser or by downloading applications?

Wireless Browsing

Wireless browsing, and even data usage over wireless networks, was marginal at best until the iPhone came out. There was no easy way to surf the internet from a mobile phone up until then, so the major use of the data network was for the Blackberry’s email service.

Since then, as Om Malik wrote in the quote above, new 3G devices such as the iPhone and BlackBerry 3G have increased the use of data, putting the backend networks under strain.

Another interesting quote on that matter is from Dimitris Mavrakis who points out in telecoms.com:

The runaway success of USB modems and the iPhone 3G have helped to make mobile broadband one of the most important strategic and commercial opportunities in the mobile industry.

While cellular providers now have to start beefing up their networks to play nicer with data, other vendors are following suite and coming out with handsets that are easier to work with on the internet.

Visualization on Consumer Electronics

When the iPhone came out, consumer electronics was… boring - Bland interfaces, usually done by engineers who select bad colors, 2D menus and windows.

From the moment the iPhone came out, it’s as if the world stood still: every consumer electronics vendor stopped everything they did feature-wise and went off to look for a 3D interface to match the iPhone. Even chip vendors came out with demos showing Open GL 3D graphics on their chips - “like the iPhone”.

It did a lot of good for the industry - it taught us that devices shouldn’t be boring.

Reflective Surfaces

The iPhone has been such a success, that the very basics of its design are being copied time and time again by others, and I am not talking about the iPhone-killer handsets which are spreading around.

And I don’t even need to go far - just look at what our own home-designer decided to do with the RSS and Email links for this blog:

Does the reflection of the words remind you of a certain product? We’re not alone in this - a lot of designers out there have been working on mimicking various design decisions made by Apple on the iPhone - from reflective surfaces to the way.

There’s More?

There are probably a few more areas that I just missed here. Feel free to add them.

Related posts

• The Carnival is Back in Town (Carnival of the Mobilists #158) (14)
• iPhone vs Android, Windows Mobile vs RIM - are these the correct battlefields? (2)
• Welcome, My Son, to the Virtual Machine (0)
• SIP PBX Implementation, Part I: To Relay or Not to Relay – That is the Question (0)
• Google Wave or Opera Unite? (0)

-

What if Microsoft was the company behind the service we all call Twitter?

Every tweet would be a maximum of 140 characters, but would need to download a 2Mb message, containing various XML markups and a ton of security.

You know - you would be able to change the font of your tweet. And the size of it. And the color. Set it to bold. Or italicize it. That kind of tweaking takes space.

Add to that encryption for privacy concerns, authentication, server signatures, etc.

You would be readying yourself for Twitter 2010

And just when you got used to the way twitter handles them RT messages.

Oh, and it will require Silverlight, .NET, ActiveX, and Internet Explorer 9 to read, of course.

They would have come up with a new SDK technology once a year

Remember the fact that you’re getting ready for Twitter 2010? So if you are a developer, that change in retweets might change the application you’ve written for twitter.

You would have to sign an EULA for each tweet you send out

You know, saying that the tweet you send is up to you, no one else is responsible for it, even if it kills your machine, or the machine of the people receiving it.

And one for each tweet you want to read, of course.

Oh - what’s that you say? Twitter just changed their terms of service?

You would have to reset your twitter client EVERY day

Some of us actually HAVE to do that, since their twitter clients gobble up memory relentlessly without giving it back to the community: I used to close my tweetdeck and reopen it about twice a day a few versions back.

Image from Andy Brudtkuhl’s blog

You would have to get used to “the blue tweet of death”

Also known as a Fail Whale in some cultures.

The new BSOD? Left: lauterhaus Right: majortk

You couldn’t use Twitter on your Mac

Or you could. Only it would be an older version of Twitter with the possibility of stopping support at any time.

You could only use Windows Media Player to view videos tweeted

While that might not be true, there has been some complaints about twitter not playing nice with open standards.

Nagging administration again…

1. I’d like to thank Sagee for supplying some of the “what if” answers above.
2. I originally wanted to write a post titled “What If Cisco Acquired Tandberg”, but most of the blogs out there beat me to it.
3. You’re more than invited to subscribe to this blog or to our blog network if VoIP interests you.
4. UPDATE: Sagee also wrote his own version - What If Twitter Developed Video Conferencing?

Oh! And if I missed some, feel free to comment below and add your own!

Related posts

• Why VoIP Security is More Important Than we Thought (0)
• VoIP Security is More Than Just Hollow Threats (2)
• Why Is Designing HD Video Endpoint Hardware So Hard? (0)
• Where Would You Draw The Line On VoIP Privacy? (0)
• Webinar: IMS Security - How to Protect Signaling Communications against Attack (0)

This is the first real post in this series, so I’d like to start off with the reason for writing this series.

We’ve set out here to create a product to assist customers with developing a high definition video client - our BEEHD. The main “theme” behind this product is to reduce the hassles involved with dealing with standards and video coding technologies to a point where our customers have only to focus on the user interface of the application.

What we “missed” is that there is an additional area where our customers and prospects face challenges - designing a hardware suitable for video conferencing. It is not that we missed that part, it’s just that we assumed our customers will be able to cope with it on their own. But there is good reason why designing such hardware isn’t that easy: the processing requirements for HD video are humungous.

Video Processing

Just to give some kind of a glimpse to what is necessary, here’s a chart that shows the amount of pixels in an image per resolution. CIF and 4CIF is where standard definition is today. HD is either 720p or 1080p in the chart.

Pixels per resolution

To make things worse, going HD also means increasing frame rates from 15 per second to at least 30 per second, which will again double the amount of data you will need to process in a given amount of time.

You might think that this only affects the video encoding and decoding parts, where a lot of processing power is required to get the job done, but there are other areas that need to be taken care of:

• High end motion camera that can capture HD resolution
• HD requires more bandwidth on video inputs and outputs (usually by means of faster clock speeds and larger bus mechanisms)
• HD requires more processing when scaling the image to fit the display

All of these issues makes a video endpoint something that can’t fit today in a single chip, which means the videophone application itself needs to be distributed between several chipsets in the solution (a software issue) AND that the data path of the video inputs and outputs are longer as they need to traverse through more chips (a hardware issue).

Audio Processing

A HD videophone system also needs to support HD Voice - the ability to work with something better than the measly sound spectrum of up to 3.3 KHz we’re used to from our phones today. and that is also not a trivial task.

Why? Because again, as video traverses several components/modules within the videophone, so does audio. And while the issues here are not related directly to processing power, they do relate to a lot of sensitivity issues that arise when dealing with audio.

Next

What I wanted to do was touch 7 of the challenges related to designing the hardware for HD video terminals. It is by no means an exhaustive list, but it does reflect the challenges that our prospects face the most.

These challenges are:

• Where to place the video coding processes?
• How does video scaling and window layouts affect the design?
• Searching for a good HD camera
• Taking care of the on screen display
• A closer look at the audio system
• Dealing with the host chip and managing connectivity
• Future-proofing the hardware

Now, as this is a work in progress, I will be very happy to hear your comments and understand where you would like assistance - this might alter the challenges I’ll discuss here on the blog or even generate new ones.

And if you are interested in the subject, feel free to subscribe to this blog either on RSS or by email, as in the next couple of months there will be a post on this specific subject once a week.

Related posts

• How to Select the Best Chip for Your Video Coding? (0)
• Trends in Video Coding Chips (0)
• What Makes our BEEHD Video Codec Different? (6)
• How to Develop Your Own Hardware For an HD Video Endpoint? (0)
• Video Coding For Dummies (1)

That’s how you do it!

Applications and not operating systems are the risk

It seems like operating systems are being targeted less and applications more:

“The analysts found that the biggest risk facing most systems is unpatched vulnerabilities in applications and that applications, not operating systems, have become the primary target of attack”.

As VoIP is always considered an application on top of the operating system, and not really a part of the operating system, hackers are more likely to go after VoIP on the application level than from a pure network level.

I would say that the main area of attack will be the VoIP protocol itself, trying to inject malformed/malicious messages for one purpose or another; or trying to hack into the application to gain access to user related information.

Vulnerabilities in applications running on web servers

Server-side hacking happens more than client-side hacking:

“Dealing with vulnerabilities in applications running on Web servers. The survey found that Web server-side applications are the target of more than 60% of all Internet attacks…”

My take? If you are building a server, you should be more careful with your code than with a client. It’s quite obvious - there’s more to gain by hacking or crashing a large server than a single user.

This also indicates that those pursuing “cloud communication” should deal with VoIP security threats from day one and not leave it for the last minute.

Don’t neglect security

I’d say that my take away here is: when it comes to VoIP, don’t neglect security. Bear in mind that it’s not just about user privacy, but the whole application’s security we’re dealing with here: Secure RTP, TLS or H.235 are irrelevant in this case; It’s the way you code and test your application that matters.

Related posts

• VoIP Security is More Than Just Hollow Threats (2)
• What If Microsoft Developed Twitter (24)
• Welcome, My Son, to the Virtual Machine (0)
• Where Would You Draw The Line On VoIP Privacy? (0)
• Modu’s Hardware vs Software: Take 2 (2)

Only problem is - time. No time to sit down and write it. To my rescue came Sean D’Souza from Psychotactics Zingers with a post on how to write a book an article at a time. So my target is writing an eBook one blog post at a time.

What will I be doing?

In the following weeks, and probably several months, I’ll be posting on the various aspects of hardware design for video telephony endpoint - from the reasoning behind developing such a product, to design considerations and finally to a checklist of things you need to do.

It’s going to be a series of posts on “Hardware for HD”. I will be interested to hear your views on the subject or the things that you are challenged with, so drop by and leave me a message.

How will you know there’s a post related to this series? Just follow the logo on the top of this page.

Related posts

• Why Is Designing HD Video Endpoint Hardware So Hard? (0)
• Trends in Video Coding Chips (0)
• How to Select the Best Chip for Your Video Coding? (0)
• What Makes our BEEHD Video Codec Different? (6)
• Why Hardware Trumps Software for Visual Communications (0)