MOBILE 3DTV blog

MOBILE3DTV project finalized

2011-06-08T11:39:00.010+03:00

Mission accomplished! The MOBILE3DTV project ended and has provided core technology for new mobile 3DTV systems. Summarizing our work in pictures, here are the posters that we presented at the presentation of the final end-to-end MOBILE3DTV system at ICT2010 in Brussels.

MOBILE3DTV @ ICT EVENT 2010

2011-05-23T13:12:00.019+03:00

The MOBILE3DTV was selected as exhibitor at the ICT event 2010 in Brussels. The project’s booth was located in the “Content & Knowledge Zone”. The concept of the booth that was developed within the project consortium was a presentation of a first end-to-end system of 3D content delivery optimization over DVB-H. For each stage, an expert sent by the task-leading partner within the consortium was present at the booth. The core of the booth was the demo of the first MOBILE3DTV end-to-end system including a DVB-H transmission of 3D content to a first prototype developed within the project. Within the demo, the visitors at the booth could follow the whole chain of the system from the provider including available contents of different genres and content creation, optimized stereo video encoding methods and error resilient transmission, to the receiver side including visual optimization, 3DTV handheld devices. Finally, all visitors were able to become real users in the user experience stage in which they were able to experience mobile 3DTV on the first available prototypes.

Beyond the overall demo of the system, specific information and small demos for each stage of the production chain were presented on additional mobile 3D video devices. The specific demos demonstrated different coding approaches like MVC or the Video+Depth approach or simulated different transmission artifacts or error resillience strategies so that visitors at the booth were able to see the challenges for which the MOBILE3DTV project provided technological and methodological solutions.

Mobile3DTV at the Mobile World Congress, Barcelona, 15-18 February 2010.

2010-02-16T10:14:00.003+02:00

The Mobile3DTV project partner MMS will be present at the MWC as part of the Texas Instruments' ecosystem of partners and developers. MMS will be demonstrating stereo camera and 3M auto-stereoscopic display integrated in to TI's popular Zoom OMAP36x-III mobile development platform to enable 3D video capture, encoding and display of stereo content. Visit TI's booth in Hall 8 - 8A84.
More information about can be found here.

MOBILE3DTV at eChallenges e-2009

2009-11-12T11:21:00.013+02:00

METU representing Mobile3DTV Project has participated in eChallenges Conference e-2009, 21 - 23 October 2009, Istanbul, Turkey; the 19th of annually held international conference series, supported by European Commission and technically co-sponsered by IEEE. Detailed information about the conference can be found here.

WP3 Leader Gozde Akar from METU gave a presentation in the Workshop titled “Turkey in European Research Programmes – Success Stories” . During the 3 days of the conference, we had a demonstration stand of our DVB-H end-to-end broadcast system in the eChallenge Exhibition area. The stand included the transmission, reception, processing and display of stereo coded video over DVB-H. Also a 17” autostereoscopic display and IPhone with 3D Sheet are used to let the visitors experience 3D in different applications.

During the conference we’ve received feedback from participants of various backgrounds including commercial, government and research organizations. Participants had a chance to experience stereo video personally with the help of an IPhone and autostereoscopic sheet.

At the end of the conference our project has been awarded with “Runner-Up Demonstration Stand Award” which is presented by Mehmet Gokgoz, Director of Turkish Research and Business Organizations (TURBO).

Update: Mobile3DTV @ NEM Summit

2009-09-24T17:27:00.009+03:00

Mobile3DTV participated successfully in this year's NEM Summit in Saint Malo, France. After we had presented our first DVB-H stereo video streaming system last year, we informed about our user studies this year.

We demonstrated our current work on optimizing the mobile 3DTV system through different user studies along the production chain. The visitors at our booth were able to experience real test scenarios. They watched the original test items of our coding study or were able to get information about different transmission artifacts.

Additionally,we informed about our view on user-centered quality evaluation and introduced our Open Profiling of Quality method.

A short coverage of us can be found in ICT Results' report. Thanks to all who visited our booth. Thanks for your interest and your fruitful feedback and discussions. CU next year at NEM Summit in Nantes.

Quality of Experience (QoE) - Current standardization activities

2009-09-07T17:01:00.010+03:00

The assessment of the Quality of Experience (QoE) has become more important in audiovisual quality assessment and MOBILE3DTV has started to evaluate user experience for mobile 3D television and video. Within standardization bodies the topic of Quality of Experience has been included into standardization activities. The goal of ETSI HF STF 354 (Specialist Task Force 354 within the Human Factors Group of ETSI) is to provide requirement guidelines for real-time multimedia services aiming at providing a good QoE. Goal is to provide objective and subjective measures of user experience for given communication situations, service prescriptions and levels of QoS [1]. The guidelines as the expression of measure are available in ETSI EG 202 534 [2] and ETSI TR 102 535 [3].

Within the International Telecommunication Union ITU standardization activities towards QoE are targeted in ITU-T SG12, the lead study group on quality of service and quality of experience within the ITU. Goal of ITU-T SG12 to be able to a) measure quality parameters in next generation networks and b) measure their impact on QoE. The challenge of providing good QoE for new multimedia systems is study item of question Q13/12 “QoE, QoS and performance requirements and assessment methods for multimedia including IPTV”. Main study items are the relation of end-user requirements to system parameters and to identify simple and efficient analysis techniques to measure and monitor QoE. All results will be included into the development of new recommendations.

References:

Brooks, P., Hestnes, B., Heiestad, S., Aaby, C. (2006). “Communicating Quality of Experience data for the development of multimedia services”. Proceedings of the 20th International Symposium on Human Factors in Telecommunication, Sophia Antipolis, France, March 21-23 2006, (available from http://www.hft.org/HFT06/HFT_06_programme.htm).
ETSI EG 202 534: "Human Factors (HF): Guidelines for real-time person-to-person communication services". ETSI Guide, 2007
ETSI TR 102 535: "Human Factors (HF): Guidelines for real-time person-to-person communication services; future requirements". ETSI Technical Report, 2007

Mobile3DTV project invited to give two workshop talks

2009-08-13T10:59:00.008+03:00

Researchers from the Project Mobile3DTV are invited to give some talks at 3D-related workshops:

Dominik Strohmeier from Ilmenau University of Technology will give a talk at the 6th Salzburger Medientag 2009 on 18. September 2009. His talk "Mobile3DTV - Meet the Users" will cover the aspects of user needs and requirements of Mobile3DTV and their relevance for the user-centered development process of Mobile3DTV. All presentations at the Medientag will be given in German.

Update: Satu Jumisko-Pyykkö from Tampere University of Technology will give a talk at 3D Media Workshop in Berlin organized by Heinrich-Hertz-Institut, Berlin, in joint collaboration with national and international research projects. The talk "User-centered quality of experience of Mobile3DTV" will target the new approaches of our research to form a holistic framework for QoE assessment. The workshop takes place from 15.-16. October 2009.

Feel free to meet us at one of the workshops.

Nokia releases the source code of its MVC implementation

2009-05-28T21:50:00.005+03:00

Nokia announced source code of their MVC codec based on their previous H.264/AVC baseline codec. Since it is baseline profile, only IPPP encoding structure can be used.

------------------------------------------------------------------------------------

Nokia announced today open availability of its MVC source code implementing H.264/MPEG-4 MVC, a leading multi-view video coding standard created jointly by the Telecommunication Standardization Sector of ITU and the Moving Picture Experts Group, MPEG of ISO/IEC. The source code is fully optimized for different uses in mobile devices and includes a player for the Maemo environment, making it easy for developers to take full advantage of the source code in an open, expandable ecosystem.

MVC is the newest addition to highly successful and widely deployed H.264/AVC family of video codecs serving multi-view and 3D video usage. It provides superior network robustness and compression performance for delivering 3D video by e.g. taking advantage of the inter-view dependencies of the different visual channels. In addition, its backwards compatibility with H.264/AVC codecs makes it widely interoperable in environments having both 2D and 3D capable devices.

Nokia acknowledges finalization of the MVC specification and is proud to contribute to the MVC community with this software release. "We are all very excited about this technology and the way it adds a whole new dimension to visual communication and entertainment. We hope this donation helps the developer community to take advantage of the full potential of the MVC technology and stimulates the creation of innovative applications and services", says Jyri Huopaniemi, Director of Nokia Research Center Tampere laboratory.

The software is available for download at: http://research.nokia.com/research/mobile3D. The implementation has also been showcased at Nokia World 2008 using a modified Nokia N800 device with an integrated auto-stereoscopic display for natural real-time 3D video playback.

Mobile3DTV featured on on ICT Results - "A third dimension for mobile phones"

2009-05-19T20:55:00.006+03:00

We are happy to announce that Mobile3DTV was featured on ICT Results. We would like to thank the ICT Results team for helping us to promote our project on the web. Here is an excerpt from the article; link to the full text is at the end.

Simulating the third dimension is something of a Holy Grail for cinema and television. The key advantage of 3-D film over the conventional two dimensions is the illusion of depth and the sense of ‘body’ the viewer experiences – as if the action is leaping out of the screen rather than occurring within it.
“The mobile market has always been much more dynamic and receptive to new technologies than the television market, as the whole idea of mobility is based on dynamism,” explains Atanas Gotchev, the scientific coordinator of the EU-funded Mobile3DTV project.
The story of 3-D television for mobile phones has been one punctuated by stops and starts. As early as 2003, Sharp launched a 3-D mobile phone in Japan and Korea’s SK Telecom launched a 3-D phone – from Samsung – in 2007, and Japan’s Hitachi just launched one in 2009. But the big challenges have been the paucity of content and coming up with a profitable business model. Apple’s iPhone also supports three-dimensional television, but can currently only be viewed with special glasses.
Mobile3DTV is developing the core elements of the next generation of three-dimensional television for mobile devices. “One major challenge is choosing the optimal format for representing 3-D video for mobile delivery,” Gotchev points out. The format should be adopted ideally by all industrial players to avoid a ‘formats war’, he suggests. For that reason, the project decided to build its system around the EU standard known as Digital Video Broadcasting – Handheld (DVB-H).
“Another challenge is to ensure a comfortable and enjoyable 3-D viewing experience," adds Gotchev. Mobile3DTV is employing so-called auto-stereoscopic displays, which produce 3-D images that do not require those awkward glasses to view them – which is good news for people who want to be incognito about their mobile viewing.
“Auto-stereoscopic displays use additional optical elements aligned on the surface of an LCD, to ensure that the observer sees different images with each eye,” explains Gotchev. “As mobile devices are normally watched by a single observer, two independent views are sufficient for satisfactory 3-D perception.”
“We have access to probably the most advanced 3-D portable display – one delivered by the Japanese giant NEC LCD,” says Gotchev.
Mobile3DTV is funded under the ICT strand of the EU’s Framework Programme for research.

Full text is available here:
http://cordis.europa.eu/ictresults/index.cfm?section=news&tpl=article&BrowsingType=Features&ID=90580

New section in our website makes it to span accross borders

2009-05-14T12:53:00.003+03:00

We have added a new section to mobile3dtv.eu called "User studies". It discusses the subjective tests on stereo-video quality performed part of Workpackage 4. In order to evaluate the user experience while watching stereoscopic video, new assessment methods have been developed jointly in IMT in Germany and IHTE in Finland.
In a similar way, our website is now shared between different countries. While most of it is hosted in Tampere University of Technology in Finland, the new section resides in TU-Ilmenau in Germany. Since all sections share common navigation bar and have the same outlook, our visitors can seamlessly navigate through our website while roaming between different European servers.

Mobile 3DTV on Youtube

2009-05-06T16:58:00.005+03:00

We have prepared a couple of short movies about our project. The movies are available on our youtube channel, and also at http://mobile3dtv.eu/publicity, our publicity webpage. Currently there are two movies available, with more to come soon. We hope you like them!

Project meeting in Saariselkä

2009-04-28T09:42:00.001+03:00

Third Mobile3DTV project meeting was held in Saariselkä, in the Finnish Lapland, from 4th to 6th of April.

Representatives of all workpackages of the project met to discuss the progress of the project and to agree about a roadmap of workpackage interaction in the system development process. All work was inspired by the beautiful winter landscape of northern Finland.

The most important outcome of the project meeting is a roadmap that coordinates the work between the workpackages for the rest of the project. This roadmap is a result of a productive review meeting in Brussels and the comments given by the reviewers.

Beside the hard work of the meeting, the participants enjoyed the great winter in Lapland. Especially the 1200m long sledge slope was as good alternative to relax in meeting breaks.

The project meeting was a success for the project and the outcome of the meeting in combination with the developed roadmap will direct the project towards a successful second year.

MOBILE3DTV at GSMA Mobile World Congers 2009 in Barcelona

2009-04-16T17:45:00.019+03:00

MOBILE3DTV demonstration was presented at GSMA Mobile World Congers on 16-19 February 2009 in Barcelona (http://www.mobileworldcongress.com/). We presented the current state of the development on the project with very spectacular demonstration. Participants in the demonstration from MMSolutions site were Nikolay Daskalov – CEO, Radoslav Ibrishimov and Gjorgji Rosikopulos.

We shared some space at the OMAP zone with Texas Instruments, which are our long term partners in many projects, and exchanged few ideas and impressions from the congress . We also met few of the partners in the MOBILE3DTV project and discussed some new concepts about the evolution of the project. During the exhibition we had the opportunity to see the “beyond tomorrow” vision of the mobile technology. We also demonstrated our achievements in the mobile image capturing technology, such as 8MP camera framework, stereo-capture module, ect.

The kit for the demonstration was developed by MMSolutions. It was based on standard TI development platform, widely spread among the mobile application developers. The content was delivered by both MMSolutions and Tampere University of Technology. The videos were specially prepared for the congress, and were of a great interest for the visitors, who saw the demonstration.

We previewed the spectators the true 3-D effect and the possibility of implementing this future technology in multimedia mobile device. Every observer shared a positive opinion for the demonstration. They confirmed that they will keep an eye on the progress of the project in Internet with interest. Based on our observations and the conversations with every spectator of the demonstration, we think that all of them will be consumers of the device in future.

In the end we would like to thank all the visitors, interested in the project, for sharing their opinion about the demonstration. We would like also to thank Atanas Gotchev and Atanas Boev for helping us with the preparation of the content and the preparation of the brochures and ad materials. Special thanks to all the other participants in the project for their help in achieving this level of the technology.

NEC provides 3.1'' 3D TFT LCD Module for Mobile3DTV terminal prototype

2009-02-20T10:23:00.003+02:00

NEC LCD Technologies announced that it will provide samples of its 3.1-inch-diagonal 3D polysilicon thin-film-transistor (TFT) liquid crystal display (LCD) module for the MOBILE3DTV terminal prototype. The MOBILE3DTV prototype will be on public display at the Mobile World Congress 2009 (MWC2009), the world’s largest mobile technologies-related event, in Barcelona February 16– 19.

MOBILE3DTV is one of the projects funded by the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007–2013) in the context of its Information & Communication Technologies (ICT) Cooperation theme. The three-year project began in January 2008 and aims to demonstrate the feasibility of mobile 3D TV broadcasting.

The MOBILE3DTV terminal prototype is being developed by a project consortium partner, MM Solutions AD, and shall be displayed at the Texas Instruments booth (Hall 8, Booth 8A 84), which provides the prototype’s CPU.

A sample of the 3.1-inch diagonal LCD module with wide quarter video graphics array (WQVGA) resolution and excellent 3D picture image for bare-eyes viewing will also be displayed in cooperation with NEC Electronics in the NEC booth (Hall 8, Booth 8A 125).

The MOBILE3DTV terminal prototype is being featured at MWC2009 to demonstrate real-time decoding and displaying of 3D video content. The 3D LCD module has been adopted because of its wide viewing areas and high-performance display which features high-resolution 3D images based on NEC LCD Technologies’ unique horizontal double-density pixel (HDDP) system.

Looking forward, NEC LCD Technologies will continue to promote research and development in pursuit of practical uses and mass production of 3D LCDs.

www.mobile3dtv.eu updated

2009-02-13T11:13:00.006+02:00

The website www.mobile3dtv.eu has been updated recently. On the website you can find interesting news about the project research, conference papers, technical reports, and software about mobile 3D television and video.

Standards related with coding of Mobile 3DTV content

2009-02-10T19:13:00.016+02:00

Standardization of digital audio and video is investigated by the Moving Picture Experts Group (MPEG), a working group of ISO/IES and the corresponding standards are issued with ISO/IES designations [1].

MPEG-C, Part 3
The purpose of ISO/IEC 23002-3 Auxiliary Video Data Representations (MPEG-C part 3) is to support all those applications where additional data needs to be efficiently attached to the individual pixels of a regular video. In ISO/IEC 23002-3 it is described how this can be achieved in a generic way by making use of existing (and even future) video codecs available within MPEG. ISO/IEC 23002-3 consists of an array of N-bit values which are associated with the individual pixels of a regular video stream. These data can be compressed like conventional luminance signals using already existing (and even future) MPEG video codecs. The format allows for optional subsampling of the auxiliary data in both the spatial and temporal domain. This can be beneficial depending on the particular application and its requirements and allowing for very low bitrates for the auxiliary data.

The specification is very flexible in the sense that it defines a new 8-bit code word aux_video_type that specifies the type of the associated data, e.g., currently a value of 0x10 signals a depth map, a value of 0x11 signals a parallax map. New values for additional data representations can be easily added to fulfill future demands. The specification is directly applicable to 3D video as it allows specifying such video in the format of single view + associated depth, where the single channel video is augmented by the per-pixel depth attached as auxiliary data. As such, it is susceptible to efficient compression. Rendering of virtual view (at least one in case of stereo) is required at the receiver side. The specification has been standardized since 2007 [2], [3], [4].

MVC
3D video (3DV) and free viewpoint video (FVV) are new types of visual media that expand the user’s experience beyond what is offered by 2D video. 3DV offers a 3D depth impression of the observed scenery, while FVV allows for an interactive selection of viewpoint and direction within a certain operating range. A common element of 3DV and FVV systems is the use of multiple views of the same scene that are transmitted to the user. Multiview Video Coding (MVC, ISO/IEC 14496-10:2008 Amendment 1) is an extension of the Advanced Video Coding (AVC) standard that provides efficient coding of such multiview video. The overall structure of MVC defines the following interfaces: The encoder receives N temporally synchronized video streams and generates one bitstream. The decoder receives the bitstream, decodes and outputs the N video signals. The video representation format is based on N views. For the case of stereo-video, that is two separate views coded together. A promising extension is to study view subsampling, i.e. one full resolution view + one subsampled view. The idea behind this approach is that the human visual system is capable to retrieve the stereo with the quality of the better channel. MVC is standard since 2008 (version 1) [5], [6].

3D Video Coding
3D Video Coding (3DVC) is a standard that targets serving a variety of 3D displays. Such displays here in focus present N views (e.g. N = 9) simultaneously to the user, so-called multi-vied displays. For efficiency reasons only a lower number K of views (K = 1,..,3) shall be transmitted. For those K views additional depth data shall be provided. At the receiver side the N views to be displayed are generated from the K transmitted views with depth by depth image based rendering (DIBR). This application scenario imposes specific constraints such as narrow angle acquisition (<>K out of N views, augmented with K depth sequences. This representation related to stereo-video generalizes the possibilities of MPEG-C, Part 3 and MVC, i.e. the two separate views can be coded together or can be reduced to single view + depth with the second view to be synthesized at the receiver. 3DVC is an ongoing MPEG activity, and a standard is expected in 2011 [7], [8], [9].

References:

Moving Pictures Experts Group, http://www.chiariglione.org/mpeg/
ISO/IEC JTC1/SC29/WG11, “Text of ISO/IEC FDIS 23002-3 Representation of Auxiliary Video and Supplemental Information”, Doc. N8768, Marrakech, Morocco, January 2007.
ISO/IEC JTC1/SC29/WG11, “Text of ISO/IEC 13818-1:2003/FDAM2 Carriage of Auxiliary Data”, Doc. N8799, Marrakech, Morocco, January 2007.
ITU T and ISO/IEC JTC 1, “Advanced video coding for generic audiovisual services”, ITU-T Rec. H.264 and ISO/IEC 14496-10 AVC, 2003, most recent Version: 2005.
ISO/IEC JTC1/SC29/WG11, “Text of ISO/IEC 14496-10:200X/FDAM 1 Multiview Video Coding”, Doc. N9978, Hannover, Germany, July 2008.
ISO/IEC JTC1/SC29/WG11, “Joint Multiview Video Model (JMVM) 8”, Doc. N9762, Archamps, France, May 2008.
ISO/IEC JTC1/SC29/WG11 “Overview of 3D Video Coding”, Doc. N9784, Archamps, France, May 2008.
ISO/IEC JTC1/SC29/WG11 “Description of Exploration Experiments in 3D Video Coding”, Doc. N9783, Archamps, France, May 2008.
ISO/IEC JTC1/SC29/WG11 “Applications and Requirements of FTV”, Doc. N9466, Shenzhen, China, Oct. 2007.

Interesting talk: "Our cell phones, ourselves"

2009-02-10T14:52:00.004+02:00

What does your cell phone mean for you? Here is an interesting talk of Jan Chipchase who works as researcher at Nokia about the way we interact with our mobile phones.

See Jan Chipchase on TED.

Enjoy this interesting talk about Jan's research and his results and learn about you and your mobile phone. And feel free to leave a comment...

Technical Report: Adaptation and optimization of coding algorithms for mobile 3DTV

2009-02-04T19:14:00.001+02:00

Adaptation and optimization of coding algorithms for mobile 3DTV
Philipp Merkle, Heribert Brust, Kristina Dix, Yongzhe Wang, Aljoscha Smolic, Technical report, November 2008

We achieved up to 35% bitrate savings from inter-view prediction compared to stereo simulcast. Inter-view prediction whether as H.264 SEI or MVC does not add substantial complexity. A representation as video plus depth is an interesting alternative for 3D video. It allows to adjust the stereo rendering at the decoder and to optimally adapt the 3D impression for any given display. However, this extended functionality comes at the cost of an increased complexity. MPEG-C Part 3 is suitable for encoding of video plus depth data. Good depth quality is essential for good overall quality.

The publication is available on www.mobile3dtv.eu.

Conference Paper: Mobile 3DTV System: Quality and User Perspective

2009-02-04T19:12:00.002+02:00

Mobile 3DTV System: Quality and User Perspective
A. Gotchev, S. Jumisko-Pyykkö, A. Boev, D. Strohmeier, "Mobile 3DTV System: Quality and User Perspective", European Workshop of Media Delivery (EUMOB’08), Oulu, Finland, July 2008.

While mobile 3DTV system components such as stereo video compression techniques, transmission channels and auto-stereoscopic displays are available and with good level of maturity, their joint work crucially depend on the quality and user acceptance. We address these two key factors by rigorously investigating all possible artifacts appearing at all processing stages and influencing different layers of the human 3D vision, and by applying a novel user-centered design to the critical components of the system.

The publication is available on www.mobile3dtv.eu.

Conference Paper: Designing for User Experience: What to Expect from Mobile 3D TV and Video?

2009-02-04T18:32:00.002+02:00

Designing for User Experience: What to Expect from Mobile 3D TV and Video?
S. Jumisko-Pyykkö, M. Weitzel, D. Strohmeier: "Designing for User Experience: What to Expect from Mobile 3D TV and Video?", uxTV Con 2008, Silicon Valley, USA, October 2008.

A long path has been taken in the side of technology to develop 3D video for consumer products, but user studies to underline the actual users’ needs and expectations are disregarded. The object of this study is to examine users’ needs, expectations and requirements for mobile 3D television and video. We conducted three user studies applying a methodological triangulation of the extensive survey, focus groups and probe studies to gather the requirements. The results are presented in the form of the requirement guideline which highlights the aspects of users, desired system and service including the interesting contents and usage contexts. Both academia and industry can benefit from these requirements when designing the further studies and development work in the user experience of 3D television and video.

The publication is available on www.mobile3dtv.eu.

Meet the users: Sanna Virtanen (28)

2009-02-03T16:06:00.011+02:00

We met Sanna for the first time in a cozy cafe in the center of Tampere. She was watching "Finland's Next Supermodel" on her mobile 3DTV device. Of course, we were courious to hear about her experiences with mobile 3D television.

Sanna is 28 years old and she has worked as Managing Consultant at WeCan OY – a medium sized software company – since two years. Sanna has a boyfriend who lives in Madrid, Spain.

Sanna is a highflyer in her job. She loves the job, although it is very exhausting and she’s always busy at work. But in the same manner in which she loves her job, she lives her life after work. She is a lifestyle woman, always up-to-date and trendy. She bought her mobile 3D phone 5 months ago. Her last phone contract was ending and as she had the possibility to choose a new phone, she decided for mobile 3D. The advertisement and the latest reports in the lifestyle magazines convinced her that this phone fulfills all her demands on functionality and quality.

“I enjoy sitting in the cafe and watch the latest lifestyle magazines or all these cute shows on TV like this ‘Finland’s next supermodel’. 3D makes everything so realistic and I enjoy diving into the content so that I can forget the day for a short time.” Having a cup of Iced Mocha she chilled on the sofa and watched the series when we met her for the first time. She passed some time after work in the cafe before her Pilates workout started.

"When I come home later, I will call Raúl. He is my boyfriend since 4 month. I hate the long-distance relationship, but luckily we now found 3D videophoning", Sanna tells us. Secretly, the 3D videophony was the final reason to decide for mobile3DTV. “You know, it is just this intimacy that I experience when Rául appears in front of me three-dimensionally. For a short time I even forget the annoying distance between us”, Sanna says with a smile on her face.

Do you know Sanna? Until now, Sanna is just a persona, a user archetype. She represents one of the user groups of mobile 3DTV according to our studies into user requirements for the new system [1][2]. You may ask how other users will look like? Read on, we will present them later.

References:

Jumisko-Pyykkö, S., Weitzel, M., Strohmeier, D. "Designing for User Experience: What to Expect from Mobile 3D TV and Video? " . Proceedings of the First International Conference on Designing Interactive User Experiences for TV and Video. October 22 - 24, 2008, Silicon Valley, California, USA.
Strohmeier, D., Weitzel, M., Jumisko-Pyykkö, S. Use scenarios: mobile 3D television and video. Proceedings of SPIE Electronic Imaging 2009, San Jose, CA, USA (2009)

Current user research in the context of Mobile3DTV

2009-02-02T11:59:00.005+02:00

Related user studies in the field of 3DTV and especially in the field of mobile TV, one can realize a big change in the field of user-centered design. Current research has introduced user-centered approaches to study user’s needs and expectations about novel systems and services. This allows better understanding of the impact of critical system components. The cyclic character of user-centered design is standardized in ISO 13407 [1].

Subjective quality has usually been studied quantitatively based on requirements of the International Telecommunication Union (ITU) [2] offering guidelines to study quality perception studies in controlled, laboratory environments. In the context of mobile television, these studies, which base on quantitative measurements of ITU recommendations, have focused on evaluating the impact of coding errors, low bitrates, and impact of changes in image size or framerates as well as bitrate share between audio and visual modalities (e.g. [3], [4], [5]). Further, there are studies into transmission errors over DVB-H channel and their impact on the perceived quality [6], but still these studies are rare.

With respect to stereo-coding, there are studies on stereo-coding errors in images and videos [7]. However, user studies in the field of 3DTV have focused more on the impact of different factors on presence, the users feeling of being there. In stereo-video content presentation, the most prominent additional features that influence the perceived quality are depth information, perceived sharpness, or naturalness (i.e. the most truthful or realistic reproduction of content) [8][9]. All these factors contribute to the concept of presence, the involvement of the user in the content; the feeling of being there [10]. Nevertheless, some work has reported simulator sickness symptoms caused by the use of stereo-video presentation. This particular impact of stereo vision adversely affects enjoyment and factors creating simulator sickness are not yet known in depth [11].

In contrast to the recommendations of the ITU, current studies in the field of mobile TV include user characteristics, usage contexts and the user’s goal of viewing. Currently there are available results of field trials or studies conducted in several countries [12][13] as well as results of prospective focus groups and online surveys [14][15]. In contrast, user studies into the field of 3DTV are still rare and Freeman’s focus group [16] is the only study which examines user requirements of 3DTV more broadly.

From the methodological point of view, there is a lot of work in progress to close the shortcomings of ITU recommendations. Researchers have introduces contextual quality evaluation procedure that is conducted in parallel to the controlled laboratory assessments [17]. Additionally, evaluation of the experienced quality of critical system components targets the future acceptability of the quality of components which were developed in isolation from the end-product [18]. A third step to understand experienced quality is to understand the interpretation of constructed quality and experienced quality factors [19].

Finally, there is one available study into the user experience of mobile 3D television. Following a methodological triangulation of online survey, focus groups and probe study to elicit user requirements, Jumisko-Pyykkö et al. [20] present a broad view on user experience of mobile TV. In all the three studies, they targeted the user experience factors of user, system and services, and contexts. Concluding their study, they present first guidelines to design user experience of mobile 3D television.

References:

ISO 13407. 1999. Human-centered design processes for interactive systems. International Standard, the International Organization for Standardization.
ITU-R BT.500-11, Methodology for the Subjective Assessment of the Quality of Television Pictures, Recommendation ITU-R BT.500-11, ITU Telecom. Standardization Sector of ITU, 2002.
H. Knoche, J. D. McCarthy, and M. A. Sasse, “Can small be beautiful?: assessing image resolution requirements for mobile tv,” in MULTIMEDIA ‟05: Proceedings of the 13th annual ACM international conference on Multimedia. New York, NY, USA: ACM Press, 2005, pp. 829–838.
S. Winkler, S. C. Faller, “Maximizing audiovisual quality at low bitrates,” Workshop on Video Processing and Quality Metrics for Consumer Electronics. Scottsdale, United States of America: January 2005.
S. Jumisko-Pyykkö, “I would like to see the subtitles and the face or at least hear the voice”: Effects of Picture ratio and Audio-video Bitrate Ratio on Perception of Quality in Mobile Television. To appear in Personalized and Mobile Digital TV Applications in Springer Multimedia Tools and Applications Series, 2007
S. Jumisko-Pyykkö, Vadakital, V., Liinasuo, M. , Hannuksela M. M., Acceptance of Audiovisual Quality in Erroneous Television Sequences over a DVB-H Channel. Proceedings of Second International Workshop in Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, USA, January 2006.
L. B. Stelmach, Tam, W. J. (1998). Stereoscopic image coding: effect of disparate image-quality in left- and right-eye views. Signal Processing: Image Communications, 14:111–117.
H. de Ridder, (1996). Naturalness and image quality: Saturation and lightness variations in color images of natural scenes. Journal of Imaging Science and Technology, 40:487–493.
A. Berthold (1997). The influence of blur on the perceived quality and sensation of depth of 2D and stereo images. Technical report, ATR Human Information Processing Research Laboratories.
W. IJsselsteijn, de Ridder, H., Freeman, J., and Avons, S. (2000a). Presence: Concept, determinants and measurement. Proceedings of the SPIE, 3959:520–529.
Häkkinen, J., Liinasuo, M., Takatalo, J., and Nyman, G. 2006. Visual comfort with mobile stereoscopic gaming. Proceedings of SPIE. Vol. 6055
Chipchase, J., Yanqing, C., Jung, Y., and Design, N. 2006. Personal Television: A Qualitative Study of Mobile TV Users. Lecture Notes in Computer Science. Vol. 4471, pp.195-204
Södergård C. (ed.). 2003. Mobile television – technology and user experiences, Report on the Mobile –TV Project. Espoo: VTT Publications 506.
Knoche, H. O., McCarthy, J. D. 2004. Mobile Users Needs and Expectations of Future Multimedia Services. In Proceedings of the WWRF12. Nov. 2004, Toronto, Canada
Carlsson, C., Walden, P. 2007. Mobile TV - To Live or Die by Content, Proc 40th HICSS 51b.
Freeman, J., Avons, S. E. 2000. Focus Group Exploration of Presence through Advanced Broadcast Services. In Proc. SPIE 3959 , 3959-3976.
Jumisko-Pyykkö, S., Hannuksela, M. M. Does Context Matter in Quality Evaluation of Mobile Television? To appear in the Proceedings of 10th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI 2008)
Jumisko-Pyykkö, S. Kumar Malamal Vadakital, V., Hannuksela, M.M. Acceptance Threshold: Bidimensional Research Method for User-Oriented Quality Evaluation Studies. To appear in International Journal of Digital Multimedia Broadcasting, 2008.
Strohmeier, D. Wahrnehmungsuntersuchung von 2D vs. 3D Displays in A/V-Applikationen mittels einer kombinierten Analysemethodik, Diploma Thesis, Ilmenau University of Technology, Germany, 2007
Jumisko-Pyykkö, S., Weitzel, M., Strohmeier, D. "Designing for User Experience: What to Expect from Mobile 3D TV and Video? " . Proceedings of the First International Conference on Designing Interactive User Experiences for TV and Video. October 22 - 24, 2008, Silicon Valley, California, USA.

Mobile3dtv artefact rendering framework is available for download

2009-02-02T11:57:00.002+02:00

We have developed a framework for simulation of mobile 3DTV artifacts. It is able to introduce an arbitrary combination of stereoscopic artifacts to a given video stream, with controlled amount of impairment for each artifact.

It is organized as a collection of Matlab functions, each one responsible for introducing a specific artifact. The configuration is stored in a text file, which describes the input and output video streams, the set of artifacts to be introduced, and the parameters for each artifact. One configuration file can specify a set of artifact parameters, which to be applied over several input video files in “batch mode”.

The framework operates on stereo-video streams (where left and right channel are provided as separate video files) or dense depth video streams (where video and depth channel are provided as two separate video files). Video is decoded into a set of frames, each frame is processed and the result is encoded in a video stream again.

The current version (1.3a) is available for download on our website, in the software download section

Mobile auto-stereoscopic displays

2009-01-30T17:51:00.007+02:00

Mobile 3D displays are designed to fulfil specific requirements. They should be sufficiently small and thin for a handheld device and should provide autostereoscopy – the ability to create 3D effect without requiring glasses. Furthermore, backwards compatibility is a desirable feature – the possibility to switch the display back to “2D” mode, when 3D content is not available. Presently, there are several announced products or prototypes of reconfigurable 3D displays.

A display produced by Sharp uses electronically switchable reconfigurable parallax barrier display based on a patterned retardation film [96]. When the “3D mode” is on, the light passing through the subpixels is selectively blocked, in order different subsets of the screen image to be seen with each eye. These subsets form the “left” and “right” views, respectively. This display has been used to produce the Sharp AL3DU laptops (discontinued). The same parallax barrier has been used for a prototype of portable MVC decoder and player, developed by Nokia Research Center [1].

A reconfigurable 2D/3D technology from Ocuity Ltd. uses a Polarisation Activated Microlens array [2]. The microlens array is made from a birefringent material such that at the surface of the lens there is a refractive index step for only one of the polarisations. A switchable polariser selects either the unrefracted light for 2D displays (i.e. the cylindrical microlens array has no impact) or the refracted light.

A display family announced by NEC provides 2D compatibility by using Horizontally Double-Density Pixels (HDDP) configuration and fixed lenticular lenses. The HDDP configuration is composed of the RGB color arrangement in horizontal stripes with the pixels doubling the horizontal resolution. When the display is to be used in 2D mode, the same image data is written in the two adjacent pixels. With this approach, the 2D/3D switching can be done entirely by software means, and different parts of the displays can be in different modes [3].

Newsight has been marketing 3D displays based on wavelength-selective optical filters and with sizes ranging from 8.4’’ to 54’’. They claim their technology allows also manufacturing portable displays of size of 2’’ or so [4].

A single user 3D display technology developed by SeeFront utilizes lenticular optical layer mounted in front of a LCD panel. A camera tracks the positions of the eyes so to guide a software module which adaptively fractionizes the stereoscopic views [5]. The optical system can meet design requirements for various size displays including portable displays with small viewing distance. Only 3D viewing mode is provided.

Beside the above reviewed display solutions, auto-stereoscopic displays based either on lenticular optics or parallax barrier technology have been marketed by a number of companies [6] - [9].

References:

K. Willner, K. Ugur, M. Salmimaa, A. Hallapuro, J. Lainema, ‘Mobile 3D Video Using MVC and N800 Internet Tablet’, 3DTV-CON 2008, Istanbul, Turkey. Pp. 69-72.
G. J. Woodgate, J. Harrold, “Autostereoscopic display technology for mobile 3DTV applications”, in Proc. SPIE Vol.6490A-19 (Stereoscopic Displays and Applications XVIII), 2007
S.Uehara, T.Hiroya, H. Kusanagi; K. Shigemura, H.Asada, “1-inch diagonal transflective 2D and 3D LCD with HDDP arrangement”, in Proc. SPIE-IS&T Electronic Imaging 2008, Stereoscopic Displays and Applications XIX, Vol. 6803, San Jose, USA, January 2008
Newsight Displays, http://www.newsight.com/3d_products/displays/
SeeFront 3D Technology, http://www.seefront.com/seefront_3d_technology.php
Alioscopy, http://www.alioscopy.eu
MasterImage, http://www.masterimage.co.kr/new_eng/solution/technology.htm
3DIS, http://www.3dis.co.kr/en/eproducts.htm
SpatialView, http://www.spatialview.com/index.cfm

Mobile 3D makes a comeback this year

2009-01-29T09:11:00.006+02:00

A long, long time ago (2003) in a country far, far away (Japan) NTT DoCoMo introduced the first mobile phone with 3D display. The phone, made by Sharp and known by the cryptic name SH251iS had 2.2 inch, parallax barrier-based 3D display, with the usual for 2003 resolution of 176 x 220px. In portrait 3D mode that makes 88 x 220px for each eye - which even back then was barely acceptable.

Fast forward to present time. In 2009, KDDI Japan offers the "WOOO Ketai H001", which is a mobile phone with 3D display from Hitachi. The specs are up to the standards of 2009 - 5Mpix camera, 1seg TV tuner, and most notably - 3.1" 3D display with resolution 800x480 pixels. The display seems to be switchable, parallax barrier 3D (there is dedicated 3D button under the keypad). In stereo-mode this would result in 400x480 pixels for each eye - much more suitable for 3D video, we would say.

Additional note about the resolution: on the KDDI website (as far as we can tell) the phone is listed as being "…3D 3.1 (inch) VGA…", without specifying the exact resolution. The guys from Akihabara news, which probably know better Japanese than us, suggest it is 845x480. However, parallax barrier usually goes with even number of horizontal pixels. Knowing some 3D displays intended for mobile use, we would expect that 800x480 is the proper resolution. We will post an update as soon as we know the details.

[via Engadget and Akihabara news]