Multimedia Communication

Sustainability in Video Encoding and Streaming

noreply@blogger.com (Unknown) — Fri, 27 Mar 2026 21:27:00 +0000

Sustainability in Video Encoding and Streaming:

Energy-Efficient Techniques and Metrics

Workshop on Media Energy Consumption Measurement and Exposure

Presenter: Christian Timmerer (Alpen-Adria-Universität Klagenfurt)

Abstract: The presentation discusses the increasing environmental impact of video streaming and highlights the urgent need for more sustainable approaches across the entire streaming pipeline. Video traffic dominates internet usage and contributes significantly to global greenhouse gas emissions, while the demand for higher quality content continues to drive up computational complexity and energy consumption in encoding, delivery, and playback.

A central insight is that there is a strong trade-off between video quality and energy consumption, where small reductions in quality can lead to substantial energy savings. By introducing energy as an explicit optimization objective, techniques such as content-aware encoding, energy-aware bitrate ladder construction, and real-time optimization for live streaming can significantly reduce energy usage while maintaining nearly the same perceptual quality.

The work also emphasizes the role of adaptive bitrate algorithms that incorporate energy consumption alongside traditional quality and buffer-based metrics. These approaches demonstrate that it is possible to simultaneously improve user experience and reduce energy consumption, indicating that sustainability and performance can be aligned rather than conflicting goals.

To enable such optimizations, the presentation introduces a range of metrics and models, including video complexity measures, quality prediction models, and machine learning-based approaches for estimating encoding and decoding energy as well as CO₂ emissions. These tools support more informed, data-driven decisions across the full streaming workflow from encoding to playback.

Another important theme is end-to-end optimization, where energy efficiency depends on the combined behavior of encoding strategies, bitrate selection, and client-side adaptation. Industry efforts confirm the practical relevance of these approaches and highlight the importance of collaboration and real-world validation.

Despite promising results, several challenges remain, including difficulties in measuring and benchmarking energy consumption, the lack of standardized methodologies, and the limited integration of energy considerations into existing workflows. Overall, the presentation argues that energy consumption should become a first-class optimization target in video streaming systems, similar to established quality metrics, to enable truly sustainable media delivery.

Keywords: sustainable streaming, energy-aware encoding, adaptive bitrate streaming, green multimedia, video compression, bitrate ladder optimization, QoE optimization, energy-quality tradeoff, video complexity analysis, CO2 footprint, energy modeling, machine learning for video, end-to-end optimization, eco-efficient streaming, real-time streaming optimization

MPEG news: a report from the 153rd meeting

noreply@blogger.com (Unknown) — Fri, 20 Feb 2026 09:34:00 +0000

This version of the blog post is also available at ACM SIGMM Records

MPEG News Archive

The 153rd MPEG meeting took place online from January 19-23, 2026. The official MPEG press release can be found here. This report highlights key outcomes from the meeting, with a focus on research directions relevant to the ACM SIGMM community:

MPEG Roadmap
Exploration on MPEG Gaussian Splat Coding (GSC)
MPEG Immersive Video 2nd edition (new white paper)

MPEG Roadmap

MPEG released an updated roadmap showing continued convergence of immersive and “beyond video” media with deployment-ready systems work. Near-term priorities include 6DoF experiences (MPEG Immersive Video v2 and 6DoF audio), volumetric representations (dynamic meshes, solid point clouds, LiDAR, and emerging Gaussian splat coding), and “coding for machines,” which treats visual and audio signals as inputs to downstream analytics rather than only for human consumption.

Research aspects: The most promising research opportunities sit at the intersections: renderer and device-aware rate-distortion-complexity optimization for volumetric content; adaptive streaming and packaging evolution (e.g., MPEG-DASH / CMAF) for interactive 6DoF services under tight latency constraints; and cross-cutting themes such as media authenticity and provenance, green and energy metadata, and exploration threads on neural-network-based compression and compression of neural networks that foreshadow AI-native multimedia pipelines.

MPEG Gaussian Splat Coding (GSC)

Gaussian Splat Coding (GSC) is MPEG’s effort to standardize how 3D Gaussian Splatting content, scenes represented as sparse “Gaussian splats” with geometry plus rich attributes (scale and rotation, opacity, and spherical-harmonics appearance for view-dependent rendering), is encoded, decoded, and evaluated so it can be exchanged and rendered consistently across platforms. The main motivation is interoperability for immersive media pipelines: enabling reproducible results, shared benchmarks, and comparable rate-distortion-complexity trade-offs for use cases spanning telepresence and immersive replay to mobile XR and digital twins, while retaining the visual strengths that made 3DGS attractive compared to heavier neural scene representations.

The work remains in an exploration phase, coordinated across ISO/IEC JTC 1/SC 29 groups WG 4 (MPEG Video Coding) and WG 7 (MPEG Coding for 3D Graphics and Haptics) through Joint Exploration Experiments covering datasets and anchors, new coding tools, software (renderer and metrics), and Common Test Conditions (CTC). A notable systems thread is “lightweight GSC” for resource-constrained devices (single-frame, low-latency tracks using geometry-based and video-based pipelines with explicit time and memory targets), alongside an “early deployment” path via amendments to existing MPEG point-cloud codecs to more natively carry Gaussian-splat parameters. In parallel, MPEG is testing whether splat-specific tools can outperform straightforward mappings in quality, bitrate, and compute for real-time and streaming-centric scenarios.

Research aspects: Relevant SIGMM directions include splat-aware compression tools and rate-distortion-complexity optimization (including tracked vs. non-tracked temporal prediction); QoE evaluation for 6DoF navigation (metrics for view and temporal consistency and splat-specific artifacts); decoder and renderer co-design for real-time and mobile lightweight profiles (progressive and LOD-friendly layouts, GPU-friendly decode); and networked delivery problems such as adaptive streaming, ROI and view-dependent transmission, and loss resilience for splat parameters. Additional opportunities include interoperability work on reproducible benchmarking, conformance testing, and practical packaging and signaling for deployment.

MPEG Immersive Video 2nd edition (white paper)

The second edition of MPEG Immersive Video defines an interoperable bitstream and decoding process for efficient 6DoF immersive scene playback, supporting translational and rotational movement with motion parallax to reduce discomfort often associated with pure 3DoF viewing. The second edition primarily extends functionality (without changing the high-level bitstream structure), adding capabilities such as capture-device information, additional projection types, and support for Simple Multi-Plane Image (MPI), alongside tools that better support geometry and attribute handling and depth-related processing.

Architecturally, MIV ingests multiple (unordered) camera views with geometry (depth and occupancy) and attributes (e.g., texture), then reduces inter-view redundancy by extracting patches and packing them into 2D “atlases” that are compressed using conventional video codecs. MIV-specific metadata signals how to reconstruct views from the atlases. The standard is built as an extension of the common Visual Volumetric Video-based Coding (V3C) bitstream framework shared with V-PCC, with profiles that preserve backward compatibility while introducing a new profile for added second-edition functionality and a tailored profile for full-plane MPI delivery.

Research aspects: Key SIGMM topics include systems-efficient 6DoF delivery (better view and patch selection and atlas packing under latency and bandwidth constraints); rate-distortion-complexity-QoE optimization that accounts for decode and render cost (especially on HMD and mobile) and motion-parallax comfort; adaptive delivery strategies (representation ladders, viewport and pose-driven bit allocation, robust packetization and error resilience for atlas video plus metadata); renderer-aware metrics and subjective protocols for multi-view temporal consistency; and deployment-oriented work such as profile and level tuning, codec-group choices (HEVC / VVC), conformance testing, and exploiting second-edition features (capture device info, depth tools, Simple MPI) for more reliable reconstruction and improved user experience.

Concluding Remarks

The meeting outcomes highlight a clear shift toward immersive and AI-enabled media systems where compression, rendering, delivery, and evaluation must be co-designed. These developments offer timely opportunities for the ACM SIGMM community to contribute reproducible benchmarks, perceptual metrics, and end-to-end streaming and systems research that can directly influence emerging standards and deployments.

The 154th MPEG meeting will be held in Santa Eulària, Spain, from April 27 to May 1, 2026. Click here for more information about MPEG meetings and ongoing developments.

Professor of Information Systems Engineering (all genders welcome)

noreply@blogger.com (Unknown) — Wed, 18 Feb 2026 13:04:00 +0000

Department of Informatics Systems

Full professorships | Full time

Application deadline: 22 March 2026

Reference code: 43/02-PERS/26

URL: https://jobs.aau.at/en/job/professor-of-information-systems-engineering-all-genders-welcome/

Announcement

The University of Klagenfurt wants to attract more women for professorships.

We are pleased to announce the following open position at the Department of Informatics Systems, Faculty of Technical Sciences, in compliance with the provisions of § 98 (permanent) or § 98 (fixed-term, max. 6 years) of the Austrian Universities Act:

Professor of Information Systems Engineering (all genders welcome)

This is a full-time position available from 1 October 2027. Depending on the candidate’s academic credentials, the employment contract can be concluded either as a permanent employment contract or as a fixed-term employment contract with the option of a permanent extension. The duration of fixed-term contracts is subject to negotiation.

With approximately 13,000 students, the University of Klagenfurt is a young, vibrant and innovative university, located at the intersection of Alpine and Mediterranean culture in an area that offers exceptionally high quality of life. As a public university pursuant to § 6 of the Austrian Universities Act, it receives federal funding. The university operates under the motto “Beyond Boundaries!”.

In accordance with its key strategic road map, the Development Plan, the university’s primary guiding principles and objectives include the pursuit of scientific excellence regarding the appointment of professors, favourable research conditions, a good faculty-student ratio, and the promotion of the development of early career researchers.

Information Systems Engineering focuses on the design, development, and management of large systems that connect people, data, and technology to support organizational goals. It combines principles of software engineering, data management, business processes, and emerging digital technologies to create solutions that enhance decision-making, optimize operations, and drive innovation.

We welcome applications addressing the engineering of Information Systems, in particular those focusing on designing, modelling, executing, verifying, and optimizing business processes. We are looking for a highly qualified and internationally visible scientist with high engagement in developing and sustaining an ambitious and innovative research and teaching programme. Candidates should also be interested in developing collaborations in the university’s Areas of Research Strength: Digitalisation and Health, Multiple Perspectives in Optimization, Networked and Autonomous Systems and/or the Cluster of Excellence “Bilateral AI”.

Your responsibilities – what awaits you

The duties of the position include:

Representing the field of Information Systems Engineering in research and teaching
Teaching in relevant degree programmes at Bachelor’s, Master’s, and Doctoral level both in English and German, as well as supervision of student projects and academic theses
Advising and mentoring of students and early career researchers
Competitive research grant acquisition and management
Collaboration with academic and industry partners
Participation in university management
Participation in third mission and public relations activities

Your profile

Habilitation or equivalent qualification in Computer Science or a relevant neighbouring field
Excellent research track record in Information Systems Engineering
Experience in the acquisition and management of competitive third-party funded research projects of a relevant volume
Teaching competence and experience at university level
Experience in the (co-)supervision of academic theses
Fluency in English

This distinguishes you additionally

Interdisciplinary experience
Scientific dissemination skills
Engagement in academic administrative duties
Competence in leadership and teamwork
Competence in gender mainstreaming and diversity management

German language skills are not a formal prerequisite, but proficiency at level B2 is expected within two years.

Why you will enjoy working with us

The salary is subject to negotiation. The minimum gross salary for the position at this level (salary group A1 for University Staff according to the Austrian Universities’ Collective Bargaining Agreement) is currently € 93,986 per year.

The university is committed to increasing the number of women among the faculty, particularly in high-level positions, and therefore specifically invites applications from qualified women. Among equally qualified candidates, women will receive preferential consideration.

People with disabilities or chronic diseases who meet the qualification criteria are explicitly invited to apply.

In accordance with the Austrian Income Tax Act, an attractive relocation tax allowance can be granted for the first five years in the case of appointments to professorships in Austria. The prerequisites are subject to examination on a case by case basis.

Please submit your application in English by e-mail to the University of Klagenfurt, Office of the Senate, attn. Mag.a (FH) Sabine Seebacher via application_professorship@aau.at no later than 22 March, 2026, including:

a mandatory principal part not exceeding five pages (https://jobs.aau.at/wp-content/uploads/specimen_main_part_application_professorship.docx). The submission of the mandatory principal part constitutes a necessary condition for the validity of your application.
one single PDF including:

a letter of motivation
a detailed scientific CV
a comprehensive list of publications, talks, and of all courses taught
a list of projects that you acquired as a PI or co-PI, including the amount of funding that was attributed to you
a research statement
a teaching statement
supplementary documents where applicable (e.g., course evaluations)
links to publicly available versions of your three most important publications within the scope of this professorship

For general information, please refer to the general information provided at https://jobs.aau.at/en/the-university-as-employer/. For specific information about the position, please contact Prof. Dr. Martin Pinzger (Tel.: +43 463 2700 3513; martin.pinzger@aau.at).

MPEG news: a report from the 152nd meeting

noreply@blogger.com (Unknown) — Fri, 28 Nov 2025 12:47:00 +0000

This version of the blog post is also available at ACM SIGMM Records

MPEG News Archive

The 152nd MPEG meeting took place in Geneva, Switzerland, from October 7 to October 11, 2025. The official MPEG press release can be found here. This column highlights key points from the meeting, amended with research aspects relevant to the ACM SIGMM community:

MPEG Systems received an Emmy® Award for the Common Media Application Format (CMAF). A separate press release regarding this achievement is available here.
JVET ratified new editions of VSEI, VVC, and HEVC
The fourth edition of Visual Volumetric Video-based Coding (V3C and V-PCC) has been finalized
Responses to the call for evidence on video compression with capability beyond VVC successfully evaluated

MPEG Systems received an Emmy® Award for the Common Media Application Format (CMAF)

On September 18, 2025, the National Academy of Television Arts & Sciences (NATAS) announced that the MPEG Systems Working Group (ISO/IEC JTC 1/SC 29/WG 3) had been selected as a recipient of a Technology & Engineering Emmy® Award for standardizing the Common Media Application Format (CMAF). But what is CMAF? CMAF (ISO/IEC 23000-19) is a media format standard designed to simplify and unify video streaming workflows across different delivery protocols and devices. Here’s a structured overview. Before CMAF, streaming services often had to produce multiple container formats, i.e., (i) ISO Base Media File Format (ISOBMFF) for MPEG-DASH and MPEG-2 Transport Stream (TS) for Apple HLS. This duplication resulted in additional encoding, packaging, and storage costs. I wrote a blog post about this some time ago here. CMAF’s main goal is to define a single, standardized segmented media format usable by both HLS and DASH, enabling “encode once, package once, deliver everywhere.”

The core concept of CMAF is that it is based on ISOBMFF, the foundation for MP4. Each CMAF stream consists of a CMAF header, CMAF media segments, and CMAF track files (a logical sequence of segments for one stream, e.g., video or audio). CMAF enables low-latency streaming by allowing progressive segment transfer, adopting chunked transfer encoding via CMAF chunks. CMAF defines interoperable profiles for codecs and presentation types for video, audio, and subtitles. Thanks to its compatibility with and adoption within existing streaming standards, CMAF bridges the gaps between DASH and HLS, creating a unified ecosystem.

Research aspects include – but are not limited to – low-latency tuning (segment/chunk size trade-offs, HTTP/3, QUIC), Quality of Experience (QoE) impact of chunk-based adaptation, synchronization of live and interactive CMAF streams, edge-assisted CMAF caching and prediction, and interoperability testing and compliance tools.

JVET ratified new editions of VSEI, VVC, and HEVC

At its 40th meeting, the Joint Video Experts Team (JVET, ISO/IEC JTC 1/SC 29/WG 5) concluded the standardization work on the next editions of three key video coding standards, advancing them to the Final Draft International Standard (FDIS) stage. Corresponding twin-text versions have also been submitted to ITU-T for consent procedures. The finalized standards include:

Versatile Supplemental Enhancement Information (VSEI) — ISO/IEC 23002-7 | ITU-T Rec. H.274
Versatile Video Coding (VVC) — ISO/IEC 23090-3 | ITU-T Rec. H.266
High Efficiency Video Coding (HEVC) — ISO/IEC 23008-2 | ITU-T Rec. H.265

The primary focus of these new editions is the extension and refinement of Supplemental Enhancement Information (SEI) messages, which provide metadata and auxiliary data to support advanced processing, interpretation, and quality management of coded video streams.

The updated VSEI specification introduces both new and refined SEI message types supporting advanced use cases:

AI-driven processing: Extensions for neural-network-based post-filtering and film grain synthesis offer standardized signalling for machine learning components in decoding and rendering pipelines.
Semantic and multimodal content: New SEI messages describe infrared, X-ray, and other modality indicators, region packing, and object mask encoding; creating interoperability points for multimodal fusion and object-aware compression research.
Pipeline optimization: Messages defining processing order and post-processing nesting support research on joint encoder-decoder optimization and edge-cloud coordination in streaming architectures.
Authenticity and generative media: A new set of messages supports digital signature embedding and generative-AI-based face encoding, raising questions for the SIGMM community about trust, authenticity, and ethical AI in media pipelines.
Metadata and interpretability: New SEIs for text description, image format metadata, and AI usage restriction requests could facilitate research into explainable media, human-AI interaction, and regulatory compliance in multimedia systems.

All VSEI features are fully compatible with the new VVC edition, and most are also supported in HEVC. The new HEVC edition further refines its multi-view profiles, enabling more robust 3D and immersive video use cases.

Research aspects of these new standard’s editions can be summarized as follows: (i) Define new standardized interfaces between neural post-processing and conventional video coding, fostering reproducible and interoperable research on learned enhancement models. (ii) Encourage exploration of metadata-driven adaptation and QoE optimization using SEI-based signals in streaming systems. (iii) Open possibilities for cross-layer system research, connecting compression, transport, and AI-based decision layers. (iv) Introduce a formal foundation for authenticity verification, content provenance, and AI-generated media signalling, relevant to current debates on trustworthy multimedia.

These updates highlight how ongoing MPEG/ITU standardization is evolving toward a more AI-aware, multimodal, and semantically rich media ecosystem, providing fertile ground for experimental and applied research in multimedia systems, coding, and intelligent media delivery.

The fourth edition of Visual Volumetric Video-based Coding (V3C and V-PCC) has been finalized

MPEG Coding of 3D Graphics and Haptics (ISO/IEC JTC 1/SC 29/WG7) has advanced MPEG-I Part 5 – Visual Volumetric Video-based Coding (V3C and V-PCC) to the Final Draft International Standard (FDIS) stage, marking its fourth edition. This revision introduces major updates to the Video-based Coding of Volumetric Content (V3C) framework, particularly enabling support for an additional bitstream instance: V-DMC (Video-based Dynamic Mesh Compression).

Previously, V3C served as the structural foundation for V-PCC (Video-based Point Cloud Compression) and MIV (MPEG Immersive Video). The new edition extends this flexibility by allowing V-DMC integration, reinforcing V3C as a generic, extensible framework for volumetric and 3D video coding. All instances follow a shared principle, i.e., using conventional 2D video codecs (e.g., HEVC, VVC) for projection-based compression, complemented by specialized tools for mapping, geometry, and metadata handling.

While V-PCC remains co-specified within Part 5, MIV (Part 12) and V-DMC (Part 29) are standardized separately. The progression to FDIS confirms the technical maturity and architectural stability of the framework.

This evolution opens new research directions as follows: (i) Unified 3D content representation, enabling comparative evaluation of point cloud, mesh, and view-based methods under one coding architecture. (ii) Efficient use of 2D codecs for 3D media, raising questions on mapping optimization, distortion modeling, and geometry-texture compression. (iii) Dynamic and interactive volumetric streaming, relevant to AR/VR, telepresence, and immersive communication research.

The fourth edition of MPEG-I Part 5 thus positions V3C as a cornerstone for future volumetric, AI-assisted, and immersive video systems, bridging standardization and cutting-edge multimedia research.

Responses to the call for evidence on video compression with capability beyond VVC successfully evaluated

The Joint Video Experts Team (JVET, ISO/IEC JTC 1/SC 29/WG 5) has completed the evaluation of submissions to its Call for Evidence (CfE) on video compression with capability beyond VVC. The CfE investigated coding technologies that may surpass the performance of the current Versatile Video Coding (VVC) standard in compression efficiency, computational complexity, and extended functionality.

A total of five submissions were assessed, complemented by ECM16 reference encodings and VTM anchor sequences with multiple runtime variants. The evaluation addressed both compression capability and encoding runtime, as well as low-latency and error-resilience features. All technologies were derived from VTM, ECM, or NNVC frameworks, featuring modified encoder configurations and coding tools rather than entirely new architectures.

Key Findings

In the compression capability test, 76 out of 120 test cases showed at least one submission with a non-overlapping confidence interval compared to the VTM anchor. Several methods outperformed ECM16 in visual quality and achieved notable compression gains at lower complexity. Neural-network-based approaches demonstrated clear perceptual improvements, particularly for 8K HDR content, while gains were smaller for gaming scenarios.
In the encoding runtime test, significant improvements were observed even under strict complexity constraints: 37 of 60 test points (at both 1× and 0.2× runtime) showed statistically significant benefits over VTM. Some submissions achieved faster encoding than VTM, with only a 35% increase in decoder runtime.

Research Relevance and Outlook

The CfE results illustrate a maturing convergence between model-based and data-driven video coding, raising research questions highly relevant for the ACM SIGMM community:

How can learned prediction and filtering networks be integrated into standard codecs while preserving interoperability and runtime control?
What methodologies can best evaluate perceptual quality beyond PSNR, especially for HDR and immersive content?
How can complexity-quality trade-offs be optimized for diverse hardware and latency requirements?

Building on these outcomes, JVET is preparing a Call for Proposals (CfP) for the next-generation video coding standard, with a draft planned for early 2026 and evaluation through 2027. Upcoming activities include refining test material, adding Reference Picture Resampling (RPR), and forming a new ad hoc group on hardware implementation complexity.

For multimedia researchers, this CfE marks a pivotal step toward AI-assisted, complexity-adaptive, and perceptually optimized compression systems, which are considered a key frontier where codec standardization meets intelligent multimedia research.

The 153rd MPEG meeting will be held online from January 19 to January 23, 2026. Click here for more information about MPEG meetings and their developments.

Happy World Standards Day 2025!

noreply@blogger.com (Unknown) — Tue, 14 Oct 2025 11:19:00 +0000

Celebrating innovation, interoperability, and collaboration through international standards.

Every year on October 14, we celebrate World Standards Day — honoring the collective efforts of experts and organizations worldwide who develop and maintain the standards that make modern digital life possible. For the Moving Picture Experts Group (MPEG), this day marks decades of work in defining the technologies that power media, streaming, and immersive experiences worldwide.

A Year of Progress and New Milestones

Over the past year, MPEG and its working groups achieved remarkable progress across video, audio, systems, and AI-driven technologies — advancing the future of multimedia communication. Hot off the press, MPEG is proud to announce another Emmy® Technology & Engineering Award — this time for the Common Media Application Format (CMAF; ISO/IEC 23000-19), a landmark standard that brought long-awaited harmonization between DASH and HLS streaming formats (among others).

Next Generation Video Coding Beyond VVC

The Joint Video Experts Team (JVET), a joint effort of ISO/IEC and ITU-T, launched a Call for Evidence exploring technologies that go beyond Versatile Video Coding (VVC).

The goal: to identify breakthroughs that significantly improve compression efficiency, runtime performance, and functionality — from HDR and 8K video to gaming and user-generated content. Depending on the results, a Call for Proposals (CfP) for the next generation of video coding may follow in 2026, opening the door to AI-enhanced compression.

The current plan foresees a draft CfP in January 2026, followed by the final CfP in July 2026 and submissions in November 2026, with evaluations scheduled for January 2027. The first version of the resulting standard is expected to be finalized within three years thereafter.

MPEG-DASH (Sixth Edition)

Adaptive streaming continues to evolve, and the sixth edition of MPEG-DASH (ISO/IEC 23009-1) marks a major step forward. New features include enhanced low-latency streaming, content steering across multiple CDNs, compact signaling for faster playback, and even support for interactive storylines — enabling richer, more dynamic media experiences. MPEG-DASH remains the foundation of scalable, interoperable video streaming used by billions of devices worldwide.

AI and Machine-Oriented Coding

MPEG’s vision for Audio and Video Coding for Machines continues to take shape. The updated Call for Proposals on Audio Coding for Machines (ACoM) invites technologies for efficiently compressing audio and multi-dimensional signals — not only for human listening but also for machine learning and AI-driven analysis. In parallel, Video Coding for Machines (VCM) is being standardized to optimize visual data for computer vision and autonomous systems, reducing bitrate while preserving task-relevant features.

Open Font Format (Fifth Edition)

MPEG Systems (WG 3) reached the Final Draft International Standard (FDIS) stage for the fifth edition of the Open Font Format (ISO/IEC 14496-22). This major update removes previous technical constraints, supporting over 64K glyphs and the entire Unicode range in a single file — a leap toward more inclusive digital typography across languages and writing systems.

3D and Volumetric Media Innovation

From Video-Based Dynamic Mesh Coding (V-DMC) to Low Latency Point Cloud Compression (L3C2), MPEG advanced two pivotal 3D graphics standards to final draft status. These technologies support real-time 3D content — from immersive AR/VR experiences to LiDAR-based perception in autonomous vehicles — enabling efficient, low-latency, and interoperable volumetric media.

Ensuring Media Authenticity

New amendments to MPEG Audio standards introduce mechanisms for Media Authenticity, allowing verification of content integrity and provenance across audio, video, and system layers. This step is essential for a trustworthy digital media ecosystem.

Genomics and AI Meet Multimedia

MPEG also looked beyond traditional media: the MPEG-G Genomics Hackathon, co-organized with partners such as Stanford Medicine, Philips, and Fudan University, challenges researchers to apply AI to microbiome data encoded in MPEG-G format. The goal: uncover new biomedical insights through standard-based, interoperable data compression.

Looking Ahead

From next-generation video compression and AI-enhanced codecs to trustworthy media and adaptive streaming, MPEG continues to define the building blocks of interoperable multimedia. As new technologies reshape how we experience and analyze content, standards ensure that innovation remains open, efficient, and globally accessible.

On this World Standards Day, we celebrate the dedication of all MPEG experts and contributors for shaping a smarter, more connected multimedia future.

Learn more at www.mpeg.org and stay tuned for updates from the next MPEG meeting in early 2026.

Full Professor of Virtual and Augmented Reality (all genders welcome)

noreply@blogger.com (Unknown) — Wed, 16 Jul 2025 13:41:00 +0000

The official and legally binding job description is available here.

The University of Klagenfurt wants to attract more qualified women for professorships.

The University of Klagenfurt is pleased to announce the following open position in the Department of Information Technology (ITEC) within the Faculty of Technical Sciences, in compliance with the provisions of Art. 98 (open-ended) or Art. 99 (limited to 5 years) of the Austrian Universities Act:

Full Professor of Virtual and Augmented Reality (all genders welcome)

This is a full-time position. Whether the position will be implemented in compliance with the provisions of Art. 98 Austrian Universities Act (open-ended) or Art. 99 of the Austrian Universities Act (limited to 5 years) will be decided in the course of the appointment procedure.

The University of Klagenfurt is a young, vibrant, and innovative university, located at the intersection of Alpine and Mediterranean culture in an area that offers an exceptionally high quality of life. As a public university pursuant to Art. 6 of the Austrian Universities Act, it receives federal funding. The Times Higher Education (THE) Young University Rankings 2021 ranked it among the 50 best young universities in the world. The university operates under the motto “Beyond Boundaries!”.

In accordance with its key strategic road map, the development plan, the university’s primary guiding principles and objectives include the pursuit of scientific excellence regarding the appointment of professors, favourable research conditions, a good faculty-student ratio, and the promotion of the development of young scientists.

The professorship will be embedded in the Department of Information Technology (ITEC; https://itec.aau.at/) within the Faculty of Technical Sciences (https://www.aau.at/en/tewi), which focuses on distributed multimedia systems, including multimedia coding, transmission, and quality of experience, AI-based multimedia analysis, game studies and engineering, as well as distributed cloud and edge computing. The department and faculty provide a vivid, friendly, and research-oriented environment. We are looking for a highly qualified and internationally recognized scientist with high engagement in developing and sustaining an ambitious and innovative research programme.

Virtual and Augmented Reality (VR/AR) are broad research fields addressing both theoretical and application-driven questions. This position offers an opportunity to focus on cutting-edge VR/AR research areas including – but not limited to – immersive media (e.g., 360° videos, 3D point clouds), AI for object recognition in VR/AR (e.g., in industry and medicine), educational and training applications, computer graphics, sensor technology, human-computer interaction, and efficient multimedia data transmission and cloud/edge processing.

The professor will be involved in teaching in a variety of degree programmes, including the Bachelor’s programmes “Applied Informatics” and “Robotics and Artificial Intelligence”, and the international Master’s programmes “Informatics” and “Game Studies and Engineering”.

The duties of the position include:

Representing the field of Virtual and Augmented Reality in research and teaching
Acquiring and managing competitive research funding
Collaborating with colleagues across the university and with industry partners
Teaching in relevant Bachelor’s, Master’s, and Doctoral programmes
Advising and mentoring students and early career researchers
Contributing to the long-term development of the department and its international standing
Advancing the department’s and faculty’s research priorities, with a commitment to interdisciplinary collaboration
Contributing to university governance and academic self-administration
Engaging in third mission activities and public outreach

Required qualifications:

Habilitation or equivalent qualification in a relevant field
Excellent research standing and publication record in Virtual and/or Augmented Reality, including theoretical and technical foundations
Experience in the acquisition of competitive third-party funded research projects of a relevant volume
Teaching experience at university level and didactic competence
Experience in the (co-)supervision of academic theses
Collaboration and social skills
Fluency in English

Desired qualifications:

Excellent scientific communication and dissemination skills
Interdisciplinary experience
Experience with academic management duties
Competence in leadership and management of teams
Competence in gender mainstreaming and diversity management
Fluency in German

German language skills are not a formal prerequisite, but proficiency at level B2 is expected within two years. The remit of the professorship requires that the successful candidate will establish Klagenfurt as primary place of work.

The salary is subject to negotiation. The minimum gross salary for the position at this level (salary group A1 for faculty according to the Austrian Universities’ Collective Bargaining Agreement) is currently € 92,500 per year.

In accordance with the Austrian Income Tax Act an attractive relocation tax allowance can be granted for the first five years in the case of appointments to professorships in Austria. The prerequisites are subject to examination on a case-by-case basis.

a mandatory principal part not exceeding five pages https://jobs.aau.at/wp-content/uploads/specimen_main_part_application_professorship.doc). The submission of the mandatory principal part mentioned above constitutes a necessary condition for the validity of your application.
one single PDF including:

a letter of motivation
a detailed scientific CV
a comprehensive list of publications, talks, and courses taught
a list of acquired third-party funded research projects, including role, funding organization, and amount of funding (in case of funding acquired within a consortium, please specify the amount attributed to you)
a compact research statement of up to two pages
supplementary documents, where applicable (e.g., course evaluations)
links to publicly available versions of your three most important publications within the scope of this professorship

For general information, please refer to the general information on our website provided at https://jobs.aau.at/en/the-university-as-employer/. For specific information about the position, please contact Prof. Dr. Christian Timmerer (christian.timmerer@aau.at).

Up to 4 Predoc Scientist Positions (all genders welcome)

noreply@blogger.com (Unknown) — Wed, 18 Jun 2025 19:17:00 +0000

The University of Klagenfurt, with approximately 1,700 employees and over 13,000 students, is located in the Alps-Adriatic region and consistently achieves excellent placements in rankings. The motto “per aspera ad astra” underscores our firm commitment to the pursuit of excellence in all activities in research, teaching, and university management. The principles of equality, diversity, health, sustainability, and compatibility of work and family life serve as the foundation for our work at the university.

The University of Klagenfurt is in the process of establishing a Karl Popper Kolleg (graduate school) entitled “FruitScope: A DroneScope for Smart Agriculture”. The following positions are open for applicants at this school with an anticipated starting date of October 1, 2025:

Up to 4 Predoc Scientist Positions (all genders welcome)

Level of employment: 75 % (30 hours per week) each
Minimum salary: € 39,005.40 per annum (gross); classification according to collective bargaining agreement: B1
Limited to: 3 years
Application deadline: August 20, 2025
Reference code: 338/25

Tasks and responsibilities:

Independent research and scientific qualification within the Karl Popper Kolleg FruitScope with the aim to acquire the Doctoral Degree in Technical Sciences
Peer-reviewed publication of scientific results in journals and at conferences
Team work and student mentoring
Active participation in public relations activities

This graduate school seeks to push the current bounds of state-of-the-art in navigation, coordination, sensing, and communication of multi agent unmanned aerial vehicles (UAVs). The groups of the involved faculty publish in international top journals and conference proceedings. Successful applicants will be encouraged and supported to publish and present their work in such journals and proceedings and will have the opportunity to cooperate with our world-renowned international partners in science and industry. We currently cooperate with partners worldwide, mainly in the USA/Canada and Europe. We specifically encourage close and open collaboration with our peers both internationally and at the University and support international exchanges with the universities and research institutions affiliated to the graduate school (e.g., ETH Zurich, MIT, CMU, NASA, UofT, U-Mich, UPenn, Georgia Tech). Our young research groups provide a dynamic, familiar, and friendly attitude and thus a collaborative and inspiring work environment with very modern infrastructure (e.g., one of the largest indoor drone halls in Europe), which is continuously updated and upgraded (e.g., soon, with one of the largest outdoor drone test fields in the world).

Prerequisites for the appointment:

Completed Master’s or Diploma degree in electrical engineering, information and communication engineering, mechanical engineering, computer science or related fields. This requirement has an extended deadline and must be fulfilled two weeks before the starting date at the latest; hence, the last possible deadline for meeting this requirement is September 17, 2025.
Proven knowledge and experience in at least one of the following areas: mobile robotics, wireless communications or sensing, multimedia communication, signal processing for communications, or machine learning
Proven programming skills in at least one of the following languages: Matlab, C/C++, Java, Python, ROS or similar
Fluency in English (both written and spoken)

Additional desired qualifications:

Good knowledge of cooperative software development (e.g., with GIT)
First scientific publication (apart from Master’s or Diploma thesis) in the area of mobile robotics, wireless sensing, or multimedia communication technology
Relevant international or practical experience
Good scientific communication and presentation skills
German language skills or willingness to acquire German language skills within the first two years of service
Social skills and ability to work independently

Our offer:

The employment contract is concluded for the position as predoc scientist and stipulates a starting salary of € 2,786.10 gross per month (14 times a year; previous experience deemed relevant to the job can be recognized).

The University of Klagenfurt also offers:

Personal and professional advanced training courses, management and career coaching, including bespoke training for women in science
Numerous attractive additional benefits, see also https://jobs.aau.at/en/the-university-as-employer/
Diversity- and family-friendly university culture
The opportunity to live and work in the attractive Alps-Adriatic region with a wide range of leisure activities in the spheres of culture, nature and sports

The application:

If you are interested in this position, please apply in English providing the following documents:

Letter of application explaining the motivation and including a statement of interest in research (indicating an idea for the research for your own doctoral degree)
Curriculum vitae (please do not include a photo)
Copies of degree certificates (Bachelor and Master)
Copies of official transcripts (Bachelor and Master) containing a list of all courses and grades
Master’s thesis. If the thesis is not available, the candidate should provide a draft or an explanation.
If an applicant has not received the Master’s degree by the application deadline, the applicant should provide a declaration, written either by a supervisor or by the candidate themselves, on the feasibility of finishing the Master’s degree before September 17, 2025.

To apply, please select the position with the reference code 338/25 in the category “Scientific Staff” using the link “Apply for this position” in the job portal at https://jobs.aau.at/en/.

Candidates must provide proof that they meet the required qualifications by August 20, 2025, at the latest. However, candidates who fulfil the required qualifications but do not yet possess the required Master’s degree can apply, provided they are able to meet this requirement at least two weeks before the starting date. Therefore, the latest possible deadline for meeting this requirement is September 17, 2025.

General information about the university as an employer can be found at https://jobs.aau.at/en/the-university-as-employer/. At the University of Klagenfurt, recruitment and staff matters are accompanied not only by the authority responsible for the recruitment procedure but also by the Equal Opportunities Working Group and, if applicable, by the Representative for Disabled Persons.

For further information on this specific vacancy, please contact:

Prof Dr. Stephan Weiss, +43 463 2700 3571, Stephan.Weiss@aau.at
Prof Dr. Christian Bettstetter, +43 463 2700 3640, Christian.Bettstetter@aau.at
Prof Dr. Bernhard Rinner, +43 463 2700 3671, Bernhard.Rinner@aau.at
Prof Dr. Christian Timmerer +43 463 2700 3621, Christian.Timmerer@aau.at

The University of Klagenfurt aims to increase the proportion of women and therefore specifically invites qualified women to apply for the position. Where the qualification is equivalent, women will be given preferential consideration.

People with disabilities or chronic diseases, who fulfil the requirements, are particularly encouraged to apply. Travel and accommodation costs incurred during the application process will not be refunded. Under exceptional circumstances online hearings may be possible. Translations into other languages serve informational purposes only. Solely the version advertised in the University Bulletin (Mitteilungsblatt) shall be legally binding.

MPEG news: a report from the 150th meeting

noreply@blogger.com (Unknown) — Fri, 09 May 2025 12:42:00 +0000

This version of the blog post is also available at ACM SIGMM Records.

MPEG News Archive

The 150th MPEG meeting was held online from 31 March to 04 April 2025. The official press release can be found here. This blog post provides the following highlights:

Requirements: MPEG-AI strategy and white paper on MPEG technologies for metaverse
JVET: Draft Joint Call for Evidence on video compression with capability beyond Versatile Video Coding (VVC)
Video: Gaussian splat coding and video coding for machines
Audio: Audio coding for machines
3DGH: 3D Gaussian splat coding

MPEG-AI Strategy

The MPEG-AI strategy envisions a future where AI and neural networks are deeply integrated into multimedia coding and processing, enabling transformative improvements in how digital content is created, compressed, analyzed, and delivered. By positioning AI at the core of multimedia systems, MPEG-AI seeks to enhance both content representation and intelligent analysis. This approach supports applications ranging from adaptive streaming and immersive media to machine-centric use cases like autonomous vehicles and smart cities. AI is employed to optimize coding efficiency, generate intelligent descriptors, and facilitate seamless interaction between content and AI systems. The strategy builds on foundational standards such as ISO/IEC 15938-13 (CDVS), 15938-15 (CDVA), and 15938-17 (Neural Network Coding), which collectively laid the groundwork for integrating AI into multimedia frameworks.

Currently, MPEG is developing a family of standards under the ISO/IEC 23888 series that includes a vision document, machine-oriented video coding, and encoder optimization for AI analysis. Future work focuses on feature coding for machines and AI-based point cloud compression to support high-efficiency 3D and visual data handling. These efforts reflect a paradigm shift from human-centric media consumption to systems that also serve intelligent machine agents. MPEG-AI maintains compatibility with traditional media processing while enabling scalable, secure, and privacy-conscious AI deployments. Through this initiative, MPEG aims to define the future of multimedia as an intelligent, adaptable ecosystem capable of supporting complex, real-time, and immersive digital experiences.

MPEG White Paper on Metaverse Technologies

The MPEG white paper on metaverse technologies (cf. MPEG white papers) outlines the pivotal role of MPEG standards in enabling immersive, interoperable, and high-quality virtual experiences that define the emerging metaverse. It identifies core metaverse parameters – real-time operation, 3D experience, interactivity, persistence, and social engagement – and maps them to MPEG’s longstanding and evolving technical contributions. From early efforts like MPEG-4’s Binary Format for Scenes (BIFS) and Animation Framework eXtension (AFX) to MPEG-V’s sensory integration, and the advanced MPEG-I suite, these standards underpin critical features such as scene representation, dynamic 3D asset compression, immersive audio, avatar animation, and real-time streaming. Key technologies like point cloud compression (V-PCC, G-PCC), immersive video (MIV), and dynamic mesh coding (V-DMC) demonstrate MPEG’s capacity to support realistic, responsive, and adaptive virtual environments. Recent efforts include neural network compression for learned scene representations (e.g., NeRFs), haptic coding formats, and scene description enhancements, all geared toward richer user engagement and broader device interoperability.

The document highlights five major metaverse use cases – virtual environments, immersive entertainment, virtual commerce, remote collaboration, and digital twins – all supported by MPEG innovations. It emphasizes the foundational role of MPEG-I standards (e.g., Parts 12, 14, 29, 39) for synchronizing immersive content, representing avatars, and orchestrating complex 3D scenes across platforms. Future challenges identified include ensuring interoperability across systems, advancing compression methods for AI-assisted scenarios, and embedding security and privacy protections. With decades of multimedia expertise and a future-focused standards roadmap, MPEG positions itself as a key enabler of the metaverse – ensuring that emerging virtual ecosystems are scalable, immersive, and universally accessible.

The MPEG white paper on metaverse technologies highlights several research opportunities, including efficient compression of dynamic 3D content (e.g., point clouds, meshes, neural representations), synchronization of immersive audio and haptics, real-time adaptive streaming, and scene orchestration. It also points to challenges in standardizing interoperable avatar formats, AI-enhanced media representation, and ensuring seamless user experiences across devices. Additional research directions include neural network compression, cross-platform media rendering, and developing perceptual metrics for immersive Quality of Experience (QoE).

Draft Joint Call for Evidence (CfE) on Video Compression beyond Versatile Video Coding (VVC)

The latest JVET AHG report on ECM software development (AHG6), documented as JVET-AL0006, shows promising results. Specifically, in the “Overall” row and “Y” column, there is a 27.06% improvement in coding efficiency compared to VVC, as shown in the figure below.

The Draft Joint Call for Evidence (CfE) on video compression beyond VVC (Versatile Video Coding), identified as document JVET-AL2026 | N 355, is being developed to explore new advancements in video compression. The CfE seeks evidence in three main areas: (a) improved compression efficiency and associated trade-offs, (b) encoding under runtime constraints, and (c) enhanced performance in additional functionalities. This initiative aims to evaluate whether new techniques can significantly outperform the current state-of-the-art VVC standard in both compression and practical deployment aspects.

The visual testing will be carried out across seven categories, including various combinations of resolution, dynamic range, and use cases: SDR Random Access UHD/4K, SDR Random Access HD, SDR Low Bitrate HD, HDR Random Access 4K, HDR Random Access Cropped 8K, Gaming Low Bitrate HD, and UGC (User-Generated Content) Random Access HD. Sequences and rate points for testing have already been defined and agreed upon. For a fair comparison, rate-matched anchors using VTM (VVC Test Model) and ECM (Enhanced Compression Model) will be generated, with new configurations to enable reduced run-time evaluations. A dry-run of the visual tests is planned during the upcoming Daejeon meeting, with ECM and VTM as reference anchors, and the CfE welcomes additional submissions. Following this dry-run, the final Call for Evidence is expected to be issued in July, with responses due in October.

The Draft Joint Call for Evidence (CfE) on video compression beyond VVC invites research into next-generation video coding techniques that offer improved compression efficiency, reduced encoding complexity under runtime constraints, and enhanced functionalities such as scalability or perceptual quality. Key research aspects include optimizing the trade-off between bitrate and visual fidelity, developing fast encoding methods suitable for constrained devices, and advancing performance in emerging use cases like HDR, 8K, gaming, and user-generated content.

3D Gaussian Splat Coding

Gaussian splatting is a real-time radiance field rendering method that represents a scene using 3D Gaussians. Each Gaussian has parameters like position, scale, color, opacity, and orientation, and together they approximate how light interacts with surfaces in a scene. Instead of ray marching (as in NeRF), it renders images by splatting the Gaussians onto a 2D image plane and blending them using a rasterization pipeline, which is GPU-friendly and much faster. Developed by Kerbl et al. (2023) it is capable of real-time rendering (60+ fps) and outperforms previous NeRF-based methods in speed and visual quality. Gaussian splat coding refers to the compression and streaming of 3D Gaussian representations for efficient storage and transmission. It's an active research area and under standardization consideration in MPEG.

MPEG technical requirements working group together with MPEG video working group started an exploration on Gaussian splat coding and the MPEG coding of 3D graphics and haptics (3DGH) working group addresses 3D Gaussian splat coding, respectively. Draft Gaussian splat coding use cases and requirements are available and various joint exploration experiments (JEEs) are conducted between meetings.

(3D) Gaussian splat coding is actively researched in academia, also in the context of streaming, e.g., like in “LapisGS: Layered Progressive 3D Gaussian Splatting for Adaptive Streaming” or “LTS: A DASH Streaming System for Dynamic Multi-Layer 3D Gaussian Splatting Scenes”. The research aspects of 3D Gaussian splat coding and streaming span a wide range of areas across computer graphics, compression, machine learning, and systems for real-time immersive media. In particular, on efficiently representing and transmitting Gaussian-based neural scene representations for real-time rendering. Key areas include compression of Gaussian parameters (position, scale, color, opacity), perceptual and geometry-aware optimizations, and neural compression techniques such as learned latent coding. Streaming challenges involve adaptive, view-dependent delivery, level-of-detail management, and low-latency rendering on edge or mobile devices. Additional research directions include standardizing file formats, integrating with scene graphs, and ensuring interoperability with existing 3D and immersive media frameworks.

MPEG Audio and Video Coding for Machines

The Call for Proposals on Audio Coding for Machines (ACoM), issued by the MPEG audio coding working group, aims to develop a standard for efficiently compressing audio, multi-dimensional signals (e.g., medical data), or extracted features for use in machine-driven applications. The standard targets use cases such as connected vehicles, audio surveillance, diagnostics, health monitoring, and smart cities, where vast data streams must be transmitted, stored, and processed with low latency and high fidelity. The ACoM system is designed in two phases: the first focusing on near-lossless compression of audio and metadata to facilitate training of machine learning models, and the second expanding to lossy compression of features optimized for specific applications. The goal is to support hybrid consumption – by machines and, where needed, humans – while ensuring interoperability, low delay, and efficient use of storage and bandwidth.

The CfP outlines technical requirements, submission guidelines, and evaluation metrics. Participants must provide decoders compatible with Linux/x86 systems, demonstrate performance through objective metrics like compression ratio, encoder/decoder runtime, and memory usage, and undergo a mandatory cross-checking process. Selected proposals will contribute to a reference model and working draft of the standard. Proponents must register by August 1, 2025, with submissions due in September, and evaluation taking place in October. The selection process emphasizes lossless reproduction, metadata fidelity, and significant improvements over a baseline codec, with a path to merge top-performing technologies into a unified solution for standardization.

Research aspects of Audio Coding for Machines (ACoM) include developing efficient compression techniques for audio and multi-dimensional data that preserve key features for machine learning tasks, optimizing encoding for low-latency and resource-constrained environments, and designing hybrid formats suitable for both machine and human consumption. Additional research areas involve creating interoperable feature representations, enhancing metadata handling for context-aware processing, evaluating trade-offs between lossless and lossy compression, and integrating machine-optimized codecs into real-world applications like surveillance, diagnostics, and smart systems.

The MPEG video coding working group approved the committee draft (CD) for ISO/IEC 23888-2 video coding for machines (VCM). VCM aims to encode visual content in a way that maximizes machine task performance, such as computer vision, scene understanding, autonomous driving, smart surveillance, robotics and IoT. Instead of preserving photorealistic quality, VCM seeks to retain features and structures important for machines, possibly at much lower bitrates than traditional video codecs. The CD introduces several new tools and enhancements aimed at improving machine-centric video processing efficiency. These include updates to spatial resampling, such as the signaling of the inner decoded picture size to better support scalable inference. For temporal resampling, the CD enables adaptive resampling ratios and introduces pre- and post-filters within the temporal resampler to maintain task-relevant temporal features. In the filtering domain, it adopts bit depth truncation techniques – integrating bit depth shifting, luma enhancement, and chroma reconstruction – to optimize both signaling efficiency and cross-platform interoperability. Luma enhancement is further refined through an integer-based implementation for luma distribution parameters, while chroma reconstruction is stabilized across different hardware platforms. Additionally, the CD proposes removing the neural network-based in-loop filter (NNLF) to simplify the pipeline. Finally, in terms of bitstream structure, it adopts a flattened structure with new signaling methods to support efficient random access and better coordination with system layers, aligning with the low-latency, high-accuracy needs of machine-driven applications.

Research in VCM focuses on optimizing video representation for downstream machine tasks, exploring task-driven compression techniques that prioritize inference accuracy over perceptual quality. Key areas include joint video and feature coding, adaptive resampling methods tailored to machine perception, learning-based filter design, and bitstream structuring for efficient decoding and random access. Other important directions involve balancing bitrate and task accuracy, enhancing robustness across platforms, and integrating machine-in-the-loop optimization to co-design codecs with AI inference pipelines.

Concluding Remarks

The 150th MPEG meeting marks significant progress across AI-enhanced media, immersive technologies, and machine-oriented coding. With ongoing work on MPEG-AI, metaverse standards, next-gen video compression, Gaussian splat representation, and machine-friendly audio and video coding, MPEG continues to shape the future of interoperable, intelligent, and adaptive multimedia systems. The research opportunities and standardization efforts outlined in this meeting provide a strong foundation for innovations that support real-time, efficient, and cross-platform media experiences for both human and machine consumption.

The 151st MPEG meeting will be held in Daejeon, Korea, from 30 June to 04 July 2025. Click here for more information about MPEG meetings and their developments.

MPEG news: a report from the 149th meeting

noreply@blogger.com (Unknown) — Fri, 14 Mar 2025 08:40:00 +0000

This blog post is based on the MPEG press release and has been modified/updated here to focus on and highlight research aspects. This version of the blog post will also be posted at ACM SIGMM Records.

MPEG News Archive

The 149th MPEG meeting took place in Geneva, Switzerland, from January 20 to 24, 2025. The official press release can be found here. MPEG promoted three standards (among others) to Final Draft International Standard (FDIS), driving innovation in next-generation, immersive audio and video coding, and adaptive streaming:

MPEG-I Immersive Audio enables realistic 3D audio with six degrees of freedom (6DoF).
MPEG Immersive Video (Second Edition) introduces advanced coding tools for volumetric video.
MPEG-DASH (Sixth Edition) enhances low-latency streaming, content steering, and interactive media.

This blog post focuses on these new standards/editions based on the press release and amended with research aspect relevant for the ACM SIGMM community.

MPEG-I Immersive Audio

At the 149th MPEG meeting, MPEG Audio Coding (WG 6) promoted ISO/IEC 23090-4 MPEG-I immersive audio to Final Draft International Standard (FDIS), marking a major milestone in the development of next-generation audio technology.

MPEG-I immersive audio is a groundbreaking standard designed for the compact and highly realistic representation of spatial sound. Tailored for Metaverse applications, including Virtual, Augmented, and Mixed Reality (VR/AR/MR), it enables seamless real-time rendering of interactive 3D audio with six degrees of freedom (6DoF). Users can not only turn their heads in any direction (pitch/yaw/roll) but also move freely through virtual environments (x/y/z), creating an unparalleled sense of immersion.

True to MPEG’s legacy, this standard is optimized for efficient distribution – even over networks with severe bitrate constraints. Unlike proprietary VR/AR audio solutions, MPEG-I Immersive Audio ensures broad interoperability, long-term stability, and suitability for both streaming and downloadable content. It also natively integrates MPEG-H 3D Audio for high-quality compression.

The standard models a wide range of real-world acoustic effects to enhance realism. It captures detailed sound source properties (e.g., level, point sources, extended sources, directivity characteristics, and Doppler effects) as well as complex environmental interactions (e.g., reflections, reverberation, diffraction, and both total and partial occlusion). Additionally, it supports diverse acoustic environments, including outdoor spaces, multiroom scenes with connecting portals, and areas with dynamic openings such as doors and windows. Its rendering engine balances computational efficiency with high-quality output, making it suitable for a variety of applications.

Further reinforcing its impact, the upcoming ISO/IEC 23090-34 Immersive audio reference software will fully implement MPEG-I immersive audio in a real-time framework. This interactive 6DoF experience will facilitate industry adoption and accelerate innovation in immersive audio. The reference software is expected to reach FDIS status by April 2025.

With MPEG-I immersive audio, MPEG continues to set the standard for the future of interactive and spatial audio, paving the way for more immersive digital experiences.

Research aspects: Research can focus on optimizing the streaming and compression of MPEG-I immersive audio for constrained networks, ensuring efficient delivery without compromising spatial accuracy. Another key area is improving real-time 6DoF audio rendering by balancing computational efficiency and perceptual realism, particularly in modeling complex acoustic effects like occlusions, reflections, and Doppler shifts for interactive VR/AR/MR applications.

MPEG Immersive Video (Second Edition)

At the 149th MPEG meeting, MPEG Video Coding (WG 4) advanced the second edition of ISO/IEC 23090-12 MPEG immersive video (MIV) to Final Draft International Standard (FDIS), marking a significant step forward in immersive video technology.

MIV enables the efficient compression, storage, and distribution of immersive video content, where multiple real or virtual cameras capture a 3D scene. Designed for next-generation applications, the standard supports playback with six degrees of freedom (6DoF), allowing users to not only change their viewing orientation (pitch/yaw/roll) but also move freely within the scene (x/y/z). By leveraging strong hardware support for widely used video formats, MPEG immersive video provides a highly flexible framework for multi-view video plus depth (MVD) and multi-plane image (MPI) video coding, making volumetric video more accessible and efficient.

With the second edition, MPEG continues to expand the capabilities of MPEG immersive video, introducing a range of new technologies to enhance coding efficiency and support more advanced immersive experiences. Key additions include:

Geometry coding using luma and chroma planes, improving depth representation
Capture device information, enabling better reconstruction of the original scene
Patch margins and background views, optimizing scene composition
Static background atlases, reducing redundant data for stationary elements
Support for decoder-side depth estimation, enhancing depth accuracy
Chroma dynamic range modification, improving color fidelity
Piecewise linear normalized disparity quantization and linear depth quantization, refining depth precision

The second edition also introduces two new profiles: (1) MIV Simple MPI profile, allowing MPI content playback with a single 2D video decoder, and (2) MIV 2 profile, a superset of existing profiles that incorporates all newly added tools.

With these advancements, MPEG immersive video continues to push the boundaries of immersive media, providing a robust and efficient solution for next-generation video applications.

Research aspects: Possible research may explore advancements in MPEG immersive video to improve compression efficiency and real-time streaming while preserving depth accuracy and spatial quality. Another key area is enhancing 6DoF video rendering by leveraging new coding tools like decoder-side depth estimation and geometry coding, enabling more precise scene reconstruction and seamless user interaction in volumetric video applications.

MPEG-DASH (Sixth Edition)

At the 149th MPEG meeting, MPEG Systems (WG 3) advanced the sixth edition of MPEG-DASH (ISO/IEC 23009-1 Media presentation description and segment formats) by promoting it to the Final Draft International Standard (FDIS), the final stage of standards development. This milestone underscores MPEG’s ongoing commitment to innovation and responsiveness to evolving market needs.

The sixth edition introduces several key enhancements to improve the flexibility and efficiency of MPEG-DASH:

Alternative media presentation support, enabling seamless switching between main and alternative streams
Content steering signaling across multiple CDNs, optimizing content delivery
Enhanced segment sequence addressing, improving low-latency streaming and faster tune-in
Compact duration signaling using patterns, reducing MPD overhead
Support for Common Media Client Data (CMCD), enabling better client-side analytics
Nonlinear playback for interactive storylines, expanding support for next-generation media experiences

With these advancements, MPEG-DASH continues to evolve as a robust and scalable solution for adaptive streaming, ensuring greater efficiency, flexibility, and enhanced user experiences across a wide range of applications.

Research aspects: While advancing MPEG-DASH for more efficient and flexible adaptive streaming has been subject to research for a while, optimizing content delivery across multiple CDNs while minimizing latency and optimizing QoE remains an open issue. Another key area is enhancing interactivity and user experiences by leveraging new features like nonlinear playback for interactive storylines and improved client-side analytics through Common Media Client Data (CMCD).

The 150th MPEG meeting will be held online from March 31 to April 04, 2025. Click here for more information about MPEG meetings and their developments.

MPEG news: a report from the 148th meeting

noreply@blogger.com (Unknown) — Fri, 06 Dec 2024 14:14:00 +0000

MPEG News Archive

The 148th MPEG meeting took place in Kemer, Türkiye, from November 4 to 8, 2024. The official press release can be found here and includes the following highlights:

Point Cloud Coding: AI-based point cloud coding & enhanced G-PCC
MPEG Systems: New Part of MPEG DASH for redundant encoding and packaging, reference software and conformance of ISOBMFF, and a new structural CMAF brand profile
Video Coding: New part of MPEG-AI and 2nd edition of conformance and reference software for MPEG Immersive Video (MIV)
MPEG completes subjective quality testing for film grain synthesis using the Film Grain Characteristics SEI message

148th MPEG Meeting, Kemer, Türkiye, November 4-8, 2024.

Point Cloud Coding

At the 148^th MPEG meeting, MPEG Coding of 3D Graphics and Haptics (WG 7) launched a new AI-based Point Cloud Coding standardization project. MPEG WG 7 reviewed six responses to a Call for Proposals (CfP) issued in April 2024 targeting the full range of point cloud formats, from dense point clouds used in immersive applications to sparse point clouds generated by Light Detection and Ranging (LiDAR) sensors in autonomous driving. With bit depths ranging from 10 to 18 bits, the CfP called for solutions that could meet the precision requirements of these varied use cases.

Among the six reviewed proposals, the leading proposal distinguished itself with a hybrid coding strategy that integrates end-to-end learning-based geometry coding and traditional attribute coding. This proposal demonstrated exceptional adaptability, capable of efficiently encoding both dense point clouds for immersive experiences and sparse point clouds from LiDAR sensors. With its unified design, the system supports inter-prediction coding using a shared model with intra-coding, applicable across various bitrate requirements without retraining. Furthermore, the proposal offers flexible configurations for both lossy and lossless geometry coding.

Performance assessments highlighted the leading proposal’s effectiveness, with significant bitrate reductions compared to traditional codecs: a 47% reduction for dense, dynamic sequences in immersive applications and a 35% reduction for sparse dynamic sequences in LiDAR data. For combined geometry and attribute coding, it achieved a 40% bitrate reduction across both dense and sparse dynamic sequences, while subjective evaluations confirmed its superior visual quality over baseline codecs.

The leading proposal has been selected as the initial test model, which can be seen as a baseline implementation for future improvements and developments. Additionally, MPEG issued a working draft and common test conditions.

Research aspects: The initial test model, like those for other codec test models, is typically available as open source. This enables both academia and industry to contribute to refining various elements of the upcoming AI-based Point Cloud Coding standard. Of particular interest is how training data and processes are incorporated into the standardization project and their impact on the final standard.

Another point cloud-related project is called Enhanced G-PCC, which introduces several advanced features to improve the compression and transmission of 3D point clouds. Notable enhancements include inter-frame coding, refined octree coding techniques, Trisoup surface coding for smoother geometry representation, and dynamic Optimal Binarization with Update On-the-fly (OBUF) modules. These updates provide higher compression efficiency while managing computational complexity and memory usage, making them particularly advantageous for real-time processing and high visual fidelity applications, such as LiDAR data for autonomous driving and dense point clouds for immersive media.

By adding this new part to MPEG-I, MPEG addresses the industry's growing demand for scalable, versatile 3D compression technology capable of handling both dense and sparse point clouds. Enhanced G-PCC provides a robust framework that meets the diverse needs of both current and emerging applications in 3D graphics and multimedia, solidifying its role as a vital component of modern multimedia systems.

MPEG Systems Updates

At its 148^th meeting, MPEG Systems (WG 3) worked on the following aspects, among others:

New Part of MPEG DASH for redundant encoding and packaging
Reference software and conformance of ISOBMFF
A new structural CMAF brand profile

The second edition of ISO/IEC 14496-32 (ISOBMFF) introduces updated reference software and conformance guidelines, and the new CMAF brand profile supports Multi-View High Efficiency Video Coding (MV-HEVC), which is compatible with devices like Apple Vision Pro and Meta Quest 3.

The new part of MPEG DASH, ISO/IEC 23009-9, addresses redundant encoding and packaging for segmented live media (REAP). The standard is designed for scenarios where redundant encoding and packaging are essential, such as 24/7 live media production and distribution in cloud-based workflows. It specifies formats for interchangeable live media ingest and stream announcements, as well as formats for generating interchangeable media presentation descriptions. Additionally, it provides failover support and mechanisms for reintegrating distributed components in the workflow, whether they involve file-based content, live inputs, or a combination of both.

Research aspects: With the FDIS of MPEG DASH REAP available, the following topics offer potential for both academic and industry-driven research aligned with the standard's objectives (in no particular order or priority):

Optimization of redundant encoding and packaging: Investigate methods to minimize resource usage (e.g., computational power, storage, and bandwidth) in redundant encoding and packaging workflows. Explore trade-offs between redundancy levels and quality of service (QoS) in segmented live media scenarios.
Interoperability of live media Ingest formats: Evaluate the interoperability of the standard's formats with existing live media workflows and tools. Develop techniques for seamless integration with legacy systems and emerging cloud-based media workflows.
Failover mechanisms for cloud-based workflows: Study the reliability and latency of failover mechanisms in distributed live media workflows. Propose enhancements to the reintegration of failed components to maintain uninterrupted service.
Standardized stream announcements and descriptions: Analyze the efficiency and scalability of stream announcement formats in large-scale live streaming scenarios. Research methods for dynamically updating media presentation descriptions during live events.
Hybrid workflow support: Investigate the challenges and opportunities in combining file-based and live input workflows within the standard. Explore strategies for adaptive workflow transitions between live and on-demand content.
Cloud-based workflow scalability: Examine the scalability of the REAP standard in high-demand scenarios, such as global live event streaming. Study the impact of cloud-based distributed workflows on latency and synchronization.
Security and resilience: Research security challenges related to redundant encoding and packaging in cloud environments. Develop techniques to enhance the resilience of workflows against cyberattacks or system failures.
Performance metrics and quality assessment: Define performance metrics for evaluating the effectiveness of REAP in live media workflows. Explore objective and subjective quality assessment methods for media streams delivered using this standard.

The current/updated status of MPEG-DASH is shown in the figure below.

MPEG-DASH status, November 2024.

Video Coding Updates

In terms of video coding, two noteworthy updates are described here:

Part 3 of MPEG-AI, ISO/IEC 23888-3 – Optimization of encoders and receiving systems for machine analysis of coded video content, reached Committee Draft Technical Report (CDTR) status
Second edition of conformance and reference software for MPEG Immersive Video (MIV). This draft includes verified and validated conformance bitstreams and encoding and decoding reference software based on version 22 of the Test model for MPEG immersive video (TMIV). The test model, objective metrics, and some other tools are publicly available at https://gitlab.com/mpeg-i-visual.

Part 3 of MPEG-AI, ISO/IEC 23888-3: This new technical report on "optimization of encoders and receiving systems for machine analysis of coded video content" is based on software experiments conducted by JVET, focusing on optimizing non-normative elements such as preprocessing, encoder settings, and postprocessing. The research explored scenarios where video signals, decoded from bitstreams compliant with the latest video compression standard, ISO/IEC 23090-3 – Versatile Video Coding (VVC), are intended for input into machine vision systems rather than for human viewing. Compared to the JVET VVC reference software encoder, which was originally optimized for human consumption, significant bit rate reductions were achieved when machine vision task precision was used as the performance criterion.

The report will include an annex with example software implementations of these non-normative algorithmic elements, applicable to VVC or other video compression standards. Additionally, it will explore the potential use of existing supplemental enhancement information messages from ISO/IEC 23002-7 – Versatile supplemental enhancement information messages for coded video bitstreams – for embedding metadata useful in these contexts.

Research aspects: (1) Focus on optimizing video encoding for machine vision tasks by refining preprocessing, encoder settings, and postprocessing to improve bit rate efficiency and task precision, compared to traditional approaches for human viewing. (2) Examine the use of metadata, specifically SEI messages from ISO/IEC 23002-7, to enhance machine analysis of compressed video, improving adaptability, performance, and interoperability.

Subjective Quality Testing for Film Grain Synthesis

At the 148^th MPEG meeting , the MPEG Joint Video Experts Team (JVET) with ITU-T SG 16 (WG 5 / JVET) and MPEG Visual Quality Assessment (AG 5) conducted a formal expert viewing experiment to assess the impact of film grain synthesis on the subjective quality of video content. This evaluation specifically focused on film grain synthesis controlled by the Film Grain Characteristics (FGC) supplemental enhancement information (SEI) message. The study aimed to demonstrate the capability of film grain synthesis to mask compression artifacts introduced by the underlying video coding schemes.

For the evaluation, FGC SEI messages were adapted to a diverse set of video sequences, including scans of original film material, digital camera noise, and synthetic film grain artificially applied to digitally captured video. The subjective performance of video reconstructed from VVC and HEVC bitstreams was compared with and without film grain synthesis. The results highlighted the effectiveness of film grain synthesis, showing a significant improvement in subjective quality and enabling bitrate savings of up to a factor of 10 for certain test points.

This study opens several avenues for further research:

Optimization of film grain synthesis techniques: Investigating how different grain synthesis methods affect the perceptual quality of video across a broader range of content and compression levels.
Compression artifact mitigation: Exploring the interaction between film grain synthesis and specific types of compression artifacts, with a focus on improving masking efficiency.
Adaptation of FGC SEI messages: Developing advanced algorithms for tailoring FGC SEI messages to dynamically adapt to diverse video characteristics, including real-time encoding scenarios.
Bitrate savings analysis: Examining the trade-offs between bitrate savings and subjective quality across various coding standards and network conditions.

The 149th MPEG meeting will be held in Geneva, Switzerland from January 20-24, 2025. Click here for more information about MPEG meetings and their developments.

Happy World Standards Day 2024

noreply@blogger.com (Unknown) — Mon, 14 Oct 2024 07:09:00 +0000

As we celebrate World Standards Day, it's important to recognize the monumental advancements the MPEG community has made over the past year. These achievements continue to influence multimedia standards worldwide, playing a crucial role in ensuring seamless, high-quality digital experiences.

ISO Base Media File Format (8th Edition): This standard has been pivotal for media streaming applications, particularly for formats like DASH (Dynamic Adaptive Streaming over HTTP) and CMAF (Common Media Application Format). The latest update facilitates more seamless media switching and continuous presentation, optimizing the user experience across different devices.
Neural Network Compression (2nd Edition): With AI technologies rapidly evolving, MPEG's neural network compression standard addresses the need for efficient storage and inference in multimedia systems. The second edition enhances reference software, providing robust tools for handling complex neural networks in applications such as image and video processing.
Low Latency, Low Complexity LiDAR Coding: As industries like autonomous vehicles and smart cities expand, this standard addresses the need for efficient and real-time processing of LiDAR data. The MPEG community has developed compression techniques that maintain low latency and complexity, enabling faster decision-making for autonomous systems.
MPEG-DASH (6th Edition): The 6th edition of MPEG-DASH brings exciting improvements in adaptive streaming. Key updates include support for new CMCD parameters for better content management and a background mode that allows players to receive updates without disrupting media playback. These advancements significantly enhance streaming efficiency and flexibility.
Video Coding for Machines (VCM): A significant addition this year has been the introduction of Video Coding for Machines. This emerging standard focuses on machine vision applications, where efficient encoding and decoding are crucial for machine learning tasks such as object detection and recognition. This innovation caters to the increasing integration of machine-based analytics in multimedia systems.
Immersive Media and Volumetric Video: MPEG’s work on volumetric video coding and standards for immersive media continues to push the boundaries of AR/VR technologies. This ensures that immersive content can be delivered across various platforms with improved consistency and performance.

These highlights exemplify MPEG's commitment to fostering innovation through multimedia standards, shaping the future of digital content. On this World Standards Day, let’s celebrate the efforts that keep the digital ecosystem thriving!

ACM Mile-High Video Conference 2025: Call for Contributions

noreply@blogger.com (Unknown) — Fri, 27 Sep 2024 11:52:00 +0000

MHV 2025: ACM Mile-High Video Conference 2025
Call for Contributions
February 18-20, 2025, The Cable Center, Denver, Colorado
https://www.mile-high.video/

ACM Mile-High Video (MHV) is a flagship industry-oriented technical conference in the area of video technologies, which has been successfully running in Denver, Colorado, starting from 2016. ACM MHV 2025 welcomes contributions from both industry and academia to share real-world problems and solutions as well as novel approaches and innovations from content production to consumption. ACM MHV 2025 will provide a unique opportunity to view the interplay of the industry and academia in the area of video technologies.

ACM MHV contributions are solicited in, but not limited to, the following areas:

Content production, encoding, and packaging

Encoding for broadcast, mobile, and OTT (incl. using AI/ML in encoding),
New and emerging audio, image, and video codecs (incl. point cloud coding, light field coding, holography coding, etc.)
Edge, network, and cloud-based coding
Perceptually optimized objective quality metrics
Quality assessment models and tools, and user experience studies
Storage applications for video processing and streaming
Accessibility
HDR

Video workflows

Virtualized headends, cloud-based workflows for production and distribution
Redundancy and resilience in content origination
Ingest protocols
Ad insertion

Content delivery and security

Developments in transport protocols and new delivery paradigms
Protection for OTT distribution and tools against piracy
Analytics

Streaming technologies

Adaptive streaming and transcoding
Low latency
Player, playback, and QoE developments
Content discovery, promotion, and recommendation systems
Protocol and Web API improvements and innovations for streaming video

Industry trends

3D/XR video (NeRF, Gaussian splatting, etc.)
Scalable and multi-view video coding deployments
Video and audio coding for machines
Cloud gaming and gaming streaming
Provenance, content authentication, and deepfakes
Energy management in video compression and streaming
Edge computing tools and applications
Hardware for content encoding, storage, and distribution

Standards and interoperability

New and developing standards in the media and delivery space
Interoperability guidelines

Reasons for academics to submit to and attend the ACM Mile-High Video 2025 event:

Networking with Industry Leaders: Engage with top minds from both academia and industry, fostering collaborations and partnerships in cutting-edge video technology.
High Visibility: Present your research to a global audience, including key industry players and decision-makers.
Innovative Content: Showcase your work in emerging areas such as metaverse, AI-driven video technologies, and advanced codecs.
Research Impact: Influence the direction of next-gen video technology and its applications across industries.

Participating in ACM MHV 2025 is a prime opportunity to boost academic careers by connecting theoretical research with real-world innovations.

Persuasive reasons for industry professionals to submit and attend ACM Mile-High Video 2025:

Access to Cutting-Edge Research: Stay at the forefront of video technology with insights into innovations in streaming, codecs, and content delivery.
Collaborate with Academia: Engage with academic researchers to explore potential partnerships that can accelerate R&D and technological breakthroughs.
Showcase Industry Leadership: Present your company’s pioneering solutions, fostering brand recognition and thought leadership in the global video tech community.
Talent and Recruitment: Discover emerging talent and ideas that could impact your organization’s future.

Participating drives industry growth through shared knowledge and collaboration!

Prospective speakers are invited to submit an extended abstract (one page ~400 words + references) that will be peer-reviewed by the ACM MHV technical program committee (TPC) for relevance, timeliness, and technical correctness.

The authors of the accepted extended abstracts will be invited to optionally submit a full-length paper (up to six pages + references) for possible inclusion into the conference proceedings. These papers must be original work (i.e., not published previously in a journal or conference) and will also be peer-reviewed by the ACM MHV TPC.

Accepted extended abstracts and full-length papers will be presented at the ACM MHV conference, and, optionally, will be published at the conference proceedings within the ACM Digital Library.

All prospective ACM authors are subject to all ACM Publications Policies, including ACM's new Publications Policy on Research Involving Human Participants and Subjects.

How to Submit an Extended Abstract

Prospective authors are invited to submit one-page extended abstracts here

Important Dates

Extended abstract submission deadline: Oct. 31, 2024 AoE (firm deadline)
Notification of extended abstract acceptance: Nov. 27, 2024 AoE
Optional: Full-length paper submission deadline: Dec. 16, 2024 AoE
Notification of full-length paper acceptance: Jan. 22, 2025 AoE
Camera-ready submission (extended abstracts/full-length papers) deadline: Jan. 31, 2025 AoE

ACM MHV 2025 Program Chairs

Christian Timmerer (AAU; christian.timmerer AT aau.at)
Dan Grois (AnyAI; dgrois AT acm.org)
Gwendal Simon (Synamedia; gsimon AT synamedia.com)
Jill Boyce (Nokia; jill.boyce AT nokia.com)
Yuriy Reznik (Brightcove; yreznik AT brightcove.com)

MPEG news: a report from the 147th meeting

noreply@blogger.com (Unknown) — Mon, 16 Sep 2024 07:10:00 +0000

MPEG News Archive

The 147th MPEG meeting was held in Sapporo, Japan from 15-19 July 2024, and the official press release can be found here. It comprises the following highlights:

ISO Base Media File Format*: The 8th edition was promoted to Final Draft International Standard, supporting seamless media presentation for DASH and CMAF.
Syntactic Description Language: Finalized as an independent standard for MPEG-4 syntax.
Low-Overhead Image File Format*: First milestone achieved for small image handling improvements.
Neural Network Compression*: Second edition for conformance and reference software promoted.
Internet of Media Things (IoMT): Progress made on reference software for distributed media tasks.

* … covered in this blog post and expanded with possible research aspects.

8th edition of ISO Base Media File Format

The ever-growing expansion of the ISO/IEC 14496-12 ISO base media file format (ISOBMFF) application area has continuously brought new technologies to the standards. During the last couple of years, MPEG Systems (WG 3) has received new technologies on ISOBMFF for more seamless support of ISO/IEC 23009 Dynamic Adaptive Streaming over HTTP (DASH) and ISO/IEC 23000-19 Common Media Application Format (CMAF) leading to the development of the 8th edition of ISO/IEC14496-12.

The new edition of the standard includes new technologies to explicitly indicate the set of tracks representing various versions of the media presentation of a single media for seamless switching and continuous presentation. Such technologies will enable more efficient processing of the ISOBMFF formatted files for DASH manifest or CMAF Fragments.

Research aspects: The central research aspect of the 8th edition of ISOBMFF, which “will enable more efficient processing,” will undoubtedly be its evaluation compared to the state-of-the-art. Standards typically define a format, but how to use it is left open to implementers. Therefore, the implementation is a crucial aspect and will allow for a comparison of performance. One such implementation of ISOBMFF is GPAC, which most likely will be among the first to implement these new features.

Low-Overhead Image File Format

ISO/IEC 23008-12 image format specification defines generic structures for storing image items and sequences based on ISO/IEC 14496-12 ISO base media file format (ISOBMFF). As it allows the use of various high-performance video compression standards for a single image or a series of images, it has been adopted by the market quickly. However, it was challenging to use it for very small-sized images such as icons or emojis. While the initial design of the standard was versatile and useful for a wide range of applications, the size of headers becomes an overhead for applications with tiny images. Thus, Amendment 3 of ISO/IEC 23008-12 low-overhead image file format aims to address this use case by adding a new compact box for storing metadata instead of the ‘Meta’ box to lower the size of the overhead.

Research aspects: The issue regarding header sizes of ISOBMFF for small files or low bitrate (in the case of video streaming) was known for some time. Therefore, amendments in these directions are appreciated while further performance evaluations are needed to confirm design choices made at this initial step of standardization.

Neural Network Compression

An increasing number of artificial intelligence applications based on artificial neural networks, such as edge-based multimedia content processing, content-adaptive video post-processing filters, or federated training, need to exchange updates of neural networks (e.g., after training on additional data or fine-tuning to specific content). For this purpose, MPEG developed a second edition of the standard for coding of neural networks for multimedia content description and analysis (NNC, ISO/IEC 15938-17, published in 2024), adding syntax for differential coding of neural network parameters as well as new coding tools. Trained models can be compressed to at least 10-20% for several architectures, even below 3%, of their original size without performance loss. Higher compression rates are possible at moderate performance degradation. In a distributed training scenario, a model update after a training iteration can be represented at 1% or less of the base model size on average without sacrificing the classification performance of the neural network.

In order to facilitate the implementation of the standard, the accompanying standard ISO/IEC 15938-18 has been updated to cover the second edition of ISO/IEC 15938-17. This standard provides a reference software for encoding and decoding NNC bitstreams, as well as a set of conformance guidelines and reference bitstreams for testing of decoder implementations. The software covers the functionalities of both editions of the standard, and can be configured to test different combinations of coding tools specified by the standard.

Research aspects: The reference software for NNC, together with the reference software for audio/video codecs, are vital tools for building complex multimedia systems and its (baseline) evaluation with respect to compression efficiency only (not speed). This is because reference software is usually designed for functionality (i.e., compression in this case) and not performance.

The 148th MPEG meeting will be held in Kemer, Türkiye, from November 04-08, 2024. Click here for more information about MPEG meetings and their developments.

University assistant predoctoral (all genders welcome) (in German: Universitätsassistent:in)

noreply@blogger.com (Unknown) — Wed, 07 Aug 2024 09:20:00 +0000

The University of Klagenfurt, with approximately 1,500 employees and over 12,000 students, is located in the Alps-Adriatic region and consistently achieves excellent placements in rankings. The motto “per aspera ad astra” underscores our firm commitment to the pursuit of excellence in all research, teaching, and university management activities. The principles of equality, diversity, health, sustainability, and compatibility of work and family life serve as the foundation for our work at the university.

The University of Klagenfurt is pleased to announce the following open position at the Department of Information Technology at the Faculty of Technical Sciences with an expected starting date of November 4, 2024:

University assistant predoctoral (all genders welcome) (in German: Universitätsassistent:in)

within the Ada Lovelace Programme (project title: Streaming of Holographic Content and its Impact on the Quality of Experience).

Level of employment: 100 % (40 hours/week)
Minimum salary: € 50,103.20 per annum (gross); Classification according to collective agreement: B1
Contract duration: 4 years
Application deadline: by September 11, 2024
Reference code: 348/24

Tasks and responsibilities:

Autonomous scientific work, including the publication of research articles in the fields of coding and streaming of holographic content, Quality of Experience (QoE), and behavioural sciences
Conducting independent scientific research with the aim of submitting a dissertation and acquiring a doctoral degree in technical sciences
Teaching exercises and lab courses (e.g., in the computer science Bachelor’s or/and Master’s programme)
Participating in research projects of the department, especially within the Ada Lovelace Programme (Streaming of Holographic Content and its Impact on the Quality of Experience)
Mentoring students
Assisting in public relations activities, science to public communication, and extra-curricular events of the department and the faculty

Prerequisites for the appointment:

Completed Diploma or Master’s degree from a recognized university in the field of computer science, information and communications engineering, electrical engineering, or related fields. The completion of this degree must be fulfilled no later than two weeks before the starting date; hence, the last possible deadline for meeting this requirement is October 20, 2024
Strong background in one or more of the following fields: multimedia systems (i.e., video/holographic content coding/streaming, Quality of Experience) and empirical research methods (i.e., statistical methods, interdisciplinary research with behavioural sciences)
Fluent in written and spoken English
Programming experience in multimedia systems

Additional desired qualifications:

Experience with scientific publications or presentations
Experience in interdisciplinary research projects, ideally in the behavioural sciences, as the project involves empirical research
Excellent ability to work with teams
Scientific curiosity and enthusiasm for research in multimedia systems and empirical research

The doctoral student will be co-supervised by Christian Timmerer, Heather Foran, and Hadi Amirpour.

Our offer:

This position serves the purposes of the vocational and scientific education of graduates of Master’s or Diploma degree programmes and sets the goal of completing a Doctoral degree / a Ph.D. in Technical Sciences. Therefore, applications by persons who have already completed a subject-specific doctoral degree or a subject-relevant Ph.D. program cannot be considered.

The employment contract is concluded for the position of university assistant (predoctoral) and stipulates a starting salary of € 3,578.80 gross per month (14 times a year; previous experience deemed relevant to the job can be recognized in accordance with the collective agreement).

The University of Klagenfurt also offers:

Personal and professional advanced training courses, management, and career coaching
Numerous attractive additional benefits, see also https://jobs.aau.at/en/the-university-as-employer/
Diversity- and family-friendly university culture
The opportunity to live and work in the attractive Alps-Adriatic region with a wide range of leisure activities in the spheres of culture, nature, and sports

The application:

If you are interested in this position, please apply in English by providing the following documents:

Letter of application/cover letter including motivation statement for the given position
Curriculum vitae (with clear information about the degrees, including date/place/grade, the experience acquired, the thesis title, the list of publications (if any), and any other relevant information)
Copy of the degree certificates and transcripts of the courses
Any certificates that can prove the fulfilment of the required and additional qualifications listed above (e.g., the submission of the final thesis if required by the degree programme, copy of publications, programming skills certificates, language skills certificates, etc.)
Final thesis or other study-related written work (like seminar reports) or excerpts thereof
If an applicant has not received the Diploma or Master’s degree by the application deadline, the applicant should provide a declaration, written either by a supervisor or by the candidate themselves, on the feasibility of finishing the Diploma or Master’s degree by October 30, 2024 at the latest.

To apply, please select the position with the reference code 348/24 in the category “Scientific Staff” using the link “Apply for this position” in the job portal at jobs.aau.at/en/.

Candidates must furnish proof that they meet the required qualifications by October 20, 2024 at the latest.

For further information on this specific vacancy, please contact Univ.-Prof. DI Dr. Christian Timmerer (christian.timmerer@aau.at). General information about the university as an employer can be found at https://jobs.aau.at/en/the-university-as-employer/. At the University of Klagenfurt, recruitment and staff matters are accompanied not only by the authority responsible for the recruitment procedure but also by the Equal Opportunities Working Group and, if necessary, by the Representative for Disabled Persons.

The University of Klagenfurt aims to increase the proportion of women and, therefore, invites explicitly qualified women to apply for the position. Where the qualification is equivalent, women will be given preferential consideration.

People with disabilities or chronic diseases, who fulfill the requirements, are particularly encouraged to apply.

Travel and accommodation costs incurred during the application process will not be refunded. Translations into other languages shall serve informational purposes only. Solely the version advertised in the University Bulletin (Mitteilungsblatt) shall be legally binding.

Successful 5-year Evaluation of Christian Doppler Laboratory ATHENA

noreply@blogger.com (Unknown) — Mon, 15 Jul 2024 06:41:00 +0000

The Christian Doppler (CD) Laboratory ATHENA was established in October 2019 to tackle current and future research and deployment challenges of HTTP Adaptive Streaming (HAS) and emerging streaming methods. The goal of CD laboratories is to conduct application-oriented basic research, promote collaboration between universities and companies, and facilitate technology transfer. They are funded through a public-private partnership between companies and the Christian Doppler Research Association, which is funded by the Ministry for Digital and Economic Affairs and the National Foundation for Research, Technology, and Development (Nationalstiftung für Forschung, Technologie und Entwicklung (FTE)). ATHENA is supported by Bitmovin as a company partner.

The CD laboratories have a duration of seven years and undergo rigorous scientific review after two and five years. This spring, the CD lab ATHENA completed its 5-year evaluation, and we have just received official notification from the CDG that we have successfully passed the review. Consequently, it is time to briefly outline the main achievements during this second phase (i.e., years 2 to 5) of the CD lab ATHENA.

Before exploring the achievements, it’s important to highlight the ongoing relevance of research in video streaming, given its dominance in today’s Internet usage. The January 2024 Sandvine Internet Phenomena report revealed that video streaming accounts for 68% of fixed/wired Internet traffic and 64% for mobile Internet traffic. Specifically, Video on Demand (VoD) represents 54% of fixed/wired and 57% of mobile traffic, while live streaming contributes to 14% of fixed/wired and 7% of mobile traffic. The major services in this domain include YouTube and Netflix, each commanding more than 10% of the overall Internet traffic, with TikTok, Amazon Prime, and Disney+ also playing significant roles.

ATHENA is structured into four work packages, each with distinct objectives as detailed below:

Content provisioning: Primarily involves video encoding for HAS, quality-aware encoding, learning-based encoding, and multi-codec HAS.
Content delivery: Addresses HAS issues by utilizing edge computing, exchanging information between CDN/SDN and clients, providing network assistance for clients, and evaluating corresponding utilities.
Content consumption: Focuses on bitrate adaptation schemes, playback improvements, context and user awareness, and studies on Quality of Experience (QoE).
End-to-end aspects: Offers a comprehensive view of application and transport layer enhancements, Quality of Experience (QoE) models, low-latency HAS, and learning-based HAS.

During the 2nd phase of ATHENA’s work, we achieved significant results, including publications in respected academic journals and conferences. Specifically, our publications were featured in key multimedia, signal processing, computer networks & wireless communication, and computing systems venues, as categorized by Google Scholar under engineering and computer science. Some of the notable publications include IEEE Communications Surveys & Tutorials (impact factor: 35.6), IEEE Transactions on Image Processing (10.6), IEEE Internet of Things Journal (10.6), IEEE Transactions on Circuits and Systems for Video Technology (8.4), and IEEE Transactions on Multimedia (7.3).

Furthermore, we focused on technology transfer by submitting 16 invention disclosures, resulting in 13 patent applications (including provisionals). Collaborating with our company partner, we obtained 6 granted patents. Additionally, we’re pleased to report on the progress of our spin-off projects, as well as the funding secured for two FFG-funded projects named APOLLO and GAIA, and an EU Horizon Europe-funded innovation action called SPIRIT.

The ATHENA team was also active in organizing scientific events such as workshops, special sessions, and special issues at IEEE ICME, ACM MM, ACM MMSys, ACM CoNEXT, IEEE ICIP, PCS, and IEEE Network. We also contributed to reproducibility in research through open source tools (e.g., Video Complexity Analyzer and LLL-CAdViSE) and datasets (e.g., Video Complexity Dataset and Multi-Codec Ultra High Definition 8K MPEG-DASH Dataset) among others.

We also note our contributions to the applications of AI in video coding & streaming, for example in video coding and video streaming as follows:

Fast Multi-Rate Encoding with Machine Learning (using Convolutional Neural Networks (CNNs))
LiDeR: Lightweight video Super Resolution for mobile devices (using Deep Neural Networks (DNNs))
Blind Visual Quality Assessment Using Vision Transformers
Video Complexity Analysis (VCA) and optimizations for per-title encoding (using Linear Regression, Random Forest, and XGBoost models)
DeepStream: Video streaming enhancements using compressed deep neural networks (using Deep Neural Networks (DNNs))
ECAS-ML: Edge-assisted adaptive bitrate switching (using Long Short-Term Memory (LSTM))
Quality Optimization of Live Streaming Services over HTTP with Reinforcement Learning (RL)

A major outcome of the second phase is the successful defense of the inaugural cohort of PhD students:

Dr. Alireza Erfanian: “Optimizing QoE and Latency of Video Streaming using Edge Computing and In-Network Intelligence”, May 25, 2023
Dr. Ekrem Çetinkaya: “Video Coding Enhancements for HTTP Adaptive Streaming using Machine Learning”, June 7, 2023
Dr. Minh Nguyen: “Policy-driven Dynamic HTTP Adaptive Streaming Player Environment”, June 30, 2023
Dr. Jesús Aguilar Armijo: “Multi-access Edge Computing for Adaptive Video Streaming”, July 10, 2023
Dr. Reza Farahani: “Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, SDN, and MEC”, August 22, 2023
Dr. Vignesh V Menon: “Content-adaptive Video Coding for HTTP Adaptive Streaming”, January 15, 2024
Dr. Babak Taraghi, “End-to-end Quality of Experience Evaluation for HTTP Adaptive Streaming”, July 10, 2024

Two postdoctoral scholars have reached a significant milestone on their path toward habilitation

Dr. Hadi Amirpour, “Video Coding for Efficient HTTP Adaptive Streaming”, February 8, 2024
Dr. Farzad Tashtarian, “How to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions”, February 27, 2023 & “End-to-End Adaptive Video Streaming Optimization”, June 26, 2024

During the second phase, each work package produced excellent publications in their domain, briefly highlighted in the following. Content provisioning (WP-1) focuses mainly on video coding for HAS (43 papers) and immersive media coding for streaming (4 papers). The former can be further subdivided into the following topic areas:

Video complexity: spatial and temporal feature extraction (4 papers)
Compression efficiency improvement of individual representations (1 paper)
Encoding parameter prediction for HAS (9 papers)
Efficient bitrate ladder construction (4 papers)
Fast multi-rate encoding (3 papers)
Data security and data hiding (7 papers)
Energy-efficient video encoding for HAS (4 papers)
Advancing video quality evaluation (7 papers)
Datasets (4 papers)

Content delivery (WP-2) dealt with SDN/CDN assistance for HAS, edge computing support for HAS, and network-embedded media streaming support, resulting in 21 papers. Content consumption (WP-3) worked on QoE enhancement mechanisms at client-side and QoE- and energy-aware content consumption (11 papers). Finally, end-to-end Aspects (WP-4) produced 15 papers in the area of end-to-end QoE improvement in multimedia video streaming. We reported 94 papers published/accepted for the ATHENA 5-year evaluation.

In this context, it is also important to highlight the collaboration within ATHENA, which has resulted in joint publications across various work packages (WPs) and with other ITEC members. For example, collaborations with Prof. Schöffmann (FWF-funded project OVID), FFG-funded projects APOLLO/GAIA, and EU-funded project SPIRIT. In addition, we would like to acknowledge our international collaborators, such as Prof. Hongjie He from Southwest Jiaotong University, Prof. Patrick Le Callet from the University of Nantes, Prof. Wassim Hamidouche from the Technology Innovation Institute (UAE), Dr. Sergey Gorinsky from IMDEA, Dr. Abdelhak Bentaleb from Concordia University, Dr. Raimund Schatz from AIT, and Prof. Pablo Cesar from CWI. We are also pleased to report the successful technology transfers to Bitmovin, particularly CAdViSE (WP-4) and WISH ABR (WP-3). Regular “Fun with ATHENA” meetups and Break-out Groups are utilized for in-depth discussions about innovations and potential technology transfers.

Over the next two years, the ATHENA project will prioritize the development of deep neural network/AI-based image and video coding within the context of HAS. This includes energy- and cost-aware video coding for HAS, immersive video coding such as volumetric video and holography, as well as Quality of Experience (QoE) and energy-aware content consumption for HAS (including energy-efficient, AI-based live video streaming) and generative AI for HAS.

Thanks to all current and former ATHENA team members: Samira Afzal, Hadi Amirpour, Jesús Aguilar Armijo, Emanuele Artioli, Christian Bauer, Alexis Boniface, Ekrem Çetinkaya, Reza Ebrahimi, Alireza Erfanian, Reza Farahani, Mohammad Ghanbari (late), Milad Ghanbari, Mohammad Ghasempour, Selina Zoë Haack, Hermann Hellwagner, Manuel Hoi, Andreas Kogler, Gregor Lammer, Armin Lachini, David Langmeier, Sandro Linder, Daniele Lorenzi, Vignesh V Menon, Minh Nguyen, Engin Orhan, Lingfeng Qu, Jameson Steiner, Nina Stiller, Babak Taraghi, Farzad Tashtarian, Yuan Yuan, and Yiying Wei. Finally, thanks to ITEC support staff Martina Steinbacher, Nina Stiller, Margit Letter, Marion Taschwer, and Rudolf Messner.

We also would like to thank the Christian Doppler Research Association for continuous support, organizing the review, and the reviewer for constructive feedback!

HTTP Adaptive Streaming – Quo Vadis? (2024)

noreply@blogger.com (Unknown) — Mon, 01 Jul 2024 15:05:00 +0000

Telecom Seminar Series at TII, Jun 27, 2024, 04:00 PM Dubai

Abstract: Video traffic on the Internet is constantly growing; networked multimedia applications consume a predominant share of the available Internet bandwidth. A major technical breakthrough and enabler in multimedia systems research and of industrial networked multimedia services certainly was the HTTP Adaptive Streaming (HAS) technique. This resulted in the standardization of MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH) which, together with HTTP Live Streaming (HLS), is widely used for multimedia delivery in today’s networks. Existing challenges in multimedia systems research deal with the trade-off between (i) the ever-increasing content complexity, (ii) various requirements with respect to time (most importantly, latency), and (iii) quality of experience (QoE). Optimizing towards one aspect usually negatively impacts at least one of the other two aspects if not both. This situation sets the stage for our research work in the ATHENA Christian Doppler (CD) Laboratory (Adaptive Streaming over HTTP and Emerging Networked Multimedia Services; https://athena.itec.aau.at/), jointly funded by public sources and industry. In this talk, we will present selected novel approaches and research results of the first year of the ATHENA CD Lab’s operation. We will highlight HAS-related research on (i) multimedia content provisioning (machine learning for video encoding); (ii) multimedia content delivery (support of edge processing and virtualized network functions for video networking); (iii) multimedia content consumption and end-to-end aspects (player-triggered segment retransmissions to improve video playout quality); and (iv) novel QoE investigations (adaptive point cloud streaming). We will also put the work into the context of international multimedia systems research.

HTTP Adaptive Streaming – Quo Vadis (2024) from Alpen-Adria-Universität

Video Streaming: Then, Now, Future

noreply@blogger.com (Unknown) — Thu, 06 Jun 2024 16:35:00 +0000

I'm happy to share my slides from my public/inaugural lecture at the University of Klagenfurt on June 5, 2022.

Title: "Video Streaming: Then, Now, Future"
June 5, 2024, 17:00, University of Klagenfurt, Hörsaal 2

In my public lecture, I provide insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. I'm also presenting provocative contributions of his own that have significantly influenced the industry. I conclude by looking at future challenges and invite the audience to join in a discussion (e.g., in the comments below).

Video Streaming: Then, Now, and in the Future from Alpen-Adria-Universität

MPEG news: a report from the 146th meeting

noreply@blogger.com (Unknown) — Sat, 18 May 2024 19:45:00 +0000

MPEG News Archive

The 146th MPEG meeting was held in Rennes, France from 22-26 April 2024, and the official press release can be found here. It comprises the following highlights:

AI-based Point Cloud Coding*: Call for proposals focusing on AI-driven point cloud encoding for applications such as immersive experiences and autonomous driving.
Object Wave Compression*: Call for interest in object wave compression for enhancing computer holography transmission.
Open Font Format: Committee Draft of the fifth edition, overcoming previous limitations like the 64K glyph encoding constraint.
Scene Description: Ratified second edition, integrating immersive media objects and extending support for various data types.
MPEG Immersive Video (MIV): New features in the second edition, enhancing the compression of immersive video content.
Video Coding Standards: New editions of AVC, HEVC, and Video CICP, incorporating additional SEI messages and extended multiview profiles.
Machine-Optimized Video Compression*: Advancement in optimizing video encoders for machine analysis.
MPEG-I Immersive Audio*: Reached Committee Draft stage, supporting high-quality, real-time interactive audio rendering for VR/AR/MR.
Video-based Dynamic Mesh Coding (V-DMC)*: Committee Draft status for efficiently storing and transmitting dynamic 3D content.
LiDAR Coding*: Enhanced efficiency and responsiveness in LiDAR data processing with the new standard reaching Committee Draft status.

* ... covered in this column.

AI-based Point Cloud Coding

MPEG issued a Call for Proposals (CfP) on AI-based point cloud coding technologies as a result from ongoing explorations regarding use cases, requirements, and the capabilities of AI-driven point cloud encoding, particularly for dynamic point clouds.

With recent significant progress in AI-based point cloud compression technologies, MPEG is keen on studying and adopting AI methodologies. MPEG is specifically looking for learning-based codecs capable of handling a broad spectrum of dynamic point clouds, which are crucial for applications ranging from immersive experiences to autonomous driving and navigation. As the field evolves rapidly, MPEG expects to receive multiple innovative proposals. These may include a unified codec, capable of addressing multiple types of point clouds, or specialized codecs tailored to meet specific requirements, contingent upon demonstrating clear advantages. MPEG has therefore publicly called for submissions of AI-based point cloud codecs, aimed at deepening the understanding of the various options available and their respective impacts. Submissions that meet the requirements outlined in the call will be invited to provide source code for further analysis, potentially laying the groundwork for a new standard in AI-based point cloud coding. MPEG welcomes all relevant contributions and looks forward to evaluating the responses.

Research aspects: In-depth analysis of algorithms, techniques, and methodologies, including a comparative study of various AI-driven point cloud compression techniques to identify the most effective approaches. Other aspects include creating or improving learning-based codecs that can handle dynamic point clouds as well as metrics for evaluating the performance of these codecs in terms of compression efficiency, reconstruction quality, computational complexity, and scalability. Finally, the assessment of how improved point cloud compression can enhance user experiences would be worthwhile to consider here also.

Object Wave Compression

A Call for Interest (CfI) in object wave compression has been issued by MPEG. Computer holography, a 3D display technology, utilizes a digital fringe pattern called a computer-generated hologram (CGH) to reconstruct 3D images from input 3D models. Holographic near-eye displays (HNEDs) reduce the need for extensive pixel counts due to their wearable design, positioning the display near the eye. This positions HNEDs as frontrunners for the early commercialization of computer holography, with significant research underway for product development. Innovative approaches facilitate the transmission of object wave data, crucial for CGH calculations, over networks. Object wave transmission offers several advantages, including independent treatment from playback device optics, lower computational complexity, and compatibility with video coding technology. These advancements open doors for diverse applications, ranging from entertainment experiences to real- time two-way spatial transmissions, revolutionizing fields such as remote surgery and virtual collaboration. As MPEG explores object wave compression for computer holography transmission, a Call for Interest seeks contributions to address market needs in this field.

Research aspects: Apart from compression efficiency, lower computation complexity, and compatibility with video coding technology, there is a range of research aspects, including the design, implementation, and evaluation of coding algorithms within the scope of this CfI. The QoE of computer-generated holograms (CGHs) together with holographic near-eye displays (HNEDs) is yet another dimension to be explored.

Machine-Optimized Video Compression

MPEG started working on a technical report regarding to the "Optimization of Encoders and Receiving Systems for Machine Analysis of Coded Video Content". In recent years, the efficacy of machine learning-based algorithms in video content analysis has steadily improved. However, an encoder designed for human consumption does not always produce compressed video conducive to effective machine analysis. This challenge lies not in the compression standard but in optimizing the encoder or receiving system. The forthcoming technical report addresses this gap by showcasing technologies and methods that optimize encoders or receiving systems to enhance machine analysis performance.

Research aspects: Video (and audio) coding for machines has been recently addressed by MPEG Video and Audio working groups, respectively. MPEG Joint Video Experts Team with ITU-T SG16, also known as JVET, joined this space with a technical report, but research aspects remain unchanged, i.e., coding efficiency, metrics, and quality aspects for machine analysis of compressed/coded video content.

MPEG-I Immersive Audio

MPEG Audio Coding enters the "immersive space" with MPEG-I immersive audio and its corresponding reference software. The MPEG-I immersive audio standard sets a new benchmark for compact and lifelike audio representation in virtual and physical spaces, catering to Virtual, Augmented, and Mixed Reality (VR/AR/MR) applications. By enabling high-quality, real-time interactive rendering of audio content with six degrees of freedom (6DoF), users can experience immersion, freely exploring 3D environments while enjoying dynamic audio. Designed in accordance with MPEG's rigorous standards, MPEG-I immersive audio ensures efficient distribution across bandwidth-constrained networks without compromising on quality. Unlike proprietary frameworks, this standard prioritizes interoperability, stability, and versatility, supporting both streaming and downloadable content while seamlessly integrating with MPEG-H 3D audio compression. MPEG-I's comprehensive modeling of real-world acoustic effects, including sound source properties and environmental characteristics, guarantees an authentic auditory experience. Moreover, its efficient rendering algorithms balance computational complexity with accuracy, empowering users to finely tune scene characteristics for desired outcomes.

Research aspects: Evaluating QoE of MPEG-I immersive audio-enabled environments as well as the efficient audio distribution across bandwidth-constrained networks without compromising on audio quality are two important research aspects to be addressed by the research community.

Video-based Dynamic Mesh Coding (V-DMC)

Video-based Dynamic Mesh Compression (V-DMC) represents a significant advancement in 3D content compression, catering to the ever-increasing complexity of dynamic meshes used across various applications, including real-time communications, storage, free-viewpoint video, augmented reality (AR), and virtual reality (VR). The standard addresses the challenges associated with dynamic meshes that exhibit time-varying connectivity and attribute maps, which were not sufficiently supported by previous standards. Video-based Dynamic Mesh Compression promises to revolutionize how dynamic 3D content is stored and transmitted, allowing more efficient and realistic interactions with 3D content globally.

Research aspects: V-DMC aims to allow "more efficient and realistic interactions with 3D content", which are subject to research, i.e., compression efficiency vs. QoE in constrained networked environments.

Low Latency, Low Complexity LiDAR Coding

Low Latency, Low Complexity LiDAR Coding underscores MPEG's commitment to advancing coding technologies required by modern LiDAR applications across diverse sectors. The new standard addresses critical needs in the processing and compression of LiDAR-acquired point clouds, which are integral to applications ranging from automated driving to smart city management. It provides an optimized solution for scenarios requiring high efficiency in both compression and real-time delivery, responding to the increasingly complex demands of LiDAR data handling. LiDAR technology has become essential for various applications that require detailed environmental scanning, from autonomous vehicles navigating roads to robots mapping indoor spaces. The Low Latency, Low Complexity LiDAR Coding standard will facilitate a new level of efficiency and responsiveness in LiDAR data processing, which is critical for the real-time decision-making capabilities needed in these applications. This standard builds on comprehensive analysis and industry feedback to address specific challenges such as noise reduction, temporal data redundancy, and the need for region-based quality of compression. The standard also emphasizes the importance of low latency coding to support real-time applications, essential for operational safety and efficiency in dynamic environments.

Research aspects: This standard effectively tackles the challenge of balancing high compression efficiency with real-time capabilities, addressing these often conflicting goals. Researchers may carefully consider these aspects and make meaningful contributions.

The 147th MPEG meeting will be held in Sapporo, Japan, from July 15-19, 2024. Click here for more information about MPEG meetings and their developments.

MPEG news: a report from the 145th meeting

noreply@blogger.com (Unknown) — Fri, 17 May 2024 19:26:00 +0000

MPEG News Archive

The 145th MPEG meeting was held online from 22-26 January 2024, and the official press release can be found here. It comprises the following highlights:

Latest Edition of the High Efficiency Image Format Standard Unveils Cutting-Edge Features for Enhanced Image Decoding and Annotation
MPEG Systems finalizes Standards supporting Interoperability Testing
MPEG finalizes the Third Edition of MPEG-D Dynamic Range Control
MPEG finalizes the Second Edition of MPEG-4 Audio Conformance
MPEG Genomic Coding extended to support Transport and File Format for Genomic Annotations
MPEG White Paper: Neural Network Coding (NNC) – Efficient Storage and Inference of Neural Networks for Multimedia Applications

This column will focus on the High Efficiency Image Format (HEIF) and interoperability testing. As usual, a brief update on MPEG-DASH et al. will be provided.

High Efficiency Image Format (HEIF)

The High Efficiency Image Format (HEIF) is a widely adopted standard in the imaging industry that continues to grow in popularity. At the 145th MPEG meeting, MPEG Systems (WG 3) ratified its third edition, which introduces exciting new features, such as progressive decoding capabilities that enhance image quality through a sequential, single-decoder instance process. With this enhancement, users can decode bitstreams in successive steps, with each phase delivering perceptible improvements in image quality compared to the preceding step. Additionally, the new edition introduces a sophisticated data structure that describes the spatial configuration of the camera and outlines the unique characteristics responsible for generating the image content. The update also includes innovative tools for annotating specific areas in diverse shapes, adding a layer of creativity and customization to image content manipulation. These annotation features cater to the diverse needs of users across various industries.

Research aspects: Progressive coding has been a part of modern image coding formats for some time now. However, the inclusion of supplementary metadata provides an opportunity to explore new use cases that can benefit both user experience (UX) and quality of experience (QoE) in academic settings.

Interoperability Testing

MPEG standards typically comprise format definitions (or specifications) to enable interoperability among products and services from different vendors. Interestingly, MPEG goes beyond these format specifications and provides reference software and conformance bitstreams, allowing conformance testing.

At the 145th MPEG meeting, MPEG Systems (WG 3) finalized two standards comprising conformance and reference software by promoting it to the Final Draft International Standard (FDIS), the final stage of standards development. The finalized standards, ISO/IEC 23090-24 and ISO/IEC 23090-25, showcase the pinnacle of conformance and reference software for scene description and visual volumetric video-based coding data, respectively.

ISO/IEC 23090-24 focuses on conformance and reference software for scene description, providing a comprehensive reference implementation and bitstream tailored for conformance testing related to ISO/IEC 23090-14, scene description. This standard opens new avenues for advancements in scene depiction technologies, setting a new standard for conformance and software reference in this domain.

Similarly, ISO/IEC 23090-25 targets conformance and reference software for the carriage of visual volumetric video-based coding data. With a dedicated reference implementation and bitstream, this standard is poised to elevate the conformance testing standards for ISO/IEC 23090-10, the carriage of visual volumetric video-based coding data. The introduction of this standard is expected to have a transformative impact on the visualization of volumetric video data.

At the same 145th MPEG meeting, MPEG Audio Coding (WG6) celebrated the completion of the second edition of ISO/IEC 14496-26, audio conformance, elevating it to the Final Draft International Standard (FDIS) stage. This significant update incorporates seven corrigenda and five amendments into the initial edition, originally published in 2010.

ISO/IEC 14496-26 serves as a pivotal standard, providing a framework for designing tests to ensure the compliance of compressed data and decoders with the requirements outlined in ISO/IEC 14496-3 (MPEG-4 Audio). The second edition reflects an evolution of the original, addressing key updates and enhancements through diligent amendments and corrigenda. This latest edition, now at the FDIS stage, marks a notable stride in MPEG Audio Coding's commitment to refining audio conformance standards and ensuring the seamless integration of compressed data within the MPEG-4 Audio framework.

These standards will be made freely accessible for download on the official ISO website, ensuring widespread availability for industry professionals, researchers, and enthusiasts alike.

Research aspects: Reference software and conformance bitstreams often serve as the basis for further research (and development) activities and, thus, are highly appreciated. For example, reference software of video coding formats (e.g., HM for HEVC, VM for VVC) can be used as a baseline when improving coding efficiency or other aspects of the coding format.

MPEG-DASH Updates

The current status of MPEG-DASH is shown in the figure below.

MPEG-DASH Status, January 2024.

The following most notable aspects have been discussed at the 145th MPEG meeting and adopted into ISO/IEC 23009-1, which will eventually become the 6th edition of the MPEG-DASH standard:

It is now possible to pass CMCD parameters sid and cid via the MPD URL.
Segment duration patterns can be signaled using SegmentTimeline.
Definition of a background mode of operation, which allows a DASH player to receive MPD updates and listen to events without possibly decrypting or rendering any media.

Additionally, the technologies under consideration (TuC) document has been updated with means to signal maximum segment rate, extend copyright license signaling, and improve haptics signaling in DASH. Finally, REAP is progressing towards FDIS but not yet there and most details will be discussed in the upcoming AhG period.

The 146th MPEG meeting will be held in Rennes, France, from April 22-26, 2024. Click here for more information about MPEG meetings and their developments.

Assistant Professor (postdoc) with QA option (tenure track) (all genders welcome)

noreply@blogger.com (Unknown) — Thu, 16 May 2024 20:41:00 +0000

Department of Information Technology

Scientific Staff | Full time

Application deadline: 12 June 2024

Reference code: 673/23 [URL]

The University of Klagenfurt, with approximately 1,500 employees and over 12,000 students, is located in the Alps-Adriatic region and consistently achieves excellent placements in rankings. The motto “per aspera ad astra” underscores our firm commitment to the pursuit of excellence in all activities in research, teaching, and university management. The principles of equality, diversity, health, sustainability, and compatibility of work and family life serve as the foundation for our work at the university.

Assistant Professor (postdoc) with QA option (tenure track) (all genders welcome)

Level of employment: 100 % (40 hours/week)

Minimum salary: € 66,532.20 per annum (gross), Classification according to collective agreement: B1 lit.b

Limited to: 6 years (with the option of transitioning to a permanent contract)

Application deadline: 12 June 2024

Reference code: 673/23

Area of responsibility

Independent research in computer science and communication technologies with the aim of habilitation
Independent delivery of courses in English and German using established and innovative methods
Participation in the research and teaching projects run by the organisational unit
Acquisition and management of third-party funded projects
Supervision of students at Bachelor, Master, and doctoral levels
Participation in organisational and administrative tasks and in quality assurance measures
Contribution to expanding the international scientific and cultural contacts of the organisational unit
Participation in public relations activities including third mission

Requirements

Doctoral degree in the field of computer science, information and communications engineering, electrical engineering or related fields completed at a domestic or foreign higher education institution
Relevant and good publication record in the field of multimedia systems
A strong background in one or both fields

(Distributed) multimedia systems, preferably covering video in the context of video coding, communication, streaming, and quality of experience (QoE);
Machine learning, preferably in the context of (distributed) multimedia systems or/and computer vision

Very good scientific communication and dissemination skills (scientific writing and oral presentations)
Excellent programming skills in multimedia systems or/and machine learning
Excellent spoken and written English skills

Desired skills

Experience in the acquisition and running of third-party funded projects and readiness to play an active role in third-party funded projects and their acquisition
Didactic competence and proven successful teaching experience
Willingness to actively participate in research, teaching, and administration
Scientific curiosity and enthusiasm for imparting knowledge
Gender mainstreaming and diversity management skills
Leadership and teamwork skills
Good spoken and written German skills

Additional information

Our offer:

This tenure track position includes the option of negotiating a qualification agreement in accordance with Section 27 of the collective agreement for university staff for the areas of research, independent teaching, management and administrative tasks, and experience gained externally (QA). The employment contract is concluded for the position as Assistant Professor (postdoc) with QA option and stipulates a starting salary of € 4,752.30 gross per month (14 times a year; previous experience deemed relevant to the job can be recognised in accordance with the collective agreement). Upon entering into the qualification agreement, the position shall be classified as an Assistant Professorship with a minimum gross salary of € 5,595.60 per month. Upon fulfilling the stipulations of the qualification agreement, the post-holder shall be promoted to tenured Associate Professor with a minimum gross salary of € 6,055.70 per month.

The University of Klagenfurt also offers:

Personal and professional advanced training courses, management, and career coaching
Numerous attractive additional benefits, see also https://jobs.aau.at/en/the-university-as-employer/
Diversity- and family-friendly university culture
The opportunity to live and work in the attractive Alps-Adriatic region with a wide range of leisure activities in the spheres of culture, nature, and sports

The application:

If you are interested in this position, please apply in German or English, providing a convincing application including the following:

Letter of application, including – but not limited to – motivation as well as a concise research and teaching statement, respectively
Curriculum vitae, including publication and lecture lists, as well as details and an explanation of research and teaching activities (please do not include a photo)

Furthermore:

Proof of all completed higher education programmes (certificates, supplements, if applicable)
Outline of the content of the doctoral programme (listing academic achievements, intermediate examinations, etc.) as well as the content of the thesis (summary)
Other documentary evidence that may be relevant to this announcement (see prerequisites and desired qualifications)
Please provide three references (contact details of persons who the university may contact by telephone for information purposes)

To apply, please select the position with the reference code 673/23 in the category “Scientific Staff” using the link “Apply for this position” in the job portal at https://jobs.aau.at/en/. >>> LINK <<<

Candidates must furnish proof that they meet the required qualifications by 12 June 2024 at the latest.

For further information on this specific vacancy, please contact Prof. Christian Timmerer (christian.timmerer@aau.at). General information about the university as an employer can be found at https://jobs.aau.at/en/the-university-as-employer/. At the University of Klagenfurt, recruitment and staff matters are accompanied not only by the authority responsible for the recruitment procedure but also by the Equal Opportunities Working Group and, if necessary, by the Representative for Disabled Persons.

As part of its human resources policy, the University of Klagenfurt places particular emphasis on anti-discrimination, equal opportunities, and diversity.

People with disabilities or chronic diseases, who fulfil the requirements, are particularly encouraged to apply.

Travel and accommodation costs incurred during the application process will not be refunded.

Translations into other languages shall serve informational purposes only. Solely the version advertised in the University Bulletin (Mitteilungsblatt) shall be legally binding.

Streaming week in Denver: MOQ interim + Mile-High Video + SVTA Segments

noreply@blogger.com (Unknown) — Wed, 17 Jan 2024 10:29:00 +0000

The next Media over QUIC (MOQ) interim meeting will be hosted by Comcast in Denver (Feb. 6-8). It is open to public participation and it is free. Details are here: https://github.com/moq-wg/wg-materials/blob/main/interim-24-02/arrangements.md

Then, the ACM Mile-High Video conference will be just a few miles away (including a Latency Party during the Super Bowl) between Feb. 11-14. Details are here: https://www.mile-high.video/technical-program

Finally, SVTA Segments 2024 will take place at the same venue on Feb. 14th. Details are here: https://segments2024.svta.org/

You can benefit from the early (and combo) registration rates for Mile-High Video and Segments.

Hat-Trick Victory: MPEG-DASH Papers Shine in ACM SIGMM Test of Time Awards

noreply@blogger.com (Unknown) — Thu, 07 Dec 2023 14:34:00 +0000

The Association for Computing Machinery (ACM) Special Interest Group on Multimedia (SIGMM) provides a Test of Time Award. The details for this award can be found here, and I took the liberty to copy the main aspects as follows.

"This award is presented every year, starting in 2020, to the authors of the paper published either 10, 11 or 12 years previously at an SIGMM sponsored or co-sponsored conference (so the 2020 award would be for papers at a 2008, 2009 or 2010 SIGMM conference). The award recognizes the paper that has had the most impact and influence on the field of Multimedia in terms of research, development, product or ideas, during the intervening years, as selected by a selection committee. The contributions the selection committee will focus on may be theoretical advances, techniques and/or software tools that have been widely used, and/or innovative applications that have had impact on multimedia computing."

Interestingly, in the past three years, papers related to MPEG-DASH were always among the winners or honorable mentions as follows:

2021 Awards

Winner (MM Systems & Networking)

Thomas Stockhammer. 2011. Dynamic adaptive streaming over HTTP --: standards and design principles. In Proceedings of the second annual ACM conference on Multimedia systems (MMSys '11). Association for Computing Machinery, New York, NY, USA, 133–144. https://dl.acm.org/doi/10.1145/1943552.1943572

Abstract: In this paper, we provide some insight and background into the Dynamic Adaptive Streaming over HTTP (DASH) specifications as available from 3GPP and in draft version also from MPEG. Specifically, the 3GPP version provides a normative description of a Media Presentation, the formats of a Segment, and the delivery protocol. In addition, it adds an informative description on how a DASH Client may use the provided information to establish a streaming service for the user. The solution supports different service types (e.g., On-Demand, Live, Time-Shift Viewing), different features (e.g., adaptive bitrate switching, multiple language support, ad insertion, trick modes, DRM) and different deployment options. Design principles and examples are provided.

2022 Awards

Honorable Mention, in the category of “Multimedia Systems- Networks”

Saamer Akhshabi, Ali C. Begen, and Constantine Dovrolis. 2011. An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP. In Proceedings of the second annual ACM conference on Multimedia systems (MMSys '11). Association for Computing Machinery, New York, NY, USA, 157–168. https://dl.acm.org/doi/10.1145/1943552.1943574

Abstract: Adaptive (video) streaming over HTTP is gradually being adopted, as it offers significant advantages in terms of both user-perceived quality and resource utilization for content and network service providers. In this paper, we focus on the rate-adaptation mechanisms of adaptive streaming and experimentally evaluate two major commercial players (Smooth Streaming, Netflix) and one open source player (OSMF). Our experiments cover three important operating conditions. First, how does an adaptive video player react to either persistent or short-term changes in the underlying network available bandwidth. Can the player quickly converge to the maximum sustainable bitrate? Second, what happens when two adaptive video players compete for available bandwidth in the bottleneck link? Can they share the resources in a stable and fair manner? And third, how does adaptive streaming perform with live content? Is the player able to sustain a short playback delay? We identify major differences between the three players, and significant inefficiencies in each of them.

2023 Awards

Honorable Mention, in the category of "MM Systems & Networking"

Stefan Lederer, Christopher Müller, and Christian Timmerer. 2012. Dynamic adaptive streaming over HTTP dataset. In Proceedings of the 3rd Multimedia Systems Conference (MMSys '12). Association for Computing Machinery, New York, NY, USA, 89–94. https://dl.acm.org/doi/10.1145/2155555.2155570

Abstract: The delivery of audio-visual content over the Hypertext Transfer Protocol (HTTP) got lot of attention in recent years and with dynamic adaptive streaming over HTTP (DASH) a standard is now available. Many papers cover this topic and present their research results, but unfortunately all of them use their own private dataset which -- in most cases -- is not publicly available. Hence, it is difficult to compare, e.g., adaptation algorithms in an objective way due to the lack of a common dataset which shall be used as basis for such experiments. In this paper, we present our DASH dataset including our DASHEncoder, an open source DASH content generation tool. We also provide basic evaluations of the different segment lengths, the influence of HTTP server settings, and, in this context, we show some of the advantages as well as problems of shorter segment lengths.

A Tutorial on Immersive Video Delivery: From Omnidirectional Video to Holography

noreply@blogger.com (Unknown) — Tue, 05 Dec 2023 16:47:00 +0000

A Tutorial on Immersive Video Delivery: From Omnidirectional Video to Holography

IEEE Communications Surveys and Tutorials

[PDF]

Jeroen van der Hooft (Ghent University, Belgium), Hadi Amirpour (AAU, Austria), Maria Torres Vega (KU Leuven, Belgium), Yago Sanchez (Fraunhofer/HHI), Raimund Schatz (AIT, Austria), Thomas Schierl (Fraunhofer/HHI, Germany), and Christian Timmerer (AAU, Austria)

Abstract: Video services are evolving from traditional two-dimensional video to virtual reality and holograms, which offer six degrees of freedom to users, enabling them to freely move around in a scene and change focus as desired. However, this increase in freedom translates into stringent requirements in terms of ultra-high bandwidth (in the order of Gigabits per second) and minimal latency (in the order of milliseconds). To realize such immersive services, the network transport, as well as the video representation and encoding, have to be fundamentally enhanced. The purpose of this tutorial article is to provide an elaborate introduction to the creation, streaming, and evaluation of immersive video. Moreover, it aims to provide lessons learned and to point at promising research paths to enable truly interactive immersive video applications toward holography.

Keywords—Immersive video delivery, 3DoF, 6DoF, omnidirectional video, volumetric video, point clouds, meshes, light fields, holography, end-to-end systems

J. van der Hooft, H. Amirpour, M. Torres Vega, Y. Sanchez, R. Schatz, T. Schierl, C. Timmerer, "A Tutorial on Immersive Video Delivery: From Omnidirectional Video to Holography," in IEEE Communications Surveys & Tutorials, vol. 25, no. 2, pp. 1336-1375, Secondquarter 2023, doi: 10.1109/COMST.2023.3263252.

Immersive Video Delivery: From Omnidirectional Video to Holography from Alpen-Adria-Universität

MPEG news: a report from the 144th meeting

noreply@blogger.com (Unknown) — Tue, 28 Nov 2023 15:50:00 +0000

The original blog post can be found at the Bitmovin Techblog and has been modified/updated here to focus on and highlight research aspects. Additionally, this version of the blog post will also be posted at ACM SIGMM Records.

MPEG News Archive

The 144th MPEG meeting was held in Hannover, Germany! For those interested, the press release with all the details is available. It’s great to see progress being made in person (cf. also the group pictures below).

Attendees of the 144th MPEG meeting in Hannover, Germany.

The main outcome of this meeting is as follows:

MPEG issues Call for Learning-Based Video Codecs for Study of Quality Assessment
MPEG evaluates Call for Proposals on Feature Compression for Video Coding for Machines
MPEG progresses ISOBMFF-related Standards for the Carriage of Network Abstraction Layer Video Data
MPEG enhances the Support of Energy-Efficient Media Consumption
MPEG ratifies the Support of Temporal Scalability for Geometry-based Point Cloud Compression
MPEG reaches the First Milestone for the Interchange of 3D Graphics Formats
MPEG announces Completion of Coding of Genomic Annotations

We have modified the press release to cater to the readers of ACM SIGMM Records and highlighted research on video technologies. This edition of the MPEG column focuses on MPEG Systems-related standards and visual quality assessment. As usual, the column will end with an update on MPEG-DASH.

Visual Quality Assessment

MPEG does not create standards in the visual quality assessment domain. However, it conducts visual quality assessments for its standards during various stages of the standardization process. For instance, it evaluates responses to call for proposals, conducts verification tests of its final standards, and so on. MPEG Visual Quality Assessment (AG 5) issued an open call to study quality assessment for learning-based video codecs. AG 5 has been conducting subjective quality evaluations for coded video content and studying their correlation with objective quality metrics. Most of these studies have focused on the High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC) standards. To facilitate the study of visual quality, MPEG maintains the Compressed Video for the study of Quality Metrics (CVQM) dataset.

With the recent advancements in learning-based video compression algorithms, MPEG is now studying compression using these codecs. It is expected that reconstructed videos compressed using learning-based codecs will have different types of distortion compared to those induced by traditional block-based motion-compensated video coding designs. To gain a deeper understanding of these distortions and their impact on visual quality, MPEG has issued a public call related to learning-based video codecs. MPEG is open to inputs in response to the call and will invite responses that meet the call’s requirements to submit compressed bitstreams for further study of their subjective quality and potential inclusion into the CVQM dataset.

Considering the rapid advancements in the development of learning-based video compression algorithms, MPEG will keep this call open and anticipates future updates to the call.

Interested parties are kindly requested to contact the MPEG AG 5 Convenor Mathias Wien (wien@lfb.rwth- aachen.de) and submit responses for review at the 145th MPEG meeting in January 2024. Further details are given in the call, issued as AG 5 document N 104 and available from the mpeg.org website.

Research aspects: Learning-based data compression (e.g., for image, audio, video content) is a hot research topic. Research on this topic relies on datasets offering a set of common test sequences, sometimes also common test conditions, that are publicly available and allow for comparison across different schemes. MPEG’s Compressed Video for the study of Quality Metrics (CVQM) dataset is such a dataset, available here, and ready to be used also by researchers and scientists outside of MPEG. The call mentioned above is open for everyone inside/outside of MPEG and allows researchers to participate in international standards efforts (note: to attend meetings, one must become a delegate of a national body).

MPEG Systems-related Standards

At the 144th MPEG meeting, MPEG Systems (WG 3) produced three news-worthy items as follows:

Progression of ISOBMFF-related standards for the carriage of Network Abstraction Layer (NAL) video data.
Enhancement of the support of energy-efficient media consumption.
Support of temporal scalability for geometry-based Point Cloud Compression (PPC).

ISO/IEC 14496-15, a part of the family of ISOBMFF-related standards, defines the carriage of Network Abstract Layer (NAL) unit structured video data such as Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), Essential Video Coding (EVC), and Low Complexity Enhancement Video Coding (LCEVC). This standard has been further improved with the approval of the Final Draft Amendment (FDAM), which adds support for enhanced features such as Picture-in-Picture (PiP) use cases enabled by VVC.

In addition to the improvements made to ISO/IEC 14496-15, separately developed amendments have been consolidated in the 7th edition of the standard. This edition has been promoted to Final Draft International Standard (FDIS), marking the final milestone of the formal standard development.

Another important standard in development is the 2nd edition of ISO/IEC14496-32 (file format reference software and conformance). This standard, currently at the Committee Draft (CD) stage of development, is planned to be completed and reach the status of Final Draft International Standard (FDIS) by the beginning of 2025. This standard will be essential for industry professionals who require a reliable and standardized method of verifying the conformance of their implementation.

MPEG Systems (WG 3) also promoted ISO/IEC 23001-11 (energy-efficient media consumption (green metadata)) Amendment 1 to Final Draft Amendment (FDAM). This amendment introduces energy-efficient media consumption (green metadata) for Essential Video Coding (EVC) and defines metadata that enables a reduction in decoder power consumption. At the same time, ISO/IEC 23001-11 Amendment 2 has been promoted to the Committee Draft Amendment (CDAM) stage of development. This amendment introduces a novel way to carry metadata about display power reduction encoded as a video elementary stream interleaved with the video it describes. The amendment is expected to be completed and reach the status of Final Draft Amendment (FDAM) by the beginning of 2025.

Finally, MPEG Systems (WG 3) promoted ISO/IEC 23090-18 (carriage of geometry-based point cloud compression data) Amendment 1 to Final Draft Amendment (FDAM). This amendment enables the compression of a single elementary stream of point cloud data using ISO/IEC 23090-9 (geometry-based point cloud compression) and storing it in more than one track of ISO Base Media File Format (ISOBMFF)-based files. This enables support for applications that require multiple frame rates within a single file and introduces a track grouping mechanism to indicate multiple tracks carrying a specific temporal layer of a single elementary stream separately.

Research aspects: MPEG Systems usually provides standards on top of existing compression standards, enabling efficient storage and delivery of media data (among others). Researchers may use these standards (including reference software and conformance bitstreams) to conduct research in the general area of multimedia systems (cf. ACM MMSys) or, specifically on green multimedia systems (cf. ACM GMSys).

MPEG-DASH Updates

The current status of MPEG-DASH is shown in the figure below with only minor updates compared to the last meeting.

MPEG-DASH Status, October 2023.

In particular, the 6th edition of MPEG-DASH is scheduled for 2024 but may not include all amendments under development. An overview of existing amendments can be found in the blog post from the last meeting. Current amendments have been (slightly) updated and progressed toward completion in the upcoming meetings. The signaling of haptics in DASH has been discussed and accepted for inclusion in the Technologies under Consideration (TuC) document. The TuC document comprises candidate technologies for possible future amendments to the MPEG-DASH standard and is publicly available here.

Research aspects: MPEG-DASH has been heavily researched in the multimedia systems, quality, and communications research communities. Adding haptics to MPEG-DASH would provide another dimension worth considering within research, including, but not limited to, performance aspects and Quality of Experience (QoE).

The 145th MPEG meeting will be online from January 22-26, 2024. Click here for more information about MPEG meetings and their developments.

MPEG news: a report from the 143rd meeting

noreply@blogger.com (Unknown) — Fri, 11 Aug 2023 10:43:00 +0000

MPEG News Archive

The 143rd MPEG meeting took place in person in Geneva, Switzerland. The official press release can be accessed here and includes the following details:

MPEG finalizes the Carriage of Uncompressed Video and Images in ISOBMFF
MPEG reaches the First Milestone for two ISOBMFF Enhancements
MPEG ratifies Third Editions of VVC and VSEI
MPEG reaches the First Milestone of AVC (11th Edition) and HEVC Amendment
MPEG Genomic Coding extended to support Joint Structured Storage and Transport of Sequencing Data, Annotation Data, and Metadata
MPEG completes Reference Software and Conformance for Geometry-based Point Cloud Compression

We have adjusted the press release to suit the audience here and emphasized research on video technologies. This blog post centers around ISOBMFF and video codecs. As always, I will conclude with an update on MPEG-DASH.

ISOBMFF Enhancements

The ISO Base Media File Format (ISOBMFF) supports the carriage of a wide range of media data such as video, audio, point clouds, haptics, etc., which has now been further extended to uncompressed videos and images.

ISO/IEC 23001-17 – Carriage of uncompressed video and images in ISOBMFF – specifies how uncompressed 2D image and video data is carried in files that comply with the ISOBMFF family of standards. This encompasses a range of data types, including monochromatic and color data, transparency (alpha) information, and depth information. The standard enables the industry to effectively exchange uncompressed video and image data while utilizing all additional information provided by the ISOBMFF, such as timing, color space, and sample aspect ratio for interoperable interpretation and/or display of uncompressed video and image data.

ISO/IEC 14496-15 (based on ISOBMFF), provides the basis for "network abstraction layer (NAL) unit structured video coding formats" such as AVC, HEVC, and VVC. The current version is the 6th edition, which has been amended to support neural-network post-filter supplemental enhancement information (SEI) messages. This amendment defines the carriage of the neural-network post-filter characteristics (NNPFC) SEI messages and the neural-network post-filter activation (NNPFA) SEI messages to enable the delivery of (i) a base post-processing filter and (ii) a series of neural network updates synchronized with the input video pictures/frames.

Research aspects: While the former, the carriage of uncompressed video and images in ISOBMFF, seems to be something obvious to be supported within a file format, the latter enables to use neural network-based post-processing filters to enhance video quality after the decoding process, which is an active field of research. The current extensions with the file format provide a baseline for the evaluation (cf. also next section).

Video Codec Enhancements

MPEG finalized the specifications of the third editions of the Versatile Video Coding (VVC, ISO/IEC 23090-3) and the Versatile Supplemental Enhancement Information (VSEI, ISO/IEC 23002-7) standards. Additionally, MPEG issued the Committee Draft (CD) text of the eleventh edition of the Advanced Video Coding (AVC, ISO/IEC 14496-10) standard and the Committee Draft Amendment (CDAM) text on top of the High Efficiency Video Coding standard (HEVC, ISO/IEC 23008-2).

These SEI messages include two systems-related SEI messages, (a) one for signaling of green metadata as specified in ISO/IEC 23001-11 and (b) the other for signaling of an alternative video decoding interface for immersive media as specified in ISO/IEC 23090-13. Furthermore, the neural network post-filter characteristics SEI message and the neural-network post-processing filter activation SEI message have been added to AVC, HEVC, and VVC.

The two SEI messages for describing and activating post-filters using neural network technology in video bitstreams could, for example, be used for reducing coding noise, spatial and temporal upsampling (i.e., super-resolution and frame interpolation), color improvement, or general denoising of the decoder output. The description of the neural network architecture itself is based on MPEG’s neural network representation standard (ISO/IEC 15938 17). As results from an exploration experiment have shown, neural network-based post-filters can deliver better results than conventional filtering methods. Processes for invoking these new post-filters have already been tested in a software framework and will be made available in an upcoming version of the VVC reference software (ISO/IEC 23090-16).

Research aspects: SEI messages for neural network post-filters (NNPF) for AVC, HEVC, and VVC, including systems supports within the ISOBMFF, is a powerful tool(box) for interoperable visual quality enhancements at the client. This tool(box) will (i) allow for Quality of Experience (QoE) assessments and (ii) enable the analysis thereof across codecs once integrated within the corresponding reference software.

MPEG-DASH Updates

The current status of MPEG-DASH is depicted in the figure below:

The latest edition of MPEG-DASH is the 5th edition (ISO/IEC 23009-1:2022) which is publicly/freely available here. There are currently three amendments under development:

ISO/IEC 23009-1:2022 Amendment 1: Preroll, nonlinear playback, and other extensions. This amendment has been ratified already and is currently being integrated into the 5th edition of part 1 of the MPEG-DASH specification.
ISO/IEC 23009-1:2022 Amendment 2: EDRAP streaming and other extensions. EDRAP stands for Extended Dependent Random Access Point and at this meeting the Draft Amendment (DAM) has been approved. EDRAP increases the coding efficiency for random access and has been adopted within VVC.
ISO/IEC 23009-1:2022 Amendment 3: Segment sequences for random access and switching. This amendment is at Committee Draft Amendment (CDAM) stage, the first milestone of the formal standardization process. This amendment aims at improving tune-in time for low latency streaming.

Additionally, MPEG Technologies under Consideration (TuC) comprises a few new work items, such as content selection and adaptation logic based on device orientation and signaling of haptics data within DASH.

Finally, part 9 of MPEG-DASH -- redundant encoding and packaging for segmented live media (REAP) -- has been promoted to Draft International Standard (DIS). It is expected to be finalized in the upcoming meetings.

Research aspects: Random access has been extensively evaluated in the context of video coding but not (low latency) streaming. Additionally, the TuC item related to content selection and adaptation logic based on device orientation raises QoE issues to be further explored.

The 144th MPEG meeting will be held in Hannover from October 16-20, 2023. Click here for more information about MPEG meetings and their developments.