Now, Anthony Orth and colleagues from RMIT University in Melbourne, Australia, present a new method to unscramble the angular information from optical fibers, opening standard endoscopy fiber bundles up to depth extraction and light field imaging: The researchers use intracore intensity patterns in the fiber bundle, which have traditionally been ignored, to reconstruct the angle of light rays at entry.

“The key observation is that the angular distribution of light is subtly hidden in the details of how these optical fiber bundles transmit light,” Orth said. “The fibres essentially ‘remember’ how light was initially sent in, and the pattern of light at the other side depends on the angle at which light entered the fiber.”

Using this additional information, the authors go on to demonstrate common light field features such as depth mapping, software refocus, and (single-shot) 3D sample visualization:

Apart from the paper’s obvious value in medical imaging (e.g. live 3D optical biopsies), the researchers also unlocked optical fiber bundles as a new class of light field sensor, beside camera arrays, microlens arrays, coded aperture masks, angle-sensitive pixels, micromirror devices, or mirror-based image multipliers.

Using a typical fiber bundle of 750 µm outer diameter and 30,000 fibers with 3.2 µm center-to-center spacing, the scientists were able to accurately map depth within the first 80 µm of the fiber bundle. However, the authors say that this limitation could be mitigated by increasing dynamic range of the system, e.g. using a camera with larger dynamic range, or by combining multiple exposures.

For more details, check out the original publication:
Orth A, Ploschner M, Wilson ER, Maksymov IS, Gibson BC. Optical fiber bundles: Ultra-slim light field imaging probes. Science Advances 5: eaav1555

The paper’s supplementary materials also contain some light field animations of the studied samples.

via, via

In contrast to most other light field solutions, the light field lens does not rely on a microlens array, but a set of mirrors inside the lens, which produces 3×3 different kaleidoscopic views through a single lens (hence the “K” in “K|Lens”). The central view, or “core view”, is used as the image reference, and subsequently enriched with information from the other views to increase resolution, reconstruct depth information and enable light field features.
Also in contrast to other products, K|Lens is an interchangeable lens that will purportedly be compatible with all major full-frame cameras.

These sample pictures were taken with the “first prototype lens” and a “commercial full-frame camera”. The EXIF information in the spring flower photo points to a Canon Eos 5D Mark III, but a photo of the lens in action shows the lens attached to a Nikon D810. The company says they are testing full-frame cameras from all major brands.

For both sample pictures, K|Lens posted the “raw” kaleidoscopic images (e.g. what the image sensor sees), the central sub-image, as well as computed depth maps. The spring flower sample also comes with computed images for software refocus and variable depth of field:

To get an idea of the range of views and depth information possible within K|Lens, we aligned the middle row of sub-images and created an animation, moving left to right and back:

According to their newsletter, the company plans to produce a pre-series lens by the end of 2019, and is in talks with three high-quality manufacturers from Germany and Japan.
While a final product is still “a long way to go”, the sample images above show that the concept works, and the company is making progress.

Author Dmitriy Vatolin (3Dvideo) details in rich detail the capabilities of Lytro’s 2016 Lytro Cinema camera and the fundamental basics behind it. He then goes on to look at what becomes possible with light field technology, with a special interest in video and film making, and finishes by describing how/when the technology could move over to smartphones.

The article is a worhtwile read, and also includes a fascinating 25 minute video interview/demonstration with Jon Karafin, then Head of Light Field Video at Lytro, published by No Film School:

Of course, Lytro closed business in early 2018, but the technology is there and has been shown to work, so it’s only a matter of time before somebody else picks up where Lytro left off.

via

While details in the press release are sparse, the display is apparently based on a 17-inch 8K display developed by JDI, and “tiled pattern” output of images “into a compressed pixel array format”. We’re assuming this last part relates to sub-images (like those in a raw light field recording), which form a light field using a microlens array or comparable optical element. Unfortunately, both company websites lack any further information about the light field display prototype.
However, reports from live demonstrations of the display mention that the 8K LCD panel has 7680×4320 pixels at 510 ppi, and the prototype supports 69 viewpoints over a viewing angle of 134°. This would give the display an effective resolution of approx. 920×520 pixels. Light field content is prepared by a Hyper Cube MHP-8000 8K player.

Nikkei XTech further reports that the technology uses polarization as a means to control the direction of light rays:

For the newly-developed light field display, a barrier substrate was attached to the TFT substrate on the side of the LCD panel’s backlight, and a polarizing plate was attached on top of it. The barrier consists of a region that blocks light and a region that transmits light. The barrier is used to control the direction of light and play 3D video supporting 69 viewpoints.

Display Daily suggests that the core of this technology may be merely an auto-stereoscopic effect, not true light field technology, but given that 69 separate views are produced for every frame, use of the term “light field” seems valid to us.

Here’s a video from one of JDI’s live demonstrations (the light field display part runs from 13:00 to 15:20):

The prototype was first presented at SID Display Week in May 2018. At the time of the press release, mass production of the display was scheduled for April 1, 2019 to March 31, 2020.

Full press release below:

Japan Display and NHK Media Technology Collaborative Research and Development of a 17.0-inch Light Field Display
– Realizes next-generation 3D video, without 3D glasses –

May 17, 2018 (Tokyo, Japan) – Japan Display Inc. (JDI)　and NHK Media Technology, Inc. (NHK-MT) today announced that they have added a video playback function to the”17.0-inch Light Field* Display” which has been under joint research and development between the two companies, and have realized next-generation 3D video. They plan to start mass production of this product during their common FY2019 period, April 1, 2019 through March 31, 2020.

The “17-inch light field display”, based on JDI’s 17-inch 8K LCD,　reproduces the reflected light from an object that corresponds to the actual visible position, thereby realizing natural-looking images without any special 3D glasses. Just by ordinary viewing it is able to express extremely realistic images.

JDI and NHK-MT have collaborated to develop a new system that converts multiple light field images arranged in a tiled pattern into a compressed pixel array format suitable for the light field display in real time. With this system it is possible to reproduce an 8K light field video by using a practical playback device.　Using the latest digitizing and computer graphics production technologies, combined with digital content compressed with newly-created non-linear algorithms, the system provides unprecedented representation of natural images.

With this new system, 3D videos may be seen without any special glasses and without eye and/or brain fatigue, and can be expected to be used in various applications, such as digital archiving, education and medical fields, etc.

JDI and NHK-MT intend to continue their collaborative work in new technological research and development to create the most life-like 3D images and video possible.

The 17.0-inch Light Field Display will be demonstrated at:

SID DISPLAY WEEK 2018
Venue: Los Angeles Convention Center, Los Angeles, California, USA
Conference: May 21 (Monday) ~ May 25 (Friday), 2018
Exhibition: May 22 (Tuesday) ~ May 24 (Thursday), 2018
URL: http://www.displayweek.org/ new window

SID’s Display Week is the premier international event for the electronic display industry, where breakthrough technologies are introduce. Display Week offers synergies unparalleled by any other display event, comprised of attendees and exhibitors representing the foremost display-engineering talent from all over the world, as well as leaders representing both the commercial and consumer markets.

*What is a “Light Field”
In the real world, light travels in an infinite variety of directions, and we recognize our three-dimensional world by the way in which we see this light. A “Light Field” simulates this infinite light concept and enables stereoscopic vision by reproducing light emitted in many different directions.

About Japan Display Inc.
Japan Display Inc. (“JDI”)is a global leader in innovative display solutions. We provide customers with advanced small- and medium-sized displays for high-end electronic applications, including smartphones, automobiles, medical equipment and VR/AR devices. Our strength lies in leading-edge low temperature poly-silicon (LTPS) technology that enables displays with high resolution, low-power consumption, narrow bezels, and design freedom, with more potential features under development.
Please visit: https://www.j-display.com/english/.

About NHK Media Technology Inc.
NHK Media Technology, Inc. is one of the major subsidiaries of NHK, Japan’s sole public broadcaster.NHK Media Technology, Inc. is responsible for the technical operations of NHK programs and the development and operation of its computer systems. We thus support NHK’s broadcasting by our state-of-the-art broadcast engineering and reliable information system technologies. As the media environment undergoes remarkable change and broadcasting services diversify, we are making full use of our twin-pillar technologies to challenge the very limits of these potentials in the industry. Our goals is to seek the better combination of broadcasting and telecommunications, and higher quality though 8K UHDTV production for the future.
Please visit: http://www.nhk-mt.co.jp/en/

via, via, via, via

Here’s how it works:
For the recording part, incoming light from a 360 degree cylindrical volume is first reflected off a parabolic mirror and into a field lens, which produces a spherical intermediate imaging volume. This intermediate volume is captured using a plenoptic camera, containing a microlens array.
To display a recorded scene, light follows the reverse path: A high-resolution display, with properties matching the camera sensor, generates light which is modulated by an (identical) microlens array and passes through the field lens. From there, the light rays reach the parabolic mirror and are directed back to their original location.
A beam-splitter, situated between the field lens and microlens array, allows switching between recording and displaying light fields.

The proposed optical configuration means that light fields can be captured and reproduced in one system, allowing viewers around the parabolic mirror to see objects in 3D and with parallax, as if they were physically there (like when the original light field was recorded).

To test a real-life version of this reciprocal light field system (see below for photos), the authors had to make some modifications to the proposed design:
Instead of a microlens array, the plenoptic camera was built using a micro photon sieve (mPSA) – an array of pinholes which focus light by means of diffraction and interference – with a Sony A7R II (42 Megapixel full-frame sensor). mPSAs have lower diffraction efficiency but are much more cost-effective.
The display part of the setup had to be scaled up due to the pixel size mismatch between the full frame image sensor and available displays. The adjusted setup utilised a 4K mobile phone display (Sony Xperia XZ Premium), and reproduced only about 25% of the entire captured image. However, the proof-of-concept system successfully demonstrated reconstruction of real images reflected from the parabolic mirror.

The upcoming research article builds upon a recent conference paper describing the general design of the system (Yöntem & Chu, 2018), and adds modelling data, simulations, as well as experimental results.
The full paper (Yöntem et al., 2019) is currently available through “Early Posting” access via institutional subscriptions, and will be published in an upcoming issue of JOSA A.
Update: The full paper (Yöntem et al., 2019) is now available in JOSA A (see below).

References:

• Yöntem A, Chu D. 2018. Design for 360-degree 3D Light-field Camera and Display. Imaging and Applied Optics 2018. OSA Technical Digest: paper 3Tu5G.6.
• Yöntem A, Li K, Chu D. In Press (2019). A reciprocal 360-degree 3D light-field image acquisition and display system. Journal of the Optical Society of America A: 36, A77-A87.

Figures were adapted with permission from Yöntem & Chu, 2018, and Yöntem et al., 2019 and the The Optical Society (OSA).

We found a 2017 patent application by CREAL3D co-founder Tomas Sluka, titled Near-eye sequential light-field projector with correct monocular depth cues, which describes a VR/AR (virtual reality, augmented reality) head-set that creates a light field through a different approach:
Instead of a single light field, the proposed device emits several smaller pinhole-aperture light fields, which are guided to the eye by use of mirrors.

These smaller light fields can be produced using a pinhole mask in front of multiple LEDs, or with individual diodes coupled with an optical fiber whose exit point acts as a point light. The author also mentions the use of fiber optics splitters, moving diodes, or moving mirrors as alternative possibilities.

Several of these sub-light fields are then selectively combined into a single light field by use of a spatial light modulator (SLM). The SLM, again, can use a range of mechanisms to selectively guide light rays or pin-hole light fields into or out of view, depending on which partial light fields are required to form a scene.
The prime SLM example described in the patent application is a digital micromirror device – a set of electronically controlled tilting mirrors – but can be “any reflective light modulator” or “a transmissive light modulator combined with a reflective surface” (e.g. a matrix of controlled transparent/opaque “pixels” in front of a static mirror).

For more details and variations, check out the original patent application: Near-eye sequential light-field projector with correct monocular depth cues (WO2018091984A1) – Google Patents