FTV (Free viewpoint TV) and Creation of Ray-Based Image Engineering
Article Sidebar
Published:
Jan 20, 2008
Main Article Content
Abstract
Free viewpoint TV (FTV) enables us to view a distant 3D world by freely changing our viewpoints as if we were there. FTV will bring an epochal change in the history of television since this function has not yet been achieved by conventional TV technology. We propose the concept of FTV and veri¯ed its feasibility using the world's ¯rst real-time system including the complete chain of operation from image capture to display. The international standardization of FTV technologies is now underway. FTV is a ray based system rather than a pixel-based system. We are creating ray-based image engineering through the development of FTV.
Article Details
How to Cite
Tanimoto, M. (2008). FTV (Free viewpoint TV) and Creation of Ray-Based Image Engineering. ECTI Transactions on Electrical Engineering, Electronics, and Communications, 6(1), 3–14. https://doi.org/10.37936/ecti-eec.200861.171741
Section
Research Article
This journal provides immediate open access to its content on the principle that making research freely available to the public supports a greater global exchange of knowledge.
- Creative Commons Copyright License
The journal allows readers to download and share all published articles as long as they properly cite such articles; however, they cannot change them or use them commercially. This is classified as CC BY-NC-ND for the creative commons license.
- Retention of Copyright and Publishing Rights
The journal allows the authors of the published articles to hold copyrights and publishing rights without restrictions.
References
[1] Masayuki Tanimoto, "Free Viewpoint Television," The Journal of Three Dimensional Images, vol.15, no.3, pp.17-22, September 2001 (in Japanese).
[2] Masayuki Tanimoto, "Overview of Free View-point Television," Signal Processing: Image Communication, vol. 21, no.6, pp.454-461, July 2006.
[3] Masayuki Tanimoto, "Free Viewpoint Television FTV," Picture Coding Symposium 2004, Special Session 5, December 2004.
[4] Masayuki Tanimoto, "FTV (Free Viewpoint Television) Creating Ray-Based Image Engineering," Proc. of ICIP2005, pp. II-25-II-28, September 2005.
[5] M. Tanimoto and T. Fujii, "FTV-Free Viewpoint Television," ISO/IEC JTC1/SC29/WG11, M8595, July 2002.
[6] Masayuki Tanimoto; "Free Viewpoint Television," OSA Topical Meeting on Digital Holography and Three Dimensional Imaging, DWD2, June 2007(Invited Paper).
[7] Akira Kubota, Aljoscha Smolic, Marcus Magnor, Masayuki Tanimoto, Tsuhan Chen, and Cha Zhang; "Multiview Imaging and 3TDTV," IEEE Signal Processing Magazine, Vol.24, No.6, pp.10-21, November 2007.
[8] T. Fujii, "A Basic Study on Integrated 3-D Visual Communication," Ph.D dissertation in engineering, The University of Tokyo, 1994 (in Japanese).
[9] T. Fujii, T. Kimoto and M. Tanimoto, "Ray Space Coding for 3D Visual Communication," Picture Coding Symposium 1996, pp. 447-451, March 1996.
[10] M. Tanimoto, A. Nakanishi, T. Fujii and T. Kimoto, "The Hierarchical Ray-Space for Scalable 3-D Image Coding," Picture Coding Symposium 2001, pp. 81-84, April 2001.
[11] T. Fujii and M. Tanimoto, "Free-viewpoint Television based on the Ray-Space representation," Proc. SPIE ITCom 2002, pp. 175-189, August 2002.
[12] M. Sekitoh, T. Fujii, T. Kimoto and M. Tanimoto, "Bird's Eye View System for ITS," IEEE, Intelligent Vehicle Symposium, pp. 119-123, 2001.
[13] M. Tanimoto and T. Fujii, "FTV Free Viewpoint Television", ISO/IEC JTC1/SC29/WG11, M8595, July 2002.
[14] "Introduction to Multi-view Video Coding", ISO/IEC JTC 1/SC 29/WG11, N7328, July 2005.
[15] "AHG on FTV (Free Viewpoint Television)", ISO/IEC JTC1/SC29/WG11 N8947, April 2007.
[16] N. Bangchang, T. Fujii and M. Tanimoto, "Experimental System of Free Viewpoint TeleVision," Proc. IST/SPIE Symposium on Electronic Imaging, Vol. 5006-66, pp. 554-563, Jan. 2003.
[17] T. Fujii, K. Mori, K. Takeda, K. Mase, M. Tanimoto and Y. Suenaga, "Multipoint Measuring System for Video and Sound: 100-camera and microphone system", IEEE 2006 International Conference on Multimedia & Expo (ICME), pp. 437-440, July 2006.
[18] K. Matsumoto, T. Yendo, T. Fujii and M. Tanimoto, "Multiple-Image Recti¯cation for FTV," Proc. of 3D Image Conference 2006, P-19, pp. 171-174, July 2006.
[19] Norishige Fukushima, Tomohiro Yendo, Toshiaki Fujii, Masayuki and Tanimoto, "A Novel Rectification Method for Two-Dimensional Camera Array by Parallelizing Locus of Feature Points," Proc. of IWAIT2008, B5-1, January 2008.
[20] Kenji Yamamoto,Tomohiro Yendo, Toshiaki Fujii,Masayuki Tanimoto, "Colour Correction for Multiple-camera System by using Correspondences," The journal of the institute of Image Information and Television Engineers, vol. 61,no. 2, pp.213-222 ,February 2007
[21] Yun He, Joern Ostermann, Masayuki Tanimoto, Aljoscha Smolic; "Introduction to the Special Section on Multiview Video Coding," IEEE Transactions on Circuits and Systems for Video Technology, Vol.17, No.11, pp.1433-435 (November 2007).
[22] Kenji Yamamoto, Masaki Kitahara, Tomohiro Yendo, Toshiaki Fujii, Masayuki Tanimoto, Shinya Shimizu, Kazuto Kamikura, and Yoshiyuki Yashima; "Multiview Video Coding Using View Interpolation and Color Correction," IEEE Transactions on Circuits and Systems for Video Technology, Vol.17, No.11, pp.1436-1449(November 2007).
[23] A. Nakanishi, T. Fujii, T. Kimoto and M. Tanimoto, "Ray-Space Data Interpolation by Adaptive Filtering using Locus of Corresponding Points on Epipolar Plane Image," The Journal of the Institute of Image Information and Television Engineers (ITE), vol. 56, no. 8, pp. 1321-1327, August 2002.
[24] M. Droese, T. Fujii and M. Tanimoto, "Ray Space Interpolation based on Filtering in Disparity Domain," Proc. 3D Conference 2004, pp. 213-216, Tokyo, Japan, June 2004.
[25] M. Droese, T. Fujii and M. Tanimoto, "Ray Space Interpolation Constraining Smooth Disparities Based On Loopy Belief Propagation," Proc. of IWSSIP 2004, pp. 247-250, Poznan, Poland, Sept. 2004.
[26] N. Fukushima, T. Yendo, T. Fujii and M. Tanimoto, "Real-time arbitrary view interpolation and rendering system using Ray-Space," Proc. SPIE Three-Dimensional TV, Video, and Display IV, Vol. 6016, pp. 250-261, Nov. 2005.
[27] N. Fukushima, T. Yendo, T. Fujii and M. Tanimoto, "An Effective Partial Interpolation Method for Ray-Space," Proc. of 3D Image Conference 2006, pp. 85-88, July 2006.
[28] N. Fukushima, T. Yendo, T. Fujii and M. Tanimoto, "Free Viewpoint Image Generation Using Multi-Pass Dynamic Programming," Proc. SPIE Stereoscopic Displays and Virtual Reality Systems XIV, Vol.6490, pp. 460-470, Feb. 2007.
[29] M. Sugawara, M. Kanazawa, K. Mitani, H. Shimamoto, T. Yamashita and F. Okano, "Ultrahigh-Definition Video System with 4000 Scanning Lines," SMPTE Motion Imaging, 112, pp. 339-346, October/November 2003.
[30] T. Fujii and M. Tanimoto, "Real-Time Ray Space Acquisition System", SPIE Electronic Imaging, vol. 5291, pp. 179-187, Jan. 2004.
[31] Keisuke Manoh, Tomohiro Yendo, Toshiaki Fujii, and Masayuki Tanimoto, "Ray-Space Acquisition System of All-Around Convergent Views using a Rotation Mirror", Proc. of SPIE, Vol. 6778, pp.67780C-1-8, September 2007.
[32] Toshiaki Fujii, Tomohiro Yendo, Masayuki Tanimoto; "Ray-Space Transmission System with Real-Time Acquisition and Display", Proc. of IEEE Lasers and Electro-optics Society Annual Meeting 2007,pp.78-79, October 2007.
[33] T. Endo, Y. Kajiki, T. Honda and M. Sato, "Cylindrical 3-D Video Display Observable from All Directions," Proc. of Pacific Graphics 2000, pp. 300-306, October 2000.
[34] Ryuji Takano, Tomohiro Yendo, Toshiaki Fujii, Masayuki Tanimoto, "Scene Separation in Ray Space," Proc. of IMPS2005, pp.31-32, November 2005.
[35] Ryuji Takano, Tomohiro Yendo, Toshiaki Fujii, Masayuki Tanimoto, "Scene Separation and Synthesis Processing in Ray-Space," Proc. of IWAIT2007, P6-23, pp.878-883, January 2007.
[36] Naoyuki Chimura, Tomohiro Yendo, Toshiaki Fujii, Masayuki Tanimoto, "New Visual Arts by Processing Ray-Space," Proc. of Electronic Imaging & the Visual Arts (EVA) 2007 Florence, pp.170-175, March 2007.
[37] Naoyuki Chimura, Tomohiro Yendo, Toshiaki Fujii, Masayuki Tanimoto, "Image Generation with Special Effects by Deforming Ray-Space," Proc. of NICOGRAPH, S1-4, November 2007.
[38] "Call for Comments on 3DAV," ISO/IEC JTC1/SC29/WG11 N6051, Oct. 2003.
[39] "Call for Evidence on Multi-view Video Coding," ISO/IEC JTC1/SC29/WG11 N6720, Oct. 2004.
[40] "Call for Proposals on Multi-view Video Coding," ISO/IEC JTC1/SC29/WG11 N7327, July 2005.
[41] "Subjective test results for the CfP on Multi-view Video Coding," ISO/IEC JTC1/SC29/WG11 N7779, January 2006.
[42] M. Tanimoto, T Fujii, S. Sakazawa and H. Kimata, "Proposal on Standardization of Free Viewpoint TV (FTV)," ISO/IEC JTC1/SC29/WG11, M13612 (JVT-T140), July 2006.
[43] M. Tanimoto, T. Fujii, H. Kimata and S. Sakazawa, "Proposal on Requirements for FTV," ISO/IEC JTC1/SC29/WG11, M14417, April 2007.
[44] "Liaison Statement from the 3D Consortium to SC29/WG11 on FTV Standardization," ISO/IEC JTC1/SC29/WG11, M 14533, April 2007.
[45] "AHG on FTV (Free Viewpoint Television)," ISO/IEC JTC1/SC29/WG11 N9170, April 2007.
[46] "Preliminary FTV Model and Requirements," ISO/IEC JTC1/SC29/WG11, N9168, July 2007.
[47] "Applications and Requirements on FTV," ISO/IEC JTC1/SC29/WG11, N9466, October 2007.
[48] M. Kawakita, K. Iizuka, H. Nakamura, I. Mizuno, T. Kurita, T. Aida, Y. Yamanouchi, H. Mitsumine, T. Fukaya, H. Kikuchi, and F. Sato, "High-definition real-time depth-mapping TV camera: HDTV Axi-Vision Camera," Optics Express, Vol. 12, Issue 12, pp. 2781-2794, June 2004.
[49] M. Tanimoto, T. Fujii and K. Suzuki, "Experiment of view synthesis using multi-view depth," ISO/IEC JTC1/SC29/WG11, M14889, October 2007.
[2] Masayuki Tanimoto, "Overview of Free View-point Television," Signal Processing: Image Communication, vol. 21, no.6, pp.454-461, July 2006.
[3] Masayuki Tanimoto, "Free Viewpoint Television FTV," Picture Coding Symposium 2004, Special Session 5, December 2004.
[4] Masayuki Tanimoto, "FTV (Free Viewpoint Television) Creating Ray-Based Image Engineering," Proc. of ICIP2005, pp. II-25-II-28, September 2005.
[5] M. Tanimoto and T. Fujii, "FTV-Free Viewpoint Television," ISO/IEC JTC1/SC29/WG11, M8595, July 2002.
[6] Masayuki Tanimoto; "Free Viewpoint Television," OSA Topical Meeting on Digital Holography and Three Dimensional Imaging, DWD2, June 2007(Invited Paper).
[7] Akira Kubota, Aljoscha Smolic, Marcus Magnor, Masayuki Tanimoto, Tsuhan Chen, and Cha Zhang; "Multiview Imaging and 3TDTV," IEEE Signal Processing Magazine, Vol.24, No.6, pp.10-21, November 2007.
[8] T. Fujii, "A Basic Study on Integrated 3-D Visual Communication," Ph.D dissertation in engineering, The University of Tokyo, 1994 (in Japanese).
[9] T. Fujii, T. Kimoto and M. Tanimoto, "Ray Space Coding for 3D Visual Communication," Picture Coding Symposium 1996, pp. 447-451, March 1996.
[10] M. Tanimoto, A. Nakanishi, T. Fujii and T. Kimoto, "The Hierarchical Ray-Space for Scalable 3-D Image Coding," Picture Coding Symposium 2001, pp. 81-84, April 2001.
[11] T. Fujii and M. Tanimoto, "Free-viewpoint Television based on the Ray-Space representation," Proc. SPIE ITCom 2002, pp. 175-189, August 2002.
[12] M. Sekitoh, T. Fujii, T. Kimoto and M. Tanimoto, "Bird's Eye View System for ITS," IEEE, Intelligent Vehicle Symposium, pp. 119-123, 2001.
[13] M. Tanimoto and T. Fujii, "FTV Free Viewpoint Television", ISO/IEC JTC1/SC29/WG11, M8595, July 2002.
[14] "Introduction to Multi-view Video Coding", ISO/IEC JTC 1/SC 29/WG11, N7328, July 2005.
[15] "AHG on FTV (Free Viewpoint Television)", ISO/IEC JTC1/SC29/WG11 N8947, April 2007.
[16] N. Bangchang, T. Fujii and M. Tanimoto, "Experimental System of Free Viewpoint TeleVision," Proc. IST/SPIE Symposium on Electronic Imaging, Vol. 5006-66, pp. 554-563, Jan. 2003.
[17] T. Fujii, K. Mori, K. Takeda, K. Mase, M. Tanimoto and Y. Suenaga, "Multipoint Measuring System for Video and Sound: 100-camera and microphone system", IEEE 2006 International Conference on Multimedia & Expo (ICME), pp. 437-440, July 2006.
[18] K. Matsumoto, T. Yendo, T. Fujii and M. Tanimoto, "Multiple-Image Recti¯cation for FTV," Proc. of 3D Image Conference 2006, P-19, pp. 171-174, July 2006.
[19] Norishige Fukushima, Tomohiro Yendo, Toshiaki Fujii, Masayuki and Tanimoto, "A Novel Rectification Method for Two-Dimensional Camera Array by Parallelizing Locus of Feature Points," Proc. of IWAIT2008, B5-1, January 2008.
[20] Kenji Yamamoto,Tomohiro Yendo, Toshiaki Fujii,Masayuki Tanimoto, "Colour Correction for Multiple-camera System by using Correspondences," The journal of the institute of Image Information and Television Engineers, vol. 61,no. 2, pp.213-222 ,February 2007
[21] Yun He, Joern Ostermann, Masayuki Tanimoto, Aljoscha Smolic; "Introduction to the Special Section on Multiview Video Coding," IEEE Transactions on Circuits and Systems for Video Technology, Vol.17, No.11, pp.1433-435 (November 2007).
[22] Kenji Yamamoto, Masaki Kitahara, Tomohiro Yendo, Toshiaki Fujii, Masayuki Tanimoto, Shinya Shimizu, Kazuto Kamikura, and Yoshiyuki Yashima; "Multiview Video Coding Using View Interpolation and Color Correction," IEEE Transactions on Circuits and Systems for Video Technology, Vol.17, No.11, pp.1436-1449(November 2007).
[23] A. Nakanishi, T. Fujii, T. Kimoto and M. Tanimoto, "Ray-Space Data Interpolation by Adaptive Filtering using Locus of Corresponding Points on Epipolar Plane Image," The Journal of the Institute of Image Information and Television Engineers (ITE), vol. 56, no. 8, pp. 1321-1327, August 2002.
[24] M. Droese, T. Fujii and M. Tanimoto, "Ray Space Interpolation based on Filtering in Disparity Domain," Proc. 3D Conference 2004, pp. 213-216, Tokyo, Japan, June 2004.
[25] M. Droese, T. Fujii and M. Tanimoto, "Ray Space Interpolation Constraining Smooth Disparities Based On Loopy Belief Propagation," Proc. of IWSSIP 2004, pp. 247-250, Poznan, Poland, Sept. 2004.
[26] N. Fukushima, T. Yendo, T. Fujii and M. Tanimoto, "Real-time arbitrary view interpolation and rendering system using Ray-Space," Proc. SPIE Three-Dimensional TV, Video, and Display IV, Vol. 6016, pp. 250-261, Nov. 2005.
[27] N. Fukushima, T. Yendo, T. Fujii and M. Tanimoto, "An Effective Partial Interpolation Method for Ray-Space," Proc. of 3D Image Conference 2006, pp. 85-88, July 2006.
[28] N. Fukushima, T. Yendo, T. Fujii and M. Tanimoto, "Free Viewpoint Image Generation Using Multi-Pass Dynamic Programming," Proc. SPIE Stereoscopic Displays and Virtual Reality Systems XIV, Vol.6490, pp. 460-470, Feb. 2007.
[29] M. Sugawara, M. Kanazawa, K. Mitani, H. Shimamoto, T. Yamashita and F. Okano, "Ultrahigh-Definition Video System with 4000 Scanning Lines," SMPTE Motion Imaging, 112, pp. 339-346, October/November 2003.
[30] T. Fujii and M. Tanimoto, "Real-Time Ray Space Acquisition System", SPIE Electronic Imaging, vol. 5291, pp. 179-187, Jan. 2004.
[31] Keisuke Manoh, Tomohiro Yendo, Toshiaki Fujii, and Masayuki Tanimoto, "Ray-Space Acquisition System of All-Around Convergent Views using a Rotation Mirror", Proc. of SPIE, Vol. 6778, pp.67780C-1-8, September 2007.
[32] Toshiaki Fujii, Tomohiro Yendo, Masayuki Tanimoto; "Ray-Space Transmission System with Real-Time Acquisition and Display", Proc. of IEEE Lasers and Electro-optics Society Annual Meeting 2007,pp.78-79, October 2007.
[33] T. Endo, Y. Kajiki, T. Honda and M. Sato, "Cylindrical 3-D Video Display Observable from All Directions," Proc. of Pacific Graphics 2000, pp. 300-306, October 2000.
[34] Ryuji Takano, Tomohiro Yendo, Toshiaki Fujii, Masayuki Tanimoto, "Scene Separation in Ray Space," Proc. of IMPS2005, pp.31-32, November 2005.
[35] Ryuji Takano, Tomohiro Yendo, Toshiaki Fujii, Masayuki Tanimoto, "Scene Separation and Synthesis Processing in Ray-Space," Proc. of IWAIT2007, P6-23, pp.878-883, January 2007.
[36] Naoyuki Chimura, Tomohiro Yendo, Toshiaki Fujii, Masayuki Tanimoto, "New Visual Arts by Processing Ray-Space," Proc. of Electronic Imaging & the Visual Arts (EVA) 2007 Florence, pp.170-175, March 2007.
[37] Naoyuki Chimura, Tomohiro Yendo, Toshiaki Fujii, Masayuki Tanimoto, "Image Generation with Special Effects by Deforming Ray-Space," Proc. of NICOGRAPH, S1-4, November 2007.
[38] "Call for Comments on 3DAV," ISO/IEC JTC1/SC29/WG11 N6051, Oct. 2003.
[39] "Call for Evidence on Multi-view Video Coding," ISO/IEC JTC1/SC29/WG11 N6720, Oct. 2004.
[40] "Call for Proposals on Multi-view Video Coding," ISO/IEC JTC1/SC29/WG11 N7327, July 2005.
[41] "Subjective test results for the CfP on Multi-view Video Coding," ISO/IEC JTC1/SC29/WG11 N7779, January 2006.
[42] M. Tanimoto, T Fujii, S. Sakazawa and H. Kimata, "Proposal on Standardization of Free Viewpoint TV (FTV)," ISO/IEC JTC1/SC29/WG11, M13612 (JVT-T140), July 2006.
[43] M. Tanimoto, T. Fujii, H. Kimata and S. Sakazawa, "Proposal on Requirements for FTV," ISO/IEC JTC1/SC29/WG11, M14417, April 2007.
[44] "Liaison Statement from the 3D Consortium to SC29/WG11 on FTV Standardization," ISO/IEC JTC1/SC29/WG11, M 14533, April 2007.
[45] "AHG on FTV (Free Viewpoint Television)," ISO/IEC JTC1/SC29/WG11 N9170, April 2007.
[46] "Preliminary FTV Model and Requirements," ISO/IEC JTC1/SC29/WG11, N9168, July 2007.
[47] "Applications and Requirements on FTV," ISO/IEC JTC1/SC29/WG11, N9466, October 2007.
[48] M. Kawakita, K. Iizuka, H. Nakamura, I. Mizuno, T. Kurita, T. Aida, Y. Yamanouchi, H. Mitsumine, T. Fukaya, H. Kikuchi, and F. Sato, "High-definition real-time depth-mapping TV camera: HDTV Axi-Vision Camera," Optics Express, Vol. 12, Issue 12, pp. 2781-2794, June 2004.
[49] M. Tanimoto, T. Fujii and K. Suzuki, "Experiment of view synthesis using multi-view depth," ISO/IEC JTC1/SC29/WG11, M14889, October 2007.