Concept, Objectives and State of the Art

i3Dpost will research and develop new methods and intelligent technologies for the extraction of structured 3D content models from video at a level of quality suitable for use in digital cinema and interactive games. The research will combine advances to the state of the art in 3D video production, 3D motion estimation, post-production tools and media semantics. The result will be film quality 3D content in a structured form, with semantic tagging, which can be manipulated in a graphic production pipeline and reused across different media platforms including films (either 2D or 3D), computer games, music and promotional videos.

The central element in most film is the actors and data capture of actors is today performed in specialised studio set-ups. The integration of multiple view 3D video analysis with on-set production will allow the creation of video quality actor and set models. Postproduction tools will be extended robustly to separate and manipulate scene elements. On-set capture of actor performance will be instrumented in a structured form for processing and rendering in a conventional graphics production pipeline whilst maintaining the visual quality of captured video for faces, body, and clothing movement. Structured representation of actor performance will enable flexible processing in post-production: it will become possible to modify camera view, movement, eye-gaze and lighting, to render novel film content. Structured representations of actors will also facilitate the retargeting of characters and performances to different platforms for production of highly realistic characters in games enabling a new generation of interactive entertainment with the visual detail and realism of films.

The overall objective is therefore to integrate 3D information extracted from the visual scene into all stages of the postproduction pipeline, supported by semantic metadata, at a quality level that meets the requirements of the film, games and professional media industry. This will be achieved by the invention of methods for recovering intelligent, structured content from on-set filming, and incorporating them in capture environments and software tools that enable the simple manipulation and reuse within post-production and the retargeting of content to interactive platforms.

The overall objective will be achieved and assessed via five Scientific and Technical Objectives:

• O1 - To capture and represent 3D data from still and moving objects as a basis for generating different kinds of
• O2 - To generate high-quality representations of sets and actors from the captured data, for use in different scenes or contexts, taking into account the dynamics of skin and clothing
• O3 ? To create software tools and plug-ins for the intelligent manipulation and repurposing of scenes, characters and faces
• O4 - To show the usefulness of the technologies in experimental production across different media and platforms
• O5 - To establish open standards for plug-ins for 3D content manipulation