Folks,

Here is a version of the work description that addresses the comments
made on the first draft. 

The changes include: 

1.       Charter section, qualification that we are working on SIP-based
systems

2.       Charter section, minor word smithing

3.       Charter section, reference to RAI WG with relevant work

4.       Scope section, clarification of interoperability requirements

5.       Scope section, replacement of second paragraph regarding
treatment of non-video non-audio media types

6.       Goals and Milestones - addition of a Problem Statement draft

 

 

Comments welcome

 

Allyn

 

Multi-streams for Telepresence Description of Work

 

 

 

Background

One branch of video conferencing has evolved that is focused on
immersive "being there" experience.  Referred to in various ways such as
virtual conferencing, telepresence or media spaces, early systems were
mainly research projects or business systems with limited deployments.
In recent years telepresence systems have seen considerable market
success.  Following the model of early systems, the first wave of
commercial systems have been typically located in specially designed
single-purpose rooms with multiple relatively large displays permitting
life size image reproduction, multiple cameras, encoders, decoders and
microphones.  These systems have several important characteristics that
are different from more traditional video conferencing systems.  

 

The first difference concerns controlling the visual viewpoint in order
to improve participant nonverbal communication. These systems preserve
essential group meeting characteristics such as eye contact, group
gestures, seating order and spatial audio by carefully orchestrating the
miking and camera angles at each of the sites . This is distinct from
the more traditional approach where the geometric relationship between
media streams is not used to preserve inter-stream communication aspects
such as eye contact and group dynamics.  

 

A second difference is manipulation of the environment to improve
immersion.  With telepresence systems, cinematographic aspects of the
local environment reproduction are carefully planned including color,
table shape, seating and lighting so that when combined with large high
quality displays, a strong sense of a "trompe l'oeil" or "being there"
immersive experience is created.  Typical video conference systems do
not include these considerations.

 

As telepresence video systems have become successful in the market,
manufacturers have started exploring delivery of the nonverbal
communication and immersive values of telepresence via smaller, less
expensive and more flexible video conferencing systems for a variety of
venues, such as individual offices, homes and kiosks. These are also
telepresence systems, since the audio and video quality is high enough
to allow clear image reproduction for nonverbal communication, they are
able to send and receive multiple media streams, and large coordinated
multi image displays are available for immersive installations.   As the
industry develops, the line between telepresence and video conferencing
may become blurred as nonverbal communication and immersive
installations become broadly available.

 

Problem

Although telepresence systems are based on open standards such as RTP,
SIP, H.264, H.323 suite, they cannot easily interoperate with each other
without operator assistance and expensive additional equipment that
translates from one vendor to another. It would be like having to make
sure all parties are on the same equipment (and network) when making a
telephone call.  A major factor in the inability of Telepresence systems
to work with each other is that there is no standard description of the
multiple streams that comprise the media flows. 

 

For example, in a multiple screen conference, the video and audio
streams sent from remote participants must be understood by receivers so
that they can be presented in a coherent and life-like manner. This
includes the ability to present the remote participants at their true
size for their apparent distance, while maintaining correct eye contact,
gesticular cues, and simultaneously providing a spatial audio sound
stage consistent with the video presentation.  The receiving device that
decides how to display the incoming information needs to understand a
number of variables such as the spatial position of the speaker, the
field of view of the cameras, the camera zoom, which media stream is
related to each of the displays, etc. 

 

Charter

The Telepresence Multi-Streams work item in DISPATCH is chartered to
define and specify for SIP-based systems the content of media
multi-stream messages and the way these will be transported. 

 

This work will provide a standard for the exchange of media semantic
information that will foster interoperable end stations and conference
bridges. It will specify  variables that describe the semantics of the
media streams and the recommended behavior to achieve interoperability.


 

This requires considering current widely deployed use cases, such as
single and multiple screens, multi-point, Scalable Video Coding (SVC),
as well as cases that are expected to be implemented using the protocol
framework produced by this work item.  The methodology for describing
the variables must allow extensibility of the variables, since
telepresence is still a young technology and may have use cases that are
not currently considered.

 

The work item will identify use cases, define requirements, and define a
method for describing and transporting information about multiple media
streams, including a specification of variables required to support the
use cases. This work item will consider the reuse of existing IETF
protocols and produce an architecture/protocol framework document
describing the protocols required to be implemented to support this
functionality.  The document will identify any enhancements required to
existing protocols as well as describing new protocol(s) for
interoperable multi-streams negotiation that may be required.

 

Relevant work to be drawn upon has been done in XCON, MMUSIC, AVT, and
FECframe.

 

Scope

The scope includes both systems that provide a fully immersive
experience, and systems that interwork with them and therefore need to
understand the same multiple stream semantics.

 

 

The focus of this work is on audio and video multiple streams.  Other
media types may be considered, however development of methodologies for
them is not within the scope of this work.

 

Interoperation with standards compliant systems is required, such as
SIP-based video conferencing systems.  However, backwards compatibility
with existing non-standards compliant telepresence systems is not
required.

 

 

 

The group will produce

- Requirements and use cases

 

- Architectural Framework describing the protocols required to be
implemented to support this functionality and identifying existing
protocol enhancements and new protocol functionality required

 

- Specification of a new protocol to support telepresence multi-streams
[if required]

 

Goals and Milestones

Nov 2010 

 

Use Cases and Requirements to IESG as Informational RFC 

Nov 2010

March 2011 

Problem Statement to IESG as Informational RFC

Architecture to IESG as Informational RFC 

March 2011

Revise Charter [IF new protocol is not required]

Nov 2011 

Submit protocol draft to IESG as Proposed Standard RFC