ISO/IEC JTC1/SC29/WG11 N3943

January 2001 - Pisa



Title:          Intellectual Property Management and Protection in MPEG Standards[1]

Source:      Requirements 

Status:       Approved

Author        Rob Koenen



Intellectual Property Management and Protection
in MPEG Standards

1         Abstract

MPEG has a long history in dealing with DRM issues. Gradually, MPEG is moving from defining hooks to proprietary systems (MPEG-2, MPEG-4 Version 1) to more encompassing standardization in Intellectual Property Management and Protection. MPEG feels that this is necessary to achieve MPEG’s most important goal: interoperability. The need for a standard rights description language has been identified in MPEG-4, MPEG-7 and MPEG-21. This will be discussed in Pisa during MPEG’s 55th meeting, which takes place the week before the W3C workshop

2         Introduction

MPEG has a long history with Digital Rights Management issues. The MPEG specific term for DRM is ‘Intellectual Property Management and Protection’ – IPMP. In this paper, DRM will refer to the general issue of digital rights management, while IPMP will denote MPEG specifics. At the invitation of Renato Ianella, work group chair, this paper will give an overview of MPEG’s work in the DRM space. This covers the existing standards MPEG-2 and MPEG-4, the upcoming multimedia description standard MPEG-7 and the Multimedia Framework MPEG-21. It will not address the details of the most recent activity in the context of MPEG-4, the IPMP Extensions (Amendment 3 to 14496:2000 - MPEG-4 Systems). The details of this work will be covered in another position paper.

While rights holders’ communities have always played an important role in driving the DRM-related work items in MPEG, crafting the solutions has been a joint effort of the rights holders, manufacturers of conditional access (CA) and DRM systems, ICT and CE industries and the academic community.


DRM is usually associated with content exclusively. MPEG has recognized that IP also exists in MPEG Technology. In principle, MPEG-4 IPMP technology can also be applied to manage the usage of this type of IP.


The goal of the work in MPEG is interoperability. Two kinds of interoperability could be distinguished:

  1. From the manufacturer’s point of view: ensuring that modules from different suppliers can be integrated in a product through clear interface agreements;
  2. From the consumer’s point of view: ensuring that content from multiple sources will play on players from different make.

It is the last type of interoperability that MPEG is most concerned with.

3         MPEG-2: digital TV

MPEG-2 [1] contains a few tools for the Identification as well as the Protection of (copyright on) content. It is important to distinguish the two.

For identification purposes, there is the copyright descriptor. Individual streams as well as collections of streams forming a program can be identified to contain copyrighted material. The copyright descriptor consists of two parts: a unique copyright identifier of 32 bits, which identifies the type of the work (like ISBN, ISSN etc.). This number points at a registration authority. Each such authority can define its own syntax and semantics for identifying the work itself, through the use of a copyright number. The copyright can be signaled at the Audio or Video level, but also (as a collection) at the Systems Level.

For enabling protection, there are the similar provisions, which can be used to:


While MPEG-2 enabled Conditional Access systems to be integrated with MPEG-2 technology, there are no provisions for achieving interworking between different systems. The Simulcrypt system, providing a limited form of interoperability in digital television, was developed using these hooks; some further semantics defined in the European Digital Video Broadcast (DVB) project.

4         MPEG-4: IPMP ‘Hooks’

In MPEG-4 [2], representatives of rights holders’ communities raised the problem of protection of content in an early stage. Recognizing the power of the standard on the one hand and the digital developments on the other, MPEG issued, in April 1997, MPEG issued a Call for Proposals for the Identification and Protection of Content in MPEG-4 [3]. That Call for Proposals (CfP) covered:

       Identification of content

       Automated monitoring and tracking of creations

       Prevention of illegal copying

       Tracking object manipulation and modification history

       Supporting transactions between Users Media Distributors and Rights Holders

The call looked at the issue from all sides, and took into account requirements from (alphabetical and not exhaustive): authors, broadcasters, collection societies, consumers, creation providers, creators, rights management agencies, media companies, media distributors, performers, producers, publishers, retailers, rights holders, telecom companies and trusted third parties.

Figure 1 – Tight integration of IPMP and MPEG-4 System

(DMIF: delivery layer; CB: Composition Buffer; DB: decoding Buffer)

In response to this Call, a number of different proposals where received, ranging from watermarking technologies to complete DRM systems. The call brought together a good number of DRM experts, who, together with MPEG Systems experts and stakeholders, defined two different pieces of technology: one for the identification of copyright, and one to enable its protection. Below is a summary of this work; a more elaborate description can be found in [4].

4.1      Identification of Intellectual Property

The Identification part – the Intellectual Property dataset – identifies:

       Whether the content is protected by a (non-standard) IPMP System

       The type of the content (Audiovisual, Audio, Visual, Still Picture, …)

       The Registration Authority that hands out unique numbers for the type of content: ISAN (International Standard Audiovisual Number), ISBN, ISRC (International Standard Recording Code), etc.

       The number that identifies the content according to such a system

       Variable length fields for titles and supplementary information.

       References to separate data streams with such information.

This identification can be applied at the level of an ‘object’, which, in MPEG-4, can have any level of granularity. It can be a complete movie but also a single sound lasting for a few seconds. The standard does not prescribe when and how often to use such descriptors. International treaties and legislation prohibit removal of this information. MPEG does not (and, due to its nature, cannot) enforce that this information be present, persistent or correct. The second part of the IPMP framework can, however, be used to technically prevent the information to be altered or removed.

4.2      Protection and more

Next to the description part, also MPEG-4 has the protection part. Protection is actually too limited a term for the purposes; MPEG views protection as an integral part of the management of content – of DRM, that is. The discussions after the CfP in 1997 led to the conclusion that 1) it was not desirable to enforce IPMP tools upon all MPEG-4 content and MPEG-4 players, and 2) it was neither feasible nor desirable, at that point in time, to standardize a complete DRM system (or more than one). The great diversity in application domains, ranging from real time communications on low complexity devices to valuable content in set top boxes. Protection technology comes at a cost, and there is always a cost/benefit trade-off, so one size does not fit all. Also, there were concerns that content from a high value domain could be ‘laundered’ through a low protection domain.


Hence, again a ‘hooks’ approach was pursued. MPEG-4 integrates the hooks tightly with the MPEG-4 Systems layer, which makes it possible to build secure MPEG-4 delivery chains in very smart and efficient ways.

The underlying philosophy is simple.  The bitstream embeds information that informs the terminal which (of possibly multiple) IPMP system should be used to process governed objects in compliance with rules declared by the content provider.  The respective IPMP systems themselves were not specified within MPEG-4.

There are two simple extensions of basic MPEG-4 systems constructs:

       IPMP-Descriptors (IPMP-Ds): a part of the MPEG-4 object descriptors that describe how an object can be accessed and decoded. These IPMP-Ds are used to denote the IPMP System that was used to encrypt the object. An independent registration authority (RA) is used so any party can register its own IPMP system and identify this without collisions.

       IPMP-Elementary Streams (IPMP-ES). All MPEG objects are represented by elementary streams, that can reference each other. These special elementary  streams can be used to convey IPMP specific data. Their syntax and semantics are not specified in the standard.

Using these tools, it is possible to have IPMP Systems act on (See Figure 1):

       The media data

       The scene description and the compositor

       The interaction messages

       Any other information conveyed in the BIFS (Binary Format for Scenes, MPEG-4’s binary language for scene description and scene animation)

In principle, these IPMP tools can be used to govern the usage of technology-related IP – or patent royalties. It is beyond the scope of this paper to describe a possible implementation.

5         Reasons and Requirements for MPEG-4’s IPMP Extensions

Fairly soon after MPEG-4, with its IPMP hooks, became an International Standard, concerns were voiced within MPEG that many similar devices might be built by different manufacturers, all MPEG-4, but many of them not interworking. The reason for this: they would incorporate different, non-interworking protection schemes, because the manufacturers would work together with different service providers. This concern was exacerbated by the developments the Secure Digital Music Initiative (SDMI – where all players gathered to discuss technology for secure delivery of music. The agreements in SDMI pave the way for electronic distribution of valuable content, but interworking would still not be guaranteed. Such a development was not thought to be in the interest of the end user, who might have to buy, for example, a number of different portable devices to be able to play content from all labels. After all, as was stated in the introduction, the very goal of a standard like MPEG-4 is creating interoperability for consumers. Therefore, MPEG-4 recently (July 2000) issued a new Call for Proposals for IPMP Technology [5]. A summary from the requirements for such a solution, taken from [5] follows here:

1.       The solution shall support access to and interaction with content while keeping the amount of hardware to a minimum. There shall be no duplication of similar devices to interact with similar content from different sources. To a lesser extent, the same applies to software. Examples of interaction with content are playback, copy, edit, create and so forth.

2.       The solution shall support easy interaction with content from different sources without swapping of physical modules; that is without requiring action on the part of the end user. Addition of modules is acceptable if it requires a one-time action, if the device supports it, and if the cost is reasonable.

3.       The solution shall support conveying to end users which conditions apply to what types of interaction with the content. An example is payment for playback.

4.       The solution shall support protection of user privacy. Note: In many countries legislation requires that no user information shall be disclosed without the explicit consent of the end user.

5.       The solution shall support service models in which the end user's identity is not disclosed to the service/content provider and/or to other parties.

6.       The solution shall support the preservation of user rights. Notes: 1) For instance, the solution shall support preservation of user rights in such events as the provider going out of business. 2 ) It is believed that an important requirement of end users is that their rights to interact with the content not be revoked for alleged misuse when the burden of disproving misuse is entirely on the end user. However, MPEG does not currently see any implications for these requirements.

7.       The solution shall support the content and the end user’s rights to interact with it to survive common accidents, e.g. an operating system crash, or a flat battery.

8.       The solution shall support MPEG-4 terminal mobility, e.g. end users should be able to use the same device in different locations.

9.       The solution shall support content mobility across MPEG-4 terminals, e.g. end users should be able to move to a different terminal and keep their rights to interact with the content. Note: Assuming easy access to the content, this mainly applies to the portability of the rights to interact with it.

10.   The solution shall support content and the end user’s rights to interact with it to survive changing to a new version of similar hardware or software. Note: Assuming easy access to the content, this mainly applies to the renewability of the rights to interact with it.

11.   The solution shall support content and the end user’s rights to interact with it to survive changing to a different type of MPEG-4 hardware. Note: Assuming easy access to the content, this mainly applies to the survivability the rights to interact with it.

12.   The solution shall support transferring, temporarily or permanently, content and the rights to interact with it to another party.

13.   The solution shall enable content owners to control which of their assets are available when, where and under what conditions.

14.   The solution shall support persistent security over time and renewability of that security.

15.   The solution shall support the flexible expression of different business models/rules, which might yet be unknown and which may change over time, markets and geography. Note: Some business models are envisaged to involve ‘super distribution’, in which content and rights to interact with it are passed along from one user to another

16.   The solution shall enable content owners to change business rules whenever and however they wish.

17.   The solution shall support implementations that are cost effective with regard to the value of the content to be managed and protected.

18.   The solution shall support fast development of products and services.

19.   The solution shall support implementations into devices that have a long life cycle, i.e. at least five years.

20.   Implementation of the solution shall be based on currently available technology.

21.   The solution shall not impose policies. Note: Imposing policies is the legitimate domain of content, service and application providers, and governments.


The CfP resulted in 13 submissions. MPEG’s Systems Group is now working with the proposers to see how the requirements can be met, and has started an extension to the MPEG-4 Systems standard in the form of an amendment. The requirements are widely supported in MPEG, and all parties participate constructively to see how much is achievable.

The work currently concentrates on various interfaces, e.g. for downloading IPMP modules and renewing security. Major issues are the management of trust and tamper resistance. In the course of the work, it has become clear that a standard rights description language may be helpful. Several responses to the CfP addressed such a language.

Another submission to the Workshop will address the technical aspects of this work.

6         MPEG-7: Descriptions are as valuable as content

MPEG-7 specifies a different kind of coding than MPEG standards, as it does not define content representation for reproduction, but for content management (including search and retrieval, filtering of broadcasts, database asset management). MPEG-7 defines descriptions of content, from very low-level (colors, shapes, sound characteristics, segmentations in space and time) to very high level (written content descriptions, semantic information, the classical ‘metadata’). Special attention has been given to Intellectual Property, which comes in a number of different forms in MPEG-7, from describing rights pertaining to content to protecting the value of MPEG-7 descriptions themselves.


MPEG-7 defines the following elements: Descriptors (Ds), Description Schemes (DSs) a Description Definitions Language (DDL – based on XML Schema Language) and again a Systems layer. Included is also a binary, streamable representation of Descriptions and the DDL, called BiM (Binary format for MPEG-7).


Like in MPEG-2 and in MPEG-4, MPEG has worked with all of its constituent industry sectors, among which rights holders, to define the requirements. An excerpt of the MPEG-7 Requirements Document V.12 [6] highlighting MPEG-7’s IPMP requirements follows here:

1.       No legal status of descriptions - MPEG-7 Descriptions by nature have no legal bearing. The standard shall be designed in such a way that this is as clear as possible. Note: The greatest concern amongst the creative sectors is associated with the dangers inherent in the misrepresentation of legal attribution to information relating to intellectual property.

2.       Describing content rights - MPEG-7 shall provide a mechanism for pointing to content rights (rights owners, contractual information). Ds and DSs shall not directly describe content rights. Note: Rights ownership can change regularly which precludes the accurate declaration of this information in MPEG-7 applications. Allocation a unique identifier to each content element could be the answer. A resolution service could ensure that parties needing this information have appropriate access to it. [Note that MPEG-4 already allows this, RK]

3.       Relationship to content management and protection measures - MPEG-7 shall accommodate, and not by design interfere with, rights management information and technological protection measures used to manage and protect content. Note: In accordance with the World Intellectual Property Organization (WIPO) treaties it is prohibited to circumvent, alter or remove:

            Rights Management Information associated with content.

            Technical Protection Measures managing and protecting copyright material.

4.       Applications distinguishing between legitimate and illegitimate content - a. MPEG-7 shall support applications that distinguish between legitimate and illegitimate content.  b. The MPEG-7 standard shall be constructed so as to allow clear and unambiguous reference, in external specifications, agreements and in legislation, to the clauses in the MPEG-7 standard that address the requirement (a) above. Notes: While it is understood how (a) could be done inside trusted domains, further work is needed to investigate how this can be implemented outside trusted domains.  On (b): The ability to reference the relevant part of the MPEG-7 standard in contracts, laws etc., will allow the enforcement of this feature where appropriate..

5.       Authentication of descriptions - MPEG-7 shall offer a mechanism to allow for the authentication of MPEG-7 Descriptions. Note: This could be useful for companies specializing in selling Descriptions and for their customers.

6.       Management and protection of descriptions - MPEG-7 shall support the management of intellectual property in Descriptions and protection from unauthorized access, use and modification Note: Descriptions may embody content that requires management and protection.

7.       Management and protection of Ds and DSs - MPEG-7 shall support the management of intellectual property in Ds and DSs and protection from unauthorized access, use and modification. Note: The design of Ds and DSs may be intellectual property requiring management and protection.

8.       Usage rules - MPEG-7 shall contain Ds/DSs that provide information on how content may be used. Note: Such a feature may provide considerable consumer benefit by, for example, providing pre-purchase information. It may also enable different creative sectors to achieve interoperability between the providers of similar services. However, it should be noted that MPEG-7 cannot override the usage rules associated with the content itself which will be governed by the usage rules of its own management and protection system.

9.       Usage history - MPEG-7 shall contain Ds/DSs that provide information on how content has been used, in accordance with privacy rules. Note: 1) Such a usage history may be anonymous. 2) Such a description scheme can be used to record how often a film in a home database has been viewed.

10.   Identification of content - MPEG-7 shall enable the identification of content by international identification conventions. Note: Each of the creative sectors can manage content identification at its own discretion through MPEG-7 Ds and DSs. Such identification is preferably accomplished as in the MPEG-4 IP Identification Data Set.

11.   Identification of content in descriptions - Where the description contains content, MPEG-7 shall enable the identification of that content by international identification conventions. Notes: 1) For example, it is possible that a representation of the content, such as a ‘thumbnail’ of an image or the MIDI file of a sound recording will be used as the basis for constructing a Description for search purposes. In these examples, it may be necessary to identify the ‘content’ in the Description. Such identification is preferably accomplished as in the MPEG-4 IP Identification Data Set. 2) There may be a need to be able to identify Descriptions.

12.   Identification of descriptions - MPEG-7 may need to enable the unique identification of Descriptions. Note: This is for further discussion.


Again, we see the distinction between identification of IP (in this case in content as well as descriptions of content) and the protection of IP. Additional elements like audit trails make it possible to not only protect, but also manage the content and the descriptions. With the abundance of electronic content, good descriptions will be as important as the content itself, and descriptions will certainly hold significant value.

The author of this paper believes that the provisions in MPEG-4 can resolve most of the requirements above. The requirements for identification are very similar as in MPEG-4, and so will the solutions be: references to external registration bodies, together with unique IDs assigned by these. Also the protection and management can and probably will be similar. If we view MPEG-7 descriptions as just another form of content (“one man’s metadata is another man’s data”), then the existing DRM tools can be used for their authentication and protection. Additional Description Schemes are probably needed, e.g. for requirement 9 (usage history).

One requirement cannot be resolved using existing tools: the usage rules, requirement 8. Hence, a requirement to develop a rights description language comes not only from the MPEG-4 project, but also from MPEG-7 (and, as we will see, it has also surfaced in the context of MPEG-21).

7         MPEG-21: Integrating the pieces

The following, taken from [7], explains MPEG-21’s goals best. Many elements exist to build an infrastructure for the delivery and consumption of multimedia content. There is, however, no 'big picture' to describe how the specification of these elements, either in existence or under development, relate to each other. The aim of MPEG-21 is 1) to understand if and how these various components fit together and 2) to discuss which new standards may be required, if gaps in the infrastructure exist and, once the above two points have been reached, 3) to actually accomplish the integration of different standards.


To cut a long story short, (see [8] and the introductory Sections of [7] for more detail), MPEG-21 currently concentrates on a number of areas in which MPEG thinks lacking standardization is hindering interoperability:

1.       Digital Item Declaration (a uniform and flexible abstraction and interoperable schema for declaring Digital Items)

2.       Content Representation (how the media resources are represented)

3.       Digital Item Identification and Description (a framework for identification and description of any entity regardless of its nature, type or granularity)

4.       Content Management and Usage (provide interfaces and protocols that enable creation, manipulation, search, access, storage, delivery, and (re)use of content across the content distribution and consumption value chain)

5.       Intellectual Property Management and Protection (the means to enable content to be persistently and reliably managed and protected across a wide range of networks and devices)

6.       Terminals and Networks (the ability to provide interoperable and transparent access to content across networks and terminals)

7.       Event Reporting (the metrics and interfaces that enable Users to understand precisely the performance of all reportable events within the framework)


See Figure 2 for a graphical overview of MPEG-21.


Calls for Proposals have been issued for the Digital Item Declaration (see [9], responses due in January 2001) and the Digital Item Identification and Description (see [10], responses due in March 2001). Again, a specification of how to deal with IPMP is an important part of the work, because it is needed for interoperability.


With respect to the management and protection of Digital Items (the MPEG-21 term for content – a loose definition), the MPEG-21 PDTR makes the following observations:

1.       Most of the e-content existent today is governed by at best rudimentary IPMP systems.

2.       No IPMP system has yet emerged as a de-facto standard.

3.       While various IPMP systems exist today, no framework exists to allow for interoperation amongst such systems.

4.       One problem for consumers interacting with e-content today is the lack of interoperability between IPMP systems.

5.       Owners of rights of e-content require the freedom to exercise their rights by choosing channels and technologies (including IPMP Systems) through which to offer and make available their content. Note: Through new technologies (e.g., the Internet), end users increasingly become owners of rights in content.

6.       Consumers of e-content may in some circumstances require the freedom to manage their privacy, which includes interacting with content anonymously.

7.       Most existing IPMP systems cannot deal with the subtleties of issues related to Intellectual Property Law.


It is MPEG-21’s aim in the IPMP area to “[…] provide a uniform framework that enables all Users to express their rights and interests in, and agreements related to, Digital Items and to have assurance that those rights, interests and agreements will be persistently and reliably managed and protected across a wide range of networks and devices.”


Figure 2 - Graphical Overview of MPEG-21

The requirements for technology to accommodate this are the same as for MPEG-4 (see Section 4). MPEG-21 will be developed consistently with MPEG-4 (and MPEG‑7), which is notably important in the IPMP area, where many overlapping requirements exist. The notion of users expressing their rights again leads to the necessity of defining a rights description language. Note that a ‘User’ in MPEG-21 context can be any party in the chain, not just the end user.


In Annex G of the PDTR, MPEG-21 further states as a possible course of action [7]:


To the extent possible, MPEG-21 should provide a uniform framework that enables the capture, codification, dissemination and reflection of updates of relevant legislation, regulations, agreements and cultural norms that together create the setting and generally accepted societal platform for commerce involving Digital Items.

The main requirements for this work area are:

1.       The Framework shall allow for languages that support the codification of usage criteria for Digital Items based upon cultural, societal and other rules.

2.       The Framework shall allow for languages that support the construction of proof of illegal use of Digital Items.

3.       The Framework to codify rules shall be flexible and extensible. This would allow for an initial implementation able to express only a limited set of rules. In order to express a larger set of rules at a later stage, further languages can be added to the framework.

4.       The Framework shall not favour any particular human language, culture or legal/administrative/political system.

5.             Note: This neutrality applies only to the Framework. Specific codification languages within the Framework may be, for example, bound to a specific human language or legal system.

6.       The Framework shall allow for efficient implementations of codification languages with respect to the size of rules and the resources needed for processing such rules.

7.       Compliant languages within the Framework shall have unambiguous semantics and predictable effects.

8.       The Framework shall provide for a mechanism to resolve conflicts between rules governing the interaction with the same Digital Item.

9.       MPEG-21 shall provide means by which codified rules can be given precedence amongst themselves within the Framework.


The following is suggested as an action plan:

1.       Adopt or extend existing rights expression languages, where appropriate, for describing contractual usage rules for Digital Items. Start from the work being done in MPEG-7, but develop new languages if needed.

2.       Expand these languages to allow the expression of rights and interest in personal data.

3.       Expand these languages to allow the expression of public policies and rules stemming from sources other than Rights Holders, such as governments and other relevant rule-making bodies. This work item may require more time than available in the first phase of the development of MPEG-21. As soon as time and resources are available, this item should be undertaken.


In January 2001, MPEG issued a ‘Call for Requirements’ to invite interested parties to submit their requirements for a Rights Language and a Rights Data Dictionary to MPEG.

8         References

[1]          ISO/IEC 13818-1: Generic coding of moving pictures and associated audio: Systems. (MPEG-2 Systems)

[2]          ISO/IEC 14496-1: Coding of Audiovisual Objects: Systems. (MPEG-4 Systems)

[3]          ISO/IEC JTC1/SC29/WG11 N1714, MPEG Requirements Group, Call for Proposals for the Identification and Protection of Content in MPEG-4, April 1997, Bristol MPEG meeting

[4]          ISO/IEC JTC1 SC29 WG11 N2614, “MPEG-4 Intellectual Property Management & Protection (IPMP) Overview & Applications Document”, December 1998, Rome MPEG meeting. (

[5]          ISO/IEC JTC1/SC29/WG11 N3543, MPEG Requirements Group, Call for Proposals for IPMP Solutions, July 2000, Beijing MPEG meeting

[6]          ISO/IEC JTC1/SC29/WG11 N3548, MPEG Requirements Group, MPEG-7 Requirements Document V.12, October 2000, La Baule MPEG meeting (

[7]          ISO/IEC JTC1/SC29/WG11 N3939, MPEG Requirements Group, MPEG-21 Proposed Draft Technical Report, January 2001, Pisa MPEG meeting (

[8]          Keith Hill, Leonardo Chiariglione, Rob Koenen, Program for MPEG-21 Workshop in Noordwijkerhout, NL, March 2000 (

[9]          ISO/IEC JTC1/SC29/WG11 N757, MPEG Requirements Group, Call for Proposals for Digital Item Declaration, October 2000, La Baule MPEG meeting (

[10]      ISO/IEC JTC1/SC29/WG11 N3757, MPEG Requirements Group, Call for Proposals for Digital Item Identification and Description, October 2000, La Baule MPEG meeting (

[11]      ISO/IEC JTC1/SC29/WG11 N757, MPEG Requirements Group, Call for Requirements for a Rights Data Dictionary and a Rights Expression Language, January 2001, Pisa MPEG meeting

[1] Note: this paper is based on the position paper submitted on behalf of MPEG for the W3C workshop on DRM, held on 22 and 23 January 2001, in Sophia Antipolis, France.