Network Working Group H.-J. Happel Internet-Draft audriga GmbH Intended status: Informational 27 January 2023 Expires: 31 July 2023 Structured and Dynamic Email draft-happel-structured-dynamic-email-00 Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 31 July 2023. Copyright Notice Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Problem statement . . . . . . . . . . . . . . . . . . . . . . 3 2.1. Lack of structured data . . . . . . . . . . . . . . . . . 3 2.2. Lack of structured interaction . . . . . . . . . . . . . 3 2.3. Current solutions . . . . . . . . . . . . . . . . . . . . 3 2.4. Goals . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Happel Expires 31 July 2023 [Page 1] Internet-Draft Structured and Dynamic Email January 2023 3. Design principles . . . . . . . . . . . . . . . . . . . . . . 5 3.1. Protocol-agnostic implementation . . . . . . . . . . . . 5 3.2. Co-existence of approaches . . . . . . . . . . . . . . . 5 3.3. Decentral semantics . . . . . . . . . . . . . . . . . . . 5 4. Solution approach . . . . . . . . . . . . . . . . . . . . . . 6 4.1. Email content (body) . . . . . . . . . . . . . . . . . . 6 4.2. Email metadata (header) . . . . . . . . . . . . . . . . . 6 4.3. Email storage . . . . . . . . . . . . . . . . . . . . . . 6 4.4. Email filtering . . . . . . . . . . . . . . . . . . . . . 6 4.5. Email context . . . . . . . . . . . . . . . . . . . . . . 7 4.6. Discovery . . . . . . . . . . . . . . . . . . . . . . . . 7 4.7. HTTP services . . . . . . . . . . . . . . . . . . . . . . 8 4.8. Trust . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5. Security considerations . . . . . . . . . . . . . . . . . . . 8 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 7. Informative References . . . . . . . . . . . . . . . . . . . 8 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 10 1. Introduction Email can be considered a successful technology, based on its broad adoption and based on the fact dozens of email-related RFCs have been written over time [MailRFCs]. Despite of these various RFCs, which are dealing with a multitude of small improvements, the major inner workings of email remain widely unchanged since their inception. In particular, email is still mostly a text-based medium, which requires a lot of human attention, even though more and more so called "transactional" emails are authored in an automated fashion. Accordingly, the task of dealing with large amounts of email is still a major challenge for many users. Most existing attempts that have been made to improve this situation focus on particular use cases. The main point of this paper is to suggest that it might be worthwhile to consider if there exists a generalizable core underneath those use cases. Such a more streamlined approach might help to condense specifications and to increase adoption by email servers and clients. Furthermore, we argue that such an approach might enable novel use cases by leveraging the specific and partially unique properties of email technology, which include: * Its ubiquitous and simple approach of addressing * Its asynchronous nature Happel Expires 31 July 2023 [Page 2] Internet-Draft Structured and Dynamic Email January 2023 * Its unique role in exchanging data betweeen the public Internet and the private information space of a user * Its established role as a wrapping or transport mechanism for other data * Its widespread support by many tools Essentially, this entails that a consciously designed approach for the automated processing of formally structured information in emails could be considered a "private API" of email users. While this certainly raises security and trust concerns, the opportunities behind such unifying approach seem worthwhile discussing. 2. Problem statement 2.1. Lack of structured data A large share of today's emails is of transactional nature, i.e., sent by automated agents or processes to human users. While those emails often have relatively clear semantics (such as _Invoice_ or _Reservation_), the medium of those emails is still human-readable text. Users and their email software cannot leverage knowledge inside and about those emails to make their processing a more efficient and enjoyable task. 2.2. Lack of structured interaction Using informal HTML email conventions, such emails can be considered as small websites, adapted and personalized for a certain use case. However, they are mostly a "dead end" for interaction. Due to the lack of explicit semantics in the underlying email, particular actions afforded by transactional mail (such as _Approval_ or _Rating_) cannot be easily made actionable for users. Also, many interactions require to switch from email to a regular web browser, resulting in a process disruption which is coined _Medienbruch_ in German language. 2.3. Current solutions Several attempts have been made to improve this situation. We discuss some of them in the following. Happel Expires 31 July 2023 [Page 3] Internet-Draft Structured and Dynamic Email January 2023 Within the IETF, various RFCs have been devised to structure certain types of email interaction. Probably most notable, iMIP [RFC2447] allows users to deal with meeting workflows in a structured manner. Further examples structure interaction with message delivery notifications [RFC8098], mailinglist subscriptions [RFC8058] or specify header-based semantic indicators for certain domains [RFC6477] or Emoji-based semantics of approval/disapproval in mailinglist discussions [RFC9078]. In more administrative use cases, special kinds of email body parts are used for abuse reporting [RFC6650] or administrative reporting [RFC6522]. In a USENET context [RFC1036], so-called "control messages" are defined for what can be considered certain API calls, such as for creating a USENET group. Looking outside the IETF, the "Email Markup" approach [EMarkup] by Google allows allows senders to integrate certain Schema.org [SchemaOrg] annotations (such as for hotel reservations or parcel delivery) in their email, such that email clients can provide customized support, including certain easy to use response actions. Email markup is still in use, but has a number of limitations: * Being a proprietary standard owned by Google * It does not support to annotate standard email elements such as headers or attachments * It is supported by only few providers * It is limited to a few fixed use cases * It is only available for senders that went through an approval process * It is not designed for human end users to send structured emails More recently, AMP email [AMPemail] (also initiated by Google) and Microsoft Actionable Messages [AM] have been specified. Compared to Email Markup, their focus is less on semantically annotating email content, but on more rich interaction, e.g., allowing the submission of simple forms. In addition, both approaches can include dynamic elements, by retrieving certain data, or even replacing the complete email body at reading time. Both approaches however suffer from similar limitations as those already pointed out for Email Markup. There also seems to be a lack of consensus, since Microsoft Outlook is reported to have removed initial support for AMP email [OutAMP]. Happel Expires 31 July 2023 [Page 4] Internet-Draft Structured and Dynamic Email January 2023 2.4. Goals To summarize, we see a need for a vendor neutral standard for structured and dynamic email. It should enable email senders to provide structured information about the semantics of their message and allow recipients (may it be human users or their agents) for rich and structured interaction with those emails. Such standard should also take into account the various RFCs specifying email semantics for specific use cases and ideally provide a more generalized framework for such use cases, which can be more consistently and easily implemented by vendors. For example, there exist various "informal" standards for certain types of email messages, that have not yet been standardized in dedicated RFCs, such as _out-of-office_ (aka _vacation notice_) emails. 3. Design principles 3.1. Protocol-agnostic implementation Structured email should work with existing protocols such as IMAP [RFC9051], SMTP [RFC5321], and novel ones such as JMAP [RFC8620]. Accordingly, most extensions would probably realized in the scope of the Internet Message Format [RFC5322] or in areas not yet addressed by standardization (see, e.g., Section 4.3, Section 4.5, Section 4.6, Section 4.7). 3.2. Co-existence of approaches A major advantage of email technology is the fact, that email is backwards compatible. For example, most HTML emails are accompanied by a plain text version to be used as a fallback in case an email client cannot deal with HTML. In a similar fashion, structured email could be provided as an additional option for clients able and willing to deal with it, thus avoiding a classic "chicken/egg" problem for novel technology. 3.3. Decentral semantics With respect to structured markup, an incremental and decentral approach is proposed. Instead of an authoritative fixed set of schemas, as in the case of Email Markup, users should be allows to create and extend schemas on their own. Happel Expires 31 July 2023 [Page 5] Internet-Draft Structured and Dynamic Email January 2023 4. Solution approach This draft can and will not yet provide a full specification for the problems stated. In this section, we will single out certain aspects that might be addressed by such specifications. 4.1. Email content (body) As described before, an (optional and alternate) structured description of the semantics of email messages is supposed be in the core of this proposal. While existing approaches such as Email Markup offer different options for how to include structured data, an alternative, machine-readable presentation of the message content in addition to human readable text and HTML representations, might be the most natural approach. 4.2. Email metadata (header) For the sake of efficient processing and perhaps also for reasons of data privacy, certain structured data about an email might also be captured in its headers. Header fields of certain message parts (such as attachments) might also be enriched by semantic annotation. 4.3. Email storage Email messages stored in IMAP are currently immutable. Receivers are not supposed to modify their actual content. Hence, the main means to add and store additional data on the receiving side are: * IMAP flags * IMAP mailboxes (sorting into folders, which can be considered as some sort of tagging) * Custom data storage on server or email client side As of today, this already yields certain data portability problems - e.g., IMAP flags are often not considered when exporting raw email message files. Accordingly, we see a need for the standardized persistent storage of data which is added by algorithmic processing or by users and their email clients. 4.4. Email filtering Once emails contain structured data about their content, this might create a demand for related extensions to the Sieve email filtering language [RFC5228]. Happel Expires 31 July 2023 [Page 6] Internet-Draft Structured and Dynamic Email January 2023 4.5. Email context Email lives in tight symbiosis with strongly related data such as address books, calendars, or tasks. However, besides specific use cases such as meeting organization, an integration between such data is mainly subject to non-standardized functionality of client software. Notably, email also operates in a wider context such as services which send transactional emails and also applications running on the same Desktop or Mobile device as the email client. Transactional emails are sent by services and organizations which are in a relation to the user. Further related data may be managed in outside applications or would be a candidate for being managed within email client software. From a broader architectural perspective, email can be considered a technology particularly suited to support novel trends in data sovereignty and decentralized data storage, since by design it bridges the public Internet data space with the private data spaces of mailbox users. Structured email will open new opportunities for exchanging data in all those cases. Accordingly, standards may help to enable and organize interoperability. 4.6. Discovery In order to allow users to send structured mails, they need to know if and which kind of structured emails a recipient can process. There might be multiple ways of conveying such information: * Headers in emails received from users, which a receiver can store/ look up * A DNS/HTTP-based discovery service, which returns structured email capability for a domain or particular sender An example for the latter could be: * User selects "john@doe.com" as recipient * Email client checks if discovery service is offered by "doe.com" * Email client queries "doe.com" for structured email capabilities * Discovery service at "doe.com" returns a document which states a number of structured data types or actions that are supported by the "john@doe.com" account Happel Expires 31 July 2023 [Page 7] Internet-Draft Structured and Dynamic Email January 2023 * The email client will allow the user to optionally select from those structured interaction options and provide an appropriate user experience 4.7. HTTP services For dynamic email features, an endpoint must be available to provide updated data for an email currently read by a user. Similar, if a structured email allows for the execution of a certain action or the submission of form data, there needs to be a receiving endpoint in place. For interoperability reasons, those endpoint APIs should be standardized. 4.8. Trust Structured and automated interaction certainly raises various security concerns (see Section 5). While not the focus of this current draft, we assume that parts of the proposed approach can also be used on a special "trust layer". E.g., certain structured message exchanges, in combination with signing or encryption could help to establish trust necessary for further structured communication. 5. Security considerations While it is clear that trust, security, and anti-spam considerations are core to the technology suggested in this document, this aspect is not discussed in depth at this stage in order to reduce complexity. While existing approaches such as Email Markup or Actionable Messages enforce strict control by registration processes, we think that a more open process is better suited for the decentralized character of email. Various trust mechanisms such as DKIM [RFC6376] validation, challenge-response emails, or trusted receivers in address books/ white lists might be considered as a prerequisite for structured email processing. 6. IANA Considerations This document has no IANA actions at this time. 7. Informative References Happel Expires 31 July 2023 [Page 8] Internet-Draft Structured and Dynamic Email January 2023 [AM] Microsoft Inc., "Actionable Messages", . [AMPemail] OpenJS Foundation, "AMP email", . [EMarkup] Google Inc., "Email Markup", . [MailRFCs] Takizawa, T., "MAIL RFCs", . [OutAMP] Steinbrinck, K., "Outlook is Turning Off AMP for Email", . [RFC1036] Horton, M. and R. Adams, "Standard for interchange of USENET messages", RFC 1036, DOI 10.17487/RFC1036, December 1987, . [RFC2447] Dawson, F., Mansour, S., and S. Silverberg, "iCalendar Message-Based Interoperability Protocol (iMIP)", RFC 2447, DOI 10.17487/RFC2447, November 1998, . [RFC5228] Guenther, P., Ed. and T. Showalter, Ed., "Sieve: An Email Filtering Language", RFC 5228, DOI 10.17487/RFC5228, January 2008, . [RFC5321] Klensin, J., "Simple Mail Transfer Protocol", RFC 5321, DOI 10.17487/RFC5321, October 2008, . [RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322, DOI 10.17487/RFC5322, October 2008, . [RFC6376] Crocker, D., Ed., Hansen, T., Ed., and M. Kucherawy, Ed., "DomainKeys Identified Mail (DKIM) Signatures", STD 76, RFC 6376, DOI 10.17487/RFC6376, September 2011, . [RFC6477] Melnikov, A. and G. Lunt, "Registration of Military Message Handling System (MMHS) Header Fields for Use in Internet Mail", RFC 6477, DOI 10.17487/RFC6477, January 2012, . Happel Expires 31 July 2023 [Page 9] Internet-Draft Structured and Dynamic Email January 2023 [RFC6522] Kucherawy, M., Ed., "The Multipart/Report Media Type for the Reporting of Mail System Administrative Messages", STD 73, RFC 6522, DOI 10.17487/RFC6522, January 2012, . [RFC6650] Falk, J. and M. Kucherawy, Ed., "Creation and Use of Email Feedback Reports: An Applicability Statement for the Abuse Reporting Format (ARF)", RFC 6650, DOI 10.17487/RFC6650, June 2012, . [RFC8058] Levine, J. and T. Herkula, "Signaling One-Click Functionality for List Email Headers", RFC 8058, DOI 10.17487/RFC8058, January 2017, . [RFC8098] Hansen, T., Ed. and A. Melnikov, Ed., "Message Disposition Notification", STD 85, RFC 8098, DOI 10.17487/RFC8098, February 2017, . [RFC8620] Jenkins, N. and C. Newman, "The JSON Meta Application Protocol (JMAP)", RFC 8620, DOI 10.17487/RFC8620, July 2019, . [RFC9051] Melnikov, A., Ed. and B. Leiba, Ed., "Internet Message Access Protocol (IMAP) - Version 4rev2", RFC 9051, DOI 10.17487/RFC9051, August 2021, . [RFC9078] Crocker, D., Signes, R., and N. Freed, "Reaction: Indicating Summary Reaction to a Message", RFC 9078, DOI 10.17487/RFC9078, August 2021, . [SchemaOrg] W3C Schema.org Community Group, "Schema.org", . Author's Address Hans-Joerg Happel audriga GmbH Email: happel@audriga.com URI: https://www.audriga.com Happel Expires 31 July 2023 [Page 10]