add option to access original message #2125

r10s · 2020-12-31T12:50:28Z

while dealing with email-compatibility (https://github.com/orgs/deltachat/projects/31 and also on one-to-one-talks), it turns out that we need an option to read the original message more urgently as i thought before.

while this is clear for html-only-mails, unfortunately, also plain-text-alternatives as offered eg. by some newletters are often very poor, having huge links etc.
even if we try harder to detect quotes, convert things etc. we will never 100%, there may always be some context loss

therefore, the idea is to have a function to check if a message is "maybe bad" (~~dc_msg_is_mime_modified()~~ dc_msg_has_html()) and another function to get the message with "rich content", always as html (~~dc_get_original_mime_html()~~ dc_get_msg_html()). the ui shall offer a button as "Show full message..." or so (there is already a pr for that on android). as the html-part is not shown and not even processed before the function is called, the fear or tracking seems to be already mitigated enough, however, for a first try, i do not want to make that too complicated - also, many html-mails are not really working without reading external content and you will not open "random" mails, but, as of now, only from senders you already started a chat with.

having the option to access the "original" message, finallly also allows us to do handle the generated "short" mail more freely - eg. we can truncate long messages, and stop display before a links spanning 20 lines etc.

ui-wise, this is pretty easy to implement, and already done for android (deltachat/deltachat-android#1763 - there are also some images). desktop already has a similar hack, detecting such messages by scanning for "[...]" and offering a link to the info screen; this should be replaced by the api mentioned above.

open things:

handle encrypted messages (can also go to a subsequent pr)

r10s · 2021-01-09T13:52:16Z

src/dc_receive_imf.rs

@@ -854,7 +867,9 @@ async fn add_parts(
                } else {
                    match ephemeral_timer {
                        EphemeralTimer::Disabled => 0,
-                        EphemeralTimer::Enabled { duration } => rcvd_timestamp + i64::from(duration)
+                        EphemeralTimer::Enabled { duration } => {


this was rewritten by cargo fmt, not sure why. it is not related to this pr.

r10s · 2021-01-09T13:54:02Z

src/dc_receive_imf.rs

+                        mime_headers.clone()
+                    } else {
+                        None
+                    },


we already had a field that was used for exactly the purpose of saving the raw mime message, although the name mime_headers is a bit misleading - i meanwhile read it as "mime and headers" :)

mime_headers is misleading indeed. Is save_mime_headers used for anything but debugging? Can it be removed?

save_mime_headers was introduced at deltachat/deltachat-core#388, not sure if it is actually in used, however, it is not much code, also, with this pr not even a wasted column anymore - so i would not remove it in this pr without further discussion.

r10s · 2021-01-09T13:55:49Z

src/simplify.rs

-pub fn simplify(mut input: String, is_chat_message: bool) -> (String, bool, Option<String>) {
+pub fn simplify(mut input: String, is_chat_message: bool) -> (String, bool, bool, Option<String>) {
+    let mut is_cut = false;
+


yip, i also thought that four return values is a value to think over doing that in another way, however, i did not want to make this pr even larget - also, beside tests, simplify() us called only at one place, so i'd leave it as is for now :)

src/originalhtml.rs

r10s · 2021-01-09T14:11:14Z

src/originalhtml.rs

+                    } else if mimetype == mime::TEXT_PLAIN {
+                        if let Ok(decoded_data) = mail.get_body() {
+                            self.plain = Some(decoded_data);
+                            self.format_flowed =


these parameter-getting lines are similar to the ones from mimeparser.rs, maybe we can move them to format_flowed.rs, however, as not much more code from format_flowed.rs is reusable (format_flowed.rs converts to plain text ignoring quotes where we want to convert to html and regard quotes), i think, it is not worth the noise.

deltachat-ffi/deltachat.h

Simon-Laux · 2021-01-09T16:31:12Z

I'm not sure if I agree calling it "dc_get_original_mime_html()" when It's clearly returning a modified version to work in browsers / converts text emails to html. Maybe "dc_get_original_mime_as_html()" would be a bit better?
Also the wording of the documentation could emphasize a bit more that the returned html is not to be trusted.
"NEVER TRUST USER INPUT"

Anyways besides that minor naming and documentation wording stuff:
The PR looks good to me (reading code on github, haven't tested it yet.)

…mime_modified-flag stays as-is, however

…handling for saved mime-structure

r10s · 2021-01-10T16:56:06Z

@Simon-Laux @link2xt i targeted most of your comments, thank you very much for that.

moreover, i added a commit that explicitly avoid setting mime-modified- aka has-html-flag for encrypted messages.

i'd like to handle/discuss that in a subsequent pr, there is probably no huge intersection between html-mails and encrypted-mails anyway.

deltachat-ffi/deltachat.h

src/html.rs

deltachat-ffi/src/lib.rs

src/dc_receive_imf.rs

src/html.rs

src/mimeparser.rs

link2xt · 2021-01-11T09:55:19Z

src/mimeparser.rs

-                        };
+
+                        self.is_mime_modified = self.is_mime_modified
+                            || ((is_forwarded || is_cut || top_quote.is_some())


top_quote.is_some() is not necessary. We display the top quote just fine, no reason to show "read more" for each unencrypted message with a quote. If we have removed User ... wrote: top line, then is_cut should be set. remove_top_quote has an is_quoted_headline variable internally which tracks this.

i also thought over that bit.

we do that only for non-delta-messages anyway - and then, there is the risk that the quote is not really a quoted messages but sth. the user entered/edited manually, so a tap on the quote cannot go to the original message and there is no chance the see the full quote (in the message, only the first lines are shown)

this risk is potentially also there also for delta-to-delta communication, eg. if the message was deleted, however, it is much larger when the user can edit the quote manually.

so, i'd leave that in for now, it is better to show one "full message" button too much than one too few :)

src/simplify.rs

link2xt · 2021-01-11T15:52:32Z

src/plaintext.rs

+        let lines = split_lines(&self.text);
+
+        let mut ret =
+            "<!DOCTYPE html>\n<html><head><meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" /></head><body>\n".to_string();


html, head, meta and body tags are all not necessary for valid HTML5 (identified by <!DOCTYPE html>), could be removed.

At least, is meta necessary here?

without the meta-tag, some browsers do display utf-8 wrong, so yes, that is needed.

as <meta> needs to be placed in <head> i would also leave them in, it is just a few characters and i am not sure, if all browsers will handle things correctly when we rely on html5 here.

standard footers meanwhile go the "contact status", so they are no longer a reason to trigger "full message view". this was already discussed when the HTML view was introduced at #2125 however, forgotten to change when the "contact status" was added at #2218 this pr will result in a cleaner chat view with less "Show Full Message..." buttons and will also save some storage (in fact, i came over that when reviewing #4129 )

r10s mentioned this pull request Dec 31, 2020

add option to view full (html) message deltachat/deltachat-android#1763

Merged

5 tasks

r10s force-pushed the save-original branch 2 times, most recently from 7944936 to ce925dd Compare January 1, 2021 16:09

r10s marked this pull request as draft January 1, 2021 16:24

r10s force-pushed the save-original branch 5 times, most recently from 10123cb to a8fb946 Compare January 2, 2021 13:14

r10s mentioned this pull request Jan 2, 2021

forbid active content when displaying html-messages #2127

Closed

r10s force-pushed the save-original branch 5 times, most recently from 114f0d5 to 89c644f Compare January 5, 2021 21:55

r10s mentioned this pull request Jan 5, 2021

add reference to RFC 3676 to standards.md #2130

Merged

r10s force-pushed the save-original branch 4 times, most recently from a62198d to 05d21f2 Compare January 9, 2021 10:26

r10s commented Jan 9, 2021

View reviewed changes

src/originalhtml.rs Outdated Show resolved Hide resolved

r10s commented Jan 9, 2021

View reviewed changes

r10s marked this pull request as ready for review January 9, 2021 14:12

Simon-Laux reviewed Jan 9, 2021

View reviewed changes

deltachat-ffi/deltachat.h Outdated Show resolved Hide resolved

Simon-Laux reviewed Jan 9, 2021

View reviewed changes

deltachat-ffi/deltachat.h Outdated Show resolved Hide resolved

r10s added 4 commits January 10, 2021 16:18

rename get_original_mime_html() to more simple get_html()

64ab118

rename api is_mime_modified() to more simple has_html(); internally, …

71de658

…mime_modified-flag stays as-is, however

rename MimeS to MimeMultipartType

607f351

do not set mime-modified flag for encrypted messages that need extra-…

77e2c0a

…handling for saved mime-structure

fix typo

d13da59

link2xt mentioned this pull request Jan 11, 2021

Add API to check if a message failed to decrypt #2131

Closed

link2xt reviewed Jan 11, 2021

View reviewed changes

r10s added 8 commits January 11, 2021 13:24

move get_msg_html() to MsgId.get_html()

6136628

incorporate more documentation suggestions from review

bb7bae1

remove unused return value from collect_texts_recursive()

ef5ce31

avoid mime_modified being mutable in write-parts-loop

d554467

move 'use futures::future::FutureExt' atop of html.rs

481e5fd

move attributes defining plain-text to a dedicated structure

f7bf527

more PlainText to separate file

3c6ce41

escape cid when building regex

e31dccf

link2xt reviewed Jan 11, 2021

View reviewed changes

let dc_get_msg_html() return NULL when calling with bad param

6492eef

link2xt approved these changes Jan 11, 2021

View reviewed changes

r10s merged commit e2688f6 into master Jan 11, 2021

r10s deleted the save-original branch January 11, 2021 16:40

link2xt mentioned this pull request Jan 15, 2021

images from multipart/related easily flood chat #1977

Closed

r10s mentioned this pull request Jan 19, 2021

forward html-messages properly #2151

Merged

4 tasks

This was referenced Jan 28, 2021

add option to view full (html) message deltachat/deltachat-desktop#2078

Closed

add option to view full (html) message deltachat/deltachat-ios#1046

Closed

r10s mentioned this pull request Mar 12, 2023

"full message view" not needed because of footers #4151

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add option to access original message #2125

add option to access original message #2125

r10s commented Dec 31, 2020 •

edited

Loading

r10s Jan 9, 2021

r10s Jan 9, 2021

link2xt Jan 11, 2021

r10s Jan 11, 2021 •

edited

Loading

r10s Jan 9, 2021

r10s Jan 9, 2021 •

edited

Loading

Simon-Laux commented Jan 9, 2021 •

edited

Loading

r10s commented Jan 10, 2021

link2xt Jan 11, 2021

r10s Jan 11, 2021

link2xt Jan 11, 2021

r10s Jan 11, 2021 •

edited

Loading

add option to access original message #2125

add option to access original message #2125

Conversation

r10s commented Dec 31, 2020 • edited Loading

r10s Jan 9, 2021

Choose a reason for hiding this comment

r10s Jan 9, 2021

Choose a reason for hiding this comment

link2xt Jan 11, 2021

Choose a reason for hiding this comment

r10s Jan 11, 2021 • edited Loading

Choose a reason for hiding this comment

r10s Jan 9, 2021

Choose a reason for hiding this comment

r10s Jan 9, 2021 • edited Loading

Choose a reason for hiding this comment

Simon-Laux commented Jan 9, 2021 • edited Loading

r10s commented Jan 10, 2021

link2xt Jan 11, 2021

Choose a reason for hiding this comment

r10s Jan 11, 2021

Choose a reason for hiding this comment

link2xt Jan 11, 2021

Choose a reason for hiding this comment

r10s Jan 11, 2021 • edited Loading

Choose a reason for hiding this comment

r10s commented Dec 31, 2020 •

edited

Loading

r10s Jan 11, 2021 •

edited

Loading

r10s Jan 9, 2021 •

edited

Loading

Simon-Laux commented Jan 9, 2021 •

edited

Loading

r10s Jan 11, 2021 •

edited

Loading