Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental: Embed videos #1465

Closed
wants to merge 6 commits into from
Closed

Experimental: Embed videos #1465

wants to merge 6 commits into from

Conversation

facelessuser
Copy link
Owner

@facelessuser facelessuser commented Sep 23, 2021

Embedding Video/Audio File Types

Resolves #897

This is an experimental branch that explores syntax to embed media. The idea is to use the existing ![]() syntax that has been historically used for images, and instead, use it for media in general.

![](mov.webm)

"Img" links are processed as normal, and once attr_list has processed them, we intercept them in a Treeprocessor. There we identify any links that appear to be media links. Once we've found such links we wrap them in their appropriate element tag (<video> or <audio>). Inside the "media" tag, the video or audio file with be specified via a <source> element.

We do have a default mapping for some known extension types, but some can be either audio or video, if our default assumption is incorrect, the user should specify the type attribute using attr_list.

Since these are processed after attr_list, this means we can apply whatever attributes we want to the element(s) and leverage those. When we are processing the attributes attached to the given tag, we can sort them based on whether they are appropriate for the <source> (or <track> -- subtitles) element or the parent container.

For instance, here we set preload to metadata for the media container.

![](mov.webm){preload=metadata}

But here, since webm can be ambiguous, and we default to assuming video, we can specify on the <source> element that the type is audio/webm. Since we are aware of which attributes are <source> specific, we can specify them together and it will all get sorted out.

![](mov.webm){type="audio/webm" preload=metadata}

Anything non-specific to either element will be attached at the parent container level.

If desired, a user can specify globally, via options, a set of default attributes to apply on every media object. If a user explicitly sets that attribute on a media object, the global value will of course be overridden.

For all videos, we will specify a fallback, the last resort for when the video or audio isn't handled. It is a simple message that allows the user to try and download the file directly.

<video controls preload="metadata">
<source src="https://dl8.webmfiles.org/elephants-dream.webm" type="video/webm">
<a download="" href="https://dl8.webmfiles.org/elephants-dream.webm">https://dl8.webmfiles.org/elephants-dream.webm</a>
</video>

If the user provides an alt, we will use that to provide a description -- in the case that no alt is specified and a title is, title can double as an alt. Assuming none of these are provided, the fallback will simply be the source link.

<video controls preload="metadata"><source src="https://dl8.webmfiles.org/elephants-dream.webm" type="video/webm"><source src="./images/elephants-dream.webm" type="video/webm">
<a download="" href="https://dl8.webmfiles.org/elephants-dream.webm">Elephant has a dream</a>
</video>

Lastly, we provide a mechanism for fallback sources. If multiple, same types of media (audio or video) are grouped together, the first one will be assumed as the main video, and all others will be assumed fallbacks if they specify the fallback attribute. This can be used to specify multiple video types in case a browser does not support one type. Additionally, .vtt sources can be used and will be included in whatever the parent is, audio or video -- .vtt sources are inserted as <track> elements, not <source>.

![A movie about stuff](mov.webm)
![](mov.mp4){fallback}
<video controls= preload="metadata">
<source src="mov.webm" type="video/webm">
<source src="mov.mp4" type="video/mp4">
<a download="" href="mov.webm">mov.webm</a>
</video>

Source-specific attributes will apply to each source, but the "media" and general attributes will be taken only from the main source.

![Audio about stuff](audio.mp3){loop=''}
![](audio.webm){fallback type="audio/webm"}
<audio controls loop preload="metadata">
<source src="audio.mp3" type="audio/mpeg">
<source src="audio.webm" type="audio/webm">
<a download="" href="audio.mp3">Audio about stuff</a>
</audio>

Embedding Media Services

Resolves #896

Embedding video services such as Youtube uses the same syntax as the above ![some alt text](youtube.link). These media links will be converted to an appropriate embedded link and wrapped in an <iframe>.

<iframe> have no concept of alt, but it is generally encouraged that a title is applied. So in this case, alt will double as a title if title is not defined. If neither alt or title is defined, something generic like YouTube video player will be provided via the default configuration.

![Random YouTube link](https://www.youtube.com/embed/ZTjzHsAU5-U)
<iframe allowfullscreen="" frameborder="0" src="https://www.youtube.com/embed/ZTjzHsAU5-U" title="Random YouTube link"></iframe>

Things are abstracted in such a way that someone could provide their own function for a specific video service, but as of right now, only: Vimeo, YouTube, and Dailymotion are supported out of the box.

Global default attributes can be defined per media service.

Feedback is welcome!

@gir-bot gir-bot added S: needs-review Needs to be reviewed and/or approved. C: magiclink Related to the magiclink extension. C: source Related to source code. labels Sep 23, 2021
@facelessuser
Copy link
Owner Author

Maybe we should split this into a new extension? It doesn't necessarily rely on anything in MagicLink really. Might keep things cleaner and easier to maintain.

@facelessuser
Copy link
Owner Author

I guess it is link related 🤷🏻. We can maybe at least break MagicLink into submodules and confine the media stuff to one submodule 🤔 .

@facelessuser
Copy link
Owner Author

Hmm. Not every user may desire the default fallback to be in Engllish...maybe we should make the message configurable, or strip it down to only what is provided by alt.

@gir-bot gir-bot added C: docs Related to documentation. C: infrastructure Related to project infrastructure. and removed C: magiclink Related to the magiclink extension. labels Sep 23, 2021
@facelessuser
Copy link
Owner Author

Move embedding logic to a new extension called Embed (need a better name?).

No more complex fallback message. Just show a download link with "alt" text or the URL. Most browsers support HTML5 video at this point. People will most likely never see this, and if they do, that's what they'll get.

@gir-bot gir-bot added the C: magiclink Related to the magiclink extension. label Sep 24, 2021
@flywire
Copy link
Contributor

flywire commented Sep 26, 2021

Feedback is welcome!

If you are interested in captions this is very much a use I had envisaged for https://github.com/flywire/caption (assuming on a line by itself) but I never managed to apply it to user-specified content types. The unresolved issue in caption is overriding defaults.

@facelessuser
Copy link
Owner Author

@flywire, I understand the idea behind this logic, but doing it this way will nest <figure> under <p> which isn't really proper. But the reasoning behind triggering off of title does make sense.

That is because ![]() is handled as inline content as <img> is considered inline. <figure> should really be handled at the block level. I currently do something like this. But yeah, it is more verbose.

Example audio

<figure markdown=1>

![](./images/file_example_MP3_700KB.mp3)

<figcaption markdown=1>This is a **song**</figcaption>
</figure>

In order to properly handle images, videos, and audios in captions, we'd either need some kind of figure container syntax or handle ![]() at the block level, which may be weird as then you can't really insert images as inline any more.

At this time, I would probably pass on auto-captioning as I think this would need more thought. While what you suggest would work, it is probably not "proper". Granted, that may not matter to some, but it's just something to note.

Use alt as a description in the message
Also, use sibling videos/audio if fallback is defined
- Allow any extension as long as a MIME type of audio/video is provided
- Remove fallback message. Only show a download link with alt text if
  provided an if not, the link will show the URL.
@BjoernSchotte
Copy link

As I'm interested in the embed feature, is there anything I can help with?

@facelessuser
Copy link
Owner Author

I think for starters, getting some general feedback on the current implementation.

@facelessuser
Copy link
Owner Author

I'm closing this issue PR for now. Moving forward, I think we'll be exploring such things with the new general-purpose blocks that are currently in beta.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C: docs Related to documentation. C: infrastructure Related to project infrastructure. C: magiclink Related to the magiclink extension. C: source Related to source code. S: needs-review Needs to be reviewed and/or approved.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Video Embeds [Feature Request] YouTube Embeds
4 participants