You can now use the media9 package to do so! Here's an example. You can embed video files, or use a YouTube link. Note that the videos won't play in Overleaf's previews; instead, download the PDF and open it in a suitable viewer e.g. Adobe Reader.

[edit] Overleaf guides

[edit] LaTeX Basics

[edit] Mathematics

[edit] Figures and tables

[edit] References and Citations

[edit] Languages

[edit] Document structure

[edit] Formatting

[edit] Fonts

[edit] Presentations

[edit] Commands

[edit] Field specific

[edit] Class files