Releases: iv-org/invidious
Version 0.16.0: API Improvements and Annotations
Hello again! This past month has seen 116 commits from 13 contributors and a couple important changes I'd like to announce.
A privacy policy is now available here. I've done my best to explain things as clearly as possible without oversimplifying, and would very much recommend reading it if you're concerned about your privacy and want to learn more about how Invidious uses your data. Please let me know if there is anything that needs clarification.
I'm also very happy to announce that a Spanish translation has been added to the site. You can use it with ?hl=es or by setting es as your default locale. As always I'm extremely grateful to translators for making the site accessible to more people.
For Administrators
Invidious now supports server-to-server push notifications. This uses PubSubHubbub to automatically handle new videos sent to an instance, which is less resource intensive and generally faster. Note that it will not pull all videos from a subscribed channel, so recommended usage is in addition to channel_threads. Using PubSub requires a valid domain that updates can be sent to, and a random string that can be used to sign updates sent to the instance. You can enable it by adding use_pubsub_feeds: true to your config.yml. See Configuration for more info.
Unfortunately there are a couple necessary changes to the DB to support liveNow and premiereTimestamp in subscription feeds. Migration scripts have been provided that should be used automatically if following the instructions here.
You can now configure default user preferences for your instance. This allows you to set default locale, player preferences, and more. See #415 for more details and example usage.
For Developers
The fields API has been added with #429 and is now supported on all JSON endpoints, thanks @afrmtbl! Syntax is straight-forward and can be used to reduce data transfer and create a simpler response for debugging. You can see an example here. I've been quite happy using it and hope it is similarly useful for others.
An /api/v1/annotations/:id endpoint has been added for pulling legacy annotation data from this archive, see below for more details. You can also access annotation data available on YouTube using ?source=youtube, although this will only return card data as legacy annotations were deleted on January 15th.
A couple minor changes to existing endpoints:
- A
premiereTimestampfield has been added to/api/v1/videos/:id - A
sort_byparam has been added to/api/v1/comments/:id, supportsnew,top.
More info is available in the documentation.
Annotations
I'm pleased to announce that annotation data is finally available from the roughly 1.4 billion videos archived as part of this project. They are accessible from the Internet Archive here or as a 355GB torrent, see here for more details. A corresponding /api/v1/annotations/:id endpoint has been added to Invidious which uses the collection from IA to provide legacy annotations.
Support for them in the player is possible thanks to this plugin developed by @afrmtbl. A PR for adding support to the site is available as #303. There's also an extension for overlaying them on top of the YouTube player (again thanks to @afrmtbl), and an extension for hooking into code still present in the YouTube player itself, developed by @tech234a.
I would recommend reading the official announcement for more details. I would like to again thank everyone that helped contribute to this project.
Finances
Donations
Expenses
- invidious-load1 (nyc1) : $10.00 (load balancer)
- invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
- invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-node5 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
- Total : $80.00
This past month the site saw a couple abnormal peaks in traffic, so an additional webserver has been added to match the increased load. The goal on Patreon has been updated to match the above expenses.
Thanks everyone!
Version 0.15.0: Preferences and Channel Playlists
The project has seen quite a bit of activity this past month. Large focus has been on fixing bugs, but there's still quite a few new features I'm happy to announce. There have been 133 commits from 15 contributors this past month.
As a couple miscellaneous changes, a couple nice screenshots have been added to the README, so folks can see more of what the site has to offer without creating an account.
The footer has also been cleaned up quite a bit, and now displays the current version, so it's easier to know what features are available from the current instance.
For Administrators
This past month there has been a minor release - 0.14.1 - which fixes a breaking change made by YouTube for their polymer redesign.
There have been several new features that unfortunately require a database migration. There are migration scripts provided in config/migrate-scripts, and the wiki has instructions for automatically applying them. I'll do my best to keep those changes to a minimum, and expect to see a corresponding script to automatically apply any new changes.
Administrator preferences have been added with #312, which allows administrators to customize their instance. Administrators can change the order of feed menus, change the default homepage, disable open registration, and several other options. There's a short 'how-to' here, and the new options are documented here.
An /api/v1/stats endpoint has been added with #356, which reports the instance version and number of active users. Statistics are disabled by default, and can be enabled in administator preferences. Statistics for the official instance are available here.
For Developers
/api/v1/channels/:ucid now provides an autoGenerated tag, which returns true for topic channels, and larger genre channels generated by YouTube. These channels don't have any videos of their own, so latestVideos will be empty. It is recommended instead to display a list of playlists generated by YouTube.
You can now pull a list of playlists from a channel with /api/v1/channels/playlists/:ucid. Supported options are documented in the wiki. Pagination is handled with a continuation token, which is generated on each call. Of note is that auto-generated channels currently have one page of results, and subsequent calls will be empty.
For quickly pulling the latest 30 videos from a channel, there is now /api/v1/channels/latest/:ucid. It is much faster than a call to /api/v1/channels/:ucid. It will not convert an author name to a valid ucid automatically, and will not return any extra data about a channel.
Preferences
In addition to administrator preferences mentioned above, you can now change your preferences without an account (see #42). I think this is quite an improvement to the usability of the site, and is much friendlier to privacy-conscious folks that don't want to make an account. Preferences will be automatically imported to a newly created account.
Several issues with sorting subscriptions have been fixed, and /manage_subscriptions has been sped up significantly. The subscription feed has also seen a bump in performance. Delayed notifications have unfortunately started becoming a problem now that there are more users on the site. Some new changes are currently being tested which should mostly resolve the issue, so expect to see more in the next release.
Channel Playlists
You can now view available playlists from a channel, and auto-generated channels are no longer empty. You can sort as you would on YouTube, and all the same functionality should be available. I'm quite pleased to finally have it implemented, since it's currently the only data available from the above mentioned auto-generated channels, and makes it much easier to consume music on the site.
There's also more discussion on improving Invidious for streaming music in #304, and adding support for music.youtube.com. I would appreciate any thoughts on how to improve that experience, since it's a very large and useful part of YouTube.
Finances
Donations
Patreon : $42.42
Liberapay : $30.97
Crypto : ~$0.00 (converted from BCH, BTC)
Total : $73.39
Expenses
invidious-load1 (nyc1) : $10.00 (load balancer)
invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
Total : $75.00
It's been very humbling to see how fast the project has grown, and I look forward to making the site even better. Thank you everyone.
Version 0.14.0: Community
This last month several contributors have made improvements specifically for the people using this project. New pages have been added to the wiki, and there is now a Matrix Server and IRC channel so it's easier and faster for people to ask questions or chat. There have been 101 commits since the last major release from 8 contributors.
It has come to my attention in the past month how many people are self-hosting, and I would like to make it easier for them to do so.
With that in mind, expect future releases to have a section for For Administrators (if any relevant changes) and For Developers (if any relevant changes).
For Administrators
This month the most notable change for administrators is releases. As always, there will be a major release each month. However, a new minor release will be made whenever there are any critical bugs that need to be fixed.
This past month is the first time there has been a minor release - 0.13.1 - which fixes a breaking change made by YouTube. Administrators using versioning for their instances will be able to rely on the latest version, and should have a system in place to upgrade their instance as soon as a new release is available.
Several new pages have been added to the wiki (as mentioned below) that will help administrators better setup their own instances. Configuration, maintenance, and instructions for updating are of note, as well as several common issues that are encountered when first setting up.
For Developers
There's now a pretty=1 parameter for most endpoints so you can view data easily from the browser, which is convenient for debugging and casual use. You can see an example here.
Unfortunately the /api/v1/insights/:id endpoint is no longer functional, as YouTube removed all publicly available analytics around a month ago. The YouTube endpoint now returns a 404, so it's unlikely it will be functional again.
Wiki
There have been a sizable number of changes to the Wiki, including a list of public Invidious instances, the list of extensions, and documentation for administrators (as mentioned above) and developers.
The wiki is editable by anyone so feel free to add anything you think is useful.
Matrix & IRC
Thee is now a Matrix Server for Invidious, so please feel free to hop on if you have any questions or want to chat. There is also a registered IRC channel: #invidious on Freenode which is bridged to Matrix.
Features
Several new features have been added, including a download button, creator hearts and comment colors, and a French translation.
There have been fixes for Google logins, missing text in locales, invalid links to genre channels, and better error handling in the player, among others.
Several fixes and features are omitted for space, so I'd recommend taking a look at the compare tab for more information.
Annotations Update
Annotations were removed January 15th, 2019 around15:00 UTC. Before they were deleted we were able to archive annotations from around 1.4 billion videos. I'd very much recommend taking a look here for more information and a list of acknowledgements. I'm extremely thankful to everyone who was able to contribute and I'm glad we were able to save such a large part of internet history.
There's been large strides in supporting them in the player as well, which you can follow in #303. You can preview the functionality at https://dev.invidio.us . Before they are added to the main site expect to see an option to disable them, both site-wide and per video.
Organizing this project has unfortunately taken up quite a bit of my time, and I've been very grateful for everyone's patience.
Finances
Donations
Expenses
invidious-load1 (nyc1) : $10.00 (load balancer)
invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
Total : $75.00
As always I'm grateful for everyone's contributions and support. I'll see you all in March.
0.13.1
Version 0.13.0: Translations, Annotations, and Tor
I hope everyone had a happy New Year! There's been a couple new additions since last release, with 44 commits from 9 contributors. It's been quite a year for the project, and I hope to continue improving the project into 2019! Starting off the new year:
Translations
I'm happy to announce support for translations has been added with a160c64. Currently, there is support for:
- Arabic (
ar) - Dutch (
nl) - English (
en-US) - German (
de) - Norwegian Bokmål (
nb_NO) - Polish (
pl) - Russian (
ru)
Which you can change in your preferences under Language. You can also add &hl=LANGUAGE to the end of any request to translate it to your preferred language, for example https://invidio.us/?hl=ru. I'd like to say thank you again to everyone who has helped translate the site! I've mentioned this before, but I'm delighted that so many people find the project useful.
Annotations
Recently, YouTube announced that all annotations will be deleted on January 15th, 2019. I believe that annotations have a very important place in YouTube's history, and announced a project to archive them.
I expect annotations to be supported in the Invidious player once archiving is complete (see #110 for details), and would also like to host them for other developers to use in their projects.
The code is available here, and contains instructions for running a worker if you would like to contribute. There's much more information available in the announcement as well for anyone who is interested.
Tor
I unfortunately missed the chance to mention this in the previous release, but I'm now happy to announce that you can now view Invidious through Tor at the following links:
kgg2m7yk5aybusll.onion
axqzx4s6s54s32yentfqojs3x5i7faxza6xo3ehd4bzzsg2ii4fv2iid.onion
Invidious is well suited to use through Tor, as it does not require any JS and is fairly lightweight. I'd recommend looking here and here for more details on how to use the onion links, and would like to say thank you to /u/whonix-os for suggesting it and providing support setting setting them up.
Popular and Trending
You can now easily view videos trending on YouTube with a16f967. It also provides support for viewing YouTube's various categories categories, such as News, Gaming, and Music. You can also change the region parameter to view trending in different countries, which should be made easier to use in the coming weeks.
A link to /feed/popular has also been added, which provides a list of videos sorted using the algorithm described here. I think it better reflects what users watch on the site, but I'd like to hear peoples' thoughts on this and on how it could be improved.
Finances
Donations
Expenses
- invidious-load1 (nyc1) : $10.00 (load balancer)
- invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
- invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
- Total : $75.00
What will happen with what's left over?
I believe this is the first month that all expenses have been fully paid for by donations. Thank you! I expect to allocate the current amount for hardware to improve performance and for hosting annotation data, as mentioned above.
Anything that is left over is kept to continue hosting the project for as long as possible. Thank you again everyone!
I think that's everything for 2018. There's lots still planned, and I'm very excited for the future of this project!
Version 0.12.0: Accessibility, Privacy, Transparency
Hello again, it's been a while! A lot has happened since the last release. Invidious has seen 134 commits from 3 contributors, and I'm quite happy with the progress that has been made. I enjoyed this past month, and I believe having a monthly release schedule allows me to focus on more long-term improvements, and I hope people enjoy these more substantial updates as well.
Accessability and Privacy
There have been quite a few improvements for user privacy, and improvements that improve accessibility for both people and software.
You can now view comments without JS with 19516ea. Currently, this functionality is limited to the first 20 comments, but expect this functionality to be improved to come as close to the JS version as possible. Folks can track progress in #204.
Invidious is now compatible with LibreJS, and provides license information here with 7f868ec. As expected, all libraries are compatible under the AGPLv3, and I'm happy to mention that no other changes were required to make Invidious compatible with LibreJS.
A DNT policy has also been added with 9194f47 for compatibility with Privacy Badger. I'm pleased to mention that here too no other changes had to be made in order for Invidious to be compatible with this extension. I expect a privacy policy to be added soon as well, so users can better understand how Invidious uses their data.
For users that are visually impaired, there is now a text CAPTCHA available so it's easier to register and login. Because of the simple front-end of the project, I expect screen readers and other software to be able to easily understand the site's interface. In combination with the ability to listen-only, I believe Invidious is much more accessible than YouTube. Folks can read #244 for more details, and I would very much appreciate any feedback on how this can be improved.
User Preferences
There have been a lot of improvements to preferences. Options for enabling audio-only by default and continuous playback (autoplay) have been added with e39dec9, with 4b76b93, respectively. Users can also now mark videos as watched from their subscription feed and view watch history by going to https://invidio.us/feed/history. I expect to add more information to history so that it's easier to use. Folks can track progress with #182. As with all data Invidious keeps, watch history can be exported here.
Users can now delete their account with b9c29bf. This will remove all user data from Invidious, including session IDs, watch history, and subscriptions. As mentioned above, it's easy to export that data and import it to a local instance, or export subscriptions for use with other applications such as FreeTube or NewPipe.
Translation and Internationalis(z)ation
Invidious has been approved for hosting by Weblate, available here. At the time of writing, translations for Arabic, Dutch, German, Polish, and Russian are currently underway. I would like to say a very big thank you to everyone working on them, and I hope to fully support them within around 2 weeks. Folks can track progress with #251.
Transperency and Finances
For the sake of transparency, I plan on publishing each month's finances. This is currently already done on Liberapay and Patreon, but there is not a total amount currently provided anywhere, and I would also like to include expenses to provide a better explanation of how patrons' money is being spent.
Donations
- Patreon: $43.60 (Patreon takes roughly 9%)
- Liberapay : $22.10
- Crypto : ~$1.25 (converted from BCH, BTC)
- Total : $66.95
Expenses
- invidious-load1 (nyc1) : $10.00 (load balancer)
- invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
- invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
- invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
- Total : $75.00
I'd be happy to provide any explanation where needed. I would also like to thank everyone who donates, it really helps and I can't say how happy I am to see that so many people find it valuable.
That's all for this month. I wish everyone the best for the holidays, and I'll see you all again in January!
Week 11: FreeTube and Styling
This past Friday I'm been very excited to see that FreeTube version 0.4.0 has been released! I'd recommend taking a look at the official patch notes, but to spoil a little bit here: FreeTube now uses the Invidious API for all requests previously sent to YouTube, and has also seen support for playlists, keyboard shortcuts, and more default settings (speed, autoplay, and subtitles). I'm happy to see that FreeTube has reached 500 stars on Github, and I think it's very much deserved. I'd recommend keeping an eye on the newly-launched FreeTube blog for updates on the project.
Quite a few styling changes have been added this past week, including channel subscriber count to the subscribe and unsubscribe buttons. The changes sound small, but they've been a very big improvement and I'm quite satisfied with how they look. Also to note is that partial support for duration in thumbnails have been added with #202. Overall, I think the site is becoming much more pleasing visually, and I hope to continue to improve it.
I've been very pleased to see Invidious in its current state, and I believe it's many times more mature compared to even a month ago. Changes have also started slowing down a bit as it's become more mature, and therefore I'd like to transition to a monthly update schedule in order to provide more comprehensive updates for everyone. I want to thank you all for helping me reach this point. I can't say how happy I am for Invidious to be where it is now.
Enjoy the rest of your week everyone, I'll see you in November!
Week 10: Subscriptions
This week I'm happy to announce that subscriptions have been drastically sped up with
35e63fa. As I mentioned last week, this essentially "caches" a user's feed, meaning that operations that previously took 20 seconds or timed out, now can load in under a second. I'd take a look at #173 for a sample benchmark. Previously features that made Invidious's feed so useful, such as filtering by unseen and by author would take too long to load, and so instead would timeout. I'm very happy that this has been fixed, and folks can get back to using these features.
Among some smaller features that have been added this week include #118, which adds, in my opinion, some very attractive subscribe and unsubscribe buttons. I think it's also a bit of a functional improvement as well, since it doesn't require a user to reload the page in order to subscribe or unsubscribe to a channel, and also gives the opportunity to put the channel's sub count on display.
An option to swap between Reddit and YouTube comments without a page reload has been added with
5eefab6, bringing it somewhat closer in functionality to the popular AlienTube extension, on which it is based (although the extension unfortunately appears now to be fragmented).
As always, there are a couple smaller improvements this week, including some minor fixes for geo-bypass with
e46e618 and 245d0b5, playlist preferences with 81b4477, and YouTube comments with 02335f3.
This coming week I'd also recommend keeping an eye on the excellent FreeTube, which is looking forward to a new release. I've been very lucky to work with @PrestonN for the past few weeks to improve the Invidious API, and I'm quite looking forward to the new release.
That's all for this week folks, thank you all again for your continued interest and support.
Week 9: Playlists
Not as much to announce this week, but I'm still quite happy to announce a couple things, namely:
Playback support for playlists has finally been added with 88430a6. You can now view playlists with the &list= query param, as you would on YouTube. You can also view mixes with the mentioned &list=, although they require some extra handling that I would like to add in the coming week, as well as adding playlist looping and shuffle. I think playback support has been a roadblock for more exciting features such as #114, and I look forward to improving the experience.
Comments have had a bit of a cosmetic upgrade with #132, which I think helps better distinguish between Reddit and YouTube comments, as it makes them appear similarly to their respective sites. You can also now switch between YouTube and Reddit comments with a push of a button, which I think is quite an improvement, especially for newer or less popular videos with fewer comments.
I've had a small breakthrough in speeding up users' subscription feeds with PostgreSQL's materialized views. Without going into too much detail, materialized views essentially cache the result of a query, making it possible to run resource-intensive queries once, rather than every time a user visits their feed. In the coming week I hope to push this out to users, and hopefully close #173.
I haven't had as much time to work on the project this week, but I'm quite happy to have added some new features. Have a great week everyone.
Week 8: Mixes
Hello again!
Mixes have been added with 20130db, which makes it easy to create a playlist of related content. See #188 for more info on how they work. Currently, they return the first 50 videos rather than a continuous feed to avoid tracking by Google/YouTube, which I think is a good trade-off between usability and privacy, and I hope other folks agree. You can create mixes by adding RD to the beginning of a video ID, an example is provided here based on Big Buck Bunny. I've been quite happy with the results returned for the mixes I've tried, and it is not limited to music, which I think is a big plus. To emulate a continuous feed provided many are used to, using the last video of each mix as a new 'seed' has worked well for me. In the coming week I'd like to to add playback support in the player to listen to these easily.
A very big thanks to @flourgaz for Docker support with #186. This is an enormous improvement in portability for the project, and opens the door for Heroku support (see #162), and seamless support on Windows. For most users, it should be as easy as running docker-compose up.
I've spent quite a bit of time this past week improving support for geo-bypass (see #92), and am happy to note that Invidious has been able to proxy ~50% of the geo-restricted videos I've tried. In addition, you can now watch geo-restricted videos if you have dash enabled as your preferred quality, for more details see #34 and #185, or last week's update. For folks interested in replicating these results for themselves, I'd take a look here for the script used, and here for a list of videos restricted in the US.
1080p has seen a fairly smooth roll-out, although there have been a couple issues reported, mainly #193, which is likely an issue in the player. I've also encountered a couple other issues myself that I would like to investigate. Although none are major, I'd like to keep 1080p opt-in for registered users another week to better address these issues.
Have an excellent week everyone.