In the learning path for the population of my media collection as it transitions from disc based media to a disk drive, I have discovered a few truths that I will share.
There are some useful tools to “rip” your media. These are the essential first step in the process, to get your media from the source into a format that can be worked with.
The next topic is what sort of transcoding you want to do. Here is where some of my early choices in media ripping have come back to haunt me. In prior efforts, I was merely attempting to get media in a format that looked good when loaded onto my iPad. That meant that there was a pretty aggressive compression applied to the files, making them both storage thrifty, and acceptable quality for playback.
Alas, those files, when streamed via my Tivo to my 52″ LCD TV, looked like, uh, shit. No other way to put it, the bit rate is just too small for decent viewing quality. Any full length movie that is less than 1G is likely to be a muddy compromise.
Historical Aspects
Early on, I wanted to have copies of media for my computer. In the old days there were some tools that I used to create DVD rips. Some open source apps that worked but had some flaws. That changed in 2009 or so when I discovered this program called “MacTheRipper”. It just worked, no fiddling with muxing, no mis-sync’d audio tracks, etc. It allowed the direct creation of unencrypted VOB files that you could process with other utilities to create AVI’s or MPG files.
For those keeping track, VOB files are MP2 files, pretty much un-compressed video streams, and what the cable companies used for transmitting “digital” cable until the 2010’s when mp4 streams became the normal.
To be able to shift them to my iPad, or to even view them conveniently on my computer, I needed to get them into another format. Enter the first utility in the workflow stream, Handbrake. This is a handy collection of back end utilities (like ffmpg, and others) with a handy GUI front end. Suddenly a lot of the magical incantations were no longer needed, and typos wouldn’t ruin your efforts (oh, how many 2+ hour conversions failed due to an incorrect flag…)
At this time, the workflow was:
- Rip the DVD to a VOB file
- Use Handbrake to convert to an appropriate format
- Use iTunes to get it on my iPad, or just watch it on my Mac
That worked, and when Handbrake added presets, there were options to quickly set them up for the various use (optimized resolutions/bit rates etc.)
This failed when the transition to Bluray Discs happened.
Early Bluray processing
As with DVD’s and the CSS encryption, Bluray discs presented some initial trouble. The key for the encryption was not fixed, as it was for DVD’s, but each manufacturer had their own key, and the discs had encryption that could be changed, as long as the manufacturer keys stayed hidden. And just as with DVD’s that didn’t stand up. (one would think that the DRM folks would give up the fight, but I suspect that it will be a never ending battle, and the general population will always figure out a way.)
The earliest BD rippers would create separate streams for audio and for the video, to create a containerized mp4 file required some additional tools to multiplex the streams. And in 2010 or so, they sucked. They sucked big tool. I was almost at the point of giving up, then two developments happened, first was AnyDVD offered a BD ripper that worked. It gave you a giant uncompressed file, but it could be reprocessed with Handbrake to make it a tolerable size.
The second was found on an obscure forum that referenced this new tool called MakeMKV. This is a utility that can start with a disc, read and de-encrypt, and then convert to an MKV file. This was the breakthrough, a 1-step process to de-encrypt, and to create an open, standards based, format for media.
Suddenly, I could create media files of near perfect quality, at a greatly reduced storage size, of all my discs.
Today
Regardless of the disc format, I toss it in the drive, run MakeMKV, and dump the file to a staging or spool folder.
From there, since I am just tossing it into the Plex repository, I just leave it as a full resolution mkv file, and let Plex transcode as needed.
For example, my Tivo (a Roamio model) has a Plex app. It limits the bitstream to 4mbit/second, so Plex transcodes it on the fly. Same when I use the Plex app on my iPad.
DVD’s seem to top out at about 3mbit/second (this is empirical, not theoretical) and they look pretty amazing streamed. A 100 minute DVD processed by Handbrake, with quality cranked up, and some de-interlacing, creates about a 2.5G file. The mkv file is about 4-5 GB.
BD’s converted and transcoded are 4 – 8 GB, down from 10 – 35GB or so. Unfortunately, using Handbrake to process the mkv files into mp4 files strips out all the subtitles. If you want to have subtitles, it will only burn them into the video (that is, they are imprinted into the video stream. Boo) So, my new workflow is to just leave BD’s as the original mkv file. Then the subtitles work (essential for some movies.)
My current collection is approaching 1.3TB in size, but virtually all my discs have been processed. The remnants are the guitar teaching videos, and some other miscellaneous discs.
Lessons Learnt
- Optimizing for space made a lot of sense in the 2009 era, where disks were large, but not huge. Today, with 4TB drives costing about $120, that is a much less important consideration.
-
Handbrake can be used on DVD’s directly if you add the libdvdcss libraries. Easy to do, and makes it one pass to rip from a DVD to a media file. Convenient, but be sure to use good settings. I had to redo a bunch of encodings because I had some settings wrong.
-
Animation DVD’s (think “Family Guy” and The “Simpsons”) are somewhat tricky. The compression algorithms work well, almost too well, and you get some really weird artifacts. I ended up re-ripping virtually all of them.
-
If you care about subtitles and are processing BD’s, Handbrake isn’t your friend. Stick to MakeMKV.
-
1.5 TB is a shitload of data. Back it up (and I have, to my moster Drobo and soon to Amazon’s glacial storage).
-
A NUC, 7th generation Core i5 version, makes a very capable streaming server. Less than $300, add your own memory and disk, and hang external drives to host the media, and you are in business. N.B.: Linux is solid, but the display manager on Ubuntu 16.04 LTS hangs. A lot. I think it is tied to runaway processes in Chrome. Fucking Google.
-
USB2.0 external drive, with a WD Green 2TB disk is fast enough. You do not need USB3 speeds, or 7200 RPM drives to serve up video. Cheap and reliable disks for the WIN!
Plex is just amazing. It works, it is easy to configure, it is simple to navigate on my Tivo, and I kick myself for not doing this sooner. I had long toyed with using my old Core i7 based system for this, but kept putting it off. Foolish.
The current workflow is use MakeMKV to rip the file, name it appropriately, and then drop it into the appropriate folder on my server. Easy-peasy
