Onlydarksets

I posted a method for converting SageTV files to iPod/iPhone compliant files a while back, but a number of the files weren’t converting properly.  I also had problems with mencoder cutting commercials out of the files.  So, I retooled the approach, and I am now getting 100% success, using the approach outlined below.

NOTE: Several of the non-alphanumeric characters do not copy and paste correctly.  If you have any problems (especially with the RunExternal actions), check that the quotes and dashes are correct.  Thanks to Steve for pointing that out in the comments.

Tools Used

• SageTV: records TV shows
• DVRMSToolbox: monitors recording folder and orchestrates conversion
  • SageTV Web Interface: provides metadata to DVRMSToolbox
  • ComSkip or ShowAnalyzer: finds commercials in TV show files for DTb
• mencoder (bundled with mplayer): removes commercials
• Handbrake: converts to h.264/aac in mp4 container
• AtomicParsley: writes metadata to mp4
• MoveOut: puts converted files in the “right” place

My Files

• “Convert to Zune/iPod/iPhone” Profile
• “Add to iTunes” Javascript (NOTE: I have tested this as a standalone script, but it is not incorporated into the workflow below)

Step 1: Installation and SageTV

Obviously, you need to have all of the above tools installed and SageTV configured and running and recording shows.  DVRMSToolbox (DTb) should be running and monitoring the SageTV recorded TV folder(s).

Step 2: Setup the DTb Profile

The problem I experienced with mencoder was due to the stdout mucking up DTb.  Turns out DTb has a RunExternalNoStatus action that ignores the output, thus obviating the problem.  Using this action for AtomicParsley sped up that app’s processing by tenfold (due to the huge amount of output of AP).  So, as a general rule, if you have having trouble getting an external app to run well (or at all) in DTb, try RunExternalNoStatus.

Now, for the meat of the process.  DTb orchestrates the various external applications.  The logical flow is:

1. Scan the TV show file for commercials.
2. Use mencoder to create a new TV show file without the commercials.
3. Use Handbrake to convert the file from MPEG to MP4 (h.264/AAC).
4. Use AtomicParsley to write metadata to the MP4 file (e.g., show, episode, air date, etc.).
5. Use DTb to rename the show to a friendly format (e.g., “Fringe – 2008-12-03 – Safe.mp4″).
6. Use MoveOut to move the shows to the correct folders (if necessary).

As implemented, here are the Actions and parameters:

1. FWThrottle: limit the number of files that are processed simultaneously
   1. UseDefaultSettings: <checked>
   2. ContextFile: InputFile
   3. Throttle: Checked
   4. ContinueOnFail: <unchecked>
2. CreateTimeout: sets a timeout – trust me, the first time mencoder hangs on “Dora the Explorer” and you don’t have “Lost” waiting for you in the morning like you were expecting, you’ll be glad.
   1. Timeout Minutes: 240
   2. ContinueOnFail: <unchecked>
3. ChangeContextValue: create a new context to store the location of the commercial file created by ComSkip/SA
   1. SourceContext: InputFile
   2. TargetContext: SageEDLFile
   3. CreateNewContext: <checked>
   4. NewContextName: SageEDLFile
   5. FindRegex: ^((.*\\)(.*)\.(.*))\z
   6. ReplaceRegex: $2$3.edl
   7. ContinueOnFail: <checked> (it’s OK to keep going if we fail for some reason – we just won’t get commercials cut out)
4. ChangeContextValue: update OutputFile context to allow for MP4 extension
   1. SourceContext: OutputFile
   2. TargetContext: OutputFile
   3. CreateNewContext: <unchecked>
   4. FindRegex: ^((.*\\)(.*)\.(.*))\z
   5. ReplaceRegex: $2$3.mp4
   6. ContinueOnFail: <unchecked>
5. ChangeContextValue: create a new context to specify location of the resulting commercial-free MPEG file
   1. SourceContext: OutputFile
   2. TargetContext: CutMpegFile
   3. CreateNewContext: <checked>
   4. NewContextName: CutMpegFile
   5. FindRegex: ^((.*\\)(.*)\.(.*))\z
   6. ReplaceRegex: $2$3.mpg
   7. ContinueOnFail: <unchecked>
6. FindCommercials: run ComSkip/SA to find commercials and write to EDL file
   1. UseDefaultSettings: <checked>
   2. ContextFile: InputFile
   3. ContinueOnFail: <checked> (again, worst case is we have commercials in the final MP4 file)
7. RunExternalNoStatus: run mencoder to cut out commercials
   1. ProcessName: <fullpath>\mencoder.exe
   2. ReadError: <unchecked>
   3. ArgumentsContext: “%InputFile%” -edl “%SageEDLFile%” -oac copy -ovc copy -of mpeg -o “%CutMpegFile%.mpg” -mpegopts muxrate=36000 -quiet
   4. ContinueOnFail: <unchecked>
8. RunExternalNoStatus: run Handbrake to convert to MP4
   1. ProcessName: <fullpath\handbrakeCLI.exe
   2. ReadError: <unchecked>
   3. ArgumentsContext: <experiment with the Handbrake GUI using a sample file to get the settings you want>
   4. ContinueOnFail: <unchecked>
9. MigrateMetadataToContext: workaround to get a nicely formatted “original air date”
   1. Metadata Pattern: %originalAirDate%
   2. TargetContext: OutputFile
   3. CreateNewContext: <checked>
   4. NewContextName: DateAndEpisode
   5. FindRegex: ((\d+-)+\d+)
   6. ReplaceRegex: $1
   7. ContinueOnFail: <unchecked>
10. RunExternalNoStatus: run AtomicParsley to add basic metadata to MP4
    1. Processname: <fullpath>\AtomicParsley.exe
    2. ReadError: <unchecked>
    3. ArgumentsContext: “%OutputFile%” –genre “TV Shows” –stik “TV Show” –TVNetwork %channelName% –TVShowName “%Title%” –TVEpisode “%episode%” –description “%description%” –category=”%category%”
    4. ContinueOnFail: <checked> (no need to crash just for the metadata)
11. AdvancedFileRename: rename the output file to something meaningful
    1. ContextFile: OutputFile
    2. NamePattern: %Title% – %originalAirDate% – %episode%
    3. WhenFileExists: AppendGuid
12. FileOperation: delete temporary MPEG file
    1. ContextFile: CutMpegFile
    2. DestinationPath: <blank>
    3. WhatOperation: Delete
13. FileOperation: move MP4 to the output directory
    1. ContextFile: OutputFile
    2. DestinationPath: <destination directory>
    3. WhatOperation: Move

Step 3: Move the files

I use MoveOut to then put some files in the kids’ directory, some in the wife’s Zune directory, and some in my directory.  You could do this all with DTb, but I find it easier to use a separate app.

What’s Next?

Zune users are all set – the ZMP automatically monitors folders, so the next time you sync your Zune, it will pick up the new MP4 files.  iPod/iPhone users, however, have to manually import the folder prior to each sync.  There are two options for getting around this.  First, you can add an extra “RunExternal” action that runs a Javascript to import the files into iTunes (available in the linked download, courtesy of Coyote).  Second, you can try to get the iTunes Library Updater to work, although there are mixed results with iTunes 7+.  I don’t have iTunes installed on my SageTV server, so this isn’t really an issue for me.

jelwood is working on an alternate approach using Quicktime chapters instead of actually cutting out the commercials.  Because not every medium I am watching these on supports Quicktime chapters (including SageTV and the Zune), I am sticking with the mencoder route.  However, this is a better option if storage space is not an issue or you have problems with ComSkip/SA getting the commercials “not quite right”.

Also, jelwood has created a patched version of AtomicParsley that allows you to set the sort order to the original air date.  I include the air date in my titles out of preference, so this is not necessary for me, but incorporating this is trivial.

Anything else?

EDIT (2009-03-18): DTb 1.2.1.6 added the MigrateMetadataToContext, which makes it easier to format the original air date to a friendlier format.