5 This script helps to convert video from one format to another.
6 This is useful for ripping DVD to mpeg4 video (XviD, DivX).
9 * automatic crop detection
10 * mp3 audio compression with resampling options
11 * automatic bitrate calculation based on desired target size
12 * optional interlace removal, b/w video optimization, video scaling
14 Run the script with no arguments to start with interactive prompts:
16 Run the script with the filename of a config to start automatic mode:
19 After Rippy is finished it saves the current configuation in a file called
20 'rippy.conf' in the local directoy. This can be used to rerun process using the
21 exact same settings by passing the filename of the conf file as an argument to
22 Rippy. Rippy will read the options from the file instead of asking you for
23 options interactively. So if you run rippy with 'dry_run=1' then you can run
24 the process again later using the 'rippy.conf' file. Don't forget to edit
25 'rippy.conf' to set 'dry_run=0'!
27 If you run rippy with 'dry_run' and 'verbose' true then the output generated is
28 valid command line commands. you could (in theory) cut-and-paste the commands
29 to a shell prompt. You will need to tweak some values such as crop area and bit
30 rate because these cannot be calculated in a dry run. This is useful if you
31 want to get an idea of what Rippy plans to do.
33 For all the trouble that Rippy goes through to calculate the best bitrate for a
34 desired target video size it sometimes fails to get it right. Sometimes the
35 final video size will differ more than you wanted from the desired size, but if
36 you are really motivated and have a lot of time on your hands then you can run
37 Rippy again with a manually calculated bitrate. After all compression is done
38 the first time Rippy will recalculate the bitrate to give you the nearly exact
39 bitrate that would have worked. You can then edit the 'rippy.conf' file; set
40 the video_bitrate with this revised bitrate; and then run Rippy all over again.
41 There is nothing like 4-pass video compression to get it right! Actually, this
42 could be done in three passes since I don't need to do the second pass
43 compression before I calculate the revised bitrate. I'm also considering an
44 enhancement where Rippy would compress ten spread out chunks, 1-minute in
45 length to estimate the bitrate.
47 Free, open source, and all that good stuff.
48 Rippy Copyright (c) 2006 Noah Spurrier
50 Permission is hereby granted, free of charge, to any person obtaining a copy
51 of this software and associated documentation files (the "Software"), to deal
52 in the Software without restriction, including without limitation the rights
53 to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
54 copies of the Software, and to permit persons to whom the Software is
55 furnished to do so, subject to the following conditions:
57 The above copyright notice and this permission notice shall be included in all
58 copies or substantial portions of the Software.
60 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
61 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
62 MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
63 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
64 DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
65 OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
66 USE OR OTHER DEALINGS IN THE SOFTWARE.
69 $Id: rippy.py 517 2008-08-18 22:23:56Z noah $
84 __revision__ = '$Revision: 11 $'
85 __all__ = ['main', __version__, __revision__]
87 GLOBAL_LOGFILE_NAME = "rippy_%d.log" % os.getpid()
88 GLOBAL_LOGFILE = open(GLOBAL_LOGFILE_NAME, "wb")
90 ###############################################################################
91 # This giant section defines the prompts and defaults used in interactive mode.
92 ###############################################################################
93 # Python dictionaries are unordered, so
94 # I have this list that maintains the order of the keys.
98 'video_source_filename',
100 'video_final_filename',
102 'video_aspect_ratio',
104 'video_encode_passes',
106 'video_fourcc_override',
108 'video_bitrate_overhead',
110 'video_deinterlace_flag',
116 'audio_raw_filename',
117 'audio_volume_boost',
120 #'audio_lowpass_filter',
121 'delete_tmp_files_flag'
124 # The 'prompts' dictionary holds all the messages shown to the user in
125 # interactive mode. The 'prompts' dictionary schema is defined as follows:
126 # prompt_key : ( default value, prompt string, help string, level of difficulty (0,1,2) )
129 'video_source_filename': ("dvd://1", 'video source filename?', """This is the filename of the video that you want to convert from.
130 It can be any file that mencoder supports.
131 You can also choose a DVD device using the dvd://1 syntax.
132 Title 1 is usually the main title on a DVD.""", 0),
133 'video_chapter': ("none", 'video chapter?', """This is the chapter number. Usually disks such as TV series seasons will be divided into chapters. Maybe be set to none.""", 0),
134 'video_final_filename': ("video_final.avi", "video final filename?", """This is the name of the final video.""", 0),
135 'audio_raw_filename': ("audiodump.wav", "audio raw filename?", """This is the audio raw PCM filename. This is prior to compression.
136 Note that mplayer automatically names this audiodump.wav, so don't change this.""", 1000),
137 #'audio_compressed_filename':("audiodump.mp3","Audio compressed filename?", """This is the name of the compressed audio that will be mixed
138 # into the final video. Normally you don't need to change this.""",2),
139 'video_length': ("none", "video length in seconds?", """This sets the length of the video in seconds. This is used to estimate the
140 bitrate for a target video file size. Set to 'calc' to have Rippy calculate
141 the length. Set to 'none' if you don't want rippy to estimate the bitrate --
142 you will have to manually specify bitrate.""", 1),
143 'video_aspect_ratio': ("calc", "aspect ratio?", """This sets the aspect ratio of the video. Most DVDs are 16/9 or 4/3.""", 1),
144 'video_scale': ("none", "video scale?", """This scales the video to the given output size. The default is to do no scaling.
145 You may type in a resolution such as 320x240 or you may use presets.
146 qntsc: 352x240 (NTSC quarter screen)
147 qpal: 352x288 (PAL quarter screen)
148 ntsc: 720x480 (standard NTSC)
149 pal: 720x576 (standard PAL)
150 sntsc: 640x480 (square pixel NTSC)
151 spal: 768x576 (square pixel PAL)""", 1),
152 'video_codec': ("mpeg4", "video codec?", """This is the video compression to use. This is passed directly to mencoder, so
153 any format that it recognizes should work. For XviD or DivX use mpeg4.
154 Almost all MS Windows systems support wmv2 out of the box.
155 Some common codecs include:
156 mjpeg, h263, h263p, h264, mpeg4, msmpeg4, wmv1, wmv2, mpeg1video, mpeg2video, huffyuv, ffv1.
158 'audio_codec': ("mp3", "audio codec?", """This is the audio compression to use. This is passed directly to mencoder, so
159 any format that it recognizes will work.
160 Some common codecs include:
162 See mencoder manual for details.""", 2),
163 'video_fourcc_override': ("XVID", "force fourcc code?", """This forces the fourcc codec to the given value. XVID is safest for Windows.
164 The following are common fourcc values:
165 FMP4 - This is the mencoder default. This is the "real" value.
166 XVID - used by Xvid (safest)
168 MP4S - Microsoft""", 2),
169 'video_encode_passes': ("1", "number of encode passes?", """This sets how many passes to use to encode the video. You can choose 1 or 2.
170 Using two pases takes twice as long as one pass, but produces a better
171 quality video. I found that the improvement is not that impressive.""", 1),
172 'verbose_flag': ("Y", "verbose output?", """This sets verbose output. If true then all commands and arguments are printed
173 before they are run. This is useful to see exactly how commands are run.""", 1),
174 'dry_run_flag': ("N", "dry run?", """This sets 'dry run' mode. If true then commands are not run. This is useful
175 if you want to see what would the script would do.""", 1),
176 'video_bitrate': ("calc", "video bitrate?", """This sets the video bitrate. This overrides video_target_size.
177 Set to 'calc' to automatically estimate the bitrate based on the
178 video final target size. If you set video_length to 'none' then
179 you will have to specify this video_bitrate.""", 1),
180 'video_target_size': ("737280000", "video final target size?", """This sets the target video size that you want to end up with.
181 This is over-ridden by video_bitrate. In other words, if you specify
182 video_bitrate then video_target_size is ignored.
183 Due to the unpredictable nature of VBR compression the final video size
184 may not exactly match. The following are common CDR sizes:
185 180MB CDR (21 minutes) holds 193536000 bytes
186 550MB CDR (63 minutes) holds 580608000 bytes
187 650MB CDR (74 minutes) holds 681984000 bytes
188 700MB CDR (80 minutes) holds 737280000 bytes""", 0),
189 'video_bitrate_overhead': ("1.0", "bitrate overhead factor?", """Adjust this value if you want to leave more room for
190 other files such as subtitle files.
191 If you specify video_bitrate then this value is ignored.""", 2),
192 'video_crop_area': ("detect", "crop area?", """This sets the crop area to remove black bars from the top or sides of the video.
193 This helps save space. Set to 'detect' to automatically detect the crop area.
194 Set to 'none' to not crop the video. Normally you don't need to change this.""", 1),
195 'video_deinterlace_flag': ("N", "is the video interlaced?", """This sets the deinterlace flag. If set then mencoder will be instructed
196 to filter out interlace artifacts (using '-vf pp=md').""", 1),
197 'video_gray_flag': ("N", "is the video black and white (gray)?", """This improves output for black and white video.""", 1),
198 'subtitle_id': ("None", "Subtitle ID stream?", """This selects the subtitle stream to extract from the source video.
199 Normally, 0 is the English subtitle stream for a DVD.
200 Subtitles IDs with higher numbers may be other languages.""", 1),
201 'audio_id': ("128", "audio ID stream?", """This selects the audio stream to extract from the source video.
202 If your source is a VOB file (DVD) then stream IDs start at 128.
203 Normally, 128 is the main audio track for a DVD.
204 Tracks with higher numbers may be other language dubs or audio commentary.""", 1),
205 'audio_sample_rate': ("32000", "audio sample rate (Hz) 48000, 44100, 32000, 24000, 12000", """This sets the rate at which the compressed audio will be resampled.
206 DVD audio is 48 kHz whereas music CDs use 44.1 kHz. The higher the sample rate
207 the more space the audio track will take. That will leave less space for video.
208 32 kHz is a good trade-off if you are trying to fit a video onto a CD.""", 1),
209 'audio_bitrate': ("96", "audio bitrate (kbit/s) 192, 128, 96, 64?", """This sets the bitrate for MP3 audio compression.
210 The higher the bitrate the more space the audio track will take.
211 That will leave less space for video. Most people find music to be acceptable
212 at 128 kBitS. 96 kBitS is a good trade-off if you are trying to fit a video onto a CD.""", 1),
213 'audio_volume_boost': ("none", "volume dB boost?", """Many DVDs have very low audio volume. This sets an audio volume boost in Decibels.
214 Values of 6 to 10 usually adjust quiet DVDs to a comfortable level.""", 1),
215 #'audio_lowpass_filter':("16","audio lowpass filter (kHz)?","""This sets the low-pass filter for the audio.
216 # Normally this should be half of the audio sample rate.
217 # This improves audio compression and quality.
218 # Normally you don't need to change this.""",1),
219 'delete_tmp_files_flag': ("N", "delete temporary files when finished?", """If Y then %s, audio_raw_filename, and 'divx2pass.log' will be deleted at the end.""" % GLOBAL_LOGFILE_NAME, 1)
222 ##############################################################################
223 # This is the important convert control function
224 ##############################################################################
227 def convert(options):
228 """This is the heart of it all -- this performs an end-to-end conversion of
229 a video from one format to another. It requires a dictionary of options.
230 The conversion process will also add some keys to the dictionary
231 such as length of the video and crop area. The dictionary is returned.
232 This options dictionary could be used again to repeat the convert process
233 (it is also saved to rippy.conf as text).
235 if options['subtitle_id'] is not None:
236 print "# extract subtitles"
237 apply_smart(extract_subtitles, options)
239 print "# do not extract subtitles."
242 # I really only need to calculate the exact video length if the user
243 # selected 'calc' for video_bitrate
245 # selected 'detect' for video_crop_area.
246 if options['video_bitrate'] == 'calc' or options[
247 'video_crop_area'] == 'detect':
248 # As strange as it seems, the only reliable way to calculate the length
249 # of a video (in seconds) is to extract the raw, uncompressed PCM audio stream
250 # and then calculate the length of that. This is because MP4 video is VBR, so
251 # you cannot get exact time based on compressed size.
252 if options['video_length'] == 'calc':
253 print "# extract PCM raw audio to %s" % (options['audio_raw_filename'])
254 apply_smart(extract_audio, options)
255 options['video_length'] = apply_smart(get_length, options)
256 print "# Length of raw audio file : %d seconds (%0.2f minutes)" % (options['video_length'], float(options['video_length']) / 60.0)
257 if options['video_bitrate'] == 'calc':
258 options['video_bitrate'] = options[
259 'video_bitrate_overhead'] * apply_smart(calc_video_bitrate, options)
260 print "# video bitrate : " + str(options['video_bitrate'])
261 if options['video_crop_area'] == 'detect':
262 options['video_crop_area'] = apply_smart(crop_detect, options)
263 print "# crop area : " + str(options['video_crop_area'])
264 print "# compression estimate"
265 print apply_smart(compression_estimate, options)
267 print "# compress video"
268 apply_smart(compress_video, options)
269 'audio_volume_boost',
271 print "# delete temporary files:",
272 if options['delete_tmp_files_flag']:
274 apply_smart(delete_tmp_files, options)
278 # Finish by saving options to rippy.conf and
279 # calclating if final_size is less than target_size.
280 o = ["# options used to create video\n"]
281 video_actual_size = get_filesize(options['video_final_filename'])
282 if options['video_target_size'] != 'none':
283 revised_bitrate = calculate_revised_bitrate(
284 options['video_bitrate'],
285 options['video_target_size'],
287 o.append("# revised video_bitrate : %d\n" % revised_bitrate)
288 for k, v in options.iteritems():
289 o.append(" %30s : %s\n" % (k, v))
291 fout = open("rippy.conf", "wb").write(''.join(o))
292 print "# final actual video size = %d" % video_actual_size
293 if options['video_target_size'] != 'none':
294 if video_actual_size > options['video_target_size']:
295 print "# FINAL VIDEO SIZE IS GREATER THAN DESIRED TARGET"
296 print "# final video size is %d bytes over target size" % (video_actual_size - options['video_target_size'])
298 print "# final video size is %d bytes under target size" % (options['video_target_size'] - video_actual_size)
299 print "# If you want to run the entire compression process all over again"
300 print "# to get closer to the target video size then trying using a revised"
301 print "# video_bitrate of %d" % revised_bitrate
305 ##############################################################################
308 def exit_with_usage(exit_code=1):
309 print globals()['__doc__']
310 print 'version:', globals()['__version__']
315 def check_missing_requirements():
316 """This list of missing requirements (mencoder, mplayer, lame, and mkvmerge).
317 Returns None if all requirements are in the execution path.
320 if pexpect.which("mencoder") is None:
321 missing.append("mencoder")
322 if pexpect.which("mplayer") is None:
323 missing.append("mplayer")
324 cmd = "mencoder -oac help"
325 (command_output, exitstatus) = run(cmd)
326 ar = re.findall("(mp3lame)", command_output)
328 missing.append("Mencoder was not compiled with mp3lame support.")
330 # if pexpect.which("lame") is None:
331 # missing.append("lame")
332 # if pexpect.which("mkvmerge") is None:
333 # missing.append("mkvmerge")
334 if len(missing) == 0:
339 def input_option(message, default_value="", help=None, level=0, max_level=0):
340 """This is a fancy raw_input function.
341 If the user enters '?' then the contents of help is printed.
343 The 'level' and 'max_level' are used to adjust which advanced options
344 are printed. 'max_level' is the level of options that the user wants
345 to see. 'level' is the level of difficulty for this particular option.
346 If this level is <= the max_level the user wants then the
347 message is printed and user input is allowed; otherwise, the
348 default value is returned automatically without user input.
350 if default_value != '':
351 message = "%s [%s] " % (message, default_value)
352 if level > max_level:
355 user_input = raw_input(message)
356 if user_input == '?':
358 elif user_input == '':
365 def progress_callback(d=None):
366 """This callback simply prints a dot to show activity.
367 This is used when running external commands with pexpect.run.
369 sys.stdout.write(".")
374 global GLOBAL_LOGFILE
375 print >>GLOBAL_LOGFILE, cmd
377 exitstatus) = pexpect.run(cmd,
378 events={pexpect.TIMEOUT: progress_callback},
381 logfile=GLOBAL_LOGFILE)
383 print "RUN FAILED. RETURNED EXIT STATUS:", exitstatus
384 print >>GLOBAL_LOGFILE, "RUN FAILED. RETURNED EXIT STATUS:", exitstatus
385 return (command_output, exitstatus)
388 def apply_smart(func, args):
389 """This is similar to func(**args), but this won't complain about
390 extra keys in 'args'. This ignores keys in 'args' that are
391 not required by 'func'. This passes None to arguments that are
392 not defined in 'args'. That's fine for arguments with a default valeue, but
393 that's a bug for required arguments. I should probably raise a TypeError.
394 The func parameter can be a function reference or a string.
395 If it is a string then it is converted to a function reference.
397 if isinstance(func, type('')):
398 if func in globals():
399 func = globals()[func]
401 raise NameError("name '%s' is not defined" % func)
402 if hasattr(func, 'im_func'): # Handle case when func is a class method.
404 argcount = func.func_code.co_argcount
405 required_args = dict([(k, args.get(k))
406 for k in func.func_code.co_varnames[:argcount]])
407 return func(**required_args)
410 def count_unique(items):
411 """This takes a list and returns a sorted list of tuples with a count of each unique item in the list.
413 count_unique(['a','b','c','a','c','c','a','c','c'])
415 [(5,'c'), (3,'a'), (1,'b')]
416 Example 2 -- get the most frequent item in a list:
417 count_unique(['a','b','c','a','c','c','a','c','c'])[0][1]
424 stats[i] = stats[i] + 1
427 stats = sorted([(v, k) for k, v in stats.items()])
432 def calculate_revised_bitrate(
436 """This calculates a revised video bitrate given the video_bitrate used,
437 the actual size that resulted, and the video_target_size.
438 This can be used if you want to compress the video all over again in an
439 attempt to get closer to the video_target_size.
441 return int(math.floor(video_bitrate * \
442 (float(video_target_size) / float(video_actual_size))))
445 def get_aspect_ratio(video_source_filename):
446 """This returns the aspect ratio of the original video.
447 This is usualy 1.78:1(16/9) or 1.33:1(4/3).
448 This function is very lenient. It basically guesses 16/9 whenever
449 it cannot figure out the aspect ratio.
451 cmd = "mplayer '%s' -vo png -ao null -frames 1" % video_source_filename
452 (command_output, exitstatus) = run(cmd)
454 "Movie-Aspect is ([0-9]+\.?[0-9]*:[0-9]+\.?[0-9]*)",
458 if ar[0] == '1.78:1':
460 if ar[0] == '1.33:1':
463 #idh = re.findall("ID_VIDEO_HEIGHT=([0-9]+)", command_output)
464 # if len(idw)==0 or len(idh)==0:
466 # print 'Could not get aspect ration. Assuming 1.78:1 (16/9).'
468 # return float(idw[0])/float(idh[0])
470 # ID_VIDEO_HEIGHT=480
471 # Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
474 def get_aid_list(video_source_filename):
475 """This returns a list of audio ids in the source video file.
476 TODO: Also extract ID_AID_nnn_LANG to associate language. Not all DVDs include this.
478 cmd = "mplayer '%s' -vo null -ao null -frames 0 -identify" % video_source_filename
479 (command_output, exitstatus) = run(cmd)
480 idl = sorted(re.findall("ID_AUDIO_ID=([0-9]+)", command_output))
484 def get_sid_list(video_source_filename):
485 """This returns a list of subtitle ids in the source video file.
486 TODO: Also extract ID_SID_nnn_LANG to associate language. Not all DVDs include this.
488 cmd = "mplayer '%s' -vo null -ao null -frames 0 -identify" % video_source_filename
489 (command_output, exitstatus) = run(cmd)
490 idl = sorted(re.findall("ID_SUBTITLE_ID=([0-9]+)", command_output))
495 video_source_filename,
499 """This extracts the given audio_id track as raw uncompressed PCM from the given source video.
500 Note that mplayer always saves this to audiodump.wav.
501 At this time there is no way to set the output audio name.
503 #cmd = "mplayer %(video_source_filename)s -vc null -vo null -aid %(audio_id)s -ao pcm:fast -noframedrop" % locals()
504 cmd = "mplayer -quiet '%(video_source_filename)s' -vc dummy -vo null -aid %(audio_id)s -ao pcm:fast -noframedrop" % locals()
512 def extract_subtitles(
513 video_source_filename,
517 """This extracts the given subtitle_id track as VOBSUB format from the given source video.
519 cmd = "mencoder -quiet '%(video_source_filename)s' -o /dev/null -nosound -ovc copy -vobsubout subtitles -vobsuboutindex 0 -sid %(subtitle_id)s" % locals()
527 def get_length(audio_raw_filename):
528 """This attempts to get the length of the media file (length is time in seconds).
529 This should not be confused with size (in bytes) of the file data.
530 This is best used on a raw PCM AUDIO file because mplayer cannot get an accurate
531 time for many compressed video and audio formats -- notably MPEG4 and MP3.
533 This returns -1 if it cannot get the length of the given file.
535 cmd = "mplayer %s -vo null -ao null -frames 0 -identify" % audio_raw_filename
536 (command_output, exitstatus) = run(cmd)
537 idl = sorted(re.findall("ID_LENGTH=([0-9.]*)", command_output))
539 print "ERROR: cannot get length of raw audio file."
540 print "command_output of mplayer identify:"
542 print "parsed command_output:"
548 def get_filesize(filename):
549 """This returns the number of bytes a file takes on storage."""
550 return os.stat(filename)[stat.ST_SIZE]
553 def calc_video_bitrate(
559 """This gives an estimate of the video bitrate necessary to
560 fit the final target size. This will take into account room to
561 fit the audio and extra space if given (for container overhead or whatnot).
562 video_target_size is in bytes,
563 audio_bitrate is bits per second (96, 128, 256, etc.) ASSUMING CBR,
564 video_length is in seconds,
565 extra_space is in bytes.
566 a 180MB CDR (21 minutes) holds 193536000 bytes.
567 a 550MB CDR (63 minutes) holds 580608000 bytes.
568 a 650MB CDR (74 minutes) holds 681984000 bytes.
569 a 700MB CDR (80 minutes) holds 737280000 bytes.
573 if extra_space is None:
575 #audio_size = os.stat(audio_compressed_filename)[stat.ST_SIZE]
576 audio_size = (audio_bitrate * video_length * 1000) / 8.0
577 video_target_size = video_target_size - audio_size - extra_space
578 return (int)(calc_video_kbitrate(video_target_size, video_length))
581 def calc_video_kbitrate(target_size, length_secs):
582 """Given a target byte size free for video data, this returns the bitrate in kBit/S.
583 For mencoder vbitrate 1 kBit = 1000 Bits -- not 1024 bits.
584 target_size = bitrate * 1000 * length_secs / 8
585 target_size = bitrate * 125 * length_secs
586 bitrate = target_size/(125*length_secs)
588 return int(target_size / (125.0 * length_secs))
591 def crop_detect(video_source_filename, video_length, dry_run_flag=0):
592 """This attempts to figure out the best crop for the given video file.
593 Basically it runs crop detect for 10 seconds on five different places in the video.
594 It picks the crop area that was most often detected.
596 skip = int(video_length / 9) # offset to skip (-ss option in mencoder)
598 cmd1 = "mencoder '%s' -quiet -ss %d -endpos %d -o /dev/null -nosound -ovc lavc -vf cropdetect" % (
599 video_source_filename, skip, sample_length)
600 cmd2 = "mencoder '%s' -quiet -ss %d -endpos %d -o /dev/null -nosound -ovc lavc -vf cropdetect" % (
601 video_source_filename, 2 * skip, sample_length)
602 cmd3 = "mencoder '%s' -quiet -ss %d -endpos %d -o /dev/null -nosound -ovc lavc -vf cropdetect" % (
603 video_source_filename, 4 * skip, sample_length)
604 cmd4 = "mencoder '%s' -quiet -ss %d -endpos %d -o /dev/null -nosound -ovc lavc -vf cropdetect" % (
605 video_source_filename, 6 * skip, sample_length)
606 cmd5 = "mencoder '%s' -quiet -ss %d -endpos %d -o /dev/null -nosound -ovc lavc -vf cropdetect" % (
607 video_source_filename, 8 * skip, sample_length)
610 (command_output1, exitstatus1) = run(cmd1)
611 (command_output2, exitstatus2) = run(cmd2)
612 (command_output3, exitstatus3) = run(cmd3)
613 (command_output4, exitstatus4) = run(cmd4)
614 (command_output5, exitstatus5) = run(cmd5)
615 idl = re.findall("-vf crop=([0-9]+:[0-9]+:[0-9]+:[0-9]+)", command_output1)
617 re.findall("-vf crop=([0-9]+:[0-9]+:[0-9]+:[0-9]+)", command_output2)
619 re.findall("-vf crop=([0-9]+:[0-9]+:[0-9]+:[0-9]+)", command_output3)
621 re.findall("-vf crop=([0-9]+:[0-9]+:[0-9]+:[0-9]+)", command_output4)
623 re.findall("-vf crop=([0-9]+:[0-9]+:[0-9]+:[0-9]+)", command_output5)
624 items_count = count_unique(idl)
625 return items_count[0][1]
628 def build_compression_command(
629 video_source_filename,
630 video_final_filename,
636 video_fourcc_override='FMP4',
638 video_crop_area=None,
639 video_aspect_ratio='16/9',
641 video_encode_passes=2,
642 video_deinterlace_flag=0,
643 audio_volume_boost=None,
644 audio_sample_rate=None,
649 # Notes:For DVD, VCD, and SVCD use acodec=mp2 and vcodec=mpeg2video:
650 # mencoder movie.avi -o movie.VOB -ovc lavc -oac lavc -lavcopts
651 # acodec=mp2:abitrate=224:vcodec=mpeg2video:vbitrate=2000
654 # build video filter (-vf) argument
657 if video_crop_area and video_crop_area.lower() != 'none':
658 video_filter = video_filter + 'crop=%s' % video_crop_area
659 if video_deinterlace_flag:
660 if video_filter != '':
661 video_filter = video_filter + ','
662 video_filter = video_filter + 'pp=md'
663 if video_scale and video_scale.lower() != 'none':
664 if video_filter != '':
665 video_filter = video_filter + ','
666 video_filter = video_filter + 'scale=%s' % video_scale
667 # optional video rotation -- were you holding your camera sideways?
668 # if video_filter != '':
669 # video_filter = video_filter + ','
670 #video_filter = video_filter + 'rotate=2'
671 if video_filter != '':
672 video_filter = '-vf ' + video_filter
675 # build chapter argument
677 if video_chapter is not None:
678 chapter = '-chapter %d-%d' % (video_chapter, video_chapter)
681 # chapter = '-chapter 2-2'
684 # build audio_filter argument
687 if audio_sample_rate:
688 if audio_filter != '':
689 audio_filter = audio_filter + ','
690 audio_filter = audio_filter + 'lavcresample=%s' % audio_sample_rate
691 if audio_volume_boost is not None:
692 if audio_filter != '':
693 audio_filter = audio_filter + ','
694 audio_filter = audio_filter + 'volume=%0.1f:1' % audio_volume_boost
695 if audio_filter != '':
696 audio_filter = '-af ' + audio_filter
698 # if audio_sample_rate:
699 # audio_filter = ('-srate %d ' % audio_sample_rate) + audio_filter
702 # build lavcopts argument
704 #lavcopts = '-lavcopts vcodec=%s:vbitrate=%d:mbd=2:aspect=%s:acodec=%s:abitrate=%d:vpass=1' % (video_codec,video_bitrate,audio_codec,audio_bitrate)
705 lavcopts = '-lavcopts vcodec=%(video_codec)s:vbitrate=%(video_bitrate)d:mbd=2:aspect=%(video_aspect_ratio)s:acodec=%(audio_codec)s:abitrate=%(audio_bitrate)d:vpass=1' % (locals())
707 lavcopts = lavcopts + ':gray'
710 if seek_skip is not None:
711 seek_filter = '-ss %s' % (str(seek_skip))
712 if seek_length is not None:
713 seek_filter = seek_filter + ' -endpos %s' % (str(seek_length))
715 # cmd = "mencoder -quiet -info comment='Arkivist' '%(video_source_filename)s' %(seek_filter)s %(chapter)s -aid %(audio_id)s -o '%(video_final_filename)s' -ffourcc %(video_fourcc_override)s -ovc lavc -oac lavc %(lavcopts)s %(video_filter)s %(audio_filter)s" % locals()
716 cmd = "mencoder -quiet -info comment='Arkivist' '%(video_source_filename)s' %(seek_filter)s %(chapter)s -aid %(audio_id)s -o '%(video_final_filename)s' -ffourcc %(video_fourcc_override)s -ovc lavc -oac mp3lame %(lavcopts)s %(video_filter)s %(audio_filter)s" % locals()
720 def compression_estimate(
722 video_source_filename,
723 video_final_filename,
729 video_fourcc_override='FMP4',
731 video_crop_area=None,
732 video_aspect_ratio='16/9',
734 video_encode_passes=2,
735 video_deinterlace_flag=0,
736 audio_volume_boost=None,
737 audio_sample_rate=None,
739 """This attempts to figure out the best compression ratio for a given set of compression options.
741 # TODO Need to account for AVI overhead.
742 skip = int(video_length / 9) # offset to skip (-ss option in mencoder)
744 cmd1 = build_compression_command(
745 video_source_filename,
746 "compression_test_1.avi",
752 video_fourcc_override,
758 video_deinterlace_flag,
764 cmd2 = build_compression_command(
765 video_source_filename,
766 "compression_test_2.avi",
772 video_fourcc_override,
778 video_deinterlace_flag,
784 cmd3 = build_compression_command(
785 video_source_filename,
786 "compression_test_3.avi",
792 video_fourcc_override,
798 video_deinterlace_flag,
804 cmd4 = build_compression_command(
805 video_source_filename,
806 "compression_test_4.avi",
812 video_fourcc_override,
818 video_deinterlace_flag,
824 cmd5 = build_compression_command(
825 video_source_filename,
826 "compression_test_5.avi",
832 video_fourcc_override,
838 video_deinterlace_flag,
849 size = get_filesize("compression_test_1.avi") + get_filesize("compression_test_2.avi") + get_filesize(
850 "compression_test_3.avi") + get_filesize("compression_test_4.avi") + get_filesize("compression_test_5.avi")
855 video_source_filename,
856 video_final_filename,
862 video_fourcc_override='FMP4',
864 video_crop_area=None,
865 video_aspect_ratio='16/9',
867 video_encode_passes=2,
868 video_deinterlace_flag=0,
869 audio_volume_boost=None,
870 audio_sample_rate=None,
877 """This compresses the video and audio of the given source video filename to the transcoded filename.
878 This does a two-pass compression (I'm assuming mpeg4, I should probably make this smarter for other formats).
881 # do the first pass video compression
883 #cmd = "mencoder -quiet '%(video_source_filename)s' -ss 65 -endpos 20 -aid %(audio_id)s -o '%(video_final_filename)s' -ffourcc %(video_fourcc_override)s -ovc lavc -oac lavc %(lavcopts)s %(video_filter)s %(audio_filter)s" % locals()
885 cmd = build_compression_command(
886 video_source_filename,
887 video_final_filename,
893 video_fourcc_override,
899 video_deinterlace_flag,
912 # If not doing two passes then return early.
913 if video_encode_passes != '2':
917 video_actual_size = get_filesize(video_final_filename)
918 if video_actual_size > video_target_size:
919 print "======================================================="
921 print "First pass compression resulted in"
922 print "actual file size greater than target size."
923 print "Second pass will be too big."
924 print "======================================================="
927 # do the second pass video compression
929 cmd = cmd.replace('vpass=1', 'vpass=2')
940 audio_compressed_filename,
941 audio_lowpass_filter=None,
942 audio_sample_rate=None,
946 """This is depricated.
947 This compresses the raw audio file to the compressed audio filename.
949 cmd = 'lame -h --athaa-sensitivity 1' # --cwlimit 11"
950 if audio_lowpass_filter:
951 cmd = cmd + ' --lowpass ' + audio_lowpass_filter
953 #cmd = cmd + ' --abr ' + audio_bitrate
954 cmd = cmd + ' --cbr -b ' + audio_bitrate
955 if audio_sample_rate:
956 cmd = cmd + ' --resample ' + audio_sample_rate
957 cmd = cmd + ' ' + audio_raw_filename + ' ' + audio_compressed_filename
961 (command_output, exitstatus) = run(cmd)
964 raise Exception('ERROR: lame failed to compress raw audio file.')
968 video_final_filename,
969 video_transcoded_filename,
970 audio_compressed_filename,
971 video_container_format,
974 """This is depricated. I used to use a three-pass encoding where I would mix the audio track separately, but
975 this never worked very well (loss of audio sync)."""
976 if video_container_format.lower() == 'mkv': # Matroska
978 video_final_filename,
979 video_transcoded_filename,
980 audio_compressed_filename,
983 if video_container_format.lower() == 'avi':
985 video_final_filename,
986 video_transcoded_filename,
987 audio_compressed_filename,
993 video_final_filename,
994 video_transcoded_filename,
995 audio_compressed_filename,
998 """This is depricated."""
999 cmd = 'mkvmerge -o %s --noaudio %s %s' % (
1000 video_final_filename, video_transcoded_filename, audio_compressed_filename)
1003 if not dry_run_flag:
1009 video_final_filename,
1010 video_transcoded_filename,
1011 audio_compressed_filename,
1014 """This is depricated."""
1016 # cmd = "mencoder -quiet -oac copy -ovc copy -o '%s' -audiofile %s '%s'" % (video_final_filename, audio_compressed_filename, video_transcoded_filename)
1017 # if verbose_flag: print cmd
1018 # if not dry_run_flag:
1023 def delete_tmp_files(audio_raw_filename, verbose_flag=0, dry_run_flag=0):
1024 global GLOBAL_LOGFILE_NAME
1025 file_list = ' '.join(
1026 [GLOBAL_LOGFILE_NAME, 'divx2pass.log', audio_raw_filename])
1027 cmd = 'rm -f ' + file_list
1030 if not dry_run_flag:
1034 ##############################################################################
1035 # This is the interactive Q&A that is used if a conf file was not given.
1036 ##############################################################################
1039 def interactive_convert():
1041 global prompts, prompts_key_order
1043 print globals()['__doc__']
1045 print "=============================================="
1046 print " Enter '?' at any question to get extra help."
1047 print "=============================================="
1050 # Ask for the level of options the user wants.
1051 # A lot of code just to print a string!
1052 level_sort = {0: '', 1: '', 2: ''}
1054 level = prompts[k][3]
1055 if level < 0 or level > 2:
1057 level_sort[level] += " " + prompts[k][1] + "\n"
1058 level_sort_string = "This sets the level for advanced options prompts. Set 0 for simple, 1 for advanced, or 2 for expert.\n"
1059 level_sort_string += "[0] Basic options:\n" + str(level_sort[0]) + "\n"
1060 level_sort_string += "[1] Advanced options:\n" + str(level_sort[1]) + "\n"
1061 level_sort_string += "[2] Expert options:\n" + str(level_sort[2])
1062 c = input_option("Prompt level (0, 1, or 2)?", "1", level_sort_string)
1063 max_prompt_level = int(c)
1066 for k in prompts_key_order:
1067 if k == 'video_aspect_ratio':
1068 guess_aspect = get_aspect_ratio(options['video_source_filename'])
1069 options[k] = input_option(
1075 elif k == 'audio_id':
1076 aid_list = get_aid_list(options['video_source_filename'])
1078 if max_prompt_level >= prompts[k][3]:
1079 if len(aid_list) > 1:
1080 print "This video has more than one audio stream. The following stream audio IDs were found:"
1081 for aid in aid_list:
1083 default_id = aid_list[0]
1086 print "Rippy was unable to get the list of audio streams from this video."
1087 print "If reading directly from a DVD then the DVD device might be busy."
1088 print "Using a default setting of stream id 128 (main audio on most DVDs)."
1090 options[k] = input_option(
1096 elif k == 'subtitle_id':
1097 sid_list = get_sid_list(options['video_source_filename'])
1099 if max_prompt_level >= prompts[k][3]:
1100 if len(sid_list) > 0:
1101 print "This video has one or more subtitle streams. The following stream subtitle IDs were found:"
1102 for sid in sid_list:
1104 #default_id = sid_list[0]
1105 default_id = prompts[k][0]
1108 print "Unable to get the list of subtitle streams from this video. It may have none."
1109 print "Setting default to None."
1111 options[k] = input_option(
1117 elif k == 'audio_lowpass_filter':
1118 lowpass_default = "%.1f" % (math.floor(
1119 float(options['audio_sample_rate']) / 2.0))
1120 options[k] = input_option(
1126 elif k == 'video_bitrate':
1127 if options['video_length'].lower() == 'none':
1128 options[k] = input_option(prompts[k][1], '1000', prompts[k][
1129 2], prompts[k][3], max_prompt_level)
1131 options[k] = input_option(prompts[k][1], prompts[k][0], prompts[
1132 k][2], prompts[k][3], max_prompt_level)
1134 # don't bother asking for video_target_size or
1135 # video_bitrate_overhead if video_bitrate was set
1136 if (k == 'video_target_size' or k == 'video_bitrate_overhead') and options[
1137 'video_bitrate'] != 'calc':
1139 # don't bother with crop area if video length is none
1140 if k == 'video_crop_area' and options[
1141 'video_length'].lower() == 'none':
1142 options['video_crop_area'] = 'none'
1144 options[k] = input_option(
1151 #options['video_final_filename'] = options['video_final_filename'] + "." + options['video_container_format']
1153 print "=========================================================================="
1154 print "Ready to Rippy!"
1156 print "The following options will be used:"
1157 for k, v in options.iteritems():
1158 print "%27s : %s" % (k, v)
1161 c = input_option("Continue?", "Y")
1162 c = c.strip().lower()
1169 def clean_options(d):
1170 """This validates and cleans up the options dictionary.
1171 After reading options interactively or from a conf file
1172 we need to make sure that the values make sense and are
1173 converted to the correct type.
1174 1. Any key with "_flag" in it becomes a boolean True or False.
1175 2. Values are normalized ("No", "None", "none" all become "none";
1176 "Calcluate", "c", "CALC" all become "calc").
1177 3. Certain values are converted from string to int.
1178 4. Certain combinations of options are invalid or override each other.
1179 This is a rather annoying function, but then so it most cleanup work.
1183 # convert all flag options to 0 or 1
1185 if isinstance(d[k], types.StringType):
1186 if d[k].strip().lower()[0] in 'yt1': # Yes, True, 1
1190 d['video_bitrate'] = d['video_bitrate'].lower()
1191 if d['video_bitrate'][0] == 'c':
1192 d['video_bitrate'] = 'calc'
1194 d['video_bitrate'] = int(float(d['video_bitrate']))
1196 d['video_target_size'] = int(d['video_target_size'])
1197 # shorthand magic numbers get automatically expanded
1198 if d['video_target_size'] == 180:
1199 d['video_target_size'] = 193536000
1200 elif d['video_target_size'] == 550:
1201 d['video_target_size'] = 580608000
1202 elif d['video_target_size'] == 650:
1203 d['video_target_size'] = 681984000
1204 elif d['video_target_size'] == 700:
1205 d['video_target_size'] = 737280000
1207 d['video_target_size'] = 'none'
1210 d['video_chapter'] = int(d['video_chapter'])
1212 d['video_chapter'] = None
1215 d['subtitle_id'] = int(d['subtitle_id'])
1217 d['subtitle_id'] = None
1220 d['video_bitrate_overhead'] = float(d['video_bitrate_overhead'])
1222 d['video_bitrate_overhead'] = -1.0
1224 d['audio_bitrate'] = int(d['audio_bitrate'])
1225 d['audio_sample_rate'] = int(d['audio_sample_rate'])
1226 d['audio_volume_boost'] = d['audio_volume_boost'].lower()
1227 if d['audio_volume_boost'][0] == 'n':
1228 d['audio_volume_boost'] = None
1230 d['audio_volume_boost'] = d['audio_volume_boost'].replace('db', '')
1231 d['audio_volume_boost'] = float(d['audio_volume_boost'])
1233 # assert (d['video_bitrate']=='calc' and d['video_target_size']!='none')
1234 # or (d['video_bitrate']!='calc' and d['video_target_size']=='none')
1236 d['video_scale'] = d['video_scale'].lower()
1237 if d['video_scale'][0] == 'n':
1238 d['video_scale'] = 'none'
1240 al = re.findall("([0-9]+).*?([0-9]+)", d['video_scale'])
1241 d['video_scale'] = al[0][0] + ':' + al[0][1]
1242 d['video_crop_area'] = d['video_crop_area'].lower()
1243 if d['video_crop_area'][0] == 'n':
1244 d['video_crop_area'] = 'none'
1245 d['video_length'] = d['video_length'].lower()
1246 if d['video_length'][0] == 'c':
1247 d['video_length'] = 'calc'
1248 elif d['video_length'][0] == 'n':
1249 d['video_length'] = 'none'
1251 d['video_length'] = int(float(d['video_length']))
1252 if d['video_length'] == 0:
1253 d['video_length'] = 'none'
1254 assert (not (d['video_length'] == 'none' and d['video_bitrate'] == 'calc'))
1260 optlist, args = getopt.getopt(sys.argv[1:], 'h?', ['help', 'h', '?'])
1261 except Exception as e:
1264 command_line_options = dict(optlist)
1265 # There are a million ways to cry for help. These are but a few of them.
1266 if [elem for elem in command_line_options if elem in [
1267 '-h', '--h', '-?', '--?', '--help']]:
1270 missing = check_missing_requirements()
1271 if missing is not None:
1273 print "=========================================================================="
1275 print "Some required external commands are missing."
1276 print "please install the following packages:"
1278 print "=========================================================================="
1280 c = input_option("Continue?", "Y")
1281 c = c.strip().lower()
1287 # cute one-line string-to-dictionary parser (two-lines if you count
1291 '([^: \t\n]*)\s*:\s*(".*"|[^ \t\n]*)',
1294 options = clean_options(options)
1297 options = interactive_convert()
1298 options = clean_options(options)
1302 if __name__ == "__main__":
1304 start_time = time.time()
1305 print time.asctime()
1307 print time.asctime()
1308 print "TOTAL TIME IN MINUTES:",
1309 print (time.time() - start_time) / 60.0
1310 except Exception as e:
1311 tb_dump = traceback.format_exc()
1312 print "=========================================================================="
1313 print "ERROR -- Unexpected exception in script."
1316 print "=========================================================================="
1317 print >>GLOBAL_LOGFILE, "=========================================================================="
1318 print >>GLOBAL_LOGFILE, "ERROR -- Unexpected exception in script."
1319 print >>GLOBAL_LOGFILE, str(e)
1320 print >>GLOBAL_LOGFILE, str(tb_dump)
1321 print >>GLOBAL_LOGFILE, "=========================================================================="