#354671 - 08/09/2012 16:27
Insanely fast video/audio encoding machine
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Neither MacBook Pro nor my Core2Duo server transcode from MPEG2 to h.264 at a respectable rate, so I'm considering building a dedicated machine to do that and any other computation-heavy work I may need in the future. Here's the catch. I'd like to do it "on the cheap"  None of the open source software seems to take advantage of things like OpenCL, so I won't be needing significant graphics power to help the process. This will all come down to two things, IMO, CPU (cores and speed) and memory. Does anyone currently have a system deployed for a similar purpose?
|
|
Top
|
|
|
|
|
#354694 - 09/09/2012 19:28
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
carpal tunnel
Registered: 30/04/2000
Posts: 3810
|
My 2009-era MacPro (quad core Nehalem CPU) running Handbrake can transcode 1080i HD MPEG2 to 720p H.264 at roughly 30fps (using all eight hyperthreads), which counts as modestly respectable. I'd imagine a newer Ivy Bridge chip would do even better. I've been pondering replacing a bunch of 2009-era iMacs in our lab with Hackintoshes of some description or another, which seems like the way to have Mac goodness without Mac priciness. (And to buy monitors once, since they're good for multiple generations fo computers.)
|
|
Top
|
|
|
|
|
#354705 - 09/09/2012 23:50
Re: Insanely fast video/audio encoding machine
[Re: DWallach]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14496
Loc: Canada
|
In theory, a pair of Raspberry Pi boards can do it in real-time (30fps) -- one as the decoder, one as the encoder.
I think I read an article recently where Ivy Bridge chips were compared to NVIDIA GPUs for this task, and their (IB) performance was also about real-time (30fps). Could be wrong there, though.
|
|
Top
|
|
|
|
|
#354718 - 10/09/2012 11:58
Re: Insanely fast video/audio encoding machine
[Re: Phoenix42]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
The issue with defining the requirements is that there are so many variables to choose from. I suppose I can pick the base requirements, but even then I can't list immediately all the encoder settings I'd be using. At simple base level though: Source: 1080i MPEG2 or 720P MPEG2 Destination: 720P h.264 23.97fps Respectable rate: 2x+ real-time or faster using something like Handbrake's AppleTV 3 preset. I don't want to spend $5000+, so by "cheap" I'm thinking maybe up to $1000. Otherwise, I was thinking that the bare minimum would be a pair of quad or 8-core processors with large cache which is easier to spec blind with no budget.
Edited by hybrid8 (10/09/2012 14:47)
|
|
Top
|
|
|
|
|
#354731 - 10/09/2012 14:47
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
veteran
Registered: 21/03/2002
Posts: 1424
Loc: MA but Irish born
|
That might be a good question to ask on the Handbrake forums. Yeah I'm sure the signal to noise ratio sucks, but you'd get an interesting thread going with a title like "$1k budget, what is a killer encoding rig", you'd just need to filter it for junk.
While Handbrake/ffmpeg can use multiple cores, I'd assume that at some point the rate of improvement slows down, but maybe that would require a pair of Bulldozers.
|
|
Top
|
|
|
|
|
#354732 - 10/09/2012 14:56
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
carpal tunnel
Registered: 08/06/1999
Posts: 7868
|
Source: 1080i MPEG2 or 720P MPEG2
Destination: 720P h.264 23.97fps
Respectable rate: 2x+ real-time or faster using something like Handbrake's AppleTV 3 preset.
Do you have a handy sample file somewhere that can be used for benchmarking? I can throw it at a few setups to report back encoding time.
|
|
Top
|
|
|
|
|
#354738 - 10/09/2012 16:06
Re: Insanely fast video/audio encoding machine
[Re: drakino]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
I have lots of samples, but they're large. I suppose I can cut one down and then run numbers on my own systems first. I'm doing some file splitting and renaming right now so I'll see if I can get together manageable sample of Dinosaur Train. The ATSC streams I record for my daughter are around 5-7GB for every hour, depending on the show.
|
|
Top
|
|
|
|
|
#354739 - 10/09/2012 16:26
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
http://twistedmelon.com/sample.mpegOk, I've made a small clip just under 00:01:30 which comes out at around 124MB. The recording has two audio tracks, the second and lower bit-rate one being descriptive video which I'll toss during transcode. I'll post up my times in a bit. I'll be doing the following: Start with Handbrake preset AppleTV 2 (720p) Change frame rate to "same as source" Audio - audio track 1 AC3 passthrough - eliminate other track Text - keep all sub-picture content, un-check forced and burned The source file details (as reported by MPEG Streamclip)
Stream: original_recording.mpg
Type: MPEG program stream
Duration: 0:29:59
Data Size: 2.30 GB
Bit Rate: 10.99 Mbps
Video Tracks:
224 MPEG-2, 1920 × 1080, 16:9, 23.976 fps, 65.00 Mbps, upper field first
Audio Tracks:
128 AC3 2/0, 48 kHz, 384 kbps
129 AC3 2/0, 48 kHz, 192 kbps
Details for the clip from Media Info:
General
Complete name : sample.mpeg
Format : MPEG-PS
File size : 125 MiB
Duration : 1mn 32s
Overall bit rate mode : Variable
Overall bit rate : 11.3 Mbps
Video
ID : 224 (0xE0)
Format : MPEG Video
Format version : Version 2
Format profile : Main@High
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Duration : 1mn 32s
Bit rate mode : Variable
Bit rate : 10.5 Mbps
Maximum bit rate : 65.0 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Active Format Description : Full frame 16:9 image
Frame rate : 29.970 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.169
Stream size : 116 MiB (93%)
Audio #1
ID : 189 (0xBD)-128 (0x80)
Format : AC-3
Format/Info : Audio Coding 3
Mode extension : CM (complete main)
Format settings, Endianness : Big
Muxing mode : DVD-Video
Duration : 1mn 32s
Bit rate mode : Constant
Bit rate : 384 Kbps
Channel(s) : 6 channels
Channel positions : Front: L C R, Side: L R, LFE
Sampling rate : 48.0 KHz
Bit depth : 16 bits
Compression mode : Lossy
Stream size : 4.23 MiB (3%)
Audio #2
ID : 189 (0xBD)-129 (0x81)
Format : AC-3
Format/Info : Audio Coding 3
Mode extension : CM (complete main)
Format settings, Endianness : Big
Muxing mode : DVD-Video
Duration : 1mn 32s
Bit rate mode : Constant
Bit rate : 192 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Bit depth : 16 bits
Compression mode : Lossy
Stream size : 2.12 MiB (2%)
Text #1
ID : 224 (0xE0)-CC1
Format : EIA-608
Muxing mode : A/53 / DTVCC Transport
Muxing mode, more info : Muxed in Video #1
Bit rate mode : Constant
Stream size : 0.00 Byte (0%)
Text #2
ID : 224 (0xE0)-1
Format : EIA-708
Muxing mode : A/53 / DTVCC Transport
Muxing mode, more info : Muxed in Video #1
Bit rate mode : Constant
Stream size : 0.00 Byte (0%)
Edited by hybrid8 (10/09/2012 17:25)
|
|
Top
|
|
|
|
|
#354740 - 10/09/2012 16:59
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Source file 124MB, destination file 62MB
The transcode took about 3 minutes and 30 seconds with an average fps of 12.59 (2611 frames total).
I'll refine the settings later to bring the final size down because I'm not happy with only a 50% savings. This sample pass also didn't include Handbrake's DeComb filter which I'll need to turn on for final conversions. The final product will also be forced to 23.976 fps.
Edited by hybrid8 (10/09/2012 17:03)
|
|
Top
|
|
|
|
|
#354744 - 10/09/2012 18:24
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
veteran
Registered: 21/03/2002
Posts: 1424
Loc: MA but Irish born
|
What am I doing in a pteranodon nest?
|
|
Top
|
|
|
|
|
#354749 - 10/09/2012 19:19
Re: Insanely fast video/audio encoding machine
[Re: Phoenix42]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Oh, I'm also totally open to any suggestions for encode settings to maximize quality and minimize size. I've been playing around but haven't found the ideal yet. Whenever I play with the bit rate (instead of using the default of ref 20 quality), I end up with something I'm not happy with.
|
|
Top
|
|
|
|
|
#354761 - 11/09/2012 02:46
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14496
Loc: Canada
|
The better torrent sites publish "how-to" guides for submissions that go into gory details about the exact tools and settings needed for high quality encodings.
Eg. Here's one such guide (attached).
Attachments
encoding guidev2.1.pdf (723 downloads)Description: Encoding guidelines
|
|
Top
|
|
|
|
|
#354764 - 11/09/2012 11:40
Re: Insanely fast video/audio encoding machine
[Re: mlord]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
I got into reading about avisynth last night and I had to stop for fear I was going to liquify my bowels. It's not too hard to understand how to create a script, it just that I could literally, spend the rest of my life playing with settings combinations. Converting the HD captures, animated and some live-action content, will be my main tasks, saving a lot of storage space that I can't afford to take up long-term on my DVR machine. But I also have some DVDs of classic 70's anime that I've just ripped without transcode that I'd love to tune-up somewhat before final conversion. A touch of saturation, contrast, line darkening and maybe a tiny bit of noise removal for example.
Edited by hybrid8 (11/09/2012 11:44)
|
|
Top
|
|
|
|
|
#354790 - 12/09/2012 02:21
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
carpal tunnel
Registered: 08/06/1999
Posts: 7868
|
Below are the Handbrake results from a few Macs I had access to. I'm not sure where it went, but I also have an H.264 USB accelerator somewhere that I could test with. Sadly no ffmpeg support ever materialized for it (CrusherHD). I was hoping to get this working on my ReadyNAS for better AirVideo transcodes. Start with Handbrake preset AppleTV 2 (720p)
Change frame rate to "same as source"
Audio - audio track 1 AC3 passthrough - eliminate other track
Text - keep all sub-picture content, un-check forced and burned 2012 Retina MBP (2.6GHz i7 3720QM Quad Core) - 0:55 encode time, 48.89 fps, 66.4MB 2008 Mac Pro (2.8GHz Xeon E5462 Dual Quad Core) - 0:57 encode time, 47.81 fps 2011 MBP (2.5GHz i7 2860QM Quad Core) - 1:12 encode time, 37.15 fps 2010 Mac Mini (2.4GHz Core 2 Duo P8600) - 5:04 encode time, 8.83 fps Also tried it in Quicktime X on the Retina MBP, using the normal 720p with Intel Quicksync. Also 0:55 encoding time, but half the CPU usage. Output file 86.8MB Quicktime X with the Apple TV 720p preset took a little longer at 1:03, similar near max CPU usage as Handbrake. Output file 81.1MB
|
|
Top
|
|
|
|
|
#354839 - 13/09/2012 21:16
Re: Insanely fast video/audio encoding machine
[Re: drakino]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Those all much better than my times.  Maybe I should consider making a Raspberry Pi cluster?
These guys call it a "supercomputer" but that's not quite fair until they measure the speed. It looks like a single quad-core i7 desktop is probably the best bet without going into the many thousands that it would cost to use two current-gen Xeons.
Edited by hybrid8 (13/09/2012 21:18)
|
|
Top
|
|
|
|
|
#354842 - 13/09/2012 21:50
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
veteran
Registered: 21/03/2002
Posts: 1424
Loc: MA but Irish born
|
I was going to ask if an 8 core AMD would be better, but based on this random single benchmark I pulled up, it is equal not better then an i7.
|
|
Top
|
|
|
|
|
#354843 - 13/09/2012 22:12
Re: Insanely fast video/audio encoding machine
[Re: Phoenix42]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Hmm... I don't know about the validity of the divx test compared to h.264. It's a shame they didn't use a benchmark that was geared toward current common encoding tasks, rather than those of 10 years ago. And that web site actually employs javascript to prevent people from copying text from their site. Annoying. But it only takes a second or two to un-tick the "enable javascript" pref, then use the site, then just turn the pref back on. How douchey. How about this: http://forums.overclockersclub.com/index.php?showtopic=189505The first graphs at the link above should be relevant and compare quite a number of different CPU/clock speeds. Now I'm also trying to find any comparisons (without over-clocking and extra coolers) to Xeon E3-1230
Edited by hybrid8 (13/09/2012 22:30)
|
|
Top
|
|
|
|
|
#354855 - 14/09/2012 09:49
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
carpal tunnel
Registered: 30/04/2000
Posts: 3810
|
There's something of a debate in the supercomputing universe about this very issue. On the one side, you've got people saying that you want a smaller number of fancy big/fast computers, since communication overhead tends to limit scalability. On the other hand, you've got people who worry about power consumption and say that large numbers of slow nodes, despite having lower utilization, might still win. Here's a researcher looking at this from the low-power perspective. I'd say that if you want maximum performance without power constraint, then you'd go for traditional CPUs. Otherwise, maybe little ones.
|
|
Top
|
|
|
|
|
#354862 - 14/09/2012 11:34
Re: Insanely fast video/audio encoding machine
[Re: DWallach]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
In the end my concerns, given a decent level of encoding performance, are primarily cost and simplicity.
|
|
Top
|
|
|
|
|
#354913 - 14/09/2012 21:00
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Ok, more research... With more thought given, I've come to the conclusion that I'd really like the possibility to run Mac OS X on whatever machine I put together. Thankfully there are some good resources for this, including forums where people are trying/recommending various product configurations: http://tonymacx86.comHere's what I'm thinking about right now. And this is subject to change depending on when I pull the trigger. CPU: Intel i7-3770K 3.5GHz (which I'd like to overclock to 4.2GHz) MB: Gigabyte GA-Z77-DS3H RAM: 16GB (2x8GB) A complete system can be had for $754 on Amazon right now. Excluding keyboard/mouse/display, which I have plenty of, but also excluding a replacement CPU fan which will probably be a good idea for overclocking. I can probably drop $90 off the price by not buying an additional drive as I can probably find one or two around here as well.
|
|
Top
|
|
|
|
|
#354919 - 14/09/2012 22:04
Re: Insanely fast video/audio encoding machine
[Re: hybrid8]
|
veteran
Registered: 21/03/2002
Posts: 1424
Loc: MA but Irish born
|
|
|
Top
|
|
|
|
|
#354920 - 14/09/2012 22:50
Re: Insanely fast video/audio encoding machine
[Re: Phoenix42]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Yup, found that site last night. The Xeon saves about $100 but I can overclock the i7...
|
|
Top
|
|
|
|
|
|
Previous Topic
Index