Hi, I'm trying to capture video and audio from a webcam in order to encode it with the Theora and Speex DirectShow-filters. For audio, this filter graph plays fine: http://www.huitl.de/mic-speexenc-speexdec-out.png. The video part is problematic. Please have a look at http://www.huitl.de/cam-theoraenc-theoradec-out.png. 1. The webcam provides the color formats RGB24, I420 and IYUV. The Theora encode filter only accepts YV12, so I can't just connect the webcam capture pin with theora's yv12-in. 2. The "Color Space Converter" can't do the conversion to YV12. 3. I found that the ffdshow raw video filter should be able to do this conversion (although I'd like to have no dependencies other than Theora and Speex). 4. When I render the Theora decode filter's output pin, DirectShow attaches an AVI decompressor, a color space converter and finally the video renderer. 5. I cannot play the filter graph. When trying to do so, GraphBuild shows: "This graph can't play", return code 0x80070000. 6. I tried to replace the AVI decompressor and the color space converter with another ffdshow raw video filter. When I connect the ffdshow-filter's output with the video renderer, GraphBuild crashes: http://www.huitl.de/cam-theoraenc-theoradec-ff-out.png I'd appreciate any help doing this: 1. Capture video from the webcam and encode with Theora. 2. Access the encoded video data in order to send it over a network. 3. Feed data received over a network to the Theora decoder, which in turn should pass it to a video renderer. Used versions: - oggcodecs_XP_0.63.7755 - ffdshow-20040828 - DirectX 9.0b + SDK 9.0b Regards, Robert -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://lists.xiph.org/pipermail/theora-dev/attachments/20040914/f2d9194b/attachment.pgp
----- Original Message ----- From: "Robert Huitl" <theora-dev@huitl.de> To: <theora-dev@xiph.org> Sent: Wednesday, September 15, 2004 12:30 AM Subject: [Theora-dev] Problems with Theora DirectShow filters>Hi, > >I'm trying to capture video and audio from a webcam in order to encode it >with >the Theora and Speex DirectShow-filters. For audio, this filter graph plays >fine: http://www.huitl.de/mic-speexenc-speexdec-out.png.It does... but i don't see the point of encoding it if you are going to decode it straight away... the encoders aren't designed to be chained back into their matching decoder.>The video part is problematic. Please have a look at >http://www.huitl.de/cam-theoraenc-theoradec-out.png.Same goes here... linking an encoder back to a decoder won't always work directly (particularly with video... as the video decoder generally expects the video renderer to provide the custom allocators to draw straight onto a directdraw surface). You should build a graph something like this if you want to capture... Audio Input Source --> Speex Encoder --> } } Ogg Mux Filter Video Input Source --> Theora Encoder --> }>1. The webcam provides the color formats RGB24, I420 and IYUV. The Theora >encode filter only accepts YV12, so I can't just connect the webcam capture >pin with theora's yv12-in.Yes... this is a problem at the moment... probably in the next release i'll include a conversion from RGB types to YUV types.>2. The "Color Space Converter" can't do the conversion to YV12.Yeah... the colour space converter is pretty useless !!>3. I found that the ffdshow raw video filter should be able to do this >conversion (although I'd like to have no dependencies other than Theora and >Speex).At the moment that is probably the only option... until i include the colour space conversion in the theora filter.>4. When I render the Theora decode filter's output pin, DirectShow attaches >an >AVI decompressor, a color space converter and finally the video renderer.One thing... you should never "Render" encoding graphs... the automatic graph building is for decoding graphs only.>5. I cannot play the filter graph. When trying to do so, GraphBuild shows: >"This graph can't play", return code 0x80070000.It's because the encoder and the decoder shouldn't be directly chained. There is no need to do this (excpet perhaps testing)>6. I tried to replace the AVI decompressor and the color space converter >with >another ffdshow raw video filter. When I connect the ffdshow-filter's >output >with the video renderer, GraphBuild crashes: >http://www.huitl.de/cam-theoraenc-theoradec-ff-out.pngHmmm... i'll have to have a look into this... quite possibly it's because the ffdshow filter expects the downstream filter to create the sample allocator... but the encoder expects the upstream filter to do it... as the upstream filter is the one that knows what sort of samples it has, and what size and how many buffers it shold create. The excpetion is generally anything that could connect to the video renderer which expects the renderer to provide the allocators.>I'd appreciate any help doing this: >1. Capture video from the webcam and encode with Theora.For the moment... you'll need to find a colour space converter that can do the conversion properly. In the next release i'll include the RGB to YUV conversions. I know there are some proprietary filters out there... i haven't had much luck finding a good free colour space converter though.>2. Access the encoded video data in order to send it over a network.At the moment... none of the filters do network output... that would typically be done by an application. I do have plans to provide a callback interface on the muxer so that your application can grab the pages and do whatever it wants with them. If you want to feed packets... you will probably need your own custom filter at this stage... eg if you wanted to use rtp. At some point i may write a generic packet grabber to feed packets back to an application.>3. Feed data received over a network to the Theora decoder, which in turn >should pass it to a video renderer.This should already work... if you open a http resource (via full url) with the ogg demux source and render it... it should work fine. But again if you want to do packet level, and not page level network transport with custom or non-http protocols, you'll need to write a custom filter (or use something like the sample grabber filter) and do the protocol/network operations in the application code. It sounds like you are wanting to do some kind of video voip application... as a general rule if rtp is not being used... it will probably be a custom protocol and should be done in application code or by writing an application specific filter. At some point i will probably write some rtp stuff... but for now i'm focussing on http streaming rather than trying to send data back out over the network. Though i will get to it at some point, it's just not the highest priority at the moment. So i'm more focussed at the moment on getting the core of the code working for the majority of the uses... specifics for particular styles of application should probably be written by the application developer as they will be application specific. I don't realyl want to spend time at this stage writing custom network protocol filters for individual applications. Cheers, Zen.
On Thursday 16 September 2004 21:40, David Barrett wrote:> In the meantime, I've been using the Intel Performance Primitives (IPP) > library for color conversion, and it's working great. It converts between > I420, YUY2, and RGB24, and a zillion other colorspaces. Plus it's > optimized for every Intel processor under the sun, has a free trial, is > free for non-commercial Linux apps, and is very reasonably priced otherwise > ($200). > > http://www.intel.com/software/products/ipp/index.htmInteresting. But IPP do not provide a DirectShow filter, do they? How do you integrate them into a filter graph? Robert -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://lists.xiph.org/pipermail/theora-dev/attachments/20040918/5271bda3/attachment.pgp
On Friday 17 September 2004 10:49, illiminable wrote:> >You can download it at www.huitl.de/theora-speex-mux.ogg. > > This has no video frames in it !! Also... since seeking takes place on the > first stream... you should really have theora as the first stream in the > file (this means on the bottom pin of the muxer)... it doesn't matter too > much for now... but when i do keyframe seeking it might.Well, my code connects the theora encoder to the muxer first, and then the audio encoder. Should I do this in reverse order and add the first stream last? I put another file online at http://www.huitl.de/theora-only.ogg. As the name says, it's only a theora stream, captured from the webcam and color-converted by ffdshow. It plays in GraphEdit (but not with the ffdshow MPEG4 filter), but it looks to fast and the graph causes a time-out at the end.> I suspect the problem is that the ffdshow filter strips the granule pos > time stamps, and attaches it's own... which are in directshow time... which > are significantly higher numbers (1 sec = 10,000,000 UNITS), so when it > muxes, all the theora frames will be bunched up at the end. Potentially > even an overflow could occur.I didn't understand this one completely, but it sounds reasonable :) With the ogg-file mentioned above, you should be able to find out if this is the case.> >P.S.: In case you can't get audio/video working, the resolution is > >320x240/RGB24@30fps (at least, that's the setting. Don't know if the > > webcam can supply that much with USB 1.1). > > Ok... was just checking... as i have noticed a bug when the resolution is > very high... in particular where you have a very wide, but not very tall > frame... i'm not sure if that is occuring in my code or in theora at this > stage. But that resolution is certainly within the range that i have > previously tested.-> http://www.huitl.de/videocap-playback.png There are two resolutions in the filter graph properties, 320x240 and 384x240. Though it might not be a problem, it looks strange to me.> PS: Could i also ask (i'm not sure if it was accidental or not)... that > except for private correspondence (or where you want to keep specific > information off the list), you keep as much of this on the list as > possible... it makes it easier to capture the information and make it > available to everyone who may be encountering similar issues. Cheers.Sure. I forgot to set up a correct reply-address for postings to this mailing list. Regards, Robert -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://lists.xiph.org/pipermail/theora-dev/attachments/20040918/45abb622/attachment.pgp