Thursday, October 15, 2009

Video processing on Haskell - easy

Almost all my past jobs were in the field on video processing, creating custom filters, decoders and encoders. Several of these jobs were to implement complete video/audio players. I’ve used ffmpeg library to perform all these tasks, except once I’ve used GStreamer. I can’t say that GStreamer is bad, but it didn’t allow me to completely control the decoding process, stucks on malformed video files and it was really painful to develop the 24x7 application.

And I returned to ffmpeg again, but than I became a haskeller :) so the idea was to made ffmpeg closer to Haskell community. Today let me introduce the hs-ffmpeg library for haskell.

Current version (0.3.2) supports basic file opening, streams enumerating, video and audio decoding. To make the process more interesting I took FFMpeg and SDL tutorial and implemented bindings in the sequence they are appeared in this tutorial. The second project, that shows how to use hs-ffmpeg and implements tutorials is ffmpeg-tutorials. It depends on SDL, so I’ve used Haskell SDL bindings to work with video and audio.

Several notes on building the sources.

Building hs-ffmpeg

You have to change hs-ffmpeg.cabal file to tune your ffmpeg library settings. In my case, ffmpeg installed under /usr/local tree, so my include dirs have /usr/local/include line and extra-lib-dirs contains /usr/local/lib value. Also in the extra-libraries you should specify the complete list of libraries, against which your version of ffmpeg have been buit.

If you do everything right, than hs-ffmpeg will build without any errors.

Patching the SDL

If you want to run ffmpeg-tutorials (sure you want :)), than you have to patch SDL bindings first. Current SDL bindings version is 0.5.5, and I’ve created SDL audio patch to this version, so do the following:

cabal fetch SDL
cd /tmp
tar xzvf ~/.cabal/packages/
cd SDL-0.5.5
patch -p1 < SDL-0.5.5-audio.patch

Than configure and build bindings. (Note: you’ll have to install SDL development libraries on to your system).

Running ffmpeg-tutorials

Currently only three tutorials are implemented. tutorial03 is completely showing capabilities of ffmpeg in video and audio decoding. Just simply run it with video fil ename as an argument to see how it decodes and shows video and plays sound.

Future work

I have only started this work, but I’d glad to hear comments and suggestions from people who interested in video processing on Haskell.

My future work plans are splitted onto several parts:

  • Continue implementing bindings to ffmpeg library

  • Improve build system, profiling

  • Started with multimedia combinators library, to program video and audio applications in more functional way (it is my dream)


1 comment:

  1. Instead of cabal fetch and then untar the file, you could just do a cabal unpack.