A short story of interlacing in HEVC

And how it nearly ruined another video coding standard.

Everyone knows that interlacing was a trick in the '80s for pseudo motion compensation with analogue video. This more or less worked because it mimicked how television worked back then. This technique was preserved when flat panels for pc and tv were introduced, for a mix of backward compatibility and technical limitations, and video coding features interlacing in MPEG2 and H264 and similar.

However as with black and white, TACS and Gopher, old technology has to be replaced with modern and efficient technology, as a trade off of users' interests and technology providers' market prospects. In case you are not familiar, interlacing is a mess to support, makes decoding slower and heavily degrades quality. People saying that interlacing saves bandwidth do not know much about video coding and bad marketing claiming that higher resolution is better than higher framerate has an effect too.

So, when ITU and then MPEG set out to establish the mandates for a new video standard capable of superseding H264, it was decided that interlacing was old enough, did more harm than good and it was time for retirement: HEVC was going to be the first video codec to officially deprecate interlacing.

Things went pretty swell during its development, until a few months before the completion of the standard. A group of US companies complained that the proposed tools were not sufficient (a set of SEI messages and treating fields like progressive frames) and heavily protested with both standardisation bodies. ITU firmly rejected the idea (with the video group chair threatening to step down) while MPEG set out to understand the needs of the industry and see if there was anything that could be done.

An ad-hoc group was established to see if there was any evidence that interlaced coding tool would have improved the situation. Things looked really shady, the Requirements group even mentioned that it was the first time that an AhG was established to look for evidence, instead of establishing an AhG because there was evidence. Several liasons from EBU and other DVB members tried to point out this absurdity while the threat of adding interlacing back in HEVC became real. Luckily the first version of the specifications got published in the meantime, so this decision didn't slow down the standardisation process.

Why so much love towards interlacing? Well in the "rebellious" group defence, it is true that interlaced content in HEVC is less performant than in H264; however it is also true that such deinterlaced content in HEVC outperforms H264 in any configuration. Truth is that mass marketed deinterlacers (commonly found in televisions for example) bring a lot of royalty income, so it is normal that companies with vested interests would prefer to have interlacing in a soon-popular video standard like HEVC. Also in markets like US where the network operator (which has control on the encoding but not on the video source) might differ from the content provider, it could be politically difficult to act as a carrier only if you have to deinterlace a video.

However these problems are actually not enough for forcing every encoder, decoder, analyser to support a deprecated technology like interlacing. Technical problems can be solved with good deinterlacers at the top of the distribution chain, while political ones can be solved amending contracts. Plus having progressive only video will definitely improve quality and let the industry concentrate on other delicate subjects, like bit depth, both properties going in favour of users' interests.

At the last MPEG meeting, the "rebellious" group which had been working on reintroducing interlacing for a year provided no real evidence that interlaced coding tools would improve HEVC at all. The only sensible solution was to disband the group over this wasted effort and support progressive video only, which is what happened luckily. So now both ITU and MPEG support progressive video only and this has finally nailed it.

Interlacing is dead, long live progressive.

Written by Vittorio Giovara (projectsymphony@gmail.com)
Published under a CC-BY-SA 3.0 license.

Libav 10 and my first year with the team

I am very glad to announce that Libav 10 has been released!

This has a bunch of features that I contributed to, in particular regarding stereoscopic video and interlaced filtering, but more importantly this release has the work of an awesome group of people which has been carried out for a whole year. This is the magic of open source!

I joined the group more or less one year ago, with some patches regarding an obscure H.264 specification which I then later reimplemented in HEVC and then I wrote a few filters I needed and then designed an API and then, wow! A whole year passed without me noticing, and I am still around, sending patches to the same group of people who welcomed someone who had problems with shifting values (sad but true story)!

I met the team both at VDD and FOSDEM and they've been the most exciting conferences I ever went to (and I went to a lot of them). I couldn't believe I was with the devteam of my favourite multimeida opensource projects I've been following since I was a kid! Until a year ago, I saw the names from the commits and the blogposts from both VideoLAN and Libav projects and I had been thinking "Oh wouldn't it be so cool to be like one of them".

The answer is yes, it definitely would, and it's something that can happen if one is really committed in it! The Libav Info page states "Being a committer is a duty, not a privilege", but it sure does feel like one.

Thanks for this exciting year guys, I look forward to the next ones.

Libav function prefixes

For future reference, all (or most of) functions and structs in libav have a prefix that indicates the exposure of that functions. Those are

• av_ meaning a public function, present in the API;
• ff_ meaning a private function, not present in the API;
• avpriv_ meaning inter-library private function, used internally across libraries only.

Source: #libav-devel

How to cross compile from Linux to Windows, x264, VLC and Libav

...using latest modern tools!

X264 and VLC are two of the most awesomest opensource software you can find on-line and of course the pose no problem when you compile them on a Unix environment. Too bad that sometimes you need to think of Windowze as well, so we need a way to crosscompile that software: in this blogpost, I'll describe how to achieve that, using modern tools on a Ubuntu 12.04 installation.

[0] Sources
It goes without saying that without the following guides, I'd have had a much harder time!
http://alex.jurkiewi.cz/blog/2010/cross-compiling-x264-for-win32-on-ubuntu-linux
https://bbs.archlinux.org/viewtopic.php?id=138128
http://wiki.videolan.org/Win32Compile
http://forum.videolan.org/viewtopic.php?f=32&t=101489
So a big thanks to all the original authors!

[1] Introduction
When you crosscompile you just use the same tools and toolchains that you are used to, gcc, ld and so on, but configured (and compiled) so that they produce executable code for a different platform. This platform can vary both in software and in hardware and it is usually identified by a triplet: the processor architecture, the ABI and the operating system.

What we are going to use here is i686-w64-mingw32, which identifies any x86 cpu since the Pentium III, the w64 ABI used on modern Windows NT systems (if I'm not wrong), and the mingw32 architecture, that is the Windows gcc variant.


[2] Prerequisites
Note that the name of the packages might be slightly different according to your distribution. We are going to need a quite recent mingw-runtime for VLC (>=3.00) which has not yet landed on Ubuntu, so we'll take it from our Debian cousins.

Execute this command

$ wget http://ftp.jp.debian.org/debian/pool/main/m/mingw-w64/mingw-w64-dev_3.0~svn4933-1_all.deb
$ sudo dpkg -i mingw-w64-dev_3.0~svn4933-1_all.deb

and then install stock dependencies




$ sudo dpkg -i gcc-mingw-w64 g++-mingw-w64
$ sudo dpkg -i pkg-config yasm subversion cvs git-core

[3] x264 and libav 
x264 has very few dependencies, just pthreads and zlib, but it reaches its full potential when all of them are satisfied (encapsulation, avisynth support and so on).

Loosely following Alex Jurkiewicz's work, we create a user-writable folder and then we prepare a script that sets some useful variables every time.

$ mkdir -p ~/win32-cross/{src,lib,include,share,bin}

#!/bin/sh

TRIPLET=i686-w64-mingw32

export CC=$TRIPLET-gcc
export CXX=$TRIPLET-g++
export CPP=$TRIPLET-cpp
export AR=$TRIPLET-ar
export RANLIB=$TRIPLET-ranlib
export ADD2LINE=$TRIPLET-addr2line
export AS=$TRIPLET-as
export LD=$TRIPLET-ld
export NM=$TRIPLET-nm
export STRIP=$TRIPLET-strip

export PATH="/usr/i586-mingw32msvc/bin:$PATH"
export PKG_CONFIG_PATH="$HOME/win32-cross/lib/pkgconfig/"

export CFLAGS="-static -static-libgcc -static-libstdc++ -I$HOME/win32-cross/include -L$HOME/win32-cross/lib -I/usr/$TRIPLET/include -L/usr/$TRIPLET/lib"
export CXXFLAGS="$CFLAGS"

exec "$@"

Please not the use of the CFLAGS variables: without all the static parameters, the executable will dynamically link gcc, so you'll need to bundle the equivalent dll. I prefer to have one single exe, so everything goes static, but I'm not really sure which flag is actually needed. If you have any idea, please drop me a line.

Anyway, let's compile latest revision of pthreads (2.9.1 as of this writing)

$ cd ~/win32-cross/src
$ wget -qO - ftp://sourceware.org/pub/pthreads-win32/pthreads-w32-2-9-1-release.tar.gz | tar xzvf -
$ cd pthreads-w32-2-9-1-release
$ make GC-static CROSS=i686-w64-mingw32-
$ cp libpthreadGC2.a ../../lib
$ cp *.h ../../include

and zlib (1.2.7) - we need to remove the references to the libc library (which is implied anyway) otherwise we will get a linkage failure

$ cd ~/win32-cross/src
$ wget -qO - http://zlib.net/zlib-1.2.7.tar.gz | tar xzvf -
$ cd zlib-1.2.7
$ ../../mingw ./configure
$ sed -i"" -e 's/-lc//' Makefile
$ make
$ DESTDIR=../.. make install prefix=

Now it's turn for libav, so that x264 can use different input chroma and other stuff. If you need libav exececutables, you might want to change the configure line so that it suits you

$ cd ~/win32-cross/src
$ git clone git://git.libav.org/libav.git
$ cd libav
$ ./configure \
      --target-os=mingw32 --cross-prefix=i686-w64-mingw32- --arch=x86 --prefix=../.. \
      --enable-memalign-hack --enable-gpl --enable-avisynth --enable-runtime-cpudetect \
      --disable-encoders --disable-muxers --disable-network --disable-devices
$ make
$ make install

and the nice tools that give more output options

$ cd ~/win32-cross/src
$ svn checkout http://ffmpegsource.googlecode.com/svn/trunk/ ffms
$ cd ffms
$ ../../mingw ./configure --host=mingw32 --with-zlib=../.. --prefix=$HOME/win32-cross
$ ../../mingw make
$ make install

$ cd $HOME/win32-x264/src
# Create a CVS auth file on your machine
$ cvs -d:pserver:anonymous@gpac.cvs.sourceforge.net:/cvsroot/gpac login
$ cvs -z3 -d:pserver:anonymous@gpac.cvs.sourceforge.net:/cvsroot/gpac co -P gpac
$ cd gpac
$ chmod +rwx configure src/Makefile
# Hardcode cross-prefix
$ sed -i'' -e 's/cross_prefix=""/cross_prefix="i686-w64-mingw32-"/' configure
$ ../../mingw ./configure --static --use-js=no --use-ft=no --use-jpeg=no \

      --use-png=no --use-faad=no --use-mad=no --use-xvid=no --use-ffmpeg=no \

      --use-ogg=no --use-vorbis=no --use-theora=no --use-openjpeg=no \

      --disable-ssl --disable-opengl --disable-wx --disable-oss-audio \

      --disable-x11-shm --disable-x11-xv --disable-fragments--use-a52=no \

      --disable-xmlrpc --disable-dvb --disable-alsa --static-mp4box \

      --extra-cflags="-I$HOME/win32-cross/include -I/usr/i686-w64-mingw32/include" \

      --extra-ldflags="-L$HOME/win32-cross/lib -L/usr/i686-w64-mingw32/lib"
# Fix pthread lib name
$ sed -i"" -e 's/pthread/pthreadGC2/' config.mak
# Add extra libs that are required but not included
$ sed -i"" -e 's/-lpthreadGC2/-lpthreadGC2 -lwinmm -lwsock32 -lopengl32 -lglu32/' config.mak
$ make
# Make will fail a few commands after building libgpac_static.a

# (i586-mingw32msvc-ar cr ../bin/gcc/libgpac_static.a ...).

# That's fine, we just need libgpac_static.a 

i686-w64-mingw32-ranlib bin/gcc/libgpac_static.a 

$ cp bin/gcc/libgpac_static.a ../../lib/
$ cp -r include/gpac ../../include/

Finally we can compile x264 at full power! The configure script will provide a list of what features have been activated, make sure everything you need is there!

$ cd ~/win32-cross/src
$ git clone git://git.videolan.org/x264.git
$ cd x264
$ ./configure --cross-prefix=i686-w64-mingw32- --host=i686-w64-mingw32 \

      --extra-cflags="-static -static-libgcc -static-libstdc++ -I$HOME/win32-cross/include" \

      --extra-ldflags="-static -static-libgcc -static-libstdc++ -L$HOME/win32-cross/lib" \

      --enable-win32thread
$ make

And you're done! Take that x264.exe file and use it wherever you want!
Most of the work here has been outlined by Alex Jurkiewicz in this guide so checkout his blog for more nice guides!


[4] VideoLAN
On the other hand, VLC has a LOT of dependencies, but thankfully it also has a nice way to get them working quickly. If you read the wiki guide, you'll notice that it will use i586-mingw32msvc everywhere, but you should definitely avoid that! In fact that one offers a very old toolchain, under which VLC will fail to compile! Also the latest versions provides much better code, x264 will weight 46MB against 38MB in one case!

So let's update every script to the more modern version i686-w64-mingw32! As usual, first of all get the sources

$ git clone git://git.videolan.org/vlc.git vlc

$ cd vlc 

And let's get the dependencies through the contrib scripts, qt4 needs to be compiled by hand as the version in Ubuntu repositories doesn't cope well with the rest of the process. I also had to remove some of the files because they were of the wrong architecture (mileage might vary here) .

$ mkdir -p contrib/win32
$ cd contrib/win32
$ ../bootstrap --host=i686-w64-mingw32
$ make prebuilt

$ make .qt4

$ rm ../i686-w64-mingw32/bin/{moc,uic,rcc}

$ cd -

We now return to the main sources folder and launch the boostrap and configure process; you need some standard automake/libtool dependencies for this.

$ ./bootstrap

$ mkdir win32 && cd win32

$ ../extras/package/win32/configure.sh --host=i686-w64-mingw32

$ ./compile

$ make package-win-common

Let's grab something to drink and celebrate when the compilation ends! You'll find all the necessary files in the vlc-x.x.x folder. A big thanks goes to the wiki authors and j-b who gave me pointers on #videolan irc.

[5] Conclusions
Whelp, that was a long run! As additional benefit you are able to customize every single piece of software to your need, eg. you can modify the libav version that you are going to use for Vlc as you wish! Also crosscompiling is often treated as black magic, but in reality is a simple process that just needs more careful configuration. Errors often are related to wrong paths or missing dependencies and sometimes a combination of both; don't lose hope and keep going until you get what you want!