Freej beginner tutorial

FreeJ is a vision mixer: a digital instrument for realtime video manipulation used in the fields of dance teather, veejaying, online streaming, medical visualisation and TV.

It runs a video engine in which multiple layers can be filtered thru effect chains and then mixed together with images, movies, live cameras, particle generators, text scrollers and vector graphics. All the resulting video mix can be shown on a screen, encoded into a movie and streamed live to the internet.

FreeJ can be controlled locally or remotely, also from multiple places at the same time, using its ascii console interface; operations can be scripted in javascript and triggered live via keyboard, mouse, MIDI controllers, Joysticks, OSC clients, Wiimotes and more devices.

FreeJ's sourcecode is written in portable C and C++ and it works on most platforms supported by the GNU C compiler, including 32bit and 64bit processors, PowerPC and various ARM flavours.

FreeJ is released free under the GNU General Public License (v3).

FEATURES

• reads and renders layers from multiple sources: webcams, TV, divx/avi movies, images, txt files and more
• can encode in Ogg/Theora video, recording on local file and streaming live to an online Icecast server

• can be controlled from remote: VJoE - VeeJay over Ethernet

• can be scripted in object oriented javascript
• supports frei0r and freeframe video plugins
• very efficient video engine with multithreaded layers
• Emacs/Vi style console with hotkeys and completion
• multiple controllers at the same time (Midi,Joystick etc.)
• 100% Free and open source, GCC 4 compliant portable code

HISTORY

This software started being developed as a digital instrument Jaromil used in dance-theater performances. Since 2001 ongoing development took inspiration from various artists and programmers: Andreas Schiffler, Roberto Paci Dalo', Tom Demeyer, Francescopaolo Isidoro, Kentaro Fukuchi, Luigi Pagliarini, Isabella Bordoni, to name just a few.

Set the VeeJay Free! was the first motto for this software.

In 2003 Kysucix joined development contributing the streaming feature and helping to include the javascript parser. He employed FreeJ in interactive installations while working with Studio Azzurro.

Since 2004 development has received support from the Netherlands Media Art Insitute.

In 2005 Mr.Goil joined development, writing programmable controllers, reviewing the scripting environment and adding more features.

In 2007 the austrian initiative Netculture lab supported Jaromil and Mr.Goil developing the BeTV release: it enhanced scriptability and streaming, with a major cleanup of the code and wider support of video plugins.

In 2008 both Jaromil and Mr.Goil are meeting regularly in code sessions which are rapidly driving the project towards a stable 1.0 release of FreeJ engine and javascript API. Meanwhile Blender2Crystal developer Caedes is experimenting with python bindings and uses of the FreeJ engine in a 3d environment.

For the time being, FreeJ is employed in various video performances, interactive installations and online TV streams, as well used for visualisations in medical analysis.

Developers are keen to accept projects and propositions in order to sustain the development activity and involve more developers, please join our mailinglist on http://lists.dyne.org to collaborate and be part of our history

GET STARTED

To start rolling with FreeJ you can have a look also to the README, INSTALL and other files distributed with the package or source code. Also, check the online documentation on http://freej.dyne.org

Most powerful uses of FreeJ involve scripting for live interaction with devices and video compositing, still a first look at the console controller can give an initial picture of how the software works.

Tutorials, examples and scripting reference are available from the website as well along with the distributed sourcecode, in the doc/ and scripts/ directories. GNU/Linux distributions usually ship these files inside /usr/share/freej and /usr/doc/FreeJ*

As development unrolls suggestions and feedback are welcome, join us on the freej mailinglist on http://lists.dyne.org and let us know your impressions and ideas.

Target of this manual

In this part you will learn:

• installing freej

• what is the basic idea of using freej, how its user interface is organized

• how to use freej to show movies and static images

• how to use layers, so you can use more than one movie
• how to mix layers with each other
• how to add visual effects to displayed movies

However, we assume that you:

• can use a GNU/Linux system (command line)

• will really do all the exercises from this tutorial -- if you only read it, you'll learn nothing. You have to put hands on!

• will let us know about any problems, comments and suggestions, so we can make this tutorial better

contents

• installation

• the first start -- how to start and stop freej

• user interface -- what is what in freej

• showing movies with freej

• using layers

• mixing two movies (fun starts)

• adding transparency

• using effects -- basics

• effects disabling and deleting

installation

requirements

To compile and run FreeJ it is necessary to have:

1. a working GNU/Linux system or Darwin/OSX -> see the documentation in README.OSX

2. SDL libraries http://www.libsdl.org

3. PNG libraries http://www.libpng.org

4. S-LANG libraries http://www.s-lang.org

Several other libraries are optional and strongly recommended to benefit from all features implemented in FreeJ, in particular Ogg/Vorbis/Theora libraries available on http://www.xiph.org

how to install

GNU/Linux users

On Debian and Ubuntu 'freej' stable packages are ready to install. Release candidates up to date with latest development are also published on: http://launchpad.org/~jaromil/+archive

On Gentoo it is also possible to 'emerge freej'

Apple/OSX users

Binary packages of FreeJ for OSX should be available from the website http://freej.dyne.org, although they might be a bit outdated and limited in features. It is also possible to compile FreeJ from source following the README.OSX and using XCode, MacPorts and Fink.

In case you are a programmer talented with OSX please consider to get in touch with us and contribute to make this software better on the Apple platform, there isn't much work to do anyway.

Windlows users

No-one of us feels to compile FreeJ on the M$ platform, but there is a quick and easy way to try this software using the dyne:bolic liveCD even without installing anything, just boot it.

• dynebolic live CD.

  

Game console users

Experimental builds of FreeJ have succeeded to work on consoles as GP2X and NintendoDS. It should be also possible to make FreeJ run on other embedded devices and we are very interested in doing so, please contact us if you are as well

installing from source code

• So yo' smart, huh? I thou' yo' head woul' be biggah! (Idiocracy, Mike Judge, 2006)

x86 PC

First, be sure to have:

• all the required libraries and tools:

cdbs, libtool, flex, bison, libsdl-dev, libpng-dev, libfreetype6-dev, libfontconfig-dev, libogg-dev, libvorbis-dev, libjpeg-dev, libslang2-dev, libtheora-dev, libavcodec-dev, libavformat-dev, libswscale-dev, libunicap2-dev, libbluetooth-dev, fftw3-dev, libjack-dev, libasound-dev, libhtml-template-perl, python-dev, swig

If you're going to create Debian packages, you need also debhelper and pkg-config

PLEASE NOTE: automake 1.9 is REQUIRED - it doesn't work with a different version (we're hardly working to fix this).

• A working GNU/Linux system with X or framebuffer video

• SDL libraries http://www.libsdl.org

• PNG library (compile with _ZLIB_ support) http://libpng.org

• To have some more assembler mmx optimized filters you need NASM:

NASM netwide assembler compiler http://sf.net/projects/nasm

then go in the filters/nasm-x86 and type

make

copy by hand the *.so filters in a ~/.freej/ directory in your home, and you're ready to rock'n roll!

Now, download freej latest stable version from

http://ftp.dyne.org/freej

extract the source archive and compile it:

   $ tar xvfz freej.tar.gz
   $ cd freej
    (if you are reading this file from the sourcecode, start from here)
   $ ./configure
   $ make
   $ sudo make install

this will install:

• effect plugins into /usr/local/lib/freej/
• freej binary in /usr/local/bin/

(or any other prefix you configured instead of the default /usr/local)

To launch it, just type

freej

and that's all!

Darwin/OSX

Please see README.OSX into the tarball.

from debian packages

Assuming you can do

apt-get install freej

to have a precompiled FreeJ, to squeeze the best out of your damn box you may want to compile this software with machine specific optimizations!

The source configure script guesses the best compiler optimization flags for your CPU.

The following commands are then necessary to set your build environment, from inside the freej source directory do:

sudo apt-get install autoconf automake1.9
sudo apt-get build-dep freej
dpkg-buildpackage -rfakeroot

INSTALL FROM GIT REPOSITORY

• (I know shit's bad right now with all that starvin' bullshit. And the dust storms. And we runnin' out of French Fries and burrito coverings. But I got a solution. - Idiocracy, Mike Judge, 2006)

Assuming you have Debian/Ubuntu (for other distro there can be some differences), with all tools needed and all developers lib installed (e.g. build-essential, the libs above etc.:

You need git:

apt-get install git-core

So, get the code:

git clone git://code.dyne.org/freej.git

After this, you will have a "freej" dir. Enter in it, and do

./autogen.sh

that create all the necessary configuration files (It may ask you to install some missing libraries or to install the required automake version 1.9). This also run the "configure" script.

If you need to enable/disable something, relaunch "configure" with your favourite options.

When you're satisfied:

make && sudo make install 

That's all!

If you want to experiment with live video, please be sure to have a working camera and to activate it from your BIOS settings (e.g. for eeepc 701), otherwise it will not work!

the first start

Create a directory, for example freej_tmp. We will put our images, movies etc in this directory and we will play inside it:

 mkdir freej_tmp
 cd freej_tmp

Download this image to freej_tmp directory. Now start freej:

 freej ipernav.png 

As a result, freej starts and shows the picture ipernav.png.

user interface

Freej consists of two windows:

• output window – in this window our movies, pictures etc are shown,

• console window – this window is used to interact with freej – give commands, load new pictures, movies etc,

  

In console window, press ctrl-h. You will see the list of shortcuts.

As you can see, [ctrl+c] means quit. Press [ctrl+c]. You are prompted to confirm that you really want to quit freej. Type yes [enter]

some more fun – movies

So far we just used static image – not so much fun. But the very same way we can use a movie.

Download this movie to freej_tmp directory. Then start freej:

 freej kury.avi

As a result, freej starts and shows the movie kury.avi.

quit the freej (remember? [ctrl+c]).

more layers

We can load both static image and the movie.

Start freej with both movie and static image:

 freej kury.avi ipernav.png 

As you can see, only chicken movie is visible (and the static image – file ipernav.png – is invisible). We are going to understand why...

Now, when we opened two images (static image and a movie) in freej, we have them on two layers.

You can see the list of layers in the console. As you can see, that layers are called VID (which means: video layer) and IMG (static image layer).

The list of layers is also shown in output window. As you can see (below), the video layer is on top of static image layer. This is why we can't see the image.

Take a look at the console window. As you can see, one of layers (VID) is highlighted. It is the current layer. You can see the detailed information about that layer in the top of console window.

You can see there the name of the file (kury.avi) and some other info, which we will understand later.

Notice that current layer doesn't mean top layer or the layer which is visible now. Current layer means just current layer.

While in the console window, you can use left and right arrow keys to change the current layer. Press the right arrow key to change the current layer to IMG layer. Notice that now at the top of the console window you can see the details of static image layer.

While in the console window, you can use page up and page down keys to move the current layer up and down. So now, when the current layer is IMG layer, press page up. As you can see, the static image layer went to the top. Now the static image is visible and the chicken movie became invisible.

Close the freej (ctrl+c, as you remember).

some words about commands, parameters, completions

Some of the console commands (e.g. CTRL-B, CTRL-E) can accept parameters. To show them, just press TAB. The completion is also working.

For example, pressing CTRL-B you have:

 [*] select Blit mode for the selected Layer - press TAB for completion:

and so, pressing TAB, you obtain:

 [*] List available blits starting with ""
 RGB    ADD     SUB     MEAN
 ABSDIFF        MULT    MULTNOR DIV
 MULTDIV2       MULTDIV4        AND     OR
 XOR    RED     GREEN   BLUE
 REDMASK        GREENMASK       BLUEMASK        NEG
 ADDB           ADDBH   SUBB    SHL
 SHLB   SHR     MULB    BIN
 SDL    ALPHA   SRCALPHA        CHROMAKEY

When choosing somethink, like a filter (pressing CTRL-E), also you can use TAB and completion:

 [*] add new Effect - press TAB for completion:
 3dflippo       Brightness      bw0r    Cartoon
 Contrast0r     delay0r Distort0r       Equaliz0r
 Flippo Gamma   Glow    Hueshift0r
 Invert0r       Mask0Mate       nosync0r        pixeliz0r
 rotozoom       Saturat0r       scanline0r      Sobel
 Squareblur     TehRoxx0r       Threshold0r     Twolay0r
 Vertigo        Water

This also give you a list of all effect.

mixing two movies

As we previously told, freej is about mixing images. Now we will start doing that.

Download another movie to freej_tmp directory. Then start freej:

 freej kury.avi term.avi 

As a result, freej starts. kury.avi is on the top layer, and is visible. term.avi is on the bottom layer and thus is invisible.

Now press key CTRL-B, and write red in output window. The effect should look like that:

Pressing key [2] we made red blit on active layer. You can see it in layer details, in the top of console window.

As you know, each picture an a computer screen has three channels: red, green and blue. Red blit means that now only red channel of the kury.avi layer is visible. Green and blue channels of this layer became transparent, so now we can see:

• the red channel of kury.avi

• the green channel of term.avi

• the blue channel of term.avi

In very similar fashion we can use green blit (it's [3] key) or blue blit ([4] key). You can also try using other blits (keys [1]-[9]). Also try switching [0] on and off – I don't know what does it do, but you will see the difference.

alpha blit

And now press key [9].

As you can see, now we use alpha blit (you can read it in the info on current layer in the console window). With alpha blit, the current layer becames partly transparent. As you can see, now the layer kury.avi became fully transparent and only term.avi is visible. This is because alpha parameter is zero:

We can change the alpha parameter with mouse. Just press [ctrl+v] in the output window. Now you can move the mouse up and down and alpha will change. When you are done, press [ctrl+v] again. Notice what happens when alpha is about 127 – the current layer becomes half transparent.

Effects

Start freej with term.avi:

 freej term.avi 

In console window press [ctrl+e] (e like effect). You will see the prompt like that:

So just press [tab] and you will see the list of available effects:

Vertigo effect is a nice one. So just type vertigo [enter]. You will see something like that:

disabling and deleting effects

Take a look at the console window. As you can see, the current layer is video layer with file term.avi:

You can also see that this current layer has the effect vertigo:

Press down arrow key in the console window to see the details on this filter:

Press [insert] key to temporarily disable and enable this effect. Press [delete] to delete this effect from this layer.

TEAM

• Denis "Jaromil" Rojo - FreeJ author and mantainer
• Silvano "Kysucix" Galliani - TXT layer, encoder, streaming
• Christoph "Mr.Goil" Rudorff - Scriptability, MIDI and Joystick

parts of included code are written by Andreas Schiffler (sdl_gfx), Jan (theorautils), Dave Griffiths (audio bus), Nemosoft (ccvt), Charles Yates (yuv2rgb), Steve Harris (liblo), Sam Lantinga (sdl_*), Jean-Christophe Hoelt (goom), L. Donnie Smith (cwiid), Olivier Debon (flash).

documentation, testing and user case studies have bee contributed by Anne-Marie Skriver, Marloes de Valk, Piotr Sobolewski

refer to the AUTHORS file for a full list of contributions

DISCUSSION

A mailinglist for further discussion about FreeJ is running on

• http://lists.dyne.org

For chat we hang around the channel #dyne on irc.dyne.org - access is free via SSL on port 9999, connection IP is kept anonymous for your own privacy.

If you have problems, you are welcome to ask on the mailinglist for help, there you will probably find some good suggestions

Please try to not mix your system's problems with FreeJ's bugs.

If you find bugs, you are welcome to report them in the bugtracking system on http://bugs.dyne.org

DISCLAIMER

FreeJ is (c) 2001 - 2008 by Denis Roio

• (c) 2004 - 2005 by Silvano Galliani (c) 2005 - 2008 by Christoph Rudorff

Statically included libraries are copyright of the respective authors.

This source code is free software; you can redistribute it and/or modify it under the terms of the GNU Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.

This source code is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Please refer to the GNU Public License for more details.

You should have received a copy of the GNU Public License along with this source code; if not, write to: Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

about this document

The original document was by Piotr Sobolewski (http://www.rozrywka.jawsieci.pl/materialy/index_EN.html)

It was revised, enlarged and rebuilt at Wintercamp 09 in Amsterdam, by Gabriele Asbesto Molesto Zaverio.

CategoryTemplate