UsualCoding.eu - code

NFQueue python bingings

Seb — Mon, 07 Jul 2008 14:04:00 +0200

A few month ago I needed python bindings for the nfqueue netfilter library (libnetfilter_queue).

I was going to write them myself since I found none, but then I stumbled across this on the netfilter mailing list.

I like this wrapper because it is simple and the code is clear. I was able to adapt my work to them very quickly.

There were some little problems though.

First of all the unchanged code cannot be used in multithreaded programs (see http://www.python.org/doc/api/threads.html).

So I wrote a little patch which adds the necessary Py_BEGIN_ALLOW_THREADS and a "stop" function to stop the queue.

The code can be found here:

nfqueue-multithreading.patch

The second problem appears only on Linux >=2.6.23 (see http://www.spinics.net/lists/netfilter/msg42063.html).

Here is the patch:

nfqueue-2.6.23-fix.patch

Special thanks to Mathieu Geli for this wrapper, it helped me a lot and saved time in my work.

By the way, I found new nfqueue bindings based on SWIG that look nice. They are brand new, but I haven't tested them yet.

Links:

MP3Camp

Seb — Fri, 26 Oct 2007 22:13:00 +0200

I have been using GNUMP3d for some time now, but a few things bother me.
For example, it uses a standalone web server to stream the music, which is useful in some cases, but requires you to set it up on a different port if you already run a web server. I know I could setup a mod_proxy or something similar, but I like it when things work right out of the box.

There are things out there like ampache, but nothing as light as GNUMP3d. That is why I coded a small streamer of my own using camping a Micro framework in ruby. There is not much code yet but it is already usable.

One of the advantages of using camping, is that it can work as a standalone server, but it can also easily be setup in a virtual host on apache, lighttpd or whatever.

Here is a screenshot of the latest (pre alpha) version:

As you can see it is very much like GNUMP3d in the way that it gives you access to playlists you can download and use in any music player.
I also added an embedded flash music player you can use to just listen in quickly.
MP3Camp supports m3u and xspf playlists and really needs some CSS job.

You can grab this early version on the svn (svn://svn.sig11.org/mp3camp).

Building a reentrant parser in C with Flex/Bison

Seb — Mon, 03 Sep 2007 20:51:00 +0000

It is not easy for beginners to find all the right information on how to create a reentrant parser with tools like flex and bison. Most examples out there are out of date or not very clear.
Once I found out, I wrote a small example of how to do it. The objective is not yet another flex/bison tutorial, but to focus solely on the reentrant code aspects.
It is a simple caculator that evaluates arithmetic expressions and it is separated in three small files:

calc.h (common header file for lexer and parser)
lexer.l (obviously the lexer)
parser.y (the parser and the main function)

First, lets see the header:

    typedef struct      parse_parm_s
    {
        void            *yyscanner;
        char            *buf;
        int             pos;
        int             length;
        double          result;
    }                   parse_parm;

    void    parse(char *buf, double *result);

    #define YYSTYPE         double
    #define YY_EXTRA_TYPE   parse_parm *

    int     yylex(YYSTYPE *, void *);
    int     yylex_init(void **);
    int     yylex_destroy(void *);
    void    yyset_extra(YY_EXTRA_TYPE, void *);
    int     yyparse(parse_parm *, void *);
    void    yyerror();

Lets focus on the parse_parm structure. What we need is a way to share data between the lexer and the parser without the use of global variables. That is where this structure comes in. In this one we have:

yyscanner which is an uninitialized pointer that will be used by the lexer for internal suff (instead of globals)
buf is the buffer that contains the data to be parsed
pos is the current position in the buffer
length is the size of the buffer
result will be the result of the evaluated expression

parse is our main parsing function. We will see it in the parser.
YYSTYPE is the type of yylval but you should know that otherwise go back to basics.
YY_EXTRA_TYPE is the type of the extra data which will be given to the lexer thanks to the yyset_extra function. In this case it is our structure.

The prototypes are forward declaration of lexer/parser functions so the compiler shuts up about warnings. I think I could generate a .h with all that but I was lazy.

Here comes the lexer:

    %{
    #include <stdlib.h>
    #include <string.h>

    #include "calc.h"
    #include "parser.tab.h"

    #define PARM    yyget_extra(yyscanner)

    #define YY_INPUT(buffer, res, max_size)             \
    do {                                                \
        if (PARM->pos >= PARM->length)                  \
            res = YY_NULL;                              \
        else                                            \
        {                                               \
            res = PARM->length - PARM->pos;             \
            res > (int)max_size ? res = max_size : 0;   \
            memcpy(buffer, PARM->buf + PARM->pos, res); \
            PARM->pos += res;                           \
        }                                               \
    } while (0)

    %}

    %option reentrant bison-bridge
    %option noyywrap
    %option nounput

    %%

    [+\-*/()] { return (*yytext); }

    [0-9]+ { *yylval = atoi(yytext); return (INT); }

    ([0-9]*.[0-9]+) { *yylval = atof(yytext); return (FLOAT); }

    [ \t\r\n] ;

    %%

The parse_parm structure is extra data so we have to access it with yyget_extra(yyscanner).

Since we read from a buffer and not stdin or a file, we have to to redefine the YY_INPUT macro (see section 10 of the flex manual 'The generated scanner').

We want the scanner to be reentrant, therefore generate no global variables. That is what the reentrant option is for. bison-bridge is used to create a bison compatible scanner and share yylval.

Finally the parser:

    %{
    #include <stdio.h>
    #include <string.h>

    #include "calc.h"

    void    parse(char *buf, double *result)
    {
        parse_parm  pp;

        pp.buf = buf;
        pp.length = strlen(buf);
        pp.pos = 0;
        *result = 0;
        yylex_init(&pp.yyscanner);
        yyset_extra(&pp, pp.yyscanner);
        yyparse(&pp, pp.yyscanner);
        *result = pp.result;
        yylex_destroy(pp.yyscanner);
    }

    %}

    %pure_parser
    %parse-param {parse_parm *parm}
    %parse-param {void *scanner}
    %lex-param {yyscan_t *scanner}

    %token INT FLOAT

    %left '-' '+'
    %left '*' '/'
    %left NEG POS

    %%

    expr: math { parm->result = $1; }
        ;

    math: math '+' math { $$ = $1 + $3; }
        | math '-' math { $$ = $1 - $3; }
        | math '*' math { $$ = $1 * $3; }
        | math '/' math { $$ = $1 / $3; }
        | '-' math %prec NEG { $$ = -$2; }
        | '+' math %prec POS { $$ = $2; }
        | '(' math ')' { $$ = $2; }
        | INT { $$ = $1; }
        | FLOAT { $$ = $1; }
        ;

    %%

yylex_init and yylex_destroy have to be called to initialize the lexer and free its ressources after the parsing. yyset_extra is used to pass the parse_parm structure to the lexer.

The pure_parser option tells bison to use no global variables and create a reentrant parser.
yyparse gets two new parameters with the parse-param. The parse_parm structure and the pointer for the lexer.
yylex gets a new parameter thanks to lex-param so the parser can pass him the pointer.

Now the parse function can be called any time to evaluate an expression and get the result. Even in multiple threads.

Here is a link to the sources in this tutorial: calc.tar.gz. You can compile it with:

    $ bison -d parser.y
    $ flex lexer.l
    $ gcc lex.yy.c parser.tab.c -ly -ll

Patching Around

Seb — Fri, 08 Jun 2007 20:14:00 +0000

The new pam_usb release has a new feature called the pamusb agent. It is basically a python daemon that you can run in your user session. It allows you to execute all kinds of events when your key is plugged in or out.
What I wanted was a small notification to know when I’m authenticated. Unfortunately there is nothing as cool as GROWL on Linux (as far as I know) so I decided to add a small tray icon to the pamusb agent.

If you are interested you can find the patched version here:

svn://svn.sig11.org/pamusb_tray/

Currently it is just GTK because I already had some code lying around but I’ll add a QT version later.
This thing gave me some ideas like adding a notification bubble and a small graphical interface to add & remove events. Maybe later…

This also reminds me that I have a small patch for Trac lying around.
Sometimes you want to use Trac as a private project management system denying access to anyone except registered users.

This patch allows you to release some pages as public ones by adding “Public” to the page name (example: “TestPublic”).
You can find it here