PulseAudio

After reading some documentation and a lot of code. A somehow working module evolves. It supports simple module argument, it can connect to a remote server and also plays audio. But the problems starting here, because threads are nice and can hurt ... [More] you because of context. For the first version it is good to hear voices in my ears coming out of speakers over a network. I implemented an early version with a lot of downsides: no latency informations threading issues no zerocopy ... What I've done wrong was attaching libpulse on pulse server mainloop. Every context and stream callback was run by the mainloop as well as read operations. Not good thing. Only pa_sink_render is working within the I/O thread. Zero copy for now is broken by libpulse, because using pa_stream_write without a free_callback will copy the samples into an internal buffer. But for the first version ok. Fixing my threading problem is easy move everything inside our thread. What means create a mainloop for the libpulse. My first thought, that could not be hard, because epoll, a modern efficient select replacement supports chaining. Just chain or combine rt_poll with our libpulse mainloop. Not a easy solution. Combining a mainloop poll and a rt poll is hard, because of two lists of filedescriptors which can not be simple appended. Both lists get generated each time poll runs. Stepping a step back and asking why should I use rt_poll? Is there another solution which fits much more the concept of using a second mainloop? A thread only working as a mainloop should also work. The only task what the thread must take case is the pa_thread_mq. [Less]

After reading some documentation and a lot of code. A somehow working module evolves. It supports simple module argument, it can connect to a remote server and also plays audio. But the problems starting here, because threads are nice and can hurt ... [More] you because of context. For the first version it is good to hear voices in my ears coming out of speakers over a network. I implemented an early version with a lot of downsides: no latency informations threading issues no zerocopy ... What I've done wrong was attaching libpulse on pulse server mainloop. Every context and stream callback was run by the mainloop as well as read operations. Not good thing. Only pa_sink_render is working within the I/O thread. Zero copy for now is broken by libpulse, because using pa_stream_write without a free_callback will copy the samples into an internal buffer. But for the first version ok. Fixing my threading problem is easy move everything inside our thread. What means create a mainloop for the libpulse. My first thought, that could not be hard, because epoll, a modern efficient select replacement supports chaining. Just chain or combine rt_poll with our libpulse mainloop. Not a easy solution. Combining a mainloop poll and a rt poll is hard, because of two lists of filedescriptors which can not be simple appended. Both lists get generated each time poll runs. Stepping a step back and asking why should I use rt_poll? Is there another solution which fits much more the concept of using a second mainloop? A thread only working as a mainloop should also work. The only task what the thread must take case is the pa_thread_mq. [Less]

On my journey to a new module-tunnel I took a closer look into other modules. Pulseaudio got modules for event handling, got modules for os driver frameworks like alsa, oss, solaris. Also there a drivers to communicate over the local network. ... [More] Starting with mod-native-protocol-tcp to allow remote pulseaudio connections, there is also support for esound, zeroconf, rtp or raop. Modify sound stream samples is also possible, take a look into the filters. To write a hardware driver or network based driver I would recommend reading null-sink, raop-sink and esound-sink. After looking around into several pulse modules. It is clear that most of the module which act as a sink have the following structure and nearby the same code beside from driver specific parts. :::c void thread_func() { / hard work / } int pa__init(pa_module*m) { /* parse + validate + copy module arguments */ pa_modargs_new(m->argument, valid_modargs) /* init driver specific */ /* setup a real time poll */ u->rtpoll = pa_rtpoll_new(); /* setup a message queue */ pa_thread_mq_init(&u->thread_mq, m->core->mainloop, u->rtpoll pa_sink_set_asyncmsgq(u->sink, u->thread_mq.inq); /* create a thread */ pa_thread_new("null-sink", thread_func, u) /* register sink */ pa_sink_put(u->sink); } int pa__done(pa_module*m) { /* deinit private structure + free everything */ } All sink modules have a real time poll running in a seperated thread, often also caled I/O thread. This is because of [http://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/Developer/Threading/|the threading model]. In summary it says "don't block the mainloop and do the hard work within the I/O loop." Back to our thread, is has to do the real work of our module. It's dequeuing data from a sink buffer, mixing and adjusting volumes. This set of tasks is named rendering. It is done by pa_sink_render. Syncronisation is done by message queues between mainloop and I/O loop named pa_thread_mq which has 2 pa_asyncmsgq one input queue for message to the thread and one output queue from thread to the mainloop. rtpoll is not as powerfull a mainloop is, but you can register at least a filehandler packed within rt_poll_item. When you write a network protocol this will be your socket. [Less]

On my journey to a new module-tunnel I took a closer look into other modules. Pulseaudio got modules for event handling, got modules for os driver frameworks like alsa, oss, solaris. Also there a drivers to communicate over the local network. ... [More] Starting with mod-native-protocol-tcp to allow remote pulseaudio connections, there is also support for esound, zeroconf, rtp or raop. Modify sound stream samples is also possible, take a look into the filters. To write a hardware driver or network based driver I would recommend reading null-sink, raop-sink and esound-sink. After looking around into several pulse modules. It is clear that most of the module which act as a sink have the following structure and nearby the same code beside from driver specific parts. :::c void thread_func() { / hard work / } int pa__init(pa_module*m) { /* parse + validate + copy module arguments */ pa_modargs_new(m->argument, valid_modargs) /* init driver specific */ /* setup a real time poll */ u->rtpoll = pa_rtpoll_new(); /* setup a message queue */ pa_thread_mq_init(&u->thread_mq, m->core->mainloop, u->rtpoll pa_sink_set_asyncmsgq(u->sink, u->thread_mq.inq); /* create a thread */ pa_thread_new("null-sink", thread_func, u) /* register sink */ pa_sink_put(u->sink); } int pa__done(pa_module*m) { /* deinit private structure + free everything */ } All sink modules have a real time poll running in a seperated thread, often also caled I/O thread. This is because of [http://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/Developer/Threading/|the threading model]. In summary it says "don't block the mainloop and do the hard work within the I/O loop." Back to our thread, is has to do the real work of our module. It's dequeuing data from a sink buffer, mixing and adjusting volumes. This set of tasks is named rendering. It is done by pa_sink_render. Syncronisation is done by message queues between mainloop and I/O loop named pa_thread_mq which has 2 pa_asyncmsgq one input queue for message to the thread and one output queue from thread to the mainloop. rtpoll is not as powerfull a mainloop is, but you can register at least a filehandler packed within rt_poll_item. When you write a network protocol this will be your socket. [Less]

On my journey to a new module-tunnel I took a closer look into other modules. Pulseaudio got modules for event handling, got modules for os driver frameworks like alsa, oss, solaris. Also there a drivers to communicate over the local network. ... [More] Starting with mod-native-protocol-tcp to allow remote pulseaudio connections, there is also support for esound, zeroconf, rtp or raop. Modify sound stream samples is also possible, take a look into the filters. To write a hardware driver or network based driver I would recommend reading null-sink, raop-sink and esound-sink. After looking around into several pulse modules. It is clear that most of the module which act as a sink have the following structure and nearby the same code beside from driver specific parts. :::c void thread_func() { / hard work / } int pa__init(pa_module*m) { /* parse + validate + copy module arguments */ pa_modargs_new(m->argument, valid_modargs) /* init driver specific */ /* setup a real time poll */ u->rtpoll = pa_rtpoll_new(); /* setup a message queue */ pa_thread_mq_init(&u->thread_mq, m->core->mainloop, u->rtpoll pa_sink_set_asyncmsgq(u->sink, u->thread_mq.inq); /* create a thread */ pa_thread_new("null-sink", thread_func, u) /* register sink */ pa_sink_put(u->sink); } int pa__done(pa_module*m) { /* deinit private structure + free everything */ } All sink modules have a real time poll running in a seperated thread, often also caled I/O thread. This is because of [http://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/Developer/Threading/|the threading model]. In summary it says "don't block the mainloop and do the hard work within the I/O loop." Back to our thread, is has to do the real work of our module. It's dequeuing data from a sink buffer, mixing and adjusting volumes. This set of tasks is named rendering. It is done by pa_sink_render. Syncronisation is done by message queues between mainloop and I/O loop named pa_thread_mq which has 2 pa_asyncmsgq one input queue for message to the thread and one output queue from thread to the mainloop. rtpoll is not as powerfull a mainloop is, but you can register at least a filehandler packed within rt_poll_item. When you write a network protocol this will be your socket. [Less]

After one month here comes my first blog post about it. My gsoc project is creating a new mod-tunnel using libpulse api. You can use module-tunnel to connect your local pulseaudio to a remote pulseaudio. This is quite usefull, because you can stream ... [More] over the network your audio from laptop to a dedicated sound system with real speakers. With pavucontrol you can move simple move a stream from local output device to the remote one. In combination with mod-zeroconf-discover and mod-zeroconf-publish it will automatic find other pulseaudio servers and connects to them. What module-tunnel really does is creating a virtual sound card on your local pulseaudio server. On the remote site it creates a client as every other normal sound application like mplayer, vlc, [...] Why I'm rewriting a complete module?' Before saying anything about it, mod-tunnel seems to be a very old module from the beginning. But it has a lot of downside beside the good one that it works as long you have a stable connection to the remote server. It can not recover from any kind of errors. mod-tunnel doing all the client protocol stack on its own instead of using the well-tested and well-written libpulse with callbacks and state machines. adding new features is difficult because of it's simple structure [Less]

After one month here comes my first blog post about it. My gsoc project is creating a new mod-tunnel using libpulse api. You can use module-tunnel to connect your local pulseaudio to a remote pulseaudio. This is quite usefull, because you can stream ... [More] over the network your audio from laptop to a dedicated sound system with real speakers. With pavucontrol you can move simple move a stream from local output device to the remote one. In combination with mod-zeroconf-discover and mod-zeroconf-publish it will automatic find other pulseaudio servers and connects to them. What module-tunnel really does is creating a virtual sound card on your local pulseaudio server. On the remote site it creates a client as every other normal sound application like mplayer, vlc, [...] Why I'm rewriting a complete module?' Before saying anything about it, mod-tunnel seems to be a very old module from the beginning. But it has a lot of downside beside the good one that it works as long you have a stable connection to the remote server. It can not recover from any kind of errors. mod-tunnel doing all the client protocol stack on its own instead of using the well-tested and well-written libpulse with callbacks and state machines. adding new features is difficult because of it's simple structure [Less]

Hi. Time for a quick update about my Summer of Code project. I'm going to talk a little about my first checklist item, refactoring the resampling code. While refactoring isn't quite exciting as implementing new features it is a necessity to assure ... [More] code maintainability. So let's take a look at the resampler interface and see what we can do. The interface consists of a pa_resampler structure which we create if we want to do some resampling. This structure holds all of our settings the resampler cares about, (sample specifications, different buffers) but also some specific data for different resampling implementations. The interesting bits are shown bellow. struct pa_resampler {... void (*impl_free)(pa_resampler *r); void (*impl_update_rates)(pa_resampler *r); void (*impl_resample)(pa_resampler *r, const pa_memchunk *in, unsigned in_samples, pa_memchunk *out, unsigned *out_samples); void (*impl_reset)(pa_resampler *r); struct { /* data specific to the trivial resampler */ unsigned o_counter; unsigned i_counter; } trivial; struct { /* data specific to the peak finder pseudo resampler */ unsigned o_counter; unsigned i_counter; float max_f[PA_CHANNELS_MAX]; int16_t max_i[PA_CHANNELS_MAX]; } peaks;...};After the implementation specific function pointers we can see multiple structures holding implementation specific data. Since the resampler can't switch implementations on the fly (without destroying and recreating a resampler) only one of these structures is used at a time. There are six of those structures contained inside of pa_resampler and some of them have only a single member which is quite pointless. Further bellow of the file we see a big init_table containing the mapping between a resampling method (an enumeration) and its initialization function. static int (* const init_table[])(pa_resampler*r) = { [PA_RESAMPLER_SRC_SINC_BEST_QUALITY] = libsamplerate_init, [PA_RESAMPLER_SRC_SINC_MEDIUM_QUALITY] = libsamplerate_init, [PA_RESAMPLER_SRC_SINC_FASTEST] = libsamplerate_init, [PA_RESAMPLER_SRC_ZERO_ORDER_HOLD] = libsamplerate_init, [PA_RESAMPLER_SRC_LINEAR] = libsamplerate_init, [PA_RESAMPLER_TRIVIAL] = trivial_init, [PA_RESAMPLER_SPEEX_FLOAT_BASE+0] = speex_init, [PA_RESAMPLER_SPEEX_FLOAT_BASE+1] = speex_init, ... [PA_RESAMPLER_SPEEX_FLOAT_BASE+10] = speex_init, [PA_RESAMPLER_SPEEX_FIXED_BASE+0] = speex_init, ... [PA_RESAMPLER_SPEEX_FIXED_BASE+10] = speex_init, ...As we can see there are quite some duplicates here. There are a total of 32 entries while only having 6 distinctive init functions. There is another big table like this containing the implementation names, this table doesn't contain any duplicates but it would be nice if we could group the implementation names into smaller implementation specific tables. So without further ado here is the first code snippet with the appropriate changes. struct pa_resampler { ... pa_resampler_implementation implementation; ...};All the implementation specific data is now contained inside a single structure. And here is how the equivalent second code snippet looks like. static pa_resampler_implementation *impl_table[] = { [PA_RESAMPLER_SRC_SINC_FASTEST] = &libsamplerate_impl, [PA_RESAMPLER_TRIVIAL] = &trivial_impl, [PA_RESAMPLER_SPEEX_FLOAT_BASE] = &speex_impl, [PA_RESAMPLER_FFMPEG] = &ffmpeg_impl, [PA_RESAMPLER_AUTO] = &auto_impl, [PA_RESAMPLER_COPY] = ©_impl, [PA_RESAMPLER_PEAKS] = &peaks_impl,};No more duplicate entries here. And at last here is how the pa_resampler_implementation structure is defined. struct pa_resampler_implementation { int (*init)(pa_resampler *r); void (*free)(pa_resampler *r); void (*update_rates)(pa_resampler *r); void (*resample)(pa_resampler *r, const pa_memchunk *in, unsigned in_samples, pa_memchunk *out, unsigned *out_samples); void (*reset)(pa_resampler *r); void *data; const char *names[PA_RESAMPLER_MAX_VARIANTS];};The implementation specific structures are replaced by a simple opaque data entry and the implementations init function takes care of the allocating. The big names table is also now split up and contained inside this structure. These changes aren't yet merged upstream and some of it may change if needed. Further details are on my github page. Thats it for now, next time there should be a more interesting topic. Thanks for your attention and bye. [Less]

Hi. Time for a quick update about my Summer of Code project. I'm going to talk a little about my first checklist item, refactoring the resampling code. While refactoring isn't quite exciting as implementing new features it is a necessity to assure ... [More] code maintainability. So let's take a look at the resampler interface and see what we can do. The interface consists of a pa_resampler structure which we create if we want to do some resampling. This structure holds all of our settings the resampler cares about, (sample specifications, different buffers) but also some specific data for different resampling implementations. The interesting bits are shown bellow. struct pa_resampler {... void (*impl_free)(pa_resampler *r); void (*impl_update_rates)(pa_resampler *r); void (*impl_resample)(pa_resampler *r, const pa_memchunk *in, unsigned in_samples, pa_memchunk *out, unsigned *out_samples); void (*impl_reset)(pa_resampler *r); struct { /* data specific to the trivial resampler */ unsigned o_counter; unsigned i_counter; } trivial; struct { /* data specific to the peak finder pseudo resampler */ unsigned o_counter; unsigned i_counter; float max_f[PA_CHANNELS_MAX]; int16_t max_i[PA_CHANNELS_MAX]; } peaks;...};After the implementation specific function pointers we can see multiple structures holding implementation specific data. Since the resampler can't switch implementations on the fly (without destroying and recreating a resampler) only one of these structures is used at a time. There are six of those structures contained inside of pa_resampler and some of them have only a single member which is quite pointless. Further bellow of the file we see a big init_table containing the mapping between a resampling method (an enumeration) and its initialization function. static int (* const init_table[])(pa_resampler*r) = { [PA_RESAMPLER_SRC_SINC_BEST_QUALITY] = libsamplerate_init, [PA_RESAMPLER_SRC_SINC_MEDIUM_QUALITY] = libsamplerate_init, [PA_RESAMPLER_SRC_SINC_FASTEST] = libsamplerate_init, [PA_RESAMPLER_SRC_ZERO_ORDER_HOLD] = libsamplerate_init, [PA_RESAMPLER_SRC_LINEAR] = libsamplerate_init, [PA_RESAMPLER_TRIVIAL] = trivial_init, [PA_RESAMPLER_SPEEX_FLOAT_BASE+0] = speex_init, [PA_RESAMPLER_SPEEX_FLOAT_BASE+1] = speex_init, ... [PA_RESAMPLER_SPEEX_FLOAT_BASE+10] = speex_init, [PA_RESAMPLER_SPEEX_FIXED_BASE+0] = speex_init, ... [PA_RESAMPLER_SPEEX_FIXED_BASE+10] = speex_init, ...As we can see there are quite some duplicates here. There are a total of 32 entries while only having 6 distinctive init functions. There is another big table like this containing the implementation names, this table doesn't contain any duplicates but it would be nice if we could group the implementation names into smaller implementation specific tables. So without further ado here is the first code snippet with the appropriate changes. struct pa_resampler { ... pa_resampler_implementation implementation; ...};All the implementation specific data is now contained inside a single structure. And here is how the equivalent second code snippet looks like. static pa_resampler_implementation *impl_table[] = { [PA_RESAMPLER_SRC_SINC_FASTEST] = &libsamplerate_impl, [PA_RESAMPLER_TRIVIAL] = &trivial_impl, [PA_RESAMPLER_SPEEX_FLOAT_BASE] = &speex_impl, [PA_RESAMPLER_FFMPEG] = &ffmpeg_impl, [PA_RESAMPLER_AUTO] = &auto_impl, [PA_RESAMPLER_COPY] = ©_impl, [PA_RESAMPLER_PEAKS] = &peaks_impl,};No more duplicate entries here. And at last here is how the pa_resampler_implementation structure is defined. struct pa_resampler_implementation { int (*init)(pa_resampler *r); void (*free)(pa_resampler *r); void (*update_rates)(pa_resampler *r); void (*resample)(pa_resampler *r, const pa_memchunk *in, unsigned in_samples, pa_memchunk *out, unsigned *out_samples); void (*reset)(pa_resampler *r); void *data; const char *names[PA_RESAMPLER_MAX_VARIANTS];};The implementation specific structures are replaced by a simple opaque data entry and the implementations init function takes care of the allocating. The big names table is also now split up and contained inside this structure. These changes aren't yet merged upstream and some of it may change if needed. Further details are on my github page. Thats it for now, next time there should be a more interesting topic. Thanks for your attention and bye. [Less]

And we’re back … PulseAudio 4.0 is out! There’s both a short and super-detailed changelog in the release notes. For the lazy, this release brings a bunch of Bluetooth stability updates, better low latency handling, performance improvements, and a ... [More] whole lot more. :) One interesting thing is that for this release, we kept a parallel next branch open while master was frozen for stabilising and releasing. As a result, we’re already well on our way to 5.0 with 52 commits since 4.0 already merged into master. And finally, I’m excited to announce PulseAudio is going to be carrying out two great projects this summer, as part of the Google Summer of Code! We are going to have Alexander Couzens (lynxis) working on a rewrite of module-tunnel using libpulse, mentored by Tanu Kaskinen. In addition to this, Damir Jelić (poljar) working on improvements to resampling, mentored by Peter Meerwald. That’s just some of the things to look forward to in coming months. I’ve got a few more things I’d like to write about, but I’ll save that for another post. [Less]