sound support: use gstreamer+pulseaudio to provide efficient sound in/out over network

[Antoine Martin]

What we have there kinda works, but there are many issues that need improving/resolving:

• bandwidth: pulseaudio's TCP protocol is a bandwidth pig... We must ensure we only use it over fast links (need better detection?)

Edit: see original description for the approach that was abandoned (pure pulseaudio)

[Antoine Martin]

Went a completely different route: pulseaudio simply could not be made to not suck up all the bandwidth, so I implemented the sound pipes using ​gstreamer's rtp module:

• initial network stuff: messages to request start/stop sound (both for server and client): r2561
• more network stuff: some tricky code to get right to set the ports to use (set by server if tunneled, by client if not!) and remove pulseaudio from xpra via a patch + command line switch (would conflict with what we do), also remove pulseaudio from win32 build: we no longer need it. r2556
• Use an external process as it appears that gstreamer+twisted+pygtk does not seem to play nice (and in any case, we need the ability to daemonize the process on the server): r2587 and r2585
• Allow users to select the sound devices to use for input and output: r2645
• Win32 needs its own .EXEs (and its own build file and DLLs, and directory...) for the gstreamer RTP code: r2653

[Antoine Martin]

This kinda works as it is, but with some limitations:

• No support on OSX
• Sound does not get disconnected when we disconnect from a session
• No selection of sound quality (same over slow links as on same box...)

[Antoine Martin]

pygst is now built for OSX but does not work on PowerPC (see #116)
This needs much more testing

[Antoine Martin]

Sound now works on OSX PowerPC, we just need to hide the process with another plist file...

About sound support:
As of r3708 we now have a pure TCP gstreamer sound support which is much cleaner/robust than the previous code with caps files, rtsp and all the "reserved ports" madness.
(just not on windows yet.. see ​ossbuild ticket 116)

This is probably enough as it is for v0.12, but there are a number of rough edges:

• we should detect when the server sound daemon exits (gio + probe?) and tell the client
• pulseaudio doesn't support multiple instances any more? (see my question ​here)
• add ability to change the sound options per session: which codec (currently hard-coded to use "vorbis"), bitrate, stereo? etc
• rename "tunnel_sink" and "tunnel_source" to something more meaningful and ensure the right ones are used (not sure about that!)
• when we disconnect a client because a new connection replaces it, we should ensure the existing sound pipelines are either dropped cleanly or kept/re-used.

Less important:

• when the connection is local (and on *nix with local pulseaudio), we could bypass gstreamer altogether and just link up the session's pulseaudio with the real one.
• advertise the pulseaudio server in the sessions with pax11publish?
• try to keep the pulse native socket in our session directory rather than in /tmp
• deal with volume control (probably not - let pavucontrol do that)

[Antoine Martin]

• multiple pulseaudio instances workaround in r3710
• OSX support using a nested background application in r3720

[Antoine Martin]

This will do for now - closing.

MS Windows support moved to #159 as this is purely a porting + building issue for that platform.