Opened 9 years ago

Closed 8 years ago

#88 closed enhancement (fixed)

sound support: use gstreamer+pulseaudio to provide efficient sound in/out over network

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 0.12
Component: Global Keywords:
Cc:

Description (last modified by Antoine Martin)

What we have there kinda works, but there are many issues that need improving/resolving:

  • bandwidth: pulseaudio's TCP protocol is a bandwidth pig... We must ensure we only use it over fast links (need better detection?)

Edit: see original description for the approach that was abandoned (pure pulseaudio)

Change History (11)

comment:1 Changed 9 years ago by Antoine Martin

Milestone: 0.9.20.9.3
Owner: changed from Antoine Martin to Antoine Martin
Status: newaccepted

comment:2 Changed 9 years ago by Antoine Martin

Summary: sound supportsound support: use pactl to remove hacks

comment:3 Changed 9 years ago by Antoine Martin

Milestone: 0.9.30.9.4

comment:4 Changed 9 years ago by Antoine Martin

Milestone: 0.9.41.0

comment:5 Changed 9 years ago by Antoine Martin

Description: modified (diff)
Summary: sound support: use pactl to remove hackssound support: use gstreamer+pulseaudio to provide efficient sound in/out over network

Went a completely different route: pulseaudio simply could not be made to not suck up all the bandwidth, so I implemented the sound pipes using gstreamer's rtp module:

  • initial network stuff: messages to request start/stop sound (both for server and client): r2561
  • more network stuff: some tricky code to get right to set the ports to use (set by server if tunneled, by client if not!) and remove pulseaudio from xpra via a patch + command line switch (would conflict with what we do), also remove pulseaudio from win32 build: we no longer need it. r2556
  • Use an external process as it appears that gstreamer+twisted+pygtk does not seem to play nice (and in any case, we need the ability to daemonize the process on the server): r2587 and r2585
  • Allow users to select the sound devices to use for input and output: r2645
  • Win32 needs its own .EXEs (and its own build file and DLLs, and directory...) for the gstreamer RTP code: r2653
Last edited 8 years ago by Antoine Martin (previous) (diff)

comment:6 Changed 9 years ago by Antoine Martin

Milestone: 1.00.11

This kinda works as it is, but with some limitations:

  • No support on OSX
  • Sound does not get disconnected when we disconnect from a session
  • No selection of sound quality (same over slow links as on same box...)

comment:7 Changed 9 years ago by Antoine Martin

pygst is now built for OSX but does not work on PowerPC (see #116)
This needs much more testing

comment:8 Changed 9 years ago by Antoine Martin

Component: ClientGlobal
Milestone: 0.110.12

comment:9 Changed 8 years ago by Antoine Martin

Sound now works on OSX PowerPC, we just need to hide the process with another plist file...

About sound support:
As of r3708 we now have a pure TCP gstreamer sound support which is much cleaner/robust than the previous code with caps files, rtsp and all the "reserved ports" madness.
(just not on windows yet.. see ossbuild ticket 116)

This is probably enough as it is for v0.12, but there are a number of rough edges:

  • we should detect when the server sound daemon exits (gio + probe?) and tell the client
  • pulseaudio doesn't support multiple instances any more? (see my question here)
  • add ability to change the sound options per session: which codec (currently hard-coded to use "vorbis"), bitrate, stereo? etc
  • rename "tunnel_sink" and "tunnel_source" to something more meaningful and ensure the right ones are used (not sure about that!)
  • when we disconnect a client because a new connection replaces it, we should ensure the existing sound pipelines are either dropped cleanly or kept/re-used.

Less important:

  • when the connection is local (and on *nix with local pulseaudio), we could bypass gstreamer altogether and just link up the session's pulseaudio with the real one.
  • advertise the pulseaudio server in the sessions with pax11publish?
  • try to keep the pulse native socket in our session directory rather than in /tmp
  • deal with volume control (probably not - let pavucontrol do that)

comment:10 Changed 8 years ago by Antoine Martin

  • multiple pulseaudio instances workaround in r3710
  • OSX support using a nested background application in r3720

comment:11 Changed 8 years ago by Antoine Martin

Resolution: fixed
Status: acceptedclosed

This will do for now - closing.

MS Windows support moved to #159 as this is purely a porting + building issue for that platform.

Note: See TracTickets for help on using tickets.