system toolbox » voip

https://tango.0pointer.de/pipermail/pulseaudio-discuss/2008-February/001275.html

I'm trying to use the Pulseaudio server on a machine which runs the 
Asterisk PBX application.  Asterisk can attach to ALSA using a console 
driver.  Basically, you call a phone extension and connect to an alsa 
driven sound card.

When I directly connect Asterisk to the sound card, everything works 
fine.  However, when I attempt to connect to the sound card through 
Pulseaudio, the Asterisk application will crash after making the second 
call connection.

The only evidence of something I currently see going wrong is that the 
alsa interface for Asterisk receives a -5 (-EIO) on a write to the alsa 
interface.  I don't know if that is the eventual cause of the crash.

I have the asterisk source code, and I understand where the alsa API 
interface is.  It looks very straight forward, much like any other alsa 
interface I've seen.  I'm certainly comfortable with modifying the code.

When attaching to the sound card through pulseaudio, I use these 
.asoundrc entries:

pcm.card0_record {
     type pulse
}

ctl.card0_record {
     type pulse
     sink alsa_output.hw_0
}

pcm.card0_playback {
     type pulse
}

ctl.card0_playback {
     type pulse
     source alsa_input.hw_0
}

my pulseaudio default.pa has these entries:

load-module module-detect
load-module module-native-protocol-unix auth-anonymous=1
load-module module-native-protocol-tcp auth-anonymous=1
load-module module-http-protocol-tcp
set-default-sink alsa_output.hw_0 
set-default-source alsa_input.hw_0

I assume what is happening is that the Asterisk alsa interface needs to 
be adjusted to be more pulseaudio friendly.

How can I get started on figuring out what needs to be changed?

I read all the information on the wiki about writing clients and 
modules, but I'm not sure it applies to what I need to do.

Can anyone point me in the right direction?

Eventually, I'm hoping to be able to use Pulseaudio to be able to tunnel 
the audio from the Asterisk alsa console interface to a voice 
recognition application on either the same machine or another machine by 
routing the audio to a null-sink and attaching the listening application 
to the .monitor of that null-sink.

I've actually have this working, however, it's usage is not reliable 
since the Asterisk application keeps core dumping every other call.  I'm 
just trying to start simple by simply attempting to attach to the 
soundcard before I finalize the usage of the null-sink.

Thanks,

Jim

http://www.nabble.com/Misterhouse-Asterisk-%2B-Pocketsphinx-HOWTO-td16800602.html

Misterhouse Asterisk + Pocketsphinx HOWTO¶

by Jim Duda Apr 21, 2008; 02:06am

The purpose of this posting is to describe my method for integrating pocketsphinx and asterisk together.

Goal¶

Dial an extension on your phone and issue voice commands to misterhouse.

Note, this HOWTO is not for the faint of heart ...

Requirements¶

- Linux only (I think)
- Pulseaudio
- Pocketsphinx
- Asterisk
- Misterhouse

You need to be comfortable with

- Asterisk, extensions, AGI interface
- Pulseaudio, null-sinks, default.pa
- Alsa, .asoundrc configuration
- Pocketsphinx
- Misterhouse (of course)

PULSEAUDIO configuration¶

We need to use a running pulseaudio server such that we can connect the asterisk application with the pocketsphinx application. Both asterisk
and pocketsphinx are designed to interface directly with the hardware layer and not designed to interface with another application.

The pulseaudio server provides a "null_sink" module, which allows an interim software mechanism to allow the asterisk and pocketsphinx to
interchange audio.
I tried once to get JACK to perform this feat, but I was unsuccessful. Pulseaudio works fine.

I suggest you get a little comfortable with the pulseaudio server first.

Step 1: Setup default.pa¶

You need these 3 lines in your /etc/pulse/default.pa file on the machine
which hosts the misterhouse application:

load-module module-null-sink sink_name=sphinx_record description="Sphinx
Recording Source" 

load-module module-null-sink sink_name=sphinx_playback
description="Sphinx Playback Source" 

load-module module-combine sink_name=combined master=<your main audio
card>,sphinx_playback

set-default-sink combined

The sphinx_record null sink is used to receive the inbound audio from asterisk, which pocketsphinx will listen to.

The sphinx_playback null sink is used by misterhouse to speak back to, which becomes the audio playback to asterisk.

The combined sink is required such that the misterhouse audio will go to both the local speakers and asterisk.

Your audio setup may vary. You might need to change specifics for your system.

You need to setup the .asoundrc for the user which runs the misterhouse
application.

.asoundrc for misterhouse user:

pcm.!default {
   type pulse
}
ctl.!default {
   type pulse
}

pcm.microphone {
     type pulse;
     device <your microphone hardware>;
}

ctl.microphone {
     type pulse;
     device <your microphone hardware>;
     source <your microphone hardware>;
}

pcm.sphinx_microphone {
     type pulse;
     device sphinx_record.monitor;
}

ctl.sphinx_microphone {
     type pulse;
     device sphinx_record.monitor;
     source sphinx_record.monitor;
}

You need to setup the .asoundrc for the user which runs the asterisk application. You need to change the "server" to the machine which runs
the misterhouse application. In my case, that machine name is "linux".

If misterhouse and asterisk run on the same host, then you can remove the server line.

.asoundrc for asterisk user

pcm.sphinx_record {
     type pulse
     device sphinx_record
     server linux
}

ctl.sphinx_record {
     type pulse
     device sphinx_record
     sink   sphinx_record
     server linux
}

pcm.sphinx_playback {
     type pulse
     device sphinx_playback.monitor
     server linux
}

ctl.sphinx_playback {
     type pulse
     device sphinx_playback.monitor
     source sphinx_playback.monitor
     server linux
}

Pocketsphinx configuration¶

You need to install pocketsphinx, see the instructions in mh/lib/PocketSphinx.pm and/or code/common/pocketsphinx_listener.pl

Enable the pocketsphinx_listener.pl code file. You should get pocketsphinx working with a microphone before attempting to integrate
with asterisk.

Asterisk configuration¶

You need the following two lines in /etc/asterisk/alsa.conf

input_device=sphinx_playback
output_device=sphinx_record

You need these lines in your /etc/asterisk/extensions.conf

[sphinx]
exten => s,1,AGI(MisterHouse.agi,"Sphinx Connect")
exten => s,n,Dial(CONSOLE/1)

exten => h,1,AGI(MisterHouse.agi,"Sphinx Disconnect")
exten => h,n,Hangup

; - Console Attachment
exten => 503,1,Goto(sphinx,s,1)

In my case, dialing extension 503 in the local context will result in a connection between the phone and the pocketsphinx application running on
the misterhouse server. You need to tailor for what you want.

You need some code in your /var/lib/asterisk/agi-bin/Misterhouse.agi file:

I have attached a copy of my version for reference.

The Misterhouse.agi code is needed such that the "Sphinx Connect" and "Sphinx Disconnect" commands are passed to misterhouse when a user dials
the extensions or hangs up the phone. This triggers the start/stop of a pocketsphinx listener.

Misterhouse configuration¶

You're going to need some code module, example, asterisk.pl, which allows communication between Asterisk and MH. I'm assuming you've
already done something similar, since you have asterisk. When asterisk detects the Sphinx extension was dialed, it sends a command to MH which
instructs MH to start a new pocketsphinx listener thread.

I have attached my version, as reference.

Summary¶

1. A user dials some extension (example 503) on Asterisk.
2. Asterisk connects the call to the alsa console interface.
3. Asterisk sends a command to MH through the AGI interface to inform MH that a user has dialed the Sphinx connection.
4. MH receives the AGI command and starts a new pocketsphinx listener thread.

The asterisk console and pocketsphinx applications are glued together by virtue of the pulseaudio server through the sphinx_record and
sphinx_playback null_sink modules, in conjunction with information in the .asoundrc files for both the asterisk user and MH user.

MH speaks "voice activation is now available and ready" through the asterisk alsa console to the user on the phone. The user speaks MH voice commands into the headset, which get forwarded to the pocketshinx application which is now listening. MH responds to voice commands.

The user hangs up. Asterisk informs MH of the hangup. MH stops the pocketsphinx listener application.

Not the greatest HOW TO, but I hope you get the idea.

Jim

#!/usr/bin/perl -w
#
# MisterHouse.agi by Robert Mann
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met:
#
#     * Redistributions of source code must retain the above copyright
#       notice, this list of conditions and the following disclaimer.
#     * Redistributions in binary form must reproduce the above copyright
#       notice, this list of conditions and the following disclaimer in the
#       documentation and/or other materials provided with the distribution.
#     * Neither the name of the <ORGANIZATION> nor the names of its
#       contributors may be used to endorse or promote products derived from
#       this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
# IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# Parts of this code is from: Eric Wieling <eric@...>
#
use Asterisk::AGI;
use IO::Socket;
use File::Basename;
use File::Temp qw/ tempfile /;

# Set this to 1 to show more on the asterisk console regarding what is happening.
$verbose = 1;

# Your misterhouse server information.  You need to use the same information that is in your mh.ini file.
$MH_ADDRESS = 'linux';
$MH_PORT = '8090';
$username = 'myuser';
$password = 'mypass';

# Currently set to cepstral but if you wish to use Festival you may comment the next line and uncomment the line below.
# $ttsengine = '/opt/theta/bin/theta';
$ttsengine = '/usr/bin/text2wave';

# This section will parse throught the AGI variables passed to the program from Asterisk
$AGI = new Asterisk::AGI;
my %input = $AGI->ReadParse();
$AGI->setcallback(\&mycallback);

# If you have verbose set to 1 then you will get a AGI enviroment dump to your Asterisk console
if ($verbose){
        verbose("AGI Environment Dump:");
        foreach $i (sort keys %input) {
                verbose(" -- $i = $input{$i}");
        }
}

$mydir = dirname($0);
$ttstype = basename($ttsengine);
$ttscommand = $ttsengine;

# Try to create a TCP socket to Misterhouse to establish communications with MisterHouse.
$remote = IO::Socket::INET->new(Proto => "tcp", PeerAddr => "$MH_ADDRESS", PeerPort => "$MH_PORT",)
        or die "cannot connect to misterhouse on port 8090 at 192.168.1.250";

# Do a simple login to Misterhouse.  Not really needed if you are behind a firewall but just in case you are not
# it is included.  Keep in mind this is all plain text so if you are worried about that you should be aware.
print $remote "Login: $username\n";
print $remote "Secret: $password\n";

# Find out what we are trying to do here.  If we are sending simple callerid info,
# Sending DTMF to track outbound calls or just sending a external command to Misterhouse.
# More can be added if we choose this method but for now these are the commands currently
# supported in this version.

if ($ARGV[0] =~ /CallerID/){
        verbose("here $ARGV[0]");
        verbose( "CallerID: $input{callerid} Line: $input{channel}\n");
        print $remote "CallerID: $input{callerid} Line: $input{channel}\n";
} elsif ($ARGV[0] =~ /DTMF/){
        $ARGV[0] =~ /DTMF: (.*)/;
        my ($extension) = $1 || $input{extension};
        $extension =~ s/\D+//g;
        verbose("DTMF: $extension CallFrom: $input{channel}");
        print $remote "DTMF: $extension CallFrom: $input{channel}\n";
} elsif ($ARGV[0] =~ /Command/){
        print $remote "$ARGV[0]\n";
} elsif ($ARGV[0] =~ /Sphinx/){
        print $remote "$ARGV[0]: extension: $input{callerid}\n";
}

# Wait for any communications back from MisterHouse if we are expecting any.  If not MisterHouse will
# close our connection for us and we will just continue on past it.
#while(<$remote>) {
# chomp;
# &speak_text("$_");
#}

# If we received any messages back from MisterHouse they will be listed here.  We should be able
# to accept any length text back as it is saved to a wav file and played back to the channel.
sub speak_text {
        my ($text) = @_;
        verbose("TTS type is: $ttstype");
        verbose("TEXT OUTPUT: $text");
        (undef, $filename) = tempfile( "/tmp/misterhouseXXXXX", SUFFIX => ".wav");
        if ($ttstype eq "theta") {
                $ENV{THETA_HOME} = dirname(dirname($ttscommand));
                system("$ttscommand -F 8000 -o $filename \"$text\"");
        }
        if ($ttstype eq "text2wave") {
                system("echo \"$text\" | $ttscommand -scale 1.5 -F 8000 -o $filename -");
        }
        $streamfile = substr($filename, 0, -4);
        $AGI->stream_file($streamfile);
        unlink($filename);
}

# Hehe.  Not really sure why this code is even here to be honest.  Have not researched it enough
# to know for sure.  It may be some sort of code needed by AGI itself.
sub mycallback {
        my ($returncode) = @_;
        print STDERR "MYCALLBACK: User Hungup ($returncode)\n";
        exit($returncode);
}

# This code simply sends verbose messages to the console of Asterisk.
sub verbose {
        if (defined($verbose)) {
                $AGI->verbose("@_", $verbose);
                return(0);
        }
}

asterisk console:

#    -- Accepting AUTHENTICATED call from 208.139.204.228, requested format = 4, actual format = 4
#    -- Executing AGI("IAX2/teliax@teliax/5", "MisterHouse.agi|"CallerID"") in new stack
#    -- Launched AGI Script /var/lib/asterisk/agi-bin/MisterHouse.agi
#  MisterHouse.agi|"CallerID": AGI Environment Dump:
#  MisterHouse.agi|"CallerID":  -- accountcode =
#  MisterHouse.agi|"CallerID":  -- callerid = +15406050674
#  MisterHouse.agi|"CallerID":  -- channel = IAX2/teliax@teliax/5
#  MisterHouse.agi|"CallerID":  -- context = home
#  MisterHouse.agi|"CallerID":  -- dnid = unknown
#  MisterHouse.agi|"CallerID":  -- enhanced = 0.0
#  MisterHouse.agi|"CallerID":  -- extension = 7818104624
#  MisterHouse.agi|"CallerID":  -- language = en
#  MisterHouse.agi|"CallerID":  -- priority = 1
#  MisterHouse.agi|"CallerID":  -- rdnis = unknown
#  MisterHouse.agi|"CallerID":  -- request = MisterHouse.agi
#  MisterHouse.agi|"CallerID":  -- type = IAX2
#  MisterHouse.agi|"CallerID":  -- uniqueid = 1120332689.141
#  MisterHouse.agi|"CallerID": here CallerID
#  MisterHouse.agi|"CallerID": CallerID: +15406050674 Line: IAX2/teliax@teliax/5
#    -- AGI Script MisterHouse.agi completed, returning 0
#    -- Executing Dial("IAX2/teliax@teliax/5", "SIP/100&SIP/101&SIP/102|20|tr") in new stack
#    -- Called 100
#    -- Called 101
#    -- Called 102
#    -- SIP/100-a651 is ringing