My research is in machine learning, and some of it is in the area of speech recognition. I do have experience with CMU Sphinx 4 (in particular with HMM's which I believe is what you are going to need), but not so much with Asterisk. I have also been doing system-level programming in linux for about 11 years so I should be able to figure out what we need for Asterisk and how it hooks into the voice recognition engine.
To be realistic, for this project to be truly effective, it would need to go beyond statistical learning and do some linguistic processing, but this would go well beyond your limits (perhaps in the future). Let me know if you are interested.