Jump to content

Jon Heese

Members
 View Profile  See their activity

  • Posts

    25

  • Joined

  • Last visited

 Content Type 

Posts posted by Jon Heese

  1. "Block oubound caller ID" not working with Sangoma/Netborder PRI

    in Trunk Setup

    Posted

    Well, well...

     

    It seems that you first have to dial a star code on the FXO line and then the number. This is almost two-stage dialling. IMHO the best solution would be if the gateway somehow can deal with the blocking. In theory, the SIP trunk should not know about the details of the termination.

     

    I am not a Sangoma expert; maybe it is worth studying the documentation there to see if there is a way to block the caller-ID.

    There are no FXO ports involved here, and as far as I know, star codes do not work on a PRI (and my testing bears this out).

     

    However, I will take your advice and contact Sangoma and see if there is any more documentation on exactly how I should go about disabling caller-id.

     

    Keep in mind that we're not talking about globally disabling outbound caller ID, but rather selectively disabling it for a specific call, or as a workaround, for a specific extension.

     

    Regards,

    Jon Heese

  3. "Block oubound caller ID" not working with Sangoma/Netborder PRI

    in Trunk Setup

    Posted

    What mode did you use on the trunk to indicate the Caller-ID? Maybe you can try "no indication" on the field "Remote Party/Privacy Indication".

    I don't fully understand your first question (or are you talking about "Remote Party/Privacy Indication"?), and I've tried all the possible values in the "Remote Party/Privacy Indication" field, including "No Indication", all with the same results.

     

    That's what I meant when I said this:

    When I enable that flag and watch the SIP packets, I see that there is an additional "Privacy: id" header added (if I'm using P-Asserted-Identity or P-Preferred-Identity) and an additional "privacy=full" added on to the end of the "Remote-Party-ID" field, if I enable that (I know, I know, it's a legacy option that shouldn't be used).

    Regards,

    Jon Heese

  4. "Block oubound caller ID" not working with Sangoma/Netborder PRI

    in Trunk Setup

    Posted

    Running pbxnsip v3.4.0.3201 (Win32) with a single trunk going to Netborder Express (running on the same server as pbxnsip), which talks to an internal Sangoma A101 single PRI card. Plugged into the Sangoma card we have a PRI from Nuvox Communications.

     

    Our issue is with blocking outbound caller ID.

     

    Optimally, we'd like to have a "*67" kind of feature, where we could selectively block outbound caller ID on a call-by-call basis, but I understand that the *67 code is a POTS feature, and won't work with a PRI. Instead, we need the switch (i.e. pbxnsip) to generate the proper signal to tell the PRI to block outbound caller ID.

     

    So, I'm testing the "Block outbound caller-ID" feature in the extension settings. When I enable that flag and watch the SIP packets, I see that there is an additional "Privacy: id" header added (if I'm using P-Asserted-Identity or P-Preferred-Identity) and an additional "privacy=full" added on to the end of the "Remote-Party-ID" field, if I enable that (I know, I know, it's a legacy option that shouldn't be used).

     

    However, the outbound caller ID is NOT being blocked, as it shows up on the callee side as normal.

     

    The Netborder logs show the same SIP headers for "privacy" turned on, but it still passes the ANI to the PRI (as it needs to in order for the call to be allowed by our provider).

     

    So, my question is: I've got three different players involved here (pbxnsip, Netborder/Sangoma, and Nuvox)... How do I figure out where the problem is?

     

    Thanks in advance for any help.

     

    Regards,

    Jon Heese

  5. Call logs all Hunt Group calls as to:"anonymous"

    in General Setup

    Posted

    What you will get is a CDR "fragment" for each segment of the call. Because it is impossible to summarize the whole call when going from one segment to another, the PBX generates a CDR record for each segment.

     

    That's the idea behind those internal CDR. In order to make the Auto Attendant look better, it pretends that the original destination was the extension. That part is debatable. However, we still have this option weather the PBX should change the From/To-fields. If you turn it off, then chances are that the original From/To fields make it through this CDR chain.

    Okay, how do I turn it off?

     

    And will turning it off affect something else negatively? i.e. Is there a reason why it's set this way by default?

     

    Also, I don't follow what you mean when you say "the original From/To fields"... Do you mean the fields that show the call as being "To: 305-247-2227"? If so, please understand that this is exactly what we DON'T want.

     

    I'm trying to be positive about this, but this has got to be the most unclear, inattentive support I've ever received for a piece of software. I don't even know whether I'm speaking to one person or a bunch of different people. Do you have a real name I can refer to you by?

     

    If one of my clients had this issue (and they WILL, once we're confident enough to sell this product), and it took them this long just to explain the problem to tech support, let alone come back with an answer, they would quickly dump me and go with someone else.

     

    Regards,

    Jon Heese

  6. Call logs all Hunt Group calls as to:"anonymous"

    in General Setup

    Posted

    Well, at the moment it is a feature. When you use the auto attendant that AA fixes the outside's callers inability to directly call and extension and it shows the call went right to the extension. For other calls like hunt groups, it should which number was dialled. In the CDR record you have more information so you can figure out where the call actually went.

     

    The alternative is that the auto attendant does not change headers. But then the headers would usually only show that the call went to a DID number of the company; IMHO that would be worse than changing the headers.

     

    BTW there is a flag in the domain settings that tells the PBX not to change the headers, did you try that out?

     

    Okay, so consider the following call routing:

     

    Incoming PSTN call

    |

    v

    Patton gateway

    |

    v

    Auto-attendant

    |

    v

    Hunt group

    |

    v

    Extension

     

    Since the above call came through an auto-attendant, shouldn't it at least show the hunt group as the final destination, not the PSTN/DID number of the company?

     

    Regards,

    Jon Heese

  8. Call logs all Hunt Group calls as to:"anonymous"

    in General Setup

    Posted

    What you see in the web interface are the From/To headers of the call.

    That's exactly what I've been trying to tell you! <_<

     

    The auto attendant changes the from/to headers when you place a call to an extension, so that it looks like the call originally went to that extension. "Supposed to be a feature", but I agree it is not very systematically.

    Can you explain this comment a little more? Are you saying that the auto-attendant changes the "To" header to reflect the dialed extension, but the ring group doesn't do this? If so, is there a reason why it behaves this way? Is there any way to make the ring group change the "To" header to properly reflect the extension that picks up the call? It's really quite useless to say that a call was made "To: 305-247-2227", considering that ALL of our incoming calls were made to that number, and what we really want to see is the extension that the caller spoke to.

     

    Oh, okay. No, I was not aware it was a logging (CDR) issue.

     

    The CDR record (see the XML file in the cdr directory) contains more detailed information. There you would find the information that you need where exactly the call went to. If you are sending the CDR out (in whatever way) you can transport that information.

     

    Are you using SOAP, CSV file or some other mechanism to send the CDR from the PBX? Maybe then the problem can be addressed easily.

    No, we're not using any logging besides the standard "e-mail call logs every night at 12 midnight". No SOAP/CSV/etc. at all. This is purely a call header issue, not something to do with the separate logging mechanism. I can also show this fact by looking at the "Current Calls" page in the web interface. When a call comes in through the ring group, and the caller is speaking with the extension that picked up, you can see in this screen that the "To" header shows "305-247-2227".

     

    Regards,

    Jon Heese

  9. Call logs all Hunt Group calls as to:"anonymous"

    in General Setup

    Posted

    Now we can pull the destination out of the Request-URI. Now the rules in http://wiki.pbxnsip.com/index.php/Inbound_Calls_on_Trunk apply. You can now also just assign an alias name to the extension, and then the PBX will send the call to the right destination.

     

    Progress is not success, I agree.

    I read through that wiki page a few times and I truly don't understand how it relates to this issue.

     

    Just to be clear, we are not having an issue with call routing. Calls are going exactly where we want, when we want. We are having an issue with call *logging*.

     

    When an outside caller calls our PBX through our PSTN lines (connected via a 4-port Patton 4114), they are sent directly to an auto-attendant, which allows them to dial an extension or wait to be connected to a ring group. If they dial an extension at that point, they are sent to that extension and the call logs show the caller as being connected to the user whose extension they dialed, like this:

     

    Time				   From 	 					 To			   Duration
2009/05/19 09:59:46	COMPANY XYZ ((305)555-4339)	Jon Heese (24)	08:07

     

    If they do not dial an extension, and instead they just wait, they are sent to the ring group, and the call logs show a call that looks like this:

     

    Time				   From 	 					 To				Duration
2009/05/19 09:57:34	COMPANY XYZ ((305)555-4339)	(305)247-2227	05:36

     

    Both calls above came through the *exact* same PSTN lines, the *exact* same Patton gateway, the *exact* same trunk in pbxnsip, and the *exact* same auto-attendant, and the first place they differ is whether they come through a ring group or go directly to an extension. This is why I think the problem is with the ring group, NOT the gateway or trunk setup.

     

    Please confirm that you understand my description above, as I feel like you may be chasing a problem that we don't have. I'm sorry if I seem dense or pedantic, but I just don't understand your logic here.

     

    Regards,

    Jon Heese

  10. Call logs all Hunt Group calls as to:"anonymous"

    in General Setup

    Posted

    If you get something else that "anonymous" then that's progress. Maybe you can just post the INVITE message that the PSTN gateway sends to the PBX, and then we should get close!

    Please forgive my skepticism, but I really don't see how this is progress at all. All we've done is change one static field ("anonymous") for another ("305-247-2227"). I just want to make sure that you realize that this number is just our main phone number here in the office, so seeing that all incoming calls (that came through the hunt group) in the logs spoke with "305-247-2227" is really quite uninformative.

     

    It seems to me that we're barking up the wrong tree, since the problem only occurs when the call comes through the hunt group. I don't quite follow how that could be an issue with our gateway configuration, since all of the calls come through the gateway the same way, but some calls are logged properly (direct extension dials) and some are not (hunt group calls).

     

    Anyway, here are the INVITE messages that show up in the logs on an incoming call:

     

    [7] 2009/05/19 10:59:50:	SIP Rx udp:192.168.10.240:5060:
INVITE sip:3052472227@192.168.10.2 SIP/2.0
Via: SIP/2.0/UDP 192.168.10.240:5060;branch=z9hG4bKf5a01bcd5
Max-Forwards: 70
Content-Length: 224
To: sip:3052472227@192.168.10.2
From: Elite Security <sip:3052455266@192.168.10.2>;tag=ec6505bcbf825e7
Call-ID: ffadce9d8dd6cdf4c231441a0e045cb6@192.168.10.2
CSeq: 597804336 INVITE
Route: <sip:192.168.10.2;lr>
Supported: timer
Remote-Party-ID: Elite Security <sip:3052455266@192.168.10.2>;party=calling;screen=no;privacy=off
Content-Type: application/sdp
Contact: sip:3052455266@192.168.10.240:5060
Supported: replaces
User-Agent: Patton SN4114 JO EUI MxSF v3.2.8.45 00A0BA03DC02 R4.2 2008-03-11 H323 SIP FXS FXO

v=0
o=MxSIP 0 147 IN IP4 192.168.10.240
s=SIP Call
c=IN IP4 192.168.10.240
t=0 0
m=audio 5158 RTP/AVP 0 8 101
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-15
a=sendrecv
[7] 2009/05/19 10:59:50:	SIP Tx udp:192.168.10.240:5060:
SIP/2.0 100 Trying
Via: SIP/2.0/UDP 192.168.10.240:5060;branch=z9hG4bKf5a01bcd5
From: Elite Security <sip:3052455266@192.168.10.2>;tag=ec6505bcbf825e7
To: <sip:3052472227@192.168.10.2>;tag=f47b8af437
Call-ID: ffadce9d8dd6cdf4c231441a0e045cb6@192.168.10.2
CSeq: 597804336 INVITE
Content-Length: 0

[5] 2009/05/19 10:59:50:	Trunk Patton SN4114 (global) sends call to account 80 in domain st2.st-aubin.com
[7] 2009/05/19 10:59:50:	SIP Tx udp:192.168.10.240:5060:
SIP/2.0 200 Ok
Via: SIP/2.0/UDP 192.168.10.240:5060;branch=z9hG4bKf5a01bcd5
From: Elite Security <sip:3052455266@192.168.10.2>;tag=ec6505bcbf825e7
To: <sip:3052472227@192.168.10.2>;tag=f47b8af437
Call-ID: ffadce9d8dd6cdf4c231441a0e045cb6@192.168.10.2
CSeq: 597804336 INVITE
Contact: <sip:3052472227@192.168.10.2:5060;transport=udp>
Supported: 100rel, replaces, norefersub
Allow-Events: refer
Allow: INVITE, ACK, CANCEL, BYE, REFER, PRACK, INFO, UPDATE
Accept: application/sdp
User-Agent: pbxnsip-PBX/3.3.1.3177
Content-Type: application/sdp
Content-Length: 216

v=0
o=- 52396 52396 IN IP4 192.168.10.2
s=-
c=IN IP4 192.168.10.2
t=0 0
m=audio 55578 RTP/AVP 0 8 101
a=rtpmap:0 pcmu/8000
a=rtpmap:8 pcma/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=sendrecv
[7] 2009/05/19 10:59:50:	Last message repeated 2 times
[7] 2009/05/19 10:59:50:	SIP Rx udp:192.168.10.240:5060:
ACK sip:3052472227@192.168.10.2:5060;transport=udp SIP/2.0
Via: SIP/2.0/UDP 192.168.10.240:5060;branch=z9hG4bKd689046f8
Max-Forwards: 70
Content-Length: 0
To: sip:3052472227@192.168.10.2;tag=f47b8af437
From: Elite Security <sip:3052455266@192.168.10.2>;tag=ec6505bcbf825e7
Call-ID: ffadce9d8dd6cdf4c231441a0e045cb6@192.168.10.2
CSeq: 597804336 ACK
Contact: sip:3052455266@192.168.10.240:5060
User-Agent: Patton SN4114 JO EUI MxSF v3.2.8.45 00A0BA03DC02 R4.2 2008-03-11 H323 SIP FXS FXO

     

    Thanks for your help so far.

     

    Regards,

    Jon Heese

  11. Call logs all Hunt Group calls as to:"anonymous"

    in General Setup

    Posted

    Okay, I figured out how to change the "To:" header on incoming calls to the gateway, and I set it to the main number, 3052472227 (all the lines go through the same SIP interface, so there was only one place to set it and they all take that setting).

     

    However, all that did was change the "To:" header to "(305)247-2227", instead of "anonymous". Calls that have come through the hunt group still don't show who they spoke with, they just show "(305)247-2227". If the caller dials an extension directly from the AA, that extension does show up in the logs.

     

    I can post log snippets if you need to see the difference between the two call strategies.

     

    Any other ideas?

     

    Regards,

    Jon Heese

    Any word on this issue yet?

     

    Regards,

    Jon Heese

  12. Call logs all Hunt Group calls as to:"anonymous"

    in General Setup

    Posted

    Okay, so any idea how to do any of this in the Patton interface?

     

    Regards,

    Jon Heese

    Okay, I figured out how to change the "To:" header on incoming calls to the gateway, and I set it to the main number, 3052472227 (all the lines go through the same SIP interface, so there was only one place to set it and they all take that setting).

     

    However, all that did was change the "To:" header to "(305)247-2227", instead of "anonymous". Calls that have come through the hunt group still don't show who they spoke with, they just show "(305)247-2227". If the caller dials an extension directly from the AA, that extension does show up in the logs.

     

    I can post log snippets if you need to see the difference between the two call strategies.

     

    Any other ideas?

     

    Regards,

    Jon Heese

  13. Call logs all Hunt Group calls as to:"anonymous"

    in General Setup

    Posted

    Even if you have one number that is rolling over, the carrier has a DID number assigned to each of the FXO lines. You can put these numbers in (they are sometimes visible when you place outbound calls, depending on your carrier uses them).

     

    I am not sure if the Patton has a problem if you assign the same number to all lines. That might also be worth a shot.

     

    Some carriers signal not only the Caller-ID also the called number. Maybe that is a problem here and we have a kind of "interop" problem on the FXO side. Though I would give this only a low probability.

    Okay, so any idea how to do any of this in the Patton interface?

     

    Regards,

    Jon Heese

  14. Call logs all Hunt Group calls as to:"anonymous"

    in General Setup

    Posted

    I would really try to nail it down at the To-header that leaves the gateway. There is no way the gateway can have a To-header "anonymous" in the PSTN world (in the SIP world, it would be possible).

     

    Maybe you just need to assign a DID number to a port and you accidentially forgot it. And the gateway right now just falls back to "anonymous" because it has nothing better.

    Okay, I understand, but I have no idea how to do what you're describing. We have 4 lines coming in to the Patton SN-4114 gateway and all of them ring in a rollover pattern based on one incoming number, 305-247-2227. Do you know where in the Patton web interface I'm supposed to configure the ports to identify with this number?

     

    You understand what I mean when I say that I'm worried this will just replace the "anonymous" in our call logs with our company's phone number, right? Still quite useless.

     

    Regards,

    Jon Heese

  15. Call logs all Hunt Group calls as to:"anonymous"

    in General Setup

    Posted

    At the beginning of the trace there is a packet received by the PBX from 192.168.10.240 where the To is already set to "anonymous". The PBX just "happily" passes that on.

     

    Check the gateway, maybe there is the DID number setup missing or somehow screwed up...

     

    In the telecom world, a "anonymous" destination is not possible yet! I want to call you, but I don't want to let you know which number I called :P .

    Thanks for your reply!

     

    Hmmm... The gateway is a Patton SN-4114, and I have to be honest: I have no earthly idea what I'm doing in its web interface 90% of the time. There is nothing intuitive about it, and I feel like I should have gone through a 9 month training class before even touching it. :) The only way I've been able to configure it correctly in the past is with step-by-step guides in the wiki, etc.

     

    I have poked around there for a few minutes just now, but I don't see anything jumping out at me.

     

    Something else bothers me about this a little: If this issue is being caused by a setting in the gateway, through which all calls come in, then why is the issue only occurring with calls that come through the Hunt Group, and not calls that are directed to an extension directly? Is there some fundamental difference with the way the call is handled that causes pbxnsip to change the "To" header for direct extension dialed calls and not Hunt Group calls?

     

    Furthermore, wouldn't changing some setting in the gateway to show the DID number of the incoming lines (instead of "anonymous") just change our To header from "anonymous" to "305-247-2227"? That's no good either. All we want is for the system to properly modify that header upon transfer out of the Hunt Group and into the individual user that picks up the call. Is this possible?

     

    Any further suggestions are greatly appreciated.

     

    Regards,

    Jon Heese

  16. Call logs all Hunt Group calls as to:"anonymous"

    in General Setup

    Posted

    Running pbxnsip v3.3.1.3177.

     

    We noticed that incoming calls that come through our main hunt group show up in the calls as being "to" anonymous, even though they connected and spoke with an extension via the hunt group.

     

    This is quite annoying for us, because when looking at the call logs, we can't see exactly who an incoming caller spoke with, unless they dialed that person's extension directly, which is rare.

     

    Is this intended behavior? A bug? Any way for us to change it to log the final connected extension as the recipient, instead of the "anonymous" hunt group?

     

    I set the log level to 7, enabled sip logging and placed a couple test calls, one via the hunt group and one via direct extension dial in the AA. Sure enough, the two log snippets differ in the fact that the hunt group call shows <sip:anonymous@192.168.10.2;user=phone>" in the To: header the whole time, where the direct extension call shows "To: 'Jon Heese' <sip:24@st2.st-aubin.com>" starting at the point where the caller dials my extension.

     

    The "From-Header" field on the hunt group is set to "Calling-Party", which I believe is the default. Plus it seems like we're having an issue with the "To:" header, not the "From:". You can see the issue in the attached log files, the message where they start to differ is on line 80 of both. You can see that the "Contact" field is set to <sip:24@72.17.170.196:5060;transport=udp> in both, but the To: header is showing "anonymous" on the hunt group call.

     

    Any ideas? Thanks in advance for any assistance you can provide.

     

    Regards,

    Jon Heese

    call_to_extension.txt

    call_to_hunt_group.txt

  17. New version of PBX (3.3.1.3177) breaks dialplan

    in Trunk Setup

    Posted

    In an effort make it easier for the users, we introduced use of country code in 3.3. Under the Domain->Settings, there's a Country Code field. Set that to 1. Also, you can set the area code if you like.

     

    Then on the trunk settings, there is field "Rewrite Global Numbers", you can set the NANPA 11 digits. These 2 settings should take care of the problem.

    Aha!

     

    We already had the country code set to 1, but changing the "Rewrite global numbers" field to "For NANPA (11 digits)" fixed this issue. Thanks for the tip.

     

    Regards,

    Jon Heese

  18. New version of PBX (3.3.1.3177) breaks dialplan

    in Trunk Setup

    Posted

    Okay, we recently upgraded from v3.2.x (I forget the exact version) to the 3.3.1.3177 executable, and everything seemed fine, until I tried to dial a long-distance number...

     

    Our dial plan is set up like this:

     

    dialplan.jpg

     

    The general idea is to make it easy for our users that are used to an old "dial 9 first" system, as well as to make 7-digit dialing possible for numbers in our area's main area code, 305. Then out-of-area dialing is handled.

     

    Our trunk is a 4-port Patton FXO box, with 4 PSTN lines plugged into it. Those lines require 1+areacode+number when dialing out-of-area numbers, and areacode+number for local (305 & 786) calls.

     

    Local numbers dial fine, with the 7-digit dialing, 786-xxx-xxxx and 305-xxx-xxxx dialing rules (and probably the 9+<local number>, I didn't check), but when I dial 1+areacode+number, I get a tritone error message about how I have to dial 1 before the number... But my dial plan IS dialing 1 before the number...

     

    Examples:

     

    I dial 18005551212, the dial plan should be following the "150" pref rule, and sending 18005551212 to the trunk, but I get the tritone telling me to dial 1 first.

     

    I dial 8005551212, the dial plan should be following the "160" pref rule (those are 10 x's), and sending 1800551212 to the trunk, but I get the tritone telling me to dial 1 first.

     

    So it's acting like it's stripping the 1 off the front of the number before it sends it to the trunk (Patton), yet the logs show that it's being sent with the 1...

     

    I tried resetting the config on my Patton gateway, with no change. I also tried a little two-stage dialing, using my "230" pref rule, which sends a *67 (caller ID block) to the trunk and then hands me dial tone on the trunk, and I can dial 18005551212 (or whatever) fine and it successfully dials and connects.

     

    So then I tried rolling back to the old (3.2.x) pbxctrl.exe executable, and voila, now it's working perfectly. So it's definitely something that changed in the dial plan handling code of the new version.

     

    Anyone know what could be causing this? How to nail down my problem? What I can do to fix it? Thanks in advance for any help.

     

    Regards,

    Jon Heese

  19. CPU maxing out, killing PBX

    in General Setup

    Posted

    Not to scare you, but...

     

    There are also other ways of DoS. If you are in the LAN, try ping -f, maybe from more than one computer. Okay, that's easy.

     

    The other nasty thing is spraying packets on the PBX, just to consume bandwidth. Especially for systems that don't have too much of it (e.g. sitting on a cable modem) and who don't have a QoS mechanism, it is easy to tease those installations badly. The experience is that the audio quality will suffer significantly. You will be chasing this for more than just days.

     

    We had a case some time ago, where the other party accidentially crashed the application, but the media part was still alive (of course that other party was not running the pbxnsip PBX ;) ). However, it was serious because that media server was sending RTP well-formatted, constantly on the PBX. It was simply comsuming so much bandwidth that we could not make phone calls over that line without mmm-aaa-jjj-ooo-rrr quality problems. Just before we were ready to call our service provider and beg him to blacklist this IP address, the media server had mercy and rebooted. Even blacklisting that address on the PBX would not have changed anything.

     

    Those guys who believe that ENUM and peer to peer will be the future should think about this scenario.

     

    Check out MPLS. It is not so stupid.

    Right.

     

    In a sense, we are lucky that this attack was "lightweight" enough that it isn't using up all of our T1's bandwidth, and all it did was crash the PBX. We are probably going to be asking our T1 provider to add a null route for this IP at the last hop on their side, just to make sure that packets aren't needlessly using up a portion of our bandwidth.

     

    Thanks for the insight.

     

    Regards,

    Jon Heese

  20. CPU maxing out, killing PBX

    in General Setup

    Posted

    So far this is not very common. But I am afraid it will be more common.

     

    The IP address white and black list were done for a reason. Yes, after finding out it saved your day. What we can learn from this case is that we need to handle this more automatically. IMHO it is not enough to use iptables in Linux to address this problem. What we need in the next version is an automatic blacklisting of addresses that fail to authenticate for so and so many times.

     

    Today the biggest limitation is usually the speed on the link to the Internet (for example, the typical CS410 case will be like that). However, as people are moving to hosted environments with very high speed links, that "natural" protection will not be there any more.

    I agree wholeheartedly. We were down for almost a full 24 hours chasing down .NET updates, license key problems, trunk issues, etc.. If this had been a client's system, instead of our own internal PBX, we would have probably lost the client, or at the very least had a lot of explaining to do as to why neither we nor the system were equipped to handle this situation more quickly.

     

    I don't blame the PBX or its developers though. A DoS is a DoS, and I probably should have recognized it earlier than I did. However, if the PBX was made smarter than me ;) (i.e. with a flow control or auto-blacklisting function), then it could have saved my butt even if I was unaware what a UDP packet even was...

     

    I talked with Pradeep on the phone a little while ago, and he agrees that it should be possible to implement a more automatic way to mitigate this problem in future versions, so here's to hoping that makes it down the chain and into the software soon!

     

    He also mentioned something I hadn't thought of yet in regards to the CS410, which is the possibility of running tcpdump on it via SSH, to determine if a similar symptom is being caused by a packet flood like this. Of course, considering that the CS410 undoubtedly runs on a single-core CPU, that may be more easily said than done. Of course, an enterprise-level router/switch would be able to analyze the packets coming in to the device, but most of our clients are quite small businesses (1-15 employees), so we would probably just pull out a hub and a laptop and run Wireshark there in that kind of situation.

     

    Regards,

    Jon Heese

  21. CPU maxing out, killing PBX

    in General Setup

    Posted

    Okay, well, after looking at a Wireshark capture from this box, I saw a bunch of packets from an IP in Amsterdam, so I tried disabling the WAN adapter on that server, and voila, the CPU usage went back down to normal and everything was fine.

     

    So, I blocked that IP address in the "Access" tab of the PBXnSIP config, re-enabled the WAN adapter and now everything's back up and running.

     

    It appears that someone was hammering the PBX with approximately 250 SIP "REGISTER" packets per second, thus crippling the process with a bunch of failed login attempts. Of course, when the system had no license installed, it summarily dismissed all incoming SIP registrations, so the problem didn't rear its ugly head until we inserted a license key and it had to process each failed login.

     

    Is this not a common occurrence? It seemed to be quite easy for this attacker to bring the system to its knees. Is the prescribed fix for this to do what we did, and block the attacking IP address with the "Access" feature?

     

    What if we have this problem on one of the PBXnSIP appliances, like the cs410? We can't do a Wireshark capture on the device, so would we have to depend on the logging or hooking up a test computer with a hub to detect these malicious SIP packets? What would your suggestion be in that case?

     

    Thanks for all who helped troubleshoot this and gave ideas.

     

    Regards,

    Jon Heese

  22. CPU maxing out, killing PBX

    in General Setup

    Posted

    there is a 3 minute key in you inbox.

     

    did you try installing on a NEW machine? does that work?

     

    matt

    Thanks for the key, Matt. It does the same thing.

     

    Yes, I tried installing on a new machine, and of course, it runs with the 3-minute license fine, but I can't insert my NFR licenses into it because they are keyed to the MAC address of this server's NIC.

     

    I would exclude that is has to do with the 3-minute key. If you have dual core, the PBX will worst-case block only one core, which leaves the other core available to log in and check what is going on.

     

    If you cannot log in, or the server does not respond to PING any more, then there is something very serious going onand I would say low probability it has something to do with the application (PBX).

     

    If you can log in, take a look at the taskmanager. If the PBX consumes the CPU or the memory, obviously then the one responsible has been found and we need to dig depper why this happens. The PAC example was something typical on what can be the problem and then it is releatively easy to fix it.

    It is a dual core server, so I can log in and watch it. The server responds to ping and other network traffic perfectly fine, the only problem is PBXnSIP, which maxes out 1 core of the CPU and totally stops responding to SIP, web, and service control requests. After killing the process, removing the license key from the pbx.xml file, and starting the service back up again, it responds fine (if I don't remove the license from pbx.xml, it will spike out immediately upon starting the service). I'd say there is a 100% chance that it's the application. ;)

     

    We have also seen other cases where a device driver freaked out and killed the machine. The fact that a software is a device driver does not imply that it is best quality...

    I'm not really sure what you mean by that last sentence, but Pradeep mentioned a possible bad USB driver in an e-mail to me a little while ago. There are no devices plugged into the server (USB or otherwise), and virtually nothing extraneous connected to it at all. It's just a Dell PowerEdge SC440 server with a network cable and a power cable plugged into it. No expansion cards at all, just a bare stable machine (been installed and running for 2-3 years) with PBXnSIP on it for the last month or two.

     

    Regards,

    Jon Heese

  23. CPU maxing out, killing PBX

    in General Setup

    Posted

    Get a new 3miute key and see if it will work on another computer...

     

    Sounds like it is NOT a key issue...now lets see if it is a pc issue...;-)

     

    matt

    Easier said that done, apparently...

     

    I tried requesting a 3-minute trial key (http://www.pbxnsip.com/sales/trial.php) once last night and once again this morning and haven't gotten anything yet... Is this supposed to be an automated process, or am I waiting for a human being to see my request and provide a key?

     

    I can just use any 3-minute key, right? Anyone have one handy they can send to me for some quick testing?

     

    This is getting kind of desperate; is there any quicker way to have this addressed by support staff?

     

    Incidentally, I don't think the key itself is the problem, since we have two different NFR keys for this machine, and both of them cause this issue. I suspect that if/when I get a 3-minute key and try to insert it, it will do the same thing. I think it's something on this machine that is causing the licensing mechanism to freak out when a key is entered.

     

    Regards,

    Jon Heese

  24. CPU maxing out, killing PBX

    in General Setup

    Posted

    Hi Jon,

     

    (your name looks awlfully familiar?)

     

    anyway, to test if license issue how about just type in a 3minute key? if it works you know what it is.

     

    also, what happens if you install on another pc? takes but a minute to try...

     

    just some ideas--hopefully helpful,

    matt

    Yes, it seems that we apparently followed you over here to PBXnSIP, Matt. ;)

     

    I just tried a couple different licenses on this machine, both commercial and free, with the same results. This license doesn't work on another machine (presumably because it's inherently linked to the MAC address of this server's NIC), but it just says "No License" on the other machine, it doesn't lock up like this, so it seems to be something particular to this server, not the license itself (which has worked fine for at least a month before today). Thanks for the suggestions.

     

    We had a similar complaint sometime back and an instance of PAC running against the PBX was causing the issue. After they closed and restarted the PAC, things were fine.

    I can't say for sure that someone didn't leave a PAC open somewhere, but I changed the web port to a different number and restarted the service (which should have rendered any running PAC client disconnected), with the same results. So I don't think it's the PAC. But thanks anyway.

     

    Same here yesturday around 2PM on the 28th.

    reboot, killing the app, removing the licence and try the 3 minutes... nothing, aboslutelly nothing worked...

     

    I have manually disabled pbxctrl and manually uninstall .NET Framwork 3.5 upgrade and 3.0 upgrade, reboot the machine and restart the server, everything seams to be fine since...

    Now there's an interesting suggestion...

     

    I do know that some Microsoft Updates had been performed on this machine recently (although not since last Friday), and sure enough, Add/Remove Programs does show .NET 3.0 and .NET 3.5 with a handful of updates on both.

     

    However, after uninstalling both .NET 3.0 and .NET 3.5 entirely, and rebooting the server, the service still pegs out when started.

     

    I suspect that something is going wrong with pbxctrl since it is targetting a specific processor and when stopping pbxctrl targeted processor is getting back to normal

    I'm not sure I understand what you're saying here, but interestingly, as you seem to imply here, only 1 of the 2 processor cores is actually pegging out. Does this shed any light on what might be happening?

     

    Thanks again for all the great suggestions, and please keep 'em coming! We are dead in the water!

     

    Regards,

    Jon Heese

  25. CPU maxing out, killing PBX

    in General Setup

    Posted

    Today, around 3:00pm, our internal PBXnSIP install (on a Windows 2003 server) came screeching to a halt. It was maxing out the CPU and was unresponsive to everything: web, SIP, service control, etc. I had to kill the pbxctrl.exe process to get the service to stop, but as soon as I started it back up again, it would go back to max CPU and become unresponsive again within 10-20 seconds, never to return.

     

    A reboot of the server made no difference, a restore from known-good tar backup made no difference, upgrading the pbxctrl.exe to 3.3.1.3177 made no difference, but I noticed that a "factory reset" (if I was quick enough to get in the web interface and do it before it locked up) did cause it to behave normally, until I tried restoring from the backup, at which point it went back to the max CPU.

     

    I noticed that if I manually remove the license key from pbx.xml, everything starts up fine. So, by my rough estimation, it's either the licensing component itself that's causing the problem, or it's something that is running *because* of the license (i.e. trunk registration, etc.) that's causing it. Either way, we are dead in the water until we can work out what's maxing out the CPU and killing the PBX.

     

    Any suggestions?

     

    Regards,

    Jon Heese

×
×
  • Create New...