mcbsys
-
Posts
78 -
Joined
-
Last visited
Content Type
Profiles
Forums
Events
Posts posted by mcbsys
-
-
11 hours ago, Vodia PBX said:
I still believe that the 401 does not matter... Anyhow in the screenshot the flag to check DNS every time is on—is there any particular reason for that? It adds another point of failure, and who knows, maybe that is causing our glitch?
Years ago I had and provider that had a different IP for each of their locations. They would use DNS to re-route to a different IP when one location went down. So I probably got in the habit then. Telnyx seems pretty stable with just the one IP (hopefully with redundant routing) so it's probably not necessary. And sure enough, with that turned off, Vodia stopped sending an email every 30 minutes. So why does Vodia consider it a status change when it refreshes the DNS, assuming the DNS doesn't change every 30 minutes?
-
You do have to manually specify SRTP (or ZRTP) in the Telnyx connection for Inbound, then Outbound matches it. I guess that then tells the PBX to send encrypted media?
-
Good news: I was mistaken. Telnyx does support TLS over a registered connection. You don't specify it; you just send the REGISTER over TLS and it accepts it. It does allow specifying that the media be encrypted and I've confirmed that that works with SRTP.
By setting registration to 6 minutes with a 50% renewal, I think I’m maintaining the connection okay. (I’ll want to do some more testing to confirm.) The one thing I don’t get is why some of the packets in my trace are still encrypted even after applying the keylogfile.
-
5 hours ago, Vodia PBX said:
You would definitively see the OPTIONS packets, e.g. by filtering SIP packets by the IP address. If you don't see them—something is wrong. Originally we did that for Teams (not sure why, probably just the "Microsoft way"), but other vendors are also starting to use OPTIONS as well and this seems to work fine.
I changed the FQDN/TLS trunk from SIP Proxy to Options and traced it for 2.5 hours, decrypting with the keylogfile. Not a single OPTIONS packet. I tried one outbound call, which failed. That shows up as one encrypted packet (in spite of the keylogfile) and 15 failed retransmissions. Maybe it's waiting on the other side to send OPTIONS first?
I was struggling to understand this setting of Options as a trunk type instead of Registration, Gateway, or Proxy. Then the mention of Teams took me to this article:
https://learn.microsoft.com/en-us/microsoftteams/direct-routing-protocols-sip
"Before an incoming or outbound call can be processed, OPTIONS messages are exchanged between SIP Proxy and the SBC. These OPTIONS messages allow SIP Proxy to provide the allowed capabilities to SBC."
That does, in fact, sound like Microsoft's approach to SIP conversations. It may be a legitimate use of OPTIONS (to discover capabilities of the other party). In that case, maybe the registration type should be called "Teams TLS with Options".
The rest of the world seems to use OPTIONS basically as a SIP "ping." Like the Asterisk article says, "OPTIONS messages are full SIP messages that a user agent can send to a peer and expect to get a standard SIP response. You can configure Asterisk to send these messages to a peer by setting the 'qualify_frequency' parameter in the peer’s aor object. At that interval, Asterisk will send the OPTIONS and will mark the peer’s contact as available and record the round trip time if it gets a response. If not, the contact is marked as unavailable..."
From what I can tell, the standard doesn't require TLS or even TCP for OPTIONS. To solve this kind of issue, where we need a keep-alive, we would need to be an setting on any trunk, "Send OPTIONS every x seconds" and then, if there's no response and TCP/TLS was active, close the connection (TCP RST).
What I still don't quite understand is whether a TLS conversation, once opened from the PBX for outbound calls, can (and will) be used by the vendor (Telnyx) for inbound calls. Can Telnyx "find" the open connection and put the inbound call on it? If so, a recurring OPTIONS would help. If not, it doesn't matter how many OPTIONS the PBX sends if the vendor always tries to start their own TLS handshake.
-
Okay I set the trunk to email on status change only, with the two explicit IP addresses:
I'm still getting an email every 30 minutes:
I expanded my tcpdump to include both IPs:
sudo tcpdump -i any -nn '(host 192.76.120.10 or host 64.16.250.10)' -v -w telnyx.pcapI captured the last re-registration at 12:03pm. Here's a screenshot of of the PCAP:
Now what? I could capture DNS traffic, but it doesn't seem confused about the IP address.
P.S. Have I ever mentioned that this forum software is pretty awesome? Rarely is it this easy to insert inline images in a forum!
-
2 hours ago, Vodia PBX said:
I have no doubt that the registration is stable. It's just the glitch with the notifications.
sip-anycast1.telnyx.comandsip-anycast1.telnyx.comseems to have a random TTL between 1 and 60 seconds, and maybe we see those glitches because it changes and then that address is not on the IP address whitelist any more. I would assume that the glitch goes away if you add the64.16.250.10and192.76.120.10.I think you are suggesting that I add those two IPs to a whitelist somewhere, but where? Most references to "whitelist" in the docs seem to be about what IP has permission to administer Vodia:
https://doc.vodia.com/docs/search?query=whitelist&searchType=articles
Does the SRV record sip-anycast1/2.telnyx.com even matter when I am registering directly to sip.telnyx.com? Here's the trunk status page; we're talking about the second, registered connection.
-
5 hours ago, Vodia PBX said:
OPTIONS work only for SIP trunks that don't register.
When I tried OPTIONS, it was without registration, with TLS. But it looked the same as SIP Proxy--I never saw an OPTIONS packet sent in the machine-level trace. How often are the OPTIONS packets supposed to go out and can I control that?
5 hours ago, Vodia PBX said:If the SIP trunk does register, you can instead the "Keep-alive time" and force the PBX to re-register every 3.5 minutes or so.
Unfortunately, it seems Telnyx doesn't allow TLS on a registered connection. I may need the "keep-alive time" if I try TCP without TLS. UDP seems fine with the default 1-hour timeout, I guess because it doesn't depend on keeping the connection open.
-
Came back to this issue in the last few days.
I believe I've uncovered the main issue: Azure closes idle connections after 4 minutes without sending a TCP Reset (TCP RST) to let the other party know. The timeout can be extended up to 30 minutes on a static IP address, but sending a TCP RST is only possible if you add another layer, a load balancer:
https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-tcp-idle-timeout
So without TCP Reset, we need some kind of keep-alive in under four minutes to keep the connection open. There’s a pretty good article on that here: https://www.asterisk.org/wanted-dead-or-alive/. I tried the network-level keep-alives suggested here https://serverfault.com/a/851251/166311 and documented here https://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html, but I had an issue with the standard registered UDP trunk where inbound calls were failing. It may have been unrelated, but I backed out the Linux changes.
Also, I’m thinking that the TLS connection is established in each direction, so even if I kept the outbound connection open from the PBX to Telnyx, unless Telnyx is also sending keep-alives, inbound calls will fail after four idle minutes. Telynx support says their TCP timeout is 3604 seconds, so just over an hour, after which I assume they would re-do the TLS handshake.
I tried the Vodia trunk of type OPTIONS, but I didn’t see an OPTIONS traffic. Is there some way to set its frequency?
Last I tried using TLS with registration:
- If I left inbound as FQDN (as defined at Telnyx) but did registration for outbound, outbound worked after a five-minute delay, but inbound failed after a delay—confirming that Telnyx didn’t restart the TLS conversation.
- If I set up the connection at Telnyx for registration in both directions, and set the proposed duration in Vodia to 6 minutes with re-registrations at 50% of expiry time, outbound worked but inbound failed immediately. I then realized that Telnyx doesn’t allow setting the inbound SIP Transport Protocol to TLS on a registered connection—that’s only available on FQDN connections.
I'm slowly coming to the conclusion that TLS won't work with Azure hosting the virtual machine unless I set up a load balancer and turn on TCP RST. Theoretically that would gracefully close the connection so the PBX would re-handshake for outbound and Telnyx would re-handshake for inbound.
-
The PBX has a static IP address.
sip.telnyx.com is consistently 192.76.120.10 in the U.S. (and in the PCAP, if I recall correctly), though they do allow 64.16.250.10 as a secondary. https://sip.telnyx.com/ The current DNS status is below.
As mentioned, I saw four packets every half hour, REGISTER, 401 Unauthorized, REGISTER, 200 OK. Does the "401 Unauthorized" challenge make the system think it lost registration, then the 200 OK that it re-gained registration, thus triggering a "status change" email?
-
18 minutes ago, Vodia PBX said:
The trunk status change emails are (supposed to be) independent from the registration metrics emails. They are only sent out when something has changed. Are the subjects exactly the same for the status change?
Yes. Here are my deleted emails, sorted by Subject and Time.
-
On 3/30/2023 at 6:48 PM, Vodia PBX said:
We do not send email on re-register. If you get an email every hour, you do have a problem that need to investigate.
Circling back to this. For the record, I'm on 68.0.28.
Yesterday, all inbound calls to the PBX suddenly started dropping. Outbound worked okay. Rebooting the PBX solved the problem, but Telnyx asked if the trunk was still registered. I didn't think to check at the time, but no one is in the customer office today, so I thought I would try some debugging. I enabled Send email on status change :
and I set up a packet trace at the Debian machine level:
sudo tcpdump -i any -nn host 192.76.120.10 -v -w telnyx.pcapResults:
1. Every 29-30 minutes, the trace has four packets: REGISTER, 401 Unauthorized, REGISTER, 200 OK. Looks like normal traffic to me. There were no calls in the 6.5 hours I tested, so I have exactly 52 packets (13 registrations).
2. Every 29-30 minutes, matching the REGISTER events, I get two emails from the PBX (total 26 emails, 13 registrations):
Subject: Phone System: Status change on trunk Telynx - Registered
Body: Trunk . This is a notification email. Do not reply.Subject: Phone System: Registration Metrics for trunk Telynx - Registered
Body:
x-RegData: RPN=sip.telnyx.com
x-RegMetrics: RRD=217 RRS=200 [the RRD number varies; RRS is always 200]So if the PBX doesn't send emails on re-register, what's going on here?
And more importantly, is there a way to only be notified when the system registration fails, i.e. when it is offline?
Thanks for your help.
-
Thanks @alex1. I may try 69.0.3 when I have time but for now I've given up on SMS to queues.
-
reg_settings.htm is the URL where you change the setting, e.g. https://phonesystem.mydomain.com/reg_settings.htm. The normal way to get there is System > Settings > General > System. Then the first thing on that page, General > System Name.
-
This is a single-tenant install with 5-15 calls per day, mostly one at a time. There isn't much to pool!
So is OPTION something I can add to a trunk and if so, how? The documentation says that the Keep-alive time setting applies to registration, which is not in use here.
I guess I can switch to registration; I just didn't think it was necessary if I have a static IP.
-
Very helpful, thanks. Not the original private certificate key that I was seeking, but the private session keys, which are more useful as they can be shared with a third party who needs to review the PCAP.
This allowed me to move forward in diagnosing a connection problem:
I may come back to this email notification issue later.
-
I've set up a TLS / FQDN Telnyx trunk pretty much following https://doc.vodia.com/docs/telnyx-secure-sip-trunking. However I'm trying to use IP-based rather than REGISTER-based authentication:
What happens is that the first outbound call works fine, but a second call to the same number five minutes later fails.
Machine-level tracing shows me the first call has a full TLS handshake, but the next call just sends an INVITE with no handshake, which Telnyx ignores. According to Telnyx support, "As far as I know Every calls requires a new TLS handshake which is failing here."
I've tried this with the trunk configured as a SIP Proxy and a SIP Gateway and still see the same behavior.
Is there some way to get Vodia to start a TLS handshake on each call? Or do I just need to use REGISTER-based authentication with TLS?
Thanks.
-
Thank you. I do at least see the extensions in the user_alias folder.
I'm all for relational databases but I'm more used to dBASE, SQL, et. al. Obviously this is some kind of XML database. Is there a browser and schema available?
-
I needed a PCAP to see all traffic on the trunk, so I captured at the Linux machine level
sudo tcpdump -i any -nn host 192.76.120.10 -v -w telnyx.pcapThe traffic is TLS-encrypted, so I need the private key, I assume from the Let's Encrypt certificate, to decrypt it.
The Vodia UI does not show the private key and the Let's Encrypt certificates are not in the usual place (/etc/letsencrypt/live). I finally found them in /usr/local/pbx/certificates/136.xml. Alas, the private key is encrypted, so I won't be able to decrypt the PCAPs.
How can I get a full, decrypted trace of all PBX traffic?
-
> Passwords that humans enter are sensitive information
Well is was more not being able to find the system-created SIP passwords that was causing trouble.
How do I map the n.xml files in the extensions folder to extension numbers?
-
On 3/31/2023 at 10:13 PM, Vodia PBX said:
The status is the SIP status for the trunk. 200 means "Ok", and other status depend. For example 408 means that the REGISTER that the PBX has sent did not return anything and timed out. Usually this is because of Internet connectivity problems.
Is there a way to generate a trunk-level PCAP inside the PBX, one that tracks _all_traffic (not just subsets attached to individual calls)?
This PBX is running on an Azure virtual machine so the Internet connection is probably pretty good.
-
The Version is 68.0.28.
I can't find any mention of "localhost" though I'm not sure I know where to look.
System > Tenants > List > pencil icon > Primary DNS address = phonesystem.mydomain.com
System > Settings > SIP > Settings > IP routing = private public
On the Debian server, under /usr/local/pbx/extensions, I have 1.xml through 6.xml. Interesting, since I only have 5 extensions. That would seem to support my suspicion that deleting x250 didn't really get rid of it.
Not only are the passwords encrypted, it seems that these XML files do not even contain the extension number. If I open every one, and look at <display_name>, I may be able to backtrack to figure out which file applies to which extension. Am I allowed to see the files that maps extensions to XML definitions? Or for that matter the indexes to all the various folders on the server?
I am really missing the simple Excel file export in 3CX that lists all extensions and passwords (web, SIP, phone UI) in plain text!
-
I think there may be some confusion over which password was causing the issue. The Grandstream "admin password" refers to the password one uses to log in to the HT802's UI. Specifically, the Grandstream provisioning XML contained something like this:
<!-- Admin password for web interface -->
<P2>kR8*gCAc&tfwG!sa</P2>which the HT802 did not like. I agree, it should be able to handle special characters, but per the documentation, it cannot.
Yes, I created kR8*gCAc&tfwG!sa in Vodia as a nice complex password under Tenant > Settings > General Settings > Provisioning Parameters. Unfortunately if I simplify that to work with the Grandstream, I will have to reprovision all the other phones as well. This is where it is a drawback to have a common admin password for all devices. I'm making a note to avoid special characters on future installs.
-
Okay thanks. Yeah that seems to disappear before it gets to me. Fortunately the System Name in the subject works.
-
11 hours ago, Scott1234 said:
You can also just input the line under reg_email like so.
"YOUR NAME HERE" <abc@abc.com>
@Scott1234, sorry but where in the UI is "reg_email"?
As mentioned above, adding a name in front of the email address at System > Settings > Messaging > Email Setup > From address was not enough.
IP-based authentication not working with Telnyx TLS / FQDN trunk
in Trunk Setup
Posted
Going back to my longer post above: am I seeing this correctly, that Vodia will not let me set up OPTIONS as a kind of keep-alive SIP "ping"? That would seem preferable over re-registering every three minutes, but I'd need to be able to set the frequency that OPTIONS are sent.