ndemou Posted December 1, 2015 Report Share Posted December 1, 2015 Looking at directories cdrt and cdri the numbering of files is not continuous. cd /opt/pbxnsip/pbx/cdrt ls | grep "337114..xml" 3371140.xml 3371141.xml 3371142.xml <---|___where's 3,4? 3371145.xml <---| 3371146.xml 3371147.xml 3371148.xml 3371149.xml It's not very common of course: ls | grep "336.....xml"|wc -l 9986 that's two CDRs out of 10000... I've checked my backups for the last two months and I have the same rare but reoccurring jumps in numbering I feel like cold water is running down my spine... P.S. Checked the logs of the last 3 days for severity levels [0]...[2] and I don't see something that I can correlate: grep -h "^\[[0-2]" /opt/pbxnsip/pbx/log/2015-1[12]-*|grep -v 'Reject call because\|connection refused\|Hangup reason:\|expired call' [0] 20151130102621: SMTP: Illegal state [2] 20151130134339: Writing to row that does not exist, ignoring request [1] 20151130225655: Timeout: Call 153 not found [2] 20151201095035: Writing to row that does not exist, ignoring request [2] 20151201095608: Writing to row that does not exist, ignoring request [0] 20151201112834: SMTP: Illegal state [2] 20151201120229: Writing to row that does not exist, ignoring request [2] 20151201120229: Last message repeated 2 times [0] 20151201131824: SMTP: Illegal state [0] 20151201142546: SMTP: Illegal state Quote Link to comment Share on other sites More sharing options...
Vodia PBX Posted December 2, 2015 Report Share Posted December 2, 2015 LOL no that can really happen. For example, the number of CDR per account can be limited, and then the PBX will bit those "dents" into the index. The writing to a row that does not exist is a little bit more serious. Next version will include the table name so that we get a better idea which table is affected. Let me know if you want to have a 5.3.2a build (which OS), which is pretty much the same like 5.3.2 at the moment. The SMTP log messages are also a little concerning. Maybe you can get us a separate log for the email client (attachment is okay) so that we can see if there is anything serious. Quote Link to comment Share on other sites More sharing options...
ndemou Posted December 2, 2015 Author Report Share Posted December 2, 2015 Regarding the loss of CDRs issue: I don't quite understand your comment (English is not my native language). Am I right in my understanding that there is nothing to worry about? Regarding "writing to a row that does not exist": I'll have to discuss it with management and get back to you on this. To help me make a my point I'd like to know a) what is it that we worry about here if you feel confident running the suggested version on a heavily loaded production system that currently runs 5.2.4 (we have a second server standing by in case of trouble but the trouble and the switch-over will not go by unnoticed by our clients) Regarding the SMTP logs: by "get us a separate log for the email client" you mean you want me to set "Log email events:" to level #9 and send you the resulting log events when I hit the error message? Quote Link to comment Share on other sites More sharing options...
ahennis@voicespring.net Posted December 2, 2015 Report Share Posted December 2, 2015 What does 5.3.2a include? Quote Link to comment Share on other sites More sharing options...
Vodia PBX Posted December 2, 2015 Report Share Posted December 2, 2015 Regarding the loss of CDRs issue: I don't quite understand your comment (English is not my native language). Am I right in my understanding that there is nothing to worry about? Regarding "writing to a row that does not exist": I'll have to discuss it with management and get back to you on this. To help me make a my point I'd like to know a) what is it that we worry about here if you feel confident running the suggested version on a heavily loaded production system that currently runs 5.2.4 (we have a second server standing by in case of trouble but the trouble and the switch-over will not go by unnoticed by our clients) Regarding the SMTP logs: by "get us a separate log for the email client" you mean you want me to set "Log email events:" to level #9 and send you the resulting log events when I hit the error message? The missing records are generally no reason for concern. We have already added the new log to the latest version, it would be great if you can upgrade to 5.3.2a but before you do that please let us know what OS you are running. For the SMTP you can just send me a private message with the SMTP log (level 9). In that message you can also include the OS version. Quote Link to comment Share on other sites More sharing options...
Vodia PBX Posted December 2, 2015 Report Share Posted December 2, 2015 What does 5.3.2a include? So far not much. There was a problem with conference rooms not showing the dial plan in the web interface, a problem with the new snom provisioning server, and some changed for the web interface for PnP. But the 5.3.2a will be a moving target as we work on features and fixes. Quote Link to comment Share on other sites More sharing options...
ahennis@voicespring.net Posted December 2, 2015 Report Share Posted December 2, 2015 We are running into the Snom provisioning server issue. Is 5.3.2a the only version that fixes the issue? We are about to load up 5.3.2 on our test server. Quote Link to comment Share on other sites More sharing options...
ndemou Posted December 2, 2015 Author Report Share Posted December 2, 2015 @Vodia: We're running CentOS 6, 32bit @ahennis: I think that your issue is unrelated to the subject of this thread. If it really is, then please begin another thread to make it easier for me, vodia and other interested readers to follow the discussion on both issues. Quote Link to comment Share on other sites More sharing options...
ahennis@voicespring.net Posted December 3, 2015 Report Share Posted December 3, 2015 Sorry, my bad. I know better than to hijack a thread. Quote Link to comment Share on other sites More sharing options...
Vodia PBX Posted December 3, 2015 Report Share Posted December 3, 2015 5.3.2a is also available for CentOS32 now. Quote Link to comment Share on other sites More sharing options...
ndemou Posted January 7, 2016 Author Report Share Posted January 7, 2016 OK after testing yesterday we've finally felt confident to upgrade the production server to 5.3.2a. What will you need to proceed? Anyway here are today's errors: grep -h "^\[[0-2]" /opt/pbxnsip/pbx/log/2016-01-07.txt|grep -v 'Reject call because\|connection refused\|Hangup reason:\|expired call\|already used before\|No billing email available for domain' [2] 20160107123707: Writing to row that does not exist, ignoring request [2] 20160107123707: Last message repeated 2 times [0] 20160107162723: SMTP: Illegal state and here are a few lines before and after the "Writing to row that does not exist," grep -B5 -A5 "Writing to row that does not exist" /opt/pbxnsip/pbx/log/2016-01-07.txt [7] 20160107123706: Port 432: Clear last request [5] 20160107123706: BYE Response: Terminate 28df04006923-568e3d9e-434d6383-2e953778-1f7428@127.0.0.1 [6] 20160107123706: 4 more requests pending to 10.13.37.102:8080 [6] 20160107123706: Port 279: Sending RTP to 91.138.139.181:45962, codec G729/8000 [3] 20160107123707: lookup_index error: Call port index is -1 [2] 20160107123707: Writing to row that does not exist, ignoring request [2] 20160107123707: Last message repeated 2 times [3] 20160107123707: lookup_index error: Call port index is -1 [5] 20160107123707: Not setting dialog state of non-existing call port (call-id=) [6] 20160107123707: 5 more requests pending to 10.13.37.102:8080 [7] 20160107123707: Port 244: Media-aware pass-through mode By the way, I also had 157 lines with the error "[1]...Call-ID xxxxx already used before" and 332 "[2] No billing email available for domain xxxxx" Quote Link to comment Share on other sites More sharing options...
ndemou Posted January 10, 2016 Author Report Share Posted January 10, 2016 To fresh your memory you asked me to install 5.3.2a in order to investigate the errors "Writing to row that does not exist, ignoring request" that appear in the logs. I did so and I still got one error per day on busy (non weekend) days. What's next? grep -B5 -A5 "Writing to row that does not exist" /opt/pbxnsip/pbx/log/2016-01-08.txt [7] 20160108150939: Port 305: Determine pass-through mode after receiving response [7] 20160108150939: Port 305: RTP pass-through mode [7] 20160108150939: Port 304: RTP pass-through mode [6] 20160108150939: Port 305: Different Codecs (local G729/8000, remote PCMA/8000), falling back to transcoding [6] 20160108150939: Port 304: Different Codecs (local PCMA/8000, remote G729/8000), falling back to transcoding [2] 20160108150940: Writing to row that does not exist, ignoring request [2] 20160108150940: Last message repeated 2 times [3] 20160108150940: lookup_index error: Call port index is -1 [3] 20160108150940: Last message repeated 2 times [5] 20160108150940: Not setting dialog state of non-existing call port (call-id=) [5] 20160108150941: set codec: codec PCMA/8000 is set to call-leg 307 And by the way on busy days, I also get hundreds of lines with the error "[1]...Call-ID xxxxx already used before" and "[2] No billing email available for domain xxxxx". I don't know if this is serious and/or related. Quote Link to comment Share on other sites More sharing options...
Vodia PBX Posted January 12, 2016 Report Share Posted January 12, 2016 Could it be that some of your phones are using the same Call-ID? For example VoIP phones that don't have the clock set don't have any random input and always generate the same Call-ID after a reboot. In that case I don't think it would be a serious problem. We have changed something in the subsystem recently that should make the PBX more robust against timeouts when calls disappear. Although I have problems making the direct connection, it might cause something like that. Quote Link to comment Share on other sites More sharing options...
ndemou Posted January 12, 2016 Author Report Share Posted January 12, 2016 Sorry but I can't tell for sure which statement of your comment corresponds to which one of the 3 errors I've mentioned. I guess that the first paragraph is about the call-id already exists error -- am I right? Anyway during the last 3 days I had ~10.000 unique call-ids recorded in my logs and 384 "call-id already exists" errors. The call-ids are like random@IP. I noticed that 80% of the errors originate from only 6.5% of the IPs. The top offender (@192.168.10.237) appears in 11% of the errors. So it appears to be something the has to do with only a limited number of phones. If however I understand your comment correctly you suggest that a reboot of the VoIP phone is required to trigger the behaviour that you think causes the event in the logs. It's highly unlikely that about 10-15 phones get rebooted so often. What will be the result for the caller? Am I right in guessing that his call will be rejected and he'll have to try again? I can't guess what the 2nd paragraph is about. Please clarify. Quote Link to comment Share on other sites More sharing options...
Vodia PBX Posted January 13, 2016 Report Share Posted January 13, 2016 Yea it is sometimes hard to keep the message threads separated. But that can also be the beauty of a forum! In the old days we had the same problem: The Call-ID generated from the PBX were not random. Actually not all. The reason is that the C-library function rand() is actually deterministic, and it always starts with the same value after a reboot. That is when we discovered the function srand. I guess that device has the same problem. If it is a soft phone, it might actually happen if it gets started up, which may happen a lot on a PC. Who ever has written the software, send them a link to the srand library function. The 2nd paragraph was about the question what the PBX should do about it. The answer is, it should still work. The Call-ID is only a part of the call identifier; only if the IP address and the To-tag are the same, the PBX will have problems telling the calls apart. This might happen if the softphone gets restarted quickly; in that case a couple of wrong CDR entries are the least problem. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.