Jump to content

Strange CDR logging format ?


rudik

Recommended Posts

Hi all,

 

I am currently implementing a new Snom One Blue PBX (2011-4.3.0.5021 Win64)and have a strange (out of the box ?) CDR logging format.

Instead of starting the file with <?xml version="1.0" encoding="utf-8"?> according to XML standard the example file called "93.xml" in the cdre-folder contains the following data ?? :

 

=== Start of file ===

TLVB c 1349258449.548 cid ,YjdlMWEzNzI4ZDVhMzhmNDhkYTkyM2NjMmIxNTM5YWQ. ct d d 1 e 1349258454.688 f +"Rudi Test" <sip:800@vijf> i ,YjdlMWEzNzI4ZDVhMzhmNDhkYTkyM2NjMmIxNTM5YWQ. o I p udp:192.168.20.142:36108 r "Rudi" <sip:061111111@vijf> rec /recordings/20121003/120049-o-061111111-800.wav s

1349258441.81 t "Rudi" <sip:061111111@vijf> u 44 vq VQSessionReport: CallTerm

LocalMetrics:

Timestamps:START=2012-10-03T10:00:49Z STOP=2012-10-03T10:00:54Z

CallID:YjdlMWEzNzI4ZDVhMzhmNDhkYTkyM2NjMmIxNTM5YWQ.

FromID:"Test 5 New"<sip:800@192.168.20.200>;tag=a817667b

ToID:"061111111" <sip:01111111@192.168.20.200>;tag=eb71e45556

SessionDesc:PT=8 PD=pcma SR=8000 FD=20 FO=160 FPP=1 PPS=50 PLC=3

LocalAddr:IP=192.168.20.200 PORT=52162 SSRC=0x48763f50

RemoteAddr:IP=192.168.20.142 PORT=37286 SSRC=0xe6cb2e02

x-UserAgent:snom-PBX/2011-4.3.0.5021

x-SIPterm:SDC=OK SDR=AN

PacketLoss:NLR=0.0 JDR=0.0

BurstGapLoss:BLD=0.0 BD=0 GLD=0.0 GD=0 GMIN=16

Delay:RTD=0 ESD=0 IAJ=0

QualityEst:MOSLQ=4.1 MOSCQ=4.1

RemoteMetrics:

JitterBuffer:JBA=3 JBR=0 JBN=40 JBM=40 JBX=80

PacketLoss:NLR=0.0 JDR=0.0

BurstGapLoss:BLD=100.0 BD=40 GLD=0.0 GD=5920 GMIN=16

Delay:RTD=0 ESD=0 IAJ=15

QualityEst:MOSLQ=4.1 MOSCQ=4.1

x-UserAgent:eyeBeam release 1009l stamp 37965

y extcall

=== End of file ===

 

Clearly this does not have a XML header but some other (unknown to me :blink: ) header. As currently the CDR XML files should be analyzed by a seperate control-system these files are useless because the control system expects XML format. I have been reading the documentation and 'normal' XML should be default for Snom One? Unfortunately in this case it is clearly not default ? What settings should be changed to 'reset' the logging to the 'good old' XML ?

 

The control-system runs great on the old version (3.4.0.3201 Win32), if possible the same format would be ideal.

 

Thanks,

Rudi

Link to comment
Share on other sites

No that's a feature. Because there are so many CDR, there is a special binary format for those entries. Background is that XML parsing cost a lot of CPU and you cannot just copy the contents because characters need to be un-escaped. TLV stands for Tag-Length-Value, and this is a primitive format to parse the contents of a stream.

Link to comment
Share on other sites

  • 1 month later...

From what I can understand the format of the new cdr files is as folows:

First 4 bytes is the string "TLVB"

After that follow 1 or more records like this:

<ll> <string of length ll> <LLLL> <string of length LLLL>

 

where:

ll = 2-bytes-integer

LLLL = 4-bytes-integer

 

I've tested the above in a few dozen cdrs

Link to comment
Share on other sites

Here's a python parser I came up with for the CDRs. It's been tested on a few hundreds CDRs (i,e and t) without any apparent problem but I can give no warranties.

 

def tlvb_parser(a):
   '''PBXnSIP v4 CDRs parser 
Acts as a generator so call it like: for key,value in tlvb_parser(data):

The CDR files have an .xml extension but are obviously not xml
They also start with TLV but they are not Type-Length-Value formated either 
From what I've seen in a few hundrend files their format is as folows:

First 4 bytes is the string "TLVB"
After that follow 1 or more records like this:
<ll> <string of length ll> <LLLL> <string of length LLLL>

where:
ll = 2-bytes-integer
LLLL = 4-bytes-integer
   '''
   # TODO: the case of 0 length for the value may be either:
   #   1) expected indicating an empty string   --or--
   #   2) unexpected indicating a corrupt file
   # Since I'm not sure I make the behaviour configurable:
   zero_len_for_value_is_expected = True
   # I also configure this sanity limit for the length of values
   maximum_expected_len_for_value = 8192

   import struct
   if a[:4]<>'TLVB':
       raise ValueError, "Invalid data (they don't start with TLVB)"
   else:
       a=a[4:]
       state='LOOKING_FOR_KEY' # initial state
       while a:
           if state=='LOOKING_FOR_KEY':
               # read key 
               l=struct.unpack('>H',a[:2])[0] # assume 1st 2 bytes are an integer representing the length of data to follow
               if l==0:
                   raise ValueError, 'Was expecting the key length but instead found 00h 00h'
               else:
                   if l>len(a)-2:
                       raise ValueError, 'Length for key points after EOF'
                   if l>255:
                       raise ValueError, 'Length for key>255 (%s) which is suspiciously large' % l
                   a=a[2:]
                   key=a[:l]
                   a=a[l:]
                   state='LOOKING_FOR_VALUE'
           else:
               # read value 
               l=struct.unpack('>I',a[:4])[0] # assume 1st 4 bytes are an integer representing the length of data to follow
               if l>len(a)-4:
                   raise ValueError, 'Length for value points after EOF'
               if l>maximum_expected_len_for_value:
                   raise ValueError, 'Length for value>%s (%s) which is suspiciously large' % (maximum_expected_len_for_value, l)
               if l==0 :
                   if not zero_len_for_value_is_expected:
                       raise ValueError, 'Was expecting the value length but instead found 00h 00h 00h 00h'
               a=a[4:]
               if l: 
                   value=a[:l] 
                   a=a[l:]
               else: 
                   value=''
               state='LOOKING_FOR_KEY' 
               yield key,value

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...