Pavlos Parissis

I tested the usage of taskset and it works as expected, all threads are running on the cpu core set by taskset. I am going to follow your advice and use taskset to bond a pbx to a specific core since the setting from pbx doesn't work as expected I hope with will not give us in issues when actually traffic is being processed. Furthermore, I modified the init script mention at LSB compliant init script thread and here is the code with a small change for using taskset to force process affinity. #!/bin/bash # ### BEGIN INIT INFO # Provides: pbx_01 # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 3 4 5 # Default-Stop: 0 1 2 6 # Short-Description: start and stop pbx_01 # Description: Init script fro pbxnsip. ### END INIT INFO # source function library . /etc/init.d/functions RETVAL=0 # Installation location CPU="02" INSTALLDIR=/pbx_service_01/pbxnsip PBX_CONFIG=$INSTALLDIR/pbx.xml PBX=pbx_01 PID_FILE=/var/run/$PBX.pid LOCK_FILE=/var/lock/subsys/$PBX PBX_OPTIONS="--dir $INSTALLDIR --config $PBX_CONFIG --pidfile $PID_FILE" # Disable the below when pbx is under pacemaker Cluster resource manager # Pacemaker doesn't like exit 5 #[ -x $INSTALLDIR/$PBX ] || exit 5 start() { echo -n "Starting PBX: " if [ -n "${CPU+x}" ] && [ ! -z $CPU ]; then daemon --pidfile $PID_FILE taskset $CPU $INSTALLDIR/$PBX $PBX_OPTIONS else daemon --pidfile $PID_FILE $INSTALLDIR/$PBX $PBX_OPTIONS fi RETVAL=$? echo [ $RETVAL -eq 0 ] && touch $LOCK_FILE return $RETVAL } stop() { echo -n "Stopping PBX: " killproc -p $PID_FILE $PBX RETVAL=$? echo [ $RETVAL -eq 0 ] && rm -f $LOCK_FILE return $RETVAL } case "$1" in start) start ;; stop) stop ;; restart) stop start ;; force-reload) stop start ;; status) status -p $PID_FILE $PBX RETVAL=$? ;; *) echo $"Usage: $0 {start|stop|restart|force-reload|status}" exit 2 esac

Initially I used taskset to control CPU core bonding and it worked very well. Since this functionality is offered by PBX, I said to use it and avoid extra changes on init script. I thought, it is a bit strange to not use application setting for controlling the application itself. Nevertheless, I will do a bit more testing with taskset and post the results here. If the result of the testing is the expected one[1], I will follow your advice to use taskset. Shall I assume no issues will pop up with that? But, you have to admit Process Affinity on linux is not working as expected. Thanks [1] all threads on the same core.

now I see something very strange with <affinity_mask>01</affinity_mask> [root@node-01 proc]# taskset -p 22756 pid 22756's current affinity mask: 1 [root@node-01 proc]# ps --pid 22756 -o psr,pid PSR PID 0 22756 [root@node-01 proc]# cat 22756/stat 22756 (pbx_01) S 1 22751 3176 0 -1 4194624 2689 0 0 0 9 19 0 0 25 0 11 0 463780 135675904 6086 4294967295 134512640 138931033 3216870704 3216867672 10126338 0 0 16789507 0 4294967295 0 0 17 0 0 0 0 several threads on the other core [root@node-01 proc]# cat 22756/task/22*/stat 22756 (pbx_01) S 1 22751 3176 0 -1 4194624 2571 0 0 0 9 7 0 0 25 0 11 0 463780 135675904 6083 4294967295 134512640 138931033 3216870704 3216867672 10126338 0 0 16789507 0 0 0 0 17 0 0 0 0 22758 (pbx_01) S 1 22751 3176 0 -1 4194624 9 0 0 0 0 0 0 0 -2 0 11 0 463804 135675904 6083 4294967295 134512640 138931033 3216870704 3085393400 10126338 0 0 16789507 0 0 0 0 -1 0 1 2 0 22759 (pbx_01) S 1 22751 3176 0 -1 4194368 11 0 0 0 0 11 0 0 25 0 11 0 463804 135675904 6083 4294967295 134512640 138931033 3216870704 3074969528 10126338 0 0 16789507 0 0 0 0 -1 0 0 0 0 22760 (pbx_01) S 1 22751 3176 0 -1 4194368 69 0 0 0 0 0 0 0 23 0 11 0 463804 135675904 6083 4294967295 134512640 138931033 3216870704 3064480432 10126338 0 0 16789507 0 0 0 0 -1 1 0 0 0 22761 (pbx_01) S 1 22751 3176 0 -1 4194368 12 0 0 0 0 0 0 0 18 0 11 0 463804 135675904 6083 4294967295 134512640 138931033 3216870704 3053990592 10126338 0 0 16789507 0 0 0 0 -1 0 0 0 0 22762 (pbx_01) S 1 22751 3176 0 -1 4194368 3 0 0 0 0 0 0 0 25 0 11 0 463804 135675904 6083 4294967295 134512640 138931033 3216870704 3043493200 10126338 0 0 16789507 0 0 0 0 -1 1 0 0 0 22763 (pbx_01) S 1 22751 3176 0 -1 4194368 3 0 0 0 0 0 0 0 25 0 11 0 463804 135675904 6083 4294967295 134512640 138931033 3216870704 3033004016 10126338 0 0 16789507 0 0 0 0 -1 1 0 0 0 22764 (pbx_01) S 1 22751 3176 0 -1 4194368 3 0 0 0 0 0 0 0 25 0 11 0 463804 135675904 6083 4294967295 134512640 138931033 3216870704 3022520580 10126338 0 0 16789507 0 0 0 0 -1 1 0 0 0 22765 (pbx_01) S 1 22751 3176 0 -1 4194368 3 0 0 0 0 0 0 0 25 0 11 0 463804 135675904 6083 4294967295 134512640 138931033 3216870704 3012030256 10126338 0 0 16789507 0 0 0 0 -1 1 0 0 0 22766 (pbx_01) S 1 22751 3176 0 -1 4194368 1 0 0 0 0 0 0 0 23 0 11 0 463804 135675904 6083 4294967295 134512640 138931033 3216870704 3001540832 10126338 0 0 16789507 0 0 0 0 -1 1 0 0 0 22767 (pbx_01) S 1 22751 3176 0 -1 4194368 1 0 0 0 0 0 0 0 25 0 11 0 463804 135675904 6083 4294967295 134512640 138931033 3216870704 2991051260 10126338 0 0 16789507 0 0 0 0 -1 1 0 0 0

Hi, I am using Process Affinity Mask to bond a CPU core with a specific PBX. so I can have on different CPU cores 2 PBX. I noticed that even I specifically set CPU core, PBX has few threads running on different CPU core with <affinity_mask>01</affinity_mask> I have PBX on 1st CPU CORE [root@node-01 proc]# taskset -p 2483 pid 2483's current affinity mask: 1 [root@node-01 proc]# ps --pid 2483 -o psr,pid PSR PID 0 2483 ## ps counts from 0 so 0 is the 1st CPU core proc fs confirms that for the parent process [root@node-01 proc]# cat 2483/stat 2483 (pbx_01) S 1 2472 9434 34816 9434 4194624 2687 0 0 0 7 34 0 0 25 0 11 0 315039 135675904 6084 4294967295 134512640 138931033 3219518768 3219515736 15430658 0 0 16781313 0 4294967295 0 0 17 0 0 0 0 but there are 4 threads running on the 2nd CPU core [root@node-01 proc]# cat 2483/task/24 2483/ 2485/ 2486/ 2487/ 2488/ 2489/ 2490/ 2491/ 2492/ 2493/ 2494/ [root@node-01 proc]# cat 2483/task/24*/stat 2483 (pbx_01) S 1 2472 9434 34816 9434 4194624 2576 0 0 0 7 7 0 0 25 0 11 0 315039 135675904 6085 4294967295 134512640 138931033 3219518768 3219515736 15430658 0 0 16781313 0 0 0 0 17 0 0 0 0 2485 (pbx_01) S 1 2472 9434 34816 9434 4194624 7 0 0 0 0 0 0 0 -2 0 11 0 315060 135675904 6085 4294967295 134512640 138931033 3219518768 3085655544 15430658 0 0 16781313 0 0 0 0 -1 0 1 2 0 2486 (pbx_01) S 1 2472 9434 34816 9434 4194368 9 0 0 0 0 27 0 0 25 0 11 0 315060 135675904 6085 4294967295 134512640 138931033 3219518768 3075231672 15430658 0 0 16781313 0 0 0 0 -1 1 0 0 0 2487 (pbx_01) S 1 2472 9434 34816 9434 4194368 71 0 0 0 0 0 0 0 18 0 11 0 315060 135675904 6085 4294967295 134512640 138931033 3219518768 3064742576 15430658 0 0 16781313 0 0 0 0 -1 0 0 0 0 2488 (pbx_01) S 1 2472 9434 34816 9434 4194368 12 0 0 0 0 0 0 0 15 0 11 0 315060 135675904 6085 4294967295 134512640 138931033 3219518768 3054252736 15430658 0 0 16781313 0 0 0 0 -1 0 0 0 0 2489 (pbx_01) S 1 2472 9434 34816 9434 4194368 3 0 0 0 0 0 0 0 25 0 11 0 315060 135675904 6085 4294967295 134512640 138931033 3219518768 3043755344 15430658 0 0 16781313 0 0 0 0 -1 1 0 0 0 2490 (pbx_01) S 1 2472 9434 34816 9434 4194368 3 0 0 0 0 0 0 0 25 0 11 0 315060 135675904 6085 4294967295 134512640 138931033 3219518768 3033266160 15430658 0 0 16781313 0 0 0 0 -1 1 0 0 0 2491 (pbx_01) S 1 2472 9434 34816 9434 4194368 2 0 0 0 0 0 0 0 25 0 11 0 315060 135675904 6085 4294967295 134512640 138931033 3219518768 3022782724 15430658 0 0 16781313 0 0 0 0 -1 0 0 0 0 2492 (pbx_01) S 1 2472 9434 34816 9434 4194368 3 0 0 0 0 0 0 0 25 0 11 0 315060 135675904 6085 4294967295 134512640 138931033 3219518768 3012292400 15430658 0 0 16781313 0 0 0 0 -1 0 0 0 0 2493 (pbx_01) S 1 2472 9434 34816 9434 4194368 1 0 0 0 0 0 0 0 20 0 11 0 315060 135675904 6085 4294967295 134512640 138931033 3219518768 3001802976 15430658 0 0 16781313 0 0 0 0 -1 1 0 0 0 2494 (pbx_01) S 1 2472 9434 34816 9434 4194368 1 0 0 0 0 0 0 0 25 0 11 0 315060 135675904 6085 4294967295 134512640 138931033 3219518768 2991313404 15430658 0 0 16781313 0 0 0 0 -1 0 0 0 0 when I have <affinity_mask>02</affinity_mask> [root@node-01 proc]# taskset -p 7888 pid 7888's current affinity mask: 2 [root@node-01 proc]# ps --pid 7888 -o psr,pid PSR PID 1 7888 only 2 threads is running on the other CPU core [root@node-01 proc]# cat 7888/stat 7888 (pbx_01) S 1 7864 9434 34816 9434 4194624 2686 0 0 0 8 25 0 0 25 0 11 0 354251 135675904 6083 4294967295 134512640 138931033 3214896976 3214893944 3728386 0 0 16781313 0 4294967295 0 0 17 1 0 0 0 [root@node-01 proc]# cat 7888/task/78*/stat 7888 (pbx_01) S 1 7864 9434 34816 9434 4194624 2573 0 0 0 8 12 0 0 25 0 11 0 354251 135675904 6083 4294967295 134512640 138931033 3214896976 3214893944 3728386 0 0 16781313 0 0 0 0 17 1 0 0 0 7890 (pbx_01) S 1 7864 9434 34816 9434 4194624 9 0 0 0 0 0 0 0 -2 0 11 0 354280 135675904 6083 4294967295 134512640 138931033 3214896976 3084991992 3728386 0 0 16781313 0 0 0 0 -1 0 1 2 0 7891 (pbx_01) S 1 7864 9434 34816 9434 4194368 9 0 0 0 0 12 0 0 25 0 11 0 354280 135675904 6083 4294967295 134512640 138931033 3214896976 3074568120 3728386 0 0 16781313 0 0 0 0 -1 0 0 0 0 7892 (pbx_01) S 1 7864 9434 34816 9434 4194368 69 0 0 0 0 0 0 0 23 0 11 0 354281 135675904 6083 4294967295 134512640 138931033 3214896976 3064079024 3728386 0 0 16781313 0 0 0 0 -1 1 0 0 0 7893 (pbx_01) S 1 7864 9434 34816 9434 4194368 12 0 0 0 0 0 0 0 18 0 11 0 354281 135675904 6083 4294967295 134512640 138931033 3214896976 3053589184 3728386 0 0 16781313 0 0 0 0 -1 1 0 0 0 7894 (pbx_01) S 1 7864 9434 34816 9434 4194368 3 0 0 0 0 0 0 0 25 0 11 0 354281 135675904 6083 4294967295 134512640 138931033 3214896976 3043091792 3728386 0 0 16781313 0 0 0 0 -1 1 0 0 0 7895 (pbx_01) S 1 7864 9434 34816 9434 4194368 3 0 0 0 0 0 0 0 25 0 11 0 354281 135675904 6083 4294967295 134512640 138931033 3214896976 3032602608 3728386 0 0 16781313 0 0 0 0 -1 1 0 0 0 7896 (pbx_01) S 1 7864 9434 34816 9434 4194368 3 0 0 0 0 0 0 0 25 0 11 0 354281 135675904 6083 4294967295 134512640 138931033 3214896976 3022119172 3728386 0 0 16781313 0 0 0 0 -1 1 0 0 0 7897 (pbx_01) S 1 7864 9434 34816 9434 4194368 3 0 0 0 0 0 0 0 25 0 11 0 354281 135675904 6083 4294967295 134512640 138931033 3214896976 3011628848 3728386 0 0 16781313 0 0 0 0 -1 1 0 0 0 7898 (pbx_01) S 1 7864 9434 34816 9434 4194368 1 0 0 0 0 0 0 0 23 0 11 0 354281 135675904 6083 4294967295 134512640 138931033 3214896976 3001139424 3728386 0 0 16781313 0 0 0 0 -1 1 0 0 0 7899 (pbx_01) S 1 7864 9434 34816 9434 4194368 1 0 0 0 0 0 0 0 25 0 11 0 354281 135675904 6083 4294967295 134512640 138931033 3214896976 2990649852 3728386 0 0 16781313 0 0 0 0 -1 1 0 0 0 why do I see this difference on CPU core allocation? I was expecting to see all threads on 1 CPU core. [root@node-01 pbxnsip]# ./pbx_01 --version Version: 3.4.0.3201 [root@node-01 pbxnsip]# cat /etc/redhat-release CentOS release 5.4 (Final)

the below output confirms what you have said [root@node-03 ~]# netstat -nap|grep pbx_01 tcp 0 0 192.168.78.10:5060 0.0.0.0:* LISTEN 6340/pbx_01 tcp 0 0 192.168.78.10:5061 0.0.0.0:* LISTEN 6340/pbx_01 tcp 0 0 192.168.78.10:80 0.0.0.0:* LISTEN 6340/pbx_01 tcp 0 0 192.168.78.10:22 0.0.0.0:* LISTEN 6353/sshd-pbx_01 tcp 0 0 192.168.78.10:443 0.0.0.0:* LISTEN 6340/pbx_01 udp 0 0 0.0.0.0:39321 0.0.0.0:* 6340/pbx_01 udp 0 0 192.168.78.10:161 0.0.0.0:* 6340/pbx_01 udp 0 0 192.168.78.10:5060 0.0.0.0:* 6340/pbx_01 udp 0 0 192.168.78.10:69 0.0.0.0:* 6340/pbx_01 udp 0 0 0.0.0.0:33122 0.0.0.0:* 6340/pbx_01 udp 0 0 :::45184 :::* 6340/pbx_01 udp 0 0 :::36295 :::* 6340/pbx_01 unix 2 [ ] DGRAM 14622 6340/pbx_01 [root@node-03 ~]# netstat -nap|grep pbx_02 tcp 0 0 192.168.78.20:5060 0.0.0.0:* LISTEN 5540/pbx_02 tcp 0 0 192.168.78.20:5061 0.0.0.0:* LISTEN 5540/pbx_02 tcp 0 0 192.168.78.20:80 0.0.0.0:* LISTEN 5540/pbx_02 tcp 0 0 192.168.78.20:22 0.0.0.0:* LISTEN 5572/sshd-pbx_02 tcp 0 0 192.168.78.20:443 0.0.0.0:* LISTEN 5540/pbx_02 udp 0 0 0.0.0.0:53900 0.0.0.0:* 5540/pbx_02 udp 0 0 0.0.0.0:49937 0.0.0.0:* 5540/pbx_02 udp 0 0 192.168.78.20:161 0.0.0.0:* 5540/pbx_02 udp 0 0 192.168.78.20:5060 0.0.0.0:* 5540/pbx_02 udp 0 0 192.168.78.20:69 0.0.0.0:* 5540/pbx_02 udp 0 0 :::46844 :::* 5540/pbx_02 udp 0 0 :::32894 :::* 5540/pbx_02 unix 2 [ ] DGRAM 13715 5540/pbx_02

Nice Thanks again

so having 2 PBXs, which listen on different IPs, on the same node shouldn't cause any issues. thanks

yes but pbx still listens on any IP for 4 ports [root@node-01 ~]# netstat -nap|grep pbx_01 tcp 0 0 192.168.78.10:5060 0.0.0.0:* LISTEN 2138/pbx_01 tcp 0 0 192.168.78.10:5061 0.0.0.0:* LISTEN 2138/pbx_01 tcp 0 0 192.168.78.10:80 0.0.0.0:* LISTEN 2138/pbx_01 tcp 0 0 192.168.78.10:22 0.0.0.0:* LISTEN 2165/sshd-pbx_01 tcp 0 0 192.168.78.10:443 0.0.0.0:* LISTEN 2138/pbx_01 udp 0 0 192.168.78.10:161 0.0.0.0:* 2138/pbx_01 udp 0 0 192.168.78.10:5060 0.0.0.0:* 2138/pbx_01 udp 0 0 192.168.78.10:69 0.0.0.0:* 2138/pbx_01 udp 0 0 0.0.0.0:54342 0.0.0.0:* 2138/pbx_01 udp 0 0 0.0.0.0:39380 0.0.0.0:* 2138/pbx_01 udp 0 0 :::49096 :::* 2138/pbx_01 udp 0 0 :::55132 :::* 2138/pbx_01

I followed your suggestion and my conf is now like this <ip_http_port>192.168.78.10:80</ip_http_port> <ip_https_port>192.168.78.10:443</ip_https_port> <ip_sip_udp_port>192.168.78.10:5060</ip_sip_udp_port> <ip_sip_tcp_port>192.168.78.10:5060</ip_sip_tcp_port> <ip_sip_tls_port>192.168.78.10:5061</ip_sip_tls_port> <ip_snmp_port>192.168.78.10:161</ip_snmp_port> <ip_snmp_community>public</ip_snmp_community> <ip_tftp_port>192.168.78.10:69</ip_tftp_port> and now listens on specific IP, expect the last 2 UDP sockets. thanks [root@node-01 ~]# netstat -nap|grep pbx_01 tcp 0 0 192.168.78.10:5060 0.0.0.0:* LISTEN 2138/pbx_01 tcp 0 0 192.168.78.10:5061 0.0.0.0:* LISTEN 2138/pbx_01 tcp 0 0 192.168.78.10:80 0.0.0.0:* LISTEN 2138/pbx_01 tcp 0 0 192.168.78.10:22 0.0.0.0:* LISTEN 2165/sshd-pbx_01 tcp 0 0 192.168.78.10:443 0.0.0.0:* LISTEN 2138/pbx_01 udp 0 0 192.168.78.10:161 0.0.0.0:* 2138/pbx_01 udp 0 0 192.168.78.10:5060 0.0.0.0:* 2138/pbx_01 udp 0 0 192.168.78.10:69 0.0.0.0:* 2138/pbx_01 udp 0 0 0.0.0.0:54342 0.0.0.0:* 2138/pbx_01 udp 0 0 0.0.0.0:39380 0.0.0.0:* 2138/pbx_01 udp 0 0 :::49096 :::* 2138/pbx_01 udp 0 0 :::55132 :::* 2138/pbx_01

Does anyone know why the setting in Bind to specific IPv4 address is ignored? Even I have set it pbx listens on all addresses. This happens on Version: 3.4.0.3201 and on CentOS 5.4

Hi all, I created a LSB compliant init script for pbx because the available init scripts aren't. The attached script passes the test mentioned here and implements the correct return status mentioned here. It has been tested on CentOS 5.4. If you are planning to but it under Cluster control pay attention to one of the comments. Have fun with it #!/bin/bash # ### BEGIN INIT INFO # Provides: pbx_01 # Required-Start: $local_fs $network # Required-Stop: $local_fs $network # Default-Start: 3 4 5 # Default-Stop: 0 1 2 6 # Short-Description: start and stop pbx_01 # Description: Init script fro pbxnsip. ### END INIT INFO # source function library . /etc/init.d/functions RETVAL=0 # Installation location INSTALLDIR=/pbx_service_01/pbxnsip PBX_CONFIG=$INSTALLDIR/pbx.xml PBX=pbx_01 PID_FILE=/var/run/$PBX.pid LOCK_FILE=/var/lock/subsys/$PBX PBX_OPTIONS="--dir $INSTALLDIR --config $PBX_CONFIG --pidfile $PID_FILE" # Disable the below when pbx is under pacemaker Cluster resource manager # Pacemaker doesn't like exit 5 [ -x $INSTALLDIR/$PBX ] || exit 5 start() { echo -n "Starting PBX: " daemon --pidfile $PID_FILE $INSTALLDIR/$PBX $PBX_OPTIONS RETVAL=$? echo [ $RETVAL -eq 0 ] && touch $LOCK_FILE return $RETVAL } stop() { echo -n "Stopping PBX: " killproc -p $PID_FILE $PBX RETVAL=$? echo [ $RETVAL -eq 0 ] && rm -f $LOCK_FILE return $RETVAL } case "$1" in start) start ;; stop) stop ;; restart) stop start ;; force-reload) stop start ;; status) status -p $PID_FILE $PBX RETVAL=$? ;; *) echo $"Usage: $0 {start|stop|restart|force-reload|status}" exit 2 esac exit $RETVA

ok fail enough

when a program crashes, I don't expect to remove the pid file. But if it is terminated gracefully, normal stop, I expect the pid file to be removed. The presents of a pid file on unix systems is quite significant. Nevertheless, I managed to do my work even with this behavior. I created a LSB compliant init script for pbx, I had to that because the current scripts on the site aren't and it was a requirement from the cluster software to use LSB compliant init script.

Does pbxnsip support HUP signal? If I send HUP via kill command, will pbx reload its configuration?

Hi, As the title states the pid file is not removed when you stop pbx, see below [root@node-01 pbxnsip]# ./pbx_01 --pidfile /var/run/pbx_01.pid [root@node-01 pbxnsip]# pidof -c -o $$ -o $PPID -o %PPID -x pbx_01 15791 [root@node-01 pbxnsip]# cat /var/run/pbx_01.pid 15791[root@node-01 pbxnsip]# [root@node-01 pbxnsip]# ps -ef|grep pbx_01 root 15791 1 1 15:57 pts/0 00:00:00 ./pbx_01 --pidfile /var/run/pbx_01.pid root 16296 23656 0 15:57 pts/0 00:00:00 grep pbx_01 [root@node-01 pbxnsip]# kill -TERM 15791 [root@node-01 pbxnsip]# cat /var/run/pbx_01.pid 1579 [root@node-01 pbxnsip]# pidof -c -o $$ -o $PPID -o %PPID -x pbx_01 [root@node-01 pbxnsip]# ps -ef|grep pbx_01 root 16895 23656 0 15:58 pts/0 00:00:00 grep pbx_01 [root@node-01 pbxnsip]# ./pbx_01 --version Version: 3.4.0.3201 The LSB specification states that "When a program is stopped, it should delete its pidfile" Does anyone know if this is solved in any newer releases?