forked from voipmonitor/sniffer
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathChangeLog
2094 lines (1614 loc) · 116 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
11.9.3 22.07.2015
=================
- use hour and day filter from alerts in fraud [VG-647]
- fix crash at exit - add call regfailedcache->prune(0) before delete sqlStore
- set limit conten_type byte for ssl v2 (
11.9.2 16.07.2015
=================
- Fix ssl / tls decoding for clients who sends ssl version 2.0 CLIENT HELLO instead of SSL 3 or TLS
- Fix cleanspool safe to not wipe more data than it should
11.9.1 16.07.2015
=================
- fix 11.9 active calls in GUI which did not reported all ringing calls (with 18X) until calls were picked up
11.9 14.07.2015
===============
- add sequential fraud alert [VG-557]
- fix issue with buffer configuration which can be too low and can cause MEMORY FULL deadlock.
- report MEMORY FULL via manager command to catch that state via GUI
- implement name_sensor configuration option which overrides ID when spooldir_by_sensorname is enabled
- fix crash when send call over HTTP is enabled and disabled
- fix prematurely end of call when 300 multiple choices received. Sniffer closed the call immediately without waiting for other SIP messages [VS-286]
- store also BYE to mysql sip history
- new option save_sip_history (more info in voipmonitor.conf) [VS-278]
11.8.1 04.07.2015
=================
- fix crash if SIP Content-Length is negative
- fix race leading to crash when ssl decryption is enabled (duplicate call tcpReassemblySip.clean)
11.8 04.07.2015
===============
- fix bug in cleanspool which can lead to crash (all versions)
- add new option spooldir_by_sensorname [VS-282]
- fix issue with string 'set buffer memory limit to ...'
- cocncurent calls fraud - add support for new filtrers - called numbers and destination ip addresses [VG-598]
- restore value opt_maxpoolsize if enough space is in spool again [VS-284]
- update destination address in process SIP MESSAGE (apply method for invite to message) [VG-615]
11.7.7 29.06.2015
=================
- fix low MOS score and false packet loss for calls with re-INVITE
11.7.6 25.06.2015
=================
- fix crash when parsing SIP packet which does not have Content-Length and there are garbage data after \r\n\r\n (in this case SIP packet will stop on \r\n\r\n)
- fix high load on some virtualizations (15% CPU when idle) - optimize calling usleep
- fix parsing multiple sip packets in one packet
11.7.5 25.06.2015
=================
- use bigint for ID in create table `register`
- fix SIP REGISTER fraud alert which counted also good REGISTER attempts
11.7.4 19.06.2015
=================
- fix race condition leading to crash in case there is SIP 3XX redirection and RTP packets for that call is still in RTP queue
11.7.3 19.06.2015
=================
- remove PID file as the latest operation to fix killing voipmonitor with init.d/voipmonitor
11.7.2 17.06.2015
=================
- if sql queue is enabled and myslq.proc is corrupted do not loose CDR
11.7.1 15.06.2015
=================
- reset jitter/loss statistics for both direction which fixes some more false negative RTP stats
11.7 15.06.2015
===============
- do not process RTP if call is hangedup to prevent false negative statistics
- fix skipping RE-INVITE SDP parsing from called which affects false negative RTP statistics
- Added rrd graph for SQL delay [VS-239]
- Fix -nan is showed instead of 0 value in SQLq RRD graph and in RRD db-files.
- move craete and drop partitions to separate thread which workarounds problem with blocking calls due to long operation on cleaning / creating partitions [VS-277]
- when SQL query cache is enabled repeat query with error indefinitely
11.6.8 11.06.2015
=================
- print average SQL query delay in syslog which helps to see if there is network problem between sniffer and SQL server [VS-239]
- fix save Q.850 reason if there is no text part which in that case whole Q.850 reason was ignored
11.6.7 10.06.2015
=================
- stop spamming syslog with: warning - packet from sensor (%i) in RTP created for sensor
- add support for send call info to url [VS-228]
- fix crash on arm7 ubuntu libc
- disable bt_sighandler
- modify rqueue - add prevent by lock between push and pop in waitForFree mode
11.6.6 03.06.2015
=================
- fix SKINNY protocol where some times recording was missing when there was >15 seconds delay between call setup and answer from phone (which is valid)
11.6.5 03.06.2015
=================
- fix crash when SIP RURI is too long
11.6.4 03.06.2015
=================
- fix threading mode 4 & defrag - do not discard ip in ip packets
11.6.3 02.06.2015
=================
- treat REGISTER with 404 as failed REGISTER
- add suport into backup mode for new custom headers tables
- fix compare files_sorter_asc in function listFilesDir which fixes loading many files in scanpcapdir mode
- add option scanpcapdir_disable_inotify which disables inotify in scanpcapdir mode (scanpcapdir_disable_inotify = yes)
- if table is missing partitions do not discard CDR after 5th unsuccesfull insert but keep it until there is partition
- fix multithreading in sql queue store mode (which is by default off)
- fix custom header partition creation in rare case
11.6.2 25.05.2015
=================
- add support for silence detector & clipping for GUI 10.20
- fix cloud commands communication
11.6.1 25.05.2015
=================
- support more skinny header versions (fixes bug where there are no caller/called informations in calls)
11.6 22.05.2015
===============
- Store all SIP responses into new SQL table so it can be searched for all error statuses [VS-10]
- Implement SIP/Q.850 reason codes (in BYE). ALTER table on cdr is needed - see sql commands when starting sniffer [VS-45]
- fix force flush tar if name contains double slash (//)
- Added new RRD graph types for packet counters (PSR-RTP count, PSSM-SIP message count, PSSR-SIP register count, PSC-calls count, PSA-all packets counter, PSS-all SIP packets). [VG-428]
- Added support for NOTIFY message in live sniffer [VS-55]
11.5.4 19.05.2015
=================
- do not close call after 60 seconds when 491 request pending received
11.5.3 18.05.2015
=================
- fix 11.5.2
11.5.2 18.05.2015
=================
- fix sql cache threads which did not start processing sql cache in daemon mode
11.5.1 14.05.2015
=================
- modify virtualudppacket option: do not transform simple tcp packets to udp
11.5 14.05.2015
===============
- new command to flush tar file via manager command (speeds up pcap download from GUI)
- new option virtualudppacket (default off) for saving sip packets as simple udp
- fix typo in .conf file sipwithoutrtptimout -> sipwithoutrtptimeout
11.4 13.05.2015
===============
- implement SQL file cache - new option query_cache (if enabled) stores every SQL query to file first so it does not consumes all memory when mysql server is unrecheable and CDR in queues will also survive sniffer restart / crash.
- add new option spooldir_by_sensor which separates spooldir by id "/var/spool/voipmonitor/id/date/..." [VS-263]
- fix race in manager command reload
- fix threading mode 3 if dedup was disabled
11.3 05.05.2015
===============
- fix empty caller name in database for >= 11.2.1
- fix duration of calls in active calls if there were no packets during the call (ie missing RTP)
- fix rare crash when audio recording is enabled during sniffing (it was not fixed properly in 11.1.1)
- add support for list of active live sniffing so GUI can close all active sniffers (if someone forget to close it) [VG-189]
- add new options passertedidentity, ppreferredidentity and remotepartypriority (parsing callerid name/num from P-[asserted|preffered]-identity instead of From header (check voipmonitor.conf)
11.2.2 04.05.2015
=================
- fix memory leak in tcpreassembly (clean empty links_
- fix memory leak for IP fragmented packets (in threading mode 4)
11.2.1 28.04.2015
=================
- fix mysql errors when creating new database / tables (on empty db)
11.2 28.04.2015
================
- sniffer can now receive traffic from remote sniffers and sniff on local interfaces at the same time (if mirror_bind_ip is set)
- when SIP/2.0 200 Auth failed is received as response to REGISTER treat it as 4XX response (some switches workaround / slow down register attack by sending 200 OK with this exact status)
- add support for upgrade sniffer by git - new configuration options upgrade_by_git = yes, git_folder = /usr/src/voipmonitor-sniffer-git (GUI support is done in the latest)
- set default packetbuffer_compress to "no" non-mirroring mode (it is not needed anymore)
- add missing if(custom_headers_cdr/message) when storing to saveToDb
- do not list call in active calls if there was BYE message
11.1.2 22.04.2015
=================
- do not connect to the database when sniffer is set to send packets to master sniffer (mirror_destination_ip) fix all => 11.X
11.1.1 22.04.2015
=================
- fix rare crash when audio recording is enabled during sniffing
- fix crash if RTP packet with padding bit set and incorrect padding length in packet and log such case (it can crash only when saveaudio was enabled or any capture rule activated audio recording)
- fix support for download pcap in cloud mode
11.1 20.04.2015
================
- implement dynamic sql tables for custom headers which are now controlled via database - no alter table on cdr* are required and adding / deleteing is now intsant. Upgrade to the GUI 10.X is required to use them. Backward compatibility with old custom_headers option is retained.
- Added packets counter for SIP type REGISTER and SIP type MESSAGE to RRD charts [VG-428]
- fix fraud refresh locking racecondition (not reported yet)
11.0.9 17.04.2015
=================
- fix cloud mode connection
11.0.8 14.04.2015
=================
- fix critical bug in 11.0.6 and 11.0.7 which can lead to corrupted pcap files in rare case.
- show md5 and error number when auto upgrade fails
11.0.7 14.04.2015
=================
- fix critical bug leading to crash when the sniffer is configured as a receiver and have multiple senders or if sniffing on multiple interfaces (eth1,eth2)
11.0.6 14.04.2015
=================
- if configuration option rtpthreads was 0 or empty rtp threads was completely disabled leading to crash. Force to set it to >= 1
- enhance buffer handling when I/O blocks writes
11.0.5 13.04.2015
=================
- fix time issue in default I/O (tar=yes) leading to MEMORY FULL
11.0.4 13.04.2015
=================
- fix memory corruption when checking RRD version on start and for every RRD graph load. Upgrade to 11.0.4 is adviced for all 11 versions.
11.0.3 10.04.2015
=================
- do not allow to set memory buffers higher than available memory to prevent swap to death.
11.0.2 10.04.2015
=================
- fix buffer freeze leading to MEMORY IS FULL
11.0.1 08.04.2015
=================
- fix crash when new DSP is enabled (fax, silence, etc..)
11.0 07.04.2015
===============
New features
============
- SIP TLS decryption support (see ssl = yes in voipmonitor.conf)
- concatenate pcap files into single tar files (enabled by default) lowering IOPS by factor 10x and more allowing to store thousands of simultaneous calls on single 7.2k RPM or network storage. More details in voipmonitor.conf tar=yes
- implement signal digital processing for G711 alaw/ulaw:
- inband dtmf decoder (option inbanddtmf in voipmonitor.conf)
- silence detector - counts time when caller or called stream is silence and number of seconds from last silence to end of call (option silencedetect in voipmonitor.conf)
- FAX CED/CNG tones detection marks call as FAX and GUI can try to convert T30 into tiff/PDF (option faxdetect)
- clipping detection - values are stored into new cdr columns cdr.caller/d_clipping_mult100. (optino clippingdetect in voipmonitor.conf
- dumpallallpackets option dumps all capture traffic into files (split by size (MB) or by 1GB if 'yes')
- add capture rules based on custom headers [VG-508]
- FreeBSD compatibility
- add support for ALAW/ULAW mono (default was always stereo)
- support different samplerate within one call (only for alaw/ulaw and one other codecs)
- possibility to disable udp defragmentation with option udpfrag
- add option to dump queued SQL queries and load them on next restart (manager command sqlvmexport and sqlexport)
- add support for x-opus codec in SDP
Optimizations
=============
- split SIP / RTP processing to more threads allowing processing >1.5Gbit (10 000 simultaneous calls) on single server without special drivers. Enabled by default.
- signifficantly lower memory consumption when processing RTP
- SET GLOBAL innodb_file_per_table=1 when creating database (only for mysql >=5.1)
- unify memory buffers option into single option max_buffer_mem which is 2000MB by default now.
- redesign configure / Makefile and do not require libssh for ordinary build (it is used only for cloud)
- speedup sniffer terminating
Bugfixes
========
- add check valid packet length and data offset to packetbufer::processPacket
- fix deadlock problem / duplicity in cdr
- fix negative skinny callid, fix timout skinny channel problem, fix onhold, create skinny channel only on OFFHOOK message
10.1.49 02.04.2015
==================
- do not crash if some mysql tables are missing or some rows disappears during sniffer reload.
10.1.48 01.04.2015
==================
- fix storing sip proxy column when there are in-dialog INVITEs
10.1.47 30.03.2015
==================
- fix racecondition when doing live sniffing
10.1.46 24.03.2015
==================
- fix racecondition leading to crash in rare cases (all versions)
10.1.45 24.03.2015
==================
- use string-time-format for calldate in active calls which fixes timezone offset in GUI
10.1.44 20.03.2015
==================
- fix autocleanspoolminpercent and autocleanmingb which did not worked when autocleanmingb was set differently than default
10.1.43 20.03.2015
==================
- enhance SIP TCP reassembler which now detects complete SIP packet sooner and fixes ignoring some packets where there is no TCP ack to them [VS-205]
10.1.42 19.03.2015
==================
- implement prefix trim in fraud option
10.1.41 17.03.2015
==================
- fix false high RTP jitter average value
- implement fraud change country / destination only for connected calls
10.1.40 10.03.2015
==================
- fix RTP jitter precision
- link with the latest libmysqlclient
10.1.39 09.03.2015
==================
- do not include SIP MESSAGE type in active calls (listcalls in manager command)
10.1.38 06.03.2015
==================
- add sips: tag support which fixes empty caller number / domain when this tag was used
10.1.37 03.03.2015
==================
- add negation tel. numbers also for tel. numbers for fraud exclude / include list (prefixed with !)
10.1.36 03.03.2015
==================
- negation record prefixed with "!" in fraud IP address exclude/include list was not working at all
10.1.35 26.02.2015
==================
- implement ERSPAN encapsulation
10.1.34 26.02.2015
==================
- fix 10.1.33 second try
10.1.33 25.02.2015
==================
- fix properly: do not use more memory then set in packetbuffer_total_maxheap in case the sniffer is in receiver mode and remote sniffer reconnect to the receiver and sends all traffic in burst (receiver will slow down)
10.1.32 25.02.2015
==================
- properly detect 32/64 bit version when doing auto upgrade
10.1.31 16.02.2015
==================
- fix SIP REGISTER saving last packet to pcap file (200 OK or any last packet was missing)
10.1.30 16.02.2015
==================
- do not check RRD library everytime GUI access sniffer over manager minimizing forking which can cause __lll_lock_wait_private deadlock (rare but it happened)
10.1.29 16.02.2015
==================
- fix audio channel swap at the end of call for OGG stereo when one of the channel is silence (all versions)
10.1.28 16.02.2015
==================
- fix > 10.1.23 versions which crashed if configured as a receiver and buffers were full
10.1.27 05.02.2015
==================
- fix GRE bad ip header detection
- fix crash when SIP packet is very small (<3B)
- fix crash when doing auto upgrade of the sniffer
10.1.26 30.01.2015
==================
- if RTP changes destination port but SSRC is the same create new RTP so the GUI can see that the change happened [VS-168]
10.1.25 30.01.2015
==================
- fix TCP manager racecondition when quick connections are made which leads to communication errors (and lags in the GUI when loading charts)
10.1.24 27.01.2015
==================
- fix G729 audio decode for 10ms variant
10.1.23 23.12.2014
==================
- do not use more memory then set in packetbuffer_total_maxheap in case the sniffer is in receiver mode and remote sniffer reconnect to the receiver and sends all traffic in burst (receiver will slow down)
10.1.22 11.12.2014
==================
- fix bad arithmetic when writing time of RTP packets (in wors case it was +- 1 sec) which can lead to show RTP packets before SIP packets in SIP history
10.1.21 11.12.2014
==================
- fix pcapscandir which ignored pcap files if they were added before some pcap file was processed (it read only first inotify event from the queue)
10.1.20 21.11.2014
==================
- fix crash in skinny when parsing multiple skinny messages in one packet (all versions)
10.1.19 21.11.2014
==================
- implement skinny_ignore_rtpip option for cisco skinny protocol in case call manager receives RTP always on the same port. More info in voipmonitor.conf
10.1.18 20.11.2014
==================
- close all file descriptors when upgrading / restarting sniffer to prevent leakage and other problems (manual restart after autoupgrade is recommended again until this version)
10.1.17 20.11.2014
==================
- clean buffers before issuing restart which might solve problem with allocating ringbuffer when upgrading from GUI
- fix potential crash in skinny packet handling
- fix updating destination number for ring group if connected call does reinvite
10.1.16 18.11.2014
==================
- fix crash in tcpreassembly
10.1.15 17.11.2014
==================
- move memory deallocation in threading mod 4 to different thread which offloads 10% for t0 thread
- add new option cdr_check_exists_callid: when storing CDR check if there is the same CDR with the same sip.Call-ID and replace it if the new one contains RTP and the old one not this option is intended for case where you need to mirror SIP signalling to RTP media nodes and every RTP voipmonitor node sends CDR based on the same SIP thus diplicating CDR and only one CDR have RTP associated. default is off
10.1.14 12.11.2014
==================
- fix crash on some linux versions when connecting to cloud (in cloud mode)
10.1.13 11.11.2014
==================
- fix billing partitions creation
10.1.12 10.11.2014
==================
- revert manager interface to use BLOCK socket which might fix recent issues with timeout
10.1.11 5.11.2014
=================
- fix crashing when SIP message is multipart/mixed (for example if contains SIPI). All versions >= 9.0
10.1.10 4.11.2014
=================
- fix issue with detect user agent for caller / called (VS-135)
- support charts for older librrd (<1.4)
- fix not storing CDR for some mysql versions / configurations if there is more sip reponse with the same text but case sensitive (200 OK | 200 ok)
- use /etc/localtime even if sniffer is statically linked with non default localtime location which fixes our 32bit sniffer builds wchih have bad timezone since 10.1.6. Print time on stdout when sniffer starts for control purpose.
- fix live audio monitor when there is duplicated RTP streams (choppy sound)
10.1.9 30.10.2014
=================
- Fix loading old files when scanpcapdir is used. (If files were moved/created in scanpcapdir and voipmonitor was not running, files were not loaded after voipmonitor started - only new files were processed.)
10.1.8 27.10.2014
=================
- 32 bit static compiled versions are now compatible with all > 2.6.18 (centos 5.10 etc.)
- add manager command cleanup_calls
10.1.7 16.10.2014
=================
- Implement new configuration option which will restrict spool cleaning based on time (for example cleanspool_enable_fromto = 1-5 will run cleaning only in night between 1-5 hours)
10.1.6 16.10.2014
=================
- Another attempt to fix stucked manager interface - clean threads with function pthread_detach() instead of pthread_attr_setdetachstate(&attr,PTHREAD_CREATE_DETACHED)
10.1.5 15.10.2014
=================
- fix parsing DTMF sip calls with DTMF RTP frames with payload 101 where payload 101 is not in SDP
10.1.4 13.10.2014
=================
- support older rrd library when generating graphs
- fix concurrent access to rrd graphs
- try to fix TCP manager problem with creating threads on 32bit (manager stuck)
10.1.3 13.10.2014
=================
- Stucked TCP manager was not properly set to non blocking socket thus the timeout never occured.
- fix pcap_dump_asyncbuffer was never applied thus always deafault to 100MB.
- add new manager commands - check_filesindex, reindexfiles_date %s - example: reindexfiles_date 2014-09-03, reindexfiles_datehour %s %i - example: reindexfiles_datehour 2014-09-03 15
10.1.2 10.10.2014
=================
- Try to fix stucked TCP manager on accept (use select before accept) and blocking all new connections until restart
10.1.1 7.10.2014
================
- add new library dependency into configure script
- remove unused rrd function to make it compatible with older librrd
10.1 6.10.2014
==============
- implement logging status information to RRD charts
10.0.42 6.10.2014
=================
- fix RTP stream for G729 10ms packetization - both reading and audio decoding which was completely ignored
10.0.41 26.9.2014
=================
- fix broken cleaning in version 10.0.40 if it was suspended due to increased I/O buffer above threshold. After upgrading you have to run "telnet localhost 5029 - reindexfiles"
10.0.40 26.9.2014
=================
- automatically suspend cleaning spooldir if async buffer > 50% to not kill all I/O and unsuspend if < 10%
- remotepartyid option is set to yes the SIP Remote-Party-ID is taken into account when storing caller/called number into the database. If callerid is anonymous and Remote-Party-ID is present the number is always taken from Remote-Party-ID if present regardless on the remotepartyid option. Default option is no.
- fix issues with memory allocation failed (if not enough memory)
10.0.39 26.9.2014
=================
- fix -v 0 parameter causing FULL BUFFER
- limit SIP REGISTER SQL queue to 1000000 then drop them to not cause buffering it until all ram is used.
- fix live sniffer for cloud mode
10.0.38 17.9.2014
=================
- fix reported packet loss (>65000) for calls with more than 2 RTP streams (bug introduced since 10.0.36)
10.0.37 12.9.2014
=================
- static builds are compatible with 2.6.18 kernels again
- fix active calls realtime audio monitor chopped sound - increase length of ringbuffer for chunkplayer
10.0.36 11.9.2014
================
- fix ignoring RTP streams in case there are more legs and MARK bit sets
- fix timezone issues for register failed when sniffer is in different timezone then mysql server
- do not print to syslog when audio fails to decode because of missing RTP
- fix duplicate errors for max_allowed_packet
- add column message.content_length
10.0.35 9.9.2014
================
- maintainance build (compiled with static libnss to prevent crashing on buggy libc)
10.0.34 9.9.2014
================
- fix G723 packetization - if not 30ms audio is out of sync
10.0.33 4.9.2014
================
- fix 10.0.28 - 10.0.32 versions where if RTP with the same SSRC changes destination IP it was treated incorrectly as new RTP stream causing low MOS score in some cases.
10.0.32 2.9.2014
================
- fix 10.0.30 and 10.0.31 which did not recognized called or caller RTP stream in some cases
10.0.31 2.9.2014
================
- fix create SQL table fraud_alert_info
- fix mysql loop connection has gone if error happens during mysql connect
10.0.30 1.9.2014
================
- fix case where call has multiple RTP streams (caller <--> rtp proxy A <--> rtp proxy B <--> called) - caller side was not decoded properly
- fix FAX calls (fax codec was not stored as payload number 1000)
- import configuration files from /etc/voipmonitor/conf.d/*
10.0.29 1.9.2014
================
- handle memory allocation failure when there is no free RAM (do not crash)
- fix support for multiple contacts for SIP REGISTER instead of replacing it with the latest REGISTER.
10.0.28 28.8.2014
=================
- fix problem with multiple SIP messages in one TCP message resulting in garbled SIP response codes
- it can happen that next SIP hop reuses the SDP unique session identifier but changing port and ip which results to replace it in the first hop in voipmonitor RTP tracker which resulted in non capturing some RTP packets. This version ensure that SDP unique RTP session is replaced only within the same SIP source IP
- fix content-type value in SIP MESSAGE db
- add --sipport command line option
- fix reading pcap with linux cooked header (when sniffing with -i any)
- modfy column type message in table message from TEXT to MEDIUMTEXT (only for table create) to support messages > 64kb
- fix issue with negated IP addresse in alerts when they are used in IP groups and not specified directly
- Set "487 Request Terminated CANCEL" ONLY if the last SIP message is CANCEL without further response messages
10.0.27 27.8.2014
=================
- fix all 10.X which can corrupt heap (if saving sip packets) when doing IP defragmentation
- fix 10.0.26 crashes and fix bogus packet message
- fix all versions bug where ARP packets and other IP protocols can cause undefined behaviour
- fix hide SIP message body in MESSAGE packet only if set opt_hide_message is set
10.0.26 21.8.2014
=================
- handle corrupted IP header packet and do not crash the sniffer
- fix header_ip address for large sip message method packets (VG-326)
- fix issue with alter table cache_number_location
10.0.25 21.8.2014
=================
- fix 10.0.24 dscp fix which caused immediate crash
10.0.24 20.8.2014
=================
- fix issue with dscp and multi rtp streams
- remove leading zeros when comparing number as international.
10.0.23 20.8.2014
=================
- fix low MOS score for all codecs with sample rate > 8khz (all wideband) and probably audio decoding
- fix no RTP stream when using SIP compat headers (content-lenght: = l:)
- trim long packet (reassembled from tcp) to max 65535 bytes
10.0.22 18.8.2014
=================
- when auto upgradeing sniffer close all connections properly so senders can reconnect
10.0.21 15.8.2014
=================
- compiler fix
10.0.20 15.8.2014
=================
- fix low MOS score when stream was properly paused > 20sec
10.0.19 14.8.2014
=================
- fix low MOS score for G729 silence suppresion / CNG
- in multiple sender one receiver (mirroring) prevent mixing RTP stream from multiple sniffers and prevent mixing DLT mixing in one pcap file.
10.0.18 12.8.2014
=================
- force RTP resync in case of reINVITE which fixes false packet loss detection and low MOS score.
10.0.17 12.8.2014
=================
- adjust jitterbuffer f1 simulator to not lower MOS score for latencies <50ms
10.0.16 7.8.2014
================
- fix cleaning of active REGISTER which did not expired
- fix clean_obsolete_dirs which cleaned more than it should (not enabled by default)
10.0.15 7.8.2014
================
- fix zombie calls when sniffer is loosing packets. Add new sipwithoutrtptimeout so rtptimeout can be keep at reasonable values (300). See new config/voipmonitor.conf comments.
- close call if there is no 18X or 2XX responses
- fix search country by phone prefix
- remove useless debug message "prevent recursion of connect to db"
- fix dtmf reorder when storing dtmf to database
- fix http issue with 100-continue before 200-OK with eq ack at end stream
- fix false high PDV in case there are DTMF frames
- fix IP capture rules which matches rules according to best fit based on higher network mask. for example 192.168.0.1/32 rule will match last if there is also rule 192.168.0.0/24.
10.0.14 5.8.2014
================
- do not report packet loss if RTP mark bit it set
10.0.13 5.8.2014
================
- fix: if RTP stream starts with CNG frame and seq number is > 0 it detected huge false packet loss and astronomic avg jitter.
10.0.12 5.8.2014
================
- fix MOS score in case there is large packet loss gap - the MOS did not reflected it at all
- fix last sip response code is always 0 if the call was canceled before the ring.
10.0.11 31.7.2014
=================
- fixes fixed jitterbuffer implementation - all versions. Certain packet loss patterns caused high MOS score.
10.0.10 30.7.2014
=================
- fix condition for ignore local ip in fraud alerts
- add sipports command to manager
- fix set local country for local number in fraud
- log to syslog when removing files in spooldir
- supress warning about ddos confusing message
10.0.9 28.7.2014
================
- automatically enable fraud modules
10.0.8 23.7.2014
================
- fix 10.0.6,10.0.7 release (crashes)
10.0.7 23.7.2014
================
- fix 10.0.6 release (crashes)
10.0.6 22.7.2014
================
- fix SQL queue which stuck when max_allow_packet in mysql is default 1MB (which is by default)
- do not add another IP/port but update the old one taken from SDP on RE-INVITE or other cases with SDP - take the unique session id from SDP header for matching which fixes problem with multiple RTP streams in calls which are not related to the call. Fixes all versions. On high throughput this would lead to less CPU and less I/O.
- fix offline reading pcap files - sql handler was terminated sooner
- fix: run pcapcommand after all packets are written to pcap file
- reset jitterbuffer simulators when receving DTMF to not affect MOS score negatively
- fix audio sync issue for g.711 CNG frames
- add siren7 decode (G.722.1) support
10.0.5 14.7.2014
================
- fix filter in live sniffer
- optimize query for select min id in database backup
- fix typo in configure script
- add plcdisable config option and --plc-disable cmd line for G711
- add information about sniffer upgrade file size to error log
- Fixed install command for libcurl4-openssl-dev
- add update_dstnum_onanswer option
- try to reconstruct both audio if 200 OK is missing (or 18X with SDP)
- support Yes or YES in voipmonitor.conf
10.0.4 30.6.2014
================
- fix high MOS scores for fixed50 and fixed200 MOS scores. Fixed50 acts like 250ms and fixed200 like 400ms. (9.3 regression - broken since 5.5.2014)
10.0.3 27.6.2014
================
- fix mirroring between 32bit and 64bit sniffers (9.3 regression)
10.0.2 27.6.2014
================
- fix mos score for g729 (if enabled in voipmonitor.conf which is not recommended anyway) which was always high even for huge packet / PDV loss
10.0.1 24.6.2014
================
- fix GRE protocol (10.0 regression)
- fix IP defragmentation
- fix crashing when decoding to WAV (10.0 regression)
- redesign timestamp reorder warning (10.0)
- fix SIP SMS hide message capture rules
10.0 19.6.2014
==============
after several months of releasing 9.4 and 10.0RC/BETA versions there is new stable version 10.0 with many enhancements and bug fixes. The most interesting part is transparent pcap file compression and saving packet optimizations which allows to store thousands of simultaneous calls which was not possible before. The sniffer is now able to write pcap files for very high number of calls per second (>6000 / sec). Read the full changelog below:
Optimizations
-------------
- reimplement saving pcap files and add native gzip pcap and graph files which is enabled by default. Zipping is done in threads which dynamically grows. Use open/write sys calls instead of libc fwrite/fopen/fclose to allow store files at >6000 calls/second rate (impossible with libc fwrite/fclose functions). Any writes to file are asynchronous by default using different queue which will not block processing packets. New parameters in config/voipmonitor.conf - pcap_dump_bufflength, pcap_dump_zip, pcap_dump_writethreads, pcap_dump_writethreads_max, pcap_dump_asyncwrite, pcap_dump_asyncbuffer. Those optimizations does not require cachedir for very high I/O loads now.
- properly dequeue remote sensors by the timestamp to prevent race conditions or if sniffing on interface = eth1,eth2
- Optimize SIP parser which is now much faster
- do not try to parse callid on UDP packet if it cannot be SIP (can save some CPU)
- save few kB for each jitterbuffer channel (3 per RTP stream) - removed unused structures
- release calls from memory sooner and do not wait on rtptimeout for some cases
- end calls sooner than the rtptimeout if there is no further communication after 401 SIP response
- be more tolerant in jitterbuffer simulater to burst RTP packets - add 200ms more so the MOS is not negatively affected
- if mysql procedure fails run repair table mysql.proc which is the most common source of the issue
- implement new mysql fine tunning options - mysqlstore_concat_limit and mysqlstore_concat* - see voipmonitor.conf for more details
- modify packetbuffer log drop statistics (heapoverrun and drop packets on interace)
- repair table mysql.proc at every sniffer start (it is broken after hard restart very often...)
- shorten register sip message timeout from 30 to 5 seconds (helps to manage RAM when there is REGISTER flood) and add option sip-register-timeout so it can be set to custom seconds
- implement autocleaning to prevent 100% disk space usage. in case the space is below 1% and below 5GB (which is default threshold) reindexfiles procedure will be executed and cleaning will be restarted. In case this will not help the new maxpoolsize will be set to size of current spool directory and will keep free space MIN(1% freespace, 5GB). Default is autoclean enabled, 1% free space or 5GB free space. New configuration options: autocleanspool = yes, autocleanspoolminpercent = 1, autocleanmingb = 5
- if filesindex directory is missing at all or there are no data past 12 hours reindexfiles will be initiated on sniffer start
- use old partition schema even if mysql is upgraded from 5.1 to 5.6 - it autodetects old partitions.
Bugfixes
--------
- SIP 200 packet was never saved if it was not BYE or INVITE - all 200 to CANCEL were missing. (probably all versions)
- if RTP src/dst port was the same as sipports the RTP was ignored (fixes all versions)
- do not end call prematurily on 481 transaction does not exist
- fix beeping in silence sequences for G711 ALAW/ULAW (all versions)
- fix g729 garbled audio for annex B / cng
- fix ignoring RTP if the first RTP is TELEVENT (flash/dtmf etc) leading to silence in audio recording
- fix if device wants to unregister (send REGISTER expires=0) and server replies 200OK expires>0 the sniffer did not deregister it from active calls
- fix reassembling TCP which would lead to crash
- fix bug causing deadlock on 32bit
- fix race condition leading to crash in get sniffer status via manager
- fix crash in case the call is one-way SIP and RTP frames are held in heap due to I/O
- do not end sniffer when sending mirroring traffic to receiver and the connection resets - properly reconnect once the receiver is up again.
- fix pcap files when sending traffic from different interfaces (TUN / ETH) to one receiver
- fix indication in CDR when packet loss in kernel / heap is detected so in the GUI CDR shows the loss might be not real
- ensure that the left channel of the was is always caller
- take size of files from OS sys calls and do not rely on pcap caplen which fixes bug in cleaning which thinkgs there is much more space consumed
- fix audio if caller/called changes codec from one sample rate to another
- fix low MOS score in case call changes packetization from 10 to 20 and back to 10 which is pretty rare but it can happen
- fix SIP TCP reassemble ignoring some TCP messages in SIP (all versions)
- fix SIP TCP reassemble if the first SIP packet does not contain Call-ID it was not tracked at all.
- fix SIP TCP reassemble duplicate TCP packet - last fragment was stored twice
- fix racecondition when storing multiple calls from file - some of calls might not be saved due to preliminary end of the program not waiting to finish all queues
- fix crash if SIP type MESSAGE has larger content-lenght than the whole message.
- fix skinny skip flag
- resolve mysql host only in main thread which fixes issues on some systems which crashes when resolv is in multiple threads. The static binary is now compatible with all centos versions again.
- fix parsing DSCP bit
- fix race condition leading to crash when saving to wav is enabled
- fix regression for all versions >9.1 when (and only for) G729 RTP streams were not scored by MOS at all and although there were loss or delays the MOS was always 4.5.
- fix typo in voipmonitor.conf - onowaytimeout -> onewaytimeout
- fix small memory leak in case there is T38 in SDP (all versions)
- fix redundant reconnect to db - flush previous query before try mysql_ping (problem since 9.1)
- fix skinny racecondition crash and add support for more version headers for some messages
- fix skinny called/calling swapped
New features
------------
- add support for capture rules by domain
- parse multiple SIP messages carried in one TCP packet
- add call per second and sip packets per second to the syslog info line
- if the call timeouts due to absolute timeout mark this in cdr.bye - 103 so the GUI can alert / see it
- support reading compressed pcap files (-r)
- add G722.1 and AMR codec support
- add option only_cdr_next for save cdr only to cdr_next table
- enable merging calls based on matching call-id. The idea is that the second leg with different call-id has in the first INVITE special header which has the call-id of the parent SIP call. The configuration option specifies name of the header.
- add syslog messages when upgrading sniffer
- add connect_duration to listcalls manager command
- add new configuraiton option absolute_timeout (default set to 4 * 3600s) which ends call longer than 4 hours by default.
- add new configuration option destroy_call_at_bye which ends call after 20 minutes (default 20*60) no matter how many RTP/SIP packets arrives after the BYE.
- parse custom headers also for SIP 2XX methods
- replace wget with libcurl when doing autoupgrades from the manager
- implement saving audio in ogg / stereo / various sample rate (new config saveaudio_stereo default is stereo now). Add new option ogg_quality so user can control quality of the ogg file
- allow RTP stream to be assigned to two independent calls with different call-id.
- store RTP mark bit into .graph file so the GUI will show where mark bits are set
- implement new cdr.a_last_rtp_from_end and cdr.b_last_rtp_from_end which stores number of seconds from last RTP packet to the end of call. This will be used to find calls where any of side ends RTP stream sooner. Such situation is not yet detectable without knowing those parameters.
- add support for other GRE encapsulation types (for IP protocol)
- when mirroring packets send id_sensor and DLT number on connect so configuration option mirror_bind_dlt is not needed anymore and allows to set id_sensor for each sender. It is not backward compatible with older versions thus the receiver must be >= 9.4 beta1
- add time shift information to syslog if detect bad packet order
- sip_send_ip = 127.0.0.1 (or 127.0.0.1:12345) sip_send_port = 12345 will send text SIP messages to TCP socket
- implement skip flag also for cisco skinny protocol
- sccp skinny: record RTP stream for some cases where signalization does not contain conference number = call reference which fixes ampty RTP
- use only first detect value for custom headers or last if set configuration value custom_headers_last_value
9.3 10.2.2014
=============
- fix crashing when decoding audio in certain cases and also when doing realtime listening
- do not connect to database if nocdr is set (-c)
- fix crash if non ascii is put into telnum filter
- fix freebsd compile
- implement storing rtp port to cdr_rtp database (cdr_rtpport option must be set to yes and it will upgrade database)
9.2 31.1.2014
=============
- do not put CNG frames into audio decoders which causes garbled g729 stream and causes low MOS score even thought there is no packet loss at all
- use separate MySQL connections for live packet sniffer and for storing REGISTER messages which adds parallelization and speedups INSERTS
- fix ipaccount for GRE protocol
- add --no-cache parameter to wget in download upgrade
- fix issue with double semicolon in sql query
- add support for DLT_IEEE802_11_RADIO reading pcap files and sniffing on raw wifi
9.1 29.1.2014
=============
- Fix MySQL deadlock in case the MySQL connection timeouts. This bug started since 8.4 - 9.0 verisons.
- Fix storing to MySQL if file is read from file (bug introduced in 8.4 and 9.0 versions) and do not fork when reading from file
- Optimize failed registrations which caused mysql overload in case of register flood
- fix configuration issue in case the datbase is partitioned and cdr_partition was not set
- fix MySQL indefinit reconnect in case the sniffer is stopping or starting which causes hang in linux bootup / shutdown process.
- fix undefined codec in case of CNG (comfort noise) frames.
- add new option saveaudio_reversestereo which reverts left and right channels
- add new manager command pcapstat which prints ringbuffer packet drop counter
- fix issue in interface option when list of interfaces contains spaces like: interface = eth0, eth1
- support mysql 5.0
9.0 17.1.2014
=============
Version 9.0 is formerly 8.4RC27 and this changelog sumarizes all changes since 8.3 (all features from 8.4)
Optimizations
-------------
- do not create RTP pcap file for every sip call but only if there is actually RTP which saves a lot of I/O
- take min MOS score from fixed2 jitterbuffer simulator by default which eliminates low mos score due to f1 and adapt sensitivness
- do not flood log with "to much INVITEs in this call" show the message only once increase MAX_IP_PER_CALL from 30 to 40
- Close SIP and SIP+RTP dump files ASAP to save some number of opened files and memory (Rob Gagnom)
- optimize thread0 packetbuffer size
- disable sendfile flood if not present in old kernel
- modify install script which will ask if the user wants to overwrite configuration file
- optimize SIP MESSAGE sql storing which accidentally access database for each SIP MESSAGE which leds in queueing sip message types
- do not create partitions every hour but every 12 hours
- add option disable_partition_operations which will not create partitions (in multisniffer environment all other sensors should have disabled partition creation which is redundant)
- enhance drop old partitions
- check if another instance of voipmonitor is running on background based on name of the configuration file (so more instances can still run on background with different configs)
- in case call is closed with 403 response shorten close window to 5 seconds so that call will not timeout on rtp_timeout (which is 5 min by default).
- change mtune and march to native in Makefile
- calls which end with 4XX or 5XX was not released from memory immediatelly and was waiting for rtptimeout if sip source port differs from the first INVITE.
Bugfixes
--------
- do not parse SIP data after content-lenght = X. This fix was already done before but only if verbose > 1