-
-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathREADME
1881 lines (1355 loc) · 68.5 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
NAME
Netdisco 1.3.1 - README
AUTHOR
Netdisco is maintained by a team of Open Source developers headed by
Eric Miller, Bill Fenner, Oliver Gorwits, Jeroen van Ingen and Max
Baker.
DESCRIPTION
Netdisco is an Open Source web-based network management tool.
Designed for moderate to large networks, configuration information and
connection data for network devices are retrieved and set by SNMP. With
Netdisco you can locate the switch port of an end-user system by IP or
MAC address. Data is stored using a SQL database for scalability and
speed.
Cisco Discovery Protocol (CDP), Foundry Discovery Protocol (FDP), Link
Layer Discovery Protocol (LLDP), and SynOptics Network Management
Protocol (SONMP) optionally provide automatic discovery of the network
topology.
The network is inventoried by both device model and operating system
(like IOS). Netdisco uses router ARP tables and L2 switch MAC forwarding
tables to locate nodes on physical ports and track them by their IP
addresses.
For each node, a time stamped history of the ports it has visited and
the IP addresses it has used is maintained. Netdisco gets all its data,
including topology information, with SNMP polls and DNS queries. It does
not use CLI access and has no need for privilege passwords. Security
features include a wire-side Wireless Access Point (AP) locator.
Netdisco was created at the University of California, Santa Cruz (UCSC),
Networking and Technology Services (NTS) department. UCSC continues to
support the development of Netdisco by providing development servers and
beer. The Netdisco project is hosted by Source Forge.
See <http://www.netdisco.org>
FEATURES
Switch Ports
From the web interface devices connected to switch and router ports are
listed by MAC address. A history of which switch ports a MAC address has
been seen at is kept. With a click the you can browse a network device
connected to an uplink port. With another click you can disable or
enable the switch port, logging the reason, user and date.
* Central location to disable/enable switch ports.
Network administrators can disable and enable ports without having
to know enable or privilege passwords. Reasons for switching on/off
ports are logged for end-of-the-year auditing and reporting. Non-IOS
savvy managers can control port access from a familiar browser
interface. This feature was designed with a University Residential
Networks (ResNet) in mind.
Only users you specify in Netdisco will have access to switch off a
port. Netdisco will also not allow people to switch off uplink ports
by accident.
* Supports Cisco VLAN Community String indexing ("public@101")
* MAC Address to switch port resolution
* IP Address to switch port resolution
* Find Switch Ports with multiple nodes attached
* Find nodes using multiple IP addresses
* Find nodes by vendor (using MAC address OUI)
Easy Administration
* Controllable through Web Interface or Command Line Interface (CLI)
* Database store for scalability and speed (Postgresql)
* Easily extendible to new network devices
* User system to restrict access and features
Network Administration and Security
* Automatic inventory and search of network hardware
* Administratively enable/disable switch ports from web interface with
logging
* Duplex Mismatch Finder
* Find Wireless Access Points (APs) from wired-side of network
* Layer Two Traceroute
Reporting
* Graphing of network topology. Clickable image-map of devices. Link
speed shown
* Statistics for number of actual nodes connected to network
Inventory of Network Devices
* by Operating System (IOS,CatOS,HP...)
* by Model, Vendor, OSI Layer, DNS Name
* Find device ports that are blocking (via Spanning Tree Protocol)
* Find devices using IP's w/out DNS entries
SUPPORTED DEVICES
Netdisco supports any Network device that talks SNMP and has basic
information available through MIB-II (RFC 1213). Additional
vendor-specific information is available for a number of devices, but
especially for Cisco, Extreme, Foundry, HP, and Nortel/Bay devices.
Device support is handled through "SNMP::Info" -- a Perl module that is
an integral part of Netdisco that handles device-specific code. See the
"Device Matrix" at <http://snmp-info.sourceforge.net> for a list of
devices that have been tested against Netdisco. SNMP::Info can be
extended for new families of devices relatively easily with a little
Perl knowledge.
SUPPORT
Please use the "netdisco-users" mailing list for all problems and
comments.
<http://lists.sourceforge.net/lists/listinfo/netdisco-users>
In case of bugs, please use the Bug interface from SourceForge page at:
<http://sourceforge.net/projects/netdisco>
GLOSSARY
Device
Any device connected to the network that contributes to the physical
topology. Devices need to be accessible via SNMP. A device usually
has multiple interfaces (ports) and can have multiple IP addresses.
Node
A node is anything connected to a device. Nodes are uniquely
identified by their MAC addresses. A node may or may not have IP
addresses associated with it.
Macsuck
Technical Answer : The process in Netdisco that goes out to all
Layer-2 devices and gets the Forwarding Tables / CAM Tables. Each
row in the table maps a MAC address to a switch port. This process
is what makes devices show up on switch ports.
Netdisco will attempt to detect uplink ports in case you are missing
topology data during macsuck. Check the logs of the macsuck /
macwalk for notifications of detected uplink ports, and add that
data to your netdisco-topology.txt.
Fun Answer - From Douglas M. McKeown :
"This is where you go to a switch (Layer 2) and find all the MAC (or
Ethernet Hardware) addresses which this device is connected to. So
you plug your Dell into your HP Switch and that HP Switch is
uplinked to your Core switch (not using the word router here. we're
talking simple, physical network connections, sort of like
electrical wires.) Well your Dell has a MAC address of let's say "A"
and amazingly, your HP switch has a MAC address of "B" and your Core
switch has an address of "1". Well if you Macsuck your Core switch,
it doesn't have your Dell connected to it, but it does have "B"
which is another switch. So you Macsuck "B" and it has MAC addresses
for 1, B and A! You don't really Macsuck an end device (your Dell).
So what do we know?
- Core (1) knows about HP Switch "B".
- HP Switch "B" knows about Core (1) and Dell "A".
- Dell "A" knows about HP Switch "B".
Does "1" know about "A" ? If it's a router it does. Otherwise it
asks who has "A" and switch "B" says, I know! So 1 goes to B which
goes to A.
Got it?"
Arpnip
The process in Netdisco that goes out to every Layer-3 device and
gets its ARP cache. Each entry in the ARP Cache maps a MAC address
to an IP address.
This process is what lets Netdisco map an Ethernet address to an IP
address. Combined with the Macsuck process, Netdisco can ultimately
resolve an IP address to a switch port.
If you have a small network that only has layer-2 devices on it, and
you use a Linux or BSD box as your router, you will need to install
net-snmp on the machine, and then have netdisco discover that
machine. Otherwise you will not be able to resolve a MAC address to
an IP address.
CDP / FDP / LLDP / SONMP
Having topology information is crucial for Netdisco to function. So.
if you network does not support one of the above Layer2 discover
protocols, you must put the information in the netdisco-topology.txt
file.
See "Topology Information" in this file.
From Douglas McKeown :
"CDP is the Cisco Discovery Protocol. Sort of an add-on for when
switches talk to switches about who's connected to whom. CDP quickly
tells other switches that it has switches connected. Netdisco really
likes CDP a lot for mapping out the network and automatically
discovering the topology. If your devices don't use CDP, then you
need to work with the netdisco-topology.txt file to create a layout
of your network."
Note that LLDP (IEEE Standard), FDP (Foundry), and SONMP
(Nortel/Bay) are supported, and anywhere you see CDP you can assume
we mean LLDP, FDP, and SONMP too.
Security Warning
WARNING! There is a potential community string exposure when
Netdisco is auto-discovering network equipment ("netdisco" -r).
If a malicious host were to implement CDP and Netdisco were to
discover that host, Netdisco would send all read-only community
strings to that device in an attempt to add it to the topology.
There are two main ways to avoid this exposure:
List addresses of valid devices
Use the discover_only and/or discover_no configuration
keywords to control what IP addresses netdisco will be
permitted to visit. "discover_only" is inclusive, and
"discover_no" is exclusive; it's recommended to use
"discover_only" if feasible.
When using this method, check the backend log for devices
visible via CDP but not via SNMP. These may point out the
need to expand the range that is discoverable, or may be
instances of this class of attack.
Additionaly "discover_no_type" can be used to prevent
netdisco from visiting certain devices based on the
device_type returned by CDP.
Disable CDP and other discovery protocols
This solution involves disabling CDP and other discovery
protocols from your user-connection ports, and leaving it on
on inter-device ports. Unfortunately, in some
configurations, user-connection ports are inter-device
ports, e.g., especially when you want to keep the ability to
easily add a phone to a port that didn't have one
previously.
Sample "IOS" Code for above:
interface range fastethernet1/1-32
no cdp enable
Make sure you don't disable CDP on any ports that are
connected to other pieces of infrastructure. Also make sure
you don't use the global command "no cdp run", since that
will disable CDP entirely.
INSTALL
See the INSTALL document for instructions and requirements to install
Netdisco.
USING NETDISCO
Components
Netdisco has three components :
1. Back-end
The back-end talks to devices via SNMP. Contained in the back-end is
the logic to create the topology, collect statistics and generate
graphs.
Most of the back-end is controlled by cron jobs.
A background daemon is put resident to run maintenance tasks
collected from the front-end. This keeps these sometimes memory
intensive tasks and code out of the httpd processes.
2. Database
Netdisco uses PostgreSQL to store all its information. Careful
abstraction of the database calls means that Netdisco can be ported
to another SQL platform easily. Hooks to use other databases are
present.
3. Front-end
The front-end operates on stored data only. This abstraction is both
for speed and security.
Some front-end administration tasks are put in a queue in the
database that a daemon running from the back-end picks up and
processes.
The number of people using Netdisco can scale with the web server
capacity, and will create no extra load on the devices.
Command-Line Options
-b || --batchmode
Batch Mode. Redirect output to log file. Log file directory set in
configuration file under datadir.
-C || --configfile file
Set Config file. Default is netdisco.conf.
-D || --debug
DEBUG. Sends copious information to STDOUT
-L || --nologging
No Log. This will not add entries to the log table.
-n || --nodestoo
Delete Nodes. Used with --expiredevice only.
-N || --newonly
New Only. On a network discovery -r, only discover found devices
that aren't in the database.
-P || --port port
Port. Specify Port for removal of nodes -e.
-S || --dumpsql
Debug. carp() SQL commands. Sets $netdisco::SQLCARP to 1.
-V || --archive
archiVe nodes. Used with -e only.
Command-Line Commands
-a || --arpwalk
Arp Walk. ArpNip each device that has Layer 3 capabilities.
-A || --arpnip device
ArpNip. ArpNip's a single device. See ArpNipper in Design.
Devices listed in "arpnip_no" in the config file are excluded. If
there is a "arpnip_only" entry in the config file, devices not
listed are excluded. See the entry below.
-B || --backup
Backup and Nightly Maintenance.
Removes
Devices and nodes that are old using the "expire_*" config file
directives (see below).
Creates
Archive data files for node,node_ip,device, and device_ip
tables.
Calls
Database cleanup routines (-K) as well.
Exports
NMIS config file if nmis_dump is set.
This routine should be run nightly.
For a full backup run sql/pg --back to backup the whole database.
-d || --discover device
Discover Device.
IP addresses and subnets listed in "discover_no" in the config file
are excluded. If there is a "discover_only" entry in the config
file, IP addresses and subnets not listed are excluded. See the
entry below.
-e || --expirenodes device
Expire Nodes for given device. Use -V to archiVe instead of delete.
Specify a port with -P to delete or archive nodes on a per port
basis.
--expire-nodes-subnet subnet
Finds all devices in given subnet and runs expire nodes on each.
Will display devices effected and then ask for confirmation.
Subnet is specified in CIDR format :
192.168.0.0/24
-E || --expiredevice device
Delete a device. Use -n to delete nodes as well.
-F || --discoverfile file
Discover Device from given File. Used to restore backed up info from
-B, and to discover devices that are not available through topology
information. Use -T to only import Topology Information.
-g || --graph
Graph. Creates graph using GraphViz. Can create image output
(png,gif) or vector output (svg).
NOTE: You can safely ignore all warnings about "size too small for
label".
Make sure you have a relatively new version of GraphViz. You need a
newer version of GraphViz if you get an error similar to:
Creating CMAP : /usr/local/netdisco/html/netmap.map
warning, language cmap not recognized, use one of: ps hpgl pcl mif...
-h || --help
Prints out command line usage.
-i || --changeip old_ip new_ip
Change IP address of device. Creates new entry, removes old one and
moves nodes over to the new one.
-I || --expireips
Expire IP Addresses from node_ip table. This will delete entries
from the node_ip table that are not matching entries (MAC Addresses)
found in the node or device_port tables.
-k || --cleanalias
alias klean-up. DANGEROUS. Deletes from the device table any IP
address that is found as an alias in the alias table.
-K || --cleannodes
Database Node Klean-up. Permanently deletes nodes matching:
1. MAC Addresses that are Switch Port Addresses
2. MAC Addresses that are listed on non-existent ports
3. MAC Addresses that exist on ports with topology information
(uplink ports)
-m || --macwalk
Mac Suck each device in the database that has Layer 2 capabilities.
-M || --macsuck device
Mac Suck given device only.
Devices listed in "macsuck_no" in the config file are excluded. If
there is a "macsuck_only" entry in the config file, devices not
listed are excluded. See the entry below.
-O || --oui
Import OUI information from oui.txt
-p || --daemon [start,stop,status,restart]
Control the Admin Daemon. Takes arguments
(start,stop,status,restart).
-r || --discoverall root_device_list
Walk the network with the given (comma-seperated) root(s). Use -N to
discover new devices only. Given root devices will always be
discovered.
-R || --refresh
Refresh devices. Will run a discover (-d) for each device in the
database.
-T || --topofile
Import Topology Data. Will import manual topology data stored in
file specified by configuration option topofile . Use -F to specify
a different file from the command line.
It is not necessary to do this after every change. This is only a
convenience switch.
-u || --user [user] [password] [port_control?] [admin?] ["full name"]
Add or Change a User. Supply all four arguments (user pw
port_control admin) for command-line control, or supply less for
interactive prompts.
It's better to use interactive prompts so that the password doesn't
get stored in your shell history file and exported to the process
table.
-v || --version
Features
Admin Daemon
The admin daemon is a copy of "netdisco" that runs in the
background. From the web "Admin Panel", jobs are put in a queue in
the database. The daemon picks up these jobs and executes them from
the back-end as user "netdisco". The daemon is restarted daily in a
cron job, or can be manually started as root :
su - netdisco -c "/usr/local/netdisco -p restart"
Port Info / Jack Search
This feature integrates Netdisco with other databases that have port
info.
Port Info was designed around data coming out of a Pinnacles
database at UCSC, and might prove to be site-specific. However, see
"port_info.html" for a good example of how to access other databases
using the "netdisco.pm" SQL routines.
Enable this feature by setting "port_info" to true in
"netdisco.conf"
Port Control
Port Control allows a user of Netdisco to administratively turn a
port on or off.
To do this the back-end requires a read-write community string for
the device in question. The admin daemon must also be enabled.
Netdisco keeps a log for each port holding information about why a
port was turned on or off.
A reason for turning switch the port is chosen from a list to
provide future audits of admin activity. The user and IP address of
the request are stored. To change the default reasons, modify the
%PORT_CONTROL_REASONS hash in "netdisco.pm"
Optionally if the "portctl_email" setting is set in "netdisco.conf",
an e-mail is sent out with a notification of the switching. Locally
at UCSC that e-mail is sent to an administrative mailing list.
To turn this feature off uncheck the "Port Control" checkbox from
all users in the "Admin Panel".
By default Netdisco will be allowed to shut off
- Switch Ports
- IP Phones
- Router Ports that are NOT uplinks
By setting certain config file directives you can allow Netdisco to
shutoff uplink ports and VLAN interfaces. But this is REALLY NOT
RECOMMENDED. See below for the required commands.
Web Console
The Web Console allows netdisco to front-end the web interface of a
switch or router. Traffic can then be routed over https, through
Netdisco's web server. An additional security layer is added by
requiring the user to be logged into Netdisco. The normal security
measures used by the device's web server are still active.
The Web console is a reverse proxy that runs on Apache. You must
enable it in "netdisco_apache.conf" and "netdisco_apache_dir.conf".
The add devices and models to the configuration lines
"web_console_vendors" and "web_console_models" in "netdisco.conf".
Netdisco Maintenance
Refreshing a device
To refresh or discover a device and its ports, use the -d command:
netdisco -d mydevice
Importing Topology Information
It is not necessary to import the topology information after
changing netdisco-topology.txt. You should however restart the admin
daemon. The topology text file is re-parsed each time you run
netdisco.
As a convenience you can use the topology file to quickly seed
Netdisco with devices. To import all the topology information at
once make sure the topology filename is set in "netdisco.conf" and
use the -T command:
netdisco -T
Aborting a process of Netdisco
Hit Ctrl-C if you are running a netdisco process, or send the job
the INT signal. The job can cleanup after itself, write out its
stats and log entries.
kill -INT jobpid
There is currently no way to stop a job inside the Admin daemon.
Send the daemon an INT signal and it will terminate after its
current job has completed.
Changing the IP Address of a Device
If a device is being replaced with a different device and a
different IP, see "Deleting a Device" below.
netdisco -i old-ip-address new-ip-address
Changing the IP address of a device will:
1. Discover the new device
2. Remove Old Device Entry, port, and aliases
3. Move the old nodes to the new device.
Auto-Deleting Old Data From the Database
In order for Netdisco to be self-maintaining data has to be taken
out of the database as well as put in. The following config file
directives are used to auto-prune stuff from the database :
expire_devices
expire_nodes
expire_nodes_archive
See each item's entry in the "Config File" Section below for more
details.
The expire data routines are called from the -B/Backup routine,
which should be running nightly via cron.
Deleting a Device
To delete a device use the -E command followed by the device name or
IP. Set -n to delete all the nodes seen on that device as well
This is rather permanent. Make sure you run -Backup before you do
this.
Deleting Nodes
Nodes consist of two components -- the switch port to MAC address
mapping in the "node" table, and the MAC address to IP mapping in
the "node_ip" table.
To remove nodes from a switch, use the Admin Panel on the web side
and choose either "Delete Nodes" or "Archive Nodes". Archiving nodes
will set the archive bit so that the data will be available, but not
always showing. You can also delete nodes from the command line
using the -e command with or without the -V flag.
Database Cleanup -K will delete nodes that seem to be extraneous.
See -K for more details.
Once you have cleared out nodes from a switch, then run -I to remove
unused node to IP mappings.
This is rather permanent. Make sure you run -Backup before you do
this.
Adding / Changing Users
The easiest way to add a user is to use the "Add User" form in the
Admin Panel. After first installing Netdisco you need to add an
admin user by running -u.
Migrating the Users table to a new host
If you are moving your Netdisco install over to another machine and
you want to keep your users table, here is the process :
source$ pg_dump -a -d -U netdisco -t users netdisco > user_dump.sql
source$ scp user_dump.sql dest:
dest$ cd /usr/local/netdisco/sql
dest$ ./pg /path/to/user_dump.sql
Localhost (127.0.0.1) is showing up on CDP Links
See "How the Switch Selects the IP Address To Include in Outbound
CDP Packets" in
ftp://ftp.hp.com/pub/networking/software/59692375_e1.pdf
Device Model comes up as 'Products.'
The device is probably newer than your Cisco MIBs. Redownload
<ftp://ftp.cisco.com/pub/mibs/v2/v2.tar.gz> and install these newest
mibs into /usr/local/share/snmp/mibs.
Things are getting Really slow
For some reason over here at UCSC, things get real slow in Postgres
after a while. Even though we are doing frequent VACUUM's on all the
data, it seems to be dragging down after a while.
This turns out to be an INDEX bloat problem on Postgres versions
less than 7.4. Recently doing this on a Postgres 7.3 install changed
the amount of space that Netdisco's database was using from 16G to
400M !!!
In order to fix this we do a "VACUUM FULL ANALYZE VERBOSE" and
"REINDEX" from "pg". This command locks each table before it does
the VACUUM, and therefore can be more thorough. It's a good idea to
take netdisco down temporarily while you do this. I do this about
once a month, or when I notice it dragging down. Use "Netdisco
Statistics" as a good metric of things slowing down. This may get
fixed with changes in VACUUM in Postgres 7.4 and above.
Procedure for doing a vacuum full (as root):
1. Shutdown the admin daemon
/usr/local/netdisco/bin/netdisco_daemon stop
2. Clear the cron tab for user netdisco
crontab -u netdisco -r
3. Comment out the netdisco config file Includes in httpd.conf
4. Restart Apache
/usr/local/apache/bin/apachectl graceful
5. Check to see if any netdisco jobs are running and wait for them
or kill them
ps
killall netdisco
6. Run REINDEX and VACUUM FULL
Before:
df -h
/usr/local/netdisco/sql/pg
# before comparison :
select relname, relpages from pg_class order by relpages desc;
REINDEX TABLE node;
REINDEX TABLE node_ip;
REINDEX TABLE device;
REINDEX TABLE device_port;
REINDEX TABLE device_port_log;
VACUUM FULL ANALYZE VERBOSE;
# after comparison :
select relname, relpages from pg_class order by relpages desc;
\q
After:
df -h
7. Restart Postgres (just for fun)
/usr/local/etc/rc.d/010.pgsql restart
OR
/etc/rc.d/init.d/pgsql restart
OR
/etc/rc.d/pgsql restart
8. Uncomment lines in httpd.conf
9. Restart Apache
/usr/local/apache/bin/apachectl graceful
10. Reload crontab for user netdisco
crontab -u netdisco /usr/local/netdisco/netdisco.crontab
11. Restart Admin Daemon
/usr/local/netdisco/bin/netdisco_daemon start
Clearing the Admin Queue
If your admin queue is just getting too long and you want to clear
it you can do it by just dropping the table and readding it.
cd sql
./pg admin.sql
Topology Information
Topology information is crucial to Netdisco's performance. It allows the
application to know which ports are uplink ports and which have
connected nodes. Ports that are uplink ports that are not marked so in
Netdisco will appear to steal MAC address entries from their rightful
ports. So it is critical to use the topology file and CDP/FDP/SONMP to
maintain a topology.
Autodetection of uplink ports
During macsuck if Netdisco finds the MAC address of a known device or
switch port, then that port is marked as an uplink. Nodes will not
collect at these switch ports, and a warning message will be printed.
Check the logs of your macsuck and macwalk jobs in order to find and
correct autodetected uplink ports. Add these ports to your
netdisco-topology.txt file.
Manual Topology Information
Netdisco will auto-discover the layer-two topology of a network using
CDP. However, many networks have parts of the topology that are not
covered by CDP.
Use the manual topology file "netdisco-topology.txt" to supply the
layout of the network if your network has devices that don't talk CDP or
misreport information.
The manual topology file only requires one side of the data to be
entered. Both directions of a link will be forced to the given data if
one side is listed.
File Format
The format of the manual topology consists of four types of lines:
#comment
Comments are delimited with a "#" They can happen on any line.
Escape as "\#" if you need to use a literal pound sign.
routername
Any line that does not start with "link:" or "alias:" is assumed to
be a the DNS name or IP address of a network device.
link:
Lines that start with "link:" connect two devices together. The
format is
link:outgoing port,destination device,Destination port
The outgoing port belongs to the device listed above the "link:"
line.
The Destination Device and Port tell Netdisco who is on the other
end of this link. The device can be a DNS name or an IP Address.
NOTE: The port names must match exactly how Netdisco sees it. Go to
the device and check it out. You might think of it as "port 1" but
Netdisco might think of it as "RMONPort26onunit1".
alias:
Not implemented for output. The backup file will have these lines
just for informations' sake. Alias IPs on a device are found during
discovery.
Many network devices like routers have multiple IP addresses
assigned to them. If the device cannot or does not supply this
information to Netdisco in a standard way, you can add IP addresses
used here.
White space in the file (except for line breaks) is ignored. Tabbing
over before "line:" lines makes it easier to read, but is not required.
File Uses
Some reasons the manual topology file is used:
1. Man in the Middle
Let's say you have two CDP speaking devices with a non-CDP speaking
device in between them
[Cisco] ---> [Bay] ---> [HP]
The Cisco and HP devices (CDP speakers) find each other and the Bay
device never appears. You would then have to add these lines to the
topology file:
ciscoswitch.my.company
link:EtherNet0/1,bayswitch.my.company,25
bayswitch.my.company
link:26,hpswitch.my.company,J3
This tells Netdisco that port "Ethernet0/1" on "ciscoswitch" is
connected to Port 25 on "bayswitch". Then in turn Port 26 on
"bayswitch" is connected to port "J3" on "hpswitch".
A note about devices that are *CDP Aware* and that implement CDP:
*CDP Aware* devices are devices that probably do not speak CDP
(probably for legal reasons) but that are smart enough not to
forward CDP packets. Cisco devices that have CDP disabled are
usually still *CDP Aware* and will not forward the packets.
Man-in-the-middle situations occur when the device both does not
speak CDP and is not *CDP Aware*.
2. Isolated Network Segment
If you have a segment of your network that is not connected
directly, or connected through a non physical link like a VPN, then
you might fudge an entry to connect that segment of the network with
the main one.
3. Attach a non-CDP speaking device
Anywhere a device that does not supply topology information is
connected to the network, an entry must be added in the manual
topology file.
Cron Jobs
Netdisco is controlled via cron jobs. Jobs are run as user "netdisco".
Multiple jobs can be run at once.
The default jobs are :
* MacSuck - Every 2 hours MacSuck all the devices in the database.
* ArpNip - Every 2 hours ArpNip all the devices in the database.
(Offset from Macsuck by 1 hours)
* Refresh Devices - Once a day refresh device information.
* Backup - Once a day backup information.
* Graph - Once a day re-create the graph.
* Walk Network - Once a week (Wed @ 14:00) try and discover new
devices.
* Restart Admin Daemon - Once a day just for good measure.
Config File
The settings in "netdisco.conf" are used both in the back-end and the
front-end.
When you make a change in the config file that is used in the web front
end, you must reload apache. The config information is shared between
processes for speed and memory performance.
su - -c "/usr/local/apache/bin/apachectl restart"
Multiple config files can be used in the back-end by calling Netdisco
with the "-C" option:
netdisco -C myotherfile.conf
General Items
domain
STRING. Trimmed from all DNS names viewed. Leave blank to show all
domain names. Add a dot in front of your value :
.ucsc.edu
home
PATH. Full path to where netdisco lives. Is the root path for all
other files and paths.
customer
STRING. Name added to page titles and heading.
customericon
STRING. URL,width,height - replaces discoball icon with custom logo.
Database Maintenance
New in version 0.93 these directives are included to help make Netdisco
more self-maintaining.
Setting these will result in permanent data removal.
expire_devices
DAYS. Devices that have not been refreshed in this number of days
will be removed. All nodes connected to this device will be removed
as well.
expire_nodes
DAYS. Nodes that have not been refreshed in this number of days will
be removed from the database. Archived and non-archived nodes are
removed. This includes SwitchPort/MAC and MAC/IP mappings.
expire_nodes_archive
DAYS. Archived data for switch-port/MAC and MAC/IP mappings older
than this number of days will be removed.
Back-End Items
arpnip_min_age
SECONDS. Prevent a device from being arpnipped for this number of
seconds after the last succesful run.
arpnip_no
LIST:string. Devices that won't be arpnipped. See "bulkwalk_no" for
syntax. If you have any layer-3 devices that have been discovered by
netdisco but are using proxy-ARP as a way to get to other devices,
place them here. Alternately, if you have many proxy-ARP clients but
one (or a handful of) central device with all of the proper ARP
info, put that in "arpnip_only".
arpnip_only
LIST:string. If present, only arpnip these devices. See
"bulkwalk_no" for syntax.
compress
EXECUTABLE. Full path and command line arguments to the compression
program used in compresslogs
compresslogs
BOOLEAN. Compress log files? See compress entry above.
datadir
PATH. Full or relative path to the directory that backups and logs
will be stored in
discover_min_age
SECONDS. Prevent a device from being refreshed for this number of
seconds after the last succesful run.
discover_no
LIST:string. IP addresses in this list will not be visited during
discovery. See bulkwalk_no for syntax, except that only hostnames,
IP addresses and subnets are valid.
discover_no_type
REGEX:string. Place a pattern here to exclude the discovery of
certain devices based on the CDP device type information. Good for
excluding a whole device class like lightweight access points or IP
phones that have CDP but don't talk SNMP.
discover_only
LIST:string. If present, discovery will be limited to only IP
addresses in this list. If you have a management VLAN, put that
subnet here to avoid discovering user devices. See bulkwalk_no for
syntax, except that only hostnames, IP addresses and subnets are
valid.
ignore_interfaces
LIST:STRING If present, device ports matching any of the items in
this list will be ignored by the discovery process. Note this may
have side effects - connected devices and nodes on those ports will
in turn also not be discovered.