User Tools

Site Tools


extensions:teemip-ip-discovery
Translations of this page:

IP Discovery

name:
IP Discovery
description:
TeemIp solution to discover IP addresses
version:
0.2.2
release:
2019-02-09
TeemIp:
2.3
code:
teemip-ip-discovery / teemip-ip-discovery-collector
state:
pilot
diffusion:
TeemIp wiki
keyword:
discovery, ping, lookup, scan
dependencies:
teemip-ip-mgmt, teemip-ipv6-mgmt, teemip-network-mgmt
download:
https://wiki.teemip.com/extensions/teemip-ip-discovery-0.2.2.zip

TeemIp IP Discovery is a solution that discovers active IPs on networks. While discovery activities are done by remote standalone collectors, discovery parameters are defined in TeemIp and discovery results are fed back and displayed within TeemIp.

Vocabulary: by “TeemIp solution” or “TeemIP”, it should be understood: TeemIp standalone or the iTop solution on top of which TeemIp as a module has been installed.

Revision History

Version Release Date Comments
0.2.2 2019-02-09 Initial production release
0.2.0 2018-09-24 Release candidate
0.1.1 2017-08-11 Beta version 2
0.1.0 2017-06-12 Beta version 1

Features

TeemIp IP Discovery provides a solution to TeemIp administrators to discover or scan their IP networks and to document within TeemIp the results of these discoveries and scans.

The solution is made of 2 components:

  • a discovery engine, called the IP Discovery Collector or the IP Discovery Application, that discovers IPs through pings, DNS reverse lookups and port scanning,
  • a TeemIp extension, called the IP Discovery Extension, that adapts TeemIp's standard data model to store the discovery outputs and enables communication between a TeemIp instance and the different discovery engines deployed on the network.

The IP Discovery collector is a standalone PHP software that relies on 3 different ways to discover its IP environment: ping, IP lookup and port scan, each of them being globally enabled or disabled per subnet. Several instances of that application can be installed on different servers. Should your network be compartmentalized by different firewalls, the different instances will allow you to bypass the restrictions imposed by the firewalls: limited pings or udp / tcp connections, DNS not available… Of course, you may install the collector on the server that hosts TeemIp but this is not necessary.

The IP Discovery extension is installed on a TeemIp instance. It can handle several IP Discovery applications.

IP Discovery Application objects in TeemIp

New TeemIp objects are brought by the IP Discovery Extension. These objects model the remote IP Discovery Applications connected to TeemIp. Each instance of them defines:

  • what discovery method to use (ping and / or IP lookup and / or port scanning),
  • what subnets to discover.

A given subnet can only be discovered by one remote application and the functions to be used for its discovery can be restricted: ie deny ping, IP lookup or scan if these have been enabled globally.

Remote IP Discovery Application

This is a standalone software that extends iTop's collectors principles. It has been built on top of iTop data collector base extension. TeemIp administrators may deploy one or several IP Discovery Collectors on different servers, including the one that hosts TeemIp. Each of these remote applications connects to TeemIp to retrieve:

  • the list of subnets they have to discover,
  • the global discovery method they need to use,
  • the restrictions that each subnet may impose.

At regular interval (defined by the administrator through a CRON command) the remote collector retrieves in TeemIp the list of networks that it has to discover and, for each of them, the list of IPs already registered. It, then, scans the network, computes the delta between what has been discovered and the situation before the scan, stores the results of its findings locally and pushes these results back to TeemIp through a standard synchronization mechanism:

  • New discovered IPs are created,
  • Ping, IP lookup or scan statuses are updated,
  • A few statics on the discovery are documented within each subnet.

Visualizing results in TeemIp

The IP Discovery extension alters the way subnets and IPs are displayed within TeemIp so that IP discovery results are visible to the end user.

Licensing

TeemIp IP Discovery solution is licensed under the terms of the GNU Affero General Public License Version 3 as published by the Free Software Foundation. This gives you legal permission to copy, distribute and/or modify it under certain conditions. Read the ’license.txt’ file in the distribution. TeemIp IP Discovery is provided AS IS with NO WARRANTY OF ANY KIND, INCLUDING THE WARRANTY OF DESIGN, MERCHANTABILITY, AND FITNESS FOR A PARTICULAR PURPOSE.  

The IP Discovery extension is available at the top of this page.

The remote IP Discovery application is accessible through a support subscription that includes consulting and assistance. Such subscription can be acquired at Combodo who you can simply contact here.

The nice binocular icon attached to the IP Discovery Application in TeemIp has been found on IconFinder and Icons Land.

Limitations

Discovery of IPv6 networks is not handled at this stage.

Requirements

On the application's side, the server must be configured to execute PHP files (minimum PHP version 5.3.6, but more recent versions are better) and it must have a web access to the TeemIp server through https. On TeemIp's side, the Discovery extension must be installed so that the application can retrieve its parameters.

Installation

IP Discovery Extension

When adding the extension on an TeemIp standalone or iTop application, just use the standard installation process.

IP Discovery Application

Application TeemIp IP Discovery Collector 0.3.0 can be downloaded here.

Expand the content of the zip archive in a folder of the server that will run the discovery.

Configuration

IP Discovery Extension

No specific configuration is required in TeemIp's configuration file or in IP configs for that extension.

IP Discovery Application

Like every iTop collector, the configuration of the application is based on the params.local.xml file located in the conf directory. It has the following XML content:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Parameters specific to the discovery instance. -->
<parameters>
    <!-- TeemIp Application -->
    <itop_url>https://localhost</itop_url>
    <itop_login>admin</itop_login>
    <itop_password>admin</itop_password>
 
    <!-- Synchronization parameters -->
    <contact_to_notify></contact_to_notify>
    <synchro_user>admin</synchro_user>
    <json_placeholders>
        <!-- IP Discovery Application UUID -->
	<discovery_application_uuid>1122335588</discovery_application_uuid>
        <!-- Name of synchronization data sources -->
	<ipv4_synchro_name>TeemIp IPv4 Discovery</ipv4_synchro_name>
        <subnetv4_synchro_name>TeemIp IPv4 Subnet Discovery</subnetv4_synchro_name>
    </json_placeholders>
 
<!-- IP Addresses Parameters -->
    <ip_default_org_id>Demo</ip_default_org_id>
    <ip_default_status>unassigned</ip_default_status>
 
<!-- Discovery functions -->
    <ping_absolute_path>/bin/</ping_absolute_path>
    <dig_absolute_path>/usr/bin/</dig_absolute_path>
 
</parameters>
Parameter Meaning Sample value
itop_url URL to the TeemIp Application. https://localhost/teemip
itop_login Login (user account) for connecting to TeemIp. Must have admin rights for executing the data synchro. admin
itop_password Password for the iTop account. admin_pwd
contact_to_notify The email address of an existing contact in TeemIp to be notified of the results of the synchronization. john.doe@demo.com
discovery_application_uuid String that uniquely identifies the remote the IP Discovery Application in TeemIp. The string is automatically created at creation time of the IP Discovery object FFF3_60D8_FEE6_520D
ipv4_synchro_name Name of the data IPv4 synchro that the discovery application will create in TeemIp. By default, the UUID will be appended to that string. TeemIp IPv4 Discovery
subnetv4_synchro_name Name of the IPv4 subnet data synchro that the discovery application will create in TeemIp. By default, the UUID will be appended to that string. TeemIp IPv4 Subnet Discovery
ip_default_org_id Organization that IPs created by the discovery applications will belong to. Demo
ip_default_status Status of IPs when create by the discovery application. Possible values: allocated, released, reserved, unassigned unassigned
ping_absolute_path Absolute path of the ping tool /bin/
dig_absolute_path Absolute path of the dig tool /usr/bin/
The Itop data collector documentation describes the configuration process in details
Since TeemIp 2.3 or iTop 2.5, the access to the REST web services is now restricted to the users having the profile REST Services User.

Usage

Create IP Discovery Applications in TeemIp

IP Discovery menu

Once installed, the extension will alter the IP Management menu by adding a sub menu group next to the IPv6 Shorcuts.

The “IP Discovery” link provides statistics on IP addresses and discovery results.

Clicking on the count figure next to an IP Discovery application provides the list of subnets discovered by the given application.

IP Discovery applications

These objects define the remote applications that actually perform the discovery work. They enable these remote applications to communicate with TeemIp, list the subnets to be discovered and define how they should be discovered.

Properties

Name Type Mandatory?
General Information
Name Alphanumeric string Yes
Organization Foreign key to a(n) Organization Yes
Status Possible values: implementation, production, obsolete No
Business criticity Possible values: medium, high, low No
Location Foreign key to a(n) Location No
Description Multiline character string No
Requestor Foreign key to a(n) Person No
Move to production date Date (year-month-day) No
UUID Alphanumeric string No
Ping Function
Ping enabled Possible values: yes, no No
Ping timeout Integer No
IP lookup Function
IP lookup enabled Possible values: yes, no No
DNS server #1 Alphanumeric string No
DNS server #2 Alphanumeric string No
Scan Function
Scan enabled Possible values: yes, no No
Port number Integer No
Protocol Possible values: tcp, udp, both No
Scan timeout Integer No

Tabs

Tab Description
Contacts All the contacts for this application
Documents All the documents linked to this object
Managed IPv4 subnets All the IPv4 subnets that the application must discover

Listing IP Discovery applications

The IP Discovery applications shorcut displays all the applications that belong to the selected organization.

Creating a new IP Discovery application

From the listing view, click on the “New…” menu to display the creation form.

  • UUID attribute uniquely identifies the remote IP Discovery application and must be unique across TeemIp, regardless the organization it belongs to. UUID is automatically generated by TeemIp when the application is created.
  • Ping, IP Lookup and Scan functions can be enabled or disabled globally for the application. If enabled, these function can be disabled individually for each subnets to be scanned.
  • DNS Server #1 and 2 can either be an IP address or a domain name.
For a given Discovery Application, the UUID defined in TeemIp MUST match the discovery_application_uuid parameter defined in the configuration file of the remote application.

Subnets

The extension enhances the IP Subnet datamodel so that an IP discovery application can look after them. For that purpose, the following attributes are added:

Properties

Name Type Mandatory?
Discovery Information
IP Discovery application Foreign key to a(n) IP Discovery application No
Last discovery date Date when the last discovery run for that subnet No
IP discovery ping enabled Foreign attribute of the application No
Ping enabled for subnet Possible values: yes, no No
Ping duration Time it took to ping the whole subnet No
IP discovery IP lookup enabled Foreign attribute of the application No
IP lookup enabled for subnet Possible values: yes, no No
IP lookup duration Time it took to lookup the whole subnet No
IP discovery scan enabled Foreign attribute of the application No
Scan enabled for subnet Possible values: yes, no No
Scan duration Time it took to scan the whole subnet No

Creating a new subnet

IP discovery attributes alter the subnet creation form as follows:

The 3 attributes Ping, IP lookup and Scan enabled for subnet may change the way the subnet's IP discovery application will discover the subnet:

  • If one of these 3 discovery functions is enabled at the application level, it may be denied at the subnet level.
  • However, if a function is disabled at the application level, it cannot be enabled at the subnet one.

Run the discovery

To launch the IP discovery and synchronization with TeemIp, run the following command (from the root directory where the IP Discovery application is installed):

php exec.php

The following (optional) command line options are available:

Option Meaning default value
--console_log_level=<level> Level of output to the console. From -1 (none) to 9 (debug). 6 (info)
--collect_only Run the discovery but do not synchronize the data with TeemIp and stores discovery results in a csv file under the data directory. false
--synchro_only Synchronizes the data previously collected and stored in the data directory with TeemIp. Does not run the discovery. false
--configure-only Checks (and updates if necessary) the synchronization data source in TeemIp and exit. Does NOT run the discovery nor the synchronization
--max_chunk_size=<size> Maximum number of items to process in one pass, for preserving the memory of the system. If there are more items to process, the application will iterate. 1000

Once invoked through the command line, the discovery application will:

  1. Connect to TeemIp to create its Synchronization Data Sources (or check their definition if they already exist and update them if needed)
  2. Connect to TeemIp to retrieve:
    1. What main discovery parameters to use,
    2. The list of subnets to discover and their specific settings (with regard the ping, ip lookup and scan functions),
    3. The list of IPs already configured in TeemIp and their attributes,
  3. Ping and / or look up and / or scan all subnets' IPs,
  4. Upload the collected data into TeemIp,
  5. Synchronize the collected data with the existing TeemIp IPs,
  6. Update the discovered subnets with their respective discovery durations.

Scheduling

Once you've run the IP Discovery interactively, the next step is to schedule its execution so that the discovery and import occurs automatically at regular intervals.

The IP Discovery application does not provide any specific scheduling mechanism, but the simple command line php exec.php can be scheduled with either cron (on Linux systems) or using the Task Scheduler on Windows.

For optimal results, don't forget to adjust the configuration parameter full_load_interval to make it consistent with the frequency of the scheduling.

Discovery functions

The IP Discovery application discovers networks through 3 standard network functions: ping, IP lookup and port scanning, each of them being individually activated for the applications. Furthermore, a function that is activated at the discovery application level can be inhibited for a subnet.

Ping

This is the standard and well known command based on icmp request. It is called with the following parameters:

Parameter Defined in Default value
ping_absolute_path /conf/params.local.xml <empty_string>
Ping timeout (s) IP Discovery Application in TeemIp 1 s

The command is invoked through the exec PHP function.

 <ping_absolute_path/>ping c1 -w<timeout> <ip_to_ping>

For instance:

 /bin/ping -c1 -w2 10.11.12.13

IP Lookup

This function checks if an entry exists in the DNS space for the IPs. It relies on the dig command and sends reverse lookups to 1 or 2 DNS servers. It is called with the following parameters:

Parameter Defined in Default value
dig_absolute_path /conf/params.local.xml <empty_string>
DNS server #1 IP Discovery Application in TeemIp - Can be a FQDN or an IP <empty_string>
DNS server #1 IP Discovery Application in TeemIp - Can be a FQDN or an IP <empty_string>

The command is invoked through the exec PHP function.

 <dig_absolute_path/>dig -x <ip_to_lookup> @<dns_server>

For instance:

 /usr/bin/dig -x 10.11.12.13 @8.8.8.8
 or
 dig -x 10.11.12.13 @mydnsserver.com
 

If no DNS server is provided, then the command is invoked without the @ and dig will rely on the DNS servers defined in the /etc/resolv.conf.

When 2 DNS servers are given then:

  • If the first one provides us with a response, no further lookup is done for the IP.
  • If the first one doesn't provide any response, the second one is tried.

Port scanning

The function is based on the PHP fsockopen function. The test is done according to the following parameters:

Parameter Defined in Default value
Port number IP Discovery Application in TeemIp none
Protocol IP Discovery Application in TeemIp none
Scan timeout (s) IP Discovery Application in TeemIp 1 s

For both UDP and TCP tests, the scan tries to open a socket on the given port.

  • For UDP, we must wait for an answer from the remote host for <scan timeout> seconds before declaring the test as failed.
  • For TCP test, answer may come faster if a remote host exists and listen on that port.

If the protocol is set to “both” then:

  • UDP is tried first.
  • If no answered is received, a test is done with TCP.

Synchronize TeemIp IPs and subnets

Discovery results of an IP Discovery Application are fed back to TeemIp through standard synchro data sources. Every IP discovery application will automatically create its own synchro data sources and will use them to push its finding and statistics into TeemIp, systematically, at the end of each discovery process.

These synchro data sources can be listed and managed from the Synchronization Data Sources menu under the Admin tools section.

TeemIp IPv4 Discovery

Default parameters for a given data source are defined in the remote IP Discovery application configuration file.

  • The UUID of the application is appended to the name of the synchro data source for an easy identification,
  • Discovered IPs that don't already exist in TeemIp are created,
  • Already existing IPs are updated,
  • No automatic deletion is done.

Reconciliation of IP addresses is done on the Organization and Address attributes. The following attributes may be affected by this synchro:

Name When ?
Organization Creation only
Status Creation only
Address Creation only
Responds to ping Creation and update
Responds to IP lookup Creation and update
FQDN from IP lookup Creation and update
Responds to scan Creation and update

TeemIp IPv4 Subnet Discovery

Default parameters for a given data source are defined in the remote IP Discovery application configuration file.

  • The UUID of the application is appended to the name of the synchro data source for an easy identification,
  • Statistics on discovery durations are updated.

Reconciliation of IP subnets is done on the Organization and Subnet IP attributes. The following attributes may be affected by this synchro:

Name When ?
Last discovery date Update
Ping duration Update
IP lookup duration Update
Scan duration Update

As usual with synchro data sources, the Status tab will provide information on every synchronization that occurred between the remote IP Discovery application and TeemIp.

View results in TeemIp

The IP Discovery Module alters the standard TeemIp datamodel so that discovery results can be visualized at both the subnet and the IP address levels.

Subnets

From the detailed display of a subnet, selecting 'List & Pick' under 'Other Actions' menu will show:

The following image may appear next to each IP of the subnet:

  • indicates that an IP pings,
  • indicates that an answer is received when IP is scanned,
  • indicates that a DNS entry exists for that IP and DNS output is provided just after the icon.

These flags reflects the latest status provided by the remote application. If an IP was pinging in the past and if the latest discovery finds out that this IP doesn't ping anymore, then the ping image will not be shown anymore.

Discovery flags (, , ) will never appear next to an IP that doesn't exist within TeemIp. This is due to the default behaviour of the application: newly discovered IPs that don't exist in TeemIp are automatically created with the status “unassigned”.

IP Addresses

The IP Discovery Module alters as well the IPAddress datamodel so that discovery informations related to IP addresses can be displayed at their level. The following attributes are added:

Properties

Name Type Can be edited?
Discovery Information
IP Discovery application name Alphanumeric string No
Last date of discovery Date [ No
Responds to ping Possible values: yes, no No
Responds to IP lookup Possible values: yes, no No
FQDN from IP lookup Alphanumeric string No
Responds to scan Possible values: yes, no No

Creating a new IP address
IP discovery attributes alter the address creation form as follows:

  • The IP discovery application is inherited from the subnet that that IP belongs to.
  • All other IP discovery parameters are, of course, read only as they are the result of the discovery process.
extensions/teemip-ip-discovery.txt · Last modified: 2019/02/13 16:46 by cnaud

";