Itential Docs & Guides | Drive Automation for Modern Networks

Itential Automation Platform (IAP) uses network and OSS adapters to facilitate the integration and interaction of IAP with various systems, applications, and environments. This section summarizes the adapter properties and parameters that are required for optimal configuration.

Network Adapters

Itential Ansible Manager

The Itential Ansible Manager (IAM) adapter is used to integrate IAP with Ansible Manager through the IAM API. Use the information in the table below to set the adapter-ansible_manager properties.

Adapter Properties

Property	Type	Description
host	String	Required. The hostname of the Ansible Manager adapter.
port	Number	Required. The port on which to connect to the adapter. Default port is 443.
authentication	Object	Required. Defines the properties used for authentication.
stub	Boolean	Optional.
protocol	String	Optional. Notifies the adapter whether to use http or https. Default is http.
healthcheck	Object	Required. Defines the types of health check settings currently supported. See
throttle	Object	Optional. Defines the properties used to throttle requests to Ansible Manager.
request	Object	Defines the properties used to handle requests/responses.
proxy	Object	Defines the properties used to handle requests/responses.
ssl	Object	Required. Defines the properties to utilize SSL authentication with Ansible Manager.

Authentication

Property	Type	Description
auth_method	String	Defines the authentication methods used for requests: basic user_password (default), static_token, request_token, no_authentication.
username	String	The username to authenticate with Ansible Manager on every request or when pulling a token that will be used in subsequent requests.
password	String	The password to authenticate with Ansible Manager on every request or when pulling a token that will be used in subsequent requests. If retrieved through an encrypt password call, use the exact return including the {code}.
token	String	Defines a static token that can be used on all requests.
token_user_field	String	The field in the token request where the username credential should be provided.
token_password_field	String	The field in the token request where the password credential should be provided.
token_result_field	String	Defines the field in the token response where the actual token will be.
token_URI_path	String	The API path used to retrieve a token.
token_timeout	Number	Defines how long a token is valid. Measured in milliseconds. Once a dynamic token is no longer valid, Itential has to pull a new token. Default is set to 0 (zero), which means the Itential platform pulls a token on every request. Maximum value is 3600000.
invalid_token_error	Number	Defines the http error that is received when a token is invalid. Notifies the adapter to pull a new token and retry the request. Default is 401.
auth_field	String	Defines the header field in which to place the token.
auth_field_format	String	Defines the format used to pass the authentication variables.

Healthcheck

The following settings are currently supported.

None - Not recommended. Itential will not run a check on Ansible Manager. Consequently, the sunable to determine if Itential can connect with Ansible Manager.
Startup - Itential will check for connectivity when the adapter initially comes up, but it will not check afterwards.
Intermittent - The adapter will check connectivity to Ansible Manager at the frequency defined in the frequency property

Property	Type	Description
type	String	The type of health check to run. Default is Intermittent.
frequency	Number	Defines how often the health check should run. Measured in milliseconds (minimum = 6000, maximum = 3600000). Default is 300000.
protocol	String	Defines the protocol (REST, SOAP, RPC, Socket, etc.) to use to check the health of the system. Default is REST.
uri_path	String	The path used to check the health of Ansible Manager. This call should be a simple request for information that does not require any parameters.

Throttle

Property	Type	Description
throttle_enabled	Boolean	Defines if the adapter should use throttling. Default is false.
number_pronghorns	Number	Defines if throttling is done in a single Itential instance or whether requests are being throttled across multiple Itential instances (minimum = 1, maximum = 20). Default is 1. Throttling a single Itential instance uses an in-memory queue so there is less overhead. Throttling across multiple Itential instances requires placing the request and queue information into a shared resource (e.g. database) so that each instance can determine what is running and what is next to run. Throttling across multiple instances requires additional I/O overhead.
sync_async	String	Defines if the queue handle requests synchronously or asynchronously.
max_in_queue	Number	Represents the maximum number of requests Itential should allow into the queue before rejecting requests (minimum = 1, maximum = 5000). Default is 1000.
concurrent_max	Number	Defines the number of requests that Itential can send to Ansible Manager at one time (minimum = 1, maximum = 1000). Default is 1; each request must be sent to Ansible Manager in a serial manner.
expire_timeout	Number	Defines the graceful timeout of the request session. After a request has completed, Itential will wait additional time prior to sending the next request. Measured in milliseconds (minimum = 0, maximum = 60000). Default is 0.
avg_runtime	Number	Represents the approximate average of how long it takes Ansible Manager to handle each request. Measured in milliseconds (minimum = 50, maximum = 60000). Default is 200. This metric has performance implications. If the number is set too low, it puts extra burden on Itential CPU and memory as the requests will continually try to run. If the number is set too high, requests may wait longer than they need to before running.The number does not need to be exact but your throttling strategy depends heavily on this number being within reason. If averages range from 50 to 250 milliseconds you might pick an average runtime somewhere in the middle so that when your Ansible Manager performance is exceptional you might run a little slower than you might like, but when it is poor you still run efficiently.

Request

Property	Type	Description
number_retires	Number	Required. Defines how many times to retry a request that has aborted or reached the limit before returning an error (minimum = 0, maximum = 20). Default is 3.
limit_retry_error	Number	Optional. Indicates the http error status code. Defines when no capacity is available and after waiting a short interval the adapter can retry the request (minimum = 0, maximum = 1000). Default is set to 0 (zero).
attempt_timeout	Number	Optional. Defines how long Itential should wait before aborting an attempt to connect. Measured in milliseconds (minimum = 1000, maximum = 300000). Default is 5000.
healthcheck_on_timeout	Boolean	Defines if the system should run a health check on timeout. Default is false. If set to true, the adapter will abort the request and run a health check until it re-establishes connectivity to Ansible Manager and then it will re-attempt the request.
archiving	Boolean	Optional. Default is false. Archives each request and response, and corresponding metrics (i.e., wait time, connection time, Ansible Manager time) in the `adapterid_results collection` in MongoDB. Before enabling this property, develop an archiving strategy for cleaning up the collection in the database so that it does not become too large, especially if the responses are large.

Proxy

Property	Type	Description
enabled	Boolean	Defines whether or not there is a proxy. Default is false.
host	String	The host name of the proxy. Default is localhost.
port	Number	The port used to connect to the proxy. Default is 443.
protocol	String	The protocol (i.e., http, https, socks 4, socks5, etc.) used to connect to the proxy. Default is http.

SSL

Property	Type	Description
enabled	Boolean	Defines whether or not SSL is enabled. Default is false. If SSL required, set to true.
accept_invalid_cert	Boolean	Defines whether the adapter should accept invalid certificates. Default is false.
ca_file	String	Defines the path name to the CA file used for SSL.
ciphers	String	Specifies a list of SSL ciphers to use.

Sample Configuration

{
    "id": "uniqueidforthisadapter",
    "properties": {
      "host": "system.access.resolved",
      "port": 443,
      "authentication": {
        "auth_method": "basic user_password",
        "username": "username",
        "password": "password",
        "token": "token",
        "token_user_field": "username",
        "token_password_field": "password",
        "token_result_field": "token",
        "token_URI_path": "/api/[version]/authenticate",
        "token_timeout": 0,
        "invalid_token_error": 401,
        "auth_field": "header.headers.X-AUTH-TOKEN",
        "auth_field_format": "${token}"
      },
      "stub": false,
      "protocol": "https",
      "healthcheck": {
        "type": "startup",
        "frequency": 300000,
        "protocol": "REST",
        "URI_Path": "/api/version/ping"
      },
      "throttle": {
        "throttle_enabled": false,
        "number_pronghorns": 1,
        "sync_async": "sync",
        "max_in_queue": 1000,
        "concurrent_max": 1,
        "expire_timeout": 0,
        "avg_runtime": 200
      },
      "request": {
        "number_retires": 3,
        "limit_retry_error": 401,
        "attempt_timeout": 5000,
        "healthcheck_on_timeout": false,
        "archiving": false
      },
      "proxy": {
        "enabled": false,
        "host": "localhost",
        "port": 9999,
        "protocol": "http"
      },
      "ssl": {
        "enabled": false,
        "accept_invalid_cert": false,
        "ca_file": "",
        "ciphers": ""
      }
    },
    "type": "AdapterClass"
  }

Ansible Tower

The Ansible Tower Adapter is used to manage and provide an API for Ansible Tower, a build/configuration/orchestration system.

Ansible Tower needs to be installed on a server that IAP can access.
Ansible Tower does not need to be installed on the same server that IAP is on.
Refer to the Minimum System Requirements (located in the Installation section) for the required version.

Currently, the adapter integrates with Ansible Tower - version XX, and will continue to be expanded over time.

Name	Method
Adhoc Command	GET, POST, DELETE
Credential	GET, POST, PUT, DELETE
Dashboard	GET
Group	GET, POST, PUT, DELETE
Host	GET, GET FILTERED (as getDevicesFiltered), POST, PUT, DELETE
Inventory	GET, POST, PUT, DELETE
Job	GET, GET RESULT, POST, PUT, DELETE
Job Event	GET
Job Template	GET, POST, PUT, DELETE, LAUNCH, LAUNCH SYNC, DRY RUN SYNC
Organization	GET, POST, PUT, DELETE (limitation of 1 in test environment)
Playbook	GET LIST
Project	GET, POST, PUT, DELETE
Workflow Job	GET, GET RESULT, POST, PUT, DELETE
Token	GET
Generic	GET, POST, PUT, DELETE

Adapter Properties

Property	Type	Description
host	String	Required. The IP or hostname of the Ansible Tower server.
port	Number	Required. The port number of the Ansible Tower server.
protocol	String	Optional. Tells the adapter whether to use http or https (http is the default).
credentials.username	String	Required. The username to use when connecting to the Ansible Tower server.
credentials.password	String	Required. The password to use when connecting to the Ansible Tower server.
credentials.token	String	Required. The token provided by your Ansible Tower license file.
stub	Boolean	Optional. Indicates whether the stub should be run instead of making calls to Ansible Tower (very useful during basic testing). The default is false which means connect to Ansible Tower.
throttle.throttle_enabled	Boolean	Optional. The throttle_enabled property defaults to false and simply states whether the adapter should use throttling or not.
throttle.number_pronghorns	Number	Optional. The number_pronghorns property defaults to 1 and states whether throttling is done in a single IAP instance or whether requests are being throttled across multiple IAPs. This is an important property for performance enhancements. Throttling in a single IAP uses an in memory queue so there is less overhead. Throttling across multiple IAPs requires putting the request and queue information into a shared resource (e.g. database), so that each IAP can determine what is running and what is next to run. This requires additional IO overhead.
throttle.sync_async	String	Optional. The sync_async property is not used at the current time. It is for future expansion of the throttling engine.
throttle.max_in_queue	Number	Optional. Max_in_queue represents the maximum number of requests that IAP should allow into the queue before rejecting requests. This is not necessarily a limit on what IAP can handle, but more about timely responses to the requests. The current default is 1000.
throttle.concurrent_max	Number	Optional. Concurrent_max defines the number of request that IAP can send to Ansible Tower at one time. The default is 1, meaning each request must be sent to Ansible Tower in a serial manner.
throttle.expire_timeout	Number	Optional. Expire_timeout defaults to 0. This is a graceful timeout of the request session. After the request has completed, IAP will wait the additional expire timeout time (in milliseconds) prior to sending the next request.
throttle.avg_runtime	Number	Optional. The average_runtime is an approximate average of how long it takes Ansible Tower to handle each request. This is an important number that has performance implications. If the number is defined too low, it puts extra burden on IAP CPU and memory as the requests will continually try to see if they can run. If the number is defined too high, requests may wait longer than they need to before running. The number does not need to be exact but the throttling strategy depends heavily on this number being within reason. If averages range from 50 to 250 milliseconds, pick an average runtime somewhere in the middle so that when Ansible Tower performance is exceptional you might be a little slower than you might like, but when it is poor you still run efficiently. Default is 200 milliseconds.
request.number_retries	Number	The number_retries tells IAP how many times to retry a request that has either aborted or taken the limit error before giving up and returning an error.
request.limit_retry_error	Number	Optional. The limit_retry_error is HTTP error status number which defines that no capacity was available and thus after waiting a short interval the adapter can retry the request. The number defaults to 0.
request.attempt_timeout	Number	Optional. The attempt_timeout is how long IAP should wait before aborting the attempt. On abort, IAP will back off the requests and run a Healthcheck until it re-establishes connectivity to Ansible Tower. Then it will re-attempt the request that aborted. The attempt timeout defaults to 5000 milliseconds.
request.archiving	Boolean	Optional. The archiving flag defaults to false. It archives the request, the results and the various times (wait time, Ansible Tower time, and overall time) in the ansibletower_results collection in MongoDB. Before enabling this capability think about how much to archive and develop a strategy for cleaning up the collection in the database so that it does not become too large, especially if the responses are large.
request.ssl.enabled	Boolean	If you require SSL then change the enabled flag to true. SSL can work two different ways, you can accept invalid certifications (only recommended for lab environments) by setting the flag to true or you can provide a CA file. If SSL is enabled and the accept invalid certifications is false, then the CA file is required.
request.ssl.accept_invalid_cert	Boolean	Flag indicating whether untrusted certificates are accepted.
request.ssl.accept_invalid_cert	Boolean	Flag indicating whether untrusted certificates are accepted.
request.ssl.ca_file	String	Path to the certificate authority chain containing a list of trusted certificates.
request.ssl.ciphers	String	The trusted set of SSL ciphers to negotiate with the remote host.

Sample Configuration

A sample Ansible Tower configuration is provided below for reference. Be sure to configure the following properties.

host
port
protocol
credentials.username
credentials.password
credentials.token
request.ssl.enabled
request.ssl.ca_file
request.ssl.ciphers

{
  "id": "ansibletower",
  "type": "AnsibleTower",
  "properties": {
    "host": "localhost",
    "port": 443,
    "protocol": "https",
    "credentials": {
      "username": "admin",
      "password": "$ENC87eb897b507afc1796db49409dd0261985802f84aad3469e",
      "token": "token"
    },
    "stub": false,
    "throttle": {
      "throttle_enabled": false,
      "number_pronghorns": 1,
      "sync_async": "sync",
      "max_in_queue": 1000,
      "concurrent_max":1,
      "expire_timeout":0,
      "avg_runtime": 200
    },
    "request": {
      "number_retires":3,
      "limit_retry_error":401,
      "attempt_timeout":5000,
      "archiving":false,
      "ssl": {
        "enabled": true,
        "accept_invalid_cert": false,
        "ca_file": "/etc/ssl/ca.cert",
        "ciphers": "DHE-RSA-AES256-SHA"
      }
    }
  }
}

NSO

The NSO Adapter connects to NSO using the web UI port and netconf port. Each time the NSO adapter establishes a connection to NSO, the adapter will read the service models advertised by NSO. To securely configure the NSO adapter, make sure the HTTP hostname is configured with the same hostname that is configured in the common name of the NSO server certificate.

NSO needs to be installed on a server that IAP can access.
NSO does not need to be installed on the same server that IAP is on.
Refer to the Minimum System Requirements (Located in the Installation section) for the required version.

Adapter Properties

Use the information in the table below to set NSO Adapter properties.

Property	Type	Description
http.host	String	Host address of the NSO web-ui.
http.port	Number	Host port of the NSO web-ui
ssl.enabled	Boolean	HTTP connection to use SSL.
ssl.acceptInvalidCerts	Boolean	Accept invalid certificates.
ssl.caFile	String	Certificate Authority file.
ssl.ciphers	String	SSL ciphers.
netconf.host	String	Host address of NSO netconf.
netconf.port	Number	Host port of NSO netconf.
netconf.protocol	String	Protocol used for netconf transactions.
netconf.frame_size	Number	Netconf frame_size to use; defaults to 64.
authenticationStrategy.type	String	Set to "dynamic".
authenticationStrategy.location	String	Set to "NSO".
authenticationStrategy.method	String	Set to either "tokenLogin" or "machineLogin".
credentials.user	String	NSO login account username.
credentials.passwd	String	NSO login account password.
commitWait	Number	Maximum timeout for a commit (in seconds).
commitQueue	Boolean	A boolean flag indicating whether commit queues will be enabled by default.
poolSize	Number	The default pool size.
yangDirs	Array of Strings	List of directories to scan for yang. Directories can include the "*" wildcard. Only required when connecting to NSO versions earlier than 4.5.2.
max_reconnect_attempts	Number	The maximum number of times IAP will try to connect to NSO before stopping. If max_reconnect_attempts and total_reconnect_window are both configured, whichever condition is met first will set the effective limit. If this is set to 0, IAP will continue to connect to NSO indefinitely using a back-off algorithm.
total_reconnect_window	Number	A maximum reconnect window (number of minutes) can also be configured. If max_reconnect_attempts and total_reconnect_window are both configured, whichever condition is met first will set the effective limit. If this is set to 0, no time window will be enforced.
min_reconnect_interval	Number	The number of seconds to wait once an NSO down event is detected before the first re-connection attempt. This value will be doubled for each subsequent attempt until the max_reconnect_interval is reached.
max_reconnect_interval	Number	When NSO becomes unavailable, IAP will start its reconnect timer at min_reconnect_interval. If the first reconnect attempt fails, the reconnect timer will be doubled each time a connection attempt fails. The process of attempting a connection and doubling the reconnect timer will continue until the reconnect timer reaches (or exceeds) the max_reconnect_interval. All future re-connect attempts will occur at this frequency.

Sample Configuration

A sample NSO adapter configuration is provided below for reference. Be sure to configure the following properties:

http.host
http.port
ssl.enabled
ssl.caFile
ssl.ciphers
netconf.host
netconf.port
credentials.user
credentials.passwd

The default configuration is to use a machineLogin for all transactions. To enable individual user authorization to be handled by NSO,NSO must be configured with an external authentication script capable of returning external group memberships for a given user id. Ensure NSO is configured for external authentication and configure the authenticationStrategy.method to be tokenLogin instead of machineLogin.

{
  "id": "nso",
  "type": "NSO",
  "properties": {
    "http": {
      "host": "localhost",
      "port": 8888
    },
    "ssl": {
      "enabled": true,
      "acceptInvalidCerts": false,
      "caFile": "/etc/ssl/ca.cert",
      "ciphers": "DHE-RSA-AES256-SHA"
    },
    "netconf": {
      "host": "localhost",
      "port": 2022,
      "protocol": "ssh",
      "frame_size": 64
    },
    "credentials": {
      "user": "admin",
      "passwd": "admin"
    },
    "authenticationStrategy": {
      "type": "dynamic",
      "location": "NSO",
      "method": "machineLogin"
    },
    "commitWait": 30000,
    "commitQueue": false,
    "max_reconnect_attempts": 0,
    "total_reconnect_window": 120,
    "min_reconnect_interval": 30,
    "max_reconnect_interval": 900,
    "poolSize": 3,
    "yangdirs": []
  }
}

Itential Tools Package

The Itential Tools Package contains tools necessary for IAP to interact with NSO. Download this project from your Itential Nexus repository.

Change to the NCS packages directory.
```
cd /var/opt/ncs/packages
```
Extract the itential_tools package.
```
tar xzf itential_tools_VERSION.tgz
```
Run the make command in the src directory of this project.
```
cd itential_tools/src && make clean all -s && cd -
```

Perform a package reload using the ncs_cli.

ncs_cli -u admin -C --noaaa > package reload

Commit Queues

NSO Manager includes a Commit Queue Manager. The Commit Queue Manager allows users to manage items in a queue to enhance overall transactional throughput and avoid choke points. The Commit Queue Manager also allows users to handle transactions on a per managed device basis.

NSO External Authentication Installation

To get started with external authentication, install the ph-auth script and then configure the ncs.conf file. The NSO external authentication script requires python and the requests module to be installed.

Install these modules directly from yum.

yum install -y python python-pip
pip install requests

Configure the external authentication script found in.
```
/var/opt/ncs/packages/itential_tools/external_auth
```
Configure the hostname and port of the IAP server.
```
PH_ADDRESS = "localhost"
PH_PORT = "3000"
```

Test the external authentication script.

python /var/opt/ncs/packages/itential_tools/external_auth/ph-auth.py

Type the following command and press Enter.
```
[test;test;]
```
You should receive the following result.
```
reject
```

NSO External Authentication Configuration

Configure the ncs.conf file to use external authentication. Both external-authentication and either location-authentication or pam should be enabled. IAP will initially make a connection using a global service account with admin privileges.

When performing transactions that can bind a particular request to a human user account, IAP will start making calls on behalf of a specific user (e.g., bobjones) instead of admin.

vi /etc/ncs/ncs.conf

...
<aaa>
    <ssh-server-key-dir>${NCS_CONFIG_DIR}/ssh</ssh-server-key-dir>

    <!-- Depending on OS - and also depending on user requirements -->
    <!-- the pam service value value must be tuned. -->

    <pam>
        <enabled>false</enabled>
        <service>system-auth</service>
    </pam>
    <external-authentication>
        <enabled>true</enabled>
        <executable>/var/opt/ncs/packages/itential_tools/external_auth/ph-auth.py</executable>
    </external-authentication>
    <local-authentication>
        <enabled>false</enabled>
    </local-authentication>

    <expiration-warning>prompt</expiration-warning>
</aaa>

Restart NSO.

service ncs restart

Configure Default Authgroup Mappings

If using the tokenLogin feature of the IAP NSO adapter, verify the users all properly map to the appropriate southbound credentials. Itential's reference configuration is accomplished by ensuring credentials are configured for the default umap inside NSO. Run this command for each authentication group; be sure to apply the appropriate device credentials as the remote-name and remote-password.

ncs_cli -u admin -C
config
devices authgroups group default default-map remote-name admin remote-password admin
commit

Sample NSO NACM Rules

To use IAP with an NSO NETCONF Access Control Model (NACM) implementation, you will need to reference the sample set of NACM rules provided with this package.

The NACM rules assume the following groups model.

Group	Description
pronghorn	A NACM group applied to the user the Itential NSO adapter will use when connecting with NSO.
users	A NACM group applied to the user the Itential NSO adapter will use when connecting with NSO.
admins *	A NACM group applied to admins of the system.

These groups are shared across both IAP and NSO. Using the sample external authentication script, Itential will provide the set of groups defined inside IAP and discovered from the configured AAA system to NSO to be applied to the NACM rule-lists.

Note: It is important that the set of groups returned by the NSO AAA provider properly match NACM rule-list group assignments.

The following rule-lists are provided in the sample rules.

List	Description
pronghorn-system	Applied to the pronghorn group.
pronghorn-users	Applied to the pronghorn user groups.
pronghorn-admins *	Applied to pronghorn and admins groupm.

Be sure to adjust the groups associated with these rule-lists to ensure they are mapped to the user accounts appropriately.

Install NACM Rules in NSO

The attached file (sample-nacm-rules.xml) can be loaded into the NSO CDB using the "load merge" feature of the ncs_cli. Use the following commands to load the sample NACM rules.

ncs_cli -u admin -C
config
load merge sample-nacm-rules.xml
commit dry-run
commit

Configure Device Whitelists

The sample NACM rules contain a rule-list for whitelisting devices to the users group. Configure the sample rule-list or create your own rule-lists to associate devices with groups of users. Add devices to the users whitelist.

ncs_cli -u admin -C
config
nacm rule-list whitelist-devices rule permit-device-mydevicename path /devices/device[name='mydevicename'] action permit
commit

Note: In a NACM-enabled system, any new device added to the system needs to have its groups defined at the time of device turnup. This applies to both manual configuration of the devices in NSO as well as any device turnup workflows that are turning up new devices on the network.

Configure Service Whitelists

The sample NACM rules contain a rule-list for whitelisting service instances to the users group. Configure the sample rule-list or create your own rule-lists to associate services with groups of users. Add all service instances for a model to the users whitelist.

ncs_cli -u admin -C
config
nacm rule-list whitelist-services rule permit-service-cisco-ios path /services/cisco-ios action permit
commit

Alternately, you can add a single instance to the users whitelist.

ncs_cli -u admin -C
config
nacm rule-list whitelist-services rule permit-service-cisco-ios-101 path /services/cisco-ios[vlan=101] action permit
commit

Note: In a NACM-enabled system, any new service instance added to the system needs to have its groups defined. Service instances may be restricted to an individual group or made accessible to multiple groups with the appropriate NACM rules.

Itential Tools Rules Engine

The Itential_Tools Rules Engine allows users to define scripts that can be executed prior to NSO command-sets being executed. These scripts will modify (add or change) values in the NSO command-sets prior to their execution.

The rules engine has two parts: a rule engine and a script engine. The script engine reads the script files that have been defined for the command that is passed in, and then reads in the defined rules before it finally passes the rules to the rule engine to be processed.

The following files are required:

File	Name	Description
command file	dl_command.json	This file is used to associate the command name to the script name.
rule script	<name of the script>.dls	This file will define the script. It is generated by the user who creates the script.

Development Tools and Libraries

The following tools and libraries can be used for development and testing of the Itential_Tools Rules Engine. The main framework for the Java-based application will be the Java Spring framework.

Testing Tool	Description
Mockito	A mocking framework for unit testing.
TestNG	A testing framework inspired from JUnit and NUnit that is designed to cover all categories of testing. The functionality of TestNG includes support for annotations and concurrent testing, along with several other features. TestNG requires JDK 7 or higher.

System Design

The Itential_Tools Rules Engine has three functional areas:

Script Engine
Rule Engine
Script Validation Engine

Itential Tools Rules Engine Design

Script Engine

The Script Engine is the main driver for script processing. It receives the command file that contains the request to modify the NSO command-set. The following processes are performed in the Script Engine:

Read in the command file.
Locate the command to get the script file to execute.
Read in the script file that needs to be executed. This file will have the defined rules to execute against the NSO command-set.
The rule engine will be called to process the rules.
The NSO command-set will be modified appropriately.
The modified command-set will be returned to the calling system.

Any errors encountered along the way will be handled appropriately and returned to the calling system.

Rule Engine

The Rule Engine is responsible for processing each rule. The input parameters will be the rule and the NSO command-set. The modified NSO command-set will be its output parameter. The following steps will be performed:

Receive parameters.
Determine which rule to process.
Perform the rule process and modify the NSO command-set.
Return the modified NSO command-set.
Handle any errors and pass back to Script Engine for its error handler to process.

Script Validation Engine

The Script Validation Engine will validate if the script has the correct format and the commands that are contained in the script are valid commands with the correct number of parameters required for the command type.

If any of the validations fail, a custom java exception will be thrown with the appropriate error message and code, and then passed back to the calling system to be handled appropriately.

Validation	Description
Parameter (required)	Script name.
Checks	Each command must be terminated with a semicolon. Each command will be validated for a valid number of parameters.

Validation Commands

$DLC_REPLACE

Four (4) parameters:

1st - Any String
2nd - Any String
3rd - Int value
4th - Int Value

$DLC_INSERT

Five (5) parameters:

1st - Any String
2nd - Any String
3rd - Before or After value
4th - Int Value
5th - Int value

Rule Command Format and Usage

Usage	Format
Replace	`$DLC_REPLACE("original string","new string", occurrence, num)`
Insert Before	`$DLC_INSERT("original string","new string",BEFORE, occurrence, num)`
Insert After	`$DLC_INSERT("original string","new string",AFTER, occurrence, num)`

Arguments

Argument	Description
original string	What the command is looking for.
new string	What will be added. This will also support the following escape characters: `\t - tab, \n - new line, \r - return`. Support of additional options as defined in the section "Supported Options within the 'new string' Argument" below.
occurrence	Number that indicates the starting position of the original string before it begins to execute the rule. For example, if there are three (3) instances of the original string and the occurrence is two (2), then the command would start processing at instance two (2).
num	Number of times the action occurs 0-x (0=all of them). For example, if there are 4 matches of original string and occurrence=3, then only the first three original strings will have the replacement. Note: In this scenario, this works in conjunction with the Occurrence setting such that if Occurrence=3 and Num=3, then only two (2) replacements would occur (i.e. the third and fourth) although Num=3.

Supported Options within the "new string" Argument

$STRING

Syntax: $STRING['my string'|'2nd string'|'3rd string']

Parameters:
- $STRING - A single value or and array or string separated by a a pipe ("|") symbol that will be printed out. An array will cause a loop will occur.
$LOOP

Syntax: $LOOP[{start}-{end}|$STRING['{string to print out}']$LOOP_VAR]

Parameters:
- {start}:[int] - The starting number of the loop.
- {end}:[int] - The ending number of the loop.
- {string to print out}:[string] - Any string you want to print out.
- $LOOP_VAR:[int] - The integer value of the current loop iteration that will be printed out where it appears either before or after the $STRING[] option. Note: Multiple references to $LOOP_VAR are allowed and supported.

Example

$DLC_INSERT("access-list","new text here",AFTER,1);
$DLC_INSERT("access-list","new text here",BEFORE,1);
$DLC_INSERT("","\n$LOOP[1-100|$STRING['vlan ']$LOOP_VAR]\n",AFTER,1,1);
$DLC_INSERT("","\n$STRING['my string'|'2nd string'|'3rd string']\n",BEFORE,1,1)

Implementation Design

Implementation of the designed Rule Engine will include the following features:

Integration into the current itential_tools java application.
A mechanism for the system to refresh the list of commands available to it without a required restart of itential_tools.
An option to bypass the refresh feature.
The ability of users to update the commands and scripts.

Rule Engine Integration

To integrate the new Rule Engine into the itential_tools application, the java jar file generated by the Rule Engine application would need to be included in the private-jar or the shared-jar directory to build the itential_tools application.

The com.itential.datalore_utils.ScriptEngine package would need to be imported into the class that is calling it. Below is a code sample to execute the command.

import com.itential.datalore_utils.ScriptEngine;

ScriptEngine scriptEngine = new ScriptEngine(sCommand, sNSOCommandSet);

try {
    sNSOCommandSet = scriptEngine.processCommand();
} catch (Exception e) {
    e.printStackTrace();
}

Integration Features

Feature	Description
Refresh command list	Currently, the command list is re-read every time processing of the command is executed.
Bypass option	A configuration parameter called bypassRuleEngine can be defined for itential_tools and passed to it. This would be defined in the application that is calling the feature set. If this parameter is not passed to itential_tools then a default value of TRUE would be set for the parameter and bypass this feature.
Update command/script by user	In the first release of this feature, all commands/scripts will be manually created/edited and copied to the installed location. In the initial release, no will be UI available to support any script manipulation.

OSS Adapters

EAI

The EAI Adapter is used to interact with the Ericsson Adaptive Inventory (EAI) platform. It currently integrates with the following:

Entity	Operation
card	(get, create, update and delete)
container	(get, create, update and delete)
equipment	(get, create, update and delete)
port	(get, create, update and delete)
shelf	(get, create, update and delete)
site	(get, create, update and delete)
slot	(get, create, update and delete)
trail	(get, create, update and delete)
generic	(get, create, update and delete)

Properties

Use the information in the table below to set the EAI Adapter properties. The host, port, and credentials are used to connect to EAI upon the adapter intially coming up. A connectivity check tells IAP the adapter has loaded successfully. The healthcheck tells IAP that it can reach EAI. The throttle section is all about throttling requests to EAI, and the request section defines properties to help handle requests.

Property Name	Type	Description
host	String	Required. The ip or hostname of the EAI server.
port	Number	Required. The port of the EAI server.
protocol	String	Optional. Tells the adapter whether to use http or https (http is the default).
credentials.db_host	String	Required. The IP or hostname of the DB server.
credentials.db_name	String	Required. The name of the DB to connect to.
credentials.token	String	Required. An authentication token used to authenticate with the EAI server.
stub	Boolean	Optional. Indicates whether the stub should run instead of making calls to EAI (very useful during basic testing). The default is false, which means connect to EAI.
throttle.throttle_enabled	Boolean	Optional. Throttle_enabled defaults to false and simply states whether the adapter should use throttling or not.
throttle.number_pronghorns	Number	Optional. Number of pronghorns defaults to 1 and states whether the throttling is done in a single IAP instance or whether requests are throttled across multiple IAPs. This is an important property for performance enhancements. Throttling in a single IAP uses an in-memory queue so there is less overhead. Throttling across multiple IAPs requires putting the request and queue information into a shared resource (e.g. database) so that each IAP can determine what is running and what is next to run. This requires additional IO overhead.
throttle.sync_async	String	Optional. Sync_async is not used at the current time (it is for future expansion of the throttling engine).
throttle.max_in_queue	Number	Optional. Max_in_queue represents the maximum number of requests that IAP should allow into the queue before rejecting requests. This is not necessarily a limit on what IAP can handle but more about timely responses to the requests. The default is currently 1000.
throttle.concurrent_max	Number	Optional. Concurrent_max defines the number of request that IAP can send to EAI at one time. The default is 1 meaning each request must be sent to EAI in a serial manner.
throttle.expire_timeout	Number	Optional. This is a graceful timeout of the request session. After the request has completed, IAP will wait the additional expire timeout time (in milliseconds) prior to sending in the next request. Expire timeout defaults to 0.
throttle.avg_runtime	Number	Optional. The average runtime is an approximate average of how long it takes EAI to handle each request. This is an important number as it has performance implications. If the number is defined too low, it puts extra burden on IAP CPU and memory as the requests will continually try to see if they can run. If the number is defined too high, requests may wait longer than they need to before running. The number does not need to be exact but your throttling strategy depends heavily on this number being within reason. If averages range from 50 to 250 milliseconds you might pick an average runtime somewhere in the middle so that when EAI performance is exceptional you might be a little slower than you might like, but when it is poor you still run efficiently. The default is 200 milliseconds.
request.number_retries	Number	The number_retries tells IAP how many times to retry a request that has either aborted or taken the limit error before giving up and returning an error.
request.limit_retry_error	Number	Optional. The limit_retry_error is HTTP error status number which defines that no capacity was available and, after waiting a short interval, the adapter can retry the request. The default is 0.
request.attempt_timeout	Number	The attempt_timeout is how long IAP should wait before aborting the attempt. On abort, IAP will back off the requests and run a healthcheck until it re-establishes connectivity to EAI and then it will re-attempt the request that aborted. The attempt timeout is optional and defaults to 5000 milliseconds.
request.archiving	Boolean	Optional. The default is false. The archiving flag archives the request, the results and various times (wait time, EAI time and overall time) in the eai_results collection in MongoDB. Before enabling this capability think about how much to archive and develop a strategy for cleaning up the collection in the database so that it does not get too large, especially if the responses are large.

Sample Configuration

A sample configuration for the EAI adapter is provided. Be sure to configure the following properties.

host
port
protocol
credentials.dbHost
credentials.dbName
credentials.token

{
  "id": "eai",
  "type": "EAI",
  "properties": {
    "host": "eai.example.com",
    "port": 443,
    "protocol": "https",
    "credentials": {
      "dbHost": "eai.example.com",
      "dbName": "granite",
      "token": "Basic ABCDEFG=="
    },
    "stub": false,
    "throttle": {
      "throttle_enabled": false,
      "number_pronghorns": 1,
      "sync_async": "sync",
      "max_in_queue": 1000,
      "concurrent_max": 1,
      "expire_timeout": 0,
      "avg_runtime": 200
    },
    "request": {
      "number_retries": 3,
      "limit_retry_error": 401,
      "attempt_timeout": 5000,
      "archiving": false
    }
  }
}

Service Now

The ServiceNow Adapter interacts with the ServiceNow platform. It currently integrates with the following.

ServiceNow	Methods
CMDB Config Item	(get, create, and update)
Incident	(get, create, and update)
Problems	(get, create, and update)
Change Request	(get, create, and update)
Catalog Request	(get, create, and update)
Catalog Request Item	(get, create, and update)
Knowledgebase Article	(get, create, and update)
User	(get, create, and update)
Group	(get, create, and update)
generic	(get, create, and update)

Properties

Use the information in the table below to set the ServiceNow Adapter properties. The host, port, and credentials are used to connect to ServiceNow upon the adapter intially coming up. A connectivity check tells IAP the adapter has loaded successfully. The healthcheck tells IAP that it can reach ServiceNow. The throttle section is all about throttling requests to ServiceNow, and the request section defines properties to help handle requests.

Property Name	Type	Description
host	String	Required. The ip or hostname of the ServiceNow server.
port	Number	Required. The port of the ServiceNow server.
protocol	String	Optional. Tells the adapter whether to use http or https (http is the default).
credentials.username	String	Required. The username to use when authenticating with ServiceNow.
credentials.password	String	Required. The password to use when authenticating with ServiceNow.
credentials.token	String	Required. The token used to authenticate with ServiceNow.
stub	Boolean	Optional. Indicates whether the stub should run instead of making calls to ServiceNow (very useful during basic testing). The default is false, which means connect to ServiceNow.
throttle.throttle_enabled	Boolean	Optional. Throttle_enabled defaults to false and simply states whether the adapter should use throttling or not.
throttle.number_pronghorns	Number	Optional. Number of pronghorns defaults to 1 and states whether the throttling is done in a single IAP instance or whether requests are throttled across multiple IAPs. This is an important property for performance enhancements. Throttling in a single IAP uses an in-memory queue so there is less overhead. Throttling across multiple IAPs requires putting the request and queue information into a shared resource (e.g. database) so that each IAP can determine what is running and what is next to run. This requires additional IO overhead.
throttle.sync_async	String	Optional. Sync_async is not used at the current time (it is for future expansion of the throttling engine).
throttle.max_in_queue	Number	Optional. Max_in_queue represents the maximum number of requests that IAP should allow into the queue before rejecting requests. This is not necessarily a limit on what IAP can handle but more about timely responses to the requests. The default is currently 1000.
throttle.concurrent_max	Number	Optional. Concurrent_max defines the number of request that IAP can send to ServiceNow at one time. The default is 1 meaning each request must be sent to ServiceNow in a serial manner.
throttle.expire_timeout	Number	Optional. This is a graceful timeout of the request session. After the request has completed, IAP will wait the additional expire timeout time (in milliseconds) prior to sending in the next request. Expire timeout defaults to 0.
throttle.avg_runtime	Number	Optional. The average runtime is an approximate average of how long it takes ServiceNow to handle each request. This is an important number as it has performance implications. If the number is defined too low, it puts extra burden on IAP CPU and memory as the requests will continually try to see if they can run. If the number is defined too high, requests may wait longer than they need to before running. The number does not need to be exact but your throttling strategy depends heavily on this number being within reason. If averages range from 50 to 250 milliseconds you might pick an average runtime somewhere in the middle so that when ServiceNow performance is exceptional you might be a little slower than you might like, but when it is poor you still run efficiently. The default is 200 milliseconds.
request.number_retries	Number	The number_retries tells IAP how many times to retry a request that has either aborted or taken the limit error before giving up and returning an error.
request.limit_retry_error	Number	Optional. The limit_retry_error is HTTP error status number which defines that no capacity was available and, after waiting a short interval, the adapter can retry the request. The default is 0.
request.attempt_timeout	Number	The attempt_timeout is how long IAP should wait before aborting the attempt. On abort, IAP will back off the requests and run a healthcheck until it re-establishes connectivity to ServiceNow and then it will re-attempt the request that aborted. The attempt timeout is optional and defaults to 5000 milliseconds.
request.archiving	Boolean	Optional. The default is false. The archiving flag archives the request, the results and various times (wait time, ServiceNow time and overall time) in the servicenow_results collection in MongoDB. Before enabling this capability consider how much to archive and develop a strategy for cleaning up the collection in the database so that it does not become too large, especially if the responses are large.

Sample Configuration

A sample configuration for the ServiceNow adapter is provided. Be sure to configure the following properties:

host
port
protocol
credentials.username
credentials.password
credentials.token

{
  "id": "servicenow",
  "type": "ServiceNow",
  "properties": {
    "host": "customer.servicenow.com",
    "port": 443,
    "protocol": "https",
    "credentials": {
      "username": "",
      "password": "",
      "token": ""
    },
    "stub": false,
    "throttle": {
      "throttle_enabled": false,
      "number_pronghorns": 1,
      "sync_async": "sync",
      "max_in_queue": 1000,
      "concurrent_max": 1,
      "expire_timeout": 0,
      "avg_runtime": 200
    },
    "request": {
      "number_retries": 3,
      "limit_retry_error": 401,
      "attempt_timeout": 5000,
      "archiving": false
    }
  }
}

SQL

The SQL Adapter allows interaction with a SQL database. This adapter can be configured for either direct SQL queries or in an object-relational mapping configuration. Reference the sample configurations for examples of each mode. Support for SSL/TLS encrypted connections will soon be available.

Properties

Property Name	Type	Description
host	String	Required. The ip or hostname of the SQL database.
port	Number	Required. The port of the SQL database.
dbName	String	The name of the SQL database.
credentials.user	String	The username to use when authenticating with the SQL database.
credentials.passwd	String	The password to use when authenticating with SQL database.
direct	Boolean	A boolean flag indicating whether a direct connection to the database will be established and exposed.
orm	Boolean	A boolean flag indicating the object-relational-mapping connection to the database will be established and exposed.
min	Number	The minimum number of concurrent connections to open. Only used for ORM connections.
max	Number	The maximum number of concurrent connections to open. Only used for ORM connections.
acquireTimeout	Number	The number of milliseconds to wait before declaring the attempt to acquire a database connection has failed. Only used for direct connections. Default is 10,000.
waitForConnection	Boolean	Determines the behavior of the adapter when no connections are available. Only used for direct connections. Default is true. If true, the request will be queued for response when a connection is available. If false, the request will error immediately.
connectionLimit	Number	The number of supported concurrent connections. Only used for direct connections. Default is 10.
queueLimit	Number	The maximum number of connection requests allowed in the connection queue. Only used for direct connections. Default is 10.

Direct Configuration

A sample configuration for direct SQL queries is provided. Be sure to configure the following properties.

host
port
dbName
credentials.user
credentials.passwd

{
  "id": "mysql",
  "type": "MySQLDriver",
  "properties": {
    "host": "localhost",
    "port": 8888,
    "dbName": "peering_insights",
    "credentials": {
      "user": "pronghorn",
      "passwd": "$ENC87eb897b507afc1796db49409dd026188a832188a7d7439b"
    },
    "direct": true,
    "orm": false,
    "acquireTimeout": 10000,
    "waitForConnection": true,
    "connectionLimit": 10,
    "queueLimit": 10
  }
}

ORM Configuration

A sample configuration for object-relational mapping is provided. Be sure to configure the following properties:.

host
port
dbName
credentials.user
credentials.passwd

{
  "id": "mysql",
  "type": "MySQLDriver",
  "properties": {
    "host": "localhost",
    "port": 8888,
    "dbName": "peering_insights",
    "credentials": {
      "user": "pronghorn",
      "passwd": "$ENC87eb897b507afc1796db49409dd026188a832188a7d7439b"
    },
    "direct": false,
    "orm": true,
    "min": 3,
    "max": 7
  }
}

Configure NSO for SSL

The following steps provide instructions to configure SSL between NSO and IAP.

NSO SSL Configuration

Upload NSO host and CA X.509 certificates to the NSO host $CERT_DIR.

Copy the NSO host private key and signed certificate to the NSO configuration directory.

NCS_CONFIG_DIR=/etc/ncs
cp $NSO_SERVER_PRIV_KEY $NCS_CONFIG_DIR/ssl/cert/host.key
cp $CERT_DIR/$NSO_SERVER_CERTIFICATE $NCS_CONFIG_DIR/ssl/cert/host.cert

Trust the CA certificate and adjust for your platform. These instructions apply to a CentOS 7 host.
```
cp $CA_SELF_SIGNED_CERT /etc/pki/ca-trust/source
update-ca-trust
```
Edit the configuration file.
```
/etc/ncs/ncs.conf
```

Enable SSL for NSO REST and JSON RPC APIs.

<ncs-config xmlns="http://tail-f.com/yang/tailf-ncs-config">
  <webui>
    <transport>
      <ssl>
        <enabled>true</enabled>
        <ip>0.0.0.0</ip>
        <port>8888</port>
        <key-file>${NCS_CONFIG_DIR}/ssl/cert/host.key</key-file>
        <cert-file>${NCS_CONFIG_DIR}/ssl/cert/host.cert</cert-file>
      </ssl>
    </transport>
  </webui>
</ncs-config>

Enable NETCONF.

<ncs-config xmlns="http://tail-f.com/yang/tailf-ncs-config">
  <netconf-north-bound>
    <enabled>true</enabled>
    <transport>
      <ssh>
        <enabled>true</enabled>
        <ip>0.0.0.0</ip>
        <port>2022</port>
      </ssh>
      <tcp>
        <enabled>false</enabled>
        <ip>127.0.0.1</ip>
        <port>2023</port>
      </tcp>
    </transport>
  </netconf-north-bound>
</ncs-config>

Restart NSO.

Note: Refer to Cisco NSO Admin and Installation guides for detailed instructions on NSO SSL configuration.

Pronghorn NSO SSL Configuration

Upload the CA X.509 certificate to Pronghorn host $CERT_DIR.

CERT_DIR=/etc/ssl/certs
CA_SELF_SIGNED_CERT=${CERT_DIR}/certificate_authority.crt
NSO_HOST=nso1.lab.itential.com

Edit /opt/pronghorn/current/properties.json.

{
    "brokers": [
        "device",
        "method",
        "service"
    ],
    "groups": [],
    "id": "nso_host_name",
    "properties": {
        "authenticationStrategy": {
            "location": "NSO",
            "method": "tokenLogin",
            "type": "dynamic"
        },
        "commitQueue": false,
        "commitWait": 5000,
        "credentials": {
            "passwd": "nso_pass",
            "user": "nso_user"
        },
        "http": {
            "host": "nso_host_name",
            "port": 8888
        },
        "max_reconnect_attempts": 0,
        "max_reconnect_interval": 900,
        "min_reconnect_interval": 30,
        "netconf": {
            "host": "nso_host_name",
            "port": 2022,
            "protocol": "ssh"
        },
        "ssh": {
            "port": 22
        },
        "ssl": {
            "acceptInvalidCerts": false,
            "caFile": "/etc/ipa/ca.crt",
            "ciphers": "DHE-RSA-AES256-SHA",
            "enabled": true
        },
        "total_reconnect_window": 0
    },
    "type": "NSO"
}

Note: While updating the cipher in SSL properties, you can use the preferred cipher in your SSL handshake with the following command from IAP server:

$ openssl s_client -connect NSOServer.com(localhost):portNumber(8080)

In the example below, the preferred cipher is DHE-RSA-AES256-SHA.

SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: 1A1D51D03BFB624A617626DCEEDACAF6AD0317DF9C34DF7F55802BAA776EC112
    Master-Key: 4B7827941C
    Start Time: 1595966521
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)

Itential Automation Platform

Help us improve by sending feedback...

On this page:

Integrations

Network Adapters

Itential Ansible Manager

Adapter Properties

Authentication

Healthcheck

Throttle

Request

Proxy

SSL

Sample Configuration

Ansible Tower

Adapter Properties

Sample Configuration

NSO

Adapter Properties

Sample Configuration

Itential Tools Package

Commit Queues

NSO External Authentication Installation

NSO External Authentication Configuration

Configure Default Authgroup Mappings

Sample NSO NACM Rules

Install NACM Rules in NSO

Configure Device Whitelists

Configure Service Whitelists

Itential Tools Rules Engine

OSS Adapters

EAI

Properties

Sample Configuration

Service Now

Properties

Sample Configuration

SQL

Properties

Direct Configuration

ORM Configuration

Configure NSO for SSL

NSO SSL Configuration

Pronghorn NSO SSL Configuration