Streaming performance data with Operations Bridge - II
Co-authored by Roy Kallol, ITOM Engineering team
By now you should be aware of the performance management capability of the Operations Bridge suite. If you missed the first part of this series, which explains how you can stream metrics from a single node using Operations Agent policies, you can read it here.
This blog post provides information about how you stream metrics from many nodes - that have been collected by a third-party system - using the Operations Connector.
For this you can use any one of the following policy types for the integration of metrics / performance counters:
Database policies enable you to collect data from database tables used by third-party systems by performing a query through a JDBC connection.
Perl script policies enable you to collect data from systems by periodically running Perl scripts which perform the actual data collection.
Structure Log File policies read entries in a log file and matches the desired text against the log file structure fields defined by using the Operations Manager (OM) pattern-matching language. When the policy finds a match and certain conditions apply, the policy responds by sending metrics data.
REST Web service policies receive and process data from third-party systems through the Operations Connector REST Web service listener since collection is done outside.
XML file policies read values in XML files and respond when the value that you choose appears in the file. XML file policies are especially suited for integrating data from third-party systems that can store their information in XML format.
Micro Focus offers several Operations Connectors that provide metric collection out of the box, so that you don’t have to write the collection policy yourself.
The table below shows some of the out-of-the-box connectors along with the metric policy type they use.
Note: The Metric Streaming Configuration policy also allows you to collect metrics but it is strictly not advisable for third-party connector use cases or any other large scale integration solutions where the number of data points being collected/received is on a large scale. The Metric streaming configuration policy should be used only for integrating metrics from a single node.
Example: Operations Connector for SCOM metric streaming
Let’s do a deep dive for Operations Connector for SCOM. The below diagram shows the overall solution flow of third-party metric data streaming by configuring the end target as Performance Engine.
The Data Forwarding policy is defined in Operations Manager i and deployed to the managed connector nodes. The Operations Agent streams the metric data to the OMi Performance Engine which acts as the central place for storing and analyzing metrics in a large-scale environment. OMi Performance Dashboard reads the metric data from the OMi Performance Engine and displays the metrics on the performance dashboard.
Why is this not in real-time for third-party connectors?
Data streamed from third-party connectors is never in real time. (Figures will depend on specific implementation characteristics. Please refer to the Performance and Sizing Guide for details.) The lack of real-time results is due to the fact that the data is from a third-party connector and the Operations connector doesn’t have control on the collection performed by the domain manager (even if collection is done in pull or push mode). There is always a gap between the time the data was collected and the time the data is displayed on the Performance Dashboard.
- Configure the SCOM Domain Manager aka SCOM server
- Install BSM/Ops Connector and Ops Connector for Microsoft SCOM
For SCOM REST Web service listener policies process the data received by the Ops Connector REST Web service listener.
- Make sure that Operations Agent version 12.03 or later is installed
Note: Operations Connector by default comes with Operations Agent installed.
- Disable the local metrics store
Since you are storing the metric data in the central store from which the Performance Dashboard gets the data for graphing, you don’t have to duplicate it by storing it on the Ops Connector node.
By disabling data logging to the local store, you can avoid performance degradation of Operations Agent’s collector component “OACORE”.
It’s advisable NOT to log more than 100 data points per second into the local metric store.
As the Operations Connector for SCOM integrates metrics data for many nodes, the local metric store should be disabled.
(Number of data points = Number of Managed Nodes x Number of Classes x Number of Metrics x Number of Instances)
Execute the following command to set OPCGENI_DISABLE_LOCAL_STORE to “true”
ovconfchg -ns eaagt -set OPCGENI_DISABLE_LOCAL_STORE true
Once the variable is set, restart the Operations Agent.
After setting the parameter to true, the local data logging of METRICS into the SQLite DB of Operations Agent is disabled. This affects all METRIC integration policies using the policy types Database, Perl, Structured Log file, Rest Web Service Listener or XML file.
Other policy types (for metric, event or topology integration) are not affected by this setting.
- Edit and Configure the Data Forwarding policy
With Data Forwarding policy, the data points are being streamed by OPCGENI. Whereas with Metric streaming configuration policy, the data is streamed by HPSENSOR.
You can also define the rules on what data is sent to which target or to discard specific data for some targets. The example shown below forwards metric data corresponding to data Domain named BsmIntSCOMMetrics.
6. Configure the Performance Engine Node details from which Performance Dashboard must request data.
7. Create a SCOM Performance Dashboard in OMi
The SCOM connector also integrates topology into OMi. Select a CI created by the SCOM topology integration (for example a node CI), then select the Data source OpsBridge Store. You should then be able to select a Class Name, followed by Metric Name and Instance Name of metrics that have been forwarded to the OpsBridge Store aka the Operations Bridge Performance Engine.
Note: Depending on the type of Operations Connector (out of the box), install the OMi Content Pack (which includes Topology Synchronization files, performance dashboard and other configuration) for the respective Ops Connector (if available) followed by enabling the respective Topology Integration policies. Once done, select a view containing the CI for creating the dashboard.
Graph plotted from historical data
You can save the respective performance dashboard as a ready-to-use template for the future.
Large scale metric streaming using multiple collectors
Using the same concept and configuration steps as above, performance data can be streamed also from multiple third-party systems, such as SCOM and Tivoli.
Note: The below are the minimum Operations Bridge capability versions required to use the metric streaming functionality to stream third-party data.
- Operations Manager i (includes OMi Performance Dashboard) 11
- OMi Performance Engine 11
- Operations Agent 03
- Respective Operations Connector
To get more information on this release and how customers are using Operations Bridge we are happy to announce the following events you can register for
Vivit event – Operations Special Interest Group Webinar – 11th October 2017 – Register here
ITOM Customer Forum, Copenhagen – October 3rd 2017 - Register here
ITOM Customer Forum, Stockholm – October 4th 2017 - Register here
ITOM Customer Forum, London – October 5th 2017 - Register here
ITOM Customer Forum, Brussels – December 7th 2017 - coming soon
Read all our news at the OpsBridge blog
Explore all the capabilities of the Operations Bridge Suite and technology integrations by visiting these sites:
- Operations Bridge Suite
- Operations Manager i
- Operations Bridge Technology Integrations eBook
- Operations Bridge Integrations page
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
- Carol Park on: Operations Bridge: What’s new in the November release
- RolfF on: Operations Manager Support Extension & Evolution to OpsBridge Q&A Webinar
- Vivek Achar on: Simplify NetApp 3-way NDMP backup with Data Protector 10.03
- Jeff_Tokar on: Launching ITOM Idea Exchange!
- Carol Park on: Sending custom metrics to Business Value Dashboard in 5 easy steps
___ on: Code your infrastructure: Use APIs to script service consumption from CSA and HCM Suite
- rlinnellsuse on: Introducing the Micro Focus ITOM Platform
- VolkerGaertner on: What’s new in Operations Bridge Suite 2017.11
- DerekW on: Announcing Micro Focus Network Operations Management Suite 2017.11: 5 new key capabilities
- Brett Simpson_1 on: How to take advantage of event-driven ad hoc downtime manage