NOTICE: Branded Content
NOTICE: Certain versions of content (“Material”) accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.
ArcSight User Discussions
cancel

Difference between a Smart Connector and Smart Collector

SOLVED
Go to solution
Respected Contributor... atulsm Respected Contributor...
Respected Contributor...

Difference between a Smart Connector and Smart Collector

In ArcMC web interface and in docs, i see reference to Smart Collector. Unfortunately i cannot find an explaination which distinguishes collector with connector. 

Technically, what is the difference between a SmartConnector and a SmartCollector and where do we use collector ?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Respected Contributor... viktor.doundako Respected Contributor...
Respected Contributor...
Solution

Re: Difference between a Smart Connector and Smart Collector

To undersand the Collectors v.s Connectors, we need to step back and look at what the SmarConnectors do.

Conceptually, the standard SmartConnectors have two main responsibilties: "Collect" raw data from various sources, and "Process" the collected data to become enriched security events and post them to a destination.

Introduced in ADP 2.30, customers can take advantage of the massive scalabilty and robustness of the Event Broker infrastructure, and move the computationaly intensive "Process" step to the highly scalable and more robust Event Broker streaming infrastructure.

This is done by using syslog Colelctors and syslog CEBs: Collectors are standalone compnents very similar to the SmartConenctors, but they only "Collect" raw syslog data like the syslog SmartConnectors do, wrap it up and post it to a dedicated eb-con-syslog topic in Event Broker. 

At that point, the Event Broker's CEB stream processors (CEB stands for Connector in Event Broker) read the data from the eb-con-syslog topic, do the parsing/normalization/enrichment/filtering processing (as the standalone SmartConnectors destination pipelines do) and post the security events on the EB topics for consumption.

In other words: as their name suggests, the syslog Collectors are lightweight component responsible for collecting raw syslog data and passing it to Event Broker for processing.

Main advantages of the new architecture:

  1. Potential for hardware consolidation and data throughput increase in the data collection layer where the Collectors are deployed: due to moving the processing to the EB streaming infrastructure.
  2. Improved stabilty and easy horizontal scalability as the data flows increase with time, or fluctuate during operations: CEBs are deployed or undeployed on the EB nodes with a single click in the ArcMC UI.
  3. Reduced network traffic due to a single data feed to Event Broker, instead of having tmultiple destinations coming from SmartConnectors
  4. The raw Syslog data is now available on the EB topic for any system that customer would like to share it with.

Note that at this time Colectors and CEBs are only available for Syslog data.

7 REPLIES
Knowledge Partner
Knowledge Partner

Re: Difference between a Smart Connector and Smart Collector

I think those are part of some legacy setup, Collectors would collect data from sources (as we do with windows event connectors for example, or a fileconnector), where the connector itself is actually fetching the data instead of listening on some port.

I never seen or used these Collectors, so i do not think you need to think about needing them for any new implementations. Though i might be wrong, and if so, anyone is free to correct me :)

alexandros_n Honored Contributor.
Honored Contributor.

Re: Difference between a Smart Connector and Smart Collector

Smartcollectors are the "connectors" (or something like that) to work with Event Broker.

Respected Contributor... atulsm Respected Contributor...
Respected Contributor...

Re: Difference between a Smart Connector and Smart Collector

Smart connectors can also be used to connect with eventbroker.

evknott1 Super Contributor.
Super Contributor.

Re: Difference between a Smart Connector and Smart Collector

This is from memory from a presentation at the last Protect Conference.

SmartCollector is in BETA and is part of a plan to move the parser functionality of the SmartConnectors into Event Broker.

The SmartCollector would perform the receipt of the events but do no parsing.  It would pass them to SmartConnector(s) in Event Broker where the parsing would take place.  From my understanding, this would allow us to reduce the network consumption between the SmartConnector and Event Broker if going to both Logger and ESM and allow higher EPS rates at each SmartCollector (as parsing would be handled in the Event Broker SmartConnector).

 

Respected Contributor... viktor.doundako Respected Contributor...
Respected Contributor...

Re: Difference between a Smart Connector and Smart Collector

Very close. :-)

Now those compoents are generally available.

Highlighted
Respected Contributor... viktor.doundako Respected Contributor...
Respected Contributor...
Solution

Re: Difference between a Smart Connector and Smart Collector

To undersand the Collectors v.s Connectors, we need to step back and look at what the SmarConnectors do.

Conceptually, the standard SmartConnectors have two main responsibilties: "Collect" raw data from various sources, and "Process" the collected data to become enriched security events and post them to a destination.

Introduced in ADP 2.30, customers can take advantage of the massive scalabilty and robustness of the Event Broker infrastructure, and move the computationaly intensive "Process" step to the highly scalable and more robust Event Broker streaming infrastructure.

This is done by using syslog Colelctors and syslog CEBs: Collectors are standalone compnents very similar to the SmartConenctors, but they only "Collect" raw syslog data like the syslog SmartConnectors do, wrap it up and post it to a dedicated eb-con-syslog topic in Event Broker. 

At that point, the Event Broker's CEB stream processors (CEB stands for Connector in Event Broker) read the data from the eb-con-syslog topic, do the parsing/normalization/enrichment/filtering processing (as the standalone SmartConnectors destination pipelines do) and post the security events on the EB topics for consumption.

In other words: as their name suggests, the syslog Collectors are lightweight component responsible for collecting raw syslog data and passing it to Event Broker for processing.

Main advantages of the new architecture:

  1. Potential for hardware consolidation and data throughput increase in the data collection layer where the Collectors are deployed: due to moving the processing to the EB streaming infrastructure.
  2. Improved stabilty and easy horizontal scalability as the data flows increase with time, or fluctuate during operations: CEBs are deployed or undeployed on the EB nodes with a single click in the ArcMC UI.
  3. Reduced network traffic due to a single data feed to Event Broker, instead of having tmultiple destinations coming from SmartConnectors
  4. The raw Syslog data is now available on the EB topic for any system that customer would like to share it with.

Note that at this time Colectors and CEBs are only available for Syslog data.

Respected Contributor... atulsm Respected Contributor...
Respected Contributor...

Re: Difference between a Smart Connector and Smart Collector

Thanks viktor, this is very clear now !

What is the format of the data going into eb-con-syslog topic. Is it CEF or Syslog (RFC 5424).

Key Links

Micro Focus is looking for research study participants to help us improve our website. Tell us what you think, and in exchange for your time we'll send you a small thank-you, like a gift card. Sign up or learn more 

Top Contributors Last 30 Days