ArcSight User Discussions
cancel

Multiple ESM on one Eventbroker?

Knowledge Partner
Knowledge Partner

Multiple ESM on one Eventbroker?

So from what i understand, is there still a limit that only one binary topic (eb-esm) can exist on one Eventbroker? Is there any way for ESM to consume non binary topics?

If both is not possible, is it any possibility to route half of the topic to one ESM, and the rest to another?

For large enterprises the limit of 1M EPS of the EB is great, but hard to justify when the receiving ESM has a limit of 100k, and would easily be fixed by adding more ESM's with the 100k limit to one EB, is this still not possible? And if not is there any plans to improve upon this?

3 REPLIES
Community Manager COEST Community Manager
Community Manager

Re: Multiple ESM on one Eventbroker?

Hey! Doing my best to find you the right answer! Stay tuned!

Highlighted
Community Manager COEST Community Manager
Community Manager

Re: Multiple ESM on one Eventbroker?

Marius - I got a response from one of our PMs, hope this helps - if still questions, please let me know!

--------------------------------------------

So from what i understand, is there still a limit that only one binary topic (eb-esm) can exist on one Eventbroker? YES

Is there any way for ESM to consume non binary topics? NO

If both is not possible, is it any possibility to route half of the topic to one ESM, and the rest to another? NO

For large enterprises the limit of 1M EPS of the EB is great, but hard to justify when the receiving ESM has a limit of 100k, and would easily be fixed by adding more ESM's with the 100k limit to one EB, is this still not possible? CORRECT And if not is there any plans to improve upon this? WE ARE INVESTIGATING SOLUTIONS - will keep you posted!

Knowledge Partner
Knowledge Partner

Re: Multiple ESM on one Eventbroker?

The issue here comes towards the target demographic, while EventBroker is an awesome tool, it might be a bit "overkill" for someone consuming maybe 10k EPS, but if you are moving into 100-500k EPS, like MSSP's or very large environments, it starts becoming a need-to-have.

Now the issue shows up, since these large customers needs maybe 10 ESM's currently to both separate and ingest data, and want to utilize Distributed Correlation, and tbh it is mostly useful when EventBroker is in front.

Currently since the limit is 100k for each ESM, or you might still be hesitant to take the step either because you are going to go above that limit, or want to still keep your data separated.

This means that you need to run one eventbroker per ESM installation, and if you want it highly redundant that means at least 6 machines (3 masters and 3 workers), 5-10 esm's then that is 60 servers already, not including distributed correlation.

Eventbroker is utilizing the functionality from Apacke Kafka (and some Kafka/Confluent i am sure), and these both support multiple binary topics, so i am a bit unsure how this ended up being a constraint on the eventbroker itself?

A Kafka connector would solve the issue temporarily, as no ESM would ingest directly from EB, but rather have one connector per ESM consuming from different non binary topics, but you then say goodbye to half of the reason EventBroker is so good, mainly to manage your data transport. This is because the connectors still have quite a low EPS rate maximum.

Hopefully maybe there is some more long term plans on this, so it could fit the largest customers as well :)

Top Contributors Last 30 Days