This SmartConnector doesn't seem to map the "host" field from the Bro HTTP log.
This matches the documentation, which doesn't list that field in the "Bro IDS HTTP Log Mappings" section, but it seems like a pretty glaring omission.
The "uri" field is mapped to "Request URL," but that contains only the relative URI and does not include the host name.
It appears the RequestUrl field is not being populated correctly. The requestProtocol, requestUrlAuthority,requestUrlHost, requestUrlPort, requestUrlFilename, and requestUrlQuery are all populated from a correctly formatted requestUrl: <protocol>://<authority>@<host>:<port>/<filename>?<query>. (per ArcSight dcumentation).
I have brought this to the attention of the connector development group and this will be addressed in a future sprint.
This will be addressed by connector development in a future SmartConnector release; I do not currently have a solid date.
The new connector does a much better job on the requestUrl and now populating the other request fields. Thanks.
However, the problem of the connector not being able to follow the folder when Bro periodically creates new ones is still present.
While I see the request fields are now populating with data, the Request URL field does not appear to be parsing correctly. It is duplicating some of the URL and in some cases, it leaves the "-" (found in bro's logs for blank fields). Examples:
Larry, are you seeing this as well?
Yes, I'm seeing the same occurrences. I can empathize though having gone through the effort of creating a flex connector for Bro. Bro has done some good things by breaking out the various parts of a URL but in this case it would be better to have it complete vice parsed. As to the hyphens in the data, they are both a benefit and a nuisance. While it is good to know sometimes that there was nothing in a specific column, it raises havoc when you try to tell the flex connector a certain column is an IP address and then a "-" shows up.....
While it is always good to have accuracy in data (requestUrl in this case), I will settle for all the necessary data being present vice exactly correct.
With the new connector version, I am more concerned with not being able to have the connector continue to follow the Bro log folder after it rotates the logs. I am currently having to re-start the connector on an hourly basis (the same interval Bro creates new log files) in order for the connector to continue finding the log files. I don't understand why this is an issue since the Multi-Folder Flex connector I created followed the folders fine. So the mechanism to do this must exist in the connector bag of goodies. I just need to know the magic configuration to make it work.
Documentation update request:
pg. 7 - List of Selected Log Types has 2 misspellings: Please update:
Thanks for your comment. We've made the corrections and you'll see them in the guide with our Q1R2 release the end of March.
Are there any new updates on this connector?
As of February 2016 release, Bro IDS Host Name has been added as an installation parameter, and no longer has to be configured separately.