Feature #5278
closedImplement more complex object data services
100%
Description
In task #5067 basic connection with NERD system based on links was implemented. It would be very useful to implement more object data services and use them in IDEA event detail view to automagically fettch additional data for objects like IP addressess. Following modules could be implemented.
- DNS resolver
- PassiveDNS resolver
Related issues
Updated by Jan Mach over 5 years ago
- Status changed from New to In Progress
Work in progress can be revieved here:
https://mentat-alt.cesnet.cz/mentat/events/a86158e5-bfab-4633-bc5f-7099cbf71eb8/show
https://mentat-alt.cesnet.cz/mentat/dnsr/search?search=195.113.144.233&submit=Search
https://mentat-alt.cesnet.cz/mentat/pdnsr/search?search=195.113.144.233&submit=Search
https://mentat-alt.cesnet.cz/mentat/nerd/search?search=193.32.163.89&submit=Search
Updated by Jan Mach over 5 years ago
- Status changed from In Progress to Feedback
- Assignee changed from Jan Mach to Pavel Kácha
In case you want to see what is cooking;-)
Updated by Jan Mach over 5 years ago
- Related to Feature #5067: Add link to NERD to IP address context menus added
Updated by Jan Mach over 5 years ago
Latest patch replaces information icon about reaching limit with clickable load button, that enables users fetch not yet fetched data on demand.
You may try it here:
https://mentat-alt.cesnet.cz/mentat/events/a86158e5-bfab-4633-bc5f-7099cbf71eb8/show
In case of weird behavior make sure to load fresh JavaScript files by opening developer console and reloading the page.
Updated by Pavel Kácha over 5 years ago
- Assignee changed from Pavel Kácha to Jan Mach
Very coool. Appreciate the work that went into it. Limit for number of items is per IP or per queried service? Also, some reasonable caching would make sense (shall I make new issue?)
Updated by Jan Mach over 5 years ago
Pavel Kácha wrote:
Very coool. Appreciate the work that went into it. Limit for number of items is per IP or per queried service?
Limit is set for (Source/Target).IP, current value is 10. Data services may register hooks for object types they are capable of handling (IP4/6,hostname,...). Additional data are then fetched from all registered services for that type. For example in case an IDEA message has 15 Source.IP4s, it will result in 10x5 API queries because there are 5 services registered to handle IP4s. If there are any Target.IP4, they will have separate limit counter. The idea behind this is to resolve some number for each object type, but not to overwhelm the service APIs.
Also, some reasonable caching would make sense (shall I make new issue?)
Yes, that is absolutely necessary before releasing it to production. There is no need for a new issue in my opinion.
Updated by Pavel Kácha over 5 years ago
Jan Mach wrote:
Pavel Kácha wrote:
Very coool. Appreciate the work that went into it. Limit for number of items is per IP or per queried service?
Limit is set for (Source/Target).IP, current value is 10. Data services may register hooks for object types they are capable of handling (IP4/6,hostname,...). Additional data are then fetched from all registered services for that type. For example in case an IDEA message has 15 Source.IP4s, it will result in 10x5 API queries because there are 5 services registered to handle IP4s. If there are any Target.IP4, they will have separate limit counter. The idea behind this is to resolve some number for each object type, but not to overwhelm the service APIs.
Ok, makes sense.
Also, some reasonable caching would make sense (shall I make new issue?)
Yes, that is absolutely necessary before releasing it to production. There is no need for a new issue in my opinion.
Ack. Thx.
Seems to me Targets don't get info from DNS/PassiveDNS, that's intentional?
Updated by Jan Mach over 5 years ago
Pavel Kácha wrote:
Jan Mach wrote:
Pavel Kácha wrote:
Very coool. Appreciate the work that went into it. Limit for number of items is per IP or per queried service?
Limit is set for (Source/Target).IP, current value is 10. Data services may register hooks for object types they are capable of handling (IP4/6,hostname,...). Additional data are then fetched from all registered services for that type. For example in case an IDEA message has 15 Source.IP4s, it will result in 10x5 API queries because there are 5 services registered to handle IP4s. If there are any Target.IP4, they will have separate limit counter. The idea behind this is to resolve some number for each object type, but not to overwhelm the service APIs.
Ok, makes sense.
Also, some reasonable caching would make sense (shall I make new issue?)
Yes, that is absolutely necessary before releasing it to production. There is no need for a new issue in my opinion.
Ack. Thx.
Seems to me Targets don't get info from DNS/PassiveDNS, that's intentional?
They do, there is just an empty result. You can see for yourself by opening the developer console and checking console logs. All services are queried for all objects except for NERD, which does not support IPv6.
Updated by Pavel Kácha over 5 years ago
Jan Mach wrote:
Pavel Kácha wrote:
Seems to me Targets don't get info from DNS/PassiveDNS, that's intentional?
They do, there is just an empty result. You can see for yourself by opening the developer console and checking console logs. All services are queried for all objects except for NERD, which does not support IPv6.
Mmmkay, my bad.
Updated by Jan Mach over 5 years ago
- Status changed from Feedback to Closed
- % Done changed from 50 to 100
I consider current state as good enough for pilot operation. Further improvements will be done in separate issues.