dp3.api.routers.entity ¶
check_etype
async
¶
Validate the entity type as a FastAPI dependency.
parse_eid
async
¶
Parse a path EID as a FastAPI dependency.
Source code in dp3/api/routers/entity.py
parse_eid_from_query
async
¶
Parse EID from query parameter (for v2 API).
Source code in dp3/api/routers/entity.py
get_eid_master_record_handler ¶
get_eid_master_record_handler(e: EntityId, date_from: AwareDatetime | None = None, date_to: AwareDatetime | None = None)
Handler for getting master record of EID
Source code in dp3/api/routers/entity.py
get_eid_snapshots_handler ¶
get_eid_snapshots_handler(e: EntityId, date_from: AwareDatetime | None = None, date_to: AwareDatetime | None = None, skip: int = 0, limit: int = 0) -> list[dict[str, Any]]
Handler for getting snapshots of EID.
Source code in dp3/api/routers/entity.py
get_eid_data_handler ¶
get_eid_data_handler(e: EntityId, date_from: AwareDatetime | None = None, date_to: AwareDatetime | None = None) -> EntityEidData
Handler for getting all data of EID.
Source code in dp3/api/routers/entity.py
get_eid_attr_value_handler ¶
get_eid_attr_value_handler(e: EntityId, attr: str, attr_location: str, date_from: AwareDatetime | None = None, date_to: AwareDatetime | None = None) -> EntityEidAttrValueOrHistory
Handler for getting an attribute value or history of EID.
Source code in dp3/api/routers/entity.py
set_eid_attr_value_handler
async
¶
set_eid_attr_value_handler(e: EntityId, attr: str, attr_location: str, request: Request) -> SuccessResponse
Handler for setting the current value of an EID attribute.
Source code in dp3/api/routers/entity.py
extend_eid_ttls_handler ¶
Handler for extending TTLs of EID.
Source code in dp3/api/routers/entity.py
delete_eid_record_handler ¶
Handler for deleting the master record and snapshots of EID.
Source code in dp3/api/routers/entity.py
get_entity_type_eids
async
¶
get_entity_type_eids(etype: str, fulltext_filters: Json = None, generic_filter: Json = None, skip: NonNegativeInt = 0, limit: NonNegativeInt = 20, sort: Annotated[list[str] | None, Query(description='example: hostname:-1 for descending sort by hostname')] = None) -> EntityEidList
List latest snapshots of all ids present in database under etype.
Contains only latest snapshot.
Uses pagination.
Setting limit to 0 is interpreted as no limit (return all results).
Returns only documents matching generic_filter and fulltext_filters
(JSON object in format: attribute - fulltext filter).
Fulltext filters are interpreted as regular expressions.
Only string values may be filtered this way. There's no validation that queried attribute
can be fulltext filtered.
Only plain and observation attributes with string-based data types can be queried.
Array and set data types are supported as well as long as they are not multi value
at the same time.
If you need to filter EIDs, use attribute fulltext_filters["eid"].
Generic filter allows filtering using generic MongoDB query (including $and, $or,
$lt, etc.).
For querying non-JSON-native types, you can use the following magic strings:
"$$IPv4{<ip address>}"- converts to IPv4Address object"$$IPv6{<ip address>}"- converts to IPv6Address object"$$int{<value>}"- may be necessary for filtering wheneiddata type is int"$$Date{<YYYY-mm-ddTHH:MM:ssZ>}"- converts specified UTC date to UTC datetime object"$$DateTs{<POSIX timestamp>}"- converts POSIX timestamp (int/float) to UTC datetime object"$$MAC{<mac address>}"- converts to MACAddress object
To query an IP prefix, use the following magic strings:
"$$IPv4Prefix{<ip address>/<prefix length>}"- matches prefix"$$IPv6Prefix{<ip address>/<prefix length>}"- matches prefix
To query a binary _ids of non-string snapshot buckets,
use the following magic string:
-
"$$Binary_ID{<EID object | Valid magic string>}"- converts to filter the exact EID object snapshots, only EID valid types are supported
There are no attribute name checks (may be added in the future).
Generic filter examples:
{"attr1": {"$gt": 10}, "attr2": {"$lt": 20}}{"ip_attr": "$$IPv4{127.0.0.1}"}- converts to IPv4Address object, exact match-
{"ip_attr": "$$IPv4Prefix{127.0.0.0/24}"}- converts to
{"ip_attr": {"$gte": "$$IPv4{127.0.0.0}", "$lte": "$$IPv4{127.0.0.255}"}}
- converts to
-
{"ip_attr": "$$IPv6{::1}"}- converts to IPv6Address object, exact match -
{"ip_attr": "$$IPv6Prefix{::1/64}"}- converts to
{"ip_attr": {"$gte": "$$IPv6{::1}", "$lte": "$$IPv6{::1:ffff:ffff:ffff:ffff}"}}
- converts to
-
{"_id": "$$Binary_ID{$$IPv4{127.0.0.1}}"}- converts to
{"_id": {"$gte": Binary(<IP bytes + min timestamp>), "$lt": Binary(<IP bytes + max timestamp>)}}
- converts to
-
{"id": "$$Binary_ID{$$IPv4Prefix{127.0.0.0/24}}"}- converts to
{"_id": {"$gte": Binary(<IP 127.0.0.0 bytes + min timestamp>), "$lte": Binary(<IP 127.0.0.255 bytes + max timestamp>)}}
- converts to
-
{"_time_created": {"$gte": "$$Date{2024-11-07T00:00:00Z}"}}- converts to
{"_time_created": datetime(2024, 11, 7, 0, 0, 0, tzinfo=timezone.utc)}
- converts to
-
{"_time_created": {"$gte": "$$DateTs{1609459200}"}}- converts to
{"_time_created": datetime(2021, 1, 1, 0, 0, 0, tzinfo=timezone.utc)}
- converts to
-
{"attr": "$$MAC{00:11:22:33:44:55}"}- converts to MACAddress object, exact match {"_id": "$$Binary_ID{$$MAC{Ab-cD-Ef-12-34-56}}"}- converts to
{"_id": {"$gte": Binary(<MAC bytes + min timestamp>), "$lt": Binary(<MAC bytes + max timestamp>)}}
- converts to
There are no attribute name checks (may be added in the future).
Generic and fulltext filters are merged - fulltext overrides conflicting keys.
Sorting is supported by entity ID using the special attribute name eid,
as well as for plain and observations attributes with primitive data types
(excluding json and multi_value observations). To sort by multiple attributes,
provide multiple sort parameters in the format 'attribute:direction' where
direction is 1 (ascending) or -1 (descending). Direction defaults to 1 (ascending)
if not provided. Examples:
?sort=eid:1, ?sort=hostname:-1&sort=ip:1
Source code in dp3/api/routers/entity.py
321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 | |
count_entity_type_eids
async
¶
count_entity_type_eids(etype: str, fulltext_filters: Json = None, generic_filter: Json = None) -> EntityEidCount
Count latest snapshots of all ids present in database under etype.
Returns only count of documents matching generic_filter and fulltext_filters,
see /entity/{etype}/get documentation for details.
Note that responses from this endpoint may take much longer than /entity/{etype}/get
for large datasets.
Source code in dp3/api/routers/entity.py
get_entity_type_raw_datapoints
async
¶
get_entity_type_raw_datapoints(etype: str, eid: str | None = None, attr: str | None = None, src: str | None = None, skip: NonNegativeInt = 0, limit: NonNegativeInt = 20) -> EntityRawDataPage
List raw datapoints from the current raw collection for troubleshooting ingestion.
Source code in dp3/api/routers/entity.py
get_eid_data
async
¶
get_eid_data(e: ParsedEid, date_from: AwareDatetime | None = None, date_to: AwareDatetime | None = None) -> EntityEidData
Get data of the entity identified by etype and eid.
Contains all snapshots and master record. Snapshots are ordered by ascending creation time.
Combines the master-record and snapshots endpoints.
Source code in dp3/api/routers/entity.py
get_eid_master_record
async
¶
get_eid_master_record(e: ParsedEid, date_from: AwareDatetime | None = None, date_to: AwareDatetime | None = None) -> EntityEidMasterRecord
Get the master record of the entity identified by etype and eid.
Source code in dp3/api/routers/entity.py
get_eid_snapshots
async
¶
get_eid_snapshots(e: ParsedEid, date_from: AwareDatetime | None = None, date_to: AwareDatetime | None = None, skip: NonNegativeInt = 0, limit: NonNegativeInt = 0) -> EntityEidSnapshots
Get snapshots of the entity identified by etype and eid.
Supports optional pagination via skip and limit.
Setting limit to 0 returns all matching snapshots.
Source code in dp3/api/routers/entity.py
get_eid_attr_value
async
¶
get_eid_attr_value(e: ParsedEid, attr: str, date_from: AwareDatetime | None = None, date_to: AwareDatetime | None = None) -> EntityEidAttrValueOrHistory
Get attribute value
Value is either of: - current value: in case of plain attribute - current value and history: in case of observation attribute - history: in case of timeseries attribute
Source code in dp3/api/routers/entity.py
set_eid_attr_value
async
¶
Set current value of attribute
Internally just creates datapoint for specified attribute and value.
This endpoint is meant for editable plain attributes -- for direct user edit on DP3 web UI.
Source code in dp3/api/routers/entity.py
get_distinct_attribute_values
async
¶
Gets distinct attribute values and their counts based on latest snapshots
Useful for displaying <select> enumeration fields.
Works for all plain and observation data types except dict and json.
Source code in dp3/api/routers/entity.py
extend_eid_ttls
async
¶
Extend TTLs of the specified entity
delete_eid_record
async
¶
Delete the master record and snapshots of the specified entity.
Notice that this does not delete any raw datapoints, or block the re-creation of the entity if new datapoints are received.