4.4.7.2. From CSV files¶

Parser for AQL queries and GUI expressions from CSV files.

class axonius_api_client.api.wizards.wizard_csv.WizardCsv(apiobj, log_level='debug')[source]¶

Bases: Wizard

Parser for AQL queries and GUI expressions from CSV files.

Notes

This wizard can create as many saved queries as you like. The first row of the CSV must always have a type value of “saved_query”. All rows under that row will be added to that saved query until another row with a type value of “saved_query” is found. Repeat this pattern as you see fit.

The description, tags, and fields columns are only used for types of “saved_query”

Examples

Create a client using axonius_api_client.connect.Connect and assume apiobj is either client.devices or client.users

>>> apiobj = client.devices  # or client.users

Define a CSV string to parse

>>> content = '''
... type,value,description,tags,fields
... saved_query,name of sq,description  of sq,"tag1,tag2, tag4","os.type,default, aws:aws_device_type"
... simple,hostname contains test,,,
... simple,os.type equals windows,,,
... saved_query,name of second sq,description  of sq,"tag1,tag2, tag4",
... simple,hostname contains test,,,
... complex,installed_software // name contains chrome // version earlier_than 82,,,
... simple,os.type equals windows,,,
... '''

Parse the CSV string into a query and GUI expressions

>>> parsed = apiobj.wizard_csv.parse(content=content)
>>> for sq in parsed:
...   sq['name']
...   sq['query'][:80]
...   len(sq['expressions'])
...
'name of sq'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.os.t'
2
'name of second sq'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.inst'
3

Or parse a CSV file directly

>>> parsed = apiobj.wizard_csv.parse(path="~/test.csv")

Use the result to create saved queries that the GUI understands

>>> sqs = [apiobj.saved_query.add(**sq) for sq in parsed]
>>> for sq in sqs:
...    sq['name']
...    sq['uuid']
...    sq['view']['query']['filter'][:80]
...
'name of sq'
'5f79e90be4557d5cbab26344'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.os.t'
'name of second sq'
'5f79e90be4557d5cbab26359'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.inst'

Parameters: log_level (typing.Union[str, int]) –

DOCS: str = '\ntype,value,description,tags,fields,asset_scope,private,always_cached,gui_page_size\n"# If type column is empty or begins with # it is ignored",,,,\n"# type of simple or complex will belong to the saved_query they are under",,,,\n"# Column descriptions for type of saved_query","Name of Saved Query","Description of Saved Query","Tags to apply to Saved Query","Columns to display in Saved Query",Optional: True or False (default False if empty),Optional: True or False (default False if empty),Optional: True or False (default False if empty),"20 or 50 or 100 (default 20 if empty)",\n"# Column descriptions for type of simple","Format -- [] represents optional items: [& | ! ( )] FIELD OPERATOR VALUE [)]","Description: Filter entry for simple fields","Only uses columns type and value",\n"# Column descriptions for type of complex","Format -- [] represents optional items: [& | ! ( )] COMPLEX-FIELD // SUB-FIELD OPERATOR VALUE[ // ...] [)]","Description: Filter entry for complex fields and their sub-fields","Only uses columns type and value",\n"# Value Flags for type of simple or complex","& Use and instead of or (default), | Use or instead of and (overrides &), ! Use not, ( Open a parentheses, ) Close a parentheses (can also be at end of entry)",,,\n"saved_query","example 1","Filters, default fields, custom fields","example,tag1,tag2","os.distribution,os.os_str,active_directory:ad_password_last_set,default,os.build"\n"simple","( hostname contains test",,,\n"simple","! hostname contains internal )",,,\n"simple","( os.type equals windows",,,\n"simple","| os.type equals os x )",,,\n"saved_query","example 2","No filters, no default fields, custom fields","example,tag3,tag4","os.distribution,os.os_str,active_directory:ad_password_last_set"\n"saved_query","example 3","No filters, default fields, no custom fields","example,tag5,tag6",\n'¶

parse(content, source='csv text string')[source]¶

Parse a CSV string into a set of saved queries.

Parameters

content (str) – CSV string
source (str) – where content came from

Return type

typing.List[dict]

parse_path(path, source='csv file {path}')[source]¶

Parse a CSV file into a set of saved queries.

Parameters

path (typing.Union[str, pathlib.Path]) – CSV file
source (str) – where csv file came from

Return type

typing.List[dict]

_load_csv(content, source)[source]¶

Load a CSV string and parse the rows into entries.

Parameters

content (str) – CSV string
source (str) – where content came from

Return type

typing.List[dict]

_process_csv(rows, columns, source)[source]¶

Process and validate the rows and columns from a CSV into entries.

Parameters

rows (typing.List[dict]) – rows from the CSV
columns (typing.List[str]) – columns from the CSV
source (str) – where content came from

Return type

typing.List[dict]

_rows_to_entries(rows, source)[source]¶

Process and validate the rows from a CSV into entries.

Parameters

rows (typing.List[dict]) – rows from the CSV
source (str) – where content came from

Return type

typing.List[dict]

_row_to_entry(row, src)[source]¶

Proccess and validate a row from a CSV into an entry.

Parameters

row (dict) – row from the CSV
src (str) – identifier of where the row came from

Return type

dict

_process_sqs(entries)[source]¶

Process all of the saved queries defined in the CSV.

Parameters: entries (typing.List[dict]) – the entries produced by parsing the rows
Return type: typing.List[dict]

_process_sq(entry, is_last)[source]¶

Process a saved query.

Parameters

entry (dict) – entry being processed by _process_sqs()
is_last (bool) – entry is the last entry being processed by _process_sqs()

Return type

int

_process_sq_entries()[source]¶: Process the entries for the current saved query.

_new_sq(entry)[source]¶

Create a new current saved query.

Parameters: entry (dict) – entry being processed by _process_sqs()

_process_desc(entry)[source]¶

Process the description key of an entry.

Parameters: entry (dict) – entry being processed by _process_sqs()
Return type: typing.Optional[str]

_process_tags(entry)[source]¶

Process the tags key of an entry.

Parameters: entry (dict) – entry being processed by _process_sqs()
Return type: typing.Optional[typing.List[str]]

_process_fields(entry)[source]¶

Process the fields key of an entry.

Parameters: entry (dict) – entry being processed by _process_sqs()
Return type: [List[str]]

_init()[source]¶: Post init setup.

__init__(apiobj, log_level='debug')¶

Query wizard builder.

Parameters

apiobj (axonius_api_client.api.assets.asset_mixin.AssetMixin) – Asset object
log_level (typing.Union[str, int]) – logging level for this object

_check_entry_keys(entry, keys)¶

Check that an entry has the required keys.

Parameters

entry (dict) – entry to check keys against
keys (typing.List[str]) – list of keys that entry must have

Raises

axonius_api_client.exceptions.WizardError – if key is not in entry, value is empty, or value is not a string

Return type

dict

_check_entry_type(etype, types)¶

Check that a supplied entry type is valid.

Parameters

etype (str) – entry type to check
types (typing.List[str]) – valid entry types to check etype against

Raises

axonius_api_client.exceptions.WizardError – if etype is not on of types

Return type

str

_check_patterns(value_raw, value, src, patterns)¶

Check that a value matches a list of regex patterns.

Parameters

value_raw (str) – raw unparsed value where value came from
value (str) – value to check patterns against
src (str) – identifier of what value represents
patterns (typing.List[str]) – list of regex patterns to check value against

Raises

axonius_api_client.exceptions.WizardError – if value is empty or does not match one of the patterns

_get_field(value, value_raw)¶

Find a field schema for the supplied field name.

Parameters

value (str) – name of field to find schema for
value_raw (str) – raw unparsed value where field name came from

Raises

axonius_api_client.exceptions.WizardError – if field can not be found or is “all” field

Return type

dict

_get_field_complex(value, value_raw)¶

Find a field schema for the supplied complex field name.

Parameters

value (str) – name of complex field to find schema for
value_raw (str) – raw unparsed value where complex field name came from

Raises

axonius_api_client.exceptions.WizardError – if field can not be found, is not a complex field, or is “all” field

Return type

dict

_get_operator(operator, field, value_raw)¶

Validate the supplied operator for an expression for the type of field.

Parameters

operator (str) – operator supplied by user
field (dict) – field schema to check that operator is valid for
value_raw (str) – raw unparsed value where operator and field came from

Return type

axonius_api_client.constants.fields.Operator

_parse_complex(entry, idx)¶

Parse an entry of type complex into GUI expressions.

Parameters

entry (dict) – entry to parse
idx (int) – index of this entry

Raises

axonius_api_client.exceptions.WizardError – if sub expr fails to parse

Return type

dict

_parse_entries(entries, source)¶

Parse a list of entries into a query and the associated GUI query wizard expressions.

Parameters

entries (typing.List[dict]) – list of entries to parse
source (str) – where entries came from

Raises

axonius_api_client.exceptions.WizardError – if an entry fails to be parsed

Return type

typing.List[dict]

_parse_exprs(entries)¶

Parse a list of entries into GUI expressions.

Parameters: entries (typing.List[dict]) – list of entries to parse
Raises: axonius_api_client.exceptions.WizardError – if an entry fails to be parsed
Return type: typing.List[dict]

_parse_flags(entry, idx, entries, tracker, is_open)¶

Parse flags from an entry.

Parameters

entry (dict) – entry to parse with Entry.VALUE key
idx (int) – index of this entry
entries (typing.List[dict]) – all entries
tracker (int) – tracker for Entry.WEIGHT key
is_open (bool) – parenthesis are currently open

Return type

dict

_parse_simple(entry, idx)¶

Parse an entry of type simple into GUI expressions.

Parameters

entry (dict) – entry to parse
idx (int) – index of this entry

Return type

dict

_parse_sub(field, value_raw, idx)¶

Parse an sub expression of an entry of type complex.

Parameters

field (dict) – complex field schema
value_raw (str) – the value split from the complex filter line
idx (int) – index of this entry

Raises

axonius_api_client.exceptions.WizardError – if sub field supplied is not a valid sub field of the complex field

Return type

dict

_split_complex(value_raw)¶

Split a complex query wizard expression into field and sub field expressions.

Parameters: value_raw (str) – the raw unparsed value to parse
Return type: typing.Tuple[str, typing.List[str]]

_split_flags(value_raw, flags=None)¶

Parse an expression and get the flags from the beginning and end.

Parameters

value_raw (str) – the value to get the flags from
flags (typing.Optional[typing.List[str]]) – flags from the entry

Raises

axonius_api_client.exceptions.WizardError – if value is empty after removing flags

Return type

typing.Tuple[typing.List[str], str]

_split_simple(value_raw)¶

Split a simple query wizard expression into field, operator, and value.

Parameters: value_raw (str) – the raw unparsed value to parse
Return type: typing.Tuple[str, str, str]

LOG: logging.Logger¶: Logger for this object.

APIOBJ¶

Asset object.

Type: axonius_api_client.api.assets.asset_mixin.AssetMixin

PARSER¶

Value parser.

Type: axonius_api_client.parsers.wizards.WizardParser