4.4.7.2. From CSV files

Parser for AQL queries and GUI expressions from CSV files.

class axonius_api_client.api.wizards.wizard_csv.WizardCsv(apiobj, log_level='info')[source]

Bases: axonius_api_client.api.wizards.wizard.Wizard

Parser for AQL queries and GUI expressions from CSV files.

Notes

This wizard can create as many saved queries as you like. The first row of the CSV must always have a type value of “saved_query”. All rows under that row will be added to that saved query until another row with a type value of “saved_query” is found. Repeat this pattern as you see fit.

The description, tags, and fields columns are only used for types of “saved_query”

Examples

Create a client using axonius_api_client.connect.Connect and assume apiobj is either client.devices or client.users

>>> apiobj = client.devices  # or client.users

Define a CSV string to parse

>>> content = '''
... type,value,description,tags,fields
... saved_query,name of sq,description  of sq,"tag1,tag2, tag4","os.type,default, aws:aws_device_type"
... simple,hostname contains test,,,
... simple,os.type equals windows,,,
... saved_query,name of second sq,description  of sq,"tag1,tag2, tag4",
... simple,hostname contains test,,,
... complex,installed_software // name contains chrome // version earlier_than 82,,,
... simple,os.type equals windows,,,
... '''

Parse the CSV string into a query and GUI expressions

>>> parsed = apiobj.wizard_csv.parse(content=content)
>>> for sq in parsed:
...   sq['name']
...   sq['query'][:80]
...   len(sq['expressions'])
...
'name of sq'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.os.t'
2
'name of second sq'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.inst'
3

Or parse a CSV file directly

>>> parsed = apiobj.wizard_csv.parse(path="~/test.csv")

Use the result to create saved queries that the GUI understands

>>> sqs = [apiobj.saved_query.add(**sq) for sq in parsed]
>>> for sq in sqs:
...    sq['name']
...    sq['uuid']
...    sq['view']['query']['filter'][:80]
...
'name of sq'
'5f79e90be4557d5cbab26344'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.os.t'
'name of second sq'
'5f79e90be4557d5cbab26359'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.inst'
Parameters

log_level (typing.Union[str, int]) –

DOCS: str = 'Example:\n\ntype,value,description,tags,fields\n"# If type column is empty or begins with # it is ignored",,,,\n"# type of simple or complex will belong to the saved_query they are under",,,,\n"# Column descriptions for type of saved_query","Name of Saved Query","Description of Saved Query","Tags to apply to Saved Query","Columns to display in Saved Query"\n"# Column descriptions for type of simple","Format -- [] represents optional items: [& | ! ( )] FIELD OPERATOR VALUE [)]","Description: Filter entry for simple fields","Only uses columns type and value",\n"# Column descriptions for type of complex","Format -- [] represents optional items: [& | ! ( )] COMPLEX-FIELD // SUB-FIELD OPERATOR VALUE[ // ...] [)]","Description: Filter entry for complex fields and their sub-fields","Only uses columns type and value",\n"# Value Flags for type of simple or complex","& Use and instead of or (default), | Use or instead of and (overrides &), ! Use not, ( Open a parentheses, ) Close a parentheses (can also be at end of entry)",,,\n"saved_query","example 1","Filters, default fields, custom fields","example,tag1,tag2","os.distribution,os.os_str,aws:aws_device_type,default,os.build"\n"simple","( hostname contains test",,,\n"simple","! hostname contains internal )",,,\n"simple","( os.type equals windows",,,\n"simple","| os.type equals os x )",,,\n"saved_query","example 2","No filters, no default fields, custom fields","example,tag3,tag4","os.distribution,os.os_str,aws:aws_device_type"\n"saved_query","example 3","No filters, default fields, no custom fields","example,tag5,tag6",\n'
parse(content, source='csv text string')[source]

Parse a CSV string into a set of saved queries.

Parameters
  • content (str) – CSV string

  • source (str) – where content came from

Return type

typing.List[dict]

parse_path(path, source='csv file {path}')[source]

Parse a CSV file into a set of saved queries.

Parameters
Return type

typing.List[dict]

_load_csv(content, source)[source]

Load a CSV string and parse the rows into entries.

Parameters
  • content (str) – CSV string

  • source (str) – where content came from

Return type

typing.List[dict]

_process_csv(rows, columns, source)[source]

Process and validate the rows and columns from a CSV into entries.

Parameters
Return type

typing.List[dict]

_rows_to_entries(rows, source)[source]

Process and validate the rows from a CSV into entries.

Parameters
Return type

typing.List[dict]

_row_to_entry(row, src)[source]

Proccess and validate a row from a CSV into an entry.

Parameters
  • row (dict) – row from the CSV

  • src (str) – identifier of where the row came from

Return type

dict

_process_sqs(entries)[source]

Process all of the saved queries defined in the CSV.

Parameters

entries (typing.List[dict]) – the entries produced by parsing the rows

Return type

typing.List[dict]

_process_sq(entry, is_last)[source]

Process a saved query.

Parameters
Return type

int

_process_sq_entries()[source]

Process the entries for the current saved query.

_new_sq(entry)[source]

Create a new current saved query.

Parameters

entry (dict) – entry being processed by _process_sqs()

_process_desc(entry)[source]

Process the description key of an entry.

Parameters

entry (dict) – entry being processed by _process_sqs()

Return type

typing.Optional[str]

_process_tags(entry)[source]

Process the tags key of an entry.

Parameters

entry (dict) – entry being processed by _process_sqs()

Return type

typing.Optional[typing.List[str]]

_process_fields(entry)[source]

Process the fields key of an entry.

Parameters

entry (dict) – entry being processed by _process_sqs()

Return type

[typing.List[str]]

_init()[source]

Post init setup.

__init__(apiobj, log_level='info')

Query wizard builder.

Parameters
_check_entry_keys(entry, keys)

Check that an entry has the required keys.

Parameters
  • entry (dict) – entry to check keys against

  • keys (typing.List[str]) – list of keys that entry must have

Raises

axonius_api_client.exceptions.WizardError – if key is not in entry, value is empty, or value is not a string

Return type

dict

_check_entry_type(etype, types)

Check that a supplied entry type is valid.

Parameters
  • etype (str) – entry type to check

  • types (typing.List[str]) – valid entry types to check etype against

Raises

axonius_api_client.exceptions.WizardError – if etype is not on of types

Return type

str

_check_patterns(value_raw, value, src, patterns)

Check that a value matches a list of regex patterns.

Parameters
  • value_raw (str) – raw unparsed value where value came from

  • value (str) – value to check patterns against

  • src (str) – identifier of what value represents

  • patterns (typing.List[str]) – list of regex patterns to check value against

Raises

axonius_api_client.exceptions.WizardError – if value is empty or does not match one of the patterns

_get_field(value, value_raw)

Find a field schema for the supplied field name.

Parameters
  • value (str) – name of field to find schema for

  • value_raw (str) – raw unparsed value where field name came from

Raises

axonius_api_client.exceptions.WizardError – if field can not be found or is “all” field

Return type

dict

_get_field_complex(value, value_raw)

Find a field schema for the supplied complex field name.

Parameters
  • value (str) – name of complex field to find schema for

  • value_raw (str) – raw unparsed value where complex field name came from

Raises

axonius_api_client.exceptions.WizardError – if field can not be found, is not a complex field, or is “all” field

Return type

dict

_get_operator(operator, field, value_raw)

Validate the supplied operator for an expression for the type of field.

Parameters
  • operator (str) – operator supplied by user

  • field (dict) – field schema to check that operator is valid for

  • value_raw (str) – raw unparsed value where operator and field came from

Return type

axonius_api_client.constants.fields.Operator

_parse_complex(entry, idx)

Parse an entry of type complex into GUI expressions.

Parameters
  • entry (dict) – entry to parse

  • idx (int) – index of this entry

Raises

axonius_api_client.exceptions.WizardError – if sub expr fails to parse

Return type

dict

_parse_entries(entries, source)

Parse a list of entries into a query and the associated GUI query wizard expressions.

Parameters
  • entries (typing.List[dict]) – list of entries to parse

  • source (str) – where entries came from

Raises

axonius_api_client.exceptions.WizardError – if an entry fails to be parsed

Return type

typing.List[dict]

_parse_exprs(entries)

Parse a list of entries into GUI expressions.

Parameters

entries (typing.List[dict]) – list of entries to parse

Raises

axonius_api_client.exceptions.WizardError – if an entry fails to be parsed

Return type

typing.List[dict]

_parse_flags(entry, idx, entries, tracker, is_open)

Parse flags from an entry.

Parameters
  • entry (dict) – entry to parse with Entry.VALUE key

  • idx (int) – index of this entry

  • entries (typing.List[dict]) – all entries

  • tracker (int) – tracker for Entry.WEIGHT key

  • is_open (bool) – parenthesis are currently open

Return type

dict

_parse_simple(entry, idx)

Parse an entry of type simple into GUI expressions.

Parameters
  • entry (dict) – entry to parse

  • idx (int) – index of this entry

Return type

dict

_parse_sub(field, value_raw, idx)

Parse an sub expression of an entry of type complex.

Parameters
  • field (dict) – complex field schema

  • value_raw (str) – the value split from the complex filter line

  • idx (int) – index of this entry

Raises

axonius_api_client.exceptions.WizardError – if sub field supplied is not a valid sub field of the complex field

Return type

dict

_split_complex(value_raw)

Split a complex query wizard expression into field and sub field expressions.

Parameters

value_raw (str) – the raw unparsed value to parse

Return type

typing.Tuple[str, typing.List[str]]

_split_flags(value_raw, flags=None)

Parse an expression and get the flags from the beginning and end.

Parameters
Raises

axonius_api_client.exceptions.WizardError – if value is empty after removing flags

Return type

typing.Tuple[typing.List[str], str]

_split_simple(value_raw)

Split a simple query wizard expression into field, operator, and value.

Parameters

value_raw (str) – the raw unparsed value to parse

Return type

typing.Tuple[str, str, str]