4.4.7.2. From CSV files

Parser for AQL queries and GUI expressions from CSV files.

class axonius_api_client.api.wizards.wizard_csv.WizardCsv(apiobj, log_level='debug')[source]

Bases: Wizard

Parser for AQL queries and GUI expressions from CSV files.

Notes

This wizard can create as many saved queries as you like. The first row of the CSV must always have a type value of “saved_query”. All rows under that row will be added to that saved query until another row with a type value of “saved_query” is found. Repeat this pattern as you see fit.

The description, tags, and fields columns are only used for types of “saved_query”

Examples

Create a client using axonius_api_client.connect.Connect and assume apiobj is either client.devices or client.users

>>> apiobj = client.devices  # or client.users

Define a CSV string to parse

>>> content = '''
... type,value,description,tags,fields
... saved_query,name of sq,description  of sq,"tag1,tag2, tag4","os.type,default, aws:aws_device_type"
... simple,hostname contains test,,,
... simple,os.type equals windows,,,
... saved_query,name of second sq,description  of sq,"tag1,tag2, tag4",
... simple,hostname contains test,,,
... complex,installed_software // name contains chrome // version earlier_than 82,,,
... simple,os.type equals windows,,,
... '''

Parse the CSV string into a query and GUI expressions

>>> parsed = apiobj.wizard_csv.parse(content=content)
>>> for sq in parsed:
...   sq['name']
...   sq['query'][:80]
...   len(sq['expressions'])
...
'name of sq'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.os.t'
2
'name of second sq'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.inst'
3

Or parse a CSV file directly

>>> parsed = apiobj.wizard_csv.parse(path="~/test.csv")

Use the result to create saved queries that the GUI understands

>>> sqs = [apiobj.saved_query.add(**sq) for sq in parsed]
>>> for sq in sqs:
...    sq['name']
...    sq['uuid']
...    sq['view']['query']['filter'][:80]
...
'name of sq'
'5f79e90be4557d5cbab26344'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.os.t'
'name of second sq'
'5f79e90be4557d5cbab26359'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.inst'
Parameters:

log_level (typing.Union[str, int]) –

DOCS: str = '\ntype,value,description,tags,fields,asset_scope,private,always_cached,gui_page_size\n"# If type column is empty or begins with # it is ignored",,,,\n"# type of simple or complex will belong to the saved_query they are under",,,,\n"# Column descriptions for type of saved_query","Name of Saved Query","Description of Saved Query","Tags to apply to Saved Query","Columns to display in Saved Query",Optional: True or False (default False if empty),Optional: True or False (default False if empty),Optional: True or False (default False if empty),"20 or 50 or 100 (default 20 if empty)",\n"# Column descriptions for type of simple","Format -- [] represents optional items: [& | ! ( )] FIELD OPERATOR VALUE [)]","Description: Filter entry for simple fields","Only uses columns type and value",\n"# Column descriptions for type of complex","Format -- [] represents optional items: [& | ! ( )] COMPLEX-FIELD // SUB-FIELD OPERATOR VALUE[ //  ...] [)]","Description: Filter entry for complex fields and their sub-fields","Only uses columns type and value",\n"# Value Flags for type of simple or complex","& Use and instead of or (default), | Use or instead of and (overrides &), ! Use not, ( Open a parentheses, ) Close a parentheses (can also be at end of entry)",,,\n"saved_query","example 1","Filters, default fields, custom fields","example,tag1,tag2","os.distribution,os.os_str,active_directory:ad_password_last_set,default,os.build"\n"simple","( hostname contains test",,,\n"simple","! hostname contains internal )",,,\n"simple","( os.type equals windows",,,\n"simple","| os.type equals os x )",,,\n"saved_query","example 2","No filters, no default fields, custom fields","example,tag3,tag4","os.distribution,os.os_str,active_directory:ad_password_last_set"\n"saved_query","example 3","No filters, default fields, no custom fields","example,tag5,tag6",\n'
parse(content, source='csv text string')[source]

Parse a CSV string into a set of saved queries.

Parameters:
  • content (str) – CSV string

  • source (str) – where content came from

Return type:

typing.List[dict]

parse_path(path, source='csv file {path}')[source]

Parse a CSV file into a set of saved queries.

Parameters:
Return type:

typing.List[dict]

_load_csv(content, source)[source]

Load a CSV string and parse the rows into entries.

Parameters:
  • content (str) – CSV string

  • source (str) – where content came from

Return type:

typing.List[dict]

_process_csv(rows, columns, source)[source]

Process and validate the rows and columns from a CSV into entries.

Parameters:
Return type:

typing.List[dict]

_rows_to_entries(rows, source)[source]

Process and validate the rows from a CSV into entries.

Parameters:
Return type:

typing.List[dict]

_row_to_entry(row, src)[source]

Proccess and validate a row from a CSV into an entry.

Parameters:
  • row (dict) – row from the CSV

  • src (str) – identifier of where the row came from

Return type:

dict

_process_sqs(entries)[source]

Process the saved queries defined in the CSV.

Parameters:

entries (typing.List[dict]) – the entries produced by parsing the rows

Return type:

typing.List[dict]

_process_sq(entry, is_last)[source]

Process a saved query.

Parameters:
Return type:

int

_process_sq_entries()[source]

Process the entries for the current saved query.

_new_sq(entry)[source]

Create a new current saved query.

Parameters:

entry (dict) – entry being processed by _process_sqs()

_process_desc(entry)[source]

Process the description key of an entry.

Parameters:

entry (dict) – entry being processed by _process_sqs()

Return type:

typing.Optional[str]

_process_tags(entry)[source]

Process the tags key of an entry.

Parameters:

entry (dict) – entry being processed by _process_sqs()

Return type:

typing.Optional[typing.List[str]]

_process_fields(entry)[source]

Process the fields key of an entry.

Parameters:

entry (dict) – entry being processed by _process_sqs()

Return type:

[List[str]]

_init()[source]

Post init setup.

__init__(apiobj, log_level='debug')

Query wizard builder.

Parameters:
_check_entry_keys(entry, keys)

Check that an entry has the required keys.

Parameters:
  • entry (dict) – entry to check keys against

  • keys (typing.List[str]) – list of keys that entry must have

Raises:

axonius_api_client.exceptions.WizardError – if key is not in entry, value is empty, or value is not a string

Return type:

dict

_check_entry_type(etype, types)

Check that a supplied entry type is valid.

Parameters:
  • etype (str) – entry type to check

  • types (typing.List[str]) – valid entry types to check etype against

Raises:

axonius_api_client.exceptions.WizardError – if etype is not on of types

Return type:

str

_check_patterns(value_raw, value, src, patterns)

Check that a value matches a list of regex patterns.

Parameters:
  • value_raw (str) – raw unparsed value where value came from

  • value (str) – value to check patterns against

  • src (str) – identifier of what value represents

  • patterns (typing.List[str]) – list of regex patterns to check value against

Raises:

axonius_api_client.exceptions.WizardError – if value is empty or does not match one of the patterns

_get_field(value, value_raw)

Find a field schema for the supplied field name.

Parameters:
  • value (str) – name of field to find schema for

  • value_raw (str) – raw unparsed value where field name came from

Raises:

axonius_api_client.exceptions.WizardError – if field can not be found or is “all” field

Return type:

dict

_get_field_complex(value, value_raw)

Find a field schema for the supplied complex field name.

Parameters:
  • value (str) – name of complex field to find schema for

  • value_raw (str) – raw unparsed value where complex field name came from

Raises:

axonius_api_client.exceptions.WizardError – if field can not be found, is not a complex field, or is “all” field

Return type:

dict

_get_operator(operator, field, value_raw)

Validate the supplied operator for an expression for the type of field.

Parameters:
  • operator (str) – operator supplied by user

  • field (dict) – field schema to check that operator is valid for

  • value_raw (str) – raw unparsed value where operator and field came from

Return type:

axonius_api_client.constants.fields.Operator

_parse_complex(entry, idx)

Parse an entry of type complex into GUI expressions.

Parameters:
  • entry (dict) – entry to parse

  • idx (int) – index of this entry

Raises:

axonius_api_client.exceptions.WizardError – if sub expr fails to parse

Return type:

dict

_parse_entries(entries, source)

Parse a list of entries into a query and the associated GUI query wizard expressions.

Parameters:
  • entries (typing.List[dict]) – list of entries to parse

  • source (str) – where entries came from

Raises:

axonius_api_client.exceptions.WizardError – if an entry fails to be parsed

Return type:

typing.List[dict]

_parse_exprs(entries)

Parse a list of entries into GUI expressions.

Parameters:

entries (typing.List[dict]) – list of entries to parse

Raises:

axonius_api_client.exceptions.WizardError – if an entry fails to be parsed

Return type:

typing.List[dict]

_parse_flags(entry, idx, entries, tracker, is_open)

Parse flags from an entry.

Parameters:
  • entry (dict) – entry to parse with Entry.VALUE key

  • idx (int) – index of this entry

  • entries (typing.List[dict]) – all entries

  • tracker (int) – tracker for Entry.WEIGHT key

  • is_open (bool) – parenthesis are currently open

Return type:

dict

_parse_simple(entry, idx)

Parse an entry of type simple into GUI expressions.

Parameters:
  • entry (dict) – entry to parse

  • idx (int) – index of this entry

Return type:

dict

_parse_sub(field, value_raw, idx)

Parse sub expression of an entry of type complex.

Parameters:
  • field (dict) – complex field schema

  • value_raw (str) – the value split from the complex filter line

  • idx (int) – index of this entry

Raises:

axonius_api_client.exceptions.WizardError – if sub-field supplied is not a valid sub-field of the complex field

Return type:

dict

_split_complex(value_raw)

Split a complex query wizard expression into field and sub-field expressions.

Parameters:

value_raw (str) – the raw unparsed value to parse

Return type:

typing.Tuple[str, typing.List[str]]

_split_flags(value_raw, flags=None)

Parse an expression and get the flags from the beginning and end.

Parameters:
Raises:

axonius_api_client.exceptions.WizardError – if value is empty after removing flags

Return type:

typing.Tuple[typing.List[str], str]

_split_simple(value_raw)

Split a simple query wizard expression into field, operator, and value.

Parameters:

value_raw (str) – the raw unparsed value to parse

Return type:

typing.Tuple[str, str, str]

LOG: logging.Logger

Logger for this object.

APIOBJ

Asset object.

Type:

axonius_api_client.api.assets.asset_mixin.AssetMixin

PARSER

Value parser.

Type:

axonius_api_client.parsers.wizards.WizardParser