4.4.7.2. From CSV files¶

Parser for AQL queries and GUI expressions from CSV files.

class axonius_api_client.api.wizards.wizard_csv.WizardCsv(apiobj, log_level='debug')[source]¶

Bases: Wizard

Parser for AQL queries and GUI expressions from CSV files.

Notes

This wizard can create as many saved queries as you like. The first row of the CSV must always have a type value of “saved_query”. All rows under that row will be added to that saved query until another row with a type value of “saved_query” is found. Repeat this pattern as you see fit.

The description, tags, and fields columns are only used for types of “saved_query”

Examples

Create a client using axonius_api_client.connect.Connect and assume apiobj is either client.devices or client.users

>>> apiobj = client.devices  # or client.users

Define a CSV string to parse

>>> content = '''
... type,value,description,tags,fields
... saved_query,name of sq,description  of sq,"tag1,tag2, tag4","os.type,default, aws:aws_device_type"
... simple,hostname contains test,,,
... simple,os.type equals windows,,,
... saved_query,name of second sq,description  of sq,"tag1,tag2, tag4",
... simple,hostname contains test,,,
... complex,installed_software // name contains chrome // version earlier_than 82,,,
... simple,os.type equals windows,,,
... '''

Parse the CSV string into a query and GUI expressions

>>> parsed = apiobj.wizard_csv.parse(content=content)
>>> for sq in parsed:
...   sq['name']
...   sq['query'][:80]
...   len(sq['expressions'])
...
'name of sq'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.os.t'
2
'name of second sq'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.inst'
3

Or parse a CSV file directly

>>> parsed = apiobj.wizard_csv.parse(path="~/test.csv")

Use the result to create saved queries that the GUI understands

>>> sqs = [apiobj.saved_query.add(**sq) for sq in parsed]
>>> for sq in sqs:
...    sq['name']
...    sq['uuid']
...    sq['view']['query']['filter'][:80]
...
'name of sq'
'5f79e90be4557d5cbab26344'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.os.t'
'name of second sq'
'5f79e90be4557d5cbab26359'
'(specific_data.data.hostname == regex("test", "i")) and (specific_data.data.inst'

Parameters:: log_level (typing.Union[str, int]) –

DOCS: str = '\ntype,value,description,tags,fields,asset_scope,private,always_cached,gui_page_size\n"# If type column is empty or begins with # it is ignored",,,,\n"# type of simple or complex will belong to the saved_query they are under",,,,\n"# Column descriptions for type of saved_query","Name of Saved Query","Description of Saved Query","Tags to apply to Saved Query","Columns to display in Saved Query",Optional: True or False (default False if empty),Optional: True or False (default False if empty),Optional: True or False (default False if empty),"20 or 50 or 100 (default 20 if empty)",\n"# Column descriptions for type of simple","Format -- [] represents optional items: [& | ! ( )] FIELD OPERATOR VALUE [)]","Description: Filter entry for simple fields","Only uses columns type and value",\n"# Column descriptions for type of complex","Format -- [] represents optional items: [& | ! ( )] COMPLEX-FIELD // SUB-FIELD OPERATOR VALUE[ // ...] [)]","Description: Filter entry for complex fields and their sub-fields","Only uses columns type and value",\n"# Value Flags for type of simple or complex","& Use and instead of or (default), | Use or instead of and (overrides &), ! Use not, ( Open a parentheses, ) Close a parentheses (can also be at end of entry)",,,\n"saved_query","example 1","Filters, default fields, custom fields","example,tag1,tag2","os.distribution,os.os_str,active_directory:ad_password_last_set,default,os.build"\n"simple","( hostname contains test",,,\n"simple","! hostname contains internal )",,,\n"simple","( os.type equals windows",,,\n"simple","| os.type equals os x )",,,\n"saved_query","example 2","No filters, no default fields, custom fields","example,tag3,tag4","os.distribution,os.os_str,active_directory:ad_password_last_set"\n"saved_query","example 3","No filters, default fields, no custom fields","example,tag5,tag6",\n'¶

parse(content, source='csv text string')[source]¶

Parse a CSV string into a set of saved queries.

Parameters:

content (str) – CSV string
source (str) – where content came from

Return type:

typing.List[dict]

parse_path(path, source='csv file {path}')[source]¶

Parse a CSV file into a set of saved queries.

Parameters:

path (typing.Union[str, pathlib.Path]) – CSV file
source (str) – where csv file came from

Return type:

typing.List[dict]

_load_csv(content, source)[source]¶

Load a CSV string and parse the rows into entries.

Parameters:

content (str) – CSV string
source (str) – where content came from

Return type:

typing.List[dict]

_process_csv(rows, columns, source)[source]¶

Process and validate the rows and columns from a CSV into entries.

Parameters:

rows (typing.List[dict]) – rows from the CSV
columns (typing.List[str]) – columns from the CSV
source (str) – where content came from

Return type:

typing.List[dict]

_rows_to_entries(rows, source)[source]¶

Process and validate the rows from a CSV into entries.

Parameters:

rows (typing.List[dict]) – rows from the CSV
source (str) – where content came from

Return type:

typing.List[dict]

_row_to_entry(row, src)[source]¶

Proccess and validate a row from a CSV into an entry.

Parameters:

row (dict) – row from the CSV
src (str) – identifier of where the row came from

Return type:

dict

_process_sqs(entries)[source]¶

Process the saved queries defined in the CSV.

Parameters:: entries (typing.List[dict]) – the entries produced by parsing the rows
Return type:: typing.List[dict]

_process_sq(entry, is_last)[source]¶

Process a saved query.

Parameters:

entry (dict) – entry being processed by _process_sqs()
is_last (bool) – entry is the last entry being processed by _process_sqs()

Return type:

int

_process_sq_entries()[source]¶: Process the entries for the current saved query.

_new_sq(entry)[source]¶

Create a new current saved query.

Parameters:: entry (dict) – entry being processed by _process_sqs()

_process_desc(entry)[source]¶

Process the description key of an entry.

Parameters:: entry (dict) – entry being processed by _process_sqs()
Return type:: typing.Optional[str]

_process_tags(entry)[source]¶

Process the tags key of an entry.

Parameters:: entry (dict) – entry being processed by _process_sqs()
Return type:: typing.Optional[typing.List[str]]

_process_fields(entry)[source]¶

Process the fields key of an entry.

Parameters:: entry (dict) – entry being processed by _process_sqs()
Return type:: [List[str]]

_init()[source]¶: Post init setup.

__init__(apiobj, log_level='debug')¶

Query wizard builder.

Parameters:

apiobj (axonius_api_client.api.assets.asset_mixin.AssetMixin) – Asset object
log_level (typing.Union[str, int]) – logging level for this object

_check_entry_keys(entry, keys)¶

Check that an entry has the required keys.

Parameters:

entry (dict) – entry to check keys against
keys (typing.List[str]) – list of keys that entry must have

Raises:

axonius_api_client.exceptions.WizardError – if key is not in entry, value is empty, or value is not a string

Return type:

dict

_check_entry_type(etype, types)¶

Check that a supplied entry type is valid.

Parameters:

etype (str) – entry type to check
types (typing.List[str]) – valid entry types to check etype against

Raises:

axonius_api_client.exceptions.WizardError – if etype is not on of types

Return type:

str

_check_patterns(value_raw, value, src, patterns)¶

Check that a value matches a list of regex patterns.

Parameters:

value_raw (str) – raw unparsed value where value came from
value (str) – value to check patterns against
src (str) – identifier of what value represents
patterns (typing.List[str]) – list of regex patterns to check value against

Raises:

axonius_api_client.exceptions.WizardError – if value is empty or does not match one of the patterns

_get_field(value, value_raw)¶

Find a field schema for the supplied field name.

Parameters:

value (str) – name of field to find schema for
value_raw (str) – raw unparsed value where field name came from

Raises:

axonius_api_client.exceptions.WizardError – if field can not be found or is “all” field

Return type:

dict

_get_field_complex(value, value_raw)¶

Find a field schema for the supplied complex field name.

Parameters:

value (str) – name of complex field to find schema for
value_raw (str) – raw unparsed value where complex field name came from

Raises:

axonius_api_client.exceptions.WizardError – if field can not be found, is not a complex field, or is “all” field

Return type:

dict

_get_operator(operator, field, value_raw)¶

Validate the supplied operator for an expression for the type of field.

Parameters:

operator (str) – operator supplied by user
field (dict) – field schema to check that operator is valid for
value_raw (str) – raw unparsed value where operator and field came from

Return type:

axonius_api_client.constants.fields.Operator

_parse_complex(entry, idx)¶

Parse an entry of type complex into GUI expressions.

Parameters:

entry (dict) – entry to parse
idx (int) – index of this entry

Raises:

axonius_api_client.exceptions.WizardError – if sub expr fails to parse

Return type:

dict

_parse_entries(entries, source)¶

Parse a list of entries into a query and the associated GUI query wizard expressions.

Parameters:

entries (typing.List[dict]) – list of entries to parse
source (str) – where entries came from

Raises:

axonius_api_client.exceptions.WizardError – if an entry fails to be parsed

Return type:

typing.List[dict]

_parse_exprs(entries)¶

Parse a list of entries into GUI expressions.

Parameters:: entries (typing.List[dict]) – list of entries to parse
Raises:: axonius_api_client.exceptions.WizardError – if an entry fails to be parsed
Return type:: typing.List[dict]

_parse_flags(entry, idx, entries, tracker, is_open)¶

Parse flags from an entry.

Parameters:

entry (dict) – entry to parse with Entry.VALUE key
idx (int) – index of this entry
entries (typing.List[dict]) – all entries
tracker (int) – tracker for Entry.WEIGHT key
is_open (bool) – parenthesis are currently open

Return type:

dict

_parse_simple(entry, idx)¶

Parse an entry of type simple into GUI expressions.

Parameters:

entry (dict) – entry to parse
idx (int) – index of this entry

Return type:

dict

_parse_sub(field, value_raw, idx)¶

Parse sub expression of an entry of type complex.

Parameters:

field (dict) – complex field schema
value_raw (str) – the value split from the complex filter line
idx (int) – index of this entry

Raises:

axonius_api_client.exceptions.WizardError – if sub-field supplied is not a valid sub-field of the complex field

Return type:

dict

_split_complex(value_raw)¶

Split a complex query wizard expression into field and sub-field expressions.

Parameters:: value_raw (str) – the raw unparsed value to parse
Return type:: typing.Tuple[str, typing.List[str]]

_split_flags(value_raw, flags=None)¶

Parse an expression and get the flags from the beginning and end.

Parameters:

value_raw (str) – the value to get the flags from
flags (typing.Optional[typing.List[str]]) – flags from the entry

Raises:

axonius_api_client.exceptions.WizardError – if value is empty after removing flags

Return type:

typing.Tuple[typing.List[str], str]

_split_simple(value_raw)¶

Split a simple query wizard expression into field, operator, and value.

Parameters:: value_raw (str) – the raw unparsed value to parse
Return type:: typing.Tuple[str, str, str]

LOG: logging.Logger¶: Logger for this object.

APIOBJ¶

Asset object.

Type:: axonius_api_client.api.assets.asset_mixin.AssetMixin

PARSER¶

Value parser.

Type:: axonius_api_client.parsers.wizards.WizardParser