yaml_tools.utils module

Shared utility code.

class yaml_tools.utils.SortedSet(iterable)

Bases: Set

Alternate set implementation favoring space over speed, while not requiring the set elements to be hashable. We also add a sort method.

sort()

class yaml_tools.utils.StrYAML(*, typ: List[Text] | Text | None = None, pure: Any = False, output: Any = None, plug_ins: Any = None)

Bases: YAML

New API likes dumping straight to file/stdout, so we subclass and create ‘inefficient’ custom string dumper.

dump(data, stream=None, **kw)

yaml_tools.utils.get_filelist(dirpath, filepattern='*.txt', debug=False)

Get path objects matching filepattern starting at dirpath and return a list of matching paths for any files found.

Parameters:

• dirpath – directory name to start file search

• filepattern – str of the form *.<ext>

• debug – increase output verbosity

Returns:

list of path strings

yaml_tools.utils.get_profile_ids(prog_opts, debug=False)

Replacement for get_filelist() when using the NIST profile ID text files (which are now packaged with the YAML config files).

yaml_tools.utils.get_profile_type(filename, debug=False)

Get oscal profile type from filename, where profile type is one of the exported profile names, ie, HIGH, MODERATE, LOW, or PRIVACY.

yaml_tools.utils.load_config(prog_name='ymltoxml', pkg='yaml_tools.data', file_encoding='utf-8', debug=False)

Load yaml configuration file and munchify the data. If local file is not found in current directory, the default will be loaded.

Parameters:

• prog_name (str) – filename of calling script (no extension)

• pkg – name of calling package.path for importlib

• file_encoding (str) – file encoding of config file

• debug (bool) – enable extra processing info

Returns:

Munch cfg obj and cfg file as Path obj

Return type:

tuple

yaml_tools.utils.process_template(tmpl_file, data_file, prog_opts)

Process jinja2 template file and context data and return rendered data. Context data is typically provided in a YAML file. Output data should be written to a new file matching the content type and using the same name as data_file with appropriate extension. Uses text_file_reader for supported file types or just plain text with any other extension names.

Parameters:

• tmpl_file – jinja template file (yaml or rst)

• data_file – context data for template (also yaml)

• prog_opts (dict) – configuration options

Return data_out:

rendered template data

yaml_tools.utils.pystache_render(*args, **kwargs)

Render pystache template with strict mode enabled.

yaml_tools.utils.replace_angles(data)

Replace angle bracket with original curly brace.

yaml_tools.utils.replace_curlys(data)

Replace original outside curly brace with angle bracket.

yaml_tools.utils.restore_xml_comments(xmls)

Turn tagged comment elements back into xml comments.

Parameters:

xmls (str) – xml (file) output from unparse

Return xmls:

processed xml string

Return type:

str

yaml_tools.utils.sort_from_parent(input_data, prog_opts)

Sort a list based on whether the target sort key has a parent key.

Parameters:

• input_data – Dict obj representing YAML input data

• prog_opts (dict) – configuration options

Return input_data:

sorted input

yaml_tools.utils.str_yaml_dumper(data, prog_opts)

Small StrYAML() dump wrapper.

yaml_tools.utils.text_data_writer(outdata, prog_opts)

Text data writer with optional formatting (default is raw); uses config setting for output format. Supports the same text data formats supported by the text_file_reader() input function:

• csv

• json

• yaml

• raw

Sends formatted data to stdout; redirect to a file as needed.

Parameters:

• outdata – data written to stdout

• prog_opts (dict) – configuration options

yaml_tools.utils.text_file_reader(file, prog_opts)

Text file reader for specific data types including raw text. Tries to handle YAML, JSON, CSV, text files with IDs, and plain ASCII text. Read and parse the file data if file is one of the expected types and return data objects. For all supported types of data, return a dictionary (or a list if input is a sequence).

Parameters:

• file (str) – filename/path to read

• prog_opts (dict) – configuration options

Return object:

file data as dict or list

Raises:

FileTypeError – if input file extension is not in EXTENSIONS