visualdynamics.io.escdf_objects

escdf_objects

Write a project to an ESCDF file and read one back, with each object written both as the standard's own type and whole, as Visual Dynamics keeps it.

The standard has types for what a test produces: geometry, channel_table, data, response_spectrum and mode. Each object of those kinds is written as that type, so any reader of the format gets the nodes, channels, records and modes from the standard fields alone. The standard has no fields for much of what a project holds: element blocks and their properties, coordinate systems, the processing record and provenance, reports, matched modes, sine and random specifications, and a geometry's view. Those are kept in the attachments every group has, one attachment per object, holding the object in its .vdyn layout (written by io.native into memory). A file written here therefore reads back complete, and a reader that does not know the attachment can ignore it. A report is attached twice: rendered as a self-contained HTML page, and as its definition, so it opens again editable.

Values in the standard fields are written in the units their labels name. A geometry's positions are in its declared length unit (position_units), each data record is in its own declared unit (ordinate_unit), and mode shape coefficients are normalized against the declared mass unit (modal_mass_unit). An object whose units are not defined is written as stored, with the unit left blank. Abscissas are written in seconds or hertz.

An activity is an object group: the group's results (data and mode shapes) are the activity's data, and the geometry and channel tables it uses are metadata at the root, linked from the activity. A named group becomes an activity with that name; an unnamed one takes the name of its geometry, or of its first member. Objects in no group form one more activity. Photos, reports, matched modes and sine specifications are metadata as well, written as parameter sets and linked to the activity whose group holds them, because the standard puts anything that is not a result at the root.

A file this program did not write is read from the standard fields: a geometry's nodes, lines, elements and per-node directions (a node whose axes are not the global ones gets a coordinate system of its own), a data record's values and units, and a mode set's frequencies, damping and shapes. The declared units are applied on reading, so positions, records and shapes are converted from the units the file names. Unit strings are written with '^' for a power, as the format's examples spell them, and either '^' or '**' is read.

Functions:

Name	Description
handles	True for a whole project, or for any one kind of object a project
to_file	A project as an ESCDF File, ready for escdf.write.
save	Write a project, or one object as a project of one, as an ESCDF
from_file	A project from an ESCDF File.
load	Read an ESCDF file as a project (see from_file).

Classes

Functions:

handles

handles(obj: Any) -> bool

True for a whole project, or for any one kind of object a project holds.

Source code in src/visualdynamics/io/escdf_objects.py

def handles(obj: Any) -> bool:
    """True for a whole project, or for any one kind of object a project
    holds."""
    return isinstance(obj, (Project, Geometry, DataArray, ShapeSet, ChannelTable,
                            Report, Photos, MatchedModes, SineSweepSpecification,
                            SineLevelSet))

to_file

to_file(project: Project, created_by: str | None = None, unit_system: Any = None) -> File

A project as an ESCDF File, ready for escdf.write.

Each object is written as the standard's type with the object whole in an attachment, and each object group becomes an activity. The project's name, object groups, provenance and settings go in a parameter set at the root named visualdynamics_project.

Parameters:

Name	Type	Description	Default
project	Project	What to write.	required
created_by	str	The file's creator; the login name when left out.	None
unit_system	UnitSystem	The system the reports are rendered in; SI when left out. It does not change the units of the standard fields, which are the objects' own declared units.	None

Returns:

Type	Description
File

Source code in src/visualdynamics/io/escdf_objects.py

def to_file(project: Project, created_by: str | None = None,
            unit_system: Any = None) -> escdf.File:
    """A project as an ESCDF `File`, ready for `escdf.write`.

    Each object is written as the standard's type with the object whole
    in an attachment, and each object group becomes an activity. The
    project's name, object groups, provenance and settings go in a
    parameter set at the root named `visualdynamics_project`.

    Parameters
    ----------
    project : Project
        What to write.
    created_by : str, optional
        The file's creator; the login name when left out.
    unit_system : UnitSystem, optional
        The system the reports are rendered in; SI when left out. It
        does not change the units of the standard fields, which are the
        objects' own declared units.

    Returns
    -------
    escdf.File
    """
    from .. import SI

    file = escdf.File(created_by=created_by or getpass.getuser(),
                      created_date=dt.datetime.now(dt.UTC))
    taken: set[str] = {PROJECT_RECORD}

    def identifier(name: str) -> str:
        base = escdf.valid_identifier(name)
        candidate, k = base, 2
        while candidate in taken:
            candidate, k = f'{base}_{k}', k + 1
        taken.add(candidate)
        return candidate

    names = {name: identifier(name) for name in project}
    metadata: dict[str, escdf.Dataset] = {}
    results: dict[str, escdf.Dataset] = {}
    for name, obj in project.items():
        key = names[name]
        if isinstance(obj, Geometry):
            dataset = _geometry_dataset(key, obj)
            _attach(dataset, [(WHOLE, _whole(obj))], _note('geometry'))
            metadata[key] = dataset
        elif isinstance(obj, ChannelTable):
            dataset = _channel_table_dataset(key, obj)
            _attach(dataset, [(WHOLE, _whole(obj))], _note('channel table'))
            metadata[key] = dataset
        elif isinstance(obj, DataArray):
            dataset = _data_dataset(key, obj)
            _attach(dataset, [(WHOLE, _whole(obj))], _note(type(obj).__name__))
            results[key] = dataset
        elif isinstance(obj, ShapeSet):
            dataset = _mode_dataset(key, obj)
            _attach(dataset, [(WHOLE, _whole(obj))], _note('shape set'))
            results[key] = dataset
        elif isinstance(obj, Report):
            from ..report import render_html

            # the project itself, not a copy of its mapping: a report
            # feature may read what the project holds beside its objects
            # (the data index)
            html = render_html(obj, project, unit_system or SI,
                               object_groups=project.object_groups)
            dataset = escdf.Dataset(key, 'parameter_set', name)
            _attach(dataset, [(f'{key}.html', np.frombuffer(html.encode('utf-8'),
                                                            dtype=np.uint8)),
                              (WHOLE, _whole(obj))],
                    _note('report') + f' {key}.html is the report rendered, '
                    'complete in one page.')
            metadata[key] = dataset
        elif isinstance(obj, Photos):
            dataset = escdf.Dataset(key, 'parameter_set', name)
            _attach(dataset, [(f'{n}.{f}', np.frombuffer(bytes(b), dtype=np.uint8))
                              for n, f, b in zip(obj.names, obj.formats, obj.images)],
                    'Photographs, one attachment each, named as taken.')
            metadata[key] = dataset
        else:
            dataset = escdf.Dataset(key, 'parameter_set', name)
            _attach(dataset, [(WHOLE, _whole(obj))], _note(type(obj).__name__))
            metadata[key] = dataset
        dataset.descriptive_name = name
    # the project's own record: what the standard has no field for
    record = escdf.Dataset(PROJECT_RECORD, 'parameter_set', project.name)
    record.values['notes'] = np.array([(
        'Visual Dynamics project record: attachments hold the object groups, '
        'the provenance of derived objects and the project settings, as JSON.')],
        dtype=object)
    settings = {'name': project.name, 'project_type': project.project_type,
                'active_geometry': project.active_geometry,
                'names': names}
    record.values['attachment_names'] = np.array(
        ['links.json', 'provenance.json', 'project.json'], dtype=object)
    record.values['attachments'] = [
        np.frombuffer(json.dumps(project.object_groups).encode('utf-8'), dtype=np.uint8),
        np.frombuffer(json.dumps(project.provenance).encode('utf-8'), dtype=np.uint8),
        np.frombuffer(json.dumps(settings).encode('utf-8'), dtype=np.uint8)]
    file.metadata[PROJECT_RECORD] = record
    file.metadata.update(metadata)
    # activities: one per object group, its results inside, its metadata linked
    placed: set[str] = set()
    activity_names: set[str] = set()
    for group in project.object_groups:
        members = [m for m in group['members'] if m in names]
        if not members:
            continue
        basis = next((m for m in members if isinstance(project[m], Geometry)), members[0])
        label = group.get('name') or basis
        key = escdf.valid_identifier(label, 'activity_')
        k = 2
        while key in activity_names:
            key, k = f'{escdf.valid_identifier(label, "activity_")}_{k}', k + 1
        activity_names.add(key)
        activity = escdf.Activity(key, label, file.created_date)
        for member in members:
            dataset_name = names[member]
            if dataset_name in results:
                activity.data[dataset_name] = results[dataset_name]
            else:
                activity.links.append(dataset_name)
            placed.add(member)
        file.activities[key] = activity
    loose = [name for name in project if name not in placed]
    if loose:
        key = escdf.valid_identifier(project.name or 'project', 'activity_')
        while key in activity_names:
            key += '_'
        activity = escdf.Activity(key, project.name or 'Unlinked objects',
                                  file.created_date)
        for member in loose:
            dataset_name = names[member]
            if dataset_name in results:
                activity.data[dataset_name] = results[dataset_name]
            else:
                activity.links.append(dataset_name)
        file.activities[key] = activity
    return file

save

save(obj: Any, path: str | PathLike, unit_system: Any = None, created_by: str | None = None, **_ignored: Any) -> None

Write a project, or one object as a project of one, as an ESCDF file.

Parameters:

Name	Type	Description	Default
obj	Project or object	What to write.	required
path	path - like	Where; .h5 is added unless it already ends in .h5, .hdf5 or .escdf.	required
unit_system	UnitSystem	The system reports are rendered in; SI when left out. The standard fields are written in each object's declared units whatever this is.	None
created_by	str	The creator recorded in the file; the login name when left out.	None

Returns:

Type	Description
None

Source code in src/visualdynamics/io/escdf_objects.py

def save(obj: Any, path: str | os.PathLike, unit_system: Any = None,
         created_by: str | None = None, **_ignored: Any) -> None:
    """Write a project, or one object as a project of one, as an ESCDF
    file.

    Parameters
    ----------
    obj : Project or object
        What to write.
    path : path-like
        Where; `.h5` is added unless it already ends in `.h5`,
        `.hdf5` or `.escdf`.
    unit_system : UnitSystem, optional
        The system reports are rendered in; SI when left out. The
        standard fields are written in each object's declared units
        whatever this is.
    created_by : str, optional
        The creator recorded in the file; the login name when left out.

    Returns
    -------
    None
    """
    path = str(path)
    if not path.endswith(SUFFIXES):
        path += SUFFIX
    if not isinstance(obj, Project):
        project = Project(getattr(obj, 'name', '') or type(obj).__name__)
        project.add(type(obj).__name__, obj)
        obj = project
    escdf.write(to_file(obj, created_by, unit_system), path)

from_file

from_file(file: File) -> Project

A project from an ESCDF File.

Each dataset becomes an object, built from its whole-object attachment when it has one and from the standard fields otherwise. A dataset whose type the specifications do not define is left out with a note. The project's own record, when the file has one, restores the object names, order, object groups and provenance; without it, each activity with two or more objects becomes an object group named for the activity.

Parameters:

Name	Type	Description	Default
file	File	As escdf.read returns it.	required

Returns:

Type	Description
Project

Source code in src/visualdynamics/io/escdf_objects.py

def from_file(file: escdf.File) -> Project:
    """A project from an ESCDF `File`.

    Each dataset becomes an object, built from its whole-object
    attachment when it has one and from the standard fields otherwise.
    A dataset whose type the specifications do not define is left out
    with a note. The project's own record, when the file has one,
    restores the object names, order, object groups and provenance;
    without it, each activity with two or more objects becomes an object
    group named for the activity.

    Parameters
    ----------
    file : escdf.File
        As `escdf.read` returns it.

    Returns
    -------
    Project
    """
    record = file.metadata.get(PROJECT_RECORD)
    settings: dict[str, Any] = {}
    links_record: list[dict[str, Any]] = []
    provenance: dict[str, Any] = {}
    if record is not None:
        for label, target in (('project.json', settings), ('links.json', links_record),
                              ('provenance.json', provenance)):
            blob = _attached(record, label)
            if blob is not None:
                loaded = json.loads(blob.decode('utf-8'))
                (target.update if isinstance(target, dict) else target.extend)(loaded)
    display = {v: k for k, v in settings.get('names', {}).items()}
    objects: dict[str, Any] = {}
    kept: dict[str, list[str]] = {}           # dataset name -> object names

    def add(name: str, dataset: escdf.Dataset) -> None:
        label = display.get(name) or dataset.descriptive_name or name
        for count, obj in enumerate(_objects_from(dataset), start=1):
            # a dataset split by abscissa: 'Run', 'Run (2)', as UNV's
            own = label if count == 1 else f'{label} ({count})'
            while own in objects:
                own += ' (2)'
            objects[own] = obj
            kept.setdefault(name, []).append(own)

    for name, dataset in file.metadata.items():
        if name != PROJECT_RECORD:
            add(name, dataset)
    for activity in file.activities.values():
        for name, dataset in activity.data.items():
            add(name, dataset)
    # in the order the project held them, when the file says; a
    # foreign file's order is its own
    order = [display[k] for k in settings.get('names', {}).values() if k in display]
    objects = {**{name: objects[name] for name in order if name in objects},
               **objects}
    project = Project(settings.get('name') or 'ESCDF import', objects,
                      settings.get('active_geometry'), settings.get('project_type'),
                      provenance=provenance)
    if links_record and all(m in objects for g in links_record for m in g['members']):
        project.object_groups = Project('x', object_groups=links_record).object_groups
    else:
        for activity in file.activities.values():
            members = [label for n in list(activity.data) + list(activity.links)
                       for label in kept.get(n, [])]
            if len(members) >= 2:
                project.link(*members, name=activity.descriptive_name or activity.name)
    return project

load

load(path: str | PathLike, **_ignored: Any) -> Project

Read an ESCDF file as a project (see from_file).

Parameters:

Name	Type	Description	Default
path	path - like	The file.	required

Returns:

Type	Description
Project

Source code in src/visualdynamics/io/escdf_objects.py

def load(path: str | os.PathLike, **_ignored: Any) -> Project:
    """Read an ESCDF file as a project (see `from_file`).

    Parameters
    ----------
    path : path-like
        The file.

    Returns
    -------
    Project
    """
    return from_file(escdf.read(path))