ragraph.datasets ¶

RaGraph built-in datasets¶

check ¶

check(name: str)

Check whether a dataset exists.

Source code in ragraph/datasets/__init__.py

def check(name: str):
    """Check whether a dataset exists."""
    available = enum()
    if name not in available:
        raise ValueError(
            "Dataset {} cannot be found. Please pick one of {}.".format(name, available)
        )

enum ¶

enum() -> List[str]

Enumerate all available graphs in datasets.

Source code in ragraph/datasets/__init__.py

def enum() -> List[str]:
    """Enumerate all available graphs in datasets."""
    return sorted(
        [
            d.name
            for d in HERE.iterdir()
            if d.is_dir()
            and not d.name.startswith(".")
            and not d.name.startswith("_")
            and not d.name == "esl"
        ]
    )

get ¶

get(name: str) -> Graph

Get a dataset.

Source code in ragraph/datasets/__init__.py

def get(name: str) -> Graph:
    """Get a dataset."""
    check(name)

    mod = importlib.import_module("ragraph.datasets.{}".format(name))
    nodes_path = HERE / name / (name + "_nodes.csv")
    edges_path = HERE / name / (name + "_edges.csv")

    node_weights = getattr(mod, "node_weights", None)
    edge_weights = getattr(mod, "edge_weights", None)
    return from_csv(
        nodes_path=nodes_path,
        edges_path=edges_path,
        csv_delimiter=";",
        iter_delimiter=";",
        node_weights=node_weights,
        edge_weights=edge_weights,
        name=name,
        kind="dataset",
    )

info ¶

info(name: str) -> str

Get information about a dataset.

Source code in ragraph/datasets/__init__.py

def info(name: str) -> str:
    """Get information about a dataset."""
    check(name)
    mod = importlib.import_module("ragraph.datasets.{}".format(name))
    doc = mod.__doc__
    return str(doc)

aircraft_engine ¶

Pratt & Whitney Aircraft Engine¶

A directed graph describing the Pratt & Whitney PW4098 commercial high bypass-ratio turbofan engine. The graph describes a combination of the actual hardware dependencies of the engine and those of the development teams involved with them. It is a weighted and directed graph featuring 60 elements, four of which are sometimes left out as they represent the integration teams and no individual hardware components. Weak and strong dependencies are distinguished using weights of 1 and 2, respectively.

Reference

Rowles, C. M. (1999). System integration analysis of a large commercial aircraft engine.

architecture_integral ¶

Integral architecture example¶

Graph describing a design problem using objects that need to be designed based on certain aspects. It is part of a set of three design problem graphs, each ideally solved with a different approach -- being a modular, integral or mixed architecture.

This particular graph describes a design problem which is ideally solved using a integral object architecture. That the architecture consists vertically integrated modules. That is, each module in one domain integrates with only one (or very few) in the next and there is no crossover (mixing and matching across domains like with a modular approach). This means that interfaces between modules in different domains are relatively few. A module's design solution should be applicable to every member.

Nodes are of kind "object" or "aspect_1", "aspect_2", and "aspect_3". The numbers represent different aspect domains. An "incidence" kind edge from an object to an aspect means that it possesses that aspect.

Authors

Tim Wilschut and Tiemen Schuijbroek, October 2019.

architecture_mix ¶

Mixed architecture example¶

Graph describing a design problem using objects that need to be designed based on certain aspects. It is part of a set of three design problem graphs, each ideally solved with a different approach -- being a modular, integral or mixed architecture.

This particular graph describes a design problem which is ideally solved using a mixed object architecture. That means that the objects are best put into modules with some, but little interfaces across domains with limited crossover (mixing and matching like with a modular approach). Each module should be designed based on all aspects that the objects members possess combined, while taking into account the shared dependencies with other domains. A module's design solution should be applicable to every member.

Nodes are of kind "object" or "aspect_1", "aspect_2", and "aspect_3". The numbers represent different aspect domains. An "incidence" kind edge from an object to an aspect means that it posesses that aspect.

Authors

Tim Wilschut and Tiemen Schuijbroek, October 2019.

architecture_modular ¶

Modular architecture example¶

Graph describing a design problem using objects that need to be designed based on certain aspects. It is part of a set of three design problem graphs, each ideally solved with a different approach -- being a modular, integral or mixed architecture.

This particular graph describes a design problem which is ideally solved using a modular object architecture. That means that the objects are best put into modules. Modules on different aspect domains can then be mixed and matched so achieve a solution for it's combined members. Each module should be designed based on all aspects that the objects members posess combined, while taking into account the shared dependencies with other domains. A module's design solution should be applicable to every member.

Nodes are of kind "object" or "aspect_1", "aspect_2", and "aspect_3". The numbers represent different aspect domains. An "incidence" kind edge from an object to an aspect means that it posesses that aspect.

Authors

Tim Wilschut and Tiemen Schuijbroek, October 2019.

climate_control ¶

Ford climate control system¶

This dataset describes a climate control system as it was to be found in Ford vehicles. Four different dependency types have been documented, being spatial, energy flow, information flow, and material flow dependencies. These are weighted from -2 to 2, where the following definitions have been used:

* +2: Required
* +1: Desired
*  0: Indifferent
* -1: Undesired
* -2: Detrimental

We have added an "adjacency" weight, which is the nonnegative sum of all dependencies between components.

Reference

Pimmler, T. U., & Eppinger, S. D. (1994). Integration Analysis of Product Decompositions. ASME Design Theory and Methodology Conference.

climate_control_mg ¶

Pre-clustered Ford climate control system using the Markov-Gamma heuristic¶

Pre-clustered using bus detection plus hierarchical clustering.

Reference

Pimmler, T. U., & Eppinger, S. D. (1994). Integration Analysis of Product Decompositions. ASME Design Theory and Methodology Conference.

compatibility ¶

Minimal compatibility analysis example¶

Contains 6 component variant nodes. They are divided in three node kinds (e.g. components), which correspond to the first character in their node names: A1, B1, B2, C1, C2, C3. For ease of usage, the "performance" weight of each node is set to it's node name's second character.

Compatibility between nodes is signalled using edges with a "compatibility" kind.

design ¶

Graph of a design problem complexity study. Components are related (have edges) to attributes that are of importance, which are of edge kind "incidence". Nodes have kind "component" or "attribute".

Reference

[Figure 1(a)] Chen, L., & Li, S. (2005). Analysis of Decomposability and Complexity for Design Problems in the Context of Decomposition. Journal of Mechanical Design, 127(4), 545. DOI: 10.1115/1.1897405

elevator175 ¶

Elevator system decomposed into 175 components¶

The Complex Elevator System is described using an undirected graph with multiple dependency types. It describes a machine-room-less elevator called the 'Kone MonoSpace'. It is designed for low- to midrise buildings and uses permanent-magnet electric motors. The five defined edge types are spatial, material, mechanical energy, electrical energy and information. It was published in a variation of 175 elements and a less granular variation of 45 elements. The less granular variation collapses a set of pre-defined modules and therefore contains less detail of the system.

Reference

Niutanen, V., Hölttä-otto, K., Rahardjo, A., & Stowe, H. M. (2017). Complex Elevator System DSM - Case for a DSM Design Sprint. In 19th International dependency and structure modeling conference, DSM 2017.

elevator45 ¶

Elevator system decomposed into 45 components¶

The Complex Elevator System is described using an undirected graph with multiple dependency types. It describes a machine-room-less elevator called the 'Kone MonoSpace'. It is designed for low- to midrise buildings and uses permanent-magnet electric motors. The five defined edge types are spatial, material, mechanical energy, electrical energy and information. It was published in a variation of 175 elements and a less granular variation of 45 elements. The less granular variation collapses a set of pre-defined modules and therefore contains less detail of the system.

Reference

Niutanen, V., Hölttä-otto, K., Rahardjo, A., & Stowe, H. M. (2017). Complex Elevator System DSM - Case for a DSM Design Sprint. In 19th International dependency and structure modeling conference, DSM 2017.

esl ¶

Elephant Specification Language datasets¶

check ¶

check(name: str)

Check whether a dataset exists.

Source code in ragraph/datasets/esl/__init__.py

def check(name: str):
    """Check whether a dataset exists."""
    available = enum()
    if name not in available:
        raise ValueError(
            "Dataset {} cannot be found. Please pick one of {}.".format(name, available)
        )

enum ¶

enum() -> List[str]

Enumerate all available ESL datasets.

Source code in ragraph/datasets/esl/__init__.py

def enum() -> List[str]:
    """Enumerate all available ESL datasets."""
    return sorted(
        [
            d.name
            for d in HERE.iterdir()
            if d.is_dir() and not d.name.startswith(".") and not d.name.startswith("_")
        ]
    )

get ¶

get(name: str)

Get a dataset.

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of the dataset to get (see `ragraph.datasets.esl.enum()`).	required

Source code in ragraph/datasets/esl/__init__.py

def get(name: str):
    """Get a dataset.

    Arguments:
        name: Name of the dataset to get (see `ragraph.datasets.esl.enum()`).
    """
    from ragraph.io.esl import from_esl

    check(name)

    esl_files = [str(p) for p in (HERE / name).glob("*.esl")]

    graph = from_esl(*esl_files)
    graph.kind = name

    return graph

info ¶

info(name: str) -> str

Get information about a dataset.

Source code in ragraph/datasets/esl/__init__.py

def info(name: str) -> str:
    """Get information about a dataset."""
    check(name)
    mod = importlib.import_module("ragraph.datasets.esl.{}".format(name))
    doc = mod.__doc__
    return str(doc)

pump ¶

A small example concerning a centrigal pump and its drive mechanism.

ford_hood ¶

Ford Motor Company Hood Development Process.

This DSM application was made to test the effectiveness of a DSM analysis to improve a highly developed iterative design process in terms of:

Reduction of product development lead time
Reduction of product development lead-time variation

As it is a process DSM, the nodes represent tasks in the design process. The edges represent dependencies between them. Each task in the design process was assigned a task volatility value, which is an indication of the probability of rework.

Node (task) weights:

EC(i) and EC(r) represent the initial and rework costs
ED(i) and ED(r) represent the initial and rework durations
Information variability (likelihood of input changing)

Edge weights:

volatility: an indication of the probability of rework when the dependent task is executed without properly waiting for its input to finish. It's value was determined using the information variability of the source (depended on) task and the sensitivity of the target (dependent) task.

Reference

Browning, Tyson R., and Steven D. Eppinger. 2002, November. Modeling Impacts of Process Architecture on Cost and Schedule Risk in Product Development. IEEE Transactions on Engineering Management 49 (4):428-442.

Reference

Krishnan, Viswanathan, Steven D. Eppinger, and Daniel E. Whitney. 1997, April. A Model-Based Framework to Overlap Product Development Activities. Management Science 43 (4):437-451.

Reference

Yassine, Ali A., Daniel E. Whitney, and Tony P. Zambito. 2001. Assessment of Rework Probabilities for Simulating Product Development Processes Using the Design Structure Matrix (DSM). ASME Design Engineering Technical Conferences, DTM-21693.

kodak3d ¶

Kodak Single-Use Camera product commonality¶

This DSM dataset was actually proposed as a "3D DSM" as it was featured in the reference papers in a 3D layered fashion. Relationships between components in a product family for different family members were shown in a 3D layered fashion. Common interfaces and unique interfaces were color coded accordingly for three family members: The "Fun Saver", "Outdoor", and "Plus Digital" single-use cameras.

This dataset contains all unique product family components as nodes, where each node is labeled with the family members it is used in. E.g. the node labels can be any combination of at least one of the camera type.

Edges indicate an interface between components, which is therefore modeled using an "adjacency" edge weight. Once again, they are labeled with the camera typename when that dependency occurs in the corresponding camera type.

Available labels for nodes or edges:

Fun Saver
Outdoor
Plus Digital

Reference

Alizon, Fabrice. 2009, February. Module-Based Design Management-Synerg'. Symposium on Product Family & Product Platform Design, Helsinki University of Technology (TKK), Helsinki, Finland.

Reference

Alizon, Fabrice, Seung K. Moon, Steven B. Shooter, and Timothy W. Simpson. 2007, September 4--7. Three Dimensional Design Structure Matrix-DSM3D. ASME Design Engineering Technical Conferences, DETCZ007-34510, Las Vegas, NV, pp. 941-948.

ledsip ¶

LED system-in-package (LEDSiP, 1200 components)¶

Reference: Wilschut, T. (2014). Analysis of the multi-disciplinary coupling structure of a LED system-in-package.

localbus ¶

Example graph containing a local bus¶

mww_lock_aspect ¶

Multi-WaterWerk project waterway lock-aspect mapping

mww_lock_eefde ¶

Multi-WaterWerk project waterway lock 'Eefde'

mww_lock_hansweert ¶

Multi-WaterWerk project waterway lock 'Hansweert'

mww_lock_sambeek ¶

Multi-WaterWerk project waterway lock 'Sambeek'

mww_lock_sluis15 ¶

Multi-WaterWerk project waterway lock 'Sluis 15'

mww_lock_volkerak ¶

Multi-WaterWerk project waterway lock 'Volkerak'

overlap ¶

Toy example with overlapping clusters¶

pathfinder ¶

NASA Mars Pathfinder Technology Risk DSM¶

The technology risk DSM (TR-DSM) helps to identify where most of the design effort should go if one wants to mitigate technological failures and improve robustness of a system. All components of a system are assigned with a technology risk factor, which indicates the probability of failure, unprovenness, or uncertainty in design. The technology risk factor at NASA is loosely defined as the inverse of the technology readiness level:

TRF	NASA Technology Readiness Level Definition	TRL
1	Actual system "flight proven" through successful mission operations	9
2	Actual system completed and "flight qualified" through test and demonstration	8
2	System prototype in a space environment	7
3	System/subsystem model or prototype demonstration in a relevant environment	6
4	Component and/or breadboard validation in relevant environment	5
4	Component and/or breadboard validation in laboratory environment	4
5	Analytical and experimental critical function and/or characteristic proof-of-concept	3
5	Technology concept and/or application formulated	2
5	Basic principles observed and reported	1

The dataset includes both an adjacency value as if the DSM were a regular product DSM with interface strength values and the technology risk value which is computed using:

1	`TR = TRF-source * TRF-target * adjacency`

Reference

Brady, Timothy K. 2002. Utilization of Dependency Structure Matrix Analysis to Assess Complex Project Designs. Proceedings of ASME Design Engineering Technical Conferences, no. DETCZ002/DTM-34031, Montreal, Canada.

shaja8 ¶

8 node directed graph example from Shaja and Sudhakar.¶

Reference

Shaja, A. S., & Sudhakar, K. (2010). Optimized sequencing of analysis components in multidisciplinary systems. Research in Engineering Design, 21(3), 173-187. DOI: 10.1007/s00163-009-0082-5

similarity ¶

Similarity analysis example¶

Example graph for similarity analysis. Products are related (have edges) to attributes that are of importance in product portfolio analysis. Edges have edge kind "incidence". Nodes have kind "product" or "attribute".

tarjans8 ¶

8 node directed graph example from Robert Tarjan.¶

Reference

[Figure 3] Tarjan, R. (1972). Depth-First Search and Linear Graph Algorithms. SIAM Journal on Computing, 1(2), 146-160. DOI: 10.1136/0201010

tss_electric ¶

Truck Steering System (TSS) [Electric]

The TSS is the subject of the industry workshop of the DSM Conference of 2023. It represents the steering system and related assemblies of a truck. Multiple datasets have been given that describe both technological solutions to the steering problem (product DSMs), as well as technology risk factors, process dependencies and organizational mappings.

The available datasets are:

tss_front: Product DSM for the "front axle only" solution with "risk" weights, as well as a process and organizational mapping.
tss_electric: Product DSM for the "electric" solution with "risk" weights.
tss_hydraulic: Product DSM for the "hydraulic" solution with "risk" weights.

The associated "risk" weights have been copied/assumed from the "front axle only" example.

The truck's intended operating area is Sweden, which imposes harsh (cold) conditions. The following risks have been identified:

Sudden loss of tire pressure can result in reduced steering control, potentially leading to a loss of vehicle control. This can strain other steering system components, such as the hub, steering knuckle, and tie rod.
Bearing failure can cause excessive friction and heat generation, affecting the hub's structural integrity. This can lead to further damage to the wheel, steering knuckle, and tie rod.
Cracking of the steering knuckle can weaken the axle assembly and may lead to a loss of steering control. It can also damage the hub and tie rod.
Bending of the knuckle arm can disrupt the steering geometry, affecting the steering system's overall performance. It can also cause excessive wear on the steering knuckle and tie rod.
Cracks in the frame can weaken the structural integrity of the entire front axle assembly, potentially leading to catastrophic failure. It can adversely affect all connected components and overall vehicle stability.
Fluid leaks can result in reduced hydraulic pressure, causing a loss of steering power and control. It may also affect the servo valve and hydraulic pump.
Bending or breaking of the tie rod can lead to loss of steering control, affecting wheel alignment and potentially causing damage to the steering knuckle and other components.
A malfunctioning angle sensor can lead to inaccurate steering data, potentially causing the ECU to make incorrect steering adjustments, impacting steering performance.
Valve issues can lead to erratic or delayed steering response, affecting overall steering system performance and potentially causing strain on the steering pump and hydraulic oil reservoir.
Cold start problems can affect the engine's ability to provide power for the hydraulic pump, reducing hydraulic pressure and steering assistance.
Speedometer malfunction doesn't directly impact steering but can affect the driver's ability to monitor vehicle speed, potentially leading to unsafe driving conditions. ECU malfunction can lead to incorrect steering commands, impacting steering performance and potentially straining other hydraulic system components.
Thickening or freezing of hydraulic oil can reduce the effectiveness of the hydraulic system, potentially leading to reduced steering control and increased strain on the hydraulic pump.
Pump malfunction can lead to reduced hydraulic pressure, affecting steering performance and potentially straining other hydraulic components.
A clogged filter can reduce hydraulic flow, potentially leading to reduced steering power and increased load on the hydraulic pump and other components.
A malfunctioning cooler can result in overheating of the hydraulic fluid, potentially reducing system efficiency and impacting other hydraulic components' performance.

tss_front ¶

Truck Steering System (TSS) [Front]

The TSS is the subject of the industry workshop of the DSM Conference of 2023. It represents the steering system and related assemblies of a truck. Multiple datasets have been given that describe both technological solutions to the steering problem (product DSMs), as well as technology risk factors, process dependencies and organizational mappings.

The available datasets are:

tss_front: Product DSM for the "front axle only" solution with "risk" weights, as well as a process and organizational mapping.
tss_electric: Product DSM for the "electric" solution with "risk" weights.
tss_hydraulic: Product DSM for the "hydraulic" solution with "risk" weights.

The associated "risk" weights have been copied/assumed from the "front axle only" example.

The truck's intended operating area is Sweden, which imposes harsh (cold) conditions. The following risks have been identified:

Sudden loss of tire pressure can result in reduced steering control, potentially leading to a loss of vehicle control. This can strain other steering system components, such as the hub, steering knuckle, and tie rod.
Bearing failure can cause excessive friction and heat generation, affecting the hub's structural integrity. This can lead to further damage to the wheel, steering knuckle, and tie rod.
Cracking of the steering knuckle can weaken the axle assembly and may lead to a loss of steering control. It can also damage the hub and tie rod.
Bending of the knuckle arm can disrupt the steering geometry, affecting the steering system's overall performance. It can also cause excessive wear on the steering knuckle and tie rod.
Cracks in the frame can weaken the structural integrity of the entire front axle assembly, potentially leading to catastrophic failure. It can adversely affect all connected components and overall vehicle stability.
Fluid leaks can result in reduced hydraulic pressure, causing a loss of steering power and control. It may also affect the servo valve and hydraulic pump.
Bending or breaking of the tie rod can lead to loss of steering control, affecting wheel alignment and potentially causing damage to the steering knuckle and other components.
A malfunctioning angle sensor can lead to inaccurate steering data, potentially causing the ECU to make incorrect steering adjustments, impacting steering performance.
Valve issues can lead to erratic or delayed steering response, affecting overall steering system performance and potentially causing strain on the steering pump and hydraulic oil reservoir.
Cold start problems can affect the engine's ability to provide power for the hydraulic pump, reducing hydraulic pressure and steering assistance.
Speedometer malfunction doesn't directly impact steering but can affect the driver's ability to monitor vehicle speed, potentially leading to unsafe driving conditions. ECU malfunction can lead to incorrect steering commands, impacting steering performance and potentially straining other hydraulic system components.
Thickening or freezing of hydraulic oil can reduce the effectiveness of the hydraulic system, potentially leading to reduced steering control and increased strain on the hydraulic pump.
Pump malfunction can lead to reduced hydraulic pressure, affecting steering performance and potentially straining other hydraulic components.
A clogged filter can reduce hydraulic flow, potentially leading to reduced steering power and increased load on the hydraulic pump and other components.
A malfunctioning cooler can result in overheating of the hydraulic fluid, potentially reducing system efficiency and impacting other hydraulic components' performance.

tss_hydraulic ¶

Truck Steering System (TSS) [Hydraulic]

The TSS is the subject of the industry workshop of the DSM Conference of 2023. It represents the steering system and related assemblies of a truck. Multiple datasets have been given that describe both technological solutions to the steering problem (product DSMs), as well as technology risk factors, process dependencies and organizational mappings.

The available datasets are:

tss_front: Product DSM for the "front axle only" solution with "risk" weights, as well as a process and organizational mapping.
tss_electric: Product DSM for the "electric" solution with "risk" weights.
tss_hydraulic: Product DSM for the "hydraulic" solution with "risk" weights.

The associated "risk" weights have been copied/assumed from the "front axle only" example.

The truck's intended operating area is Sweden, which imposes harsh (cold) conditions. The following risks have been identified:

Sudden loss of tire pressure can result in reduced steering control, potentially leading to a loss of vehicle control. This can strain other steering system components, such as the hub, steering knuckle, and tie rod.
Bearing failure can cause excessive friction and heat generation, affecting the hub's structural integrity. This can lead to further damage to the wheel, steering knuckle, and tie rod.
Cracking of the steering knuckle can weaken the axle assembly and may lead to a loss of steering control. It can also damage the hub and tie rod.
Bending of the knuckle arm can disrupt the steering geometry, affecting the steering system's overall performance. It can also cause excessive wear on the steering knuckle and tie rod.
Cracks in the frame can weaken the structural integrity of the entire front axle assembly, potentially leading to catastrophic failure. It can adversely affect all connected components and overall vehicle stability.
Fluid leaks can result in reduced hydraulic pressure, causing a loss of steering power and control. It may also affect the servo valve and hydraulic pump.
Bending or breaking of the tie rod can lead to loss of steering control, affecting wheel alignment and potentially causing damage to the steering knuckle and other components.
A malfunctioning angle sensor can lead to inaccurate steering data, potentially causing the ECU to make incorrect steering adjustments, impacting steering performance.
Valve issues can lead to erratic or delayed steering response, affecting overall steering system performance and potentially causing strain on the steering pump and hydraulic oil reservoir.
Cold start problems can affect the engine's ability to provide power for the hydraulic pump, reducing hydraulic pressure and steering assistance.
Speedometer malfunction doesn't directly impact steering but can affect the driver's ability to monitor vehicle speed, potentially leading to unsafe driving conditions. ECU malfunction can lead to incorrect steering commands, impacting steering performance and potentially straining other hydraulic system components.
Thickening or freezing of hydraulic oil can reduce the effectiveness of the hydraulic system, potentially leading to reduced steering control and increased strain on the hydraulic pump.
Pump malfunction can lead to reduced hydraulic pressure, affecting steering performance and potentially straining other hydraulic components.
A clogged filter can reduce hydraulic flow, potentially leading to reduced steering power and increased load on the hydraulic pump and other components.
A malfunctioning cooler can result in overheating of the hydraulic fluid, potentially reducing system efficiency and impacting other hydraulic components' performance.

ucav ¶

UCAV (unmanned combat aerial vehicle) preliminary design process.¶

The graph contains 14 preliminary design activities. Each phase consists of an initial activity to define design several disciplines-such as aerodynamics, propulsion, stability and control (S&C), distribute a design configuration (a design concept proposed to satisfy the DR&O). Then, requirements and objectives (DR&O), followed by a couple of activities to create and mechanical and electrical, weights, and performance - each evaluate the configuration from their own perspective.

The nodes contain a minimum, mean and max weight for both duration and cost, as well as an improvement curve value. The improvement curve determines the savings in work for each successive iteration of an activity. The edges contain a binary weight and rework probability and rework impact annotations.

When an activity is performed before all activities with edges to that activity has been succesfully executed, there is a rework probability that the activity needs to be redone, the impact then indicates the percentage that needs to be redone and the improvement curve decreases this in successive iterations.

Reference

Eppinger, S. D., & Browning, T. R. (2012). Design Structure Matrix - Methods and Applications.