Apps

basicApp.py

class BasicApp(title, subtitle, description, page_type, layout=[], logo=None, footer=None)

Bases: object

Defines what an App is in the report_manager. Other Apps will inherit basic functionality from this class. Attributes: Title, subtitle, description, logo, footer.

property title

property subtitle

property description

property page_type

property logo

property footer

property layout

add_to_layout(section)

extend_layout(sections)

get_HTML_title()

get_HTML_subtitle()

get_HTML_description()

add_basic_layout()

Calls class functions to setup the layout: title, subtitle, description, logo and footer.

build_page()

Builds page basic layout.

dataUploadApp.py

class DataUploadApp(title, subtitle, description, layout=[], logo=None, footer=None)

Bases: apps.basicApp.BasicApp

Defines what the dataUpload App is in the report_manager. Used to upload experimental and clinical data to correct project folder.

Warning

There is a size limit of 55MB. Files bigger than this will have to be moved manually.

buildPage()

Builds page with the basic layout from basicApp.py and adds relevant Dash components for project data upload.

dataUpload.py

get_data_upload_queries()

Reads the YAML file containing the queries relevant to parsing of clinical data and returns a Python object (dict[dict]).

Returns

Nested dictionary.

get_new_subject_identifier(driver)

Queries the database for the last subject identifier and returns a new sequential identifier.

Parameters

• driver (py2neo driver) – py2neo driver, which provides the connection to the neo4j graph database.

• projectId (str) – external project identifier (from the graph database).

Returns

Subject identifier.

Return type

str

get_new_biosample_identifier(driver)

Queries the database for the last biological sample internal identifier and returns a new sequential identifier.

Parameters

driver – py2neo driver, which provides the connection to the neo4j graph database.

Returns

Biological sample identifier.

get_new_analytical_sample_identifier(driver)

Queries the database for the last analytical sample internal identifier and returns a new sequential identifier. :param driver: py2neo driver, which provides the connection to the neo4j graph database.

Returns

Analytical sample identifier.

get_subjects_enrolled_in_project(driver, projectId)

Extracts the number of subjects included in a given project.

Parameters

• driver (py2neo driver) – py2neo driver, which provides the connection to the neo4j graph database.

• projectId (str) – external project identifier (from the graph database).

Returns

Number of subjects.

Return type

Numpy ndarray

check_samples_in_project(driver, projectId)

check_external_ids_in_db(driver, projectId)

remove_samples_nodes_db(driver, projectId)

create_new_subjects(driver, data, projectId)

Parameters

• driver – py2neo driver, which provides the connection to the neo4j graph database.

• data – pandas Dataframe with clinical data as columns and samples as rows.

• projectId (string) – project identifier.

Returns

Pandas DataFrame where new biological sample internal identifiers have been added.

create_new_biosamples(driver, data)

Parameters

• driver – py2neo driver, which provides the connection to the neo4j graph database.

• data – pandas Dataframe with clinical data as columns and samples as rows.

Returns

Pandas DataFrame where new biological sample internal identifiers have been added.

create_new_ansamples(driver, data)

Parameters

• driver – py2neo driver, which provides the connection to the neo4j graph database.

• data – pandas Dataframe with clinical data as columns and samples as rows.

Returns

Pandas DataFrame where new analytical sample internal identifiers have been added.

create_experiment_internal_identifiers(driver, projectId, data, directory, filename)

create_mapping_cols_clinical(driver, data, directory, filename, separator='|')

Parameters

• driver (py2neo driver) – py2neo driver, which provides the connection to the neo4j graph database.

• data – pandas Dataframe with clinical data as columns and samples as rows.

• separator (str) – character used to separate multiple entries in an attribute.

Returns

Pandas Dataframe with all clinical data and graph database internal identifiers.

get_project_information(driver, project_id)

homepageApp.py

class HomePageApp(title, subtitle, description, layout=[], logo=None, footer=None)

Bases: apps.basicApp.BasicApp

Defines what the HomePage App is in the report_manager. Enables the tracking of the number of entitites, relationships, projects, and others, currently in the grapha database.

buildPage()

Builds page with the basic layout from basicApp.py and adds all the relevant plots from homepageStats.py.

homepageStats.py

size_converter(value)

Converts a given value to the highest possible unit, maintaining two decimals.

Parameters

or float value (int) –

Returns

String with converted value and units.

get_query()

Reads the YAML file containing the queries relevant for graph database stats, parses the given stream and returns a Python object (dict[dict]).

Returns

Nested dictionary.

get_db_schema()

Retrieves the database schema

Returns

network with all the database nodes and how they are related

get_db_stats_data()

Retrieves all the stats data from the graph database and returns them as a dictionary.

Returns

Dictionary of dataframes.

plot_store_size_components(dfs, title, args)

Plots the store size of different components of the graph database, as a Pie Chart.

Parameters

• dfs (dict) – dictionary of json objects.

• title (str) – title of the Dash div where plot is located.

• args (dict) – see below.

Arguments

• valueCol (str) – name of the column with the values to be plotted.

• textCol (str) – name of the column containing information for the hoverinfo parameter.

• height (str) – height of the plot.

• width (str) – width of the plot.

Returns

New Dash div containing title and pie chart.

plot_node_rel_per_label(dfs, title, args, focus='nodes')

Plots the number of nodes or relationships (depending on ‘focus’) per label, contained in the grapha database.

Parameters

• dfs (dict) – dictionary of json objects.

• title (str) – title of the Dash div where plot is located.

Paeam str focus

plot number of nodes per label (‘nodes’) or the number of relationships per type (‘relationships’).

Returns

New Dash div containing title and barplot.

indicator(color, text, id_value)

Builds a new Dash div styled as a container, with borders and background.

Parameters

• color (str) – background color of the container (RGB or Hex colors).

• text (str) – name to be plotted inside the container.

• id_value (str) – identifier of the container.

Returns

Dash div containing title and an html.P element.

quick_numbers_panel()

Creates a panel of Dash containers where an overviem of the graph database numbers can be plotted.

Returns

List of Dash components.

importsApp.py

class ImportsApp(title, subtitle, description, layout=[], logo=None, footer=None)

Bases: apps.basicApp.BasicApp

Defines what the imports App is in the report_manager. Enables the tracking of the number of imported entitites and relationships, updates and file sizes.

buildPage()

Builds page with the basic layout from basicApp.py and adds all the relevant plots from imports.py.

imports.py

get_stats_data(filename, n=3)

Reads graph database stats file and filters for the last ‘n’ full and partial independent imports, returning a Pandas DataFrame.

Parameters

• filename (str) – path to stats file (including filename and ‘.hdf’ extension).

• n (int) – number of independent imports to plot.

Returns

Pandas Dataframe with different entities and relationships as rows and columns:

select_last_n_imports(stats_file, n=3)

Selects which independent full and partial imports should be plotted based on n.

Parameters

• stats_file – pandas DataFrame with stats data.

• n (int) – number of independent imports to select.

Returns

List of import ids to be plotted according to selection criterion.

remove_legend_duplicates(figure)

Removes duplicated legend items.

Parameters

figure – plotly graph object figure.

get_databases_entities_relationships(stats_file, key='full', options='databases')

Builds dictionary from stats file. Depending on ‘options’, keys and values can differ. If options is set to ‘dates’, keys are dates of the imports and values are databases imported at each date; if ‘databases’, keys are databases and values are entities and relationships created from each database; if ‘entities’, keys are databases and values are entities created from each database; if ‘relationships’, keys are databases and values are relationships created from each database.

Parameters

• stats_file – pandas DataFrame with stats data.

• key (str) – use only full, partial or both kinds of imports (‘full’, ‘partial’, ‘all’).

• options (str) – name of the variables to be used as keys in the output dictionary (‘dates’, ‘databases’, ‘entities’ or ‘relationships’).

Returns

Dictionary.

set_colors(dictionary)

This function takes the values in a dictionary and attributes them an RGB color.

Parameters

dictionary (dict) – dictionary with variables to be attributed a color, as values.

Returns

Dictionary where ‘dictionary’ values are keys and random RGB colors are the values.

get_dropdown_menu(fig, options_dict, add_button=True, equal_traces=True, number_traces=2)

Builds a list for the dropdown menu, based on a plotly figure traces and a dictionary with the options to be used in the dropdown.

Parameters

• fig – plotly graph object figure.

• options_dict – dictionary where keys are used as dropdown options and values data points.

• add_button (bool) – add option to display all dropdown options simultaneously.

• equal_traces (bool) – defines if all dropdown options have the same number of traces each. If True, define ‘number_traces’ as well. If False, number of traces will be the same as the number of values for each ‘options_dict’ key.

• number_traces (int) – number of traces created for each ‘options_dict’ key.

Returns

List of nested structures. Each dictionary within updatemenus[0][‘buttons’][0] corresponds to one dropdown menu options and contains information on which traces are visible, label and method.

get_totals_per_date(stats_file, key='full', import_types=False)

Summarizes stats file to a Pandas DataFrame with import dates and total number of imported entities and relationships.

Parameters

• stats_file – pandas DataFrame with stats data.

• key (str) – use only full or partial imports (‘full’, ‘partial’).

• import_types (bool) – breakdown importing stats into entities or relationships related.

Returns

Pandas DataFrame with independent import dates as rows and imported numbers as columns.

get_imports_per_database_date(stats_file)

Summarizes stats file to a Pandas DataFrame with import dates, databases and total number of imported entities and relationships per database.

Parameters

stats_file – pandas DataFrame with stats data.

Returns

Pandas DataFrame with independent import dates and databases as rows and imported numbers as columns.

plot_total_number_imported(stats_file, plot_title)

Creates plot with overview of imports numbers per date.

Parameters

• stats_file – pandas DataFrame with stats data.

• plot_title (str) – title of the plot.

Returns

Line plot figure within the <div id=”_dash-app-content”>.

plot_total_numbers_per_date(stats_file, plot_title)

Plots number of entities and relationships imported per date, with scaled markers reflecting numbers rations.

Parameters

• stats_file – pandas DataFrame with stats data.

• plot_title (str) – title of the plot.

Returns

Scatter plot figure within the <div id=”_dash-app-content”>, with scalled markers.

plot_databases_numbers_per_date(stats_file, plot_title, key='full', dropdown=False, dropdown_options='dates')

Grouped horizontal barplot showing the number of entities and relationships imported from each biomedical database.

Parameters

• stats_file – pandas DataFrame with stats data.

• plot_title (str) – title of the plot.

• key (str) – use only full or partial imports (‘full’, ‘partial’).

• dropdown (bool) – add dropdown menu to figure or not.

• dropdown_options (str) – name of the variables to be used as options in the dropdown menu (‘dates’, ‘databases’, ‘entities’ or ‘relationships’).

Returns

Horizontal barplot figure within the <div id=”_dash-app-content”>.

plot_import_numbers_per_database(stats_file, plot_title, key='full', subplot_titles=('', ''), colors=True, plots_1='entities', plots_2='relationships', dropdown=True, dropdown_options='databases')

Creates plotly multiplot figure with breakdown of imported numbers and size of the respective files, per database and import type (entities or relationships).

Parameters

• stats_file – pandas DataFrame with stats data.

• plot_title (str) – title of the plot.

• key (str) – use only full or partial imports (‘full’, ‘partial’).

• subplot_titles (tuple) – title of the subplots (tuple of strings, one for each subplot).

• colors (bool) – define standard colors for entities and for relationships.

• plots_1 (str) – name of the variable plotted.

• plots_2 (str) – name of the variable plotted.

• dropdown (bool) – add dropdown menu to figure or not.

• dropdown_options (str) – name of the variables to be used as options in the dropdown menu (‘dates’, ‘databases’, ‘entities’ or ‘relationships’).

Returns

Multi-scatterplot figure within the <div id=”_dash-app-content”>.

initialApp.py

loginApp.py

class LoginApp(title, subtitle, description, layout=[], logo=None, footer=None)

Bases: apps.basicApp.BasicApp

Defines the login App Enables user to access the reports

buildPage()

Builds page with the basic layout from basicApp.py and adds the login form.

projectApp.py

class ProjectApp(id, projectId, title, subtitle, description, layout=[], logo=None, footer=None, force=False)

Bases: apps.basicApp.BasicApp

Defines what a project App is in the report_manager. Includes multiple tabs for different data types.

property id

Retrieves page identifier.

property session_id

Retrieves session identifier.

property project_id

Retrieves project identifier.

property configuration_files

Retrieves project configuration files.

property force

Retrieves attribute force (whether or not the project report needs to be regenerated).

build_header()

build_page()

Builds project and generates the report. For each data type in the report (e.g. ‘proteomics’, ‘clinical’), creates a designated tab. A button to download the entire project and report is added.

projectCreationApp.py

class ProjectCreationApp(title, subtitle, description, layout=[], logo=None, footer=None)

Bases: apps.basicApp.BasicApp

Defines what the project creation App is in the report_manager. Includes multiple fill in components to gather project information and metadata.

buildPage()

Builds page with the basic layout from basicApp.py and adds relevant Dash components for project creation.

projectCreation.py

get_project_creation_queries()

Reads the YAML file containing the queries relevant to user creation, parses the given stream and returns a Python object (dict[dict]).

Returns

Nested dictionary.

check_if_node_exists(driver, node, node_property, value)

Queries the graph database and checks if a node with a specific property and property value already exists. :param driver: py2neo driver, which provides the connection to the neo4j graph database. :type driver: py2neo driver :param str node: node to be matched in the database. :param str node_property: property of the node. :param value: property value. :type value: str, int, float or bool :return: Pandas dataframe with user identifier if User with node_property and value already exists, if User does not exist, returns and empty dataframe.

get_new_project_identifier(driver, projectId)

Queries the database for the last project external identifier and returns a new sequential identifier.

Parameters

• driver (py2neo driver) – py2neo driver, which provides the connection to the neo4j graph database.

• projectId (str) – internal project identifier (CPxxxxxxxxxxxx).

Returns

Project external identifier.

Return type

str

get_subject_number_in_project(driver, projectId)

Extracts the number of subjects included in a given project.

Parameters

• driver (py2neo driver) – py2neo driver, which provides the connection to the neo4j graph database.

• projectId (str) – external project identifier (from the graph database).

Returns

Integer with the number of subjects.

create_new_project(driver, projectId, data, separator='|')

Creates a new project in the graph database, following the steps:

1. Retrieves new project external identifier and creates project node and relationships in the graph database.

2. Creates subjects, timepoints and intervention nodes.

3. Saves all the entities and relationships to tab-delimited files.

4. Returns the number of projects created and the project external identifier.

Parameters

• driver (py2neo driver) – py2neo driver, which provides the connection to the neo4j graph database.

• projectId (str) – internal project identifier (CPxxxxxxxxxxxx).

• data – pandas Dataframe with project as row and other attributes as columns.

• separator (str) – character used to separate multiple entries in a project attribute.

Returns

Two strings: number of projects created and the project external identifier.