{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "\"FMP\"\n", "\"AudioLabs\"\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "\"B\"\n", "

libfmp

\n", "
\n", "\n", "
\n", "\n", "

\n", "This notebook gives a short introduction to the libfmp-package, the Python package that accompanies the FMP notebooks. After a short introduction to Python modules and packages, we explain how to the libfmp-package is structured and how it can be used. The libfmp-package is also provided at GitHub and PyPi (being \n", "installable via pip install libfmp). Further details can be found below and in the following publication:\n", "\n", "

\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Python Modules\n", "\n", "A Python module is basically a file with an extension `.py` containing Python code. The content of a module can be accessed with the `import` statement. As an example, we consider the file `b_test_module.py` contained in the folder `libfmp/B/`. When the `import` statement is executed, the interpreter searches for `b_test_module.py` in a list of directories which specifies the search paths for modules. The variable `sys.path` (which is part of the module `sys`) yields the list of directories. It is initialized from the environment variable `PYTHONPATH` (plus an installation-dependent default). The list contained in `sys.path` can be extended using the function `sys.path.append`. The following example illustrates these concepts:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2024-02-15T09:01:14.961052Z", "iopub.status.busy": "2024-02-15T09:01:14.960845Z", "iopub.status.idle": "2024-02-15T09:01:14.964837Z", "shell.execute_reply": "2024-02-15T09:01:14.964205Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['/Users/zal/dev/Python_FMP_notebooks/FMP/B', '/Users/zal/miniconda3/envs/FMP/lib/python38.zip', '/Users/zal/miniconda3/envs/FMP/lib/python3.8', '/Users/zal/miniconda3/envs/FMP/lib/python3.8/lib-dynload', '', '/Users/zal/miniconda3/envs/FMP/lib/python3.8/site-packages', '/Users/zal/miniconda3/envs/FMP/lib/python3.8/site-packages/IPython/extensions', '/Users/zal/.ipython'] \n", "\n", "['/Users/zal/dev/Python_FMP_notebooks/FMP/B', '/Users/zal/miniconda3/envs/FMP/lib/python38.zip', '/Users/zal/miniconda3/envs/FMP/lib/python3.8', '/Users/zal/miniconda3/envs/FMP/lib/python3.8/lib-dynload', '', '/Users/zal/miniconda3/envs/FMP/lib/python3.8/site-packages', '/Users/zal/miniconda3/envs/FMP/lib/python3.8/site-packages/IPython/extensions', '/Users/zal/.ipython', '..'] \n", "\n" ] } ], "source": [ "import sys\n", "print(sys.path, '\\n')\n", "\n", "import os\n", "sys.path.append('..')\n", "print(sys.path, '\\n')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once the directory of the module is in the search path, we can use the `import` statement. Let us come back to our example `test_module.py`, which has the following content:\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2024-02-15T09:01:14.995029Z", "iopub.status.busy": "2024-02-15T09:01:14.994823Z", "iopub.status.idle": "2024-02-15T09:01:15.000657Z", "shell.execute_reply": "2024-02-15T09:01:15.000116Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\"\"\"\n", "Module: libfmp.b.b_test_module\n", "Author: Meinard Mueller\n", "License: The MIT license, https://opensource.org/licenses/MIT\n", "\n", "This file is part of the FMP Notebooks (https://www.audiolabs-erlangen.de/FMP).\n", "\"\"\"\n", "\n", "string = 'This is a test function'\n", "a, b, c = 1, 2, 3\n", "\n", "\n", "def add(a, b=0, c=0):\n", " \"\"\"Function to add three numbers\n", "\n", " | Notebook: B/B_libfmp.ipynb and\n", " | Notebook: B/B_PythonBasics.ipynb\n", "\n", " Args:\n", " a (float): First number\n", " b (float): Second number (default: 0)\n", " c (float): Third number (default: 0)\n", "\n", " Returns:\n", " d (float): Sum\n", " \"\"\"\n", " d = a + b + c\n", " print('Addition: ', a, ' + ', b, ' + ', c, ' = ', d)\n", " return d\n", "\n" ] } ], "source": [ "import os\n", "\n", "fn = os.path.join('..', 'libfmp', 'b', 'b_test_module.py')\n", "with open(fn, 'r', encoding='utf-8') as stream:\n", " content_text = stream.read()\n", " \n", "print(content_text)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The following options import the module `b_test_module` or some of its elements:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2024-02-15T09:01:15.003299Z", "iopub.status.busy": "2024-02-15T09:01:15.003120Z", "iopub.status.idle": "2024-02-15T09:01:17.366806Z", "shell.execute_reply": "2024-02-15T09:01:17.366105Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Addition: 1 + 2 + 3 = 6\n", "Addition: 4 + 5 + 0 = 9\n", "Addition: 6 + 0 + 0 = 6\n", "Addition: 1 + 2 + 3 = 6\n" ] } ], "source": [ "import libfmp.b.b_test_module\n", "result = libfmp.b.b_test_module.add(libfmp.b.b_test_module.a, \n", " libfmp.b.b_test_module.b, \n", " libfmp.b.b_test_module.c)\n", "\n", "from libfmp.b.b_test_module import add\n", "result = add(4, 5)\n", "\n", "from libfmp.b.b_test_module import add as s\n", "result = s(6)\n", "\n", "from libfmp.b.b_test_module import *\n", "result = add(a, b, c)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The file variable `test_module.__file__` determines the path where the module was found. Furthermore, when a `.py`-file is imported as a module, Python sets the variable `__name__` to the name of the module. Finally, the help-function shows the documentation of the specified module." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2024-02-15T09:01:17.369748Z", "iopub.status.busy": "2024-02-15T09:01:17.369454Z", "iopub.status.idle": "2024-02-15T09:01:17.373569Z", "shell.execute_reply": "2024-02-15T09:01:17.373053Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Directory of module: /Users/zal/dev/Python_FMP_notebooks/FMP/B/../libfmp/b/b_test_module.py\n", "Name of module: libfmp.b.b_test_module\n", "=======================================\n", "Help on module libfmp.b.b_test_module in libfmp.b:\n", "\n", "NAME\n", " libfmp.b.b_test_module\n", "\n", "DESCRIPTION\n", " Module: libfmp.b.b_test_module\n", " Author: Meinard Mueller\n", " License: The MIT license, https://opensource.org/licenses/MIT\n", " \n", " This file is part of the FMP Notebooks (https://www.audiolabs-erlangen.de/FMP).\n", "\n", "FUNCTIONS\n", " add(a, b=0, c=0)\n", " Function to add three numbers\n", " \n", " | Notebook: B/B_libfmp.ipynb and\n", " | Notebook: B/B_PythonBasics.ipynb\n", " \n", " Args:\n", " a (float): First number\n", " b (float): Second number (default: 0)\n", " c (float): Third number (default: 0)\n", " \n", " Returns:\n", " d (float): Sum\n", "\n", "DATA\n", " a = 1\n", " b = 2\n", " c = 3\n", " string = 'This is a test function'\n", "\n", "FILE\n", " /Users/zal/dev/Python_FMP_notebooks/FMP/libfmp/b/b_test_module.py\n", "\n", "\n" ] } ], "source": [ "print('Directory of module:', libfmp.b.b_test_module.__file__)\n", "print('Name of module:', libfmp.b.b_test_module.__name__)\n", "print('=======================================')\n", "help(libfmp.b.b_test_module)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that any `.py`-file that contains a module can be also be executed as a Python script (e.g., `python test_module.py`). In the case that a file is run as a script, the variable `__name__` is set to the string `'__main__'`. This allows for placing additional statements in the module that are executed only when being run as a script (and not when imported as a module). For example, one can place these elements in a conditional (`if`) block as follows:\n", "```\n", "if (__name__ == '__main__'):\n", " Statements only executed when run as a script\n", "``` " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Python Packages\n", "\n", "A Python package is a namespace that consists of a directory, which in turn may contain subdirectories (subpackages) and files (modules). The naming convention follows the hierarchical file structure using **dot notation**. Opposed to normal directories, a package in Python typically contains a particular file called `__init__.py` (until Python 3.3, the existence of such a file was even mandatory). This file is automatically executed when the package (or a module in the package) is imported. For example, this allows for initializing package-specific data or for automatically importing specific modules from a package.\n", "\n", "Continuing our example above, the directory `B` can be regarded as a package containing an initialization file `__init__.py`. The following examples show how to import packages and modules:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2024-02-15T09:01:17.376239Z", "iopub.status.busy": "2024-02-15T09:01:17.375982Z", "iopub.status.idle": "2024-02-15T09:01:17.379979Z", "shell.execute_reply": "2024-02-15T09:01:17.379477Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Addition: 1 + 2 + 3 = 6\n", "Addition: 1 + 2 + 3 = 6\n" ] } ], "source": [ "import sys\n", "import os\n", "sys.path.append('..')\n", "\n", "import libfmp.b.b_test_module\n", "result = libfmp.b.b_test_module.add(libfmp.b.b_test_module.a, \n", " libfmp.b.b_test_module.b, \n", " libfmp.b.b_test_module.c)\n", "\n", "from libfmp.b import b_test_module\n", "result = b_test_module.add(b_test_module.a, b_test_module.b, b_test_module.c)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The Package `libfmp` \n", "\n", "In the FMP notebooks, the naming conventions and the code structure are carefully matched to the mathematical notation used in the textbook [Müller, FMP, Springer 2015] to establish a close relationship between theory and practice. Furthermore, the programming style is kept explicit and straightforward with a flat, functional hierarchy. Most code examples are directly specified in the code cells interleaved with text cells containing explanations. This approach leads to redundancies, where individual code fragments may occur several times in the FMP notebooks. In this sense, the FMP notebooks are not designed to serve as a toolbox per se. Nevertheless, we have assembled a Python package called `libfmp`, which serves several purposes. \n", "\n", "* First, `libfmp` contains some simple reference implementation for the most essential functionalities introduced in the FMP notebooks. \n", "* Second, the package allows for reducing some redundancies in the FMP notebook at places where implementation issues are not the focus. \n", "* Third, we hope that `libfmp` provides some instructive examples on how to document and build up modules, packages, and libraries in Python." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The package `libfmp` is organized along with the structure of the FMP notebooks. Containing subpackages called `B`, `C1`, `C2`, ..., `C8`, the file structure of `libfmp` is as follows:\n", "\n", "```\n", "libfmp\n", "├── __init__.py\n", "├── C1\n", "│ ├── __init__.py\n", "│ ├── ...\n", "⋮ \n", "└── C8\n", " ├── __init__.py\n", " ├── ...\n", "```\n", "\n", "\n", "The package `libfmp` is located in the main directory of the FMP notebooks (which is the parent directory of the current notebook `B_libfmp.ipynb`). Therefore, to access `libfmp` from the current path, we need to add the parent directory (`'..'`) to Python's search path. In the following example, the package `libfmp` is imported and its help page is displayed." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2024-02-15T09:01:17.382584Z", "iopub.status.busy": "2024-02-15T09:01:17.382321Z", "iopub.status.idle": "2024-02-15T09:01:17.389732Z", "shell.execute_reply": "2024-02-15T09:01:17.389212Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on package libfmp:\n", "\n", "NAME\n", " libfmp\n", "\n", "PACKAGE CONTENTS\n", " b (package)\n", " c1 (package)\n", " c2 (package)\n", " c3 (package)\n", " c4 (package)\n", " c5 (package)\n", " c6 (package)\n", " c7 (package)\n", " c8 (package)\n", "\n", "FILE\n", " /Users/zal/dev/Python_FMP_notebooks/FMP/libfmp/__init__.py\n", "\n", "\n" ] } ], "source": [ "import sys\n", "sys.path.append('..')\n", "import libfmp\n", "help(libfmp)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `__init__.py`-file of `libfmp` is empty. This means that no functionality is available at this stage. To import the modules contained in the subpackages, one needs to import the subpackages individually. The `__init__.py` files of the subpackages `B`, `C1`, `C2`, ..., `C8` are not empty; they contain a list of all functions that are considered essential for a user of the FMP notebooks and `libfmp`. In the following example, the subpackage `C8` is imported, and the help page is displayed." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2024-02-15T09:01:17.392135Z", "iopub.status.busy": "2024-02-15T09:01:17.391953Z", "iopub.status.idle": "2024-02-15T09:01:17.399563Z", "shell.execute_reply": "2024-02-15T09:01:17.399111Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on package libfmp.c8 in libfmp:\n", "\n", "NAME\n", " libfmp.c8\n", "\n", "PACKAGE CONTENTS\n", " c8s1_hps\n", " c8s2_f0\n", " c8s2_salience\n", " c8s3_nmf\n", "\n", "FILE\n", " /Users/zal/dev/Python_FMP_notebooks/FMP/libfmp/c8/__init__.py\n", "\n", "\n" ] } ], "source": [ "import sys\n", "sys.path.append('..')\n", "from libfmp import c8\n", "help(c8)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we show the content of `__init__.py` of `C8`." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2024-02-15T09:01:17.401886Z", "iopub.status.busy": "2024-02-15T09:01:17.401704Z", "iopub.status.idle": "2024-02-15T09:01:17.405041Z", "shell.execute_reply": "2024-02-15T09:01:17.404599Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "from .c8s1_hps import median_filter_horizontal, \\\n", " median_filter_vertical, \\\n", " convert_l_sec_to_frames, \\\n", " convert_l_hertz_to_bins, \\\n", " make_integer_odd, \\\n", " hps, \\\n", " generate_audio_tag_html_list, \\\n", " hrps, \\\n", " experiment_hrps_parameter\n", "\n", "from .c8s2_salience import principal_argument, \\\n", " compute_if, \\\n", " f_coef, \\\n", " frequency_to_bin_index, \\\n", " p_bin, \\\n", " compute_y_lf_bin, \\\n", " p_bin_if, \\\n", " compute_salience_rep\n", "\n", "from .c8s2_f0 import hz_to_cents, \\\n", " cents_to_hz, \\\n", " sonify_trajectory_with_sinusoid, \\\n", " visualize_salience_traj_constraints, \\\n", " define_transition_matrix, \\\n", " compute_trajectory_dp, \\\n", " convert_ann_to_constraint_region, \\\n", " compute_trajectory_cr, \\\n", " compute_traj_from_audio, \\\n", " convert_trajectory_to_mask_bin, \\\n", " convert_trajectory_to_mask_cent, \\\n", " separate_melody_accompaniment\n", "\n", "from .c8s3_nmf import nmf, \\\n", " plot_nmf_factors, \\\n", " pitch_from_annotation, \\\n", " template_pitch, \\\n", " init_nmf_template_pitch, \\\n", " init_nmf_activation_score, \\\n", " init_nmf_template_pitch_onset, \\\n", " init_nmf_activation_score_onset\n", "\n" ] } ], "source": [ "import os\n", "fn = os.path.join('..', 'libfmp', 'c8', '__init__.py')\n", "with open(fn, 'r', encoding='utf-8') as stream:\n", " content_text = stream.read() \n", "print(content_text)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that using `__init__.py` allows a user to access the specified functions without the need to specify the filenames the functions are contained in. For example, the function `c8.c8s1_hpa.hps` can be directly accessed via `c8.hps`. Besides its easier access, the convention also allows users to easily rename the filename `c8s1_hps` with a single update in the `__init__.py` without any further changes in the code." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Documentation of Functions\n", "\n", "For documenting the functions contained in `libfmp`, we follow standard Python style conventions as formulated in the [Google Python Style Guide](https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings). Most of the `libfmp`-functions are contained in some FMP notebook, where one finds a detailed explanation of the application, the underlying theory, and implementation issues. The FMP notebooks also provide illustrative examples, experiments with different parameter settings, and a discussion of results. In the `Docstring` of a `libfmp`-function, we specify the FMP notebook where the function is explained and developed. Using the `help`-function, the following example shows the docstring of the function `libfmp.c8.hps`. In particular, the information `Notebook: C8/C8S1_HPS.ipynb` shows that this function is introduced in the [FMP notebook on harmonic–percussive separation](../C8/C8S1_HPS.html) (HPS) with the filename `C8/C8S1_HPS.ipynb`." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2024-02-15T09:01:17.407473Z", "iopub.status.busy": "2024-02-15T09:01:17.407293Z", "iopub.status.idle": "2024-02-15T09:01:17.409961Z", "shell.execute_reply": "2024-02-15T09:01:17.409536Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on function hps in module libfmp.c8.c8s1_hps:\n", "\n", "hps(x, Fs, N, H, L_h, L_p, L_unit='physical', mask='binary', eps=0.001, detail=False)\n", " Harmonic-percussive separation (HPS) algorithm\n", " \n", " Notebook: C8/C8S1_HPS.ipynb\n", " \n", " Args:\n", " x (np.ndarray): Input signal\n", " Fs (scalar): Sampling rate of x\n", " N (int): Frame length\n", " H (int): Hopsize\n", " L_h (float): Horizontal median filter length given in seconds or frames\n", " L_p (float): Percussive median filter length given in Hertz or bins\n", " L_unit (str): Adjusts unit, either 'pyhsical' or 'indices' (Default value = 'physical')\n", " mask (str): Either 'binary' or 'soft' (Default value = 'binary')\n", " eps (float): Parameter used in soft maskig (Default value = 0.001)\n", " detail (bool): Returns detailed information (Default value = False)\n", " \n", " Returns:\n", " x_h (np.ndarray): Harmonic signal\n", " x_p (np.ndarray): Percussive signal\n", " details (dict): Dictionary containing detailed information; returned if ``detail=True``\n", "\n" ] } ], "source": [ "help(c8.hps) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# The Package `libfmp` at GitHub\n", "\n", "The Python package `libfmp` is also provided at [GitHub](https://github.com/), which is a code hosting platform for the free and open source distributed version control system [Git](https://git-scm.com/). The packages `libfmp` is exported from the FMP notebooks in a fully automatic process and then pushed to the following [GitHub repository](https://github.com/meinardmueller/libfmp):\n", "\n", "
\n", " https://github.com/meinardmueller/libfmp\n", "
\n", "\n", "\"GitHub\"\n", "\n", "The folder `libfmp` in the [GitHub repository](https://github.com/meinardmueller/libfmp) coincides with the sources contained in the FMP notebooks' folder `FMP/libfmp`.\n", "The remaining files of the Github repository are also contained in the FMP notebooks' main folder (provided with the prefix `libfmp_` which is removed in the automated export).\n", "The content of the folder `docs` contains sources for the libfmp API documentation, which is based on the documentation generation package [sphinx](https://www.sphinx-doc.org). An HTML export of this documentation, is available at:\n", "\n", "
\n", " https://meinardmueller.github.io/libfmp\n", "
\n", "\n", "\n", "\"GitHub\"\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The Package `libfmp` at PyPi\n", "\n", "The [Python Package Index (PyPI)](https://pypi.org/) is a repository of software for the Python programming language, which helps users to find and install software developed and shared by the Python community. The package `libfmp` has been exported automatically to PyPi and is accessible at:\n", "\n", "
\n", " https://pypi.org/project/libfmp/\n", "
\n", "\n", "\"PyPi\"\n", "\n", "The [command `pip`](https://pip.pypa.io/en/stable/) is used to look for packages in PyPI, to calculate the packages' dependencies, and to install them. The PyPi-version of `libfmp` can be installed (within a given Python environment) in the following way:\n", "\n", "
\n", " pip install libfmp\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "Acknowledgment: This notebook was created by Meinard Müller and Frank Zalkow.\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "
\"C0\"\"C1\"\"C2\"\"C3\"\"C4\"\"C5\"\"C6\"\"C7\"\"C8\"
" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.16" } }, "nbformat": 4, "nbformat_minor": 1 }