Description

Matlab Plotting Library, known as MPL, is widely used in areas like scientific research and industrial simulation. It's especially useful when it comes to plotting diagrams side by side for purposes like comparison analysis. While MPL is written in Python, Matplotlib for C++ translates a set of MPL in way of corresponding wrappers in C++ APIs, making it possible for C++ developers to draw MPL style diagrams in C++.

This article describes the overall steps required to add MPL capability in C++ code.

Prerequisites

• macOS Catalina version 10.15.7 (19H524)
• miniConda3 version 4.10.1 (optional)

Notes: MiniConda is used for packages management, it's suggested but not mandatory if you have your preferred package management system, such as brew. So going forward, almost all the dependant libraries will be downloaded in and referred from the conda environment. In addition, two env variables CONDA_HOME and OPENCV_DIR are set and exported, you will see them throughout the article, change them as required to fit your needs. Note also that Conda will create and export an env variable named CONDA_PREFIX, which points to the active Conda environment.

export OPENCV_DIR=/Users/nling/opencv/installation/OpenCV-master-debug
export CONDA_HOME=/Users/nling/opt/miniconda3

Environments

1. Install Annoconda3 (skip if done)
   Some useful Conda commands:

• Check Conda version.

conda --version

• Check created Conda environments.
  below are some sample outputs, the asterisk sign (*) indicates the current environment you are working on, base is the default environment.

base                  *  /Users/nling/opt/miniconda3
cling                    /Users/nling/opt/miniconda3/envs/cling
kotlin                   /Users/nling/opt/miniconda3/envs/kotlin
xeus-cling               /Users/nling/opt/miniconda3/envs/xeus-cling
xeus-cling-dev           /Users/nling/opt/miniconda3/envs/xeus-cling-dev

• Work on or exit a Conda env.

conda activate xeus-cling-dev
conda deactivate

• Check available Jupyter virtual kernels (required Jupyter being installed in advance)

// conda install jupyterlab -c conda-forge
jupyter kernelspec list

2. Install matplotlib and numpy (in the default base environment, changed as necessary)

(base) ➜  ~ conda install matplotlib numpy -c conda-forge

Run the below command to double-check they are properly installed.

(base) ➜  ~ conda list | egrep "numpy|matplotlib"

Sample output

matplotlib                3.4.2            py39h6e9494a_0    conda-forge
matplotlib-base           3.4.2            py39hb07454d_0    conda-forge
matplotlib-inline         0.1.2              pyhd8ed1ab_2    conda-forge
numpy                     1.20.2           py39h4b4dc7a_0
numpy-base                1.20.2           py39he0bd621_0

3. Download source code of matplotlibcpp.h and (optionally) place it somewhere typically in the search path for library headers, for example.

cp matplotlibcpp.h ${OPENCV_DIR}/include/opencv4

4. Write demo code to plot a simple line figure.

#include "matplotlibcpp.h"
#include <vector>

namespace plt = matplotlibcpp;

int main() {
  std::vector<double> y = {1, 3, 2, 4};
  plt::plot(y);
  plt::show();
}

5. Compile the demo code with clang

/usr/bin/clang++  MatplotDemo.cpp                                       \
-o MatplotDemo                                                          \
-std=c++17                                                              \
-stdlib=libc++                                                          \
-g                                                                      \
-I${OPENCV_DIR}/include/opencv4                                         \
-I${CONDA_HOME}/include/python3.9                                       \
-I${CONDA_HOME}/lib/python3.9/site-packages/numpy/core/include          \
-L${OPENCV_DIR}/lib                                                     \
-L${CONDA_HOME}/lib                                                     \
-lpython3.9                                                             \
-lopencv_core                                                           \
-lopencv_imgproc                                                        \
-lopencv_imgcodecs                                                      \
-lopencv_highgui                                                        \
-Wl,-rpath,${OPENCV_DIR}/lib                                            \
-Wl,-rpath,${CONDA_HOME}/lib  

Options:

• -std=c++17, compile for standard ISO C++ 2017.
• -stdlib=libc++, specify c++ standard library, options can be libstdc++ and libc++.
• -g, generate and control debug info outputs.
• -o, specify the name and location of the executable file.
• -I, specify the location to search for header files.
• -L, specify the location to search for libraries to be linked with.
• -l, specify library name to be linked with.
• -W, pass comma-separated arguments to the linker. Here -rpath,/Users/nling/opt/anaconda3/lib is passed to the linker so the final executable is able to load and link with the target libraries at run-time.

6. Run the demo executable and you should be able to see onscreen the plotted demo figure.

   
   
   MatplotDemo

Show cv::Mat-formmetd OpenCV image

The matplotlibcpp.h tool comes with support for displaying Matlab-style images, see the sample code imshow for reference. Moreover, it provides supports for displaying images represented as cv::Mat data structure (requires a macro named WITH_OPENCV be defined first).

Here is a simple example that categorizes and extracts each individual character, using CCA(Connected Component Analysis) algorithm, from the "TRUTH" image shown as follows.

Original TRUTH Image

The sample code that employees connectedComponents to detect labels and plots the connected labels in a single figure.

#define WITH_OPENCV

#include <iostream>

#include <matplotlibcpp.h>

namespace plt = matplotlibcpp;

using namespace cv;
using namespace std;

int main(int argc, char const *argv[])
{
    Mat img = imread("truth.png", IMREAD_GRAYSCALE);
    Mat imgThreshed;
    threshold(img, imgThreshed, 127, 255, THRESH_BINARY);

    Mat imgLabels;
    int nComponents = connectedComponents(imgThreshed, imgLabels);
    cout << "nComponents:" << nComponents << endl;

    double minVal, maxVal;
    Point minLoc, maxLoc;
    minMaxLoc(imgLabels, &minVal, &maxVal, &minLoc, &maxLoc);
    cout << "minVal:" << minVal << ", loc:" << minLoc  << endl;
    cout << "maxVal:" << maxVal << ", loc:" << maxLoc  << endl;
    imgLabels.convertTo(imgLabels, CV_8U);

    Mat coloredLabels = imgLabels.clone();
    coloredLabels = 255 * (coloredLabels - minVal) / (maxVal - minVal);
    applyColorMap(coloredLabels, coloredLabels, COLORMAP_JET);
    plt::figure();
    plt::title("Colored Image Segments");
    plt::imshow(coloredLabels);
    // plt::save("colored_image_segments.png");

    plt::figure();
    plt::title("Plot Connected Labels");
    plt::subplot(2, 3, 1);
    plt::imshow(imgLabels == 0);
    plt::subplot(2, 3, 2);
    plt::imshow(imgLabels == 1);
    plt::subplot(2, 3, 3);
    plt::imshow(imgLabels == 2);
    plt::subplot(2, 3, 4);
    plt::imshow(imgLabels == 3);
    plt::subplot(2, 3, 5);
    plt::imshow(imgLabels == 4);
    plt::subplot(2, 3, 6);
    plt::imshow(imgLabels == 5);

    // auto-adjust spacing between subplots
    plt::tight_layout();    
    // map<string, double> params = {
    //     {"left",   0.125},
    //     {"bottom", 0.11 },
    //     {"right",  0.9  },
    //     {"top",    0.88 },
    //     {"wspace", 0.2  },
    //     {"hspace", 0.8  }
    // };
    // plt::subplots_adjust(params);
    // plt::subplot_tool();

    plt::show();

    // save after being shown to avoid spacing issue.
    // plt::save("plot_connected_labels.png");

    return 0;
}

colored_image_segments.png

Plot Connected Labels

Possible Problems and Solutions

1. Incompatible library version

(base) ➜  MatplotDemo MatplotDemo
dyld: Library not loaded: /usr/local/opt/glib/lib/libglib-2.0.0.dylib
  Referenced from: /usr/local/opt/harfbuzz/lib/libharfbuzz.0.dylib
  Reason: Incompatible library version: libharfbuzz.0.dylib requires version 6801.0.0 or later, but libglib-2.0.0.dylib provides version 6601.0.0

2. Non-GUI backend issue
   “UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.” when plotting figure with pyplot on Pycharm
   • Solution1: change backend via a call to plt::backend("tkagg");
   • Solution2: Add backend config backend : tkAgg in file ~/.matplotlib/matplotlibrc.

Solution1 solved the non-GUI backend issue meanwhile created following Error loading module, Solution2 didn't work.

By the way, MPL works fine in pure Python script.

(base) ➜  MatplotDemo workon OpenCV-master-py3
(OpenCV-master-py3) (base) ➜  MatplotDemo python
Python 3.9.5 (default, May  4 2021, 03:33:11)
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib.pyplot as plt
>>> y = [1, 3, 2, 4]
>>> plt.plot(y)
>>> plt.show()

3. Error loading module

(base) ➜  MatplotDemo MatplotDemo
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Error loading module matplotlib.pyplot!
[1]    98996 abort      MatplotDemo

The above issues were ALL caused due to below mistakenly exported library path /Users/nling/opt/anaconda3/lib, remove or comment out the DYLD_LIBRARY_PATH sovled the issues.

# export DYLD_LIBRARY_PATH=$OPENCV_DIR/lib:/Users/nling/opt/anaconda3/lib

The DYLD_LIBRARY_PATH was there for linker issues where OpenCV-related libraries could not be found at run-time.

4. Image not found at run time

(base) ➜  MatplotDemo MatplotDemo
dyld: Library not loaded: @rpath/libpython3.8.dylib
  Referenced from: /Users/nling/opencv/samples/MatplotDemo/MatplotDemo
  Reason: image not found
[1]    3813 abort      MatplotDemo

Solution: Pass the required runtime paths (represented as @rpath in the compiled executable) to the linker through -Wl compile option.

-Wl,-rpath,/Users/nling/opt/anaconda3/lib

inspired by this issue clang fails with -Wl,rpath=....

As a workaround mentioned in @rpath what?, @rpath can be specified using the install_name_tool command.

install_name_tool -add_rpath /Users/nling/opt/anaconda3/lib MatplotDemo

otool can be used to check if the compiled executable contains LC_RPATH entries whose value will be used in searching for dylib specified with @rpath. Again, the LC_RPATH entries are added via the -Wl,-rpath,/Users/nling/opt/anaconda3/lib linker options.

otool -l MatplotDemo

...
Load command 13
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name @rpath/libpython3.8.dylib (offset 24)
   time stamp 2 Thu Jan  1 08:00:02 1970
      current version 3.8.0
compatibility version 3.8.0
...
Load command 16
          cmd LC_RPATH
      cmdsize 72
         path /Users/nling/opencv/installation/OpenCV-master-debug/lib (offset 12)
Load command 17
          cmd LC_RPATH
      cmdsize 48
         path /Users/nling/opt/anaconda3/lib (offset 12)

References

1. Matplotlib for C++