VHDL-embedded Python code

[LarsAsplund]

This PR contains work toward enabling embedded Python code within VHDL

For the first iteration I've focused on providing the lowest possible threshold for embedded Python and I found that to be the "eval and exec model" where the user can create any valid Python string and have that evaluated as an expression or executed as code by a Python interpreter.

The main limitation to what can be done is basically that variables passed between Python and VHDL must have a type that can be represented in both domains. For the first set of examples I also limited myself to a small set of scalar and vector/list types. Supporting remaining types is just more of the same and will be added as I move along.

Simplicity also means:

• Good error messages when the code strings created in VHDL fail when being executed or evaluated in Python. Obviously such a test must also fail.
• The user should not need any knowledge of C, C compilers, compiler versions,  foreign language interfaces etc. In fact, they should not even have to know that C and foreign language interfaces are the means by which this is achieved. Knowing VHDL and Python should be sufficient. In reality I may not reach that goal fully but for Riviera-Pro and Active-HDL, the only thing the user has to remember is to tick the "include C stuff" box during installation of the tool.

Riviera-PRO and Active-HDL use VHPI but it is important that the user interfaces do not leak VHPI specific stuff. The same API should be possible to use on all simulators regardless of what foreign language interface being used in the background
So far I've only worked on VHPI and on Windows.

Some work has also been done on running the Python code in separate process(es) (as in separate applications). Rather than passing code strings, the VHDL code would communicate with the concurrent Python actors using message passing or simple data queues. This work will be completed in a second iteration and in a separate PR. The goal of that use model is more towards performance.

Feedback on the eval and exec model is appreciated and a good starting point is to review the provided example tests. If you have an Riviera-PRO or Active-HDL license you can run the examples. If not, you should get a good feel for the concepts by just looking at the example code.

I would also appreciate more use case examples. So far I have these (note that some are expected to fail)

For you who don't have any of the currently supported simulators, here is a screenshot of what the user interactions provided by some tests may look like. I use PySimpleGUI which creates these types of interactions from one-liners

Currently tested simulators

Riviera-PRO

• Version 2023.10, 64-bit
• Windows 10
• Python 3.12
• GCC shipped with simulator (gcc.exe (MinGW-W64 x86_64-ucrt-posix-seh, built by Brecht Sanders) 12.3.0). Note that it is an option that must be selected during installation

Active-HDL

• Version 14, 64-bit
• Windows 10
• Python 3.12
• GCC shipped with simulator (gcc.exe (x86_64-posix-seh-rev0, Built by MinGW-W64 project) 4.8.4) doesn't work with the VHPI application. Not investigated further but that GCC version is more than 10 years old. In-place replacement with a newer version such that the GCC provided by Riviera-PRO works

Questa

• Version 2023.3, 64-bit
• Windows 10
• Python 3.12
• GCC provided by Siemens as a zip file to be unzipped in the installation directory (gcc.exe (GCC) 7.4.0)

nvc

• Version 1.11.2
• Windows 10/MSYS2
• Python 3.11.7
• GCC 13.2.0

GHDL

• GHDL 4.0.0-dev (3.0.0.r750.g2135cbf14) [Dunoon edition]. Compiled with GNAT Version: 13.2.0. llvm 17.0.6 code generator
• Windows 10/MSYS2
• Python 3.11.7
• GCC 13.2.0

[joshrsmith]

Very interesting. This is like cocotb in reverse. 😆

@crdavis12

[LarsAsplund]

@joshrsmith Yes, it is reversed in the sense that the simulation is driven from VHDL rather than from Python. The reason is that we want to use VHDL for what is good for and only use Python when VHDL runs out of steam. Doing pin-wiggling from Python is a bit inefficient and that is not a price I want to pay for getting access to the areas where Python is a good fit.

[javValverde]

Great effort, this certainly opens up a lot of possibilities!

I'm still really interested in communication between python and VHDL with messages.
I would love for python to send a message to VHDL, and block until VHDL answers to the message.

[LarsAsplund]

@javValverde Yes, that will be step 2. I thought I should start with something simpler to find all the different quirks in VHPI, FLI, and the others. I also got FLI working so now the same examples can be run with no modification under Questa. Will do some refactoring before pushing that.

[cmarqu]

@LarsAsplund Not sure if you are aware, but Questa also supports VHPI since about two years.

[LarsAsplund]

@cmarqu Yes, I discovered that after I started with FLI. It was a bit hidden. However, very few VHPI functions are supported so currently it is not possible to use it for our purposes.

[LudvigVidlid]

This looks so nice :D

[nickg]

The master branch of NVC now has some very basic support for VHPI foreign sub-programs. There's some examples here:

test/regress/vhpi14.vhd
test/vhpi/vhpi14.c

Although at the moment I don't support Aldec's format for the foreign attribute.

[nickg]

This needs -fPIC too on Linux.

[nickg]

I think this needs a call to vhpi_release_handle(parameter_handle) here or a it leaks a handle on each invocation.

[nickg]

Does "VHPI python python_setup" work? ("python" here is the libraryName argument passed to vhpi_register_foreignf().) I think passing the path to the native library here and the semicolon separator is an Aldec extension that's not present in the VHDL LRM.

[LarsAsplund]

The semicolon is something I took from Aldec's examples. However, it works well without it so I will have it removed.

It seems like just python should be enough to identify the shared library if I put the libraries directory in the PATH environment variable. Are you also using that approach to for finding the libraries?

[LarsAsplund]

Btw, what would be the benefits of using VHPI over VHPIDIRECT in this case?

[nickg]

It seems like just python should be enough to identify the shared library if I put the libraries directory in the PATH environment variable. Are you also using that approach to for finding the libraries?

No, I'm rely on the user loading the library explicitly with --load and then the "VHPI libraryName modelName" should match the corresponding libraryName and modelName arguments passed to vhpi_register_foreignf(). I'm actually having a hard time understanding how the LRM intended these to be used. Does Aldec treat the first libraryName as the path to the shared library?

Btw, what would be the benefits of using VHPI over VHPIDIRECT in this case?

Honestly probably not many benefits. In theory if every simulator implemented the VHPI standard you'd only need one backend here rather than one per simulator.

[willkamp]

@LarsAsplund , Thanks for your efforts, this looks like a really powerful addition to the VUnit ecosystem. I look forward to this PR making it into the main branch and a release.

I've had a hack at compile_fli_application() in python_pkg.py to make it work in my environment - Ubuntu + python 3.10 + Intel Questa (from Quartus 24.1). I have the tests running :) 18 of 28 pass, and 10 error cases failing as expected.

We're quite interested is using something like this to build python based predictors of our signal processing in the Square Kilometre Array Telescope Correlator and Beamformer.

[LarsAsplund]

@willkamp Do you have any details on the hacks you had to do?