Embedding Python in C++
This is how I embedded Python in my C++ projects. My goal was to have a console that would take in Python in a REPL fashion. I also wanted plugins in Python that use an embedded Python module; embedded inside the program. I will also show you how to make such a module and add it to your embedded Python console.
Get Visual Studio 15 Express for Desktop from Microsoft’s website. Get 7-Zip for extracting compressed files. Get Python 3.6. Windows x86-64 executable installer. Install Python 3.6 to “C:\Python36”. This provides all the header files you need to reference in our project. It also has the “python36.dll” you will need to copy to your project. You also need the Windows x86-64 embeddable zip file. Extract the Gzipped source tarball to your ‘C’ directory. You should now have “C:\python-3.6.0-embed-amd64”. This has the standard library pre-compiled in a ZIP file. The pre-compiled ZIP file can be used to include with your project when you ship it.
Open Visual Studio 2015 and create a new C++ Win32 Project. The template is under Templates/VisualC++/Win32 in the tree on the left side of the create project menu. It will look like this:
My project directory is “C:\Users\Justin\Documents\Visual Studio 2015\Projects\EmbeddingPythonTutorial”. Your project directory will be similar. Once you click next you will see a screen similar to this:
In the EmbeddedPythonTutorial Windows project. Go to the solution explorer pain and right click the project. Then go to settings in the context menu. Then in the window that opens go to VC++ directories. Then select “Include Directories”. Add the directory “C:\Python36\include”. Add C:\Python36\libs to the “Library Directories.”
Click “OK” and you will be back in the project screen. At the top of the IDE make sure you have “x64” selected in your “Solution Platforms.”
Copy “C:\Python36\python36.dll” to “C:\Users\Justin\Documents\Visual Studio 2015\Projects\EmbeddingPythonTutorial\EmbeddingPythonTutorial” if this directory does not exist you just need to run the program in debug mode once and it will be created.
At the top of your file “EmbeddedPythonTutorial.cpp” add the following code:
This gets around Python trying to use debug code even when you use your project
in debug mode. Our project only uses the 3.6.0 release version. This tutorial
is not for compiling Python. We are pulling the header file “Python.h” from the
directory you just added to “Include Directories.” The “Python.h” file has all
Py_ functions you will need for this project.
Now in the function
Your code should look like the following. I have included some code before and after for you to find the correct location.
Now you also need to close up Python when your program finishes. To do that
add a call to
Py_Finalize before the return statement in
Including the Python Libraries
When you end up sending your project out it would be nice not to have to require the user to install Python 3.6. Well, they do not have to. You can include the default Python libraries as a ZIP file in your program.
Download “Windows x86 embeddable zip file” this has a python36.zip file inside
of it that has all the Python libraries pre-compiled. You can make your program
use this instead of the files at C:\Python36\Lib. Change the
to use the zip file. Copy the ZIP file to your project directory. Then make
Py_SetPath look like this:
Testing our Python Setup
Set the library path for your embedded Python. This code goes inside the
wWinMain above the line
I have mine setup like the following so that when I am debugging the project the program is using my local libraries instead of the pre-compiled libraries inside of a ZIP file.
At this point you should run the project to make sure you got all of our settings correct and all of our files in the right place. Note the path used if the program is not in debug. It will be the file “python36.zip” you must copy that file from “C:\python-3.6.0-embed-amd64\python36.zip” to the same directory that your executable is. You must also ship the file “python36.zip” with your program and the python36.dll in order for it to work on someone else’s machine.
Everything should look like this project at this point.
Embedded Python Console
Now you will add code to make Python work like a console. In order to do this you will need to do a little Win32 programming. You need a text box to enter text for the Python console. You need another text box for the Python results.
Input Text Box
WndProc you will add the code for the input text box. Inside
the switch statement you need to add a new case. Add
case WM_CREATE: just
case WM_COMMAND. Then inside the
case WM_CREATE add the
Just above the switch statement add the following code:
Now run the program. You will see a text box near the left side of the window. Click in it and type. That is where you will insert Python code for your Python Console. Next you will add the output text box.
Output Text Box
The output text box will be able to display multiple lines. Add the following right after the input box decleration.
Now add this code next to the code that created the input text box.
Run the program. You should see a grayed-out text box below the input text box. This is our output text box.
Sending the Input to Python
Sending the input to the Python interperter requires knowing when to take the input from the user and send it to the Python interperter. You are going to use the enter key.
In order to capture the enter key press the default window procedure for the input text box needs to be replaced. Just below the code to create the input text box add the following:
At the top of the file in the section of Global Variables add the following:
Near the bottom of the file, below the function
About() add this window
Inside the above code do you see the call to statement
SetWindowText(g_input_window, L"");? This sets the window text of your
edit box. In order for this to work you need to make the input box a global
variable. Move the code
to the globals section near the top of the file. Now run the program. Type some text into the text box and hit the enter key. The text should disappear. That’s how you know your window procedure is capturing the windows key.
Sending the Input to the Python Interperter
To send the input to the Python interperter you must call it with
[PyRun_String](https://docs.python.org/3.4/c-api/veryhigh.html#c.PyRun_String). You can’t spend to much time in the GUI thread though doing any
long task. If you do your program will hang and people won’t want to use it.
To overcome this you will put the input in a global variable and set an event
for a seperate thread to know there is new input ready.
#include <process.h>. Add that line to the top of the file. This
provides the functions
wWinMain function add the following just after the call to
In between the global variables section and the forward declerations section add the following code, which creates the varibles needed for your Python thread:
In the forward declerations section add the
Add code to close up the Python thread right before the code to close Python.
Now you will add the code for the Win32 event. The event will hold the Python thread in a yielding state until the event is set. Then the Python thread will run its code. The event lets your thread know there is data ready to be parsed by Python. The data came from your input box. If you don’t use the Win32 event system and just loop your program will eat all of your CPU. Always follow your system’s event/scheduling mechanisms. That way your CPU isn’t just hogged by a loop. You will add code in a few places to get the event set up. Add code to the section “Python Interperter Variables” at the top.
Initialize the event in the
wWinMain just before the
Since our thread will be waiting for this event to run you need to set the event
when you are trying to close the program so the Python thread dies gracefully.
In between where you set the
g_python_thread_done = TRUE; and where
you close the thread handle (
the event and wait for it. The code should look like this.
PythonThreadFunc add the code so that the event system waits
for your new Python inpute event,
g_python_input_event. The code goes
inside the while loop.
If you set this event in order to close the thread the thread needs code to check for that and end gracefully.
Now you need a variable to hold the Python input data when the user hits the enter key. At the top create a varible named “g_python_input”.
InputBoxProc you need to uncomment the two lines that were
commented out when you first wrote code there. One writes the text from the
input box to our global Python input variable. The other line sets the event
that you use to tell the thread new input is available.
For the above code to work the output box needs to be a global variable so that
you have easy access to it. Make the output box global by removing it from the
WndProc and putting it next to the
input_text_box. This is
the line you are looking for…
Hopefully this section wasn’t like drawing a
horse. If it was,
let me know. It was a big step though in getting the infrastructure you need to
handle input and not break the GUI thread. If you entered everything correctly
then hitting enter in the input box should make a
>>> show up in the output
box. Here is the code up to this point.
Getting the Python Output
You will now have Python parse the user’s input and return the result. The result can be an error, nothing, or whatever else you can see when using the Python console. This part of the project also takes quite a bit of work.
You need to modify the display hook so you can grab the data in a Python variable.
PythonThreadFunc add the following code at the top of the function
above the while loop.
You can now grab the results of entering a Python statement by grabbing the
data in the variable
__result or grab the exception information in
__traceback. You will add the code to do this now. Just after the if
statement to check in the user’s input was empty append the following code.
Run the program and type the following as two seperate commands.
You should see the output:
Now enter “z” into your text box. You should see the following traceback in your output box.
This means you have our embedded Python console set up correctly. The code at this point is EmbeddedingPythonTutorial_Part4.zip.
Adding a Python Module
You are going to make a custom module and add it to your embedded Python console system. You will be able to make calls to the embedded module through your Python input box.
Create a new file in the solution by hitting the hot key
“Visual C++” in the tree on the left and then select “C++ File”. Now in the
“Name” box at the bottom, enter “awesome_module.cpp”. Click “Add”. Enter
Ctrl+Shift+A again to add another file to the project. Select “Header File”,
and enter “awesome_module.h” for the name. Click “Add”. Now go to the header
file “awesome_module.h”. Enter this code:
Now go to the source file “awesome_module.cpp”. Add the following code:
All of this code come from following the Python lesson on extending: A Simple Example. More information is in the Extending/Embedding FAQ Refer to this if these documents if you want to keep adding to your module. You can add more variables and functions to your module.
Go to your “EmbeddingPythonTutorial.cpp” file. Add the module header file to
your “EmbeddingPythonTutorial.cpp” file. Add the following code just below
Now you need to have Python import the module. Go to where we call
wWinMain. Add the call to
This tell Python where to find the module “awesome”. Python knows the module is
avaiable at that
PyInit function. The embedded console must still call
import awesome in order to have the module available. Go down to the function
PythonThreadFunc. Modify the line
to look like this:
Run the program. In your input box type
awesome.best_actor you should see
Now you have the foundation of an embedded module to extend your embedded Python console. The code at this point is EmbeddingPythonTutorial_Part5.zip.