Scripting Extension

These are notes and current status of extending openmw scripting through SWIG. STATUS: Python scripts can be run in the console if they contain mwscript instructions but not mwscript functions functions!

These are notes on the scripting extension work, see: First described for python (SWIG):

https://forum.openmw.org/viewtopic.php?f=6&t=617&start=60#p29966

LUA implementation:

https://forum.openmw.org/viewtopic.php?f=6&t=617&start=70#p30090

https://forum.openmw.org/viewtopic.php?f=6&t=617&start=70#p30093

How to use:

$ sudo apt-get install python3-dev swig
$ git clone https://github.com/maqifrnswa/openmw/tree/python-scripting 
$ cd openmw/
$ mkdir build && cd build
$ make -j
$ [make your python script, see example below]
$ PYTHONPATH=. ./openmw --start="Seyda Neen" --skip-menu

then in game, hit "~" to get the console, and type your script name into the console to run it, eg "test.py".

Example script:

from openmw import *

#test an instruction, self reference
print("hello i'm a criminal")
setpccrimelevel(100000000)

#test an instruction, no implicit from commandline
print("hello i'm flying")
setflying("player",1)

how to write scripts: All commands are lowercase and take the same arguments as before, even optional ones. Commands that can take a reference (commands with ->) now have a first argument which is the reference. That first argument is required. If it is an implicit reference, use "self". Example 1:

player->AddItem, "Gold_001", 200 

is now

additem("player","Gold_001",200)

Example 2:

AIWander, 0, 0, 0 

is now

aiwander("self",0,0,0)

Example 3:

"urzul gra-agum"->AIWander, 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0

is now

aiwander("urzul gra-agum", 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0)

there's still a bunch of debugging output that will show up in the terminal, bear with it for the moment.

Overview:

• I think a post-1.0 goal, of mine at least, will still be to see if I can get an easy generic scripting interface.
• It will be a seperate fork from openmw, as I think a unified scripting language and zini's enhancements to the existing implementation would be best for the community. It's generally better to improve one implementation rather than create several new ones. If, in the future, such extension of scripting is found desireable, then there is no problem incorporating it into openmw.
• There are some things that python/other languages can do better and can be implemented faster than writing a new scripting engine from scratch. For example, you can use all of python's existing modules and capabillities that could enable things like dynamic web content/interaction in games. Some people are already used to lua and python scripting, they could potentially write more complicated scripts or more easily integrate scripts. For discussion on python versus lua, I found this very helpful:http://lua-users.org/wiki/LuaVersusPython -- I think both could be usefull in different cases.
• Existing scripting IDEs can be used, most scripting language already has built-in verbose debugging. It's possible to generate embedded consoles for testing scripts on the fly. Scripts can be edited and recompiled on the fly, in game.
• Security will be an issue (running untrusted scripts), but it something people already deal with in the modding community (e.g, Blender Python Scripting)Security. Concerns can be addressed at a minimum by making users opt in to using python, and can be extended to only allow running scripts signed by a trusted certificate/key.

Implementation: https://github.com/maqifrnswa/openmw/tree/python-scripting

• a new executable, extensionstool, generates bindings. A header and cpp file are generated containing that creates compiled code for every extension based on the arguments passed to the function, then sends the compiled code to an interpreter. The interpreter is declared by the console or by the new opcode, the bindings just have a pointer to the interpreter that is owned by whomever created it.
• SWIG compiles the bindings and creates a module _openmw.so and python module openmw.py.
• Right now, the console intercepts all *.py files so the console command "filename.py" will run the python script filename.py

• import openmw

  will import the openmw commands as openmw.COMMANDS, or use

  from openmw import *

  to use whatever you need to use.
• All commands are lowercase and take the same arguments as before. Commands that can take a reference (commands with ->) now have a first argument which is the reference. For example:

 player->AddItem, "Gold_001", 200 

is now

 additem("player","Gold_001",200)

TODO:

• Create a new opcode and scripting command for "StartExternalScript". This new opcode will have execute() function that take a runtime with the name of a script file, create an embedded interpretter for whatever language, and execute the script in the embedded interpretter.

namespace MWScript
{
    namespace External
    {
        class OpStartExternalScript : public Interpreter::Opcode0
        {
            public:
                virtual void execute (Interpreter::Runtime& runtime)
                {
                    std::string scriptfile = runtime.getStringLiteral (runtime[0].mInteger);
                    runtime.pop()
                    MWScript::Context context = &runtime.getContext();
                    Interpreter::Interpreter interpreter;
                    MWScript::installOpcodes(interpreter);
                    MWScriptExtensions::interpreter = &interpreter; //is a pointer
                    Py_Initialize();
                    FILE *file_1 = fopen(scriptfile.c_str(),"r");
                    PyRun_SimpleFileEx(file_1,scriptfile.c_str(),1);
                    Py_Finalize();
                }
        }

        void installOpcodes (Interpreter::Interpreter& interpreter)
        {
            interpreter.installSegment5 (Compiler::External::opcodeStartExternalScript, new OpStartExternalScript);
        }
    }
}

Outstanding questions:

• How to handle saving variables? Between frames, variables can be saved by saving the environment. That also allows us to possibly think of creating each running script as a seperate object in python and make the interpretter environment persistent. Then add the variables to save upon exist (and load upon initiation) through a second binding function. Or we can create multiple independent environments.
• How fast will this be? i'm guessing slower than the MWScript system, but will it be prohibitively slower?
• should the bindings be compiled with the executible openmw instead of _openmw.so? Right now they are in _openmw.so, which seems to work. The only problem is that MWScriptExtensions::context and MWScriptExtensions::interpreter are defined in both console.cpp and openmwbindings.cpp.